2023-10-10T20:06:28.078 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-10T20:06:28.079 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --archive /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465 --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/orig.config.yaml 2023-10-10T20:06:28.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T20:06:28.216 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465 branch: wip-yuri3-testing-2023-10-10-0720-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: yweinste@redhat.com first_in_suite: false job_id: '7420465' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-10-0720-quincy ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-10-0720-quincy sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2048 sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-10-0720-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGi6gJsTVaMUlBg6Me2PzbT0mPxCeJVMjlPY47SvIQiGWjBu3snTBdzepBBQn2t82Ii4rS/ZdXuBT+0mjcs1yd4= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYkYz3NSyUPaG3igeoA55L9b/m1sneuW3V9rXMKdl1KO1GrLsuBTmFe2n4Be3W4R8N7xmc2kremECT0tzLawKs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-10_18:16:52 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-10T20:06:28.217 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa; will attempt to use it 2023-10-10T20:06:28.218 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa/tasks 2023-10-10T20:06:28.218 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-10T20:06:28.220 INFO:teuthology.task.internal:Checking packages... 2023-10-10T20:06:28.246 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '46d1f59b549730ba7f93d940a3d32b94c45a4cf7' 2023-10-10T20:06:28.247 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-10T20:06:28.247 INFO:teuthology.packaging:ref: None 2023-10-10T20:06:28.248 INFO:teuthology.packaging:tag: None 2023-10-10T20:06:28.248 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-10-0720-quincy 2023-10-10T20:06:28.248 INFO:teuthology.packaging:sha1: 46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:06:28.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-10-0720-quincy 2023-10-10T20:06:28.539 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1374.g46d1f59b 2023-10-10T20:06:28.540 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-10T20:06:28.546 INFO:teuthology.task.internal:no buildpackages task found 2023-10-10T20:06:28.547 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-10T20:06:28.564 INFO:teuthology.task.internal:Saving configuration 2023-10-10T20:06:28.583 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-10T20:06:28.590 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:06:28.615 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:00:59.485629', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGi6gJsTVaMUlBg6Me2PzbT0mPxCeJVMjlPY47SvIQiGWjBu3snTBdzepBBQn2t82Ii4rS/ZdXuBT+0mjcs1yd4='} 2023-10-10T20:06:28.637 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:00:59.483661', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYkYz3NSyUPaG3igeoA55L9b/m1sneuW3V9rXMKdl1KO1GrLsuBTmFe2n4Be3W4R8N7xmc2kremECT0tzLawKs='} 2023-10-10T20:06:28.637 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-10T20:06:28.644 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-10T20:06:28.644 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-10T20:06:28.645 INFO:teuthology.run_tasks:Running task console_log... 2023-10-10T20:06:28.721 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d21a1cb80>, signals=[15]) 2023-10-10T20:06:28.722 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-10T20:06:28.728 INFO:teuthology.task.internal:Opening connections... 2023-10-10T20:06:28.729 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:06:28.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:06:28.800 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:06:28.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:06:28.867 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-10T20:06:28.874 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-10-10T20:06:28.892 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-10-10T20:06:28.892 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-10-10T20:06:28.948 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T20:06:28.948 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T20:06:28.948 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-10-10T20:06:28.949 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-10-10T20:06:28.949 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.4" 2023-10-10T20:06:28.949 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-10-10T20:06:28.949 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T20:06:28.950 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-10-10T20:06:28.950 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T20:06:28.950 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T20:06:28.950 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T20:06:28.951 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T20:06:28.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:06:28.951 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T20:06:28.951 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T20:06:28.952 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T20:06:28.952 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T20:06:28.953 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-10-10T20:06:28.971 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-10-10T20:06:28.988 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-10-10T20:06:28.988 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-10-10T20:06:29.044 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T20:06:29.044 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T20:06:29.044 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2023-10-10T20:06:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2023-10-10T20:06:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.4" 2023-10-10T20:06:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2023-10-10T20:06:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T20:06:29.046 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2023-10-10T20:06:29.046 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T20:06:29.046 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T20:06:29.046 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T20:06:29.047 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T20:06:29.047 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:06:29.047 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T20:06:29.047 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T20:06:29.047 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T20:06:29.048 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T20:06:29.048 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-10-10T20:06:29.071 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-10T20:06:29.080 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-10T20:06:29.086 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-10T20:06:29.087 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2023-10-10T20:06:29.089 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-10-10T20:06:29.105 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-10T20:06:29.111 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-10T20:06:29.111 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2023-10-10T20:06:29.146 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-10-10T20:06:29.160 INFO:teuthology.orchestra.run.smithi148.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-10T20:06:29.162 INFO:teuthology.orchestra.run.smithi161.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-10T20:06:29.163 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-10T20:06:29.218 INFO:teuthology.run_tasks:Running task kernel... 2023-10-10T20:06:29.234 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-10T20:06:29.235 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-10T20:06:29.236 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-10T20:06:29.236 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-10T20:06:29.237 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-10T20:06:29.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:06:29.253 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-10-10T20:06:29.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:06:29.254 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2023-10-10T20:06:29.308 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:06:29.308 INFO:teuthology.task.kernel:Running kernel on smithi161: 4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:06:29.308 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2023-10-10T20:06:29.311 INFO:teuthology.orchestra.run.smithi148.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:06:29.311 INFO:teuthology.task.kernel:Running kernel on smithi148: 4.18.0-305.25.1.el8_4.x86_64 2023-10-10T20:06:29.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2023-10-10T20:06:30.750 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:30.750 INFO:teuthology.orchestra.run.smithi161.stdout:Unable to read consumer identity 2023-10-10T20:06:30.840 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:30.841 INFO:teuthology.orchestra.run.smithi148.stdout:Unable to read consumer identity 2023-10-10T20:06:32.238 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-10-10T20:06:32.337 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-10-10T20:06:33.254 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-10-10T20:06:34.006 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-10-10T20:06:36.537 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 4.8 MB/s | 24 kB 00:00 2023-10-10T20:06:37.258 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-10T20:06:37.259 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-10T20:06:37.259 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-10T20:06:37.286 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-10T20:06:37.407 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-10-10T20:06:37.409 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-10-10T20:06:37.409 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-10-10T20:06:37.513 DEBUG:teuthology.orchestra.run.smithi161:> echo no | sudo yum reinstall kernel || true 2023-10-10T20:06:37.856 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:37.857 INFO:teuthology.orchestra.run.smithi161.stdout:Unable to read consumer identity 2023-10-10T20:06:38.020 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-10T20:06:38.020 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-10T20:06:38.020 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-10T20:06:38.153 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:02 ago on Tue 10 Oct 2023 08:06:36 PM UTC. 2023-10-10T20:06:38.169 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2023-10-10T20:06:38.170 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2023-10-10T20:06:38.170 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2023-10-10T20:06:38.280 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:06:38.280 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:38.281 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:38.293 DEBUG:teuthology.orchestra.run.smithi148:> echo no | sudo yum reinstall kernel || true 2023-10-10T20:06:38.311 INFO:teuthology.orchestra.run.smithi161.stderr:Error: No packages marked for reinstall. 2023-10-10T20:06:38.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum reinstall -y kernel || true 2023-10-10T20:06:38.640 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:38.640 INFO:teuthology.orchestra.run.smithi148.stdout:Unable to read consumer identity 2023-10-10T20:06:38.707 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:38.707 INFO:teuthology.orchestra.run.smithi161.stdout:Unable to read consumer identity 2023-10-10T20:06:38.933 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:01 ago on Tue 10 Oct 2023 08:06:37 PM UTC. 2023-10-10T20:06:38.992 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:02 ago on Tue 10 Oct 2023 08:06:36 PM UTC. 2023-10-10T20:06:39.074 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:06:39.074 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.074 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:Error: No packages marked for reinstall. 2023-10-10T20:06:39.129 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:06:39.129 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.129 INFO:teuthology.orchestra.run.smithi161.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:Error: No packages marked for reinstall. 2023-10-10T20:06:39.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum reinstall -y kernel || true 2023-10-10T20:06:39.191 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:06:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.317 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.317 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.318 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-10T20:06:39.318 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-10T20:06:39.318 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-10T20:06:39.318 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2023-10-10T20:06:39.319 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:06:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.444 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2023-10-10T20:06:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2023-10-10T20:06:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2023-10-10T20:06:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.02 2023-10-10T20:06:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 99.el8_4.1 2023-10-10T20:06:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2023-10-10T20:06:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-10T20:06:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:Group : System Environment/Base 2023-10-10T20:06:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 9264631 2023-10-10T20:06:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2023-10-10T20:06:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-10T20:06:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-10T20:06:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-10T20:06:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-10T20:06:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:Relocations : (not relocatable) 2023-10-10T20:06:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : Red Hat, Inc. 2023-10-10T20:06:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : Red Hat, Inc. 2023-10-10T20:06:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-10T20:06:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2023-10-10T20:06:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2023-10-10T20:06:39.478 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:06:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-10T20:06:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-10T20:06:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-10T20:06:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2023-10-10T20:06:39.479 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:06:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2023-10-10T20:06:39.481 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-10T20:06:39.481 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:39.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-10T20:06:39.495 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-10-10T20:06:39.495 INFO:teuthology.orchestra.run.smithi148.stdout:Unable to read consumer identity 2023-10-10T20:06:39.775 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:02 ago on Tue 10 Oct 2023 08:06:37 PM UTC. 2023-10-10T20:06:39.913 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-10T20:06:39.914 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.914 INFO:teuthology.orchestra.run.smithi148.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-10T20:06:39.928 INFO:teuthology.orchestra.run.smithi148.stderr:Error: No packages marked for reinstall. 2023-10-10T20:06:39.984 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:06:40.109 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.110 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.110 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.110 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-10T20:06:40.111 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-10T20:06:40.111 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-10T20:06:40.111 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2023-10-10T20:06:40.111 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2023-10-10T20:06:40.235 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2023-10-10T20:06:40.263 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2023-10-10T20:06:40.264 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2023-10-10T20:06:40.264 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.02 2023-10-10T20:06:40.264 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 99.el8_4.1 2023-10-10T20:06:40.264 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2023-10-10T20:06:40.264 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-10T20:06:40.265 INFO:teuthology.orchestra.run.smithi148.stdout:Group : System Environment/Base 2023-10-10T20:06:40.265 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 9264631 2023-10-10T20:06:40.265 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2023-10-10T20:06:40.265 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-10T20:06:40.265 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-10T20:06:40.266 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-10T20:06:40.266 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-10T20:06:40.266 INFO:teuthology.orchestra.run.smithi148.stdout:Relocations : (not relocatable) 2023-10-10T20:06:40.266 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : Red Hat, Inc. 2023-10-10T20:06:40.266 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : Red Hat, Inc. 2023-10-10T20:06:40.267 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-10T20:06:40.267 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2023-10-10T20:06:40.267 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2023-10-10T20:06:40.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:06:40.267 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-10T20:06:40.268 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-10T20:06:40.268 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-10T20:06:40.268 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2023-10-10T20:06:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:06:40.268 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2023-10-10T20:06:40.269 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-10T20:06:40.270 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:40.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-10T20:06:40.718 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2023-10-10T20:06:41.606 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2023-10-10T20:06:42.630 INFO:teuthology.orchestra.run.smithi161.stderr:done 2023-10-10T20:06:42.632 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-10-10T20:06:42.686 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.KzqBKVuJVf 2023-10-10T20:06:42.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KzqBKVuJVf 2023-10-10T20:06:42.762 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.KzqBKVuJVf 2023-10-10T20:06:42.922 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.KzqBKVuJVf is 6KB 2023-10-10T20:06:42.971 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.KzqBKVuJVf 2023-10-10T20:06:42.984 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2023-10-10T20:06:43.047 INFO:teuthology.orchestra.run.smithi161.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-10T20:06:43.047 INFO:teuthology.orchestra.run.smithi161.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-10T20:06:43.047 INFO:teuthology.orchestra.run.smithi161.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-10T20:06:43.047 INFO:teuthology.orchestra.run.smithi161.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-10T20:06:43.048 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:43.213 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2023-10-10T20:06:43.558 INFO:teuthology.orchestra.run.smithi148.stderr:done 2023-10-10T20:06:43.559 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2023-10-10T20:06:43.581 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.J00Ava4bP5 2023-10-10T20:06:43.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.J00Ava4bP5 2023-10-10T20:06:43.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.J00Ava4bP5 2023-10-10T20:06:43.815 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.J00Ava4bP5 is 6KB 2023-10-10T20:06:43.834 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.J00Ava4bP5 2023-10-10T20:06:43.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2023-10-10T20:06:43.919 INFO:teuthology.orchestra.run.smithi148.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-10T20:06:43.920 INFO:teuthology.orchestra.run.smithi148.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-10T20:06:43.920 INFO:teuthology.orchestra.run.smithi148.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-10T20:06:43.920 INFO:teuthology.orchestra.run.smithi148.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-10T20:06:43.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-10T20:06:44.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2023-10-10T20:07:13.244 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-10T20:07:13.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:07:13.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:07:14.120 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-10T20:07:14.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:07:14.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:07:31.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-10-10T20:07:32.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-10-10T20:07:40.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:07:40.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:07:41.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:07:41.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:07:43.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-10-10T20:07:44.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-10-10T20:07:55.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:07:55.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:07:56.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:07:56.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:08:14.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-10-10T20:08:29.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:08:29.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:08:30.908 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-10-10T20:08:31.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:08:31.198 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-10T20:08:31.198 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2023-10-10T20:08:31.252 INFO:teuthology.orchestra.run.smithi148.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-10T20:08:31.253 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:08:31.253 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-10T20:08:31.253 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-10T20:08:32.254 DEBUG:teuthology.parallel:result is None 2023-10-10T20:08:34.078 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-10-10T20:08:34.216 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:08:34.216 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-10T20:08:34.217 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-10-10T20:08:34.230 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-10T20:08:34.231 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-10T20:08:34.231 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-10T20:08:34.231 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-10T20:08:35.232 DEBUG:teuthology.parallel:result is None 2023-10-10T20:08:35.232 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-10T20:08:35.240 INFO:teuthology.task.internal:Creating test directory... 2023-10-10T20:08:35.240 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-10T20:08:35.244 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-10T20:08:35.259 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-10T20:08:35.266 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-10T20:08:35.276 INFO:teuthology.task.internal:Creating archive directory... 2023-10-10T20:08:35.276 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-10T20:08:35.304 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-10T20:08:35.334 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-10T20:08:35.340 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-10T20:08:35.341 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-10T20:08:35.363 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-10T20:08:35.453 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:08:35.465 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:08:35.624 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:08:35.652 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-10T20:08:35.654 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-10T20:08:35.662 INFO:teuthology.task.internal:Configuring sudo... 2023-10-10T20:08:35.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-10T20:08:35.664 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-10T20:08:35.691 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-10T20:08:35.700 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-10T20:08:35.701 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-10T20:08:35.736 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-10T20:08:35.751 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:08:35.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:08:35.886 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:08:35.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:08:36.014 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:08:36.014 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-10T20:08:36.082 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:08:36.126 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-10T20:08:36.186 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:08:36.225 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-10T20:08:36.268 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:08:36.269 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-10T20:08:36.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2023-10-10T20:08:36.343 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-10-10T20:08:36.401 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:08:36.442 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:08:36.686 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-10T20:08:36.695 INFO:teuthology.task.internal:Starting timer... 2023-10-10T20:08:36.695 INFO:teuthology.run_tasks:Running task pcp... 2023-10-10T20:08:36.705 INFO:teuthology.run_tasks:Running task selinux... 2023-10-10T20:08:36.713 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-10T20:08:36.714 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2023-10-10T20:08:36.767 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: [ OK ] 2023-10-10T20:08:36.769 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2023-10-10T20:08:36.829 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: [ OK ] 2023-10-10T20:08:36.831 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-10T20:08:36.832 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2023-10-10T20:08:36.850 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2023-10-10T20:08:36.851 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2023-10-10T20:08:36.873 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2023-10-10T20:08:36.874 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2023-10-10T20:08:36.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:08:36.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:08:36.927 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:08:36.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:08:36.953 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-10T20:08:36.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2023-10-10T20:08:37.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2023-10-10T20:08:37.048 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-10T20:08:37.057 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-10T20:08:37.063 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-10T20:08:37.172 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-10T20:08:37.193 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-10T20:08:37.194 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-10T20:16:48.696 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2023-10-10T20:16:48.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:16:48.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:16:48.770 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-10-10T20:16:48.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-10-10T20:16:48.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:16:48.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T20:16:48.900 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-10-10T20:16:48.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-10-10T20:16:48.966 INFO:teuthology.run_tasks:Running task clock... 2023-10-10T20:16:48.981 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-10T20:16:48.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-10T20:16:48.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:16:48.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-10T20:16:48.985 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:16:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-10T20:16:49.026 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-10T20:16:49.035 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-10T20:16:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-10T20:16:49.060 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2023-10-10T20:16:49.070 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2023-10-10T20:16:49.082 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2023-10-10T20:16:49.083 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-10T20:16:49.092 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2023-10-10T20:16:49.098 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-10T20:16:49.106 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-10T20:16:49.120 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-10T20:16:49.160 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2023-10-10T20:16:49.163 INFO:teuthology.orchestra.run.smithi148.stdout:210 Number of sources = 4 2023-10-10T20:16:49.163 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:16:49.164 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2023-10-10T20:16:49.164 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.164 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.164 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.165 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-10-10T20:16:49.191 INFO:teuthology.orchestra.run.smithi161.stdout:210 Number of sources = 4 2023-10-10T20:16:49.191 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:16:49.191 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-10-10T20:16:49.192 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.192 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.192 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.193 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:16:49.194 INFO:teuthology.run_tasks:Running task install... 2023-10-10T20:16:49.204 DEBUG:teuthology.task.install:project ceph 2023-10-10T20:16:49.204 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}} 2023-10-10T20:16:49.205 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'} 2023-10-10T20:16:49.205 INFO:teuthology.task.install:Using flavor: default 2023-10-10T20:16:49.214 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-10T20:16:49.214 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-10T20:16:49.215 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'tag': None, 'wait_for_package': False} 2023-10-10T20:16:49.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:16:49.219 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'tag': None, 'wait_for_package': False} 2023-10-10T20:16:49.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:16:49.455 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/ 2023-10-10T20:16:49.455 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1374.g46d1f59b 2023-10-10T20:16:49.462 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/ 2023-10-10T20:16:49.462 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1374.g46d1f59b 2023-10-10T20:16:49.562 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-10T20:16:49.563 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:16:49.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-10T20:16:49.589 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-10T20:16:49.589 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-10T20:16:49.630 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-10-0720-quincy/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-10T20:16:49.630 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:16:49.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-10T20:16:49.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-10T20:16:49.658 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-10T20:16:49.658 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/46d1f59b549730ba7f93d940a3d32b94c45a4cf7/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-10T20:16:49.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-10T20:16:49.731 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-10T20:16:49.797 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-10T20:16:49.799 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2023-10-10T20:16:49.800 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2023-10-10T20:16:49.862 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2023-10-10T20:16:49.863 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2023-10-10T20:16:50.191 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-10-10T20:16:50.254 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-10-10T20:16:50.542 INFO:teuthology.orchestra.run.smithi148.stdout:44 files removed 2023-10-10T20:16:50.594 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-10T20:16:50.633 INFO:teuthology.orchestra.run.smithi161.stdout:44 files removed 2023-10-10T20:16:50.684 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-10T20:16:50.948 INFO:teuthology.orchestra.run.smithi148.stdout:Updating Subscription Management repositories. 2023-10-10T20:16:51.036 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-10-10T20:16:51.981 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 154 kB/s | 78 kB 00:00 2023-10-10T20:16:52.035 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 184 kB/s | 78 kB 00:00 2023-10-10T20:16:52.205 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 50 kB/s | 9.2 kB 00:00 2023-10-10T20:16:52.446 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 25 kB/s | 9.2 kB 00:00 2023-10-10T20:16:52.541 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.7 kB/s | 1.4 kB 00:00 2023-10-10T20:16:52.800 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-10-10T20:16:52.922 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 3.2 kB/s | 1.4 kB 00:00 2023-10-10T20:16:53.111 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-10-10T20:16:53.873 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-10-10T20:16:54.596 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2023-10-10T20:16:57.165 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 3.9 MB/s | 24 kB 00:00 2023-10-10T20:16:57.517 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 MB/s | 33 MB 00:00 2023-10-10T20:16:57.945 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-10T20:16:58.301 INFO:teuthology.orchestra.run.smithi148.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 120 MB/s | 33 MB 00:00 2023-10-10T20:17:02.781 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 108 MB/s | 38 MB 00:00 2023-10-10T20:17:03.422 INFO:teuthology.orchestra.run.smithi148.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 117 MB/s | 38 MB 00:00 2023-10-10T20:17:11.071 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:17:11.071 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:17:11.082 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-10T20:17:11.083 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-10T20:17:11.084 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-10T20:17:11.085 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-10T20:17:11.086 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-10T20:17:11.089 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-10T20:17:11.089 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-10T20:17:11.576 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-10-10T20:17:11.587 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================================================== 2023-10-10T20:17:11.587 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-10-10T20:17:11.588 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================================================== 2023-10-10T20:17:11.588 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-10-10T20:17:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-10T20:17:11.588 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 6.4 k 2023-10-10T20:17:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.2 M 2023-10-10T20:17:11.589 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 876 k 2023-10-10T20:17:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 161 k 2023-10-10T20:17:11.590 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 1.6 M 2023-10-10T20:17:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 121 k 2023-10-10T20:17:11.591 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 1.7 M 2023-10-10T20:17:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 7.4 M 2023-10-10T20:17:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 47 k 2023-10-10T20:17:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:17:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 52 M 2023-10-10T20:17:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 267 k 2023-10-10T20:17:11.593 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 80 k 2023-10-10T20:17:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 255 k 2023-10-10T20:17:11.594 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 18 k 2023-10-10T20:17:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-10T20:17:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-10T20:17:11.595 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-10T20:17:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 25 k 2023-10-10T20:17:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 797 k 2023-10-10T20:17:11.596 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 130 k 2023-10-10T20:17:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-10T20:17:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-10T20:17:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-10T20:17:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 190 k 2023-10-10T20:17:11.597 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 371 k 2023-10-10T20:17:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 358 k 2023-10-10T20:17:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 107 k 2023-10-10T20:17:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 90 k 2023-10-10T20:17:11.598 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:17:11.599 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 173 k 2023-10-10T20:17:11.599 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-10T20:17:11.599 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2023-10-10T20:17:11.599 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:17:11.600 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.6 M 2023-10-10T20:17:11.600 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-10-10T20:17:11.600 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-10T20:17:11.601 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 23 M 2023-10-10T20:17:11.601 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 23 k 2023-10-10T20:17:11.602 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 2.3 M 2023-10-10T20:17:11.602 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 238 k 2023-10-10T20:17:11.602 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.9 M 2023-10-10T20:17:11.603 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:17:11.603 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 14 k 2023-10-10T20:17:11.603 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 24 k 2023-10-10T20:17:11.603 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-10T20:17:11.604 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-10T20:17:11.604 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-10T20:17:11.604 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-10T20:17:11.604 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-10T20:17:11.605 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 174 k 2023-10-10T20:17:11.605 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-10T20:17:11.605 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-10T20:17:11.605 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-10T20:17:11.606 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-10T20:17:11.606 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-10T20:17:11.606 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 507 k 2023-10-10T20:17:11.607 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-10T20:17:11.607 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.6 M 2023-10-10T20:17:11.607 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-10T20:17:11.607 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-10T20:17:11.608 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-10T20:17:11.608 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-10T20:17:11.608 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-10T20:17:11.609 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-10T20:17:11.609 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-10T20:17:11.609 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-10T20:17:11.610 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-10T20:17:11.610 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-10T20:17:11.610 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-10T20:17:11.610 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-10T20:17:11.611 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-10T20:17:11.611 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-10T20:17:11.611 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-10T20:17:11.612 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-10T20:17:11.612 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 45 k 2023-10-10T20:17:11.612 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 115 k 2023-10-10T20:17:11.613 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-10T20:17:11.613 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-10T20:17:11.614 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-10T20:17:11.614 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-10T20:17:11.614 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-10T20:17:11.615 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-10T20:17:11.615 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-10T20:17:11.615 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-10T20:17:11.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-10T20:17:11.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:17:11.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-10T20:17:11.617 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-10T20:17:11.617 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-10T20:17:11.617 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-10T20:17:11.618 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-10T20:17:11.618 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-10T20:17:11.618 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-10T20:17:11.619 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-10T20:17:11.619 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-10T20:17:11.619 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-10T20:17:11.619 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-10T20:17:11.620 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-10T20:17:11.620 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-10T20:17:11.620 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-10T20:17:11.620 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-10T20:17:11.620 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-10T20:17:11.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-10T20:17:11.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-10T20:17:11.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-10T20:17:11.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-10T20:17:11.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-10T20:17:11.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-10T20:17:11.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-10T20:17:11.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-10T20:17:11.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-10T20:17:11.623 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-10-10T20:17:11.624 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================================================== 2023-10-10T20:17:11.625 INFO:teuthology.orchestra.run.smithi161.stdout:Install 118 Packages 2023-10-10T20:17:11.625 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 2 Packages 2023-10-10T20:17:11.625 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:17:11.626 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 202 M 2023-10-10T20:17:11.626 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-10-10T20:17:11.715 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:17:11.716 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-10T20:17:11.726 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-10T20:17:11.727 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-10T20:17:11.728 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-10T20:17:11.729 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-10T20:17:11.731 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-10T20:17:11.733 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-10T20:17:11.734 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-10T20:17:11.785 INFO:teuthology.orchestra.run.smithi161.stdout:(1/120): ceph-17.2.6-1374.g46d1f59b.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-10T20:17:12.082 INFO:teuthology.orchestra.run.smithi161.stdout:(2/120): ceph-fuse-17.2.6-1374.g46d1f59b.el8.x8 2.9 MB/s | 876 kB 00:00 2023-10-10T20:17:12.141 INFO:teuthology.orchestra.run.smithi161.stdout:(3/120): ceph-immutable-object-cache-17.2.6-137 2.7 MB/s | 161 kB 00:00 2023-10-10T20:17:12.222 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2023-10-10T20:17:12.233 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================================================== 2023-10-10T20:17:12.233 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2023-10-10T20:17:12.234 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================================================== 2023-10-10T20:17:12.234 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2023-10-10T20:17:12.234 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-10T20:17:12.234 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 6.4 k 2023-10-10T20:17:12.234 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.2 M 2023-10-10T20:17:12.235 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 876 k 2023-10-10T20:17:12.235 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 161 k 2023-10-10T20:17:12.235 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 1.6 M 2023-10-10T20:17:12.235 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 121 k 2023-10-10T20:17:12.235 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 1.7 M 2023-10-10T20:17:12.236 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 7.4 M 2023-10-10T20:17:12.236 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 47 k 2023-10-10T20:17:12.236 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:17:12.236 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 52 M 2023-10-10T20:17:12.236 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 267 k 2023-10-10T20:17:12.237 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 80 k 2023-10-10T20:17:12.237 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 255 k 2023-10-10T20:17:12.237 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 18 k 2023-10-10T20:17:12.237 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-10T20:17:12.237 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-10T20:17:12.238 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-10T20:17:12.238 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 25 k 2023-10-10T20:17:12.238 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 797 k 2023-10-10T20:17:12.238 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 130 k 2023-10-10T20:17:12.239 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-10T20:17:12.239 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-10T20:17:12.239 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-10T20:17:12.239 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 190 k 2023-10-10T20:17:12.239 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 371 k 2023-10-10T20:17:12.240 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 358 k 2023-10-10T20:17:12.240 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 107 k 2023-10-10T20:17:12.240 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 90 k 2023-10-10T20:17:12.240 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:17:12.240 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 173 k 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.5 M 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.6 M 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2023-10-10T20:17:12.241 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-10T20:17:12.242 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 23 M 2023-10-10T20:17:12.242 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 23 k 2023-10-10T20:17:12.242 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 2.3 M 2023-10-10T20:17:12.242 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 238 k 2023-10-10T20:17:12.243 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 3.9 M 2023-10-10T20:17:12.243 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 17 M 2023-10-10T20:17:12.243 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1374.g46d1f59b.el8 ceph-noarch 14 k 2023-10-10T20:17:12.243 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 24 k 2023-10-10T20:17:12.243 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 174 k 2023-10-10T20:17:12.244 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-10T20:17:12.245 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-10T20:17:12.245 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-10T20:17:12.245 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-10T20:17:12.245 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-10T20:17:12.245 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 507 k 2023-10-10T20:17:12.246 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-10T20:17:12.246 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 5.6 M 2023-10-10T20:17:12.246 INFO:teuthology.orchestra.run.smithi148.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-10T20:17:12.246 INFO:teuthology.orchestra.run.smithi148.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-10T20:17:12.246 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-10T20:17:12.247 INFO:teuthology.orchestra.run.smithi148.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-10T20:17:12.247 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-10T20:17:12.247 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-10T20:17:12.247 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-10T20:17:12.247 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-10T20:17:12.248 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-10T20:17:12.248 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-10T20:17:12.248 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-10T20:17:12.248 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-10T20:17:12.248 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-10T20:17:12.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-10T20:17:12.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-10T20:17:12.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-10T20:17:12.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 45 k 2023-10-10T20:17:12.249 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:17.2.6-1374.g46d1f59b.el8 ceph 115 k 2023-10-10T20:17:12.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-10T20:17:12.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-10T20:17:12.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-10T20:17:12.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-10T20:17:12.250 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-10T20:17:12.251 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-10T20:17:12.251 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-10T20:17:12.251 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-10T20:17:12.251 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-10T20:17:12.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:17:12.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-10T20:17:12.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-10T20:17:12.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-10T20:17:12.252 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-10T20:17:12.253 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-10T20:17:12.254 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-10T20:17:12.254 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-10T20:17:12.255 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-10T20:17:12.255 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-10T20:17:12.255 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-10T20:17:12.255 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-10T20:17:12.256 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-10T20:17:12.256 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-10T20:17:12.257 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-10T20:17:12.257 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-10T20:17:12.257 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-10T20:17:12.258 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-10T20:17:12.258 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-10T20:17:12.258 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-10T20:17:12.258 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-10T20:17:12.259 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-10T20:17:12.259 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-10T20:17:12.260 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-10T20:17:12.260 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-10T20:17:12.260 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-10T20:17:12.260 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-10T20:17:12.261 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-10T20:17:12.261 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-10T20:17:12.261 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-10T20:17:12.262 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-10T20:17:12.262 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-10T20:17:12.262 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================================================== 2023-10-10T20:17:12.263 INFO:teuthology.orchestra.run.smithi148.stdout:Install 118 Packages 2023-10-10T20:17:12.264 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2023-10-10T20:17:12.264 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:17:12.265 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 202 M 2023-10-10T20:17:12.265 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2023-10-10T20:17:12.578 INFO:teuthology.orchestra.run.smithi161.stdout:(4/120): ceph-mds-17.2.6-1374.g46d1f59b.el8.x86 5.3 MB/s | 2.3 MB 00:00 2023-10-10T20:17:12.592 INFO:teuthology.orchestra.run.smithi148.stdout:(1/120): ceph-17.2.6-1374.g46d1f59b.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-10-10T20:17:12.717 INFO:teuthology.orchestra.run.smithi161.stdout:(5/120): ceph-base-17.2.6-1374.g46d1f59b.el8.x8 4.8 MB/s | 5.2 MB 00:01 2023-10-10T20:17:12.999 INFO:teuthology.orchestra.run.smithi161.stdout:(6/120): ceph-mgr-17.2.6-1374.g46d1f59b.el8.x86 3.7 MB/s | 1.6 MB 00:00 2023-10-10T20:17:13.260 INFO:teuthology.orchestra.run.smithi161.stdout:(7/120): ceph-common-17.2.6-1374.g46d1f59b.el8. 14 MB/s | 23 MB 00:01 2023-10-10T20:17:13.498 INFO:teuthology.orchestra.run.smithi161.stdout:(8/120): ceph-mon-17.2.6-1374.g46d1f59b.el8.x86 5.0 MB/s | 3.9 MB 00:00 2023-10-10T20:17:13.573 INFO:teuthology.orchestra.run.smithi161.stdout:(9/120): ceph-selinux-17.2.6-1374.g46d1f59b.el8 325 kB/s | 24 kB 00:00 2023-10-10T20:17:13.960 INFO:teuthology.orchestra.run.smithi148.stdout:(2/120): ceph-fuse-17.2.6-1374.g46d1f59b.el8.x8 641 kB/s | 876 kB 00:01 2023-10-10T20:17:14.237 INFO:teuthology.orchestra.run.smithi148.stdout:(3/120): ceph-immutable-object-cache-17.2.6-137 583 kB/s | 161 kB 00:00 2023-10-10T20:17:15.175 INFO:teuthology.orchestra.run.smithi161.stdout:(10/120): ceph-radosgw-17.2.6-1374.g46d1f59b.el 8.8 MB/s | 17 MB 00:01 2023-10-10T20:17:15.195 INFO:teuthology.orchestra.run.smithi148.stdout:(4/120): ceph-base-17.2.6-1374.g46d1f59b.el8.x8 1.8 MB/s | 5.2 MB 00:02 2023-10-10T20:17:15.220 INFO:teuthology.orchestra.run.smithi161.stdout:(11/120): cephfs-mirror-17.2.6-1374.g46d1f59b.e 5.5 MB/s | 255 kB 00:00 2023-10-10T20:17:15.273 INFO:teuthology.orchestra.run.smithi161.stdout:(12/120): libcephfs-devel-17.2.6-1374.g46d1f59b 475 kB/s | 25 kB 00:00 2023-10-10T20:17:15.399 INFO:teuthology.orchestra.run.smithi161.stdout:(13/120): libcephfs2-17.2.6-1374.g46d1f59b.el8. 6.2 MB/s | 797 kB 00:00 2023-10-10T20:17:15.456 INFO:teuthology.orchestra.run.smithi161.stdout:(14/120): libcephsqlite-17.2.6-1374.g46d1f59b.e 3.0 MB/s | 174 kB 00:00 2023-10-10T20:17:15.511 INFO:teuthology.orchestra.run.smithi161.stdout:(15/120): librados-devel-17.2.6-1374.g46d1f59b. 2.3 MB/s | 130 kB 00:00 2023-10-10T20:17:15.564 INFO:teuthology.orchestra.run.smithi148.stdout:(5/120): ceph-mgr-17.2.6-1374.g46d1f59b.el8.x86 4.3 MB/s | 1.6 MB 00:00 2023-10-10T20:17:15.573 INFO:teuthology.orchestra.run.smithi161.stdout:(16/120): libradosstriper1-17.2.6-1374.g46d1f59 8.0 MB/s | 507 kB 00:00 2023-10-10T20:17:15.946 INFO:teuthology.orchestra.run.smithi148.stdout:(6/120): ceph-common-17.2.6-1374.g46d1f59b.el8. 6.4 MB/s | 23 MB 00:03 2023-10-10T20:17:16.278 INFO:teuthology.orchestra.run.smithi161.stdout:(17/120): librgw2-17.2.6-1374.g46d1f59b.el8.x86 7.9 MB/s | 5.6 MB 00:00 2023-10-10T20:17:16.321 INFO:teuthology.orchestra.run.smithi161.stdout:(18/120): python3-ceph-argparse-17.2.6-1374.g46 1.0 MB/s | 45 kB 00:00 2023-10-10T20:17:16.378 INFO:teuthology.orchestra.run.smithi148.stdout:(7/120): ceph-mon-17.2.6-1374.g46d1f59b.el8.x86 4.8 MB/s | 3.9 MB 00:00 2023-10-10T20:17:16.453 INFO:teuthology.orchestra.run.smithi161.stdout:(19/120): ceph-osd-17.2.6-1374.g46d1f59b.el8.x8 4.8 MB/s | 17 MB 00:03 2023-10-10T20:17:16.454 INFO:teuthology.orchestra.run.smithi161.stdout:(20/120): python3-ceph-common-17.2.6-1374.g46d1 865 kB/s | 115 kB 00:00 2023-10-10T20:17:16.510 INFO:teuthology.orchestra.run.smithi161.stdout:(21/120): python3-rados-17.2.6-1374.g46d1f59b.e 6.5 MB/s | 371 kB 00:00 2023-10-10T20:17:16.512 INFO:teuthology.orchestra.run.smithi161.stdout:(22/120): python3-cephfs-17.2.6-1374.g46d1f59b. 3.2 MB/s | 190 kB 00:00 2023-10-10T20:17:16.566 INFO:teuthology.orchestra.run.smithi161.stdout:(23/120): python3-rbd-17.2.6-1374.g46d1f59b.el8 6.3 MB/s | 358 kB 00:00 2023-10-10T20:17:16.585 INFO:teuthology.orchestra.run.smithi161.stdout:(24/120): python3-rgw-17.2.6-1374.g46d1f59b.el8 1.4 MB/s | 107 kB 00:00 2023-10-10T20:17:16.616 INFO:teuthology.orchestra.run.smithi161.stdout:(25/120): rbd-fuse-17.2.6-1374.g46d1f59b.el8.x8 1.8 MB/s | 90 kB 00:00 2023-10-10T20:17:16.668 INFO:teuthology.orchestra.run.smithi161.stdout:(26/120): rbd-nbd-17.2.6-1374.g46d1f59b.el8.x86 3.3 MB/s | 173 kB 00:00 2023-10-10T20:17:16.677 INFO:teuthology.orchestra.run.smithi148.stdout:(8/120): ceph-mds-17.2.6-1374.g46d1f59b.el8.x86 967 kB/s | 2.3 MB 00:02 2023-10-10T20:17:16.717 INFO:teuthology.orchestra.run.smithi161.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1374.g 474 kB/s | 23 kB 00:00 2023-10-10T20:17:16.744 INFO:teuthology.orchestra.run.smithi148.stdout:(9/120): ceph-selinux-17.2.6-1374.g46d1f59b.el8 364 kB/s | 24 kB 00:00 2023-10-10T20:17:16.764 INFO:teuthology.orchestra.run.smithi161.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1374.g46d1f59 2.5 MB/s | 121 kB 00:00 2023-10-10T20:17:17.017 INFO:teuthology.orchestra.run.smithi161.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1374.g46d1f 6.7 MB/s | 1.7 MB 00:00 2023-10-10T20:17:17.276 INFO:teuthology.orchestra.run.smithi161.stdout:(30/120): rbd-mirror-17.2.6-1374.g46d1f59b.el8. 5.0 MB/s | 3.5 MB 00:00 2023-10-10T20:17:17.357 INFO:teuthology.orchestra.run.smithi161.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1374.g46 2.9 MB/s | 238 kB 00:00 2023-10-10T20:17:17.433 INFO:teuthology.orchestra.run.smithi161.stdout:(32/120): ceph-mgr-rook-17.2.6-1374.g46d1f59b.e 628 kB/s | 47 kB 00:00 2023-10-10T20:17:17.508 INFO:teuthology.orchestra.run.smithi161.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1374.g4 194 kB/s | 14 kB 00:00 2023-10-10T20:17:17.592 INFO:teuthology.orchestra.run.smithi161.stdout:(34/120): ceph-volume-17.2.6-1374.g46d1f59b.el8 3.1 MB/s | 267 kB 00:00 2023-10-10T20:17:17.672 INFO:teuthology.orchestra.run.smithi161.stdout:(35/120): cephadm-17.2.6-1374.g46d1f59b.el8.noa 1.0 MB/s | 80 kB 00:00 2023-10-10T20:17:17.749 INFO:teuthology.orchestra.run.smithi161.stdout:(36/120): cephfs-top-17.2.6-1374.g46d1f59b.el8. 234 kB/s | 18 kB 00:00 2023-10-10T20:17:17.889 INFO:teuthology.orchestra.run.smithi161.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 8.5 MB/s | 7.4 MB 00:00 2023-10-10T20:17:17.891 INFO:teuthology.orchestra.run.smithi161.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.1 MB/s | 306 kB 00:00 2023-10-10T20:17:17.915 INFO:teuthology.orchestra.run.smithi161.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 3.1 MB/s | 75 kB 00:00 2023-10-10T20:17:17.973 INFO:teuthology.orchestra.run.smithi161.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 708 kB/s | 59 kB 00:00 2023-10-10T20:17:17.977 INFO:teuthology.orchestra.run.smithi161.stdout:(41/120): python3-asyncssh-2.7.0-2.el8.noarch.r 7.5 MB/s | 476 kB 00:00 2023-10-10T20:17:18.002 INFO:teuthology.orchestra.run.smithi161.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 7.1 MB/s | 185 kB 00:00 2023-10-10T20:17:18.008 INFO:teuthology.orchestra.run.smithi161.stdout:(43/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.2 MB/s | 44 kB 00:00 2023-10-10T20:17:18.022 INFO:teuthology.orchestra.run.smithi161.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 1.7 MB/s | 33 kB 00:00 2023-10-10T20:17:18.031 INFO:teuthology.orchestra.run.smithi161.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 711 kB/s | 16 kB 00:00 2023-10-10T20:17:18.069 INFO:teuthology.orchestra.run.smithi161.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.6 MB/s | 173 kB 00:00 2023-10-10T20:17:18.089 INFO:teuthology.orchestra.run.smithi161.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 4.0 MB/s | 81 kB 00:00 2023-10-10T20:17:18.108 INFO:teuthology.orchestra.run.smithi161.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 578 kB/s | 11 kB 00:00 2023-10-10T20:17:18.125 INFO:teuthology.orchestra.run.smithi161.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 4.0 MB/s | 384 kB 00:00 2023-10-10T20:17:18.128 INFO:teuthology.orchestra.run.smithi161.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 915 kB/s | 18 kB 00:00 2023-10-10T20:17:18.154 INFO:teuthology.orchestra.run.smithi161.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.8 MB/s | 49 kB 00:00 2023-10-10T20:17:18.174 INFO:teuthology.orchestra.run.smithi161.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 2.9 MB/s | 59 kB 00:00 2023-10-10T20:17:18.189 INFO:teuthology.orchestra.run.smithi161.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 18 MB/s | 1.2 MB 00:00 2023-10-10T20:17:18.194 INFO:teuthology.orchestra.run.smithi161.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 2.9 MB/s | 60 kB 00:00 2023-10-10T20:17:18.214 INFO:teuthology.orchestra.run.smithi161.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 11 MB/s | 283 kB 00:00 2023-10-10T20:17:18.215 INFO:teuthology.orchestra.run.smithi161.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 775 kB/s | 16 kB 00:00 2023-10-10T20:17:18.219 INFO:teuthology.orchestra.run.smithi148.stdout:(10/120): ceph-osd-17.2.6-1374.g46d1f59b.el8.x8 7.4 MB/s | 17 MB 00:02 2023-10-10T20:17:18.234 INFO:teuthology.orchestra.run.smithi161.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.7 MB/s | 34 kB 00:00 2023-10-10T20:17:18.249 INFO:teuthology.orchestra.run.smithi161.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 5.6 MB/s | 196 kB 00:00 2023-10-10T20:17:18.254 INFO:teuthology.orchestra.run.smithi161.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 3.2 MB/s | 64 kB 00:00 2023-10-10T20:17:18.255 INFO:teuthology.orchestra.run.smithi148.stdout:(11/120): cephfs-mirror-17.2.6-1374.g46d1f59b.e 7.1 MB/s | 255 kB 00:00 2023-10-10T20:17:18.269 INFO:teuthology.orchestra.run.smithi161.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 973 kB/s | 19 kB 00:00 2023-10-10T20:17:18.274 INFO:teuthology.orchestra.run.smithi161.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 1.2 MB/s | 24 kB 00:00 2023-10-10T20:17:18.283 INFO:teuthology.orchestra.run.smithi148.stdout:(12/120): libcephfs-devel-17.2.6-1374.g46d1f59b 896 kB/s | 25 kB 00:00 2023-10-10T20:17:18.289 INFO:teuthology.orchestra.run.smithi161.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 1.5 MB/s | 29 kB 00:00 2023-10-10T20:17:18.293 INFO:teuthology.orchestra.run.smithi161.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.4 MB/s | 27 kB 00:00 2023-10-10T20:17:18.310 INFO:teuthology.orchestra.run.smithi161.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 11 MB/s | 255 kB 00:00 2023-10-10T20:17:18.318 INFO:teuthology.orchestra.run.smithi161.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 10 MB/s | 251 kB 00:00 2023-10-10T20:17:18.330 INFO:teuthology.orchestra.run.smithi161.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 3.1 MB/s | 61 kB 00:00 2023-10-10T20:17:18.338 INFO:teuthology.orchestra.run.smithi161.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 4.1 MB/s | 85 kB 00:00 2023-10-10T20:17:18.351 INFO:teuthology.orchestra.run.smithi161.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.1 MB/s | 23 kB 00:00 2023-10-10T20:17:18.354 INFO:teuthology.orchestra.run.smithi161.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-10-10T20:17:18.358 INFO:teuthology.orchestra.run.smithi148.stdout:(13/120): libcephfs2-17.2.6-1374.g46d1f59b.el8. 10 MB/s | 797 kB 00:00 2023-10-10T20:17:18.389 INFO:teuthology.orchestra.run.smithi148.stdout:(14/120): libcephsqlite-17.2.6-1374.g46d1f59b.e 5.4 MB/s | 174 kB 00:00 2023-10-10T20:17:18.413 INFO:teuthology.orchestra.run.smithi161.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 23 MB/s | 1.7 MB 00:00 2023-10-10T20:17:18.418 INFO:teuthology.orchestra.run.smithi148.stdout:(15/120): librados-devel-17.2.6-1374.g46d1f59b. 4.4 MB/s | 130 kB 00:00 2023-10-10T20:17:18.450 INFO:teuthology.orchestra.run.smithi161.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.2 MB/s | 44 kB 00:00 2023-10-10T20:17:18.466 INFO:teuthology.orchestra.run.smithi161.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.7 MB/s | 27 kB 00:00 2023-10-10T20:17:18.481 INFO:teuthology.orchestra.run.smithi148.stdout:(16/120): libradosstriper1-17.2.6-1374.g46d1f59 8.0 MB/s | 507 kB 00:00 2023-10-10T20:17:18.484 INFO:teuthology.orchestra.run.smithi161.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 9.0 MB/s | 161 kB 00:00 2023-10-10T20:17:18.504 INFO:teuthology.orchestra.run.smithi161.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 16 MB/s | 320 kB 00:00 2023-10-10T20:17:18.524 INFO:teuthology.orchestra.run.smithi161.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 23 MB/s | 457 kB 00:00 2023-10-10T20:17:18.638 INFO:teuthology.orchestra.run.smithi161.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 51 MB/s | 15 MB 00:00 2023-10-10T20:17:18.653 INFO:teuthology.orchestra.run.smithi161.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.7 MB/s | 39 kB 00:00 2023-10-10T20:17:18.675 INFO:teuthology.orchestra.run.smithi161.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 30 MB/s | 688 kB 00:00 2023-10-10T20:17:18.693 INFO:teuthology.orchestra.run.smithi161.stdout:(79/120): python3-babel-2.5.1-5.el8.noarch.rpm 28 MB/s | 4.8 MB 00:00 2023-10-10T20:17:18.696 INFO:teuthology.orchestra.run.smithi161.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2023-10-10T20:17:18.709 INFO:teuthology.orchestra.run.smithi161.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 14 MB/s | 225 kB 00:00 2023-10-10T20:17:18.714 INFO:teuthology.orchestra.run.smithi161.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.2 MB/s | 126 kB 00:00 2023-10-10T20:17:18.733 INFO:teuthology.orchestra.run.smithi161.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.8 MB/s | 187 kB 00:00 2023-10-10T20:17:18.757 INFO:teuthology.orchestra.run.smithi161.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 22 MB/s | 538 kB 00:00 2023-10-10T20:17:18.771 INFO:teuthology.orchestra.run.smithi161.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 60 MB/s | 3.7 MB 00:00 2023-10-10T20:17:18.776 INFO:teuthology.orchestra.run.smithi161.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 15 MB/s | 302 kB 00:00 2023-10-10T20:17:18.790 INFO:teuthology.orchestra.run.smithi161.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 8.5 MB/s | 157 kB 00:00 2023-10-10T20:17:18.794 INFO:teuthology.orchestra.run.smithi161.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 6.1 MB/s | 110 kB 00:00 2023-10-10T20:17:18.809 INFO:teuthology.orchestra.run.smithi161.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-10-10T20:17:18.811 INFO:teuthology.orchestra.run.smithi161.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-10-10T20:17:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:(91/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.3 MB/s | 54 kB 00:00 2023-10-10T20:17:18.830 INFO:teuthology.orchestra.run.smithi161.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 33 MB/s | 709 kB 00:00 2023-10-10T20:17:18.911 INFO:teuthology.orchestra.run.smithi161.stdout:(93/120): openblas-0.3.12-1.el8.x86_64.rpm 57 MB/s | 4.6 MB 00:00 2023-10-10T20:17:18.926 INFO:teuthology.orchestra.run.smithi161.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 48 MB/s | 4.7 MB 00:00 2023-10-10T20:17:18.936 INFO:teuthology.orchestra.run.smithi161.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 29 MB/s | 713 kB 00:00 2023-10-10T20:17:18.943 INFO:teuthology.orchestra.run.smithi161.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2023-10-10T20:17:18.949 INFO:teuthology.orchestra.run.smithi161.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.2 MB/s | 43 kB 00:00 2023-10-10T20:17:18.961 INFO:teuthology.orchestra.run.smithi161.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.5 MB/s | 155 kB 00:00 2023-10-10T20:17:18.965 INFO:teuthology.orchestra.run.smithi161.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-10-10T20:17:18.980 INFO:teuthology.orchestra.run.smithi161.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.4 MB/s | 48 kB 00:00 2023-10-10T20:17:18.991 INFO:teuthology.orchestra.run.smithi148.stdout:(17/120): ceph-radosgw-17.2.6-1374.g46d1f59b.el 6.4 MB/s | 17 MB 00:02 2023-10-10T20:17:18.991 INFO:teuthology.orchestra.run.smithi161.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 8.7 MB/s | 229 kB 00:00 2023-10-10T20:17:18.997 INFO:teuthology.orchestra.run.smithi161.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 2.6 MB/s | 43 kB 00:00 2023-10-10T20:17:19.013 INFO:teuthology.orchestra.run.smithi161.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.1 MB/s | 18 kB 00:00 2023-10-10T20:17:19.015 INFO:teuthology.orchestra.run.smithi148.stdout:(18/120): librgw2-17.2.6-1374.g46d1f59b.el8.x86 10 MB/s | 5.6 MB 00:00 2023-10-10T20:17:19.031 INFO:teuthology.orchestra.run.smithi148.stdout:(19/120): python3-ceph-argparse-17.2.6-1374.g46 1.1 MB/s | 45 kB 00:00 2023-10-10T20:17:19.032 INFO:teuthology.orchestra.run.smithi161.stdout:(104/120): openssl-devel-1.1.1g-15.el8_3.x86_64 56 MB/s | 2.3 MB 00:00 2023-10-10T20:17:19.037 INFO:teuthology.orchestra.run.smithi148.stdout:(20/120): python3-ceph-common-17.2.6-1374.g46d1 5.2 MB/s | 115 kB 00:00 2023-10-10T20:17:19.040 INFO:teuthology.orchestra.run.smithi161.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 22 MB/s | 605 kB 00:00 2023-10-10T20:17:19.049 INFO:teuthology.orchestra.run.smithi161.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 5.1 MB/s | 83 kB 00:00 2023-10-10T20:17:19.057 INFO:teuthology.orchestra.run.smithi161.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 3.4 MB/s | 59 kB 00:00 2023-10-10T20:17:19.065 INFO:teuthology.orchestra.run.smithi161.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-10T20:17:19.069 INFO:teuthology.orchestra.run.smithi148.stdout:(21/120): python3-rados-17.2.6-1374.g46d1f59b.e 11 MB/s | 371 kB 00:00 2023-10-10T20:17:19.070 INFO:teuthology.orchestra.run.smithi148.stdout:(22/120): python3-cephfs-17.2.6-1374.g46d1f59b. 4.8 MB/s | 190 kB 00:00 2023-10-10T20:17:19.072 INFO:teuthology.orchestra.run.smithi161.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.5 MB/s | 38 kB 00:00 2023-10-10T20:17:19.080 INFO:teuthology.orchestra.run.smithi161.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.4 MB/s | 86 kB 00:00 2023-10-10T20:17:19.086 INFO:teuthology.orchestra.run.smithi161.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.3 MB/s | 58 kB 00:00 2023-10-10T20:17:19.095 INFO:teuthology.orchestra.run.smithi161.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-10-10T20:17:19.096 INFO:teuthology.orchestra.run.smithi148.stdout:(23/120): python3-rbd-17.2.6-1374.g46d1f59b.el8 13 MB/s | 358 kB 00:00 2023-10-10T20:17:19.104 INFO:teuthology.orchestra.run.smithi161.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 30 MB/s | 559 kB 00:00 2023-10-10T20:17:19.115 INFO:teuthology.orchestra.run.smithi148.stdout:(24/120): python3-rgw-17.2.6-1374.g46d1f59b.el8 2.3 MB/s | 107 kB 00:00 2023-10-10T20:17:19.115 INFO:teuthology.orchestra.run.smithi161.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 31 MB/s | 639 kB 00:00 2023-10-10T20:17:19.117 INFO:teuthology.orchestra.run.smithi148.stdout:(25/120): rbd-fuse-17.2.6-1374.g46d1f59b.el8.x8 4.1 MB/s | 90 kB 00:00 2023-10-10T20:17:19.118 INFO:teuthology.orchestra.run.smithi161.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 5.0 MB/s | 70 kB 00:00 2023-10-10T20:17:19.130 INFO:teuthology.orchestra.run.smithi161.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-10-10T20:17:19.133 INFO:teuthology.orchestra.run.smithi161.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.3 MB/s | 97 kB 00:00 2023-10-10T20:17:19.150 INFO:teuthology.orchestra.run.smithi148.stdout:(26/120): rbd-nbd-17.2.6-1374.g46d1f59b.el8.x86 5.2 MB/s | 173 kB 00:00 2023-10-10T20:17:19.172 INFO:teuthology.orchestra.run.smithi148.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1374.g 1.1 MB/s | 23 kB 00:00 2023-10-10T20:17:19.198 INFO:teuthology.orchestra.run.smithi148.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1374.g46d1f59 4.4 MB/s | 121 kB 00:00 2023-10-10T20:17:19.416 INFO:teuthology.orchestra.run.smithi148.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1374.g46d1f 7.8 MB/s | 1.7 MB 00:00 2023-10-10T20:17:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:(30/120): rbd-mirror-17.2.6-1374.g46d1f59b.el8. 6.7 MB/s | 3.5 MB 00:00 2023-10-10T20:17:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1374.g46 3.3 MB/s | 238 kB 00:00 2023-10-10T20:17:19.748 INFO:teuthology.orchestra.run.smithi161.stdout:(118/120): librbd1-17.2.6-1374.g46d1f59b.el8.x8 5.8 MB/s | 3.6 MB 00:00 2023-10-10T20:17:19.757 INFO:teuthology.orchestra.run.smithi148.stdout:(32/120): ceph-mgr-rook-17.2.6-1374.g46d1f59b.e 826 kB/s | 47 kB 00:00 2023-10-10T20:17:19.806 INFO:teuthology.orchestra.run.smithi148.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1374.g4 309 kB/s | 14 kB 00:00 2023-10-10T20:17:19.858 INFO:teuthology.orchestra.run.smithi148.stdout:(34/120): ceph-volume-17.2.6-1374.g46d1f59b.el8 4.9 MB/s | 267 kB 00:00 2023-10-10T20:17:19.906 INFO:teuthology.orchestra.run.smithi148.stdout:(35/120): cephadm-17.2.6-1374.g46d1f59b.el8.noa 1.6 MB/s | 80 kB 00:00 2023-10-10T20:17:19.951 INFO:teuthology.orchestra.run.smithi148.stdout:(36/120): cephfs-top-17.2.6-1374.g46d1f59b.el8. 399 kB/s | 18 kB 00:00 2023-10-10T20:17:20.056 INFO:teuthology.orchestra.run.smithi161.stdout:(119/120): librados2-17.2.6-1374.g46d1f59b.el8. 3.7 MB/s | 3.5 MB 00:00 2023-10-10T20:17:20.215 INFO:teuthology.orchestra.run.smithi148.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 9.3 MB/s | 7.4 MB 00:00 2023-10-10T20:17:20.312 INFO:teuthology.orchestra.run.smithi148.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 851 kB/s | 306 kB 00:00 2023-10-10T20:17:20.369 INFO:teuthology.orchestra.run.smithi148.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-10-10T20:17:20.462 INFO:teuthology.orchestra.run.smithi148.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 236 kB/s | 59 kB 00:00 2023-10-10T20:17:20.474 INFO:teuthology.orchestra.run.smithi148.stdout:(41/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.5 MB/s | 476 kB 00:00 2023-10-10T20:17:20.529 INFO:teuthology.orchestra.run.smithi148.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.3 MB/s | 185 kB 00:00 2023-10-10T20:17:20.530 INFO:teuthology.orchestra.run.smithi148.stdout:(43/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 649 kB/s | 44 kB 00:00 2023-10-10T20:17:20.578 INFO:teuthology.orchestra.run.smithi148.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 677 kB/s | 33 kB 00:00 2023-10-10T20:17:20.585 INFO:teuthology.orchestra.run.smithi148.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 296 kB/s | 16 kB 00:00 2023-10-10T20:17:20.633 INFO:teuthology.orchestra.run.smithi148.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.1 MB/s | 173 kB 00:00 2023-10-10T20:17:20.685 INFO:teuthology.orchestra.run.smithi148.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.5 MB/s | 81 kB 00:00 2023-10-10T20:17:20.733 INFO:teuthology.orchestra.run.smithi148.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 230 kB/s | 11 kB 00:00 2023-10-10T20:17:20.754 INFO:teuthology.orchestra.run.smithi148.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-10-10T20:17:20.782 INFO:teuthology.orchestra.run.smithi148.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 362 kB/s | 18 kB 00:00 2023-10-10T20:17:20.832 INFO:teuthology.orchestra.run.smithi148.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 975 kB/s | 49 kB 00:00 2023-10-10T20:17:20.882 INFO:teuthology.orchestra.run.smithi148.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-10-10T20:17:20.905 INFO:teuthology.orchestra.run.smithi148.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 7.7 MB/s | 1.2 MB 00:00 2023-10-10T20:17:20.932 INFO:teuthology.orchestra.run.smithi148.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-10T20:17:20.960 INFO:teuthology.orchestra.run.smithi148.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 5.1 MB/s | 283 kB 00:00 2023-10-10T20:17:20.980 INFO:teuthology.orchestra.run.smithi148.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 322 kB/s | 16 kB 00:00 2023-10-10T20:17:21.009 INFO:teuthology.orchestra.run.smithi148.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 691 kB/s | 34 kB 00:00 2023-10-10T20:17:21.037 INFO:teuthology.orchestra.run.smithi148.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 3.4 MB/s | 196 kB 00:00 2023-10-10T20:17:21.058 INFO:teuthology.orchestra.run.smithi148.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-10-10T20:17:21.086 INFO:teuthology.orchestra.run.smithi148.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 386 kB/s | 19 kB 00:00 2023-10-10T20:17:21.107 INFO:teuthology.orchestra.run.smithi148.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 494 kB/s | 24 kB 00:00 2023-10-10T20:17:21.134 INFO:teuthology.orchestra.run.smithi148.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 603 kB/s | 29 kB 00:00 2023-10-10T20:17:21.156 INFO:teuthology.orchestra.run.smithi148.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 568 kB/s | 27 kB 00:00 2023-10-10T20:17:21.194 INFO:teuthology.orchestra.run.smithi148.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 4.2 MB/s | 255 kB 00:00 2023-10-10T20:17:21.210 INFO:teuthology.orchestra.run.smithi148.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 4.5 MB/s | 251 kB 00:00 2023-10-10T20:17:21.244 INFO:teuthology.orchestra.run.smithi148.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-10T20:17:21.260 INFO:teuthology.orchestra.run.smithi148.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-10-10T20:17:21.293 INFO:teuthology.orchestra.run.smithi148.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 466 kB/s | 23 kB 00:00 2023-10-10T20:17:21.297 INFO:teuthology.orchestra.run.smithi148.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-10T20:17:21.354 INFO:teuthology.orchestra.run.smithi148.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-10-10T20:17:21.377 INFO:teuthology.orchestra.run.smithi161.stdout:(120/120): ceph-test-17.2.6-1374.g46d1f59b.el8. 6.6 MB/s | 52 MB 00:07 2023-10-10T20:17:21.381 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-10-10T20:17:21.382 INFO:teuthology.orchestra.run.smithi161.stdout:Total 21 MB/s | 202 MB 00:09 2023-10-10T20:17:21.382 INFO:teuthology.orchestra.run.smithi148.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.5 MB/s | 44 kB 00:00 2023-10-10T20:17:21.397 INFO:teuthology.orchestra.run.smithi148.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-10T20:17:21.412 INFO:teuthology.orchestra.run.smithi148.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-10T20:17:21.429 INFO:teuthology.orchestra.run.smithi148.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-10T20:17:21.433 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-10-10T20:17:21.450 INFO:teuthology.orchestra.run.smithi148.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 22 MB/s | 457 kB 00:00 2023-10-10T20:17:21.548 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-10-10T20:17:21.548 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-10-10T20:17:21.576 INFO:teuthology.orchestra.run.smithi148.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 52 MB/s | 15 MB 00:00 2023-10-10T20:17:21.593 INFO:teuthology.orchestra.run.smithi148.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.3 MB/s | 39 kB 00:00 2023-10-10T20:17:21.624 INFO:teuthology.orchestra.run.smithi148.stdout:(78/120): python3-babel-2.5.1-5.el8.noarch.rpm 27 MB/s | 4.8 MB 00:00 2023-10-10T20:17:21.629 INFO:teuthology.orchestra.run.smithi148.stdout:(79/120): bison-3.0.4-10.el8.x86_64.rpm 19 MB/s | 688 kB 00:00 2023-10-10T20:17:21.642 INFO:teuthology.orchestra.run.smithi148.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 14 MB/s | 259 kB 00:00 2023-10-10T20:17:21.651 INFO:teuthology.orchestra.run.smithi148.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 9.8 MB/s | 225 kB 00:00 2023-10-10T20:17:21.658 INFO:teuthology.orchestra.run.smithi148.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.8 MB/s | 126 kB 00:00 2023-10-10T20:17:21.682 INFO:teuthology.orchestra.run.smithi148.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 7.8 MB/s | 187 kB 00:00 2023-10-10T20:17:21.704 INFO:teuthology.orchestra.run.smithi148.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 24 MB/s | 538 kB 00:00 2023-10-10T20:17:21.714 INFO:teuthology.orchestra.run.smithi148.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 60 MB/s | 3.7 MB 00:00 2023-10-10T20:17:21.719 INFO:teuthology.orchestra.run.smithi148.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-10-10T20:17:21.731 INFO:teuthology.orchestra.run.smithi148.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.1 MB/s | 157 kB 00:00 2023-10-10T20:17:21.736 INFO:teuthology.orchestra.run.smithi148.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 6.4 MB/s | 110 kB 00:00 2023-10-10T20:17:21.750 INFO:teuthology.orchestra.run.smithi148.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-10-10T20:17:21.753 INFO:teuthology.orchestra.run.smithi148.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-10-10T20:17:21.773 INFO:teuthology.orchestra.run.smithi148.stdout:(91/120): libtool-2.4.6-25.el8.x86_64.rpm 30 MB/s | 709 kB 00:00 2023-10-10T20:17:21.774 INFO:teuthology.orchestra.run.smithi148.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.6 MB/s | 54 kB 00:00 2023-10-10T20:17:21.852 INFO:teuthology.orchestra.run.smithi148.stdout:(93/120): openblas-0.3.12-1.el8.x86_64.rpm 59 MB/s | 4.6 MB 00:00 2023-10-10T20:17:21.866 INFO:teuthology.orchestra.run.smithi148.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 51 MB/s | 4.7 MB 00:00 2023-10-10T20:17:21.874 INFO:teuthology.orchestra.run.smithi148.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 33 MB/s | 713 kB 00:00 2023-10-10T20:17:21.879 INFO:teuthology.orchestra.run.smithi148.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.3 MB/s | 19 kB 00:00 2023-10-10T20:17:21.891 INFO:teuthology.orchestra.run.smithi148.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.4 MB/s | 43 kB 00:00 2023-10-10T20:17:21.894 INFO:teuthology.orchestra.run.smithi148.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-10-10T20:17:21.908 INFO:teuthology.orchestra.run.smithi148.stdout:(99/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.4 MB/s | 48 kB 00:00 2023-10-10T20:17:21.910 INFO:teuthology.orchestra.run.smithi148.stdout:(100/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-10T20:17:21.925 INFO:teuthology.orchestra.run.smithi148.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-10T20:17:21.927 INFO:teuthology.orchestra.run.smithi148.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 2.5 MB/s | 43 kB 00:00 2023-10-10T20:17:21.942 INFO:teuthology.orchestra.run.smithi148.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.1 MB/s | 18 kB 00:00 2023-10-10T20:17:21.964 INFO:teuthology.orchestra.run.smithi148.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-10-10T20:17:21.970 INFO:teuthology.orchestra.run.smithi148.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 51 MB/s | 2.3 MB 00:00 2023-10-10T20:17:21.977 INFO:teuthology.orchestra.run.smithi148.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 6.1 MB/s | 83 kB 00:00 2023-10-10T20:17:21.988 INFO:teuthology.orchestra.run.smithi148.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 3.3 MB/s | 59 kB 00:00 2023-10-10T20:17:21.991 INFO:teuthology.orchestra.run.smithi148.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-10-10T20:17:22.004 INFO:teuthology.orchestra.run.smithi148.stdout:(109/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.7 MB/s | 86 kB 00:00 2023-10-10T20:17:22.005 INFO:teuthology.orchestra.run.smithi148.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.3 MB/s | 38 kB 00:00 2023-10-10T20:17:22.019 INFO:teuthology.orchestra.run.smithi148.stdout:(111/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.4 MB/s | 47 kB 00:00 2023-10-10T20:17:22.020 INFO:teuthology.orchestra.run.smithi148.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.4 MB/s | 58 kB 00:00 2023-10-10T20:17:22.043 INFO:teuthology.orchestra.run.smithi148.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 23 MB/s | 559 kB 00:00 2023-10-10T20:17:22.045 INFO:teuthology.orchestra.run.smithi148.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 25 MB/s | 639 kB 00:00 2023-10-10T20:17:22.059 INFO:teuthology.orchestra.run.smithi148.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.3 MB/s | 70 kB 00:00 2023-10-10T20:17:22.064 INFO:teuthology.orchestra.run.smithi148.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 11 MB/s | 200 kB 00:00 2023-10-10T20:17:22.075 INFO:teuthology.orchestra.run.smithi148.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 5.8 MB/s | 97 kB 00:00 2023-10-10T20:17:22.409 INFO:teuthology.orchestra.run.smithi148.stdout:(118/120): librbd1-17.2.6-1374.g46d1f59b.el8.x8 11 MB/s | 3.6 MB 00:00 2023-10-10T20:17:22.532 INFO:teuthology.orchestra.run.smithi148.stdout:(119/120): librados2-17.2.6-1374.g46d1f59b.el8. 7.4 MB/s | 3.5 MB 00:00 2023-10-10T20:17:23.220 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-10-10T20:17:23.225 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-10-10T20:17:27.169 INFO:teuthology.orchestra.run.smithi148.stdout:(120/120): ceph-test-17.2.6-1374.g46d1f59b.el8. 5.0 MB/s | 52 MB 00:10 2023-10-10T20:17:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2023-10-10T20:17:27.176 INFO:teuthology.orchestra.run.smithi148.stdout:Total 13 MB/s | 202 MB 00:14 2023-10-10T20:17:27.230 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2023-10-10T20:17:27.347 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2023-10-10T20:17:27.347 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2023-10-10T20:17:28.193 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-10-10T20:17:28.420 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-10T20:17:28.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-10T20:17:28.833 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-10T20:17:28.992 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-10T20:17:29.026 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2023-10-10T20:17:29.030 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2023-10-10T20:17:29.182 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-10T20:17:29.557 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-10T20:17:29.782 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:17:30.186 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:17:30.699 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:17:30.822 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:17:31.032 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:17:31.116 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:17:31.469 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:17:32.136 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-10T20:17:32.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:17:32.434 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:17:32.644 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-10T20:17:32.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-10T20:17:33.074 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:17:33.180 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:17:33.357 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:17:33.424 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:17:33.445 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2023-10-10T20:17:33.575 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-10T20:17:33.640 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-10T20:17:33.740 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-10T20:17:33.827 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-10T20:17:33.862 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-10T20:17:34.027 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-10T20:17:34.040 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-10T20:17:34.170 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-10T20:17:34.419 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-10T20:17:34.576 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-10T20:17:34.715 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-10T20:17:34.800 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-10T20:17:34.985 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:17:35.236 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-10T20:17:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 5/122 2023-10-10T20:17:35.415 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-10T20:17:35.544 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:17:35.674 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:17:35.869 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:17:35.969 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:17:36.176 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-10T20:17:36.324 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:17:36.451 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-10T20:17:36.638 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-10T20:17:36.835 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-10T20:17:37.014 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-10T20:17:37.077 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-10T20:17:37.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:17:37.301 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:17:37.365 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-10T20:17:37.531 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-10T20:17:37.625 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-10T20:17:37.804 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 26/122 2023-10-10T20:17:37.925 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 27/122 2023-10-10T20:17:38.005 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-10T20:17:38.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 28/122 2023-10-10T20:17:38.577 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:17:38.698 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:17:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:17:38.897 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 14/122 2023-10-10T20:17:39.040 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-10T20:17:39.185 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-10T20:17:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-10T20:17:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-10T20:17:39.525 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-10T20:17:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-10T20:17:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-10T20:17:40.045 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-10T20:17:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-10T20:17:40.310 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-10T20:17:40.347 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-10T20:17:40.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-10T20:17:40.848 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-10T20:17:41.035 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-10T20:17:41.770 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-10T20:17:42.030 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-10T20:17:42.233 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-10T20:17:42.389 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-10T20:17:42.625 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-10T20:17:42.694 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-10T20:17:42.920 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-10T20:17:43.017 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-10T20:17:43.171 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-10T20:17:43.235 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-10T20:17:43.316 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 26/122 2023-10-10T20:17:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-10T20:17:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 27/122 2023-10-10T20:17:43.640 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 38/122 2023-10-10T20:17:43.682 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 28/122 2023-10-10T20:17:43.794 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 39/122 2023-10-10T20:17:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 40/122 2023-10-10T20:17:44.200 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-10T20:17:44.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-10T20:17:44.531 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-10T20:17:44.674 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-10T20:17:44.740 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-10T20:17:44.842 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-10T20:17:45.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-10T20:17:45.048 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-10T20:17:45.191 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-10T20:17:45.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-10T20:17:45.470 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-10T20:17:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-10T20:17:45.718 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-10T20:17:45.859 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-10T20:17:45.960 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-10T20:17:46.084 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-10T20:17:46.314 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-10T20:17:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-10T20:17:47.146 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-10T20:17:47.377 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-10T20:17:47.559 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-10T20:17:47.729 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-10T20:17:47.808 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-10T20:17:48.005 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-10T20:17:48.169 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-10T20:17:48.180 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-10T20:17:48.355 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-10T20:17:48.470 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-10T20:17:48.527 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-10T20:17:48.706 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-10T20:17:48.786 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-10T20:17:48.894 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-10T20:17:49.021 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-10T20:17:49.127 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 38/122 2023-10-10T20:17:49.228 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-10T20:17:49.305 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 39/122 2023-10-10T20:17:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-10T20:17:49.496 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 40/122 2023-10-10T20:17:49.565 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-10T20:17:49.665 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-10T20:17:49.746 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-10T20:17:49.812 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-10T20:17:49.960 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-10T20:17:49.974 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-10T20:17:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-10T20:17:50.214 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-10T20:17:50.312 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-10T20:17:50.440 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-10T20:17:50.491 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-10T20:17:50.603 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-10T20:17:50.661 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-10T20:17:50.874 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-10T20:17:50.956 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-10T20:17:51.124 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-10T20:17:51.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-10T20:17:51.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-10T20:17:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-10T20:17:51.625 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-10T20:17:51.843 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-10T20:17:51.861 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-10T20:17:52.065 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-10T20:17:52.128 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-10T20:17:52.240 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-10T20:17:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-10T20:17:52.421 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-10T20:17:52.605 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-10T20:17:52.749 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-10T20:17:52.815 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-10T20:17:53.016 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-10T20:17:53.184 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-10T20:17:53.325 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-10T20:17:53.504 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-10T20:17:53.548 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-10T20:17:53.729 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-10T20:17:53.867 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-10T20:17:53.966 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-10T20:17:54.063 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-10T20:17:54.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-10T20:17:54.312 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-10T20:17:54.419 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-10T20:17:54.461 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-10T20:17:54.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-10T20:17:54.678 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-10T20:17:54.840 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-10T20:17:55.009 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-10T20:17:55.089 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-10T20:17:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-10T20:17:55.286 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-10T20:17:55.413 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-10T20:17:55.431 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-10T20:17:55.636 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-10T20:17:55.646 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-10T20:17:55.854 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-10T20:17:56.090 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-10T20:17:56.373 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-10T20:17:56.616 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-10T20:17:56.816 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-10T20:17:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-10T20:17:58.260 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-10T20:17:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-10T20:17:58.750 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-10T20:17:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-10T20:17:59.394 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-10T20:17:59.729 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-10T20:17:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-10T20:18:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-10T20:18:00.358 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-10T20:18:00.784 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:01.003 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:01.599 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:01.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:18:01.977 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:18:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:18:05.427 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:05.673 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:06.260 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 87/122 2023-10-10T20:18:06.519 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:18:06.606 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 88/122 2023-10-10T20:18:06.791 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:18:20.188 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:18:20.189 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2023-10-10T20:18:20.189 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2023-10-10T20:18:20.189 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-10-10T20:18:20.189 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2023-10-10T20:18:20.190 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2023-10-10T20:18:20.190 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2023-10-10T20:18:20.190 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2023-10-10T20:18:20.190 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2023-10-10T20:18:20.190 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-10-10T20:18:20.191 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:20.600 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:18:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:18:22.239 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:18:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:18:22.600 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:18:22.991 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:18:23.149 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:18:23.166 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:18:24.721 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 89/122 2023-10-10T20:18:24.722 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2023-10-10T20:18:24.722 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2023-10-10T20:18:24.722 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2023-10-10T20:18:24.722 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /run 2023-10-10T20:18:24.722 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2023-10-10T20:18:24.723 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2023-10-10T20:18:24.723 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2023-10-10T20:18:24.723 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2023-10-10T20:18:24.723 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2023-10-10T20:18:24.723 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:25.087 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:18:26.047 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:18:26.225 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:18:26.430 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 95/122 2023-10-10T20:18:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 90/122 2023-10-10T20:18:26.628 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 96/122 2023-10-10T20:18:26.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:18:26.823 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:18:27.012 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 91/122 2023-10-10T20:18:27.038 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:18:27.161 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:18:27.246 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:18:27.557 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 92/122 2023-10-10T20:18:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:18:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 93/122 2023-10-10T20:18:28.219 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:18:28.448 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:18:28.546 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:18:28.935 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 100/122 2023-10-10T20:18:29.130 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:18:29.167 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:18:29.356 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:18:29.367 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:18:30.634 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:18:30.768 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 94/122 2023-10-10T20:18:30.959 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 95/122 2023-10-10T20:18:30.970 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 103/122 2023-10-10T20:18:31.148 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 96/122 2023-10-10T20:18:31.213 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:18:31.301 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:18:31.344 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:18:31.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:18:31.515 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:18:31.561 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 97/122 2023-10-10T20:18:31.689 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:18:31.747 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:18:32.049 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:18:32.240 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:18:32.727 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 98/122 2023-10-10T20:18:32.901 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:18:33.000 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 99/122 2023-10-10T20:18:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 100/122 2023-10-10T20:18:33.522 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:18:33.560 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 101/122 2023-10-10T20:18:33.693 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:18:33.704 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 102/122 2023-10-10T20:18:35.278 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 103/122 2023-10-10T20:18:35.505 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:18:35.600 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 104/122 2023-10-10T20:18:35.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:18:35.803 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 105/122 2023-10-10T20:18:35.972 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:18:36.332 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 106/122 2023-10-10T20:18:36.501 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:18:37.183 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:18:37.498 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 108/122 2023-10-10T20:18:37.721 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-10T20:18:38.441 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-10T20:18:38.849 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-10T20:18:39.030 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 111/122 2023-10-10T20:18:39.217 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-10T20:18:39.360 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 113/122 2023-10-10T20:18:39.518 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-10T20:18:39.655 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 115/122 2023-10-10T20:18:39.846 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 116/122 2023-10-10T20:18:39.989 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 117/122 2023-10-10T20:18:40.244 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-10T20:18:40.454 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-10T20:18:40.506 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-10T20:18:40.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-10T20:18:40.672 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-10T20:18:40.672 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:18:40.874 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:18:40.874 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:41.446 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 107/122 2023-10-10T20:18:41.746 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 108/122 2023-10-10T20:18:41.788 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:41.788 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 1/122 2023-10-10T20:18:41.788 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2/122 2023-10-10T20:18:41.788 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 3/122 2023-10-10T20:18:41.789 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:18:41.789 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 5/122 2023-10-10T20:18:41.789 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:18:41.789 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:18:41.789 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:18:41.790 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 9/122 2023-10-10T20:18:41.790 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:18:41.790 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 11/122 2023-10-10T20:18:41.790 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 12/122 2023-10-10T20:18:41.790 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:18:41.791 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 14/122 2023-10-10T20:18:41.791 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 15/122 2023-10-10T20:18:41.791 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 16/122 2023-10-10T20:18:41.791 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 17/122 2023-10-10T20:18:41.791 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 18/122 2023-10-10T20:18:41.792 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 19/122 2023-10-10T20:18:41.792 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 20/122 2023-10-10T20:18:41.792 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 21/122 2023-10-10T20:18:41.793 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 22/122 2023-10-10T20:18:41.793 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 23/122 2023-10-10T20:18:41.793 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 24/122 2023-10-10T20:18:41.793 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 25/122 2023-10-10T20:18:41.793 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 26/122 2023-10-10T20:18:41.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 27/122 2023-10-10T20:18:41.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 28/122 2023-10-10T20:18:41.803 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 29/122 2023-10-10T20:18:41.803 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 30/122 2023-10-10T20:18:41.803 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 31/122 2023-10-10T20:18:41.804 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 32/122 2023-10-10T20:18:41.804 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 33/122 2023-10-10T20:18:41.804 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 34/122 2023-10-10T20:18:41.804 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 35/122 2023-10-10T20:18:41.804 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 36/122 2023-10-10T20:18:41.805 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 37/122 2023-10-10T20:18:41.805 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 38/122 2023-10-10T20:18:41.805 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-10T20:18:41.805 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-10T20:18:41.805 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-10T20:18:41.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-10T20:18:41.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-10T20:18:41.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-10T20:18:41.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-10T20:18:41.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-10T20:18:41.807 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-10T20:18:41.807 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-10T20:18:41.807 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-10T20:18:41.807 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-10T20:18:41.807 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-10T20:18:41.808 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-10T20:18:41.808 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-10T20:18:41.808 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-10T20:18:41.808 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-10T20:18:41.808 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-10T20:18:41.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-10T20:18:41.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-10T20:18:41.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-10T20:18:41.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-10T20:18:41.809 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-10T20:18:41.810 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-10T20:18:41.810 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-10T20:18:41.810 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-10T20:18:41.810 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-10T20:18:41.811 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-10T20:18:41.812 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-10T20:18:41.812 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-10T20:18:41.812 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-10T20:18:41.812 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-10T20:18:41.814 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-10T20:18:41.814 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-10T20:18:41.814 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-10T20:18:41.814 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-10T20:18:41.814 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-10T20:18:41.815 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-10T20:18:41.815 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-10T20:18:41.815 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-10T20:18:41.815 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-10T20:18:41.815 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-10T20:18:41.816 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-10T20:18:41.816 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-10T20:18:41.816 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-10T20:18:41.816 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-10T20:18:41.816 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-10T20:18:41.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-10T20:18:41.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-10T20:18:41.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-10T20:18:41.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-10T20:18:41.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-10T20:18:41.818 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-10T20:18:41.818 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-10T20:18:41.818 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-10T20:18:41.818 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-10T20:18:41.818 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-10T20:18:41.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-10T20:18:41.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-10T20:18:41.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-10T20:18:41.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-10T20:18:41.819 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-10T20:18:41.820 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-10T20:18:41.821 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-10T20:18:41.822 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-10T20:18:41.822 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 119/122 2023-10-10T20:18:41.822 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-10T20:18:41.822 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 121/122 2023-10-10T20:18:41.892 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-10T20:18:42.606 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-10T20:18:42.928 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-10T20:18:43.155 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 111/122 2023-10-10T20:18:43.328 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-10T20:18:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 113/122 2023-10-10T20:18:43.613 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-10T20:18:43.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 115/122 2023-10-10T20:18:43.941 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 116/122 2023-10-10T20:18:44.055 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:44.055 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Oct 2023 08:17:02 PM UTC. 2023-10-10T20:18:44.134 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 117/122 2023-10-10T20:18:44.344 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-10T20:18:44.525 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-10T20:18:44.583 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-10T20:18:44.742 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-10T20:18:44.751 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-10T20:18:44.751 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:18:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-10T20:18:44.936 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:45.760 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 1/122 2023-10-10T20:18:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2/122 2023-10-10T20:18:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 3/122 2023-10-10T20:18:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 4/122 2023-10-10T20:18:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1374.g46d1f 5/122 2023-10-10T20:18:45.761 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 6/122 2023-10-10T20:18:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 7/122 2023-10-10T20:18:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 8/122 2023-10-10T20:18:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 9/122 2023-10-10T20:18:45.762 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 10/122 2023-10-10T20:18:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 11/122 2023-10-10T20:18:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 12/122 2023-10-10T20:18:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 13/122 2023-10-10T20:18:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_ 14/122 2023-10-10T20:18:45.763 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 15/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 16/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_6 17/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86 18/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 19/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el 20/122 2023-10-10T20:18:45.764 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8. 21/122 2023-10-10T20:18:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_6 22/122 2023-10-10T20:18:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 23/122 2023-10-10T20:18:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 24/122 2023-10-10T20:18:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 25/122 2023-10-10T20:18:45.765 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 26/122 2023-10-10T20:18:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 27/122 2023-10-10T20:18:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 28/122 2023-10-10T20:18:45.766 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b. 29/122 2023-10-10T20:18:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noa 30/122 2023-10-10T20:18:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.n 31/122 2023-10-10T20:18:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d 32/122 2023-10-10T20:18:45.767 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el 33/122 2023-10-10T20:18:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 34/122 2023-10-10T20:18:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.e 35/122 2023-10-10T20:18:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 36/122 2023-10-10T20:18:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 37/122 2023-10-10T20:18:45.768 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 38/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-10T20:18:45.769 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-10T20:18:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-10T20:18:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-10T20:18:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-10T20:18:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-10T20:18:45.770 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-10T20:18:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-10T20:18:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-10T20:18:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-10T20:18:45.771 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-10T20:18:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-10T20:18:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-10T20:18:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-10T20:18:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-10T20:18:45.772 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-10T20:18:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-10T20:18:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-10T20:18:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-10T20:18:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-10T20:18:45.773 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-10T20:18:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-10T20:18:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-10T20:18:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-10T20:18:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-10T20:18:45.774 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-10T20:18:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-10T20:18:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-10T20:18:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-10T20:18:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-10T20:18:45.775 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-10T20:18:45.776 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-10T20:18:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-10T20:18:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-10T20:18:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-10T20:18:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-10T20:18:45.777 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-10T20:18:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-10T20:18:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-10T20:18:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-10T20:18:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-10T20:18:45.778 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-10T20:18:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-10T20:18:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-10T20:18:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-10T20:18:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-10T20:18:45.779 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-10T20:18:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-10T20:18:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-10T20:18:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-10T20:18:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-10T20:18:45.780 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-10T20:18:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-10T20:18:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-10T20:18:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-10T20:18:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-10T20:18:45.781 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-10T20:18:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-10T20:18:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-10T20:18:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-10T20:18:45.782 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-10T20:18:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-10T20:18:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-10T20:18:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-10T20:18:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-10T20:18:45.783 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-10T20:18:45.784 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 119/122 2023-10-10T20:18:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-10T20:18:45.785 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 121/122 2023-10-10T20:18:46.017 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-10-10T20:18:46.159 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:46.160 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2023-10-10T20:18:46.160 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.160 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.160 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-10-10T20:18:46.161 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-7.el8.noarch 2023-10-10T20:18:46.161 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2023-10-10T20:18:46.161 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.161 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.161 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.162 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.163 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.164 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.165 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.165 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.165 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.165 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:46.165 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.5-12.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-10T20:18:46.166 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-10T20:18:46.167 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-10T20:18:46.167 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.167 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.167 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.167 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-10T20:18:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-10T20:18:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-10T20:18:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-10T20:18:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-10T20:18:46.168 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-10T20:18:46.169 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-10T20:18:46.170 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-10T20:18:46.170 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-10T20:18:46.170 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-10T20:18:46.170 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-10T20:18:46.170 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-10T20:18:46.171 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-10T20:18:46.171 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-10T20:18:46.171 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-10T20:18:46.171 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-10T20:18:46.171 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-10T20:18:46.172 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-10T20:18:46.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-10T20:18:46.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-10T20:18:46.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-10T20:18:46.172 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-10T20:18:46.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-10T20:18:46.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.173 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.174 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-10T20:18:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-10T20:18:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-10T20:18:46.175 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-10T20:18:46.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-10T20:18:46.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-10T20:18:46.177 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-10T20:18:46.177 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-10T20:18:46.177 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-10T20:18:46.178 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-10T20:18:46.178 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-10T20:18:46.178 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-10T20:18:46.178 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-10T20:18:46.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-10T20:18:46.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-10T20:18:46.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-10T20:18:46.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-10T20:18:46.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-10T20:18:46.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-10T20:18:46.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-10T20:18:46.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-10T20:18:46.181 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-10T20:18:46.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-10T20:18:46.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-10T20:18:46.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-10T20:18:46.182 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-10T20:18:46.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-10T20:18:46.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-10T20:18:46.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-10T20:18:46.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-10T20:18:46.183 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-10T20:18:46.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-10T20:18:46.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-10T20:18:46.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-10T20:18:46.184 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-10T20:18:46.184 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-10T20:18:46.185 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-10T20:18:46.186 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-10T20:18:46.186 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:46.187 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-10-10T20:18:46.459 DEBUG:teuthology.parallel:result is None 2023-10-10T20:18:48.001 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-10T20:18:48.001 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Oct 2023 08:17:03 PM UTC. 2023-10-10T20:18:50.280 INFO:teuthology.orchestra.run.smithi148.stdout:Installed products updated. 2023-10-10T20:18:50.438 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:50.438 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2023-10-10T20:18:50.438 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.439 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.439 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2023-10-10T20:18:50.439 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.1-7.el8.noarch 2023-10-10T20:18:50.439 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.0.4-10.el8.x86_64 2023-10-10T20:18:50.439 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.440 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.441 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.441 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.441 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.441 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.441 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.442 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.442 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.442 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.442 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:17.2.6-1374.g46d1f59b.el8.noarch 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.1-9.el8.x86_64 2023-10-10T20:18:50.444 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-10T20:18:50.445 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-10T20:18:50.445 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.5-12.el8.x86_64 2023-10-10T20:18:50.445 INFO:teuthology.orchestra.run.smithi148.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-10T20:18:50.445 INFO:teuthology.orchestra.run.smithi148.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-10T20:18:50.445 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-10T20:18:50.446 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.446 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.446 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.446 INFO:teuthology.orchestra.run.smithi148.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-10T20:18:50.446 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-10T20:18:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-10T20:18:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-10T20:18:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-10T20:18:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.447 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.448 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-10T20:18:50.448 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.448 INFO:teuthology.orchestra.run.smithi148.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-10T20:18:50.448 INFO:teuthology.orchestra.run.smithi148.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-10T20:18:50.448 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-10T20:18:50.449 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-10T20:18:50.449 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-10T20:18:50.449 INFO:teuthology.orchestra.run.smithi148.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-10T20:18:50.449 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-10T20:18:50.449 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-10T20:18:50.450 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-10T20:18:50.450 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-10T20:18:50.450 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-10T20:18:50.450 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-10T20:18:50.450 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-10T20:18:50.451 INFO:teuthology.orchestra.run.smithi148.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-10T20:18:50.451 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-10T20:18:50.451 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-10T20:18:50.451 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-10T20:18:50.451 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-10T20:18:50.452 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-10T20:18:50.452 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.452 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.452 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.452 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-10T20:18:50.453 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-10T20:18:50.453 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-10T20:18:50.453 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-10T20:18:50.453 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-10T20:18:50.454 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-10T20:18:50.454 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-10T20:18:50.454 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-10T20:18:50.454 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-10T20:18:50.454 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-10T20:18:50.455 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-10T20:18:50.455 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-10T20:18:50.455 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-10T20:18:50.455 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-10T20:18:50.455 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-10T20:18:50.456 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-10T20:18:50.456 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-10T20:18:50.456 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-10T20:18:50.456 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-10T20:18:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-10T20:18:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-2.6-1.el8.noarch 2023-10-10T20:18:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-10T20:18:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-10T20:18:50.457 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-10T20:18:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-10T20:18:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-10T20:18:50.458 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-10T20:18:50.459 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.459 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-10T20:18:50.459 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-10T20:18:50.459 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-10T20:18:50.460 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-10T20:18:50.460 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-10T20:18:50.460 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-10T20:18:50.460 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-10T20:18:50.460 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-10T20:18:50.461 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-10T20:18:50.461 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-10T20:18:50.461 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-10T20:18:50.461 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-10T20:18:50.461 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-10T20:18:50.462 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.462 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.462 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:17.2.6-1374.g46d1f59b.el8.x86_64 2023-10-10T20:18:50.462 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-10T20:18:50.462 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-10T20:18:50.463 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-10T20:18:50.463 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-10T20:18:50.463 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-10T20:18:50.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:50.464 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2023-10-10T20:18:50.697 DEBUG:teuthology.parallel:result is None 2023-10-10T20:18:50.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:18:50.837 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-10T20:18:50.858 INFO:teuthology.orchestra.run.smithi148.stdout:17.2.6-1374.g46d1f59b.el8 2023-10-10T20:18:50.858 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1374.g46d1f59b.el8 2023-10-10T20:18:50.859 INFO:teuthology.task.install:The correct ceph version 17.2.6-1374.g46d1f59b is installed. 2023-10-10T20:18:50.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 2023-10-10T20:18:50.997 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-10T20:18:51.021 INFO:teuthology.orchestra.run.smithi161.stdout:17.2.6-1374.g46d1f59b.el8 2023-10-10T20:18:51.021 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1374.g46d1f59b.el8 2023-10-10T20:18:51.022 INFO:teuthology.task.install:The correct ceph version 17.2.6-1374.g46d1f59b is installed. 2023-10-10T20:18:51.022 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-10T20:18:51.023 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:51.023 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-10T20:18:51.062 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:51.062 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-10T20:18:51.104 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-10T20:18:51.104 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:51.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2023-10-10T20:18:51.129 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-10T20:18:51.193 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:51.194 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-10-10T20:18:51.220 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-10T20:18:51.286 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-10T20:18:51.287 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:51.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-10T20:18:51.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-10T20:18:51.376 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:51.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-10T20:18:51.402 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-10T20:18:51.467 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-10T20:18:51.467 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:51.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2023-10-10T20:18:51.492 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-10T20:18:51.556 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:51.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2023-10-10T20:18:51.581 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-10T20:18:51.646 INFO:teuthology.run_tasks:Running task ceph... 2023-10-10T20:18:51.718 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-10T20:18:51.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2023-10-10T20:18:51.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2023-10-10T20:18:51.745 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-10T20:18:51.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-10T20:18:51.785 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-10T20:18:51.809 INFO:tasks.ceph:Creating extra log directories... 2023-10-10T20:18:51.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-10T20:18:51.849 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-10T20:18:51.880 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-10T20:18:51.880 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-10T20:18:51.880 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-10T20:18:51.881 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:51.881 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-10T20:18:51.925 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-10T20:18:52.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-10T20:18:52.062 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:52.063 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-10T20:18:52.077 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-10T20:18:52.188 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-10T20:18:52.218 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-10T20:18:52.219 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-10T20:18:52.219 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465', 'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7420465', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2048, 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-10-0720-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_46d1f59b549730ba7f93d940a3d32b94c45a4cf7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGi6gJsTVaMUlBg6Me2PzbT0mPxCeJVMjlPY47SvIQiGWjBu3snTBdzepBBQn2t82Ii4rS/ZdXuBT+0mjcs1yd4=', 'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYkYz3NSyUPaG3igeoA55L9b/m1sneuW3V9rXMKdl1KO1GrLsuBTmFe2n4Be3W4R8N7xmc2kremECT0tzLawKs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '46d1f59b549730ba7f93d940a3d32b94c45a4cf7', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-10_18:16:52', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-10T20:18:52.220 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-10T20:18:52.222 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-10T20:18:52.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2023-10-10T20:18:52.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2023-10-10T20:18:52.331 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:52.331 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2023-10-10T20:18:52.354 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-10T20:18:52.355 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2023-10-10T20:18:52.409 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-10T20:18:52.409 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:52.410 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 217056 Links: 1 2023-10-10T20:18:52.410 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:52.410 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:52.410 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-10-10 20:18:45.357100922 +0000 2023-10-10T20:18:52.411 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-10-10 20:16:16.269559898 +0000 2023-10-10T20:18:52.411 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-10-10 20:16:16.269559898 +0000 2023-10-10T20:18:52.411 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-10-10T20:18:52.411 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-10T20:18:52.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-10-10T20:18:52.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-10-10T20:18:52.474 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000257177 s, 2.0 MB/s 2023-10-10T20:18:52.474 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-10T20:18:52.530 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2023-10-10T20:18:52.585 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-10T20:18:52.586 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:52.586 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 218796 Links: 1 2023-10-10T20:18:52.586 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-10-10 20:18:45.357100922 +0000 2023-10-10T20:18:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-10-10 20:16:16.768547850 +0000 2023-10-10T20:18:52.587 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-10-10 20:16:16.768547850 +0000 2023-10-10T20:18:52.587 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-10-10T20:18:52.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-10T20:18:52.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-10-10T20:18:52.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-10-10T20:18:52.649 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000249847 s, 2.0 MB/s 2023-10-10T20:18:52.650 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-10T20:18:52.705 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2023-10-10T20:18:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-10T20:18:52.760 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 222360 Links: 1 2023-10-10T20:18:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-10-10 20:18:45.357100922 +0000 2023-10-10T20:18:52.761 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-10-10 20:16:17.269535754 +0000 2023-10-10T20:18:52.762 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-10-10 20:16:17.269535754 +0000 2023-10-10T20:18:52.762 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-10-10T20:18:52.762 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-10T20:18:52.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-10-10T20:18:52.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-10-10T20:18:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000228179 s, 2.2 MB/s 2023-10-10T20:18:52.825 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-10T20:18:52.880 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2023-10-10T20:18:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-10T20:18:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:52.936 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 220635 Links: 1 2023-10-10T20:18:52.936 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:52.936 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:52.937 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2023-10-10 20:18:45.357100922 +0000 2023-10-10T20:18:52.937 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2023-10-10 20:16:17.761523875 +0000 2023-10-10T20:18:52.937 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2023-10-10 20:16:17.761523875 +0000 2023-10-10T20:18:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2023-10-10T20:18:52.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-10T20:18:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2023-10-10T20:18:52.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2023-10-10T20:18:52.999 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000191057 s, 2.7 MB/s 2023-10-10T20:18:53.000 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-10T20:18:53.055 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:53.055 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:53.056 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-10-10T20:18:53.072 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-10T20:18:53.072 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-10-10T20:18:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-10T20:18:53.128 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:53.128 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 221285 Links: 1 2023-10-10T20:18:53.128 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:53.129 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:53.129 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-10-10 20:18:41.340989554 +0000 2023-10-10T20:18:53.129 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-10-10 20:16:19.371742417 +0000 2023-10-10T20:18:53.129 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-10-10 20:16:19.371742417 +0000 2023-10-10T20:18:53.129 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-10-10T20:18:53.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-10T20:18:53.192 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-10-10T20:18:53.192 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-10-10T20:18:53.192 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000255981 s, 2.0 MB/s 2023-10-10T20:18:53.193 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-10T20:18:53.250 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-10-10T20:18:53.306 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-10T20:18:53.307 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:53.307 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 218761 Links: 1 2023-10-10T20:18:53.307 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:53.308 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:53.308 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-10-10 20:18:41.340989554 +0000 2023-10-10T20:18:53.308 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-10-10 20:16:19.887731717 +0000 2023-10-10T20:18:53.308 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-10-10 20:16:19.887731717 +0000 2023-10-10T20:18:53.308 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-10-10T20:18:53.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-10T20:18:53.371 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-10-10T20:18:53.371 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-10-10T20:18:53.371 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000222707 s, 2.3 MB/s 2023-10-10T20:18:53.372 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-10T20:18:53.429 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-10-10T20:18:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-10T20:18:53.484 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:53.484 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 219966 Links: 1 2023-10-10T20:18:53.484 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:53.484 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:53.485 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-10-10 20:18:41.340989554 +0000 2023-10-10T20:18:53.485 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-10-10 20:16:20.422720623 +0000 2023-10-10T20:18:53.485 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-10-10 20:16:20.422720623 +0000 2023-10-10T20:18:53.485 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-10-10T20:18:53.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-10T20:18:53.547 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-10-10T20:18:53.547 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-10-10T20:18:53.547 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00022878 s, 2.2 MB/s 2023-10-10T20:18:53.548 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-10T20:18:53.605 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-10-10T20:18:53.660 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-10T20:18:53.661 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-10T20:18:53.661 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 219019 Links: 1 2023-10-10T20:18:53.661 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-10T20:18:53.661 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-10-10T20:18:53.662 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-10-10 20:18:41.340989554 +0000 2023-10-10T20:18:53.662 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-10-10 20:16:20.914710421 +0000 2023-10-10T20:18:53.662 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-10-10 20:16:20.914710421 +0000 2023-10-10T20:18:53.662 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-10-10T20:18:53.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-10T20:18:53.723 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-10-10T20:18:53.724 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-10-10T20:18:53.724 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000197174 s, 2.6 MB/s 2023-10-10T20:18:53.725 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-10T20:18:53.780 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:53.780 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-10T20:18:53.781 INFO:tasks.ceph:Generating config... 2023-10-10T20:18:53.782 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-10T20:18:53.782 INFO:tasks.ceph:[client] debug client = 20 2023-10-10T20:18:53.783 INFO:tasks.ceph:[client] debug ms = 1 2023-10-10T20:18:53.783 INFO:tasks.ceph:[client] pid file = 2023-10-10T20:18:53.783 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-10T20:18:53.783 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-10T20:18:53.784 INFO:tasks.ceph:[client.0] debug client = 20 2023-10-10T20:18:53.784 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-10-10T20:18:53.784 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-10-10T20:18:53.784 INFO:tasks.ceph:[client.1] debug client = 20 2023-10-10T20:18:53.785 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-10-10T20:18:53.785 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-10-10T20:18:53.785 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-10T20:18:53.786 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-10T20:18:53.786 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-10T20:18:53.786 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-10T20:18:53.786 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-10T20:18:53.787 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-10T20:18:53.787 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-10T20:18:53.787 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-10T20:18:53.787 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-10T20:18:53.788 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-10T20:18:53.788 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-10T20:18:53.788 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-10T20:18:53.788 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-10T20:18:53.789 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-10T20:18:53.789 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-10T20:18:53.789 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-10T20:18:53.789 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-10-10T20:18:53.790 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-10-10T20:18:53.790 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-10-10T20:18:53.790 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-10T20:18:53.790 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-10T20:18:53.791 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-10T20:18:53.791 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-10T20:18:53.791 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-10T20:18:53.791 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-10T20:18:53.792 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-10T20:18:53.792 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-10T20:18:53.792 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-10T20:18:53.793 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-10T20:18:53.793 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-10-10T20:18:53.793 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-10T20:18:53.793 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-10T20:18:53.794 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-10T20:18:53.794 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-10-10T20:18:53.794 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-10-10T20:18:53.794 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-10-10T20:18:53.795 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-10T20:18:53.795 INFO:tasks.ceph:Setting up mon.a... 2023-10-10T20:18:53.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-10T20:18:53.852 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.keyring 2023-10-10T20:18:53.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-10T20:18:53.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:18:53.959 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.148'), ('mon.b', '172.21.15.161'), ('mon.c', '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]')] 2023-10-10T20:18:53.960 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.148,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-10T20:18:53.961 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:53.961 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-10T20:18:54.014 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.148 --add b 172.21.15.161 --addv c '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:54.096 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:54.096 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: generated fsid 374035e8-0861-4d26-af5d-2eadca412b24 2023-10-10T20:18:54.096 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = octopus 2023-10-10T20:18:54.097 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 0 2023-10-10T20:18:54.097 INFO:teuthology.orchestra.run.smithi148.stdout:fsid 374035e8-0861-4d26-af5d-2eadca412b24 2023-10-10T20:18:54.097 INFO:teuthology.orchestra.run.smithi148.stdout:last_changed 2023-10-10T20:18:54.095730+0000 2023-10-10T20:18:54.097 INFO:teuthology.orchestra.run.smithi148.stdout:created 2023-10-10T20:18:54.095730+0000 2023-10-10T20:18:54.097 INFO:teuthology.orchestra.run.smithi148.stdout:min_mon_release 15 (octopus) 2023-10-10T20:18:54.098 INFO:teuthology.orchestra.run.smithi148.stdout:election_strategy: 1 2023-10-10T20:18:54.098 INFO:teuthology.orchestra.run.smithi148.stdout:0: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.a 2023-10-10T20:18:54.098 INFO:teuthology.orchestra.run.smithi148.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2023-10-10T20:18:54.098 INFO:teuthology.orchestra.run.smithi148.stdout:2: [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] mon.c 2023-10-10T20:18:54.099 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-10T20:18:54.100 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-10T20:18:54.154 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 374035e8-0861-4d26-af5d-2eadca412b24... 2023-10-10T20:18:54.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-10T20:18:54.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-10T20:18:54.236 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2023-10-10T20:18:54.237 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2023-10-10T20:18:54.237 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-10T20:18:54.237 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2023-10-10T20:18:54.238 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.238 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2023-10-10T20:18:54.238 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.238 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2023-10-10T20:18:54.239 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.239 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2023-10-10T20:18:54.239 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2023-10-10T20:18:54.239 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.239 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2023-10-10T20:18:54.240 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2023-10-10T20:18:54.240 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.240 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-10T20:18:54.240 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2023-10-10T20:18:54.240 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2023-10-10T20:18:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-10T20:18:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2023-10-10T20:18:54.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.242 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2023-10-10T20:18:54.242 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2023-10-10T20:18:54.242 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2023-10-10T20:18:54.242 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2023-10-10T20:18:54.242 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2023-10-10T20:18:54.243 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.244 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-10T20:18:54.244 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.244 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2023-10-10T20:18:54.244 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.244 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2023-10-10T20:18:54.245 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.245 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2023-10-10T20:18:54.245 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2023-10-10T20:18:54.245 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2023-10-10T20:18:54.245 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2023-10-10T20:18:54.246 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2023-10-10T20:18:54.246 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 374035e8-0861-4d26-af5d-2eadca412b24 2023-10-10T20:18:54.246 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.246 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2023-10-10T20:18:54.247 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2023-10-10T20:18:54.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.247 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2023-10-10T20:18:54.247 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2023-10-10T20:18:54.247 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2023-10-10T20:18:54.248 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.248 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2023-10-10T20:18:54.248 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2023-10-10T20:18:54.248 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.248 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2023-10-10T20:18:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2023-10-10T20:18:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2023-10-10T20:18:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2023-10-10T20:18:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2023-10-10T20:18:54.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.250 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2023-10-10T20:18:54.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.250 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2023-10-10T20:18:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2023-10-10T20:18:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2023-10-10T20:18:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2023-10-10T20:18:54.252 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.252 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2023-10-10T20:18:54.252 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2023-10-10T20:18:54.252 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2023-10-10T20:18:54.252 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2023-10-10T20:18:54.253 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-10-10T20:18:54.254 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2023-10-10T20:18:54.254 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2023-10-10T20:18:54.254 INFO:teuthology.orchestra.run.smithi148.stdout: memstore device bytes = 200000000 2023-10-10T20:18:54.254 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2023-10-10T20:18:54.254 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2023-10-10T20:18:54.255 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2023-10-10T20:18:54.255 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 1.0 2023-10-10T20:18:54.255 INFO:teuthology.orchestra.run.smithi148.stdout: osd mon report interval = 5 2023-10-10T20:18:54.255 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = memstore 2023-10-10T20:18:54.255 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2023-10-10T20:18:54.256 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.256 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2023-10-10T20:18:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-10-10T20:18:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2023-10-10T20:18:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2023-10-10T20:18:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2023-10-10T20:18:54.257 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:18:54.258 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:18:54.258 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2023-10-10T20:18:54.258 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.258 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2023-10-10T20:18:54.258 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-10-10T20:18:54.259 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2023-10-10T20:18:54.259 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2023-10-10T20:18:54.259 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2023-10-10T20:18:54.259 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2023-10-10T20:18:54.259 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.260 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-10T20:18:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2023-10-10T20:18:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2023-10-10T20:18:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2023-10-10T20:18:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-10T20:18:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-10T20:18:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2023-10-10T20:18:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2023-10-10T20:18:54.262 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull ratio = 0.6 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.7 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.6 2023-10-10T20:18:54.263 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:18:54.264 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2023-10-10T20:18:54.264 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2023-10-10T20:18:54.264 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2023-10-10T20:18:54.265 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2023-10-10T20:18:54.265 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-10T20:18:54.265 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-10T20:18:54.266 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2023-10-10T20:18:54.266 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2023-10-10T20:18:54.266 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-10-10T20:18:54.266 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2023-10-10T20:18:54.267 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2023-10-10T20:18:54.267 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2023-10-10T20:18:54.267 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2023-10-10T20:18:54.267 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2023-10-10T20:18:54.267 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2023-10-10T20:18:54.268 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2023-10-10T20:18:54.269 INFO:teuthology.orchestra.run.smithi148.stdout: debug objectcacher = 20 2023-10-10T20:18:54.269 INFO:teuthology.orchestra.run.smithi148.stdout: debug objecter = 20 2023-10-10T20:18:54.269 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2023-10-10T20:18:54.269 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2023-10-10T20:18:54.269 INFO:teuthology.orchestra.run.smithi148.stdout: debug objectcacher = 20 2023-10-10T20:18:54.270 INFO:teuthology.orchestra.run.smithi148.stdout: debug objecter = 20 2023-10-10T20:18:54.270 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2023-10-10T20:18:54.270 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2023-10-10T20:18:54.270 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2023-10-10T20:18:54.270 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2023-10-10T20:18:54.271 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2023-10-10T20:18:54.271 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2023-10-10T20:18:54.271 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2023-10-10T20:18:54.271 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2023-10-10T20:18:54.271 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2023-10-10T20:18:54.272 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2023-10-10T20:18:54.273 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2023-10-10T20:18:54.273 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2023-10-10T20:18:54.274 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-10T20:18:54.274 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2023-10-10T20:18:54.274 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.274 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2023-10-10T20:18:54.274 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.275 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2023-10-10T20:18:54.275 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.275 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2023-10-10T20:18:54.275 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2023-10-10T20:18:54.275 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.276 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2023-10-10T20:18:54.276 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2023-10-10T20:18:54.276 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.276 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-10T20:18:54.276 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2023-10-10T20:18:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2023-10-10T20:18:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-10T20:18:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2023-10-10T20:18:54.278 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.278 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2023-10-10T20:18:54.278 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2023-10-10T20:18:54.278 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2023-10-10T20:18:54.278 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-10T20:18:54.279 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2023-10-10T20:18:54.280 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.280 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-10T20:18:54.280 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.280 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2023-10-10T20:18:54.280 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.281 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2023-10-10T20:18:54.281 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.281 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2023-10-10T20:18:54.281 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2023-10-10T20:18:54.281 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2023-10-10T20:18:54.282 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.148,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2023-10-10T20:18:54.282 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on skipped message = False 2023-10-10T20:18:54.282 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 374035e8-0861-4d26-af5d-2eadca412b24 2023-10-10T20:18:54.282 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.282 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2023-10-10T20:18:54.283 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2023-10-10T20:18:54.283 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.283 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2023-10-10T20:18:54.283 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2023-10-10T20:18:54.283 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2023-10-10T20:18:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2023-10-10T20:18:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2023-10-10T20:18:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2023-10-10T20:18:54.285 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.286 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2023-10-10T20:18:54.286 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.286 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2023-10-10T20:18:54.286 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.286 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2023-10-10T20:18:54.287 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.287 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2023-10-10T20:18:54.287 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2023-10-10T20:18:54.287 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.287 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2023-10-10T20:18:54.288 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2023-10-10T20:18:54.288 INFO:teuthology.orchestra.run.smithi161.stdout: bdev async discard = True 2023-10-10T20:18:54.288 INFO:teuthology.orchestra.run.smithi161.stdout: bdev enable discard = True 2023-10-10T20:18:54.288 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore allocator = bitmap 2023-10-10T20:18:54.288 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2023-10-10T20:18:54.289 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2023-10-10T20:18:54.289 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 1/20 2023-10-10T20:18:54.289 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 1/20 2023-10-10T20:18:54.289 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-10-10T20:18:54.289 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2023-10-10T20:18:54.290 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 4/10 2023-10-10T20:18:54.290 INFO:teuthology.orchestra.run.smithi161.stdout: memstore device bytes = 200000000 2023-10-10T20:18:54.290 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2023-10-10T20:18:54.290 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2023-10-10T20:18:54.290 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2023-10-10T20:18:54.291 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 1.0 2023-10-10T20:18:54.291 INFO:teuthology.orchestra.run.smithi161.stdout: osd mon report interval = 5 2023-10-10T20:18:54.291 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = memstore 2023-10-10T20:18:54.291 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2023-10-10T20:18:54.291 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.292 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2023-10-10T20:18:54.292 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-10-10T20:18:54.292 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2023-10-10T20:18:54.292 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-10-10T20:18:54.292 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2023-10-10T20:18:54.293 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-10-10T20:18:54.294 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-10-10T20:18:54.294 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2023-10-10T20:18:54.294 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-10-10T20:18:54.294 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2023-10-10T20:18:54.294 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2023-10-10T20:18:54.295 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-10-10T20:18:54.295 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2023-10-10T20:18:54.295 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2023-10-10T20:18:54.295 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-10-10T20:18:54.295 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.296 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-10T20:18:54.296 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2023-10-10T20:18:54.296 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2023-10-10T20:18:54.296 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.296 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2023-10-10T20:18:54.297 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.298 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2023-10-10T20:18:54.298 INFO:teuthology.orchestra.run.smithi161.stdout: mon op complaint time = 120 2023-10-10T20:18:54.298 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull ratio = 0.6 2023-10-10T20:18:54.298 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.7 2023-10-10T20:18:54.298 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.6 2023-10-10T20:18:54.299 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-10-10T20:18:54.299 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2023-10-10T20:18:54.299 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2023-10-10T20:18:54.299 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2023-10-10T20:18:54.299 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2023-10-10T20:18:54.300 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-10T20:18:54.300 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-10T20:18:54.300 INFO:teuthology.orchestra.run.smithi161.stdout: client mount timeout = 600 2023-10-10T20:18:54.300 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2023-10-10T20:18:54.300 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = "" 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2023-10-10T20:18:54.301 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2023-10-10T20:18:54.302 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2023-10-10T20:18:54.302 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2023-10-10T20:18:54.302 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2023-10-10T20:18:54.302 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2023-10-10T20:18:54.302 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2023-10-10T20:18:54.303 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2023-10-10T20:18:54.303 INFO:teuthology.orchestra.run.smithi161.stdout: debug objectcacher = 20 2023-10-10T20:18:54.303 INFO:teuthology.orchestra.run.smithi161.stdout: debug objecter = 20 2023-10-10T20:18:54.303 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2023-10-10T20:18:54.303 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2023-10-10T20:18:54.304 INFO:teuthology.orchestra.run.smithi161.stdout: debug objectcacher = 20 2023-10-10T20:18:54.304 INFO:teuthology.orchestra.run.smithi161.stdout: debug objecter = 20 2023-10-10T20:18:54.304 INFO:teuthology.orchestra.run.smithi161.stdout:[mds] 2023-10-10T20:18:54.304 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds = 20 2023-10-10T20:18:54.304 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug frag = True 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug scatterstat = True 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: mds op complaint time = 180 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: mds verify scatter = True 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2023-10-10T20:18:54.305 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2023-10-10T20:18:54.306 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2023-10-10T20:18:54.306 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-10T20:18:54.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-10T20:18:54.364 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-10T20:18:54.365 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:54.365 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-10T20:18:54.381 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:54.381 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-10T20:18:54.435 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:18:54.436 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:54.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-10T20:18:54.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:18:54.510 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:18:54.510 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:54.564 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:18:54.565 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:54.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-10T20:18:54.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-10T20:18:54.600 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:18:54.600 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:54.654 INFO:tasks.ceph:Setting up mon nodes... 2023-10-10T20:18:54.655 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-10T20:18:54.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-10T20:18:54.703 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-10T20:18:54.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-10T20:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-10T20:18:54.768 INFO:tasks.ceph:Setting up mds nodes... 2023-10-10T20:18:54.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-10T20:18:54.817 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-10T20:18:54.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-10T20:18:54.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-10T20:18:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-10T20:18:54.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-10T20:18:54.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-10T20:18:55.006 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-10T20:18:55.008 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-10T20:18:55.033 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-10T20:18:55.120 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-10T20:18:55.123 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-10T20:18:55.147 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-10T20:18:55.148 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-10T20:18:55.186 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-10T20:18:55.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-10T20:18:55.276 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-10T20:18:55.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-10T20:18:55.326 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-10T20:18:55.339 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-10T20:18:55.416 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-10T20:18:55.428 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-10T20:18:55.429 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-10T20:18:55.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-10T20:18:55.455 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:55.455 INFO:tasks.ceph:role: osd.0 2023-10-10T20:18:55.456 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:18:55.456 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-10T20:18:55.527 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:55.527 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2023-10-10T20:18:55.528 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:55.528 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:55.528 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:55.529 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:55.529 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:55.529 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:18:55.956 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-10-10T20:18:55.957 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-10-10T20:18:55.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-10T20:18:56.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-10T20:18:56.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-10T20:18:56.230 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:56.230 INFO:tasks.ceph:role: osd.1 2023-10-10T20:18:56.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:18:56.231 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-10T20:18:56.303 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:56.303 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:56.303 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:56.303 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:56.304 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:18:56.726 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-10-10T20:18:56.728 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-10-10T20:18:56.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-10T20:18:56.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-10T20:18:56.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-10T20:18:56.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:56.924 INFO:tasks.ceph:role: osd.2 2023-10-10T20:18:56.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:18:56.925 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-10T20:18:56.995 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2023-10-10T20:18:56.996 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:56.997 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:56.997 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:56.997 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:18:57.447 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-10-10T20:18:57.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-10-10T20:18:57.450 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-10T20:18:57.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-10T20:18:57.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-10T20:18:57.721 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:57.722 INFO:tasks.ceph:role: osd.3 2023-10-10T20:18:57.722 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:18:57.722 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-10T20:18:57.794 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 2023-10-10T20:18:57.795 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:57.796 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:57.796 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:57.796 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:18:58.242 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2023-10-10T20:18:58.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2023-10-10T20:18:58.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-10T20:18:58.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-10T20:18:58.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:58.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.522+0000 7f4ad0e48540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-10T20:18:58.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.522+0000 7f4ad0e48540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-10T20:18:58.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.523+0000 7f4ad0e48540 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-10-10T20:18:58.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-10T20:18:58.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:58.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.637+0000 7fbe8fb61540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-10T20:18:58.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.637+0000 7fbe8fb61540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-10T20:18:58.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.637+0000 7fbe8fb61540 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-10-10T20:18:58.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-10T20:18:58.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:58.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.787+0000 7ff83df9c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-10T20:18:58.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.788+0000 7ff83df9c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-10T20:18:58.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.788+0000 7ff83df9c540 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-10-10T20:18:58.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-10T20:18:58.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:18:58.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.940+0000 7f6ab3214540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-10T20:18:58.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.941+0000 7f6ab3214540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-10T20:18:58.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:18:58.941+0000 7f6ab3214540 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-10-10T20:18:58.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-10T20:18:59.014 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-10T20:18:59.042 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:59.042 INFO:tasks.ceph:role: osd.4 2023-10-10T20:18:59.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:18:59.043 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-10T20:18:59.115 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:59.116 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:59.116 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:59.116 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2023-10-10T20:18:59.116 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:59.117 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:59.117 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:59.117 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:59.117 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:59.117 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:18:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-10-10T20:18:59.551 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-10-10T20:18:59.552 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-10T20:18:59.738 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-10T20:18:59.769 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-10T20:18:59.834 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:18:59.834 INFO:tasks.ceph:role: osd.5 2023-10-10T20:18:59.834 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:18:59.835 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-10T20:18:59.906 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:18:59.906 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:18:59.906 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:18:59.907 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2023-10-10T20:18:59.907 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:18:59.907 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:18:59.907 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:18:59.907 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:18:59.908 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:18:59.908 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:19:00.328 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-10-10T20:19:00.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-10-10T20:19:00.330 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-10T20:19:00.434 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-10T20:19:00.464 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-10T20:19:00.529 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:19:00.529 INFO:tasks.ceph:role: osd.6 2023-10-10T20:19:00.529 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:19:00.530 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-10T20:19:00.601 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:19:00.602 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:19:00.602 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:19:00.602 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2023-10-10T20:19:00.602 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:19:00.603 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:19:00.603 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:19:00.603 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:19:00.603 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:19:00.603 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:19:01.051 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-10-10T20:19:01.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-10-10T20:19:01.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-10T20:19:01.220 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-10T20:19:01.253 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-10T20:19:01.317 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-10T20:19:01.318 INFO:tasks.ceph:role: osd.7 2023-10-10T20:19:01.318 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:19:01.318 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-10T20:19:01.390 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-10T20:19:01.390 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-10T20:19:01.390 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-10T20:19:01.391 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2023-10-10T20:19:01.391 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-10T20:19:01.391 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-10-10T20:19:01.391 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-10T20:19:01.391 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-10T20:19:01.392 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-10T20:19:01.392 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-10T20:19:01.833 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-10-10T20:19:01.836 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-10-10T20:19:01.837 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-10T20:19:02.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-10T20:19:02.049 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:02.264 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.262+0000 7fabf8e40540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-10T20:19:02.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.262+0000 7fabf8e40540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-10T20:19:02.265 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.263+0000 7fabf8e40540 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-10-10T20:19:02.275 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-10T20:19:02.301 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:02.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.382+0000 7f37f0d1d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-10T20:19:02.384 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.382+0000 7f37f0d1d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-10T20:19:02.385 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.382+0000 7f37f0d1d540 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-10-10T20:19:02.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-10T20:19:02.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:02.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.534+0000 7fc6f7229540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-10T20:19:02.536 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.534+0000 7fc6f7229540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-10T20:19:02.537 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.534+0000 7fc6f7229540 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-10-10T20:19:02.545 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-10T20:19:02.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:02.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.688+0000 7fdfc50cb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-10T20:19:02.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.688+0000 7fdfc50cb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-10T20:19:02.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:02.689+0000 7fdfc50cb540 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-10-10T20:19:02.696 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-10T20:19:02.761 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-10T20:19:02.761 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:02.762 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-10T20:19:02.791 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:02.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-10T20:19:02.862 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:02.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-10T20:19:02.930 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:02.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-10T20:19:02.997 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:02.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-10T20:19:03.065 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:03.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-10T20:19:03.130 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:03.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-10T20:19:03.196 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-10T20:19:03.223 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.223 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-10T20:19:03.288 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.288 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-10T20:19:03.352 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-10T20:19:03.417 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-10T20:19:03.481 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-10T20:19:03.545 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.545 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-10T20:19:03.608 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:03.609 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-10T20:19:03.626 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:03.627 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-10T20:19:03.684 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.684 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-10T20:19:03.699 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:03.700 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-10T20:19:03.754 INFO:tasks.ceph:Adding keys to all mons... 2023-10-10T20:19:03.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-10T20:19:03.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-10T20:19:03.798 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2023-10-10T20:19:03.798 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA== 2023-10-10T20:19:03.798 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2023-10-10T20:19:03.799 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug== 2023-10-10T20:19:03.799 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2023-10-10T20:19:03.799 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow== 2023-10-10T20:19:03.799 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2023-10-10T20:19:03.800 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q== 2023-10-10T20:19:03.800 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2023-10-10T20:19:03.800 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw== 2023-10-10T20:19:03.800 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2023-10-10T20:19:03.800 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg== 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA== 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg== 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2023-10-10T20:19:03.801 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA== 2023-10-10T20:19:03.802 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2023-10-10T20:19:03.802 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ== 2023-10-10T20:19:03.802 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2023-10-10T20:19:03.803 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA== 2023-10-10T20:19:03.803 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2023-10-10T20:19:03.803 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg== 2023-10-10T20:19:03.804 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2023-10-10T20:19:03.804 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg== 2023-10-10T20:19:03.804 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2023-10-10T20:19:03.804 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA== 2023-10-10T20:19:03.804 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2023-10-10T20:19:03.805 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA== 2023-10-10T20:19:03.805 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2023-10-10T20:19:03.805 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ== 2023-10-10T20:19:03.805 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2023-10-10T20:19:03.805 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ== 2023-10-10T20:19:03.806 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2023-10-10T20:19:03.806 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA== 2023-10-10T20:19:03.817 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2023-10-10T20:19:03.817 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA== 2023-10-10T20:19:03.818 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2023-10-10T20:19:03.818 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug== 2023-10-10T20:19:03.818 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2023-10-10T20:19:03.818 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow== 2023-10-10T20:19:03.819 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2023-10-10T20:19:03.819 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q== 2023-10-10T20:19:03.819 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2023-10-10T20:19:03.819 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw== 2023-10-10T20:19:03.819 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2023-10-10T20:19:03.820 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg== 2023-10-10T20:19:03.820 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2023-10-10T20:19:03.820 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA== 2023-10-10T20:19:03.820 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2023-10-10T20:19:03.820 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg== 2023-10-10T20:19:03.821 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2023-10-10T20:19:03.821 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA== 2023-10-10T20:19:03.821 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2023-10-10T20:19:03.821 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ== 2023-10-10T20:19:03.821 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2023-10-10T20:19:03.822 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA== 2023-10-10T20:19:03.822 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2023-10-10T20:19:03.822 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg== 2023-10-10T20:19:03.822 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2023-10-10T20:19:03.822 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg== 2023-10-10T20:19:03.823 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2023-10-10T20:19:03.823 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA== 2023-10-10T20:19:03.823 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2023-10-10T20:19:03.823 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA== 2023-10-10T20:19:03.824 INFO:teuthology.orchestra.run.smithi161.stdout:[client.2] 2023-10-10T20:19:03.824 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ== 2023-10-10T20:19:03.824 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2023-10-10T20:19:03.825 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ== 2023-10-10T20:19:03.825 INFO:teuthology.orchestra.run.smithi161.stdout:[client.3] 2023-10-10T20:19:03.825 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA== 2023-10-10T20:19:03.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:19:03.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:19:03.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:03.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:03.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:03.969 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:04.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.021 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.070 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.120 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:19:04.221 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-10T20:19:04.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:04.270 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:04.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:04.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-10T20:19:04.372 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.422 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-10T20:19:04.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.626 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.628 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-10T20:19:04.851 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-10T20:19:04.852 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-10T20:19:04.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:19:04.968 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-10T20:19:04.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-10T20:19:05.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:19:05.110 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-10T20:19:05.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-10T20:19:05.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-10T20:19:05.291 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-10T20:19:05.317 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:05.320 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-10T20:19:05.369 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-10T20:19:05.369 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-10T20:19:05.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-10T20:19:05.376 INFO:tasks.ceph.mon.a:Started 2023-10-10T20:19:05.376 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-10T20:19:05.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-10T20:19:05.411 INFO:tasks.ceph.mon.b:Started 2023-10-10T20:19:05.412 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-10T20:19:05.412 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-10T20:19:05.414 INFO:tasks.ceph.mon.c:Started 2023-10-10T20:19:05.415 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-10T20:19:05.415 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-10T20:19:05.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-10T20:19:05.418 INFO:tasks.ceph.mgr.y:Started 2023-10-10T20:19:05.418 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-10T20:19:05.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-10T20:19:05.420 INFO:tasks.ceph.mgr.x:Started 2023-10-10T20:19:05.421 INFO:tasks.ceph:Setting crush tunables to default 2023-10-10T20:19:05.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd crush tunables default 2023-10-10T20:19:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 Processor -- start 2023-10-10T20:19:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 -- start start 2023-10-10T20:19:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f4070d30 con 0x7fd1f406ca90 2023-10-10T20:19:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f4070700 con 0x7fd1f406ecc0 2023-10-10T20:19:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.540+0000 7fd1fa3fe700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f406f710 con 0x7fd1f4159c30 2023-10-10T20:19:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54268/0 (socket says 172.21.15.148:54268) 2023-10-10T20:19:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f899b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd1f406ca90 0x7fd1f415a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50814/0 (socket says 172.21.15.148:50814) 2023-10-10T20:19:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 learned_addr learned my addr 172.21.15.148:0/265050015 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 msgr2=0x7fd1f40711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-10T20:19:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 msgr2=0x7fd1f40711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-10T20:19:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-10T20:19:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.541+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-10T20:19:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016302307 0 0) 0x7fd1f4070d30 con 0x7fd1f406ca90 2023-10-10T20:19:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1dc003570 con 0x7fd1f406ca90 2023-10-10T20:19:05.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137998598 0 0) 0x7fd1f4070700 con 0x7fd1f406ecc0 2023-10-10T20:19:05.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f4070d30 con 0x7fd1f406ecc0 2023-10-10T20:19:05.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44445011 0 0) 0x7fd1f4070d30 con 0x7fd1f406ecc0 2023-10-10T20:19:05.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 msgr2=0x7fd1f40711c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:05.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 --2- 172.21.15.148:0/265050015 >> v2:172.21.15.148:3300/0 conn(0x7fd1f4159c30 0x7fd1f40711c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 >> v1:172.21.15.148:6789/0 conn(0x7fd1f406ca90 legacy=0x7fd1f415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f406f9f0 con 0x7fd1f406ecc0 2023-10-10T20:19:05.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fd1e8001470 con 0x7fd1f406ecc0 2023-10-10T20:19:05.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 >> v1:172.21.15.161:6789/0 conn(0x7fd1f406ecc0 legacy=0x7fd1f406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 0x7fd1dc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1dc004810 0x7fd1dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc005230 0x7fd1dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f406f710 con 0x7fd1dc003bf0 2023-10-10T20:19:05.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00c160 con 0x7fd1dc005230 2023-10-10T20:19:05.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc00c320 con 0x7fd1dc004810 2023-10-10T20:19:05.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.737+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 0x7fd1dc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f899b700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc005230 0x7fd1dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f37fe700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1dc004810 0x7fd1dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1dc004810 msgr2=0x7fd1dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1dc004810 0x7fd1dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc005230 msgr2=0x7fd1dc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc005230 0x7fd1dc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 -- 172.21.15.148:0/265050015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4070d30 con 0x7fd1dc003bf0 2023-10-10T20:19:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f899b700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc005230 0x7fd1dc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f899b700 1 -- 172.21.15.148:0/265050015 reap_dead start 2023-10-10T20:19:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 0x7fd1dc003fe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009060 tx=0x7fd1e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7258785ed32d25ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400c070 con 0x7fd1dc003bf0 2023-10-10T20:19:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e400e5e0 con 0x7fd1dc003bf0 2023-10-10T20:19:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.738+0000 7fd1f2ffd700 1 -- 172.21.15.148:0/265050015 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400e790 con 0x7fd1dc003bf0 2023-10-10T20:19:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 msgr2=0x7fd1dc003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 0x7fd1dc003fe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009060 tx=0x7fd1e4008040 comp rx=0 tx=0).stop 2023-10-10T20:19:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/265050015 shutdown_connections 2023-10-10T20:19:05.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/265050015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc003bf0 0x7fd1dc003fe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/265050015 >> 172.21.15.148:0/265050015 conn(0x7fd1f40cb030 msgr2=0x7fd1f406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:05.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/265050015 shutdown_connections 2023-10-10T20:19:05.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.739+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/265050015 wait complete. 2023-10-10T20:19:05.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.741+0000 7fd1fa3fe700 1 Processor -- start 2023-10-10T20:19:05.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.741+0000 7fd1fa3fe700 1 -- start start 2023-10-10T20:19:05.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 0x7fd1f41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1f40afde0 0x7fd1f40c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f416bd50 con 0x7fd1f40adb30 2023-10-10T20:19:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40adfc0 con 0x7fd1dc003bf0 2023-10-10T20:19:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1fa3fe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40ae2a0 con 0x7fd1f40afde0 2023-10-10T20:19:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54326/0 (socket says 172.21.15.148:54326) 2023-10-10T20:19:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f3fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 0x7fd1f41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 -- 172.21.15.148:0/3110074852 learned_addr learned my addr 172.21.15.148:0/3110074852 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f899b700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1f40afde0 0x7fd1f40c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:05.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 -- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1f40afde0 msgr2=0x7fd1f40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1f40afde0 0x7fd1f40c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 -- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 msgr2=0x7fd1f41532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 0x7fd1f41532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 -- 172.21.15.148:0/3110074852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f416b700 con 0x7fd1f40adb30 2023-10-10T20:19:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f3fff700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 0x7fd1f41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.742+0000 7fd1f37fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000fd0 tx=0x7fd1e8002e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800d7f0 con 0x7fd1f40adb30 2023-10-10T20:19:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e800d950 con 0x7fd1f40adb30 2023-10-10T20:19:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f416b9e0 con 0x7fd1f40adb30 2023-10-10T20:19:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800dd10 con 0x7fd1f40adb30 2023-10-10T20:19:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f40c9860 con 0x7fd1f40adb30 2023-10-10T20:19:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd1e8014900 con 0x7fd1f40adb30 2023-10-10T20:19:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.743+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd1e801d7e0 con 0x7fd1f40adb30 2023-10-10T20:19:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.744+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f40c9a80 con 0x7fd1f40adb30 2023-10-10T20:19:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.747+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd1e803c6d0 con 0x7fd1f40adb30 2023-10-10T20:19:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.845+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd1f416bec0 con 0x7fd1f40adb30 2023-10-10T20:19:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.848+0000 7fd1dbfff700 1 -- 172.21.15.148:0/3110074852 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd1e802b020 con 0x7fd1f40adb30 2023-10-10T20:19:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 msgr2=0x7fd1f40af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000fd0 tx=0x7fd1e8002e40 comp rx=0 tx=0).stop 2023-10-10T20:19:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 shutdown_connections 2023-10-10T20:19:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1f40afde0 0x7fd1f40c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1dc003bf0 0x7fd1f41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 --2- 172.21.15.148:0/3110074852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1f40adb30 0x7fd1f40af8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:05.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 >> 172.21.15.148:0/3110074852 conn(0x7fd1f40cb030 msgr2=0x7fd1f4067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:05.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 shutdown_connections 2023-10-10T20:19:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:05.849+0000 7fd1fa3fe700 1 -- 172.21.15.148:0/3110074852 wait complete. 2023-10-10T20:19:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2023-10-10T20:19:05.864 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-10T20:19:05.865 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:05.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-10T20:19:05.891 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:05.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-10T20:19:05.956 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:05.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-10T20:19:06.021 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2023-10-10T20:19:06.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-10T20:19:06.085 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:06.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-10T20:19:06.112 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:06.113 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-10T20:19:06.178 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:06.178 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-10T20:19:06.243 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-10-10T20:19:06.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-10T20:19:06.307 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-10-10T20:19:06.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.462+0000 7fd80db13700 1 Processor -- start 2023-10-10T20:19:06.464 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.462+0000 7fd80db13700 1 -- start start 2023-10-10T20:19:06.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd80db13700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8080cbf10 0x7fd80815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.466 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd80db13700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cae70 con 0x7fd8080678b0 2023-10-10T20:19:06.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd80db13700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cb150 con 0x7fd8080c9390 2023-10-10T20:19:06.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd80db13700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cb430 con 0x7fd8080cbf10 2023-10-10T20:19:06.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd806ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8080cbf10 0x7fd80815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd807fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd8080c9390 0x7fd80815a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34684/0 (socket says 172.21.15.161:34684) 2023-10-10T20:19:06.467 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd806ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8080cbf10 0x7fd80815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53910/0 (socket says 172.21.15.161:53910) 2023-10-10T20:19:06.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.463+0000 7fd807fff700 1 -- 172.21.15.161:0/3479170150 learned_addr learned my addr 172.21.15.161:0/3479170150 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-10-10T20:19:06.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd806ffd700 1 -- 172.21.15.161:0/3479170150 >> v1:172.21.15.161:6789/0 conn(0x7fd8080c9390 legacy=0x7fd80815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd806ffd700 1 -- 172.21.15.161:0/3479170150 >> v1:172.21.15.148:6789/0 conn(0x7fd8080678b0 legacy=0x7fd80815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd8067fc700 1 -- 172.21.15.161:0/3479170150 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856108153 0 0) 0x7fd8080cb150 con 0x7fd8080c9390 2023-10-10T20:19:06.468 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd806ffd700 1 -- 172.21.15.161:0/3479170150 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080cb710 con 0x7fd8080cbf10 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd806ffd700 1 --2- 172.21.15.161:0/3479170150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080cbf10 0x7fd80815e290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098e0 tx=0x7fd7f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365de9c6c5c72397 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd8067fc700 1 -- 172.21.15.161:0/3479170150 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0009e80 con 0x7fd8080cbf10 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd8067fc700 1 -- 172.21.15.161:0/3479170150 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000b760 con 0x7fd8080cbf10 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.464+0000 7fd8067fc700 1 -- 172.21.15.161:0/3479170150 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000bae0 con 0x7fd8080cbf10 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 -- 172.21.15.161:0/3479170150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080cbf10 msgr2=0x7fd80815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.469 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 --2- 172.21.15.161:0/3479170150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080cbf10 0x7fd80815e290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098e0 tx=0x7fd7f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 -- 172.21.15.161:0/3479170150 shutdown_connections 2023-10-10T20:19:06.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 --2- 172.21.15.161:0/3479170150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080cbf10 0x7fd80815e290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 -- 172.21.15.161:0/3479170150 >> 172.21.15.161:0/3479170150 conn(0x7fd808069870 msgr2=0x7fd808069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:06.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 -- 172.21.15.161:0/3479170150 shutdown_connections 2023-10-10T20:19:06.470 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.465+0000 7fd80db13700 1 -- 172.21.15.161:0/3479170150 wait complete. 2023-10-10T20:19:06.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.466+0000 7fd80db13700 1 Processor -- start 2023-10-10T20:19:06.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.466+0000 7fd80db13700 1 -- start start 2023-10-10T20:19:06.473 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8080c9390 0x7fd808150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8080cbf10 0x7fd80815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b3430 con 0x7fd8080c9390 2023-10-10T20:19:06.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b3710 con 0x7fd8080678b0 2023-10-10T20:19:06.474 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b39f0 con 0x7fd8080cbf10 2023-10-10T20:19:06.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53920/0 (socket says 172.21.15.161:53920) 2023-10-10T20:19:06.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 -- 172.21.15.161:0/2302976341 learned_addr learned my addr 172.21.15.161:0/2302976341 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:06.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd807fff700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8080cbf10 0x7fd80815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.475 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd806ffd700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8080c9390 0x7fd808150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 -- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8080cbf10 msgr2=0x7fd80815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8080cbf10 0x7fd80815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 -- 172.21.15.161:0/2302976341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8080c9390 msgr2=0x7fd808150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8080c9390 0x7fd808150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.476 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 -- 172.21.15.161:0/2302976341 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f0009420 con 0x7fd8080678b0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd8077fe700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c5c0 tx=0x7fd7f800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8011590 con 0x7fd8080678b0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80116f0 con 0x7fd8080678b0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.467+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8080b3cd0 con 0x7fd8080678b0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.468+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8011ab0 con 0x7fd8080678b0 2023-10-10T20:19:06.477 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.468+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8080c7570 con 0x7fd8080678b0 2023-10-10T20:19:06.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.468+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd7f801c6c0 con 0x7fd8080678b0 2023-10-10T20:19:06.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.468+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd7f8012770 con 0x7fd8080678b0 2023-10-10T20:19:06.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.468+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80804c3c0 con 0x7fd8080678b0 2023-10-10T20:19:06.478 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.471+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd7f8017070 con 0x7fd8080678b0 2023-10-10T20:19:06.553 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.552+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7fd808068da0 con 0x7fd8080678b0 2023-10-10T20:19:06.557 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.556+0000 7fd7ff7fe700 1 -- 172.21.15.161:0/2302976341 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd7f801cba0 con 0x7fd8080678b0 2023-10-10T20:19:06.557 INFO:teuthology.orchestra.run.smithi161.stdout:0 2023-10-10T20:19:06.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 msgr2=0x7fd8080c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.559 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c5c0 tx=0x7fd7f800c8d0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 shutdown_connections 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8080cbf10 0x7fd80815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8080678b0 0x7fd8080c7030 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 --2- 172.21.15.161:0/2302976341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8080c9390 0x7fd808150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 >> 172.21.15.161:0/2302976341 conn(0x7fd808069870 msgr2=0x7fd80815ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:06.560 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 shutdown_connections 2023-10-10T20:19:06.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.558+0000 7fd80db13700 1 -- 172.21.15.161:0/2302976341 wait complete. 2023-10-10T20:19:06.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-10-10T20:19:06.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.688+0000 7f950fb19700 1 Processor -- start 2023-10-10T20:19:06.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950fb19700 1 -- start start 2023-10-10T20:19:06.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950fb19700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95080cc870 0x7f95080cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950fb19700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9508157010 con 0x7f950815b2e0 2023-10-10T20:19:06.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950fb19700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95080cbbb0 con 0x7f9508156c40 2023-10-10T20:19:06.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950fb19700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950815b710 con 0x7f95080cc870 2023-10-10T20:19:06.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950d8b5700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9508156c40 0x7f95080c98a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34694/0 (socket says 172.21.15.161:34694) 2023-10-10T20:19:06.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.689+0000 7f950d8b5700 1 -- 172.21.15.161:0/2156557922 learned_addr learned my addr 172.21.15.161:0/2156557922 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-10-10T20:19:06.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950e0b6700 1 --2- 172.21.15.161:0/2156557922 >> v2:172.21.15.148:3300/0 conn(0x7f95080cc870 0x7f95080cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090822391 0 0) 0x7f94f8001110 con 0x7f9508156c40 2023-10-10T20:19:06.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f4003570 con 0x7f9508156c40 2023-10-10T20:19:06.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372126461 0 0) 0x7f9508157010 con 0x7f950815b2e0 2023-10-10T20:19:06.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f8001110 con 0x7f950815b2e0 2023-10-10T20:19:06.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950e0b6700 1 -- 172.21.15.161:0/2156557922 >> v1:172.21.15.161:6789/0 conn(0x7f9508156c40 legacy=0x7f95080c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950e0b6700 1 -- 172.21.15.161:0/2156557922 >> v1:172.21.15.148:6789/0 conn(0x7f950815b2e0 legacy=0x7f95080cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950e0b6700 1 -- 172.21.15.161:0/2156557922 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95080c9410 con 0x7f95080cc870 2023-10-10T20:19:06.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950e0b6700 1 --2- 172.21.15.161:0/2156557922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95080cc870 0x7f95080cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9500009e90 tx=0x7f9500009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4c3646a3dad32 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000e070 con 0x7f95080cc870 2023-10-10T20:19:06.695 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000bb90 con 0x7f95080cc870 2023-10-10T20:19:06.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.690+0000 7f950c8b3700 1 -- 172.21.15.161:0/2156557922 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500012430 con 0x7f95080cc870 2023-10-10T20:19:06.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 -- 172.21.15.161:0/2156557922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95080cc870 msgr2=0x7f95080cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 --2- 172.21.15.161:0/2156557922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95080cc870 0x7f95080cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9500009e90 tx=0x7f9500009780 comp rx=0 tx=0).stop 2023-10-10T20:19:06.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 -- 172.21.15.161:0/2156557922 shutdown_connections 2023-10-10T20:19:06.696 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 --2- 172.21.15.161:0/2156557922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95080cc870 0x7f95080cb670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 -- 172.21.15.161:0/2156557922 >> 172.21.15.161:0/2156557922 conn(0x7f95080ce570 msgr2=0x7f95080ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:06.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 -- 172.21.15.161:0/2156557922 shutdown_connections 2023-10-10T20:19:06.697 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.691+0000 7f950fb19700 1 -- 172.21.15.161:0/2156557922 wait complete. 2023-10-10T20:19:06.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.692+0000 7f950fb19700 1 Processor -- start 2023-10-10T20:19:06.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.692+0000 7f950fb19700 1 -- start start 2023-10-10T20:19:06.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95080cc870 0x7f95080b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.700 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f950815b2e0 0x7f95080c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c6ab0 con 0x7f950815b2e0 2023-10-10T20:19:06.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9508070700 con 0x7f9508156c40 2023-10-10T20:19:06.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950fb19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9508070950 con 0x7f95080cc870 2023-10-10T20:19:06.701 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d8b5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95080cc870 0x7f95080b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53928/0 (socket says 172.21.15.161:53928) 2023-10-10T20:19:06.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 -- 172.21.15.161:0/486759140 learned_addr learned my addr 172.21.15.161:0/486759140 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:06.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950e0b6700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f950815b2e0 0x7f95080c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.702 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 -- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95080cc870 msgr2=0x7f95080b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95080cc870 0x7f95080b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 -- 172.21.15.161:0/486759140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f950815b2e0 msgr2=0x7f95080c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f950815b2e0 0x7f95080c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 -- 172.21.15.161:0/486759140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009a20 con 0x7f9508156c40 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.693+0000 7f950d0b4700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f950400c790 tx=0x7f9504007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.703 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504010510 con 0x7f9508156c40 2023-10-10T20:19:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9504010670 con 0x7f9508156c40 2023-10-10T20:19:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9508070c30 con 0x7f9508156c40 2023-10-10T20:19:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504010a10 con 0x7f9508156c40 2023-10-10T20:19:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95080711f0 con 0x7f9508156c40 2023-10-10T20:19:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f950401c6c0 con 0x7f9508156c40 2023-10-10T20:19:06.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.694+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f9504012800 con 0x7f9508156c40 2023-10-10T20:19:06.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.695+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9508071410 con 0x7f9508156c40 2023-10-10T20:19:06.705 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.697+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9504017070 con 0x7f9508156c40 2023-10-10T20:19:06.781 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.780+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7f950804c3c0 con 0x7f9508156c40 2023-10-10T20:19:06.785 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.784+0000 7f94fdffb700 1 -- 172.21.15.161:0/486759140 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f950401cba0 con 0x7f9508156c40 2023-10-10T20:19:06.786 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:19:06.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 msgr2=0x7f95080c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f950400c790 tx=0x7f9504007e10 comp rx=0 tx=0).stop 2023-10-10T20:19:06.787 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 shutdown_connections 2023-10-10T20:19:06.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f950815b2e0 0x7f95080c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95080cc870 0x7f95080b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 --2- 172.21.15.161:0/486759140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9508156c40 0x7f95080c6030 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 >> 172.21.15.161:0/486759140 conn(0x7f95080ce570 msgr2=0x7f95080c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:06.788 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 shutdown_connections 2023-10-10T20:19:06.789 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.785+0000 7f950fb19700 1 -- 172.21.15.161:0/486759140 wait complete. 2023-10-10T20:19:06.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-10-10T20:19:06.905 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:06.903+0000 7f7e6e0cd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-10T20:19:06.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.915+0000 7fcfa9fcb700 1 Processor -- start 2023-10-10T20:19:06.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.915+0000 7fcfa9fcb700 1 -- start start 2023-10-10T20:19:06.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa9fcb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 0x7fcfa415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa9fcb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfa406aa00 0x7fcfa40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa9fcb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40b4d70 con 0x7fcfa40b6580 2023-10-10T20:19:06.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa9fcb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40cd8d0 con 0x7fcfa406aa00 2023-10-10T20:19:06.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa9fcb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40b4440 con 0x7fcfa40cce30 2023-10-10T20:19:06.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa3fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcfa40b6580 0x7fcfa406a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34708/0 (socket says 172.21.15.161:34708) 2023-10-10T20:19:06.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa3fff700 1 -- 172.21.15.161:0/1219585533 learned_addr learned my addr 172.21.15.161:0/1219585533 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-10-10T20:19:06.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 0x7fcfa415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa97ca700 1 --2- 172.21.15.161:0/1219585533 >> v2:172.21.15.148:3300/0 conn(0x7fcfa406aa00 0x7fcfa40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa37fe700 1 -- 172.21.15.161:0/1219585533 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471526282 0 0) 0x7fcfa40b4d70 con 0x7fcfa40b6580 2023-10-10T20:19:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa37fe700 1 -- 172.21.15.161:0/1219585533 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf94003570 con 0x7fcfa40b6580 2023-10-10T20:19:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 -- 172.21.15.161:0/1219585533 >> v2:172.21.15.148:3300/0 conn(0x7fcfa406aa00 msgr2=0x7fcfa40b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 --2- 172.21.15.161:0/1219585533 >> v2:172.21.15.148:3300/0 conn(0x7fcfa406aa00 0x7fcfa40b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 -- 172.21.15.161:0/1219585533 >> v1:172.21.15.161:6789/0 conn(0x7fcfa40b6580 legacy=0x7fcfa406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 -- 172.21.15.161:0/1219585533 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40b4720 con 0x7fcfa40cce30 2023-10-10T20:19:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.916+0000 7fcfa8fc9700 1 --2- 172.21.15.161:0/1219585533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 0x7fcfa415aca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf980098e0 tx=0x7fcf980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03547c513233f78 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa37fe700 1 -- 172.21.15.161:0/1219585533 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98011630 con 0x7fcfa40cce30 2023-10-10T20:19:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa37fe700 1 -- 172.21.15.161:0/1219585533 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf98011790 con 0x7fcfa40cce30 2023-10-10T20:19:06.922 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa37fe700 1 -- 172.21.15.161:0/1219585533 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98011b30 con 0x7fcfa40cce30 2023-10-10T20:19:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/1219585533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 msgr2=0x7fcfa415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/1219585533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 0x7fcfa415aca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf980098e0 tx=0x7fcf980091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/1219585533 shutdown_connections 2023-10-10T20:19:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/1219585533 >> v2:172.21.15.148:3300/0 conn(0x7fcfa406aa00 0x7fcfa40b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.923 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/1219585533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40cce30 0x7fcfa415aca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/1219585533 >> 172.21.15.161:0/1219585533 conn(0x7fcfa4068cb0 msgr2=0x7fcfa40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:06.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/1219585533 shutdown_connections 2023-10-10T20:19:06.924 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.917+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/1219585533 wait complete. 2023-10-10T20:19:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.918+0000 7fcfa9fcb700 1 Processor -- start 2023-10-10T20:19:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 -- start start 2023-10-10T20:19:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.927 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 0x7fcfa40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40b2940 con 0x7fcfa40cce30 2023-10-10T20:19:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40b1d30 con 0x7fcfa406aa00 2023-10-10T20:19:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa9fcb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40b2010 con 0x7fcfa40b6580 2023-10-10T20:19:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa3fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.928 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa8fc9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa8fc9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53932/0 (socket says 172.21.15.161:53932) 2023-10-10T20:19:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa3fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:58366/0 (socket says 172.21.15.161:58366) 2023-10-10T20:19:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.919+0000 7fcfa8fc9700 1 -- 172.21.15.161:0/2877736245 learned_addr learned my addr 172.21.15.161:0/2877736245 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa97ca700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 0x7fcfa40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:06.929 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 -- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 msgr2=0x7fcfa414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 -- 172.21.15.161:0/2877736245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 msgr2=0x7fcfa40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 0x7fcfa40b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 -- 172.21.15.161:0/2877736245 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98009420 con 0x7fcfa40b6580 2023-10-10T20:19:06.930 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa8fc9700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa97ca700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 0x7fcfa40b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa3fff700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b6990 tx=0x7fcf90007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005b30 con 0x7fcfa40b6580 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf90005cb0 con 0x7fcfa40b6580 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa40b22f0 con 0x7fcfa40b6580 2023-10-10T20:19:06.931 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90007320 con 0x7fcfa40b6580 2023-10-10T20:19:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.920+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa416ab90 con 0x7fcfa40b6580 2023-10-10T20:19:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.921+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcf9001c6c0 con 0x7fcfa40b6580 2023-10-10T20:19:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.921+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fcf90005e20 con 0x7fcfa40b6580 2023-10-10T20:19:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.921+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa40b2ab0 con 0x7fcfa40b6580 2023-10-10T20:19:06.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:06.923+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fcf90017070 con 0x7fcfa40b6580 2023-10-10T20:19:07.006 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.005+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7fcfa40b4a00 con 0x7fcfa40b6580 2023-10-10T20:19:07.010 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.009+0000 7fcfa0ff9700 1 -- 172.21.15.161:0/2877736245 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf9001cba0 con 0x7fcfa40b6580 2023-10-10T20:19:07.010 INFO:teuthology.orchestra.run.smithi161.stdout:2 2023-10-10T20:19:07.011 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 msgr2=0x7fcfa40b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b6990 tx=0x7fcf90007d50 comp rx=0 tx=0).stop 2023-10-10T20:19:07.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 shutdown_connections 2023-10-10T20:19:07.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfa40cce30 0x7fcfa40b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.012 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfa406aa00 0x7fcfa414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 --2- 172.21.15.161:0/2877736245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfa40b6580 0x7fcfa40b1670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 >> 172.21.15.161:0/2877736245 conn(0x7fcfa4068cb0 msgr2=0x7fcfa40d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 shutdown_connections 2023-10-10T20:19:07.013 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.010+0000 7fcfa9fcb700 1 -- 172.21.15.161:0/2877736245 wait complete. 2023-10-10T20:19:07.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-10-10T20:19:07.039 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.037+0000 7f7e6e0cd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-10T20:19:07.116 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.114+0000 7f7e6e0cd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-10T20:19:07.138 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.137+0000 7f5925718700 1 Processor -- start 2023-10-10T20:19:07.139 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.137+0000 7f5925718700 1 -- start start 2023-10-10T20:19:07.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f5925718700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5920156c40 0x7f592015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f5925718700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59200cb670 0x7f59200c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.141 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f5925718700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5920157010 con 0x7f59200cc160 2023-10-10T20:19:07.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f5925718700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200cbdf0 con 0x7f59200cb670 2023-10-10T20:19:07.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f5925718700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200c9410 con 0x7f5920156c40 2023-10-10T20:19:07.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f591effd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5920156c40 0x7f592015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f591f7fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f59200cc160 0x7f592006e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34712/0 (socket says 172.21.15.161:34712) 2023-10-10T20:19:07.142 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f591f7fe700 1 -- 172.21.15.161:0/1036356682 learned_addr learned my addr 172.21.15.161:0/1036356682 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-10-10T20:19:07.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.138+0000 7f591e7fc700 1 --2- 172.21.15.161:0/1036356682 >> v2:172.21.15.148:3300/0 conn(0x7f59200cb670 0x7f59200c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591effd700 1 -- 172.21.15.161:0/1036356682 >> v2:172.21.15.148:3300/0 conn(0x7f59200cb670 msgr2=0x7f59200c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:07.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591effd700 1 --2- 172.21.15.161:0/1036356682 >> v2:172.21.15.148:3300/0 conn(0x7f59200cb670 0x7f59200c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591effd700 1 -- 172.21.15.161:0/1036356682 >> v1:172.21.15.161:6789/0 conn(0x7f59200cc160 legacy=0x7f592006e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.143 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591effd700 1 -- 172.21.15.161:0/1036356682 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200c8840 con 0x7f5920156c40 2023-10-10T20:19:07.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591dffb700 1 -- 172.21.15.161:0/1036356682 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842472212 0 0) 0x7f5920157010 con 0x7f59200cc160 2023-10-10T20:19:07.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591effd700 1 --2- 172.21.15.161:0/1036356682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920156c40 0x7f592015b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f592006e0e0 tx=0x7f59140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23573ecddc5bf773 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591dffb700 1 -- 172.21.15.161:0/1036356682 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011630 con 0x7f5920156c40 2023-10-10T20:19:07.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591dffb700 1 -- 172.21.15.161:0/1036356682 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914011790 con 0x7f5920156c40 2023-10-10T20:19:07.144 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f591dffb700 1 -- 172.21.15.161:0/1036356682 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011b30 con 0x7f5920156c40 2023-10-10T20:19:07.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 -- 172.21.15.161:0/1036356682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920156c40 msgr2=0x7f592015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 --2- 172.21.15.161:0/1036356682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920156c40 0x7f592015b2e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f592006e0e0 tx=0x7f59140091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 -- 172.21.15.161:0/1036356682 shutdown_connections 2023-10-10T20:19:07.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 --2- 172.21.15.161:0/1036356682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920156c40 0x7f592015b2e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.145 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 --2- 172.21.15.161:0/1036356682 >> v2:172.21.15.148:3300/0 conn(0x7f59200cb670 0x7f59200c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.139+0000 7f5925718700 1 -- 172.21.15.161:0/1036356682 >> 172.21.15.161:0/1036356682 conn(0x7f59200ce570 msgr2=0x7f59200ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.140+0000 7f5925718700 1 -- 172.21.15.161:0/1036356682 shutdown_connections 2023-10-10T20:19:07.146 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.140+0000 7f5925718700 1 -- 172.21.15.161:0/1036356682 wait complete. 2023-10-10T20:19:07.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.141+0000 7f5925718700 1 Processor -- start 2023-10-10T20:19:07.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.141+0000 7f5925718700 1 -- start start 2023-10-10T20:19:07.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.141+0000 7f5925718700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 0x7f59200c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.149 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5920156c40 0x7f592006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200b25e0 con 0x7f5920156c40 2023-10-10T20:19:07.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592016a450 con 0x7f59200cb670 2023-10-10T20:19:07.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592016a730 con 0x7f59200cc160 2023-10-10T20:19:07.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.150 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53946/0 (socket says 172.21.15.161:53946) 2023-10-10T20:19:07.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 -- 172.21.15.161:0/1620122300 learned_addr learned my addr 172.21.15.161:0/1620122300 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591e7fc700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 0x7f59200c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591f7fe700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5920156c40 0x7f592006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.151 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 -- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 msgr2=0x7f59200c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 0x7f59200c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 -- 172.21.15.161:0/1620122300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5920156c40 msgr2=0x7f592006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5920156c40 0x7f592006eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 -- 172.21.15.161:0/1620122300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914009420 con 0x7f59200cb670 2023-10-10T20:19:07.152 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591e7fc700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 0x7f59200c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f591effd700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f59140091e0 tx=0x7f5914017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011900 con 0x7f59200cb670 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914011a60 con 0x7f59200cb670 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592016aa10 con 0x7f59200cb670 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914025640 con 0x7f59200cb670 2023-10-10T20:19:07.153 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592016e520 con 0x7f59200cb670 2023-10-10T20:19:07.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.142+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5914025cf0 con 0x7f59200cb670 2023-10-10T20:19:07.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.143+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f591402eca0 con 0x7f59200cb670 2023-10-10T20:19:07.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.143+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592004c3c0 con 0x7f59200cb670 2023-10-10T20:19:07.154 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.145+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5914033060 con 0x7f59200cb670 2023-10-10T20:19:07.228 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.227+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f59200cbf40 con 0x7f59200cb670 2023-10-10T20:19:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.232+0000 7f5906ffd700 1 -- 172.21.15.161:0/1620122300 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5914015020 con 0x7f59200cb670 2023-10-10T20:19:07.233 INFO:teuthology.orchestra.run.smithi161.stdout:3 2023-10-10T20:19:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 msgr2=0x7f592016dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f59140091e0 tx=0x7f5914017000 comp rx=0 tx=0).stop 2023-10-10T20:19:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 shutdown_connections 2023-10-10T20:19:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5920156c40 0x7f592006eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59200cb670 0x7f592016dfe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 --2- 172.21.15.161:0/1620122300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59200cc160 0x7f59200c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 >> 172.21.15.161:0/1620122300 conn(0x7f59200ce570 msgr2=0x7f5920069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 shutdown_connections 2023-10-10T20:19:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.233+0000 7f5925718700 1 -- 172.21.15.161:0/1620122300 wait complete. 2023-10-10T20:19:07.247 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-10-10T20:19:07.304 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.302+0000 7f7e6e0cd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:19:07.305 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.303+0000 7fbb4940d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-10T20:19:07.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.363+0000 7f775a82a700 1 Processor -- start 2023-10-10T20:19:07.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.363+0000 7f775a82a700 1 -- start start 2023-10-10T20:19:07.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f775a82a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77540ca0b0 0x7f7754159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.368 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f775a82a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f775a82a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7754156c10 con 0x7f77540ca480 2023-10-10T20:19:07.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f775a82a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77541583f0 con 0x7f77540ca0b0 2023-10-10T20:19:07.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f775a82a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775415aa10 con 0x7f7754069d70 2023-10-10T20:19:07.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f7758dc7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.369 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f7758dc7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:58398/0 (socket says 172.21.15.161:58398) 2023-10-10T20:19:07.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.364+0000 7f7758dc7700 1 -- 172.21.15.161:0/1211949389 learned_addr learned my addr 172.21.15.161:0/1211949389 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7753fff700 1 --2- 172.21.15.161:0/1211949389 >> v2:172.21.15.148:3300/0 conn(0x7f77540ca0b0 0x7f7754159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7758dc7700 1 -- 172.21.15.161:0/1211949389 >> v2:172.21.15.148:3300/0 conn(0x7f77540ca0b0 msgr2=0x7f7754159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:07.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7758dc7700 1 --2- 172.21.15.161:0/1211949389 >> v2:172.21.15.148:3300/0 conn(0x7f77540ca0b0 0x7f7754159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.370 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7758dc7700 1 -- 172.21.15.161:0/1211949389 >> v1:172.21.15.148:6789/0 conn(0x7f77540ca480 legacy=0x7f7754069660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7758dc7700 1 -- 172.21.15.161:0/1211949389 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775415acf0 con 0x7f7754069d70 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7758dc7700 1 --2- 172.21.15.161:0/1211949389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c0d0 tx=0x7f774c00a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80df9737ab39395 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7752ffd700 1 -- 172.21.15.161:0/1211949389 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00f070 con 0x7f7754069d70 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7752ffd700 1 -- 172.21.15.161:0/1211949389 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00a2e0 con 0x7f7754069d70 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.365+0000 7f7752ffd700 1 -- 172.21.15.161:0/1211949389 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cd80 con 0x7f7754069d70 2023-10-10T20:19:07.371 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 -- 172.21.15.161:0/1211949389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 msgr2=0x7f775415a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 --2- 172.21.15.161:0/1211949389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c0d0 tx=0x7f774c00a950 comp rx=0 tx=0).stop 2023-10-10T20:19:07.372 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 -- 172.21.15.161:0/1211949389 shutdown_connections 2023-10-10T20:19:07.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 --2- 172.21.15.161:0/1211949389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7754069d70 0x7f775415a3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 --2- 172.21.15.161:0/1211949389 >> v2:172.21.15.148:3300/0 conn(0x7f77540ca0b0 0x7f7754159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 -- 172.21.15.161:0/1211949389 >> 172.21.15.161:0/1211949389 conn(0x7f7754067660 msgr2=0x7f7754067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 -- 172.21.15.161:0/1211949389 shutdown_connections 2023-10-10T20:19:07.373 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.366+0000 7f775a82a700 1 -- 172.21.15.161:0/1211949389 wait complete. 2023-10-10T20:19:07.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.367+0000 7f775a82a700 1 Processor -- start 2023-10-10T20:19:07.374 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.367+0000 7f775a82a700 1 -- start start 2023-10-10T20:19:07.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.376 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77540ca0b0 0x7f77540c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77540c9450 con 0x7f77540ca0b0 2023-10-10T20:19:07.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775416a870 con 0x7f7754069d70 2023-10-10T20:19:07.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7754097cc0 con 0x7f77540ca480 2023-10-10T20:19:07.377 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7753fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:58404/0 (socket says 172.21.15.161:58404) 2023-10-10T20:19:07.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7753fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53952/0 (socket says 172.21.15.161:53952) 2023-10-10T20:19:07.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 -- 172.21.15.161:0/563917868 learned_addr learned my addr 172.21.15.161:0/563917868 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.378 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f77537fe700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77540ca0b0 0x7f77540c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 -- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 msgr2=0x7f775409a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 -- 172.21.15.161:0/563917868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77540ca0b0 msgr2=0x7f77540c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77540ca0b0 0x7f77540c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.379 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 -- 172.21.15.161:0/563917868 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c00abf0 con 0x7f77540ca480 2023-10-10T20:19:07.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7753fff700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:07.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f7758dc7700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f774c013000 tx=0x7f774c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0082a0 con 0x7f77540ca480 2023-10-10T20:19:07.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c009df0 con 0x7f77540ca480 2023-10-10T20:19:07.380 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0266d0 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.368+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77540c9170 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.369+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775416a500 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.369+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f774c003040 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.369+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f774c026d50 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.369+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775416a080 con 0x7f77540ca480 2023-10-10T20:19:07.381 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.373+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f775416a080 con 0x7f77540ca480 2023-10-10T20:19:07.443 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.442+0000 7fbb4940d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-10T20:19:07.456 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.455+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f775416a9e0 con 0x7f77540ca480 2023-10-10T20:19:07.460 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.459+0000 7f773bfff700 1 -- 172.21.15.161:0/563917868 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f774c0380b0 con 0x7f77540ca480 2023-10-10T20:19:07.460 INFO:teuthology.orchestra.run.smithi161.stdout:4 2023-10-10T20:19:07.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 msgr2=0x7f775414ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.461 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f774c013000 tx=0x7f774c019000 comp rx=0 tx=0).stop 2023-10-10T20:19:07.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 shutdown_connections 2023-10-10T20:19:07.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77540ca480 0x7f775414ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7754069d70 0x7f775409a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 --2- 172.21.15.161:0/563917868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77540ca0b0 0x7f77540c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.462 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 >> 172.21.15.161:0/563917868 conn(0x7f7754067660 msgr2=0x7f77540caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 shutdown_connections 2023-10-10T20:19:07.463 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.460+0000 7f775a82a700 1 -- 172.21.15.161:0/563917868 wait complete. 2023-10-10T20:19:07.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-10-10T20:19:07.477 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.475+0000 7f7e6e0cd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-10T20:19:07.525 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.524+0000 7fbb4940d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-10T20:19:07.554 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.552+0000 7f7e6e0cd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-10T20:19:07.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.592+0000 7f346881b700 1 Processor -- start 2023-10-10T20:19:07.594 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.592+0000 7f346881b700 1 -- start start 2023-10-10T20:19:07.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.593+0000 7f346881b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f34640caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.596 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.593+0000 7f346881b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f346415a730 0x7f34640c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346881b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346415abc0 con 0x7f34640abce0 2023-10-10T20:19:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346881b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640c9100 con 0x7f346415a730 2023-10-10T20:19:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346881b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3464158050 con 0x7f34640ab910 2023-10-10T20:19:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f346415a730 0x7f34640c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3462d9d700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f34640abce0 0x7f34640cb970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34718/0 (socket says 172.21.15.161:34718) 2023-10-10T20:19:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3462d9d700 1 -- 172.21.15.161:0/3488342164 learned_addr learned my addr 172.21.15.161:0/3488342164 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-10-10T20:19:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346259c700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f34640caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 -- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 msgr2=0x7f34640caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f34640caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 -- 172.21.15.161:0/3488342164 >> v1:172.21.15.161:6789/0 conn(0x7f34640abce0 legacy=0x7f34640cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 -- 172.21.15.161:0/3488342164 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346415aea0 con 0x7f346415a730 2023-10-10T20:19:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346159a700 1 -- 172.21.15.161:0/3488342164 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657509780 0 0) 0x7f346415abc0 con 0x7f34640abce0 2023-10-10T20:19:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f3461d9b700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f346415a730 0x7f34640c2d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3454009910 tx=0x7f34540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bbfdd88d3d3d4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346159a700 1 -- 172.21.15.161:0/3488342164 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400b640 con 0x7f346415a730 2023-10-10T20:19:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346159a700 1 -- 172.21.15.161:0/3488342164 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345400b7a0 con 0x7f346415a730 2023-10-10T20:19:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.594+0000 7f346159a700 1 -- 172.21.15.161:0/3488342164 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400bb60 con 0x7f346415a730 2023-10-10T20:19:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 -- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f346415a730 msgr2=0x7f34640c2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f346415a730 0x7f34640c2d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3454009910 tx=0x7f34540091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 -- 172.21.15.161:0/3488342164 shutdown_connections 2023-10-10T20:19:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f34640caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 --2- 172.21.15.161:0/3488342164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f346415a730 0x7f34640c2d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 -- 172.21.15.161:0/3488342164 >> 172.21.15.161:0/3488342164 conn(0x7f3464067660 msgr2=0x7f3464067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 -- 172.21.15.161:0/3488342164 shutdown_connections 2023-10-10T20:19:07.601 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.595+0000 7f346881b700 1 -- 172.21.15.161:0/3488342164 wait complete. 2023-10-10T20:19:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.596+0000 7f346881b700 1 Processor -- start 2023-10-10T20:19:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.596+0000 7f346881b700 1 -- start start 2023-10-10T20:19:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34640abce0 0x7f34640af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346415a730 0x7f34640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640b0250 con 0x7f346415a730 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640b0530 con 0x7f34640abce0 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346881b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34640b0810 con 0x7f34640ab910 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.605 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f3461d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34640abce0 0x7f34640af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:58422/0 (socket says 172.21.15.161:58422) 2023-10-10T20:19:07.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 -- 172.21.15.161:0/1430363096 learned_addr learned my addr 172.21.15.161:0/1430363096 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f3462d9d700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346415a730 0x7f34640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.606 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 -- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34640abce0 msgr2=0x7f34640af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34640abce0 0x7f34640af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 -- 172.21.15.161:0/1430363096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346415a730 msgr2=0x7f34640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346415a730 0x7f34640ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 -- 172.21.15.161:0/1430363096 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454009420 con 0x7f34640ab910 2023-10-10T20:19:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f346259c700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f345800e930 tx=0x7f34580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.597+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458013680 con 0x7f34640ab910 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34580137e0 con 0x7f34640ab910 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34640b0af0 con 0x7f34640ab910 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458013b80 con 0x7f34640ab910 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640c4130 con 0x7f34640ab910 2023-10-10T20:19:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3458005c80 con 0x7f34640ab910 2023-10-10T20:19:07.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f34580065c0 con 0x7f34640ab910 2023-10-10T20:19:07.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.598+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346404c3c0 con 0x7f34640ab910 2023-10-10T20:19:07.609 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.600+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3458021060 con 0x7f34640ab910 2023-10-10T20:19:07.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.682+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f34640cad30 con 0x7f34640ab910 2023-10-10T20:19:07.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.686+0000 7f34527fc700 1 -- 172.21.15.161:0/1430363096 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3458017070 con 0x7f34640ab910 2023-10-10T20:19:07.687 INFO:teuthology.orchestra.run.smithi161.stdout:5 2023-10-10T20:19:07.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 msgr2=0x7f3464149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.688 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f345800e930 tx=0x7f34580091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 shutdown_connections 2023-10-10T20:19:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346415a730 0x7f34640ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34640ab910 0x7f3464149340 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 --2- 172.21.15.161:0/1430363096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34640abce0 0x7f34640af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.689 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 >> 172.21.15.161:0/1430363096 conn(0x7f3464067660 msgr2=0x7f3464154040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 shutdown_connections 2023-10-10T20:19:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.687+0000 7f346881b700 1 -- 172.21.15.161:0/1430363096 wait complete. 2023-10-10T20:19:07.690 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.688+0000 7f7e6e0cd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-10T20:19:07.701 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-10-10T20:19:07.721 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.720+0000 7fbb4940d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:19:07.786 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:07.784+0000 7f7e6e0cd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-10T20:19:07.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.818+0000 7fe3ff123700 1 Processor -- start 2023-10-10T20:19:07.819 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.818+0000 7fe3ff123700 1 -- start start 2023-10-10T20:19:07.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3ff123700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3f8159ec0 0x7fe3f80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.822 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3ff123700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3f815a290 0x7fe3f80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3ff123700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80b0790 con 0x7fe3f806a5a0 2023-10-10T20:19:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3ff123700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80ce1f0 con 0x7fe3f8159ec0 2023-10-10T20:19:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3ff123700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b0b20 con 0x7fe3f815a290 2023-10-10T20:19:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3f815a290 0x7fe3f80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3f815a290 0x7fe3f80afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53966/0 (socket says 172.21.15.161:53966) 2023-10-10T20:19:07.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 -- 172.21.15.161:0/3430790203 learned_addr learned my addr 172.21.15.161:0/3430790203 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3fcebf700 1 --2- 172.21.15.161:0/3430790203 >> v2:172.21.15.148:3300/0 conn(0x7fe3f8159ec0 0x7fe3f80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 -- 172.21.15.161:0/3430790203 >> v2:172.21.15.148:3300/0 conn(0x7fe3f8159ec0 msgr2=0x7fe3f80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:07.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 --2- 172.21.15.161:0/3430790203 >> v2:172.21.15.148:3300/0 conn(0x7fe3f8159ec0 0x7fe3f80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 -- 172.21.15.161:0/3430790203 >> v1:172.21.15.148:6789/0 conn(0x7fe3f806a5a0 legacy=0x7fe3f815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 -- 172.21.15.161:0/3430790203 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80b04b0 con 0x7fe3f815a290 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f7fff700 1 --2- 172.21.15.161:0/3430790203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f815a290 0x7fe3f80afe80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8089910 tx=0x7fe3e4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972d9848081d436d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.819+0000 7fe3f77fe700 1 -- 172.21.15.161:0/3430790203 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4009e70 con 0x7fe3f815a290 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3f77fe700 1 -- 172.21.15.161:0/3430790203 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400b860 con 0x7fe3f815a290 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3f77fe700 1 -- 172.21.15.161:0/3430790203 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400bc00 con 0x7fe3f815a290 2023-10-10T20:19:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 -- 172.21.15.161:0/3430790203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f815a290 msgr2=0x7fe3f80afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 --2- 172.21.15.161:0/3430790203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f815a290 0x7fe3f80afe80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8089910 tx=0x7fe3e4009250 comp rx=0 tx=0).stop 2023-10-10T20:19:07.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 -- 172.21.15.161:0/3430790203 shutdown_connections 2023-10-10T20:19:07.826 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 --2- 172.21.15.161:0/3430790203 >> v2:172.21.15.148:3300/0 conn(0x7fe3f8159ec0 0x7fe3f80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 --2- 172.21.15.161:0/3430790203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f815a290 0x7fe3f80afe80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 -- 172.21.15.161:0/3430790203 >> 172.21.15.161:0/3430790203 conn(0x7fe3f80d1880 msgr2=0x7fe3f80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 -- 172.21.15.161:0/3430790203 shutdown_connections 2023-10-10T20:19:07.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.820+0000 7fe3ff123700 1 -- 172.21.15.161:0/3430790203 wait complete. 2023-10-10T20:19:07.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.821+0000 7fe3ff123700 1 Processor -- start 2023-10-10T20:19:07.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.821+0000 7fe3ff123700 1 -- start start 2023-10-10T20:19:07.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.830 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 0x7fe3f80c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 0x7fe3f80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:07.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c9e70 con 0x7fe3f8159ec0 2023-10-10T20:19:07.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815c750 con 0x7fe3f806a5a0 2023-10-10T20:19:07.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3ff123700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815ca30 con 0x7fe3f815a290 2023-10-10T20:19:07.831 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3fcebf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3fcebf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53974/0 (socket says 172.21.15.161:53974) 2023-10-10T20:19:07.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3fcebf700 1 -- 172.21.15.161:0/928016160 learned_addr learned my addr 172.21.15.161:0/928016160 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:07.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3fd6c0700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 0x7fe3f80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.822+0000 7fe3f7fff700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 0x7fe3f80c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:07.832 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 -- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 msgr2=0x7fe3f8159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 -- 172.21.15.161:0/928016160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 msgr2=0x7fe3f80c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 0x7fe3f80c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 -- 172.21.15.161:0/928016160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40094f0 con 0x7fe3f815a290 2023-10-10T20:19:07.833 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fcebf700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:07.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f7fff700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 0x7fe3f80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:07.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3fd6c0700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 0x7fe3f80c75f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c860 tx=0x7fe3f0001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:07.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0012630 con 0x7fe3f815a290 2023-10-10T20:19:07.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0012790 con 0x7fe3f815a290 2023-10-10T20:19:07.834 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f815cd10 con 0x7fe3f815a290 2023-10-10T20:19:07.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0012b50 con 0x7fe3f815a290 2023-10-10T20:19:07.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8159690 con 0x7fe3f815a290 2023-10-10T20:19:07.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe3f001b400 con 0x7fe3f815a290 2023-10-10T20:19:07.835 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.823+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fe3f0011980 con 0x7fe3f815a290 2023-10-10T20:19:07.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.824+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f804c3c0 con 0x7fe3f815a290 2023-10-10T20:19:07.836 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.826+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe3f0011c60 con 0x7fe3f815a290 2023-10-10T20:19:07.904 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.903+0000 7fbb4940d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-10T20:19:07.910 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.909+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fe3f80ceab0 con 0x7fe3f815a290 2023-10-10T20:19:07.918 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.917+0000 7fe3f4ff9700 1 -- 172.21.15.161:0/928016160 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe3f0011e00 con 0x7fe3f815a290 2023-10-10T20:19:07.918 INFO:teuthology.orchestra.run.smithi161.stdout:6 2023-10-10T20:19:07.919 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 msgr2=0x7fe3f80c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:07.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 0x7fe3f80c75f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c860 tx=0x7fe3f0001690 comp rx=0 tx=0).stop 2023-10-10T20:19:07.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 shutdown_connections 2023-10-10T20:19:07.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f815a290 0x7fe3f80c75f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.920 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3f806a5a0 0x7fe3f8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 --2- 172.21.15.161:0/928016160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3f8159ec0 0x7fe3f80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 >> 172.21.15.161:0/928016160 conn(0x7fe3f80d1880 msgr2=0x7fe3f80cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 shutdown_connections 2023-10-10T20:19:07.921 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:07.918+0000 7fe3ff123700 1 -- 172.21.15.161:0/928016160 wait complete. 2023-10-10T20:19:07.931 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-10-10T20:19:07.985 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:07.984+0000 7fbb4940d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-10T20:19:08.090 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.089+0000 7f8435031700 1 Processor -- start 2023-10-10T20:19:08.091 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.089+0000 7f8435031700 1 -- start start 2023-10-10T20:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8435031700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84300b17f0 0x7f843015e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8435031700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84300b0400 0x7f84300b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8435031700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843015a4e0 con 0x7f8430159d00 2023-10-10T20:19:08.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8435031700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300696e0 con 0x7f84300b0400 2023-10-10T20:19:08.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8435031700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300b1ef0 con 0x7f84300b17f0 2023-10-10T20:19:08.094 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84300b17f0 0x7f843015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84300b17f0 0x7f843015e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53988/0 (socket says 172.21.15.161:53988) 2023-10-10T20:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 -- 172.21.15.161:0/4228333777 learned_addr learned my addr 172.21.15.161:0/4228333777 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f8434830700 1 --2- 172.21.15.161:0/4228333777 >> v2:172.21.15.148:3300/0 conn(0x7f84300b0400 0x7f84300b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 -- 172.21.15.161:0/4228333777 >> v2:172.21.15.148:3300/0 conn(0x7f84300b0400 msgr2=0x7f84300b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 --2- 172.21.15.161:0/4228333777 >> v2:172.21.15.148:3300/0 conn(0x7f84300b0400 0x7f84300b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.090+0000 7f842f7fe700 1 -- 172.21.15.161:0/4228333777 >> v1:172.21.15.148:6789/0 conn(0x7f8430159d00 legacy=0x7f843015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f842f7fe700 1 -- 172.21.15.161:0/4228333777 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300b1c90 con 0x7f84300b17f0 2023-10-10T20:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f842f7fe700 1 --2- 172.21.15.161:0/4228333777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84300b17f0 0x7f843015e300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84300642e0 tx=0x7f8420009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d4efa5c8324ead server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f842effd700 1 -- 172.21.15.161:0/4228333777 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420009e70 con 0x7f84300b17f0 2023-10-10T20:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f842effd700 1 -- 172.21.15.161:0/4228333777 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842000b860 con 0x7f84300b17f0 2023-10-10T20:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f842effd700 1 -- 172.21.15.161:0/4228333777 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842000bc00 con 0x7f84300b17f0 2023-10-10T20:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 -- 172.21.15.161:0/4228333777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84300b17f0 msgr2=0x7f843015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 --2- 172.21.15.161:0/4228333777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84300b17f0 0x7f843015e300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84300642e0 tx=0x7f8420009250 comp rx=0 tx=0).stop 2023-10-10T20:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 -- 172.21.15.161:0/4228333777 shutdown_connections 2023-10-10T20:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 --2- 172.21.15.161:0/4228333777 >> v2:172.21.15.148:3300/0 conn(0x7f84300b0400 0x7f84300b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 --2- 172.21.15.161:0/4228333777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84300b17f0 0x7f843015e300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 -- 172.21.15.161:0/4228333777 >> 172.21.15.161:0/4228333777 conn(0x7f843006c700 msgr2=0x7f8430069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:08.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 -- 172.21.15.161:0/4228333777 shutdown_connections 2023-10-10T20:19:08.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.091+0000 7f8435031700 1 -- 172.21.15.161:0/4228333777 wait complete. 2023-10-10T20:19:08.098 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.092+0000 7f8435031700 1 Processor -- start 2023-10-10T20:19:08.099 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 -- start start 2023-10-10T20:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 0x7f8430141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84300b17f0 0x7f8430143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.102 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430153ad0 con 0x7f84300b17f0 2023-10-10T20:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84301453b0 con 0x7f8430159d00 2023-10-10T20:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.093+0000 7f8435031700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8430142340 con 0x7f84300b0400 2023-10-10T20:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8434830700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 0x7f8430141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8434830700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53998/0 (socket says 172.21.15.161:53998) 2023-10-10T20:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8434830700 1 -- 172.21.15.161:0/3401929659 learned_addr learned my addr 172.21.15.161:0/3401929659 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-10-10T20:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842f7fe700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84300b17f0 0x7f8430143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 -- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 msgr2=0x7f8430143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 -- 172.21.15.161:0/3401929659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84300b17f0 msgr2=0x7f8430143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84300b17f0 0x7f8430143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 -- 172.21.15.161:0/3401929659 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200094f0 con 0x7f84300b0400 2023-10-10T20:19:08.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8434830700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:08.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f842ffff700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 0x7f8430141e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f842400c490 tx=0x7f8424007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:08.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424011590 con 0x7f84300b0400 2023-10-10T20:19:08.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424011710 con 0x7f84300b0400 2023-10-10T20:19:08.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84301447a0 con 0x7f84300b0400 2023-10-10T20:19:08.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424011ad0 con 0x7f84300b0400 2023-10-10T20:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8430144d60 con 0x7f84300b0400 2023-10-10T20:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f842401c6c0 con 0x7f84300b0400 2023-10-10T20:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.094+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f8424000fe0 con 0x7f84300b0400 2023-10-10T20:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.095+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843006e200 con 0x7f84300b0400 2023-10-10T20:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.097+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8424017070 con 0x7f84300b0400 2023-10-10T20:19:08.129 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:08.128+0000 7fbb4940d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-10T20:19:08.182 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.181+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f843004bb80 con 0x7f84300b0400 2023-10-10T20:19:08.190 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.189+0000 7f8417fff700 1 -- 172.21.15.161:0/3401929659 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f842401cba0 con 0x7f84300b0400 2023-10-10T20:19:08.191 INFO:teuthology.orchestra.run.smithi161.stdout:7 2023-10-10T20:19:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 msgr2=0x7f8430141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 0x7f8430141e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f842400c490 tx=0x7f8424007ec0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 shutdown_connections 2023-10-10T20:19:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430159d00 0x7f8430143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84300b0400 0x7f8430141e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 --2- 172.21.15.161:0/3401929659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84300b17f0 0x7f8430143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 >> 172.21.15.161:0/3401929659 conn(0x7f843006c700 msgr2=0x7f84301547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:08.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 shutdown_connections 2023-10-10T20:19:08.193 INFO:teuthology.orchestra.run.smithi161.stderr:2023-10-10T20:19:08.190+0000 7f8435031700 1 -- 172.21.15.161:0/3401929659 wait complete. 2023-10-10T20:19:08.203 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-10T20:19:08.203 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-10T20:19:08.205 INFO:tasks.ceph.osd.0:Started 2023-10-10T20:19:08.205 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-10T20:19:08.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-10T20:19:08.208 INFO:tasks.ceph.osd.1:Started 2023-10-10T20:19:08.208 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-10T20:19:08.208 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-10T20:19:08.211 INFO:tasks.ceph.osd.2:Started 2023-10-10T20:19:08.212 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-10T20:19:08.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-10T20:19:08.214 INFO:tasks.ceph.osd.3:Started 2023-10-10T20:19:08.215 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-10T20:19:08.215 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-10T20:19:08.229 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:08.228+0000 7fbb4940d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-10T20:19:08.230 INFO:tasks.ceph.osd.4:Started 2023-10-10T20:19:08.231 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-10T20:19:08.231 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-10T20:19:08.233 INFO:tasks.ceph.osd.5:Started 2023-10-10T20:19:08.234 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-10T20:19:08.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-10T20:19:08.238 INFO:tasks.ceph.osd.6:Started 2023-10-10T20:19:08.238 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-10T20:19:08.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-10T20:19:08.242 INFO:tasks.ceph.osd.7:Started 2023-10-10T20:19:08.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:08.293 INFO:tasks.ceph.osd.0.smithi148.stderr:2023-10-10T20:19:08.291+0000 7f7ac8e1e540 -1 Falling back to public interface 2023-10-10T20:19:08.295 INFO:tasks.ceph.osd.2.smithi148.stderr:2023-10-10T20:19:08.293+0000 7f18e934b540 -1 Falling back to public interface 2023-10-10T20:19:08.306 INFO:tasks.ceph.osd.1.smithi148.stderr:2023-10-10T20:19:08.304+0000 7f2eea56f540 -1 Falling back to public interface 2023-10-10T20:19:08.307 INFO:tasks.ceph.osd.0.smithi148.stderr:2023-10-10T20:19:08.305+0000 7f7ac8e1e540 -1 osd.0 0 log_to_monitors true 2023-10-10T20:19:08.313 INFO:tasks.ceph.osd.2.smithi148.stderr:2023-10-10T20:19:08.311+0000 7f18e934b540 -1 osd.2 0 log_to_monitors true 2023-10-10T20:19:08.317 INFO:tasks.ceph.osd.3.smithi148.stderr:2023-10-10T20:19:08.316+0000 7fbc39c79540 -1 Falling back to public interface 2023-10-10T20:19:08.321 INFO:tasks.ceph.osd.1.smithi148.stderr:2023-10-10T20:19:08.319+0000 7f2eea56f540 -1 osd.1 0 log_to_monitors true 2023-10-10T20:19:08.338 INFO:tasks.ceph.osd.3.smithi148.stderr:2023-10-10T20:19:08.337+0000 7fbc39c79540 -1 osd.3 0 log_to_monitors true 2023-10-10T20:19:08.351 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:08.349+0000 7f7e6e0cd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-10T20:19:08.354 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f858a225540 -1 Falling back to public interface 2023-10-10T20:19:08.354 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f4d1e22a540 -1 Falling back to public interface 2023-10-10T20:19:08.355 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f65392d1540 -1 Falling back to public interface 2023-10-10T20:19:08.355 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f6ffca58540 -1 Falling back to public interface 2023-10-10T20:19:08.355 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f858464e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-10-10T20:19:08.356 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f4d18653700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-10-10T20:19:08.356 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f65336fa700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-10-10T20:19:08.357 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f858464e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-10-10T20:19:08.357 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f65336fa700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-10-10T20:19:08.357 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f6ff6e81700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-10-10T20:19:08.358 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f65336fa700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2023-10-10T20:19:08.358 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f4d18653700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6803/0: (98) Address already in use 2023-10-10T20:19:08.359 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.353+0000 7f65336fa700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2023-10-10T20:19:08.369 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:08.368+0000 7f4d1e22a540 -1 osd.5 0 log_to_monitors true 2023-10-10T20:19:08.370 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:08.368+0000 7f65392d1540 -1 osd.6 0 log_to_monitors true 2023-10-10T20:19:08.370 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-10-10T20:19:08.369+0000 7f6ffca58540 -1 osd.7 0 log_to_monitors true 2023-10-10T20:19:08.371 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:08.370+0000 7f858a225540 -1 osd.4 0 log_to_monitors true 2023-10-10T20:19:08.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.389+0000 7fa2669a7700 1 Processor -- start 2023-10-10T20:19:08.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.389+0000 7fa2669a7700 1 -- start start 2023-10-10T20:19:08.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa2669a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2600afd90 0x7fa2600c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa2669a7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26006f1a0 0x7fa26015b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa2669a7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2600ca270 con 0x7fa2600b0160 2023-10-10T20:19:08.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa2669a7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600c87c0 con 0x7fa26006f1a0 2023-10-10T20:19:08.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa2669a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26015b770 con 0x7fa2600afd90 2023-10-10T20:19:08.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26006f1a0 0x7fa26015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa264f44700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa2600b0160 0x7fa260070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50854/0 (socket says 172.21.15.148:50854) 2023-10-10T20:19:08.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa264f44700 1 -- 172.21.15.148:0/3140923556 learned_addr learned my addr 172.21.15.148:0/3140923556 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:08.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25ffff700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2600afd90 0x7fa2600c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 -- 172.21.15.148:0/3140923556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2600afd90 msgr2=0x7fa2600c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:08.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2600afd90 0x7fa2600c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 -- 172.21.15.148:0/3140923556 >> v1:172.21.15.148:6789/0 conn(0x7fa2600b0160 legacy=0x7fa260070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 -- 172.21.15.148:0/3140923556 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26006e3e0 con 0x7fa26006f1a0 2023-10-10T20:19:08.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25effd700 1 -- 172.21.15.148:0/3140923556 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336923201 0 0) 0x7fa2600ca270 con 0x7fa2600b0160 2023-10-10T20:19:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25f7fe700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26006f1a0 0x7fa26015b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098e0 tx=0x7fa2540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1e30027d4eed95 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25effd700 1 -- 172.21.15.148:0/3140923556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400b640 con 0x7fa26006f1a0 2023-10-10T20:19:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25effd700 1 -- 172.21.15.148:0/3140923556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400b7a0 con 0x7fa26006f1a0 2023-10-10T20:19:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.390+0000 7fa25effd700 1 -- 172.21.15.148:0/3140923556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400bb40 con 0x7fa26006f1a0 2023-10-10T20:19:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 -- 172.21.15.148:0/3140923556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26006f1a0 msgr2=0x7fa26015b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26006f1a0 0x7fa26015b230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098e0 tx=0x7fa2540091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 -- 172.21.15.148:0/3140923556 shutdown_connections 2023-10-10T20:19:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2600afd90 0x7fa2600c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 --2- 172.21.15.148:0/3140923556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26006f1a0 0x7fa26015b230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 -- 172.21.15.148:0/3140923556 >> 172.21.15.148:0/3140923556 conn(0x7fa2600ca4f0 msgr2=0x7fa2600ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 -- 172.21.15.148:0/3140923556 shutdown_connections 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.391+0000 7fa2669a7700 1 -- 172.21.15.148:0/3140923556 wait complete. 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.392+0000 7fa2669a7700 1 Processor -- start 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.392+0000 7fa2669a7700 1 -- start start 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 0x7fa260154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2600b0160 0x7fa2600b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:08.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c32e0 con 0x7fa2600afd90 2023-10-10T20:19:08.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c35c0 con 0x7fa2600b0160 2023-10-10T20:19:08.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa2669a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600c38a0 con 0x7fa26006f1a0 2023-10-10T20:19:08.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54392/0 (socket says 172.21.15.148:54392) 2023-10-10T20:19:08.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 -- 172.21.15.148:0/2549642174 learned_addr learned my addr 172.21.15.148:0/2549642174 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:08.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa264f44700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2600b0160 0x7fa2600b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25ffff700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 0x7fa260154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 -- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 msgr2=0x7fa260154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 0x7fa260154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 -- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2600b0160 msgr2=0x7fa2600b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2600b0160 0x7fa2600b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 -- 172.21.15.148:0/2549642174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009420 con 0x7fa2600afd90 2023-10-10T20:19:08.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25ffff700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 0x7fa260154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:08.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.393+0000 7fa25f7fe700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa254013000 tx=0x7fa254017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:08.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254000d10 con 0x7fa2600afd90 2023-10-10T20:19:08.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400bb20 con 0x7fa2600afd90 2023-10-10T20:19:08.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2600c3b80 con 0x7fa2600afd90 2023-10-10T20:19:08.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011670 con 0x7fa2600afd90 2023-10-10T20:19:08.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2600cce90 con 0x7fa2600afd90 2023-10-10T20:19:08.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa254011ca0 con 0x7fa2600afd90 2023-10-10T20:19:08.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.394+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fa2540193b0 con 0x7fa2600afd90 2023-10-10T20:19:08.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.395+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26004c3c0 con 0x7fa2600afd90 2023-10-10T20:19:08.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.397+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa254029070 con 0x7fa2600afd90 2023-10-10T20:19:08.455 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:08.453+0000 7f7e6e0cd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-10T20:19:08.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.533+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2600c7ea0 con 0x7fa2600afd90 2023-10-10T20:19:08.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.534+0000 7fa247fff700 1 -- 172.21.15.148:0/2549642174 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fa254015020 con 0x7fa2600afd90 2023-10-10T20:19:08.537 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:08.537 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":10,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:08.182843+0000","last_up_change":"0.000000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:08.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 msgr2=0x7fa2600c2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:08.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa254013000 tx=0x7fa254017000 comp rx=0 tx=0).stop 2023-10-10T20:19:08.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 shutdown_connections 2023-10-10T20:19:08.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2600b0160 0x7fa2600b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa26006f1a0 0x7fa260154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 --2- 172.21.15.148:0/2549642174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2600afd90 0x7fa2600c2820 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:08.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 >> 172.21.15.148:0/2549642174 conn(0x7fa2600ca4f0 msgr2=0x7fa26015fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:08.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 shutdown_connections 2023-10-10T20:19:08.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:08.535+0000 7fa2669a7700 1 -- 172.21.15.148:0/2549642174 wait complete. 2023-10-10T20:19:08.549 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-10T20:19:08.549 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-10T20:19:08.565 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:08.563+0000 7f7e6e0cd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-10T20:19:09.145 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:09.143+0000 7f7e6e0cd180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-10T20:19:09.283 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:09.281+0000 7f7e6e0cd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:19:09.356 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:09.375 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:09.374+0000 7f7e6e0cd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-10T20:19:09.467 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.465+0000 7f0015d12700 1 Processor -- start 2023-10-10T20:19:09.468 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.465+0000 7f0015d12700 1 -- start start 2023-10-10T20:19:09.470 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0015d12700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015a620 0x7f001015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:09.470 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0015d12700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f001015d580 0x7f0010158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:09.470 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0015d12700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001015cbb0 con 0x7f0010157870 2023-10-10T20:19:09.470 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0015d12700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001015bdb0 con 0x7f001015d580 2023-10-10T20:19:09.470 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0015d12700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001015bfa0 con 0x7f001015a620 2023-10-10T20:19:09.471 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f000ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f001015d580 0x7f0010158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:09.471 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f000f7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0010157870 0x7f001015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50868/0 (socket says 172.21.15.148:50868) 2023-10-10T20:19:09.471 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f000f7fe700 1 -- 172.21.15.148:0/2483146335 learned_addr learned my addr 172.21.15.148:0/2483146335 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:09.471 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.466+0000 7f0007fff700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015a620 0x7f001015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:09.472 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000ffff700 1 -- 172.21.15.148:0/2483146335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015a620 msgr2=0x7f001015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:09.472 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000ffff700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015a620 0x7f001015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.472 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000ffff700 1 -- 172.21.15.148:0/2483146335 >> v1:172.21.15.148:6789/0 conn(0x7f0010157870 legacy=0x7f001015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:09.473 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000ffff700 1 -- 172.21.15.148:0/2483146335 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001015c280 con 0x7f001015d580 2023-10-10T20:19:09.473 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000ffff700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015d580 0x7f0010158110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e60 tx=0x7f0008009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fada37002eedda server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:09.473 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000effd700 1 -- 172.21.15.148:0/2483146335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800e070 con 0x7f001015d580 2023-10-10T20:19:09.473 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000effd700 1 -- 172.21.15.148:0/2483146335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800bc30 con 0x7f001015d580 2023-10-10T20:19:09.473 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.467+0000 7f000effd700 1 -- 172.21.15.148:0/2483146335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008012430 con 0x7f001015d580 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 -- 172.21.15.148:0/2483146335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015d580 msgr2=0x7f0010158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015d580 0x7f0010158110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e60 tx=0x7f0008009780 comp rx=0 tx=0).stop 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 -- 172.21.15.148:0/2483146335 shutdown_connections 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015d580 0x7f0010158110 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 --2- 172.21.15.148:0/2483146335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015a620 0x7f001015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.474 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 -- 172.21.15.148:0/2483146335 >> 172.21.15.148:0/2483146335 conn(0x7f0010068ce0 msgr2=0x7f00100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:09.475 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 -- 172.21.15.148:0/2483146335 shutdown_connections 2023-10-10T20:19:09.475 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.468+0000 7f0015d12700 1 -- 172.21.15.148:0/2483146335 wait complete. 2023-10-10T20:19:09.478 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.469+0000 7f0015d12700 1 Processor -- start 2023-10-10T20:19:09.478 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.469+0000 7f0015d12700 1 -- start start 2023-10-10T20:19:09.478 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0010157870 0x7f001016af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:09.479 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:09.479 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015d580 0x7f001014c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:09.479 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0010153750 con 0x7f001015a620 2023-10-10T20:19:09.479 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001016b4d0 con 0x7f0010157870 2023-10-10T20:19:09.479 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0010152b40 con 0x7f001015d580 2023-10-10T20:19:09.480 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:09.480 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54404/0 (socket says 172.21.15.148:54404) 2023-10-10T20:19:09.480 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 -- 172.21.15.148:0/2107667124 learned_addr learned my addr 172.21.15.148:0/2107667124 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:09.480 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f000ffff700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015d580 0x7f001014c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:09.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f000f7fe700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0010157870 0x7f001016af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:09.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 -- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015d580 msgr2=0x7f001014c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:09.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015d580 0x7f001014c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 -- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0010157870 msgr2=0x7f001016af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:09.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0010157870 0x7f001016af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.482 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 -- 172.21.15.148:0/2107667124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0008009a20 con 0x7f001015a620 2023-10-10T20:19:09.482 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0007fff700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efff800d860 tx=0x7efff8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:09.482 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff80125f0 con 0x7f001015a620 2023-10-10T20:19:09.482 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff8012750 con 0x7f001015a620 2023-10-10T20:19:09.483 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0010152e20 con 0x7f001015a620 2023-10-10T20:19:09.483 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8012af0 con 0x7f001015a620 2023-10-10T20:19:09.483 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.470+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00101533e0 con 0x7f001015a620 2023-10-10T20:19:09.483 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.471+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7efff8013480 con 0x7f001015a620 2023-10-10T20:19:09.483 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.471+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1879+0+0 (secure 0 0 0) 0x7efff801dc30 con 0x7f001015a620 2023-10-10T20:19:09.484 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.471+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001004c3c0 con 0x7f001015a620 2023-10-10T20:19:09.484 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.473+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7efff80138c0 con 0x7f001015a620 2023-10-10T20:19:09.545 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.543+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00101538c0 con 0x7f001015a620 2023-10-10T20:19:09.546 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.544+0000 7f00077fe700 1 -- 172.21.15.148:0/2107667124 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7013 (secure 0 0 0) 0x7efff802c090 con 0x7f001015a620 2023-10-10T20:19:09.546 INFO:teuthology.misc.health.smithi148.stdout: 2023-10-10T20:19:09.546 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":11,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:09.186055+0000","last_up_change":"0.000000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:09.547 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 msgr2=0x7f0010069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:09.548 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efff800d860 tx=0x7efff8008040 comp rx=0 tx=0).stop 2023-10-10T20:19:09.548 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 shutdown_connections 2023-10-10T20:19:09.548 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f001015d580 0x7f001014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.548 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0010157870 0x7f001016af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.549 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 --2- 172.21.15.148:0/2107667124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f001015a620 0x7f0010069300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:09.549 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 >> 172.21.15.148:0/2107667124 conn(0x7f0010068ce0 msgr2=0x7f0010063380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:09.549 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 shutdown_connections 2023-10-10T20:19:09.549 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:09.545+0000 7f0015d12700 1 -- 172.21.15.148:0/2107667124 wait complete. 2023-10-10T20:19:09.557 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-10T20:19:09.634 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:09.632+0000 7f7e6e0cd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-10T20:19:09.711 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:09.709+0000 7f7e6e0cd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-10T20:19:09.821 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:09.820+0000 7fbb4940d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-10T20:19:09.898 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:09.897+0000 7fbb4940d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-10T20:19:09.975 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:09.974+0000 7fbb4940d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-10T20:19:10.205 INFO:tasks.ceph.osd.0.smithi148.stderr:2023-10-10T20:19:10.204+0000 7f7ac0241700 -1 osd.0 0 waiting for initial osdmap 2023-10-10T20:19:10.206 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-10-10T20:19:10.205+0000 7f6ff3e7b700 -1 osd.7 0 waiting for initial osdmap 2023-10-10T20:19:10.207 INFO:tasks.ceph.osd.0.smithi148.stderr:2023-10-10T20:19:10.205+0000 7f7aba03a700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.207 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-10-10T20:19:10.207+0000 7f6fedc74700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.213 INFO:tasks.ceph.osd.2.smithi148.stderr:2023-10-10T20:19:10.211+0000 7f18e076e700 -1 osd.2 0 waiting for initial osdmap 2023-10-10T20:19:10.213 INFO:tasks.ceph.osd.1.smithi148.stderr:2023-10-10T20:19:10.211+0000 7f2ee1992700 -1 osd.1 0 waiting for initial osdmap 2023-10-10T20:19:10.214 INFO:tasks.ceph.osd.3.smithi148.stderr:2023-10-10T20:19:10.211+0000 7fbc3189d700 -1 osd.3 0 waiting for initial osdmap 2023-10-10T20:19:10.214 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:10.212+0000 7f65306f4700 -1 osd.6 0 waiting for initial osdmap 2023-10-10T20:19:10.215 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:10.213+0000 7f4d1564d700 -1 osd.5 0 waiting for initial osdmap 2023-10-10T20:19:10.215 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:10.213+0000 7f8581e49700 -1 osd.4 0 waiting for initial osdmap 2023-10-10T20:19:10.216 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-10-10T20:19:10.214+0000 7f652a4ed700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.216 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-10-10T20:19:10.214+0000 7f4d0f446700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.216 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-10-10T20:19:10.214+0000 7f857b441700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.217 INFO:tasks.ceph.osd.1.smithi148.stderr:2023-10-10T20:19:10.213+0000 7f2edb78b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.217 INFO:tasks.ceph.osd.2.smithi148.stderr:2023-10-10T20:19:10.213+0000 7f18da567700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.218 INFO:tasks.ceph.osd.3.smithi148.stderr:2023-10-10T20:19:10.213+0000 7fbc2ae95700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-10T20:19:10.556 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:10.554+0000 7f7e6e0cd180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-10T20:19:10.571 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:10.570+0000 7fbb4940d180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-10T20:19:10.629 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:10.627+0000 7f7e6e0cd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-10T20:19:10.702 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:10.700+0000 7f7e6e0cd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-10T20:19:10.709 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:10.707+0000 7fbb4940d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-10T20:19:10.801 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:10.800+0000 7fbb4940d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-10T20:19:10.834 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:10.832+0000 7f7e6e0cd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-10T20:19:10.918 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:10.916+0000 7f7e6e0cd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-10T20:19:11.049 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:11.048+0000 7fbb4940d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-10T20:19:11.068 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:11.067+0000 7f7e6e0cd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-10T20:19:11.125 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:11.124+0000 7fbb4940d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-10T20:19:11.146 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:11.144+0000 7f7e6e0cd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-10T20:19:11.224 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:11.222+0000 7f7e6e0cd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-10T20:19:11.300 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:11.298+0000 7f7e6e0cd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-10T20:19:11.964 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:11.963+0000 7fbb4940d180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-10T20:19:12.037 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.036+0000 7fbb4940d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-10T20:19:12.112 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.111+0000 7fbb4940d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-10T20:19:12.247 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.246+0000 7fbb4940d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-10T20:19:12.330 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:12.328+0000 7f7e274cc700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-10T20:19:12.330 INFO:tasks.ceph.mgr.y.smithi148.stderr:2023-10-10T20:19:12.328+0000 7f7e274cc700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-10T20:19:12.348 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.347+0000 7fbb4940d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-10T20:19:12.501 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.500+0000 7fbb4940d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-10T20:19:12.578 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.577+0000 7fbb4940d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-10T20:19:12.655 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.654+0000 7fbb4940d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-10T20:19:12.730 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-10-10T20:19:12.729+0000 7fbb4940d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-10T20:19:16.364 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:16.481 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.479+0000 7fac4f991700 1 Processor -- start 2023-10-10T20:19:16.482 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 -- start start 2023-10-10T20:19:16.484 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac480b0e50 0x7fac480c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.484 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac480b1220 0x7fac4806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.484 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4815de60 con 0x7fac4806dad0 2023-10-10T20:19:16.485 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4815e140 con 0x7fac480b0e50 2023-10-10T20:19:16.485 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.480+0000 7fac4f991700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4815e420 con 0x7fac480b1220 2023-10-10T20:19:16.485 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4df2e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fac4806dad0 0x7fac480cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35660/0 (socket says 172.21.15.148:35660) 2023-10-10T20:19:16.485 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4df2e700 1 -- 172.21.15.148:0/1793580040 learned_addr learned my addr 172.21.15.148:0/1793580040 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:16.486 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 --2- 172.21.15.148:0/1793580040 >> v2:172.21.15.161:3300/0 conn(0x7fac480b0e50 0x7fac480c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.486 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4cf2c700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac480b1220 0x7fac4806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.486 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac3ffff700 1 -- 172.21.15.148:0/1793580040 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830894327 0 0) 0x7fac4815de60 con 0x7fac4806dad0 2023-10-10T20:19:16.486 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac3ffff700 1 -- 172.21.15.148:0/1793580040 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac34003570 con 0x7fac4806dad0 2023-10-10T20:19:16.486 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 -- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac480b1220 msgr2=0x7fac4806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:16.487 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac480b1220 0x7fac4806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.487 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 -- 172.21.15.148:0/1793580040 >> v1:172.21.15.148:6789/0 conn(0x7fac4806dad0 legacy=0x7fac480cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.487 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 -- 172.21.15.148:0/1793580040 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4815e700 con 0x7fac480b0e50 2023-10-10T20:19:16.487 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac4d72d700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac480c9180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac44009910 tx=0x7fac440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a152302b306aa018 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.487 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac3ffff700 1 -- 172.21.15.148:0/1793580040 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44011630 con 0x7fac480b0e50 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac3ffff700 1 -- 172.21.15.148:0/1793580040 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44011790 con 0x7fac480b0e50 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.481+0000 7fac3ffff700 1 -- 172.21.15.148:0/1793580040 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44011b30 con 0x7fac480b0e50 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 -- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 msgr2=0x7fac480c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac480c9180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac44009910 tx=0x7fac440091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 -- 172.21.15.148:0/1793580040 shutdown_connections 2023-10-10T20:19:16.488 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac480c9180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.489 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 --2- 172.21.15.148:0/1793580040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac480b1220 0x7fac4806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.489 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 -- 172.21.15.148:0/1793580040 >> 172.21.15.148:0/1793580040 conn(0x7fac48069870 msgr2=0x7fac48069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:16.489 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 -- 172.21.15.148:0/1793580040 shutdown_connections 2023-10-10T20:19:16.489 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.482+0000 7fac4f991700 1 -- 172.21.15.148:0/1793580040 wait complete. 2023-10-10T20:19:16.492 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.483+0000 7fac4f991700 1 Processor -- start 2023-10-10T20:19:16.492 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.483+0000 7fac4f991700 1 -- start start 2023-10-10T20:19:16.493 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac4806dad0 0x7fac480b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.493 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac48153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.493 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.493 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480cb1e0 con 0x7fac480b1220 2023-10-10T20:19:16.493 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48155090 con 0x7fac480b0e50 2023-10-10T20:19:16.494 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4f991700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac480b1e20 con 0x7fac4806dad0 2023-10-10T20:19:16.494 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.494 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48218/0 (socket says 172.21.15.148:48218) 2023-10-10T20:19:16.494 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 -- 172.21.15.148:0/3211711990 learned_addr learned my addr 172.21.15.148:0/3211711990 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:16.494 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4cf2c700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac48153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.495 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4d72d700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac4806dad0 0x7fac480b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.495 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 -- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac4806dad0 msgr2=0x7fac480b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.495 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac4806dad0 0x7fac480b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.495 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 -- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 msgr2=0x7fac48153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.495 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac48153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.496 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.484+0000 7fac4df2e700 1 -- 172.21.15.148:0/3211711990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009420 con 0x7fac480b1220 2023-10-10T20:19:16.496 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac4cf2c700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac48153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:16.496 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac4df2e700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac4000cc40 tx=0x7fac40001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.496 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4000a980 con 0x7fac480b1220 2023-10-10T20:19:16.496 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4000aae0 con 0x7fac480b1220 2023-10-10T20:19:16.497 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac48154760 con 0x7fac480b1220 2023-10-10T20:19:16.497 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4000ae60 con 0x7fac480b1220 2023-10-10T20:19:16.497 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.485+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac48154d20 con 0x7fac480b1220 2023-10-10T20:19:16.497 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.486+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac48155220 con 0x7fac480b1220 2023-10-10T20:19:16.497 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.486+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac40007300 con 0x7fac480b1220 2023-10-10T20:19:16.498 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.486+0000 7fac3d7fa700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.498 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.487+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fac400845c0 con 0x7fac480b1220 2023-10-10T20:19:16.498 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.487+0000 7fac4d72d700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.498 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.487+0000 7fac4d72d700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 0x7fac2c0693e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac440093a0 tx=0x7fac44017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.499 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.491+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac40051c00 con 0x7fac480b1220 2023-10-10T20:19:16.675 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.673+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac4804c3c0 con 0x7fac480b1220 2023-10-10T20:19:16.676 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.674+0000 7fac3d7fa700 1 -- 172.21.15.148:0/3211711990 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fac40055220 con 0x7fac480b1220 2023-10-10T20:19:16.676 INFO:teuthology.misc.health.smithi148.stdout: 2023-10-10T20:19:16.676 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":16,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:14.224957+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:16.679 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 msgr2=0x7fac2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.679 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 0x7fac2c0693e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac440093a0 tx=0x7fac44017000 comp rx=0 tx=0).stop 2023-10-10T20:19:16.679 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 msgr2=0x7fac480ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.679 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac4000cc40 tx=0x7fac40001a80 comp rx=0 tx=0).stop 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 shutdown_connections 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.676+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fac2c066f30 0x7fac2c0693e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac480b1220 0x7fac480ca6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac4806dad0 0x7fac480b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 --2- 172.21.15.148:0/3211711990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac480b0e50 0x7fac48153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.680 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 >> 172.21.15.148:0/3211711990 conn(0x7fac48069870 msgr2=0x7fac4806c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:16.681 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 shutdown_connections 2023-10-10T20:19:16.681 INFO:teuthology.misc.health.smithi148.stderr:2023-10-10T20:19:16.677+0000 7fac4f991700 1 -- 172.21.15.148:0/3211711990 wait complete. 2023-10-10T20:19:16.689 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-10T20:19:16.689 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-10T20:19:16.690 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:19:16.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:19:16.693 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:19:16.693 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:19:16.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:19:16.695 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:19:16.695 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:19:16.696 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:19:16.699 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:19:16.699 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:19:16.699 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:19:16.701 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:19:16.702 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-10T20:19:16.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:16.766 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:19:16.767 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:19:16.782 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:19:16.783 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:19:16.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.822+0000 7fd52fea9700 1 Processor -- start 2023-10-10T20:19:16.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 -- start start 2023-10-10T20:19:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 0x7fd5280c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd52806f1a0 0x7fd52815b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280ca270 con 0x7fd5280b0160 2023-10-10T20:19:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5280c87c0 con 0x7fd52806f1a0 2023-10-10T20:19:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52fea9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52815b770 con 0x7fd5280afd90 2023-10-10T20:19:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52d444700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd52806f1a0 0x7fd52815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52d444700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd52806f1a0 0x7fd52815b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48260/0 (socket says 172.21.15.148:48260) 2023-10-10T20:19:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.823+0000 7fd52d444700 1 -- 172.21.15.148:0/2516075289 learned_addr learned my addr 172.21.15.148:0/2516075289 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52dc45700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 0x7fd5280c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52d444700 1 -- 172.21.15.148:0/2516075289 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 msgr2=0x7fd5280c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:16.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52d444700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 0x7fd5280c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52d444700 1 -- 172.21.15.148:0/2516075289 >> v1:172.21.15.161:6789/0 conn(0x7fd5280b0160 legacy=0x7fd528070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52d444700 1 -- 172.21.15.148:0/2516075289 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52806e3e0 con 0x7fd52806f1a0 2023-10-10T20:19:16.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52dc45700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 0x7fd5280c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:16.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52d444700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd52806f1a0 0x7fd52815b230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5180099b0 tx=0x7fd518009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5910f06569eacfcc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52cc43700 1 -- 172.21.15.148:0/2516075289 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800b860 con 0x7fd52806f1a0 2023-10-10T20:19:16.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52cc43700 1 -- 172.21.15.148:0/2516075289 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51800b9c0 con 0x7fd52806f1a0 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.824+0000 7fd52cc43700 1 -- 172.21.15.148:0/2516075289 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800bd60 con 0x7fd52806f1a0 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 -- 172.21.15.148:0/2516075289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd52806f1a0 msgr2=0x7fd52815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd52806f1a0 0x7fd52815b230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd5180099b0 tx=0x7fd518009250 comp rx=0 tx=0).stop 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 -- 172.21.15.148:0/2516075289 shutdown_connections 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5280afd90 0x7fd5280c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 --2- 172.21.15.148:0/2516075289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd52806f1a0 0x7fd52815b230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 -- 172.21.15.148:0/2516075289 >> 172.21.15.148:0/2516075289 conn(0x7fd5280ca4f0 msgr2=0x7fd5280ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:16.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 -- 172.21.15.148:0/2516075289 shutdown_connections 2023-10-10T20:19:16.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.825+0000 7fd52fea9700 1 -- 172.21.15.148:0/2516075289 wait complete. 2023-10-10T20:19:16.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 Processor -- start 2023-10-10T20:19:16.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 -- start start 2023-10-10T20:19:16.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd52806f1a0 0x7fd528159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5280afd90 0x7fd5280b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280c28c0 con 0x7fd5280b0160 2023-10-10T20:19:16.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528159b70 con 0x7fd5280afd90 2023-10-10T20:19:16.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.826+0000 7fd52fea9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd528159e50 con 0x7fd52806f1a0 2023-10-10T20:19:16.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48268/0 (socket says 172.21.15.148:48268) 2023-10-10T20:19:16.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 -- 172.21.15.148:0/1934014539 learned_addr learned my addr 172.21.15.148:0/1934014539 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:16.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52dc45700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd52806f1a0 0x7fd528159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52d444700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5280afd90 0x7fd5280b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 -- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd52806f1a0 msgr2=0x7fd528159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd52806f1a0 0x7fd528159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 -- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5280afd90 msgr2=0x7fd5280b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5280afd90 0x7fd5280b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 -- 172.21.15.148:0/1934014539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180094f0 con 0x7fd5280b0160 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52e446700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c840 tx=0x7fd520001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520012630 con 0x7fd5280b0160 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5200127b0 con 0x7fd5280b0160 2023-10-10T20:19:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520012b70 con 0x7fd5280b0160 2023-10-10T20:19:16.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52815a130 con 0x7fd5280b0160 2023-10-10T20:19:16.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.827+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52815a410 con 0x7fd5280b0160 2023-10-10T20:19:16.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.828+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52815a600 con 0x7fd5280b0160 2023-10-10T20:19:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.828+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd52001a3a0 con 0x7fd5280b0160 2023-10-10T20:19:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.829+0000 7fd51dffb700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 0x7fd50c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.829+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd520084310 con 0x7fd5280b0160 2023-10-10T20:19:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.829+0000 7fd52dc45700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 0x7fd50c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.829+0000 7fd52dc45700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 0x7fd50c0693b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd524006f00 tx=0x7fd524008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:16.833+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5200518d0 con 0x7fd5280b0160 2023-10-10T20:19:17.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.017+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd52804c3c0 con 0x7fd5280b0160 2023-10-10T20:19:17.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.018+0000 7fd51dffb700 1 -- 172.21.15.148:0/1934014539 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fd520054ef0 con 0x7fd5280b0160 2023-10-10T20:19:17.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:17.020 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:14.224957+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:17.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 msgr2=0x7fd50c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 0x7fd50c0693b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd524006f00 tx=0x7fd524008040 comp rx=0 tx=0).stop 2023-10-10T20:19:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 msgr2=0x7fd528146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c840 tx=0x7fd520001670 comp rx=0 tx=0).stop 2023-10-10T20:19:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 shutdown_connections 2023-10-10T20:19:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd50c066f00 0x7fd50c0693b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5280b0160 0x7fd528146390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd52806f1a0 0x7fd528159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 --2- 172.21.15.148:0/1934014539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5280afd90 0x7fd5280b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 >> 172.21.15.148:0/1934014539 conn(0x7fd5280ca4f0 msgr2=0x7fd52815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 shutdown_connections 2023-10-10T20:19:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.020+0000 7fd52fea9700 1 -- 172.21.15.148:0/1934014539 wait complete. 2023-10-10T20:19:17.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:19:17.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.148+0000 7fa687c44700 1 Processor -- start 2023-10-10T20:19:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 -- start start 2023-10-10T20:19:17.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa68006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa68006aae0 0x7fa6800cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6800b0790 con 0x7fa680159ec0 2023-10-10T20:19:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68006af60 con 0x7fa68006aae0 2023-10-10T20:19:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.149+0000 7fa687c44700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800ce1f0 con 0x7fa68015a290 2023-10-10T20:19:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa68006aae0 0x7fa6800cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6859e0700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa680159ec0 0x7fa6800af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35680/0 (socket says 172.21.15.148:35680) 2023-10-10T20:19:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6859e0700 1 -- 172.21.15.148:0/2803138559 learned_addr learned my addr 172.21.15.148:0/2803138559 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 -- 172.21.15.148:0/2803138559 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 msgr2=0x7fa68006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6851df700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa68006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6849de700 1 -- 172.21.15.148:0/2803138559 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439871913 0 0) 0x7fa6800b0790 con 0x7fa680159ec0 2023-10-10T20:19:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa68006a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 -- 172.21.15.148:0/2803138559 >> v1:172.21.15.148:6789/0 conn(0x7fa680159ec0 legacy=0x7fa6800af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 -- 172.21.15.148:0/2803138559 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6800b0b20 con 0x7fa68006aae0 2023-10-10T20:19:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6861e1700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa68006aae0 0x7fa6800cf590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa678009e60 tx=0x7fa678009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57b604777f228b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6849de700 1 -- 172.21.15.148:0/2803138559 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800e070 con 0x7fa68006aae0 2023-10-10T20:19:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6849de700 1 -- 172.21.15.148:0/2803138559 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67800bc30 con 0x7fa68006aae0 2023-10-10T20:19:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.150+0000 7fa6849de700 1 -- 172.21.15.148:0/2803138559 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678012430 con 0x7fa68006aae0 2023-10-10T20:19:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 -- 172.21.15.148:0/2803138559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa68006aae0 msgr2=0x7fa6800cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa68006aae0 0x7fa6800cf590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa678009e60 tx=0x7fa678009780 comp rx=0 tx=0).stop 2023-10-10T20:19:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 -- 172.21.15.148:0/2803138559 shutdown_connections 2023-10-10T20:19:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa68006aae0 0x7fa6800cf590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 --2- 172.21.15.148:0/2803138559 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa68006a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 -- 172.21.15.148:0/2803138559 >> 172.21.15.148:0/2803138559 conn(0x7fa6800d1880 msgr2=0x7fa6800cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 -- 172.21.15.148:0/2803138559 shutdown_connections 2023-10-10T20:19:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.151+0000 7fa687c44700 1 -- 172.21.15.148:0/2803138559 wait complete. 2023-10-10T20:19:17.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.152+0000 7fa687c44700 1 Processor -- start 2023-10-10T20:19:17.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.152+0000 7fa687c44700 1 -- start start 2023-10-10T20:19:17.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa68006aae0 0x7fa68015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa6800b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015d000 con 0x7fa680159ec0 2023-10-10T20:19:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800b5ab0 con 0x7fa68006aae0 2023-10-10T20:19:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800b4ea0 con 0x7fa68015a290 2023-10-10T20:19:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48286/0 (socket says 172.21.15.148:48286) 2023-10-10T20:19:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 -- 172.21.15.148:0/1115200112 learned_addr learned my addr 172.21.15.148:0/1115200112 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6859e0700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa68006aae0 0x7fa68015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6861e1700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa6800b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 -- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 msgr2=0x7fa6800b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa6800b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 -- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa68006aae0 msgr2=0x7fa68015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa68006aae0 0x7fa68015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 -- 172.21.15.148:0/1115200112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009a20 con 0x7fa680159ec0 2023-10-10T20:19:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6861e1700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa6800b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa6851df700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c8b0 tx=0x7fa670008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011600 con 0x7fa680159ec0 2023-10-10T20:19:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670011760 con 0x7fa680159ec0 2023-10-10T20:19:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6800b5180 con 0x7fa680159ec0 2023-10-10T20:19:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011b00 con 0x7fa680159ec0 2023-10-10T20:19:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.153+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6800b5740 con 0x7fa680159ec0 2023-10-10T20:19:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.154+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6800b5c40 con 0x7fa680159ec0 2023-10-10T20:19:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.155+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa670021350 con 0x7fa680159ec0 2023-10-10T20:19:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.155+0000 7fa675ffb700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.155+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa670083f40 con 0x7fa680159ec0 2023-10-10T20:19:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.155+0000 7fa6859e0700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.156+0000 7fa6859e0700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 0x7fa6640693e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa68015d320 tx=0x7fa67c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.160+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa670054ef0 con 0x7fa680159ec0 2023-10-10T20:19:17.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.342+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa68004c3c0 con 0x7fa680159ec0 2023-10-10T20:19:17.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.342+0000 7fa675ffb700 1 -- 172.21.15.148:0/1115200112 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6700150a0 con 0x7fa680159ec0 2023-10-10T20:19:17.345 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 msgr2=0x7fa6640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 0x7fa6640693e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa68015d320 tx=0x7fa67c007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 msgr2=0x7fa6801422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c8b0 tx=0x7fa670008040 comp rx=0 tx=0).stop 2023-10-10T20:19:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 shutdown_connections 2023-10-10T20:19:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa68015a290 0x7fa6800b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa68006aae0 0x7fa68015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 --2- 172.21.15.148:0/1115200112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa680159ec0 0x7fa6801422e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 >> 172.21.15.148:0/1115200112 conn(0x7fa6800d1880 msgr2=0x7fa680156950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:17.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 shutdown_connections 2023-10-10T20:19:17.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.344+0000 7fa687c44700 1 -- 172.21.15.148:0/1115200112 wait complete. 2023-10-10T20:19:17.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.471+0000 7fcdb9f8d700 1 Processor -- start 2023-10-10T20:19:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.471+0000 7fcdb9f8d700 1 -- start start 2023-10-10T20:19:17.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.471+0000 7fcdb9f8d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcdb41523e0 0x7fcdb406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.471+0000 7fcdb9f8d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdb4158e50 0x7fcdb40cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb9f8d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb406c010 con 0x7fcdb4152010 2023-10-10T20:19:17.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb9f8d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb406bd10 con 0x7fcdb4158e50 2023-10-10T20:19:17.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb9f8d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb415c060 con 0x7fcdb41523e0 2023-10-10T20:19:17.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdb4158e50 0x7fcdb40cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcdb4152010 0x7fcdb415b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35688/0 (socket says 172.21.15.148:35688) 2023-10-10T20:19:17.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdb4158e50 0x7fcdb40cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48288/0 (socket says 172.21.15.148:48288) 2023-10-10T20:19:17.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb37fe700 1 -- 172.21.15.148:0/1506507568 learned_addr learned my addr 172.21.15.148:0/1506507568 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:17.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb2ffd700 1 --2- 172.21.15.148:0/1506507568 >> v2:172.21.15.161:3300/0 conn(0x7fcdb41523e0 0x7fcdb406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 -- 172.21.15.148:0/1506507568 >> v2:172.21.15.161:3300/0 conn(0x7fcdb41523e0 msgr2=0x7fcdb406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 --2- 172.21.15.148:0/1506507568 >> v2:172.21.15.161:3300/0 conn(0x7fcdb41523e0 0x7fcdb406c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 -- 172.21.15.148:0/1506507568 >> v1:172.21.15.148:6789/0 conn(0x7fcdb4152010 legacy=0x7fcdb415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 -- 172.21.15.148:0/1506507568 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb415c340 con 0x7fcdb4158e50 2023-10-10T20:19:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb27fc700 1 -- 172.21.15.148:0/1506507568 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640825562 0 0) 0x7fcdb406c010 con 0x7fcdb4152010 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb3fff700 1 --2- 172.21.15.148:0/1506507568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb40cdf30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40ceae0 tx=0x7fcdac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87377904791f99a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb27fc700 1 -- 172.21.15.148:0/1506507568 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00e070 con 0x7fcdb4158e50 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb27fc700 1 -- 172.21.15.148:0/1506507568 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00bc30 con 0x7fcdb4158e50 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.472+0000 7fcdb27fc700 1 -- 172.21.15.148:0/1506507568 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac012430 con 0x7fcdb4158e50 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/1506507568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 msgr2=0x7fcdb40cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/1506507568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb40cdf30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40ceae0 tx=0x7fcdac009780 comp rx=0 tx=0).stop 2023-10-10T20:19:17.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/1506507568 shutdown_connections 2023-10-10T20:19:17.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/1506507568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb40cdf30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/1506507568 >> v2:172.21.15.161:3300/0 conn(0x7fcdb41523e0 0x7fcdb406c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/1506507568 >> 172.21.15.148:0/1506507568 conn(0x7fcdb4069870 msgr2=0x7fcdb4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/1506507568 shutdown_connections 2023-10-10T20:19:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.473+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/1506507568 wait complete. 2023-10-10T20:19:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.474+0000 7fcdb9f8d700 1 Processor -- start 2023-10-10T20:19:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.474+0000 7fcdb9f8d700 1 -- start start 2023-10-10T20:19:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4152010 0x7fcdb4142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb41523e0 0x7fcdb414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4142b70 con 0x7fcdb4158e50 2023-10-10T20:19:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb414ea80 con 0x7fcdb4152010 2023-10-10T20:19:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb9f8d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb414d540 con 0x7fcdb41523e0 2023-10-10T20:19:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4152010 0x7fcdb4142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48298/0 (socket says 172.21.15.148:48298) 2023-10-10T20:19:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 -- 172.21.15.148:0/3810202930 learned_addr learned my addr 172.21.15.148:0/3810202930 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb2ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb41523e0 0x7fcdb414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 -- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb41523e0 msgr2=0x7fcdb414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb41523e0 0x7fcdb414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 -- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4152010 msgr2=0x7fcdb4142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4152010 0x7fcdb4142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 -- 172.21.15.148:0/3810202930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009a20 con 0x7fcdb4158e50 2023-10-10T20:19:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcdb3fff700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b6c0 tx=0x7fcdac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac027470 con 0x7fcdb4158e50 2023-10-10T20:19:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac0275d0 con 0x7fcdb4158e50 2023-10-10T20:19:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.475+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac027990 con 0x7fcdb4158e50 2023-10-10T20:19:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.476+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb414f270 con 0x7fcdb4158e50 2023-10-10T20:19:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.476+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb414ecb0 con 0x7fcdb4158e50 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.476+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb404c3c0 con 0x7fcdb4158e50 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.477+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdac024070 con 0x7fcdb4158e50 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.477+0000 7fcd9b7fe700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.477+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcdac086840 con 0x7fcdb4158e50 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.477+0000 7fcdb37fe700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.478+0000 7fcdb37fe700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 0x7fcda00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007160 tx=0x7fcda4007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:17.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.482+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdac053e00 con 0x7fcdb4158e50 2023-10-10T20:19:17.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.704+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdb4151ba0 con 0x7fcdb4158e50 2023-10-10T20:19:17.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.705+0000 7fcd9b7fe700 1 -- 172.21.15.148:0/3810202930 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fcdac02c020 con 0x7fcdb4158e50 2023-10-10T20:19:17.707 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:17.708 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-10T20:19:17.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 msgr2=0x7fcda00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 0x7fcda00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007160 tx=0x7fcda4007f50 comp rx=0 tx=0).stop 2023-10-10T20:19:17.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 msgr2=0x7fcdb4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:17.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b6c0 tx=0x7fcdac019000 comp rx=0 tx=0).stop 2023-10-10T20:19:17.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 shutdown_connections 2023-10-10T20:19:17.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb4158e50 0x7fcdb4068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4152010 0x7fcdb4142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 --2- 172.21.15.148:0/3810202930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb41523e0 0x7fcdb414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 >> 172.21.15.148:0/3810202930 conn(0x7fcdb4069870 msgr2=0x7fcdb414c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 shutdown_connections 2023-10-10T20:19:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:17.707+0000 7fcdb9f8d700 1 -- 172.21.15.148:0/3810202930 wait complete. 2023-10-10T20:19:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 2 2023-10-10T20:19:19.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.841+0000 7fea25220700 1 Processor -- start 2023-10-10T20:19:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.841+0000 7fea25220700 1 -- start start 2023-10-10T20:19:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea25220700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea25220700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea25220700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea20157c30 con 0x7fea2015a330 2023-10-10T20:19:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea25220700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea201584f0 con 0x7fea20157050 2023-10-10T20:19:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea25220700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea20157670 con 0x7fea20156570 2023-10-10T20:19:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea1f59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:55032/0 (socket says 172.21.15.148:55032) 2023-10-10T20:19:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1f59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:45420/0 (socket says 172.21.15.148:45420) 2023-10-10T20:19:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.842+0000 7fea1659c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fea2015a330 0x7fea20156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:37456/0 (socket says 172.21.15.148:37456) 2023-10-10T20:19:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 -- 172.21.15.148:0/550738305 learned_addr learned my addr 172.21.15.148:0/550738305 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 -- 172.21.15.148:0/550738305 >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 msgr2=0x7fea2006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 --2- 172.21.15.148:0/550738305 >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 -- 172.21.15.148:0/550738305 >> v1:172.21.15.161:6789/0 conn(0x7fea2015a330 legacy=0x7fea20156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 -- 172.21.15.148:0/550738305 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20157950 con 0x7fea20156570 2023-10-10T20:19:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1f59e700 1 --2- 172.21.15.148:0/550738305 >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1ed9d700 1 --2- 172.21.15.148:0/550738305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea1000bc50 tx=0x7fea1000b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf45b68e96c9a7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1e59c700 1 -- 172.21.15.148:0/550738305 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1000dbf0 con 0x7fea20156570 2023-10-10T20:19:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1e59c700 1 -- 172.21.15.148:0/550738305 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1000dd50 con 0x7fea20156570 2023-10-10T20:19:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.843+0000 7fea1e59c700 1 -- 172.21.15.148:0/550738305 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea100196d0 con 0x7fea20156570 2023-10-10T20:19:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 -- 172.21.15.148:0/550738305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 msgr2=0x7fea2006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 --2- 172.21.15.148:0/550738305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fea1000bc50 tx=0x7fea1000b4f0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 -- 172.21.15.148:0/550738305 shutdown_connections 2023-10-10T20:19:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 --2- 172.21.15.148:0/550738305 >> v2:172.21.15.161:3300/0 conn(0x7fea20157050 0x7fea2006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 --2- 172.21.15.148:0/550738305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea20156570 0x7fea2006d380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 -- 172.21.15.148:0/550738305 >> 172.21.15.148:0/550738305 conn(0x7fea20067660 msgr2=0x7fea20067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 -- 172.21.15.148:0/550738305 shutdown_connections 2023-10-10T20:19:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.844+0000 7fea25220700 1 -- 172.21.15.148:0/550738305 wait complete. 2023-10-10T20:19:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.845+0000 7fea25220700 1 Processor -- start 2023-10-10T20:19:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.845+0000 7fea25220700 1 -- start start 2023-10-10T20:19:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea20157050 0x7fea200c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea2015a330 0x7fea200caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c4e50 con 0x7fea20156570 2023-10-10T20:19:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c4240 con 0x7fea20157050 2023-10-10T20:19:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c4520 con 0x7fea2015a330 2023-10-10T20:19:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48302/0 (socket says 172.21.15.148:48302) 2023-10-10T20:19:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 -- 172.21.15.148:0/204670746 learned_addr learned my addr 172.21.15.148:0/204670746 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1f59e700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea2015a330 0x7fea200caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1659c700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea20157050 0x7fea200c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 -- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea2015a330 msgr2=0x7fea200caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea2015a330 0x7fea200caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 -- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea20157050 msgr2=0x7fea200c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea20157050 0x7fea200c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 -- 172.21.15.148:0/204670746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009a20 con 0x7fea20156570 2023-10-10T20:19:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea1ed9d700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea10006fd0 tx=0x7fea1001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1000dec0 con 0x7fea20156570 2023-10-10T20:19:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1001e050 con 0x7fea20156570 2023-10-10T20:19:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1000b790 con 0x7fea20156570 2023-10-10T20:19:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.846+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea100108d0 con 0x7fea20156570 2023-10-10T20:19:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.847+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea200c4ae0 con 0x7fea20156570 2023-10-10T20:19:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.847+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2004c3c0 con 0x7fea20156570 2023-10-10T20:19:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.848+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea10024470 con 0x7fea20156570 2023-10-10T20:19:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.848+0000 7fea177fe700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.848+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fea10014070 con 0x7fea20156570 2023-10-10T20:19:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.848+0000 7fea1659c700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.849+0000 7fea1659c700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 0x7fea000693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea200c2a60 tx=0x7fea08007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:19.853+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea10056010 con 0x7fea20156570 2023-10-10T20:19:20.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.077+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea200cacf0 con 0x7fea20156570 2023-10-10T20:19:20.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.078+0000 7fea177fe700 1 -- 172.21.15.148:0/204670746 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7fea10059630 con 0x7fea20156570 2023-10-10T20:19:20.079 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:20.079 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-10T20:19:20.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 msgr2=0x7fea000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 0x7fea000693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea200c2a60 tx=0x7fea08007e70 comp rx=0 tx=0).stop 2023-10-10T20:19:20.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 msgr2=0x7fea200ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea10006fd0 tx=0x7fea1001b000 comp rx=0 tx=0).stop 2023-10-10T20:19:20.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 shutdown_connections 2023-10-10T20:19:20.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fea00066f30 0x7fea000693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea2015a330 0x7fea200caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea20156570 0x7fea200ad9e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 --2- 172.21.15.148:0/204670746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea20157050 0x7fea200c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 >> 172.21.15.148:0/204670746 conn(0x7fea20067660 msgr2=0x7fea20069940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 shutdown_connections 2023-10-10T20:19:20.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.080+0000 7fea25220700 1 -- 172.21.15.148:0/204670746 wait complete. 2023-10-10T20:19:20.084 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 3 2023-10-10T20:19:20.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:20.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.206+0000 7fa054f32700 1 Processor -- start 2023-10-10T20:19:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.206+0000 7fa054f32700 1 -- start start 2023-10-10T20:19:20.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa054f32700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa050157870 0x7fa05015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa054f32700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa050157c40 0x7fa050158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa054f32700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05015cbb0 con 0x7fa05015a620 2023-10-10T20:19:20.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa054f32700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05015bfa0 con 0x7fa050157c40 2023-10-10T20:19:20.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa054f32700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05015c280 con 0x7fa050157870 2023-10-10T20:19:20.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa04ed9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa050157c40 0x7fa050158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa045d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa05015a620 0x7fa05015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35698/0 (socket says 172.21.15.148:35698) 2023-10-10T20:19:20.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa045d9b700 1 -- 172.21.15.148:0/1184333000 learned_addr learned my addr 172.21.15.148:0/1184333000 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:20.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa04e59c700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa050157870 0x7fa05015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa04ed9d700 1 -- 172.21.15.148:0/1184333000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa050157870 msgr2=0x7fa05015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:20.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa04ed9d700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa050157870 0x7fa05015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.207+0000 7fa04ed9d700 1 -- 172.21.15.148:0/1184333000 >> v1:172.21.15.148:6789/0 conn(0x7fa05015a620 legacy=0x7fa05015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04ed9d700 1 -- 172.21.15.148:0/1184333000 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05015c560 con 0x7fa050157c40 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04dd9b700 1 -- 172.21.15.148:0/1184333000 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512497567 0 0) 0x7fa05015cbb0 con 0x7fa05015a620 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04ed9d700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157c40 0x7fa050158110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa048009e60 tx=0x7fa048009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eed3c74d6908ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04dd9b700 1 -- 172.21.15.148:0/1184333000 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800e070 con 0x7fa050157c40 2023-10-10T20:19:20.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04dd9b700 1 -- 172.21.15.148:0/1184333000 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04800bb60 con 0x7fa050157c40 2023-10-10T20:19:20.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa04dd9b700 1 -- 172.21.15.148:0/1184333000 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048012430 con 0x7fa050157c40 2023-10-10T20:19:20.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 -- 172.21.15.148:0/1184333000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157c40 msgr2=0x7fa050158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157c40 0x7fa050158110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa048009e60 tx=0x7fa048009780 comp rx=0 tx=0).stop 2023-10-10T20:19:20.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 -- 172.21.15.148:0/1184333000 shutdown_connections 2023-10-10T20:19:20.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157c40 0x7fa050158110 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 --2- 172.21.15.148:0/1184333000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa050157870 0x7fa05015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 -- 172.21.15.148:0/1184333000 >> 172.21.15.148:0/1184333000 conn(0x7fa050068ce0 msgr2=0x7fa0500690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 -- 172.21.15.148:0/1184333000 shutdown_connections 2023-10-10T20:19:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.208+0000 7fa054f32700 1 -- 172.21.15.148:0/1184333000 wait complete. 2023-10-10T20:19:20.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 Processor -- start 2023-10-10T20:19:20.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 -- start start 2023-10-10T20:19:20.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa050157c40 0x7fa0501460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa05015a620 0x7fa05016d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050143590 con 0x7fa050157870 2023-10-10T20:19:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500c60f0 con 0x7fa050157c40 2023-10-10T20:19:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa054f32700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500c63d0 con 0x7fa05015a620 2023-10-10T20:19:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48308/0 (socket says 172.21.15.148:48308) 2023-10-10T20:19:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.210+0000 7fa04e59c700 1 -- 172.21.15.148:0/4268692627 learned_addr learned my addr 172.21.15.148:0/4268692627 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04ed9d700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa05015a620 0x7fa05016d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 -- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa05015a620 msgr2=0x7fa05016d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa05015a620 0x7fa05016d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 -- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa050157c40 msgr2=0x7fa0501460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa050157c40 0x7fa0501460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 -- 172.21.15.148:0/4268692627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009a20 con 0x7fa050157870 2023-10-10T20:19:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa04e59c700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa04000ec80 tx=0x7fa0400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000c9d0 con 0x7fa050157870 2023-10-10T20:19:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04000cb30 con 0x7fa050157870 2023-10-10T20:19:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0500c66b0 con 0x7fa050157870 2023-10-10T20:19:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040013470 con 0x7fa050157870 2023-10-10T20:19:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.211+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0500c6c70 con 0x7fa050157870 2023-10-10T20:19:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.212+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0500c6e90 con 0x7fa050157870 2023-10-10T20:19:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.212+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa040013820 con 0x7fa050157870 2023-10-10T20:19:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.213+0000 7fa046ffd700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 0x7fa0300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.213+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa040084570 con 0x7fa050157870 2023-10-10T20:19:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.213+0000 7fa045d9b700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 0x7fa0300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.213+0000 7fa045d9b700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 0x7fa0300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa05015b9a0 tx=0x7fa038007bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.217+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa040051b30 con 0x7fa050157870 2023-10-10T20:19:20.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.400+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa05004c3c0 con 0x7fa050157870 2023-10-10T20:19:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.401+0000 7fa046ffd700 1 -- 172.21.15.148:0/4268692627 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fa040055150 con 0x7fa050157870 2023-10-10T20:19:20.402 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:20.403 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:14.224957+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 msgr2=0x7fa0300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 0x7fa0300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa05015b9a0 tx=0x7fa038007bd0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 msgr2=0x7fa0500c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa04000ec80 tx=0x7fa0400091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 shutdown_connections 2023-10-10T20:19:20.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa030066f30 0x7fa0300693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa05015a620 0x7fa05016d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa050157870 0x7fa0500c5b10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 --2- 172.21.15.148:0/4268692627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa050157c40 0x7fa0501460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 >> 172.21.15.148:0/4268692627 conn(0x7fa050068ce0 msgr2=0x7fa0500cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 shutdown_connections 2023-10-10T20:19:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.403+0000 7fa054f32700 1 -- 172.21.15.148:0/4268692627 wait complete. 2023-10-10T20:19:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:19:20.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:20.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.535+0000 7f935df0a700 1 Processor -- start 2023-10-10T20:19:20.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.535+0000 7f935df0a700 1 -- start start 2023-10-10T20:19:20.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935df0a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93580cce30 0x7f935815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935df0a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935df0a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93580b4d70 con 0x7f935806a830 2023-10-10T20:19:20.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935df0a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935806ac70 con 0x7f93580cce30 2023-10-10T20:19:20.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935df0a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cd8d0 con 0x7f93580b6580 2023-10-10T20:19:20.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93580cce30 0x7f935815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f9357fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93580cce30 0x7f935815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:45440/0 (socket says 172.21.15.148:45440) 2023-10-10T20:19:20.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 -- 172.21.15.148:0/4139559626 learned_addr learned my addr 172.21.15.148:0/4139559626 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:20.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f9357fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:55068/0 (socket says 172.21.15.148:55068) 2023-10-10T20:19:20.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935d709700 1 --1- 172.21.15.148:0/4139559626 >> v1:172.21.15.161:6789/0 conn(0x7f935806a830 0x7f935815d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:37462/0 (socket says 172.21.15.148:37462) 2023-10-10T20:19:20.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 -- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 msgr2=0x7f935806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:20.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 -- 172.21.15.148:0/4139559626 >> v1:172.21.15.161:6789/0 conn(0x7f935806a830 legacy=0x7f935815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f935cf08700 1 -- 172.21.15.148:0/4139559626 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580b4440 con 0x7f93580cce30 2023-10-10T20:19:20.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.536+0000 7f9357fff700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:20.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935cf08700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93580cce30 0x7f935815aca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93500099b0 tx=0x7f9350009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2298fed44adedb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f93577fe700 1 -- 172.21.15.148:0/4139559626 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350009ea0 con 0x7f93580cce30 2023-10-10T20:19:20.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f93577fe700 1 -- 172.21.15.148:0/4139559626 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000b860 con 0x7f93580cce30 2023-10-10T20:19:20.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f93577fe700 1 -- 172.21.15.148:0/4139559626 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935000bc00 con 0x7f93580cce30 2023-10-10T20:19:20.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 -- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93580cce30 msgr2=0x7f935815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93580cce30 0x7f935815aca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93500099b0 tx=0x7f9350009250 comp rx=0 tx=0).stop 2023-10-10T20:19:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 -- 172.21.15.148:0/4139559626 shutdown_connections 2023-10-10T20:19:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93580cce30 0x7f935815aca0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 --2- 172.21.15.148:0/4139559626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580b6580 0x7f935806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 -- 172.21.15.148:0/4139559626 >> 172.21.15.148:0/4139559626 conn(0x7f9358068cb0 msgr2=0x7f93580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 -- 172.21.15.148:0/4139559626 shutdown_connections 2023-10-10T20:19:20.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.537+0000 7f935df0a700 1 -- 172.21.15.148:0/4139559626 wait complete. 2023-10-10T20:19:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.538+0000 7f935df0a700 1 Processor -- start 2023-10-10T20:19:20.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 -- start start 2023-10-10T20:19:20.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f935806a830 0x7f935814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580cce30 0x7f935816a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358152c70 con 0x7f93580b6580 2023-10-10T20:19:20.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9358153ab0 con 0x7f935806a830 2023-10-10T20:19:20.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f935df0a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9358152ea0 con 0x7f93580cce30 2023-10-10T20:19:20.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f9357fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f9357fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48312/0 (socket says 172.21.15.148:48312) 2023-10-10T20:19:20.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.539+0000 7f9357fff700 1 -- 172.21.15.148:0/1417433681 learned_addr learned my addr 172.21.15.148:0/1417433681 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:20.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f935d709700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580cce30 0x7f935816a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f935cf08700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f935806a830 0x7f935814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 -- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580cce30 msgr2=0x7f935816a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580cce30 0x7f935816a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 -- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f935806a830 msgr2=0x7f935814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f935806a830 0x7f935814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 -- 172.21.15.148:0/1417433681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348009a20 con 0x7f93580b6580 2023-10-10T20:19:20.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9357fff700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f934800c170 tx=0x7f934800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800cdb0 con 0x7f93580b6580 2023-10-10T20:19:20.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800f040 con 0x7f93580b6580 2023-10-10T20:19:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480146b0 con 0x7f93580b6580 2023-10-10T20:19:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500094f0 con 0x7f93580b6580 2023-10-10T20:19:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.540+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9358153460 con 0x7f93580b6580 2023-10-10T20:19:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.541+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358153c40 con 0x7f93580b6580 2023-10-10T20:19:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.541+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9348003020 con 0x7f93580b6580 2023-10-10T20:19:20.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.542+0000 7f9354ff9700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 0x7f93380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.542+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9348084da0 con 0x7f93580b6580 2023-10-10T20:19:20.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.542+0000 7f935cf08700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 0x7f93380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.542+0000 7f935cf08700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 0x7f93380693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93500099b0 tx=0x7f9350017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.546+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9348052360 con 0x7f93580b6580 2023-10-10T20:19:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.726+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93580b4720 con 0x7f93580b6580 2023-10-10T20:19:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.727+0000 7f9354ff9700 1 -- 172.21.15.148:0/1417433681 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9348055980 con 0x7f93580b6580 2023-10-10T20:19:20.728 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:20.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 msgr2=0x7f93380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 0x7f93380693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93500099b0 tx=0x7f9350017000 comp rx=0 tx=0).stop 2023-10-10T20:19:20.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 msgr2=0x7f93580b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f934800c170 tx=0x7f934800b040 comp rx=0 tx=0).stop 2023-10-10T20:19:20.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 shutdown_connections 2023-10-10T20:19:20.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9338066f30 0x7f93380693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93580cce30 0x7f935816a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f935806a830 0x7f935814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 --2- 172.21.15.148:0/1417433681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93580b6580 0x7f93580b2df0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 >> 172.21.15.148:0/1417433681 conn(0x7f9358068cb0 msgr2=0x7f9358150480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 shutdown_connections 2023-10-10T20:19:20.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.729+0000 7f935df0a700 1 -- 172.21.15.148:0/1417433681 wait complete. 2023-10-10T20:19:20.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:19:20.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.856+0000 7fcb33fff700 1 Processor -- start 2023-10-10T20:19:20.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.856+0000 7fcb33fff700 1 -- start start 2023-10-10T20:19:20.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb33fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb3415dda0 0x7fcb3415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb33fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb340cb7c0 con 0x7fcb340cd6e0 2023-10-10T20:19:20.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb33fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3415e850 con 0x7fcb3415dda0 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb33fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340cc360 con 0x7fcb340cdab0 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb337fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb3415dda0 0x7fcb3415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb337fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb3415dda0 0x7fcb3415cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48328/0 (socket says 172.21.15.148:48328) 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb32ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb340cd6e0 0x7fcb340cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35712/0 (socket says 172.21.15.148:35712) 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb337fe700 1 -- 172.21.15.148:0/2444311718 learned_addr learned my addr 172.21.15.148:0/2444311718 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:20.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.857+0000 7fcb2a7fc700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb337fe700 1 -- 172.21.15.148:0/2444311718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 msgr2=0x7fcb3415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb337fe700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb337fe700 1 -- 172.21.15.148:0/2444311718 >> v1:172.21.15.148:6789/0 conn(0x7fcb340cd6e0 legacy=0x7fcb340cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb337fe700 1 -- 172.21.15.148:0/2444311718 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3415e290 con 0x7fcb3415dda0 2023-10-10T20:19:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb327fc700 1 -- 172.21.15.148:0/2444311718 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486718507 0 0) 0x7fcb340cb7c0 con 0x7fcb340cd6e0 2023-10-10T20:19:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb337fe700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3415dda0 0x7fcb3415cbd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009f20 tx=0x7fcb2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf8300160521815 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb327fc700 1 -- 172.21.15.148:0/2444311718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00e070 con 0x7fcb3415dda0 2023-10-10T20:19:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb327fc700 1 -- 172.21.15.148:0/2444311718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c00ba80 con 0x7fcb3415dda0 2023-10-10T20:19:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb327fc700 1 -- 172.21.15.148:0/2444311718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0092e0 con 0x7fcb3415dda0 2023-10-10T20:19:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 -- 172.21.15.148:0/2444311718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3415dda0 msgr2=0x7fcb3415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3415dda0 0x7fcb3415cbd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009f20 tx=0x7fcb2c009840 comp rx=0 tx=0).stop 2023-10-10T20:19:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 -- 172.21.15.148:0/2444311718 shutdown_connections 2023-10-10T20:19:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3415dda0 0x7fcb3415cbd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2444311718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 -- 172.21.15.148:0/2444311718 >> 172.21.15.148:0/2444311718 conn(0x7fcb34067660 msgr2=0x7fcb34067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 -- 172.21.15.148:0/2444311718 shutdown_connections 2023-10-10T20:19:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.858+0000 7fcb33fff700 1 -- 172.21.15.148:0/2444311718 wait complete. 2023-10-10T20:19:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 Processor -- start 2023-10-10T20:19:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 -- start start 2023-10-10T20:19:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb3415dda0 0x7fcb340c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3414f050 con 0x7fcb340cd6e0 2023-10-10T20:19:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340b47e0 con 0x7fcb3415dda0 2023-10-10T20:19:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb33fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340b5360 con 0x7fcb340cdab0 2023-10-10T20:19:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb32ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb32ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48334/0 (socket says 172.21.15.148:48334) 2023-10-10T20:19:20.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.860+0000 7fcb32ffd700 1 -- 172.21.15.148:0/2008429671 learned_addr learned my addr 172.21.15.148:0/2008429671 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:20.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb337fe700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb3415dda0 0x7fcb340c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb2a7fc700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 -- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 msgr2=0x7fcb3414eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3414eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 -- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb3415dda0 msgr2=0x7fcb340c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb3415dda0 0x7fcb340c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 -- 172.21.15.148:0/2008429671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009ae0 con 0x7fcb340cd6e0 2023-10-10T20:19:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb2a7fc700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb32ffd700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c7e0 tx=0x7fcb24007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24007300 con 0x7fcb340cd6e0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24007460 con 0x7fcb340cd6e0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340c8e30 con 0x7fcb340cd6e0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24004830 con 0x7fcb340cd6e0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.861+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb340b4d10 con 0x7fcb340cd6e0 2023-10-10T20:19:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.862+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34089280 con 0x7fcb340cd6e0 2023-10-10T20:19:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.862+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb24004ba0 con 0x7fcb340cd6e0 2023-10-10T20:19:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.863+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 0x7fcb140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.863+0000 7fcb2a7fc700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 0x7fcb140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.863+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcb24083710 con 0x7fcb340cd6e0 2023-10-10T20:19:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.863+0000 7fcb2a7fc700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 0x7fcb140693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcb3416d810 tx=0x7fcb1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:20.867+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb24050d50 con 0x7fcb340cd6e0 2023-10-10T20:19:21.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.055+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcb3404bb80 con 0x7fcb340cd6e0 2023-10-10T20:19:21.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.055+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/2008429671 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcb24018020 con 0x7fcb340cd6e0 2023-10-10T20:19:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:21.057 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:19:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:19:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:19:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:19:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:19:21.058 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:19:21.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 msgr2=0x7fcb140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 0x7fcb140693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcb3416d810 tx=0x7fcb1c008040 comp rx=0 tx=0).stop 2023-10-10T20:19:21.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 msgr2=0x7fcb340ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c7e0 tx=0x7fcb24007e10 comp rx=0 tx=0).stop 2023-10-10T20:19:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 shutdown_connections 2023-10-10T20:19:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb14066f30 0x7fcb140693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb3415dda0 0x7fcb340c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb340cd6e0 0x7fcb340ca7c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 --2- 172.21.15.148:0/2008429671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb340cdab0 0x7fcb3414eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 >> 172.21.15.148:0/2008429671 conn(0x7fcb34067660 msgr2=0x7fcb34156250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 shutdown_connections 2023-10-10T20:19:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.057+0000 7fcb33fff700 1 -- 172.21.15.148:0/2008429671 wait complete. 2023-10-10T20:19:21.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:19:21.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:19:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.188+0000 7f33444cd700 1 Processor -- start 2023-10-10T20:19:21.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.188+0000 7f33444cd700 1 -- start start 2023-10-10T20:19:21.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.188+0000 7f33444cd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 0x7f333c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.188+0000 7f33444cd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f333c0b46f0 0x7f333c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.188+0000 7f33444cd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0b17b0 con 0x7f333c0b6a10 2023-10-10T20:19:21.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f33444cd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0b5020 con 0x7f333c0b46f0 2023-10-10T20:19:21.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f33444cd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0b1a60 con 0x7f333c159ef0 2023-10-10T20:19:21.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f333c0b46f0 0x7f333c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341a68700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f333c0b6a10 0x7f333c0b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35728/0 (socket says 172.21.15.148:35728) 2023-10-10T20:19:21.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341a68700 1 -- 172.21.15.148:0/1481429685 learned_addr learned my addr 172.21.15.148:0/1481429685 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:21.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342269700 1 --2- 172.21.15.148:0/1481429685 >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 0x7f333c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 -- 172.21.15.148:0/1481429685 >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 msgr2=0x7f333c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 --2- 172.21.15.148:0/1481429685 >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 0x7f333c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 -- 172.21.15.148:0/1481429685 >> v1:172.21.15.148:6789/0 conn(0x7f333c0b6a10 legacy=0x7f333c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341267700 1 -- 172.21.15.148:0/1481429685 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202540097 0 0) 0x7f333c0b17b0 con 0x7f333c0b6a10 2023-10-10T20:19:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 -- 172.21.15.148:0/1481429685 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0b1d40 con 0x7f333c0b46f0 2023-10-10T20:19:21.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342269700 1 --2- 172.21.15.148:0/1481429685 >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 0x7f333c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3342a6a700 1 --2- 172.21.15.148:0/1481429685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b46f0 0x7f333c0b4ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f333c08a710 tx=0x7f3334009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5095881ccfe188a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341267700 1 -- 172.21.15.148:0/1481429685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400e070 con 0x7f333c0b46f0 2023-10-10T20:19:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341267700 1 -- 172.21.15.148:0/1481429685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333400bc30 con 0x7f333c0b46f0 2023-10-10T20:19:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.189+0000 7f3341267700 1 -- 172.21.15.148:0/1481429685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334012430 con 0x7f333c0b46f0 2023-10-10T20:19:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 -- 172.21.15.148:0/1481429685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b46f0 msgr2=0x7f333c0b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 --2- 172.21.15.148:0/1481429685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b46f0 0x7f333c0b4ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f333c08a710 tx=0x7f3334009780 comp rx=0 tx=0).stop 2023-10-10T20:19:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 -- 172.21.15.148:0/1481429685 shutdown_connections 2023-10-10T20:19:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 --2- 172.21.15.148:0/1481429685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b46f0 0x7f333c0b4ae0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 --2- 172.21.15.148:0/1481429685 >> v2:172.21.15.161:3300/0 conn(0x7f333c159ef0 0x7f333c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 -- 172.21.15.148:0/1481429685 >> 172.21.15.148:0/1481429685 conn(0x7f333c09a0a0 msgr2=0x7f333c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 -- 172.21.15.148:0/1481429685 shutdown_connections 2023-10-10T20:19:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.190+0000 7f33444cd700 1 -- 172.21.15.148:0/1481429685 wait complete. 2023-10-10T20:19:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.191+0000 7f33444cd700 1 Processor -- start 2023-10-10T20:19:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.191+0000 7f33444cd700 1 -- start start 2023-10-10T20:19:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f333c0b46f0 0x7f333c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f333c159ef0 0x7f333c09def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0af030 con 0x7f333c0b6a10 2023-10-10T20:19:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0af310 con 0x7f333c159ef0 2023-10-10T20:19:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0af5f0 con 0x7f333c0b46f0 2023-10-10T20:19:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48350/0 (socket says 172.21.15.148:48350) 2023-10-10T20:19:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 -- 172.21.15.148:0/4005566115 learned_addr learned my addr 172.21.15.148:0/4005566115 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3342269700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f333c0b46f0 0x7f333c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3342a6a700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f333c159ef0 0x7f333c09def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 -- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f333c0b46f0 msgr2=0x7f333c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f333c0b46f0 0x7f333c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 -- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f333c159ef0 msgr2=0x7f333c09def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f333c159ef0 0x7f333c09def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 -- 172.21.15.148:0/4005566115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334009a20 con 0x7f333c0b6a10 2023-10-10T20:19:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f3341a68700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f332c00c810 tx=0x7f332c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c005c00 con 0x7f333c0b6a10 2023-10-10T20:19:21.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c005d60 con 0x7f333c0b6a10 2023-10-10T20:19:21.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c007300 con 0x7f333c0b6a10 2023-10-10T20:19:21.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c0af8d0 con 0x7f333c0b6a10 2023-10-10T20:19:21.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.192+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c158080 con 0x7f333c0b6a10 2023-10-10T20:19:21.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.193+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c0afdf0 con 0x7f333c0b6a10 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.194+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f332c01b3b0 con 0x7f333c0b6a10 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.194+0000 7f33327fc700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 0x7f33200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.194+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f332c0836a0 con 0x7f333c0b6a10 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.194+0000 7f3342269700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 0x7f33200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.194+0000 7f3342269700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 0x7f33200693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f333800aca0 tx=0x7f3338008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.198+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f333c0afdf0 con 0x7f333c0b6a10 2023-10-10T20:19:21.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.381+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f333c0afdf0 con 0x7f333c0b6a10 2023-10-10T20:19:21.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.473+0000 7f33327fc700 1 -- 172.21.15.148:0/4005566115 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f333c0afdf0 con 0x7f333c0b6a10 2023-10-10T20:19:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 msgr2=0x7f33200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 0x7f33200693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f333800aca0 tx=0x7f3338008040 comp rx=0 tx=0).stop 2023-10-10T20:19:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 msgr2=0x7f333c09d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f332c00c810 tx=0x7f332c007db0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 shutdown_connections 2023-10-10T20:19:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3320066f30 0x7f33200693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f333c159ef0 0x7f333c09def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f333c0b46f0 0x7f333c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 --2- 172.21.15.148:0/4005566115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f333c0b6a10 0x7f333c09d9b0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f332c00c810 tx=0x7f332c007db0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 >> 172.21.15.148:0/4005566115 conn(0x7f333c09a0a0 msgr2=0x7f333c15cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 shutdown_connections 2023-10-10T20:19:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.477+0000 7f33444cd700 1 -- 172.21.15.148:0/4005566115 wait complete. 2023-10-10T20:19:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:19:21.490 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:19:21.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.610+0000 7f1567994700 1 Processor -- start 2023-10-10T20:19:21.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 -- start start 2023-10-10T20:19:21.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f156006d690 0x7f1560067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f156006da60 0x7f156015e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156015a250 con 0x7f15600cdf70 2023-10-10T20:19:21.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15600cbfd0 con 0x7f156006d690 2023-10-10T20:19:21.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.611+0000 7f1567994700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156015e940 con 0x7f156006da60 2023-10-10T20:19:21.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565f31700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f15600cdf70 0x7f15600cc750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35740/0 (socket says 172.21.15.148:35740) 2023-10-10T20:19:21.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565f31700 1 -- 172.21.15.148:0/4042349382 learned_addr learned my addr 172.21.15.148:0/4042349382 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:21.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1564f2f700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f156006da60 0x7f156015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 --2- 172.21.15.148:0/4042349382 >> v2:172.21.15.161:3300/0 conn(0x7f156006d690 0x7f1560067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966331191 0 0) 0x7f156015a250 con 0x7f15600cdf70 2023-10-10T20:19:21.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f154c003570 con 0x7f15600cdf70 2023-10-10T20:19:21.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480327974 0 0) 0x7f154c003570 con 0x7f15600cdf70 2023-10-10T20:19:21.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f156006da60 msgr2=0x7f156015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:21.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f156006da60 0x7f156015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 >> v2:172.21.15.161:3300/0 conn(0x7f156006d690 msgr2=0x7f1560067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:21.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 --2- 172.21.15.148:0/4042349382 >> v2:172.21.15.161:3300/0 conn(0x7f156006d690 0x7f1560067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560067c40 con 0x7f15600cdf70 2023-10-10T20:19:21.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f15580017f0 con 0x7f15600cdf70 2023-10-10T20:19:21.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 >> v1:172.21.15.148:6789/0 conn(0x7f15600cdf70 legacy=0x7f15600cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f154c003a60 0x7f154c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c004810 0x7f154c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f154c005180 0x7f154c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600cbfd0 con 0x7f154c003a60 2023-10-10T20:19:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156015e940 con 0x7f154c005180 2023-10-10T20:19:21.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c00bfa0 con 0x7f154c004810 2023-10-10T20:19:21.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f154c003a60 0x7f154c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1564f2f700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c004810 0x7f154c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565f31700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f154c005180 0x7f154c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 -- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c004810 msgr2=0x7f154c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c004810 0x7f154c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 -- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f154c005180 msgr2=0x7f154c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f154c005180 0x7f154c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565730700 1 -- 172.21.15.148:0/4042349382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c003570 con 0x7f154c003a60 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.612+0000 7f1565f31700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f154c005180 0x7f154c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:21.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1565730700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f154c003a60 0x7f154c003e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c30 tx=0x7f155c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17d353f4073540c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00e710 con 0x7f154c003a60 2023-10-10T20:19:21.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00e870 con 0x7f154c003a60 2023-10-10T20:19:21.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1557fff700 1 -- 172.21.15.148:0/4042349382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00ec30 con 0x7f154c003a60 2023-10-10T20:19:21.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 -- 172.21.15.148:0/4042349382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f154c003a60 msgr2=0x7f154c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 --2- 172.21.15.148:0/4042349382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f154c003a60 0x7f154c003e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c30 tx=0x7f155c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1565f31700 1 -- 172.21.15.148:0/4042349382 reap_dead start 2023-10-10T20:19:21.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 -- 172.21.15.148:0/4042349382 shutdown_connections 2023-10-10T20:19:21.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 -- 172.21.15.148:0/4042349382 >> 172.21.15.148:0/4042349382 conn(0x7f1560069870 msgr2=0x7f1560069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:21.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 -- 172.21.15.148:0/4042349382 shutdown_connections 2023-10-10T20:19:21.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.613+0000 7f1567994700 1 -- 172.21.15.148:0/4042349382 wait complete. 2023-10-10T20:19:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.614+0000 7f1567994700 1 Processor -- start 2023-10-10T20:19:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.614+0000 7f1567994700 1 -- start start 2023-10-10T20:19:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c003a60 0x7f156016cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15601577c0 0x7f1560157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560158170 con 0x7f15600c8500 2023-10-10T20:19:21.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156016de70 con 0x7f15601577c0 2023-10-10T20:19:21.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1567994700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156016d5d0 con 0x7f154c003a60 2023-10-10T20:19:21.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1564f2f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1564f2f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48376/0 (socket says 172.21.15.148:48376) 2023-10-10T20:19:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.615+0000 7f1564f2f700 1 -- 172.21.15.148:0/260529062 learned_addr learned my addr 172.21.15.148:0/260529062 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1565730700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c003a60 0x7f156016cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1565f31700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15601577c0 0x7f1560157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 -- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c003a60 msgr2=0x7f156016cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c003a60 0x7f156016cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 -- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15601577c0 msgr2=0x7f1560157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:21.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15601577c0 0x7f1560157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:21.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 -- 172.21.15.148:0/260529062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156016d8b0 con 0x7f15600c8500 2023-10-10T20:19:21.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1564f2f700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1550011d40 tx=0x7f15500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000fac0 con 0x7f15600c8500 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155000fc20 con 0x7f15600c8500 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156016db90 con 0x7f15600c8500 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550014440 con 0x7f15600c8500 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.616+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15600c8970 con 0x7f15600c8500 2023-10-10T20:19:21.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.617+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156004c3c0 con 0x7f15600c8500 2023-10-10T20:19:21.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.618+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1550014870 con 0x7f15600c8500 2023-10-10T20:19:21.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.618+0000 7f15557fa700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:21.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.618+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f1550018030 con 0x7f15600c8500 2023-10-10T20:19:21.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.618+0000 7f1565730700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:21.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.618+0000 7f1565730700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f155c0091e0 tx=0x7f155c01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:21.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.623+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1550052260 con 0x7f15600c8500 2023-10-10T20:19:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:21.806+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1560067f20 con 0x7f15600c8500 2023-10-10T20:19:22.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:19:22.224 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:19:22.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.480+0000 7f15557fa700 1 -- 172.21.15.148:0/260529062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f15500220d0 con 0x7f15600c8500 2023-10-10T20:19:22.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.485+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 msgr2=0x7f15440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:22.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.485+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f155c0091e0 tx=0x7f155c01a000 comp rx=0 tx=0).stop 2023-10-10T20:19:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.485+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 msgr2=0x7f15601545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.485+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1550011d40 tx=0x7f15500091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 shutdown_connections 2023-10-10T20:19:22.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15601577c0 0x7f1560157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f154c003a60 0x7f156016cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 --2- 172.21.15.148:0/260529062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15600c8500 0x7f15601545d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.486+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 >> 172.21.15.148:0/260529062 conn(0x7f1560069870 msgr2=0x7f15600ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:22.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.490+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 shutdown_connections 2023-10-10T20:19:22.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.490+0000 7f1567994700 1 -- 172.21.15.148:0/260529062 wait complete. 2023-10-10T20:19:22.493 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:19:22.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:19:22.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 Processor -- start 2023-10-10T20:19:22.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 -- start start 2023-10-10T20:19:22.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdbe8154440 0x7fdbe815da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe815e1e0 con 0x7fdbe815af30 2023-10-10T20:19:22.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe815dfd0 con 0x7fdbe8154440 2023-10-10T20:19:22.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.620+0000 7fdbed832700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe815e4c0 con 0x7fdbe815b380 2023-10-10T20:19:22.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdbe8154440 0x7fdbe815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdbe8154440 0x7fdbe815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49738/0 (socket says 172.21.15.148:49738) 2023-10-10T20:19:22.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/134639560 learned_addr learned my addr 172.21.15.148:0/134639560 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:22.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe77fe700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/134639560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 msgr2=0x7fdbe80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:22.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe80cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/134639560 >> v1:172.21.15.161:6789/0 conn(0x7fdbe815af30 legacy=0x7fdbe8067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/134639560 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe815e7a0 con 0x7fdbe8154440 2023-10-10T20:19:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe6ffd700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe815da90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80099b0 tx=0x7fdbd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fee0723e877db96 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe67fc700 1 -- 172.21.15.148:0/134639560 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800b820 con 0x7fdbe8154440 2023-10-10T20:19:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe67fc700 1 -- 172.21.15.148:0/134639560 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800b980 con 0x7fdbe8154440 2023-10-10T20:19:22.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.621+0000 7fdbe67fc700 1 -- 172.21.15.148:0/134639560 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800bd40 con 0x7fdbe8154440 2023-10-10T20:19:22.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 -- 172.21.15.148:0/134639560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 msgr2=0x7fdbe815da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:22.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe815da90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80099b0 tx=0x7fdbd8009250 comp rx=0 tx=0).stop 2023-10-10T20:19:22.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 -- 172.21.15.148:0/134639560 shutdown_connections 2023-10-10T20:19:22.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe815da90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 --2- 172.21.15.148:0/134639560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 -- 172.21.15.148:0/134639560 >> 172.21.15.148:0/134639560 conn(0x7fdbe80d0dd0 msgr2=0x7fdbe80d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:22.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 -- 172.21.15.148:0/134639560 shutdown_connections 2023-10-10T20:19:22.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.622+0000 7fdbed832700 1 -- 172.21.15.148:0/134639560 wait complete. 2023-10-10T20:19:22.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.623+0000 7fdbed832700 1 Processor -- start 2023-10-10T20:19:22.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.623+0000 7fdbed832700 1 -- start start 2023-10-10T20:19:22.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbe815af30 0x7fdbe8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8154d50 con 0x7fdbe8154440 2023-10-10T20:19:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8157670 con 0x7fdbe815af30 2023-10-10T20:19:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbed832700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80b5440 con 0x7fdbe815b380 2023-10-10T20:19:22.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49740/0 (socket says 172.21.15.148:49740) 2023-10-10T20:19:22.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/2868975665 learned_addr learned my addr 172.21.15.148:0/2868975665 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:22.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbde7fc700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbe815af30 0x7fdbe8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe77fe700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 msgr2=0x7fdbe81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe81567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbe815af30 msgr2=0x7fdbe8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbe815af30 0x7fdbe8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 -- 172.21.15.148:0/2868975665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80094f0 con 0x7fdbe8154440 2023-10-10T20:19:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbe6ffd700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006fd0 tx=0x7fdbd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.624+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8006a90 con 0x7fdbe8154440 2023-10-10T20:19:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.625+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8020070 con 0x7fdbe8154440 2023-10-10T20:19:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.625+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe80b5720 con 0x7fdbe8154440 2023-10-10T20:19:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.625+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8011e90 con 0x7fdbe8154440 2023-10-10T20:19:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.625+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe80b5ce0 con 0x7fdbe8154440 2023-10-10T20:19:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.625+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe804c3c0 con 0x7fdbe8154440 2023-10-10T20:19:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.626+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbd8022030 con 0x7fdbe8154440 2023-10-10T20:19:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.626+0000 7fdbdf7fe700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 0x7fdbc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.626+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fdbd8086310 con 0x7fdbe8154440 2023-10-10T20:19:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.626+0000 7fdbde7fc700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 0x7fdbc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.627+0000 7fdbde7fc700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 0x7fdbc80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b2560 tx=0x7fdbd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:22.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.631+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbd80535e0 con 0x7fdbe8154440 2023-10-10T20:19:22.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:22.856+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdbe804c3c0 con 0x7fdbe8154440 2023-10-10T20:19:23.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.494+0000 7fdbdf7fe700 1 -- 172.21.15.148:0/2868975665 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdbd8056c00 con 0x7fdbe8154440 2023-10-10T20:19:23.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 msgr2=0x7fdbc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 0x7fdbc80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b2560 tx=0x7fdbd0009040 comp rx=0 tx=0).stop 2023-10-10T20:19:23.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 msgr2=0x7fdbe8154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006fd0 tx=0x7fdbd8016000 comp rx=0 tx=0).stop 2023-10-10T20:19:23.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 shutdown_connections 2023-10-10T20:19:23.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdbc8066f30 0x7fdbc80693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbe815b380 0x7fdbe81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe8154440 0x7fdbe8154810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 --2- 172.21.15.148:0/2868975665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbe815af30 0x7fdbe8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 >> 172.21.15.148:0/2868975665 conn(0x7fdbe80d0dd0 msgr2=0x7fdbe8153640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:23.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 shutdown_connections 2023-10-10T20:19:23.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.497+0000 7fdbed832700 1 -- 172.21.15.148:0/2868975665 wait complete. 2023-10-10T20:19:23.501 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 2 and data pool 3 2023-10-10T20:19:23.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.634+0000 7f06ec2aa700 1 Processor -- start 2023-10-10T20:19:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.634+0000 7f06ec2aa700 1 -- start start 2023-10-10T20:19:23.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06ec2aa700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f06e4159e10 0x7f06e40b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06ec2aa700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e415db80 con 0x7f06e415a1e0 2023-10-10T20:19:23.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06ec2aa700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e415de60 con 0x7f06e40cde50 2023-10-10T20:19:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06ec2aa700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e415e800 con 0x7f06e4159e10 2023-10-10T20:19:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06e9845700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06e415a1e0 0x7f06e40cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48192/0 (socket says 172.21.15.148:48192) 2023-10-10T20:19:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.635+0000 7f06e9845700 1 -- 172.21.15.148:0/1388276747 learned_addr learned my addr 172.21.15.148:0/1388276747 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041209744 0 0) 0x7f06e415db80 con 0x7f06e415a1e0 2023-10-10T20:19:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d0003570 con 0x7f06e415a1e0 2023-10-10T20:19:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 909183103 0 0) 0x7f06d0003570 con 0x7f06e415a1e0 2023-10-10T20:19:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 >> v2:172.21.15.161:3300/0 conn(0x7f06e4159e10 msgr2=0x7f06e40b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 --2- 172.21.15.148:0/1388276747 >> v2:172.21.15.161:3300/0 conn(0x7f06e4159e10 0x7f06e40b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 >> v1:172.21.15.161:6789/0 conn(0x7f06e40cde50 legacy=0x7f06e40cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e415e140 con 0x7f06e415a1e0 2023-10-10T20:19:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f06d4001470 con 0x7f06e415a1e0 2023-10-10T20:19:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 >> v1:172.21.15.148:6789/0 conn(0x7f06e415a1e0 legacy=0x7f06e40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06d0003a60 0x7f06d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 0x7f06d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0005310 0x7f06d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415e800 con 0x7f06d0004810 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d000c2d0 con 0x7f06d0005310 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d000c490 con 0x7f06d0003a60 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 0x7f06d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06ea046700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06d0003a60 0x7f06d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06ea847700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0005310 0x7f06d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 -- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06d0003a60 msgr2=0x7f06d0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06d0003a60 0x7f06d0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 -- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0005310 msgr2=0x7f06d000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0005310 0x7f06d000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 -- 172.21.15.148:0/1388276747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0003570 con 0x7f06d0004810 2023-10-10T20:19:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06ea847700 1 -- 172.21.15.148:0/1388276747 reap_dead start 2023-10-10T20:19:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06ea046700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06d0003a60 0x7f06d0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9845700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 0x7f06d0004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c390 tx=0x7f06d4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28af3d4f6f081483 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.636+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400cc60 con 0x7f06d0004810 2023-10-10T20:19:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d400cdc0 con 0x7f06d0004810 2023-10-10T20:19:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06e9044700 1 -- 172.21.15.148:0/1388276747 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4005b70 con 0x7f06d0004810 2023-10-10T20:19:23.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 -- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 msgr2=0x7f06d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 0x7f06d0004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c390 tx=0x7f06d4001e70 comp rx=0 tx=0).stop 2023-10-10T20:19:23.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 -- 172.21.15.148:0/1388276747 shutdown_connections 2023-10-10T20:19:23.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/1388276747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06d0004810 0x7f06d0004c80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 -- 172.21.15.148:0/1388276747 >> 172.21.15.148:0/1388276747 conn(0x7f06e4067660 msgr2=0x7f06e4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:23.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 -- 172.21.15.148:0/1388276747 shutdown_connections 2023-10-10T20:19:23.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.637+0000 7f06ec2aa700 1 -- 172.21.15.148:0/1388276747 wait complete. 2023-10-10T20:19:23.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.638+0000 7f06ec2aa700 1 Processor -- start 2023-10-10T20:19:23.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.638+0000 7f06ec2aa700 1 -- start start 2023-10-10T20:19:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0004810 0x7f06e4159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06e40c51a0 0x7f06e40c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40d0250 con 0x7f06e40c5610 2023-10-10T20:19:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4159790 con 0x7f06d0004810 2023-10-10T20:19:23.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ec2aa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40c5bd0 con 0x7f06e40c51a0 2023-10-10T20:19:23.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49758/0 (socket says 172.21.15.148:49758) 2023-10-10T20:19:23.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 -- 172.21.15.148:0/2580409889 learned_addr learned my addr 172.21.15.148:0/2580409889 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:23.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea046700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0004810 0x7f06e4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 -- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06e40c51a0 msgr2=0x7f06e40c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06e40c51a0 0x7f06e40c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 -- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0004810 msgr2=0x7f06e4159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0004810 0x7f06e4159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 -- 172.21.15.148:0/2580409889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40c5eb0 con 0x7f06e40c5610 2023-10-10T20:19:23.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.639+0000 7f06ea847700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06e4089970 tx=0x7f06dc0014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc006d90 con 0x7f06e40c5610 2023-10-10T20:19:23.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc013420 con 0x7f06e40c5610 2023-10-10T20:19:23.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e40c6190 con 0x7f06e40c5610 2023-10-10T20:19:23.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0137e0 con 0x7f06e40c5610 2023-10-10T20:19:23.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e40c49d0 con 0x7f06e40c5610 2023-10-10T20:19:23.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.640+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e40cfd40 con 0x7f06e40c5610 2023-10-10T20:19:23.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.641+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06dc013bd0 con 0x7f06e40c5610 2023-10-10T20:19:23.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.641+0000 7f06da7fc700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.641+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06dc084340 con 0x7f06e40c5610 2023-10-10T20:19:23.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.641+0000 7f06ea046700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.642+0000 7f06ea046700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 0x7f06c80693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f06e404d1b0 tx=0x7f06e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.646+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06e40cfd40 con 0x7f06e40c5610 2023-10-10T20:19:23.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.831+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06e40cfd40 con 0x7f06e40c5610 2023-10-10T20:19:23.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.831+0000 7f06da7fc700 1 -- 172.21.15.148:0/2580409889 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f06e40cfd40 con 0x7f06e40c5610 2023-10-10T20:19:23.832 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:23.833 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:22.856863+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:23.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 msgr2=0x7f06c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 0x7f06c80693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f06e404d1b0 tx=0x7f06e0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:23.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 msgr2=0x7f06e40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f06e4089970 tx=0x7f06dc0014f0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 shutdown_connections 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06c8066f30 0x7f06c80693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06e40c5610 0x7f06e40c76d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06d0004810 0x7f06e4159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 --2- 172.21.15.148:0/2580409889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06e40c51a0 0x7f06e40c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 >> 172.21.15.148:0/2580409889 conn(0x7f06e4067660 msgr2=0x7f06e40cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 shutdown_connections 2023-10-10T20:19:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.834+0000 7f06ec2aa700 1 -- 172.21.15.148:0/2580409889 wait complete. 2023-10-10T20:19:23.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.960+0000 7fc3765a1700 1 Processor -- start 2023-10-10T20:19:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.960+0000 7fc3765a1700 1 -- start start 2023-10-10T20:19:23.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc3765a1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc37015a620 0x7fc37015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc3765a1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc3765a1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37015cbb0 con 0x7fc370157870 2023-10-10T20:19:23.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc3765a1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37015bdb0 con 0x7fc37015a620 2023-10-10T20:19:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc3765a1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015bfa0 con 0x7fc37015d580 2023-10-10T20:19:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc36f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc37015a620 0x7fc37015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc36ffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc370157870 0x7fc37015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48194/0 (socket says 172.21.15.148:48194) 2023-10-10T20:19:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc36f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc37015a620 0x7fc37015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49760/0 (socket says 172.21.15.148:49760) 2023-10-10T20:19:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc36ffff700 1 -- 172.21.15.148:0/312617984 learned_addr learned my addr 172.21.15.148:0/312617984 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.961+0000 7fc374b3e700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36f7fe700 1 -- 172.21.15.148:0/312617984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 msgr2=0x7fc370158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36f7fe700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36f7fe700 1 -- 172.21.15.148:0/312617984 >> v1:172.21.15.148:6789/0 conn(0x7fc370157870 legacy=0x7fc37015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36f7fe700 1 -- 172.21.15.148:0/312617984 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37015c280 con 0x7fc37015a620 2023-10-10T20:19:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36effd700 1 -- 172.21.15.148:0/312617984 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691675200 0 0) 0x7fc37015cbb0 con 0x7fc370157870 2023-10-10T20:19:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36f7fe700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc37015d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3640098e0 tx=0x7fc3640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e299e1a13ef4561c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36effd700 1 -- 172.21.15.148:0/312617984 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400b640 con 0x7fc37015a620 2023-10-10T20:19:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36effd700 1 -- 172.21.15.148:0/312617984 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36400b7a0 con 0x7fc37015a620 2023-10-10T20:19:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc36effd700 1 -- 172.21.15.148:0/312617984 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400bb40 con 0x7fc37015a620 2023-10-10T20:19:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 -- 172.21.15.148:0/312617984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 msgr2=0x7fc37015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc37015d040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3640098e0 tx=0x7fc3640091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 -- 172.21.15.148:0/312617984 shutdown_connections 2023-10-10T20:19:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 --2- 172.21.15.148:0/312617984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc37015d040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 -- 172.21.15.148:0/312617984 >> 172.21.15.148:0/312617984 conn(0x7fc370068ce0 msgr2=0x7fc3700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 -- 172.21.15.148:0/312617984 shutdown_connections 2023-10-10T20:19:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.962+0000 7fc3765a1700 1 -- 172.21.15.148:0/312617984 wait complete. 2023-10-10T20:19:23.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.963+0000 7fc3765a1700 1 Processor -- start 2023-10-10T20:19:23.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 -- start start 2023-10-10T20:19:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc370157870 0x7fc3700b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700b0fc0 con 0x7fc37015a620 2023-10-10T20:19:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700b12a0 con 0x7fc370157870 2023-10-10T20:19:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.964+0000 7fc3765a1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700b1580 con 0x7fc37015d580 2023-10-10T20:19:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49764/0 (socket says 172.21.15.148:49764) 2023-10-10T20:19:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 -- 172.21.15.148:0/4178589214 learned_addr learned my addr 172.21.15.148:0/4178589214 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36ffff700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc370157870 0x7fc3700b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc374b3e700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 -- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 msgr2=0x7fc370159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 -- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc370157870 msgr2=0x7fc3700b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc370157870 0x7fc3700b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 -- 172.21.15.148:0/4178589214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009420 con 0x7fc37015a620 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc36f7fe700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc364013000 tx=0x7fc364017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400bb20 con 0x7fc37015a620 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364000c90 con 0x7fc37015a620 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700b1860 con 0x7fc37015a620 2023-10-10T20:19:23.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364011650 con 0x7fc37015a620 2023-10-10T20:19:23.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.965+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3700b0c00 con 0x7fc37015a620 2023-10-10T20:19:23.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.966+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37004c3c0 con 0x7fc37015a620 2023-10-10T20:19:23.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.967+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc364011a00 con 0x7fc37015a620 2023-10-10T20:19:23.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.967+0000 7fc357fff700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 0x7fc35c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:23.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.967+0000 7fc36ffff700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 0x7fc35c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:23.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.967+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc364085b00 con 0x7fc37015a620 2023-10-10T20:19:23.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.968+0000 7fc36ffff700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 0x7fc35c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc360007fd0 tx=0x7fc360007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:23.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:23.972+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc364052df0 con 0x7fc37015a620 2023-10-10T20:19:24.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.156+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc37015c560 con 0x7fc37015a620 2023-10-10T20:19:24.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.157+0000 7fc357fff700 1 -- 172.21.15.148:0/4178589214 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fc364015090 con 0x7fc37015a620 2023-10-10T20:19:24.158 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:24.158 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:22.856863+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:24.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 msgr2=0x7fc35c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:24.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 0x7fc35c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc360007fd0 tx=0x7fc360007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:24.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 msgr2=0x7fc3700b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:24.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc364013000 tx=0x7fc364017000 comp rx=0 tx=0).stop 2023-10-10T20:19:24.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 shutdown_connections 2023-10-10T20:19:24.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc35c066f30 0x7fc35c0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc37015d580 0x7fc370159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc370157870 0x7fc3700b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 --2- 172.21.15.148:0/4178589214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc37015a620 0x7fc3700b06c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 >> 172.21.15.148:0/4178589214 conn(0x7fc370068ce0 msgr2=0x7fc37014c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:24.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 shutdown_connections 2023-10-10T20:19:24.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.159+0000 7fc3765a1700 1 -- 172.21.15.148:0/4178589214 wait complete. 2023-10-10T20:19:24.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:19:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.288+0000 7f18db5d3700 1 Processor -- start 2023-10-10T20:19:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.288+0000 7f18db5d3700 1 -- start start 2023-10-10T20:19:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f18d40cd300 0x7f18d415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d40d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18d40d12a0 0x7f18d406a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d415e7a0 con 0x7f18d40d12a0 2023-10-10T20:19:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d41513f0 con 0x7f18d40cd300 2023-10-10T20:19:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18db5d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40d1700 con 0x7f18d40b6760 2023-10-10T20:19:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18d40d12a0 0x7f18d406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18d40d12a0 0x7f18d406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49772/0 (socket says 172.21.15.148:49772) 2023-10-10T20:19:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 -- 172.21.15.148:0/698820418 learned_addr learned my addr 172.21.15.148:0/698820418 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d936f700 1 --2- 172.21.15.148:0/698820418 >> v2:172.21.15.161:3300/0 conn(0x7f18d40cd300 0x7f18d415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d8b6e700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d40d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 -- 172.21.15.148:0/698820418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 msgr2=0x7f18d40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:24.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d40d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 -- 172.21.15.148:0/698820418 >> v2:172.21.15.161:3300/0 conn(0x7f18d40cd300 msgr2=0x7f18d415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:24.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 --2- 172.21.15.148:0/698820418 >> v2:172.21.15.161:3300/0 conn(0x7f18d40cd300 0x7f18d415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 -- 172.21.15.148:0/698820418 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d415db20 con 0x7f18d40d12a0 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d8b6e700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d40d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d9b70700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40d12a0 0x7f18d406a510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4089ea0 tx=0x7f18c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a978253d78508c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d3fff700 1 -- 172.21.15.148:0/698820418 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c4009e70 con 0x7f18d40d12a0 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d3fff700 1 -- 172.21.15.148:0/698820418 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c400b860 con 0x7f18d40d12a0 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.289+0000 7f18d3fff700 1 -- 172.21.15.148:0/698820418 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c400bc00 con 0x7f18d40d12a0 2023-10-10T20:19:24.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 -- 172.21.15.148:0/698820418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40d12a0 msgr2=0x7f18d406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:24.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40d12a0 0x7f18d406a510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4089ea0 tx=0x7f18c4009250 comp rx=0 tx=0).stop 2023-10-10T20:19:24.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 -- 172.21.15.148:0/698820418 shutdown_connections 2023-10-10T20:19:24.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40d12a0 0x7f18d406a510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 --2- 172.21.15.148:0/698820418 >> v2:172.21.15.161:3300/0 conn(0x7f18d40cd300 0x7f18d415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 --2- 172.21.15.148:0/698820418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d40d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 -- 172.21.15.148:0/698820418 >> 172.21.15.148:0/698820418 conn(0x7f18d4068d90 msgr2=0x7f18d4069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:24.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 -- 172.21.15.148:0/698820418 shutdown_connections 2023-10-10T20:19:24.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.290+0000 7f18db5d3700 1 -- 172.21.15.148:0/698820418 wait complete. 2023-10-10T20:19:24.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.291+0000 7f18db5d3700 1 Processor -- start 2023-10-10T20:19:24.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.291+0000 7f18db5d3700 1 -- start start 2023-10-10T20:19:24.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d414a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 0x7f18d40b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d414e690 con 0x7f18d40cd300 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d415f010 con 0x7f18d40d12a0 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18db5d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d414aa20 con 0x7f18d40b6760 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d8b6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d8b6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49778/0 (socket says 172.21.15.148:49778) 2023-10-10T20:19:24.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d8b6e700 1 -- 172.21.15.148:0/690053470 learned_addr learned my addr 172.21.15.148:0/690053470 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:24.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d936f700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d414a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d9b70700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 0x7f18d40b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.292+0000 7f18d8b6e700 1 -- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 msgr2=0x7f18d414a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:24.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d8b6e700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d414a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d8b6e700 1 -- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 msgr2=0x7f18d40b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:24.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d8b6e700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 0x7f18d40b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:24.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d8b6e700 1 -- 172.21.15.148:0/690053470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40094f0 con 0x7f18d40cd300 2023-10-10T20:19:24.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d9b70700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 0x7f18d40b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:24.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d8b6e700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18c800d860 tx=0x7f18c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:24.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80125f0 con 0x7f18d40cd300 2023-10-10T20:19:24.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8012750 con 0x7f18d40cd300 2023-10-10T20:19:24.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d415ece0 con 0x7f18d40cd300 2023-10-10T20:19:24.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012ad0 con 0x7f18d40cd300 2023-10-10T20:19:24.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.293+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d414e3b0 con 0x7f18d40cd300 2023-10-10T20:19:24.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.294+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d415f1a0 con 0x7f18d40cd300 2023-10-10T20:19:24.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.294+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18c8014430 con 0x7f18d40cd300 2023-10-10T20:19:24.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.294+0000 7f18d17fa700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 0x7f18c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:24.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.295+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18c80849d0 con 0x7f18d40cd300 2023-10-10T20:19:24.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.295+0000 7f18d936f700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 0x7f18c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:24.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.295+0000 7f18d936f700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 0x7f18c00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007600 tx=0x7f18cc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:24.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.299+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18c80556d0 con 0x7f18d40cd300 2023-10-10T20:19:24.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:24.526+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18d404c3c0 con 0x7f18d40cd300 2023-10-10T20:19:25.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.510+0000 7f18d17fa700 1 -- 172.21.15.148:0/690053470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18c8055260 con 0x7f18d40cd300 2023-10-10T20:19:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 msgr2=0x7f18c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 0x7f18c00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007600 tx=0x7f18cc00b040 comp rx=0 tx=0).stop 2023-10-10T20:19:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 msgr2=0x7f18d414d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18c800d860 tx=0x7f18c8008040 comp rx=0 tx=0).stop 2023-10-10T20:19:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 shutdown_connections 2023-10-10T20:19:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18c0066f30 0x7f18c00693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18d40d12a0 0x7f18d40b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18d40b6760 0x7f18d414a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 --2- 172.21.15.148:0/690053470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d40cd300 0x7f18d414d720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 >> 172.21.15.148:0/690053470 conn(0x7f18d4068d90 msgr2=0x7f18d414f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 shutdown_connections 2023-10-10T20:19:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.513+0000 7f18db5d3700 1 -- 172.21.15.148:0/690053470 wait complete. 2023-10-10T20:19:25.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:19:25.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.644+0000 7f1208483700 1 Processor -- start 2023-10-10T20:19:25.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.644+0000 7f1208483700 1 -- start start 2023-10-10T20:19:25.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1208483700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f120015b9a0 0x7f12000caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1208483700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f120006d370 0x7f120006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1208483700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120015c8d0 con 0x7f1200152300 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1208483700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12000b1400 con 0x7f120015b9a0 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1208483700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120015be20 con 0x7f120006d370 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f120015b9a0 0x7f12000caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120621f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1200152300 0x7f1200158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48196/0 (socket says 172.21.15.148:48196) 2023-10-10T20:19:25.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120621f700 1 -- 172.21.15.148:0/3441784749 learned_addr learned my addr 172.21.15.148:0/3441784749 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:25.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1206a20700 1 --2- 172.21.15.148:0/3441784749 >> v2:172.21.15.161:3300/0 conn(0x7f120006d370 0x7f120006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 -- 172.21.15.148:0/3441784749 >> v2:172.21.15.161:3300/0 conn(0x7f120006d370 msgr2=0x7f120006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:25.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 --2- 172.21.15.148:0/3441784749 >> v2:172.21.15.161:3300/0 conn(0x7f120006d370 0x7f120006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 -- 172.21.15.148:0/3441784749 >> v1:172.21.15.148:6789/0 conn(0x7f1200152300 legacy=0x7f1200158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120521d700 1 -- 172.21.15.148:0/3441784749 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665433100 0 0) 0x7f120015c8d0 con 0x7f1200152300 2023-10-10T20:19:25.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 -- 172.21.15.148:0/3441784749 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000c9370 con 0x7f120015b9a0 2023-10-10T20:19:25.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f1205a1e700 1 --2- 172.21.15.148:0/3441784749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000caaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12000cc4d0 tx=0x7f11f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fff880da28e1f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:25.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120521d700 1 -- 172.21.15.148:0/3441784749 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f400e070 con 0x7f120015b9a0 2023-10-10T20:19:25.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120521d700 1 -- 172.21.15.148:0/3441784749 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f400bc30 con 0x7f120015b9a0 2023-10-10T20:19:25.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.645+0000 7f120521d700 1 -- 172.21.15.148:0/3441784749 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4012430 con 0x7f120015b9a0 2023-10-10T20:19:25.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 -- 172.21.15.148:0/3441784749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 msgr2=0x7f12000caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 --2- 172.21.15.148:0/3441784749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000caaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12000cc4d0 tx=0x7f11f4009780 comp rx=0 tx=0).stop 2023-10-10T20:19:25.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 -- 172.21.15.148:0/3441784749 shutdown_connections 2023-10-10T20:19:25.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 --2- 172.21.15.148:0/3441784749 >> v2:172.21.15.161:3300/0 conn(0x7f120006d370 0x7f120006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 --2- 172.21.15.148:0/3441784749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000caaf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 -- 172.21.15.148:0/3441784749 >> 172.21.15.148:0/3441784749 conn(0x7f1200069870 msgr2=0x7f1200069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:25.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 -- 172.21.15.148:0/3441784749 shutdown_connections 2023-10-10T20:19:25.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.646+0000 7f1208483700 1 -- 172.21.15.148:0/3441784749 wait complete. 2023-10-10T20:19:25.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.647+0000 7f1208483700 1 Processor -- start 2023-10-10T20:19:25.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.647+0000 7f1208483700 1 -- start start 2023-10-10T20:19:25.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f120006d370 0x7f120014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1200152300 0x7f12000b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120014fc70 con 0x7f120015b9a0 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000b3e80 con 0x7f120006d370 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120015ad30 con 0x7f1200152300 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49796/0 (socket says 172.21.15.148:49796) 2023-10-10T20:19:25.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 -- 172.21.15.148:0/4042504878 learned_addr learned my addr 172.21.15.148:0/4042504878 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:25.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f120621f700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f120006d370 0x7f120014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1205a1e700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1200152300 0x7f12000b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 -- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1200152300 msgr2=0x7f12000b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1200152300 0x7f12000b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 -- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f120006d370 msgr2=0x7f120014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:25.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f120006d370 0x7f120014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:25.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 -- 172.21.15.148:0/4042504878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4009a20 con 0x7f120015b9a0 2023-10-10T20:19:25.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1206a20700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d890 tx=0x7f11f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:25.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00125f0 con 0x7f120015b9a0 2023-10-10T20:19:25.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f0012770 con 0x7f120015b9a0 2023-10-10T20:19:25.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12000b3550 con 0x7f120015b9a0 2023-10-10T20:19:25.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.648+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012b30 con 0x7f120015b9a0 2023-10-10T20:19:25.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.649+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12000b3b10 con 0x7f120015b9a0 2023-10-10T20:19:25.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.649+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120004c3c0 con 0x7f120015b9a0 2023-10-10T20:19:25.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.650+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11f001e460 con 0x7f120015b9a0 2023-10-10T20:19:25.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.650+0000 7f11fe7fc700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 0x7f11ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:25.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.650+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11f00849f0 con 0x7f120015b9a0 2023-10-10T20:19:25.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.650+0000 7f120621f700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 0x7f11ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:25.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.651+0000 7f120621f700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 0x7f11ec0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11f8007fd0 tx=0x7f11f8007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:25.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.655+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f0051c60 con 0x7f120015b9a0 2023-10-10T20:19:25.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:25.883+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1200151c60 con 0x7f120015b9a0 2023-10-10T20:19:26.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.514+0000 7f11fe7fc700 1 -- 172.21.15.148:0/4042504878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11f0055280 con 0x7f120015b9a0 2023-10-10T20:19:26.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 msgr2=0x7f11ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 0x7f11ec0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11f8007fd0 tx=0x7f11f8007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:26.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 msgr2=0x7f12000b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11f000d890 tx=0x7f11f0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:26.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 shutdown_connections 2023-10-10T20:19:26.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11ec066f30 0x7f11ec0693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f120015b9a0 0x7f12000b29e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f120006d370 0x7f120014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 --2- 172.21.15.148:0/4042504878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1200152300 0x7f12000b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 >> 172.21.15.148:0/4042504878 conn(0x7f1200069870 msgr2=0x7f12000cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 shutdown_connections 2023-10-10T20:19:26.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.516+0000 7f1208483700 1 -- 172.21.15.148:0/4042504878 wait complete. 2023-10-10T20:19:26.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:26.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.645+0000 7ff67f288700 1 Processor -- start 2023-10-10T20:19:26.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.645+0000 7ff67f288700 1 -- start start 2023-10-10T20:19:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67f288700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff678159be0 0x7ff678068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67f288700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff6780b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67f288700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff678067800 con 0x7ff678159fb0 2023-10-10T20:19:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67f288700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6780701f0 con 0x7ff678159be0 2023-10-10T20:19:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67f288700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff678067ae0 con 0x7ff6780cf060 2023-10-10T20:19:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67da85700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff678159fb0 0x7ff6780ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48200/0 (socket says 172.21.15.148:48200) 2023-10-10T20:19:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67da85700 1 -- 172.21.15.148:0/3452444887 learned_addr learned my addr 172.21.15.148:0/3452444887 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:26.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67ea87700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff6780b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67e286700 1 --2- 172.21.15.148:0/3452444887 >> v2:172.21.15.161:3300/0 conn(0x7ff678159be0 0x7ff678068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325595552 0 0) 0x7ff678067800 con 0x7ff678159fb0 2023-10-10T20:19:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.646+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff664003570 con 0x7ff678159fb0 2023-10-10T20:19:26.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291791446 0 0) 0x7ff664003570 con 0x7ff678159fb0 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 msgr2=0x7ff6780b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff6780b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 >> v2:172.21.15.161:3300/0 conn(0x7ff678159be0 msgr2=0x7ff678068db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 --2- 172.21.15.148:0/3452444887 >> v2:172.21.15.161:3300/0 conn(0x7ff678159be0 0x7ff678068db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff678067dc0 con 0x7ff678159fb0 2023-10-10T20:19:26.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7ff668001470 con 0x7ff678159fb0 2023-10-10T20:19:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 >> v1:172.21.15.148:6789/0 conn(0x7ff678159fb0 legacy=0x7ff6780ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff664003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff664003fe0 0x7ff664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780701f0 con 0x7ff664004d50 2023-10-10T20:19:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678067ae0 con 0x7ff664003fe0 2023-10-10T20:19:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400ba90 con 0x7ff6780cf060 2023-10-10T20:19:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67da85700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff664003fe0 0x7ff664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67e286700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff664003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 -- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 msgr2=0x7ff664003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6780cf060 0x7ff664003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 -- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff664003fe0 msgr2=0x7ff664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff664003fe0 0x7ff664004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 -- 172.21.15.148:0/3452444887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664003570 con 0x7ff664004d50 2023-10-10T20:19:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67ea87700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6640051c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e210 tx=0x7ff67000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5377b3cfe19ec657 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:26.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000ea60 con 0x7ff664004d50 2023-10-10T20:19:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67000f040 con 0x7ff664004d50 2023-10-10T20:19:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.647+0000 7ff67d284700 1 -- 172.21.15.148:0/3452444887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000ee10 con 0x7ff664004d50 2023-10-10T20:19:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 -- 172.21.15.148:0/3452444887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 msgr2=0x7ff6640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 --2- 172.21.15.148:0/3452444887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6640051c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e210 tx=0x7ff67000b040 comp rx=0 tx=0).stop 2023-10-10T20:19:26.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67ea87700 1 -- 172.21.15.148:0/3452444887 reap_dead start 2023-10-10T20:19:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 -- 172.21.15.148:0/3452444887 shutdown_connections 2023-10-10T20:19:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 -- 172.21.15.148:0/3452444887 >> 172.21.15.148:0/3452444887 conn(0x7ff67806bc70 msgr2=0x7ff67806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 -- 172.21.15.148:0/3452444887 shutdown_connections 2023-10-10T20:19:26.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.648+0000 7ff67f288700 1 -- 172.21.15.148:0/3452444887 wait complete. 2023-10-10T20:19:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.649+0000 7ff67f288700 1 Processor -- start 2023-10-10T20:19:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.649+0000 7ff67f288700 1 -- start start 2023-10-10T20:19:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff67815c6d0 0x7ff6781569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 0x7ff67816ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678157360 con 0x7ff664004d50 2023-10-10T20:19:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678155820 con 0x7ff678156ef0 2023-10-10T20:19:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67f288700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780b30a0 con 0x7ff67815c6d0 2023-10-10T20:19:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49812/0 (socket says 172.21.15.148:49812) 2023-10-10T20:19:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 -- 172.21.15.148:0/1960637175 learned_addr learned my addr 172.21.15.148:0/1960637175 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67da85700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff67815c6d0 0x7ff6781569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67ea87700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 0x7ff67816ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 -- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff67815c6d0 msgr2=0x7ff6781569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff67815c6d0 0x7ff6781569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 -- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 msgr2=0x7ff67816ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 0x7ff67816ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 -- 172.21.15.148:0/1960637175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67815cb40 con 0x7ff664004d50 2023-10-10T20:19:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67ea87700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 0x7ff67816ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff67e286700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff674011b10 tx=0x7ff6740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.650+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400f930 con 0x7ff664004d50 2023-10-10T20:19:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.651+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400fa90 con 0x7ff664004d50 2023-10-10T20:19:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.651+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67815cdb0 con 0x7ff664004d50 2023-10-10T20:19:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.651+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400fe10 con 0x7ff664004d50 2023-10-10T20:19:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.651+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6780b2a50 con 0x7ff664004d50 2023-10-10T20:19:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.652+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67806c990 con 0x7ff664004d50 2023-10-10T20:19:26.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.652+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff674014650 con 0x7ff664004d50 2023-10-10T20:19:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.652+0000 7ff66e7fc700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.652+0000 7ff67da85700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:26.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.652+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff674018030 con 0x7ff664004d50 2023-10-10T20:19:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.653+0000 7ff67da85700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff668000fd0 tx=0x7ff668009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:26.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.656+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6740521b0 con 0x7ff664004d50 2023-10-10T20:19:26.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.883+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67815d1f0 con 0x7ff664004d50 2023-10-10T20:19:26.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.884+0000 7ff66e7fc700 1 -- 172.21.15.148:0/1960637175 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7ff6740557d0 con 0x7ff664004d50 2023-10-10T20:19:26.885 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:26.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 msgr2=0x7ff65c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff668000fd0 tx=0x7ff668009040 comp rx=0 tx=0).stop 2023-10-10T20:19:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 msgr2=0x7ff6780b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff674011b10 tx=0x7ff6740091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 shutdown_connections 2023-10-10T20:19:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff678156ef0 0x7ff67816ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff664004d50 0x7ff6780b3530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 --2- 172.21.15.148:0/1960637175 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff67815c6d0 0x7ff6781569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 >> 172.21.15.148:0/1960637175 conn(0x7ff67806bc70 msgr2=0x7ff67806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 shutdown_connections 2023-10-10T20:19:26.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:26.886+0000 7ff67f288700 1 -- 172.21.15.148:0/1960637175 wait complete. 2023-10-10T20:19:26.890 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:26.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:27.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.012+0000 7fe2cf497700 1 Processor -- start 2023-10-10T20:19:27.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.012+0000 7fe2cf497700 1 -- start start 2023-10-10T20:19:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cf497700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 0x7fe2c80b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cf497700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2c8159e40 0x7fe2c815d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cf497700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c815e890 con 0x7fe2c80caf10 2023-10-10T20:19:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cf497700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c815a2e0 con 0x7fe2c8159e40 2023-10-10T20:19:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cf497700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c815e2d0 con 0x7fe2c8159a70 2023-10-10T20:19:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2c8159e40 0x7fe2c815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cec96700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe2c80caf10 0x7fe2c80cb310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48208/0 (socket says 172.21.15.148:48208) 2023-10-10T20:19:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2c8159e40 0x7fe2c815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49814/0 (socket says 172.21.15.148:49814) 2023-10-10T20:19:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cec96700 1 -- 172.21.15.148:0/2305865653 learned_addr learned my addr 172.21.15.148:0/2305865653 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2ce495700 1 --2- 172.21.15.148:0/2305865653 >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 0x7fe2c80b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 -- 172.21.15.148:0/2305865653 >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 msgr2=0x7fe2c80b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 --2- 172.21.15.148:0/2305865653 >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 0x7fe2c80b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 -- 172.21.15.148:0/2305865653 >> v1:172.21.15.148:6789/0 conn(0x7fe2c80caf10 legacy=0x7fe2c80cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cd493700 1 -- 172.21.15.148:0/2305865653 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465855350 0 0) 0x7fe2c815e890 con 0x7fe2c80caf10 2023-10-10T20:19:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2cdc94700 1 -- 172.21.15.148:0/2305865653 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c815e5b0 con 0x7fe2c8159e40 2023-10-10T20:19:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.013+0000 7fe2ce495700 1 --2- 172.21.15.148:0/2305865653 >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 0x7fe2c80b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cdc94700 1 --2- 172.21.15.148:0/2305865653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159e40 0x7fe2c815d960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806e8f0 tx=0x7fe2b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bff74ae708ebcf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cd493700 1 -- 172.21.15.148:0/2305865653 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800b6b0 con 0x7fe2c8159e40 2023-10-10T20:19:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cd493700 1 -- 172.21.15.148:0/2305865653 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800b810 con 0x7fe2c8159e40 2023-10-10T20:19:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cd493700 1 -- 172.21.15.148:0/2305865653 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800bbb0 con 0x7fe2c8159e40 2023-10-10T20:19:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 -- 172.21.15.148:0/2305865653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159e40 msgr2=0x7fe2c815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 --2- 172.21.15.148:0/2305865653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159e40 0x7fe2c815d960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806e8f0 tx=0x7fe2b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 -- 172.21.15.148:0/2305865653 shutdown_connections 2023-10-10T20:19:27.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 --2- 172.21.15.148:0/2305865653 >> v2:172.21.15.161:3300/0 conn(0x7fe2c8159a70 0x7fe2c80b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 --2- 172.21.15.148:0/2305865653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159e40 0x7fe2c815d960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 -- 172.21.15.148:0/2305865653 >> 172.21.15.148:0/2305865653 conn(0x7fe2c806bc70 msgr2=0x7fe2c806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 -- 172.21.15.148:0/2305865653 shutdown_connections 2023-10-10T20:19:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.014+0000 7fe2cf497700 1 -- 172.21.15.148:0/2305865653 wait complete. 2023-10-10T20:19:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.015+0000 7fe2cf497700 1 Processor -- start 2023-10-10T20:19:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.015+0000 7fe2cf497700 1 -- start start 2023-10-10T20:19:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2c80caf10 0x7fe2c80ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2c8159e40 0x7fe2c80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80cda70 con 0x7fe2c8159a70 2023-10-10T20:19:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80ce870 con 0x7fe2c80caf10 2023-10-10T20:19:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cf497700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80cdc60 con 0x7fe2c8159e40 2023-10-10T20:19:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cdc94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cdc94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49820/0 (socket says 172.21.15.148:49820) 2023-10-10T20:19:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cdc94700 1 -- 172.21.15.148:0/870338066 learned_addr learned my addr 172.21.15.148:0/870338066 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cec96700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2c8159e40 0x7fe2c80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2ce495700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2c80caf10 0x7fe2c80ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.016+0000 7fe2cdc94700 1 -- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2c8159e40 msgr2=0x7fe2c80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cdc94700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2c8159e40 0x7fe2c80cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cdc94700 1 -- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2c80caf10 msgr2=0x7fe2c80ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cdc94700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2c80caf10 0x7fe2c80ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cdc94700 1 -- 172.21.15.148:0/870338066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8009420 con 0x7fe2c8159a70 2023-10-10T20:19:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cdc94700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806eba0 tx=0x7fe2b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800bb90 con 0x7fe2c8159a70 2023-10-10T20:19:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b8011420 con 0x7fe2c8159a70 2023-10-10T20:19:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c80cdf40 con 0x7fe2c8159a70 2023-10-10T20:19:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80117e0 con 0x7fe2c8159a70 2023-10-10T20:19:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.017+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c80ce500 con 0x7fe2c8159a70 2023-10-10T20:19:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.018+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c806e120 con 0x7fe2c8159a70 2023-10-10T20:19:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.018+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2b8011c10 con 0x7fe2c8159a70 2023-10-10T20:19:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.018+0000 7fe2be7fc700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.019+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2b8085fb0 con 0x7fe2c8159a70 2023-10-10T20:19:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.019+0000 7fe2ce495700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.019+0000 7fe2ce495700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d560 tx=0x7fe2c40075a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.023+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2b80b75a0 con 0x7fe2c8159a70 2023-10-10T20:19:27.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.206+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2c80c9700 con 0x7fe2c8159a70 2023-10-10T20:19:27.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.206+0000 7fe2be7fc700 1 -- 172.21.15.148:0/870338066 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fe2b8053220 con 0x7fe2c8159a70 2023-10-10T20:19:27.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:27.208 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:27.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 msgr2=0x7fe2ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d560 tx=0x7fe2c40075a0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 msgr2=0x7fe2c80c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2c806eba0 tx=0x7fe2b8017000 comp rx=0 tx=0).stop 2023-10-10T20:19:27.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 shutdown_connections 2023-10-10T20:19:27.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2c8159e40 0x7fe2c80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2c80caf10 0x7fe2c80ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 --2- 172.21.15.148:0/870338066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8159a70 0x7fe2c80c6620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 >> 172.21.15.148:0/870338066 conn(0x7fe2c806bc70 msgr2=0x7fe2c80681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 shutdown_connections 2023-10-10T20:19:27.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.208+0000 7fe2cf497700 1 -- 172.21.15.148:0/870338066 wait complete. 2023-10-10T20:19:27.221 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:19:27.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:27.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.335+0000 7ffa7f922700 1 Processor -- start 2023-10-10T20:19:27.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 -- start start 2023-10-10T20:19:27.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa781510f0 0x7ffa7806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa78070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa78071080 con 0x7ffa780c9af0 2023-10-10T20:19:27.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa780ca0b0 con 0x7ffa781510f0 2023-10-10T20:19:27.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7f922700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa78068a10 con 0x7ffa78070700 2023-10-10T20:19:27.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7d6be700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa781510f0 0x7ffa7806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7d6be700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa781510f0 0x7ffa7806ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49828/0 (socket says 172.21.15.148:49828) 2023-10-10T20:19:27.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7cebd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffa780c9af0 0x7ffa7815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48218/0 (socket says 172.21.15.148:48218) 2023-10-10T20:19:27.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.336+0000 7ffa7d6be700 1 -- 172.21.15.148:0/444068120 learned_addr learned my addr 172.21.15.148:0/444068120 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:27.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7debf700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa78070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7d6be700 1 -- 172.21.15.148:0/444068120 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 msgr2=0x7ffa78070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:27.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7d6be700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa78070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7d6be700 1 -- 172.21.15.148:0/444068120 >> v1:172.21.15.148:6789/0 conn(0x7ffa780c9af0 legacy=0x7ffa7815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7d6be700 1 -- 172.21.15.148:0/444068120 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78068cf0 con 0x7ffa781510f0 2023-10-10T20:19:27.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa6ffff700 1 -- 172.21.15.148:0/444068120 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621867752 0 0) 0x7ffa78071080 con 0x7ffa780c9af0 2023-10-10T20:19:27.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa7d6be700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7806ecc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa740099b0 tx=0x7ffa74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf7f82b4dd64d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa6ffff700 1 -- 172.21.15.148:0/444068120 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74009ea0 con 0x7ffa781510f0 2023-10-10T20:19:27.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa6ffff700 1 -- 172.21.15.148:0/444068120 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7400b860 con 0x7ffa781510f0 2023-10-10T20:19:27.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.337+0000 7ffa6ffff700 1 -- 172.21.15.148:0/444068120 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400bc00 con 0x7ffa781510f0 2023-10-10T20:19:27.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 -- 172.21.15.148:0/444068120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 msgr2=0x7ffa7806ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7806ecc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa740099b0 tx=0x7ffa74009250 comp rx=0 tx=0).stop 2023-10-10T20:19:27.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 -- 172.21.15.148:0/444068120 shutdown_connections 2023-10-10T20:19:27.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa78070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 --2- 172.21.15.148:0/444068120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7806ecc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 -- 172.21.15.148:0/444068120 >> 172.21.15.148:0/444068120 conn(0x7ffa780ca4f0 msgr2=0x7ffa780ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 -- 172.21.15.148:0/444068120 shutdown_connections 2023-10-10T20:19:27.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.338+0000 7ffa7f922700 1 -- 172.21.15.148:0/444068120 wait complete. 2023-10-10T20:19:27.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.339+0000 7ffa7f922700 1 Processor -- start 2023-10-10T20:19:27.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.339+0000 7ffa7f922700 1 -- start start 2023-10-10T20:19:27.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa780ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 0x7ffa7814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7814f500 con 0x7ffa781510f0 2023-10-10T20:19:27.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7814f7e0 con 0x7ffa780c9af0 2023-10-10T20:19:27.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7814fac0 con 0x7ffa78070700 2023-10-10T20:19:27.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49832/0 (socket says 172.21.15.148:49832) 2023-10-10T20:19:27.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 -- 172.21.15.148:0/1191988394 learned_addr learned my addr 172.21.15.148:0/1191988394 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7d6be700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa780ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7cebd700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 0x7ffa7814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 -- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 msgr2=0x7ffa780ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa780ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 -- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 msgr2=0x7ffa7814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 0x7ffa7814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 -- 172.21.15.148:0/1191988394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa740094f0 con 0x7ffa781510f0 2023-10-10T20:19:27.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7cebd700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 0x7ffa7814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:27.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7debf700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009e90 tx=0x7ffa7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000cc20 con 0x7ffa781510f0 2023-10-10T20:19:27.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7000f040 con 0x7ffa781510f0 2023-10-10T20:19:27.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7814fda0 con 0x7ffa781510f0 2023-10-10T20:19:27.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70014480 con 0x7ffa781510f0 2023-10-10T20:19:27.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.340+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa781542a0 con 0x7ffa781510f0 2023-10-10T20:19:27.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.341+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa780b2620 con 0x7ffa781510f0 2023-10-10T20:19:27.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.342+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa70014830 con 0x7ffa781510f0 2023-10-10T20:19:27.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.342+0000 7ffa6d7fa700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 0x7ffa5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.342+0000 7ffa7d6be700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 0x7ffa5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.342+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa70084fa0 con 0x7ffa781510f0 2023-10-10T20:19:27.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.342+0000 7ffa7d6be700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 0x7ffa5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa740099b0 tx=0x7ffa74017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.346+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa70055ca0 con 0x7ffa781510f0 2023-10-10T20:19:27.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.526+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa7804c3c0 con 0x7ffa5c066f30 2023-10-10T20:19:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.533+0000 7ffa6d7fa700 1 -- 172.21.15.148:0/1191988394 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251342 (secure 0 0 0) 0x7ffa7804c3c0 con 0x7ffa5c066f30 2023-10-10T20:19:27.537 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:27.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 msgr2=0x7ffa5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 0x7ffa5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa740099b0 tx=0x7ffa74017000 comp rx=0 tx=0).stop 2023-10-10T20:19:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 msgr2=0x7ffa7814d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009e90 tx=0x7ffa7000b040 comp rx=0 tx=0).stop 2023-10-10T20:19:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 shutdown_connections 2023-10-10T20:19:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffa5c066f30 0x7ffa5c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa781510f0 0x7ffa7814d890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa78070700 0x7ffa780ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 --2- 172.21.15.148:0/1191988394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa780c9af0 0x7ffa7814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.536+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 >> 172.21.15.148:0/1191988394 conn(0x7ffa780ca4f0 msgr2=0x7ffa7815ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.537+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 shutdown_connections 2023-10-10T20:19:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.537+0000 7ffa7f922700 1 -- 172.21.15.148:0/1191988394 wait complete. 2023-10-10T20:19:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:27.549 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-10T20:19:27.323359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1528,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560971,"statfs":{"total":1600000000,"available":1598435066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.850780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475532+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475305+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474447+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475233+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475266+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477677+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474855+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477987+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475685+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477858+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476194+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475565+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475893+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477214+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476018+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.478095+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474632+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475705+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477572+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477697+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476900+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-10T20:19:14.273312+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:14.273312+0000","last_peered":"2023-10-10T20:19:14.273312+0000","last_clean":"2023-10-10T20:19:14.273312+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:14.273312+0000","last_undegraded":"2023-10-10T20:19:14.273312+0000","last_fullsized":"2023-10-10T20:19:14.273312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474681+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475107+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475743+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475605+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474417+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476726+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476343+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474372+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477780+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475747+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477723+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":43,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195269,"statfs":{"total":200000000,"available":199955636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:27.555 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:27.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.703+0000 7f7345e31700 1 Processor -- start 2023-10-10T20:19:27.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.703+0000 7f7345e31700 1 -- start start 2023-10-10T20:19:27.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.704+0000 7f7345e31700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 0x7f734015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.704+0000 7f7345e31700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.704+0000 7f7345e31700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734015cbb0 con 0x7f734015d580 2023-10-10T20:19:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.704+0000 7f7345e31700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734015bfa0 con 0x7f7340157870 2023-10-10T20:19:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.704+0000 7f7345e31700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734015c280 con 0x7f734015a620 2023-10-10T20:19:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 0x7f734015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 0x7f734015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:58330/0 (socket says 172.21.15.148:58330) 2023-10-10T20:19:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:57342/0 (socket says 172.21.15.148:57342) 2023-10-10T20:19:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733f7fe700 1 -- 172.21.15.148:0/770247758 learned_addr learned my addr 172.21.15.148:0/770247758 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733ffff700 1 --1- 172.21.15.148:0/770247758 >> v1:172.21.15.161:6789/0 conn(0x7f734015d580 0x7f7340158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:60196/0 (socket says 172.21.15.148:60196) 2023-10-10T20:19:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 -- 172.21.15.148:0/770247758 >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 msgr2=0x7f734015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 --2- 172.21.15.148:0/770247758 >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 0x7f734015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 -- 172.21.15.148:0/770247758 >> v1:172.21.15.161:6789/0 conn(0x7f734015d580 legacy=0x7f7340158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 -- 172.21.15.148:0/770247758 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734015c560 con 0x7f734015a620 2023-10-10T20:19:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733effd700 1 --2- 172.21.15.148:0/770247758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73340098e0 tx=0x7f73340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f237ee08a65368 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733e7fc700 1 -- 172.21.15.148:0/770247758 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400b640 con 0x7f734015a620 2023-10-10T20:19:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733e7fc700 1 -- 172.21.15.148:0/770247758 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733400b7a0 con 0x7f734015a620 2023-10-10T20:19:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.705+0000 7f733e7fc700 1 -- 172.21.15.148:0/770247758 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400bb40 con 0x7f734015a620 2023-10-10T20:19:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 -- 172.21.15.148:0/770247758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 msgr2=0x7f734015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 --2- 172.21.15.148:0/770247758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73340098e0 tx=0x7f73340091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 -- 172.21.15.148:0/770247758 shutdown_connections 2023-10-10T20:19:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 --2- 172.21.15.148:0/770247758 >> v2:172.21.15.161:3300/0 conn(0x7f7340157870 0x7f734015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 --2- 172.21.15.148:0/770247758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015a620 0x7f734015d040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 -- 172.21.15.148:0/770247758 >> 172.21.15.148:0/770247758 conn(0x7f7340068ce0 msgr2=0x7f73400690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 -- 172.21.15.148:0/770247758 shutdown_connections 2023-10-10T20:19:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.706+0000 7f7345e31700 1 -- 172.21.15.148:0/770247758 wait complete. 2023-10-10T20:19:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.707+0000 7f7345e31700 1 Processor -- start 2023-10-10T20:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.707+0000 7f7345e31700 1 -- start start 2023-10-10T20:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f734015a620 0x7f734016bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015d580 0x7f7340152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73401510f0 con 0x7f7340157870 2023-10-10T20:19:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7340150800 con 0x7f734015a620 2023-10-10T20:19:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f7345e31700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7340150aa0 con 0x7f734015d580 2023-10-10T20:19:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49842/0 (socket says 172.21.15.148:49842) 2023-10-10T20:19:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 -- 172.21.15.148:0/3860185095 learned_addr learned my addr 172.21.15.148:0/3860185095 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733effd700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f734015a620 0x7f734016bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733ffff700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015d580 0x7f7340152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 -- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015d580 msgr2=0x7f7340152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015d580 0x7f7340152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 -- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f734015a620 msgr2=0x7f734016bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f734015a620 0x7f734016bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 -- 172.21.15.148:0/3860185095 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009420 con 0x7f7340157870 2023-10-10T20:19:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f733f7fe700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7330009e30 tx=0x7f733000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73300092e0 con 0x7f7340157870 2023-10-10T20:19:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.708+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733000f040 con 0x7f7340157870 2023-10-10T20:19:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.709+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7340150d80 con 0x7f7340157870 2023-10-10T20:19:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.709+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000caa0 con 0x7f7340157870 2023-10-10T20:19:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.709+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734016b8b0 con 0x7f7340157870 2023-10-10T20:19:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.709+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73401513d0 con 0x7f7340157870 2023-10-10T20:19:27.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.710+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7330014480 con 0x7f7340157870 2023-10-10T20:19:27.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.710+0000 7f73277fe700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 0x7f732c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:27.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.710+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7330084b80 con 0x7f7340157870 2023-10-10T20:19:27.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.710+0000 7f733effd700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 0x7f732c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:27.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.711+0000 7f733effd700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 0x7f732c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f734015aa20 tx=0x7f7334017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:27.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.714+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7330051d50 con 0x7f7340157870 2023-10-10T20:19:27.783 INFO:tasks.ceph.mds.d.smithi161.stderr:2023-10-10T20:19:27.782+0000 7f890359e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:19:27.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.892+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7340151230 con 0x7f732c066f30 2023-10-10T20:19:27.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.899+0000 7f73277fe700 1 -- 172.21.15.148:0/3860185095 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251342 (secure 0 0 0) 0x7f7334006c50 con 0x7f732c066f30 2023-10-10T20:19:27.903 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 msgr2=0x7f732c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 0x7f732c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f734015aa20 tx=0x7f7334017000 comp rx=0 tx=0).stop 2023-10-10T20:19:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 msgr2=0x7f734016b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7330009e30 tx=0x7f733000b040 comp rx=0 tx=0).stop 2023-10-10T20:19:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 shutdown_connections 2023-10-10T20:19:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f732c066f30 0x7f732c0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f734015d580 0x7f7340152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7340157870 0x7f734016b370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 --2- 172.21.15.148:0/3860185095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f734015a620 0x7f734016bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 >> 172.21.15.148:0/3860185095 conn(0x7f7340068ce0 msgr2=0x7f73400690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 shutdown_connections 2023-10-10T20:19:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:27.902+0000 7f7345e31700 1 -- 172.21.15.148:0/3860185095 wait complete. 2023-10-10T20:19:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:27.915 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-10T20:19:27.323359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1528,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560971,"statfs":{"total":1600000000,"available":1598435066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.850780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475532+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475305+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474447+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475233+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475266+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477677+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474855+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477987+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475685+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477858+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476194+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475565+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475893+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477214+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476018+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.478095+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474632+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475705+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477572+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477697+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476900+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-10T20:19:14.273312+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:14.273312+0000","last_peered":"2023-10-10T20:19:14.273312+0000","last_clean":"2023-10-10T20:19:14.273312+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:14.273312+0000","last_undegraded":"2023-10-10T20:19:14.273312+0000","last_fullsized":"2023-10-10T20:19:14.273312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474681+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475107+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475743+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475605+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474417+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476726+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476343+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474372+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477780+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475747+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477723+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":43,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195269,"statfs":{"total":200000000,"available":199955636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:27.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:28.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.070+0000 7f6ac793e700 1 Processor -- start 2023-10-10T20:19:28.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.070+0000 7f6ac793e700 1 -- start start 2023-10-10T20:19:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac793e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ac0157c40 0x7f6ac0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac793e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac015cbb0 con 0x7f6ac015a620 2023-10-10T20:19:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac793e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac015bfa0 con 0x7f6ac0157870 2023-10-10T20:19:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac793e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac015bdb0 con 0x7f6ac0157c40 2023-10-10T20:19:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac4ed9700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6ac015a620 0x7f6ac015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48230/0 (socket says 172.21.15.148:48230) 2023-10-10T20:19:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac4ed9700 1 -- 172.21.15.148:0/3443779654 learned_addr learned my addr 172.21.15.148:0/3443779654 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.071+0000 7f6ac5edb700 1 --2- 172.21.15.148:0/3443779654 >> v2:172.21.15.161:3300/0 conn(0x7f6ac0157c40 0x7f6ac0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489917566 0 0) 0x7f6ac015cbb0 con 0x7f6ac015a620 2023-10-10T20:19:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aac003570 con 0x7f6ac015a620 2023-10-10T20:19:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054095561 0 0) 0x7f6ac015bfa0 con 0x7f6ac0157870 2023-10-10T20:19:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac015cbb0 con 0x7f6ac0157870 2023-10-10T20:19:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ac5edb700 1 -- 172.21.15.148:0/3443779654 >> v1:172.21.15.161:6789/0 conn(0x7f6ac0157870 legacy=0x7f6ac015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ac5edb700 1 -- 172.21.15.148:0/3443779654 >> v1:172.21.15.148:6789/0 conn(0x7f6ac015a620 legacy=0x7f6ac015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ac5edb700 1 -- 172.21.15.148:0/3443779654 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac015c280 con 0x7f6ac0157c40 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ac5edb700 1 --2- 172.21.15.148:0/3443779654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157c40 0x7f6ac0158110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009e60 tx=0x7f6ab8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db96874580d2580 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800e070 con 0x7f6ac0157c40 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab800bab0 con 0x7f6ac0157c40 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.072+0000 7f6ab7fff700 1 -- 172.21.15.148:0/3443779654 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800be70 con 0x7f6ac0157c40 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 -- 172.21.15.148:0/3443779654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157c40 msgr2=0x7f6ac0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 --2- 172.21.15.148:0/3443779654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157c40 0x7f6ac0158110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009e60 tx=0x7f6ab8009780 comp rx=0 tx=0).stop 2023-10-10T20:19:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 -- 172.21.15.148:0/3443779654 shutdown_connections 2023-10-10T20:19:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 --2- 172.21.15.148:0/3443779654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157c40 0x7f6ac0158110 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 -- 172.21.15.148:0/3443779654 >> 172.21.15.148:0/3443779654 conn(0x7f6ac0068ce0 msgr2=0x7f6ac00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:28.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 -- 172.21.15.148:0/3443779654 shutdown_connections 2023-10-10T20:19:28.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.073+0000 7f6ac793e700 1 -- 172.21.15.148:0/3443779654 wait complete. 2023-10-10T20:19:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.074+0000 7f6ac793e700 1 Processor -- start 2023-10-10T20:19:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.074+0000 7f6ac793e700 1 -- start start 2023-10-10T20:19:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 0x7f6ac0132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6ac015a620 0x7f6ac014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac00b2f20 con 0x7f6ac0157c40 2023-10-10T20:19:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0151a90 con 0x7f6ac0157870 2023-10-10T20:19:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac793e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0151d70 con 0x7f6ac015a620 2023-10-10T20:19:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49852/0 (socket says 172.21.15.148:49852) 2023-10-10T20:19:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 -- 172.21.15.148:0/1493124379 learned_addr learned my addr 172.21.15.148:0/1493124379 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac56da700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 0x7f6ac0132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac5edb700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6ac015a620 0x7f6ac014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 -- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6ac015a620 msgr2=0x7f6ac014d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6ac015a620 0x7f6ac014d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 -- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 msgr2=0x7f6ac0132620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 0x7f6ac0132620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 -- 172.21.15.148:0/1493124379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8009a20 con 0x7f6ac0157c40 2023-10-10T20:19:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac56da700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 0x7f6ac0132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ac4ed9700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c580 tx=0x7f6ab0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.075+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0005b30 con 0x7f6ac0157c40 2023-10-10T20:19:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.076+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0005c90 con 0x7f6ac0157c40 2023-10-10T20:19:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.076+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac0152050 con 0x7f6ac0157c40 2023-10-10T20:19:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.076+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0007300 con 0x7f6ac0157c40 2023-10-10T20:19:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.076+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac0132b60 con 0x7f6ac0157c40 2023-10-10T20:19:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.076+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac004c3c0 con 0x7f6ac0157c40 2023-10-10T20:19:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.077+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ab001b3b0 con 0x7f6ac0157c40 2023-10-10T20:19:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.077+0000 7f6ab57fa700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 0x7f6aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.077+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ab0083c00 con 0x7f6ac0157c40 2023-10-10T20:19:28.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.077+0000 7f6ac56da700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 0x7f6aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.078+0000 7f6ac56da700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 0x7f6aa40693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6abc003990 tx=0x7f6abc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.082+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ab0050dd0 con 0x7f6ac0157c40 2023-10-10T20:19:28.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.258+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ac0151c00 con 0x7f6aa4066f30 2023-10-10T20:19:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.265+0000 7f6ab57fa700 1 -- 172.21.15.148:0/1493124379 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251342 (secure 0 0 0) 0x7f6ac0151c00 con 0x7f6aa4066f30 2023-10-10T20:19:28.275 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 msgr2=0x7f6aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 0x7f6aa40693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6abc003990 tx=0x7f6abc009040 comp rx=0 tx=0).stop 2023-10-10T20:19:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 msgr2=0x7f6ac00b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c580 tx=0x7f6ab0007e40 comp rx=0 tx=0).stop 2023-10-10T20:19:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 shutdown_connections 2023-10-10T20:19:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6aa4066f30 0x7f6aa40693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6ac015a620 0x7f6ac014d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ac0157870 0x7f6ac0132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 --2- 172.21.15.148:0/1493124379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ac0157c40 0x7f6ac00b25b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 >> 172.21.15.148:0/1493124379 conn(0x7f6ac0068ce0 msgr2=0x7f6ac00b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 shutdown_connections 2023-10-10T20:19:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.268+0000 7f6ac793e700 1 -- 172.21.15.148:0/1493124379 wait complete. 2023-10-10T20:19:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:28.282 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-10T20:19:27.323359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1528,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560971,"statfs":{"total":1600000000,"available":1598435066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.850780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475532+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475305+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474447+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475233+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475266+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477677+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474855+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477987+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475685+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477858+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476194+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475565+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475893+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477214+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476018+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.478095+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474632+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475705+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477572+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477697+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476900+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-10T20:19:14.273312+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:14.273312+0000","last_peered":"2023-10-10T20:19:14.273312+0000","last_clean":"2023-10-10T20:19:14.273312+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:14.273312+0000","last_undegraded":"2023-10-10T20:19:14.273312+0000","last_fullsized":"2023-10-10T20:19:14.273312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474681+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475107+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475743+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475605+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474417+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476726+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476343+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474372+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477780+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475747+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477723+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":43,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195269,"statfs":{"total":200000000,"available":199955636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:28.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-10-10T20:19:28.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-10-10T20:19:28.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-10-10T20:19:28.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-10-10T20:19:28.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-10-10T20:19:28.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-10-10T20:19:28.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-10-10T20:19:28.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-10-10T20:19:28.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-10-10T20:19:28.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-10-10T20:19:28.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-10-10T20:19:28.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-10-10T20:19:28.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-10-10T20:19:28.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-10-10T20:19:28.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-10-10T20:19:28.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-10-10T20:19:28.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-10-10T20:19:28.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-10-10T20:19:28.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-10-10T20:19:28.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-10-10T20:19:28.305 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475532+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:19:28.305 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-10-10T20:19:28.306 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.306 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2023-10-10T20:19:28.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475305+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:19:28.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-10-10T20:19:28.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-10-10T20:19:28.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474447+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-10-10T20:19:28.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-10-10T20:19:28.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475233+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-10-10T20:19:28.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2023-10-10T20:19:28.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475266+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:19:28.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-10-10T20:19:28.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-10-10T20:19:28.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477677+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:19:28.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-10-10T20:19:28.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2023-10-10T20:19:28.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474855+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:19:28.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-10-10T20:19:28.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-10-10T20:19:28.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477987+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:19:28.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-10-10T20:19:28.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-10-10T20:19:28.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475685+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:19:28.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-10-10T20:19:28.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-10-10T20:19:28.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477858+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:19:28.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-10-10T20:19:28.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-10-10T20:19:28.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.476194+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-10-10T20:19:28.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-10-10T20:19:28.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475565+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:19:28.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-10-10T20:19:28.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-10-10T20:19:28.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475893+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:19:28.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-10-10T20:19:28.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-10-10T20:19:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477214+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:19:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-10-10T20:19:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-10-10T20:19:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.476018+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:19:28.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-10-10T20:19:28.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-10-10T20:19:28.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.478095+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:19:28.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-10-10T20:19:28.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-10-10T20:19:28.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474632+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:19:28.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-10-10T20:19:28.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-10-10T20:19:28.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475705+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-10-10T20:19:28.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-10-10T20:19:28.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477572+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:19:28.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-10-10T20:19:28.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-10-10T20:19:28.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-10-10T20:19:28.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477697+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:19:28.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-10-10T20:19:28.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-10-10T20:19:28.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.476900+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:19:28.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-10-10T20:19:28.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-10-10T20:19:28.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474681+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-10-10T20:19:28.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475107+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-10-10T20:19:28.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-10-10T20:19:28.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475743+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-10-10T20:19:28.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-10-10T20:19:28.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475605+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:19:28.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-10-10T20:19:28.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-10-10T20:19:28.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474417+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-10-10T20:19:28.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-10-10T20:19:28.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.476726+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-10-10T20:19:28.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-10-10T20:19:28.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.476343+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-10-10T20:19:28.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-10-10T20:19:28.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.474372+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:19:28.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-10-10T20:19:28.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-10-10T20:19:28.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477780+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:19:28.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-10-10T20:19:28.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-10-10T20:19:28.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.475747+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:19:28.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-10-10T20:19:28.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-10-10T20:19:28.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:19:21.468881+0000', 'last_change': '2023-10-10T20:19:21.477723+0000', 'last_active': '2023-10-10T20:19:21.468881+0000', 'last_peered': '2023-10-10T20:19:21.468881+0000', 'last_clean': '2023-10-10T20:19:21.468881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:21.468881+0000', 'last_undegraded': '2023-10-10T20:19:21.468881+0000', 'last_fullsized': '2023-10-10T20:19:21.468881+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:21.468881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:19:28.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-10-10T20:19:28.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-10-10T20:19:28.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-10-10T20:19:28.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-10-10T20:19:28.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-10-10T20:19:28.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-10-10T20:19:28.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-10-10T20:19:28.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-10-10T20:19:28.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-10-10T20:19:28.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-10-10T20:19:28.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-10-10T20:19:28.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-10-10T20:19:28.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-10-10T20:19:28.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-10-10T20:19:28.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-10T20:19:22.472711+0000', 'last_change': '2023-10-10T20:19:22.472711+0000', 'last_active': '2023-10-10T20:19:22.472711+0000', 'last_peered': '2023-10-10T20:19:22.472711+0000', 'last_clean': '2023-10-10T20:19:22.472711+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:19:22.472711+0000', 'last_undegraded': '2023-10-10T20:19:22.472711+0000', 'last_fullsized': '2023-10-10T20:19:22.472711+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'last_clean_scrub_stamp': '2023-10-10T20:19:22.472711+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-10T20:19:28.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:19:28.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.482+0000 7f363b2df700 1 Processor -- start 2023-10-10T20:19:28.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.482+0000 7f363b2df700 1 -- start start 2023-10-10T20:19:28.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363b2df700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363b2df700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f36340cb4f0 0x7f36340cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363b2df700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363415de60 con 0x7f36340b1220 2023-10-10T20:19:28.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363b2df700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36340cbed0 con 0x7f36340cb4f0 2023-10-10T20:19:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363b2df700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363415e140 con 0x7f36340b0e50 2023-10-10T20:19:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363907b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363987c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f36340cb4f0 0x7f36340cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363887a700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f36340b1220 0x7f363406d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:60208/0 (socket says 172.21.15.148:60208) 2023-10-10T20:19:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363887a700 1 -- 172.21.15.148:0/3234100041 learned_addr learned my addr 172.21.15.148:0/3234100041 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:28.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363987c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f36340cb4f0 0x7f36340cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:58360/0 (socket says 172.21.15.148:58360) 2023-10-10T20:19:28.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.483+0000 7f363907b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:57364/0 (socket says 172.21.15.148:57364) 2023-10-10T20:19:28.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363987c700 1 -- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 msgr2=0x7f36340c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:28.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363987c700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363987c700 1 -- 172.21.15.148:0/3234100041 >> v1:172.21.15.161:6789/0 conn(0x7f36340b1220 legacy=0x7f363406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363987c700 1 -- 172.21.15.148:0/3234100041 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363415e420 con 0x7f36340cb4f0 2023-10-10T20:19:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f362bfff700 1 -- 172.21.15.148:0/3234100041 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210550558 0 0) 0x7f363415de60 con 0x7f36340b1220 2023-10-10T20:19:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363907b700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f363987c700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340cb4f0 0x7f36340cb990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f362c0098e0 tx=0x7f362c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb888f1b16b73e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f362bfff700 1 -- 172.21.15.148:0/3234100041 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00b640 con 0x7f36340cb4f0 2023-10-10T20:19:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f362bfff700 1 -- 172.21.15.148:0/3234100041 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00b7a0 con 0x7f36340cb4f0 2023-10-10T20:19:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.484+0000 7f362bfff700 1 -- 172.21.15.148:0/3234100041 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00bb40 con 0x7f36340cb4f0 2023-10-10T20:19:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 -- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340cb4f0 msgr2=0x7f36340cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340cb4f0 0x7f36340cb990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f362c0098e0 tx=0x7f362c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 -- 172.21.15.148:0/3234100041 shutdown_connections 2023-10-10T20:19:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340cb4f0 0x7f36340cb990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 --2- 172.21.15.148:0/3234100041 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36340c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 -- 172.21.15.148:0/3234100041 >> 172.21.15.148:0/3234100041 conn(0x7f3634069870 msgr2=0x7f3634069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 -- 172.21.15.148:0/3234100041 shutdown_connections 2023-10-10T20:19:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.485+0000 7f363b2df700 1 -- 172.21.15.148:0/3234100041 wait complete. 2023-10-10T20:19:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.486+0000 7f363b2df700 1 Processor -- start 2023-10-10T20:19:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.486+0000 7f363b2df700 1 -- start start 2023-10-10T20:19:28.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36341555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340b1220 0x7f3634158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36340b2810 con 0x7f36340cb4f0 2023-10-10T20:19:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36340b2af0 con 0x7f36340b1220 2023-10-10T20:19:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36340b2dd0 con 0x7f36340b0e50 2023-10-10T20:19:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49868/0 (socket says 172.21.15.148:49868) 2023-10-10T20:19:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 -- 172.21.15.148:0/2532133686 learned_addr learned my addr 172.21.15.148:0/2532133686 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363907b700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36341555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363887a700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340b1220 0x7f3634158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 -- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 msgr2=0x7f36341555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36341555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 -- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340b1220 msgr2=0x7f3634158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340b1220 0x7f3634158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 -- 172.21.15.148:0/2532133686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3630009a20 con 0x7f36340cb4f0 2023-10-10T20:19:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363987c700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f362c0093f0 tx=0x7f362c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00b910 con 0x7f36340cb4f0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00ba70 con 0x7f36340cb4f0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c025670 con 0x7f36340cb4f0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.487+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c009420 con 0x7f36340cb4f0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.488+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36340b3390 con 0x7f36340cb4f0 2023-10-10T20:19:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.488+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36340b35b0 con 0x7f36340cb4f0 2023-10-10T20:19:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.489+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f362c023070 con 0x7f36340cb4f0 2023-10-10T20:19:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.489+0000 7f36297fa700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 0x7f36180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.489+0000 7f363907b700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 0x7f36180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.489+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f362c085a00 con 0x7f36340cb4f0 2023-10-10T20:19:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.490+0000 7f363907b700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 0x7f36180693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fd0 tx=0x7f363000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.494+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f362c052c70 con 0x7f36340cb4f0 2023-10-10T20:19:28.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.754+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f363404c3c0 con 0x7f36340cb4f0 2023-10-10T20:19:28.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.755+0000 7f36297fa700 1 -- 172.21.15.148:0/2532133686 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1102 (secure 0 0 0) 0x7f362c015460 con 0x7f36340cb4f0 2023-10-10T20:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:28.756 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1696969151,"num_in_osds":8,"osd_in_since":1696969148,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1564934,"bytes_avail":1598435066,"bytes_total":1600000000,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4273}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-10T20:19:05.721237+0000","services":{}},"progress_events":{}} 2023-10-10T20:19:28.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 msgr2=0x7f36180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 0x7f36180693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fd0 tx=0x7f363000b040 comp rx=0 tx=0).stop 2023-10-10T20:19:28.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 msgr2=0x7f3634158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f362c0093f0 tx=0x7f362c017000 comp rx=0 tx=0).stop 2023-10-10T20:19:28.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 shutdown_connections 2023-10-10T20:19:28.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3618066f30 0x7f36180693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36340cb4f0 0x7f3634158ea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f36340b0e50 0x7f36341555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 --2- 172.21.15.148:0/2532133686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36340b1220 0x7f3634158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 >> 172.21.15.148:0/2532133686 conn(0x7f3634069870 msgr2=0x7f3634069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 shutdown_connections 2023-10-10T20:19:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.757+0000 7f363b2df700 1 -- 172.21.15.148:0/2532133686 wait complete. 2023-10-10T20:19:28.770 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:19:28.770 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:28.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.883+0000 7fb056471700 1 Processor -- start 2023-10-10T20:19:28.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.883+0000 7fb056471700 1 -- start start 2023-10-10T20:19:28.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb056471700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb056471700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 0x7fb05015b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb056471700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0500ca270 con 0x7fb0500b0160 2023-10-10T20:19:28.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb056471700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0500c87c0 con 0x7fb05006f1a0 2023-10-10T20:19:28.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb056471700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05015b770 con 0x7fb0500afd90 2023-10-10T20:19:28.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb04f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 0x7fb05015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb054a0e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb0500b0160 0x7fb050070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:60210/0 (socket says 172.21.15.148:60210) 2023-10-10T20:19:28.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.884+0000 7fb054a0e700 1 -- 172.21.15.148:0/2399720902 learned_addr learned my addr 172.21.15.148:0/2399720902 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:28.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04effd700 1 -- 172.21.15.148:0/2399720902 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839152608 0 0) 0x7fb0500ca270 con 0x7fb0500b0160 2023-10-10T20:19:28.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04effd700 1 -- 172.21.15.148:0/2399720902 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb038003570 con 0x7fb0500b0160 2023-10-10T20:19:28.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04ffff700 1 -- 172.21.15.148:0/2399720902 >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 msgr2=0x7fb05015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:28.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04ffff700 1 --2- 172.21.15.148:0/2399720902 >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 0x7fb05015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04ffff700 1 -- 172.21.15.148:0/2399720902 >> v1:172.21.15.161:6789/0 conn(0x7fb0500b0160 legacy=0x7fb050070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04ffff700 1 -- 172.21.15.148:0/2399720902 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05006e3e0 con 0x7fb0500afd90 2023-10-10T20:19:28.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04f7fe700 1 --2- 172.21.15.148:0/2399720902 >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 0x7fb05015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:28.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04ffff700 1 --2- 172.21.15.148:0/2399720902 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c9c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098e0 tx=0x7fb0400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=156b29b5075204f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04effd700 1 -- 172.21.15.148:0/2399720902 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011630 con 0x7fb0500afd90 2023-10-10T20:19:28.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04effd700 1 -- 172.21.15.148:0/2399720902 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040011790 con 0x7fb0500afd90 2023-10-10T20:19:28.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.885+0000 7fb04effd700 1 -- 172.21.15.148:0/2399720902 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011b30 con 0x7fb0500afd90 2023-10-10T20:19:28.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 -- 172.21.15.148:0/2399720902 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 msgr2=0x7fb0500c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 --2- 172.21.15.148:0/2399720902 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c9c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098e0 tx=0x7fb0400091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 -- 172.21.15.148:0/2399720902 shutdown_connections 2023-10-10T20:19:28.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 --2- 172.21.15.148:0/2399720902 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c9c20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 --2- 172.21.15.148:0/2399720902 >> v2:172.21.15.161:3300/0 conn(0x7fb05006f1a0 0x7fb05015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 -- 172.21.15.148:0/2399720902 >> 172.21.15.148:0/2399720902 conn(0x7fb0500ca4f0 msgr2=0x7fb0500ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:28.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 -- 172.21.15.148:0/2399720902 shutdown_connections 2023-10-10T20:19:28.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.886+0000 7fb056471700 1 -- 172.21.15.148:0/2399720902 wait complete. 2023-10-10T20:19:28.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.887+0000 7fb056471700 1 Processor -- start 2023-10-10T20:19:28.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.887+0000 7fb056471700 1 -- start start 2023-10-10T20:19:28.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0500b0160 0x7fb05016dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05016ead0 con 0x7fb05006f1a0 2023-10-10T20:19:28.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500c34d0 con 0x7fb0500b0160 2023-10-10T20:19:28.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb056471700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500c28c0 con 0x7fb0500afd90 2023-10-10T20:19:28.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49874/0 (socket says 172.21.15.148:49874) 2023-10-10T20:19:28.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 -- 172.21.15.148:0/2816305554 learned_addr learned my addr 172.21.15.148:0/2816305554 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb054a0e700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0500b0160 0x7fb05016dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04f7fe700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 -- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 msgr2=0x7fb0500c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 -- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0500b0160 msgr2=0x7fb05016dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0500b0160 0x7fb05016dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 -- 172.21.15.148:0/2816305554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009420 con 0x7fb05006f1a0 2023-10-10T20:19:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04f7fe700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.888+0000 7fb04ffff700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0500ca450 tx=0x7fb040017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011b10 con 0x7fb05006f1a0 2023-10-10T20:19:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400253f0 con 0x7fb05006f1a0 2023-10-10T20:19:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb040009420 con 0x7fb05006f1a0 2023-10-10T20:19:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040025790 con 0x7fb05006f1a0 2023-10-10T20:19:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0500c2e80 con 0x7fb05006f1a0 2023-10-10T20:19:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.889+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05004c3c0 con 0x7fb05006f1a0 2023-10-10T20:19:28.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.890+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb040025b80 con 0x7fb05006f1a0 2023-10-10T20:19:28.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.890+0000 7fb037fff700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 0x7fb03c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:28.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.890+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb040085f00 con 0x7fb05006f1a0 2023-10-10T20:19:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.890+0000 7fb04f7fe700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 0x7fb03c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.891+0000 7fb04f7fe700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 0x7fb03c0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb0500cd300 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:28.895+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0400530d0 con 0x7fb05006f1a0 2023-10-10T20:19:29.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.075+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0500b38b0 con 0x7fb03c066f30 2023-10-10T20:19:29.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.081+0000 7fb037fff700 1 -- 172.21.15.148:0/2816305554 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251342 (secure 0 0 0) 0x7fb0500b38b0 con 0x7fb03c066f30 2023-10-10T20:19:29.085 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:29.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 msgr2=0x7fb03c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:29.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 0x7fb03c0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb0500cd300 tx=0x7fb044008040 comp rx=0 tx=0).stop 2023-10-10T20:19:29.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 msgr2=0x7fb05014c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:29.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0500ca450 tx=0x7fb040017000 comp rx=0 tx=0).stop 2023-10-10T20:19:29.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 shutdown_connections 2023-10-10T20:19:29.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb03c066f30 0x7fb03c0693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:29.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0500b0160 0x7fb05016dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:29.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05006f1a0 0x7fb05014c600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:29.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 --2- 172.21.15.148:0/2816305554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0500afd90 0x7fb0500c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:29.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 >> 172.21.15.148:0/2816305554 conn(0x7fb0500ca4f0 msgr2=0x7fb050157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:29.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 shutdown_connections 2023-10-10T20:19:29.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:29.085+0000 7fb056471700 1 -- 172.21.15.148:0/2816305554 wait complete. 2023-10-10T20:19:29.093 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:29.097 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-10T20:19:27.323359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1528,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560971,"statfs":{"total":1600000000,"available":1598435066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.850780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475532+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475305+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474447+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475233+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475266+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477677+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474855+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477987+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475685+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477858+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476194+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475565+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475893+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477214+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476018+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.478095+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474632+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475705+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477572+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477697+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:45:18.762217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476900+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-10T20:19:14.273312+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:14.273312+0000","last_peered":"2023-10-10T20:19:14.273312+0000","last_clean":"2023-10-10T20:19:14.273312+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:14.273312+0000","last_undegraded":"2023-10-10T20:19:14.273312+0000","last_fullsized":"2023-10-10T20:19:14.273312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474681+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475107+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475743+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475605+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474417+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476726+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.476343+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.474372+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477780+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.475747+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:19:21.468881+0000","last_change":"2023-10-10T20:19:21.477723+0000","last_active":"2023-10-10T20:19:21.468881+0000","last_peered":"2023-10-10T20:19:21.468881+0000","last_clean":"2023-10-10T20:19:21.468881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:21.468881+0000","last_undegraded":"2023-10-10T20:19:21.468881+0000","last_fullsized":"2023-10-10T20:19:21.468881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:19:22.472711+0000","last_change":"2023-10-10T20:19:22.472711+0000","last_active":"2023-10-10T20:19:22.472711+0000","last_peered":"2023-10-10T20:19:22.472711+0000","last_clean":"2023-10-10T20:19:22.472711+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:19:22.472711+0000","last_undegraded":"2023-10-10T20:19:22.472711+0000","last_fullsized":"2023-10-10T20:19:22.472711+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359463,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":43,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195269,"statfs":{"total":200000000,"available":199955636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195268,"statfs":{"total":200000000,"available":199954625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:32.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.229+0000 7f36999aa700 1 Processor -- start 2023-10-10T20:19:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.229+0000 7f36999aa700 1 -- start start 2023-10-10T20:19:32.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.230+0000 7f36999aa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.230+0000 7f36999aa700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f36940b04f0 0x7f36940b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.230+0000 7f36999aa700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36940b0e90 con 0x7f369415a8b0 2023-10-10T20:19:32.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.230+0000 7f36999aa700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369415e190 con 0x7f36940b04f0 2023-10-10T20:19:32.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.230+0000 7f36999aa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b23a0 con 0x7f369415e620 2023-10-10T20:19:32.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:57398/0 (socket says 172.21.15.148:57398) 2023-10-10T20:19:32.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f3692ffd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f369415a8b0 0x7f36940b18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:60220/0 (socket says 172.21.15.148:60220) 2023-10-10T20:19:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 -- 172.21.15.148:0/2877815281 learned_addr learned my addr 172.21.15.148:0/2877815281 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f36937fe700 1 --2- 172.21.15.148:0/2877815281 >> v2:172.21.15.161:3300/0 conn(0x7f36940b04f0 0x7f36940b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f36927fc700 1 -- 172.21.15.148:0/2877815281 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928193516 0 0) 0x7f3684001110 con 0x7f369415a8b0 2023-10-10T20:19:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 -- 172.21.15.148:0/2877815281 >> v2:172.21.15.161:3300/0 conn(0x7f36940b04f0 msgr2=0x7f36940b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 --2- 172.21.15.148:0/2877815281 >> v2:172.21.15.161:3300/0 conn(0x7f36940b04f0 0x7f36940b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 -- 172.21.15.148:0/2877815281 >> v1:172.21.15.161:6789/0 conn(0x7f369415a8b0 legacy=0x7f36940b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 -- 172.21.15.148:0/2877815281 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940b20c0 con 0x7f369415e620 2023-10-10T20:19:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f368a7fc700 1 --2- 172.21.15.148:0/2877815281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f367c0099b0 tx=0x7f367c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e910626a411c71d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f36927fc700 1 -- 172.21.15.148:0/2877815281 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c009ea0 con 0x7f369415e620 2023-10-10T20:19:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f36927fc700 1 -- 172.21.15.148:0/2877815281 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c00b860 con 0x7f369415e620 2023-10-10T20:19:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.231+0000 7f36927fc700 1 -- 172.21.15.148:0/2877815281 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c00bc00 con 0x7f369415e620 2023-10-10T20:19:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 -- 172.21.15.148:0/2877815281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 msgr2=0x7f369415ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 --2- 172.21.15.148:0/2877815281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f367c0099b0 tx=0x7f367c009250 comp rx=0 tx=0).stop 2023-10-10T20:19:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 -- 172.21.15.148:0/2877815281 shutdown_connections 2023-10-10T20:19:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 --2- 172.21.15.148:0/2877815281 >> v2:172.21.15.161:3300/0 conn(0x7f36940b04f0 0x7f36940b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 --2- 172.21.15.148:0/2877815281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415e620 0x7f369415ac80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 -- 172.21.15.148:0/2877815281 >> 172.21.15.148:0/2877815281 conn(0x7f369406c700 msgr2=0x7f3694069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 -- 172.21.15.148:0/2877815281 shutdown_connections 2023-10-10T20:19:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.232+0000 7f36999aa700 1 -- 172.21.15.148:0/2877815281 wait complete. 2023-10-10T20:19:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.233+0000 7f36999aa700 1 Processor -- start 2023-10-10T20:19:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.233+0000 7f36999aa700 1 -- start start 2023-10-10T20:19:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36940b04f0 0x7f36940c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415a8b0 0x7f36940adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940c93a0 con 0x7f369415e620 2023-10-10T20:19:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36940c8500 con 0x7f36940b04f0 2023-10-10T20:19:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36999aa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940c87e0 con 0x7f369415a8b0 2023-10-10T20:19:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49880/0 (socket says 172.21.15.148:49880) 2023-10-10T20:19:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 -- 172.21.15.148:0/1134107438 learned_addr learned my addr 172.21.15.148:0/1134107438 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f368a7fc700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415a8b0 0x7f36940adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f3692ffd700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36940b04f0 0x7f36940c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 -- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415a8b0 msgr2=0x7f36940adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415a8b0 0x7f36940adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 -- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36940b04f0 msgr2=0x7f36940c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36940b04f0 0x7f36940c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.234+0000 7f36937fe700 1 -- 172.21.15.148:0/1134107438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c0094f0 con 0x7f369415e620 2023-10-10T20:19:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f36937fe700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e90 tx=0x7f368c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00cb70 con 0x7f369415e620 2023-10-10T20:19:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c00f040 con 0x7f369415e620 2023-10-10T20:19:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c014480 con 0x7f369415e620 2023-10-10T20:19:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36940c8ac0 con 0x7f369415e620 2023-10-10T20:19:32.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.235+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36940c8210 con 0x7f369415e620 2023-10-10T20:19:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.236+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369406e230 con 0x7f369415e620 2023-10-10T20:19:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.236+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f368c014830 con 0x7f369415e620 2023-10-10T20:19:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.237+0000 7f368b7fe700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 0x7f36740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.237+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f368c085000 con 0x7f369415e620 2023-10-10T20:19:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.237+0000 7f3692ffd700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 0x7f36740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.237+0000 7f3692ffd700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 0x7f36740693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3684007fd0 tx=0x7f3684007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.241+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f368c0521d0 con 0x7f369415e620 2023-10-10T20:19:32.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.422+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f369415db40 con 0x7f3674066f30 2023-10-10T20:19:32.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.429+0000 7f368b7fe700 1 -- 172.21.15.148:0/1134107438 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260163 (secure 0 0 0) 0x7f369415db40 con 0x7f3674066f30 2023-10-10T20:19:32.434 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:32.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 msgr2=0x7f36740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 0x7f36740693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3684007fd0 tx=0x7f3684007b90 comp rx=0 tx=0).stop 2023-10-10T20:19:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 msgr2=0x7f36941430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e90 tx=0x7f368c00b040 comp rx=0 tx=0).stop 2023-10-10T20:19:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 shutdown_connections 2023-10-10T20:19:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3674066f30 0x7f36740693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f369415e620 0x7f36941430a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36940b04f0 0x7f36940c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 --2- 172.21.15.148:0/1134107438 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f369415a8b0 0x7f36940adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 >> 172.21.15.148:0/1134107438 conn(0x7f369406c700 msgr2=0x7f36940b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:32.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 shutdown_connections 2023-10-10T20:19:32.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.432+0000 7f36999aa700 1 -- 172.21.15.148:0/1134107438 wait complete. 2023-10-10T20:19:32.439 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:32.446 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-10T20:19:31.324406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560793,"statfs":{"total":1600000000,"available":1598252722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001810"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.500183+0000","last_change":"2023-10-10T20:19:23.499934+0000","last_active":"2023-10-10T20:19:23.500183+0000","last_peered":"2023-10-10T20:19:23.500183+0000","last_clean":"2023-10-10T20:19:23.500183+0000","last_became_active":"2023-10-10T20:19:23.499640+0000","last_became_peered":"2023-10-10T20:19:23.499640+0000","last_unstale":"2023-10-10T20:19:23.500183+0000","last_undegraded":"2023-10-10T20:19:23.500183+0000","last_fullsized":"2023-10-10T20:19:23.500183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488778+0000","last_change":"2023-10-10T20:19:23.488718+0000","last_active":"2023-10-10T20:19:23.488778+0000","last_peered":"2023-10-10T20:19:23.488778+0000","last_clean":"2023-10-10T20:19:23.488778+0000","last_became_active":"2023-10-10T20:19:23.488207+0000","last_became_peered":"2023-10-10T20:19:23.488207+0000","last_unstale":"2023-10-10T20:19:23.488778+0000","last_undegraded":"2023-10-10T20:19:23.488778+0000","last_fullsized":"2023-10-10T20:19:23.488778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488280+0000","last_change":"2023-10-10T20:19:23.487828+0000","last_active":"2023-10-10T20:19:23.488280+0000","last_peered":"2023-10-10T20:19:23.488280+0000","last_clean":"2023-10-10T20:19:23.488280+0000","last_became_active":"2023-10-10T20:19:23.487010+0000","last_became_peered":"2023-10-10T20:19:23.487010+0000","last_unstale":"2023-10-10T20:19:23.488280+0000","last_undegraded":"2023-10-10T20:19:23.488280+0000","last_fullsized":"2023-10-10T20:19:23.488280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492279+0000","last_change":"2023-10-10T20:19:23.492180+0000","last_active":"2023-10-10T20:19:23.492279+0000","last_peered":"2023-10-10T20:19:23.492279+0000","last_clean":"2023-10-10T20:19:23.492279+0000","last_became_active":"2023-10-10T20:19:23.491680+0000","last_became_peered":"2023-10-10T20:19:23.491680+0000","last_unstale":"2023-10-10T20:19:23.492279+0000","last_undegraded":"2023-10-10T20:19:23.492279+0000","last_fullsized":"2023-10-10T20:19:23.492279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492487+0000","last_change":"2023-10-10T20:19:23.492418+0000","last_active":"2023-10-10T20:19:23.492487+0000","last_peered":"2023-10-10T20:19:23.492487+0000","last_clean":"2023-10-10T20:19:23.492487+0000","last_became_active":"2023-10-10T20:19:23.492183+0000","last_became_peered":"2023-10-10T20:19:23.492183+0000","last_unstale":"2023-10-10T20:19:23.492487+0000","last_undegraded":"2023-10-10T20:19:23.492487+0000","last_fullsized":"2023-10-10T20:19:23.492487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494017+0000","last_change":"2023-10-10T20:19:23.493863+0000","last_active":"2023-10-10T20:19:23.494017+0000","last_peered":"2023-10-10T20:19:23.494017+0000","last_clean":"2023-10-10T20:19:23.494017+0000","last_became_active":"2023-10-10T20:19:23.492771+0000","last_became_peered":"2023-10-10T20:19:23.492771+0000","last_unstale":"2023-10-10T20:19:23.494017+0000","last_undegraded":"2023-10-10T20:19:23.494017+0000","last_fullsized":"2023-10-10T20:19:23.494017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493478+0000","last_change":"2023-10-10T20:19:23.493170+0000","last_active":"2023-10-10T20:19:23.493478+0000","last_peered":"2023-10-10T20:19:23.493478+0000","last_clean":"2023-10-10T20:19:23.493478+0000","last_became_active":"2023-10-10T20:19:23.492278+0000","last_became_peered":"2023-10-10T20:19:23.492278+0000","last_unstale":"2023-10-10T20:19:23.493478+0000","last_undegraded":"2023-10-10T20:19:23.493478+0000","last_fullsized":"2023-10-10T20:19:23.493478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485806+0000","last_change":"2023-10-10T20:19:23.485444+0000","last_active":"2023-10-10T20:19:23.485806+0000","last_peered":"2023-10-10T20:19:23.485806+0000","last_clean":"2023-10-10T20:19:23.485806+0000","last_became_active":"2023-10-10T20:19:23.485215+0000","last_became_peered":"2023-10-10T20:19:23.485215+0000","last_unstale":"2023-10-10T20:19:23.485806+0000","last_undegraded":"2023-10-10T20:19:23.485806+0000","last_fullsized":"2023-10-10T20:19:23.485806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495009+0000","last_change":"2023-10-10T20:19:23.494933+0000","last_active":"2023-10-10T20:19:23.495009+0000","last_peered":"2023-10-10T20:19:23.495009+0000","last_clean":"2023-10-10T20:19:23.495009+0000","last_became_active":"2023-10-10T20:19:23.494414+0000","last_became_peered":"2023-10-10T20:19:23.494414+0000","last_unstale":"2023-10-10T20:19:23.495009+0000","last_undegraded":"2023-10-10T20:19:23.495009+0000","last_fullsized":"2023-10-10T20:19:23.495009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499228+0000","last_change":"2023-10-10T20:19:23.499169+0000","last_active":"2023-10-10T20:19:23.499228+0000","last_peered":"2023-10-10T20:19:23.499228+0000","last_clean":"2023-10-10T20:19:23.499228+0000","last_became_active":"2023-10-10T20:19:23.498639+0000","last_became_peered":"2023-10-10T20:19:23.498639+0000","last_unstale":"2023-10-10T20:19:23.499228+0000","last_undegraded":"2023-10-10T20:19:23.499228+0000","last_fullsized":"2023-10-10T20:19:23.499228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498244+0000","last_change":"2023-10-10T20:19:23.497453+0000","last_active":"2023-10-10T20:19:23.498244+0000","last_peered":"2023-10-10T20:19:23.498244+0000","last_clean":"2023-10-10T20:19:23.498244+0000","last_became_active":"2023-10-10T20:19:23.497111+0000","last_became_peered":"2023-10-10T20:19:23.497111+0000","last_unstale":"2023-10-10T20:19:23.498244+0000","last_undegraded":"2023-10-10T20:19:23.498244+0000","last_fullsized":"2023-10-10T20:19:23.498244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493732+0000","last_change":"2023-10-10T20:19:23.493307+0000","last_active":"2023-10-10T20:19:23.493732+0000","last_peered":"2023-10-10T20:19:23.493732+0000","last_clean":"2023-10-10T20:19:23.493732+0000","last_became_active":"2023-10-10T20:19:23.492611+0000","last_became_peered":"2023-10-10T20:19:23.492611+0000","last_unstale":"2023-10-10T20:19:23.493732+0000","last_undegraded":"2023-10-10T20:19:23.493732+0000","last_fullsized":"2023-10-10T20:19:23.493732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495418+0000","last_change":"2023-10-10T20:19:23.495368+0000","last_active":"2023-10-10T20:19:23.495418+0000","last_peered":"2023-10-10T20:19:23.495418+0000","last_clean":"2023-10-10T20:19:23.495418+0000","last_became_active":"2023-10-10T20:19:23.495214+0000","last_became_peered":"2023-10-10T20:19:23.495214+0000","last_unstale":"2023-10-10T20:19:23.495418+0000","last_undegraded":"2023-10-10T20:19:23.495418+0000","last_fullsized":"2023-10-10T20:19:23.495418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483124+0000","last_change":"2023-10-10T20:19:23.483055+0000","last_active":"2023-10-10T20:19:23.483124+0000","last_peered":"2023-10-10T20:19:23.483124+0000","last_clean":"2023-10-10T20:19:23.483124+0000","last_became_active":"2023-10-10T20:19:23.482827+0000","last_became_peered":"2023-10-10T20:19:23.482827+0000","last_unstale":"2023-10-10T20:19:23.483124+0000","last_undegraded":"2023-10-10T20:19:23.483124+0000","last_fullsized":"2023-10-10T20:19:23.483124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492517+0000","last_change":"2023-10-10T20:19:23.492454+0000","last_active":"2023-10-10T20:19:23.492517+0000","last_peered":"2023-10-10T20:19:23.492517+0000","last_clean":"2023-10-10T20:19:23.492517+0000","last_became_active":"2023-10-10T20:19:23.492262+0000","last_became_peered":"2023-10-10T20:19:23.492262+0000","last_unstale":"2023-10-10T20:19:23.492517+0000","last_undegraded":"2023-10-10T20:19:23.492517+0000","last_fullsized":"2023-10-10T20:19:23.492517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493999+0000","last_change":"2023-10-10T20:19:23.493726+0000","last_active":"2023-10-10T20:19:23.493999+0000","last_peered":"2023-10-10T20:19:23.493999+0000","last_clean":"2023-10-10T20:19:23.493999+0000","last_became_active":"2023-10-10T20:19:23.492475+0000","last_became_peered":"2023-10-10T20:19:23.492475+0000","last_unstale":"2023-10-10T20:19:23.493999+0000","last_undegraded":"2023-10-10T20:19:23.493999+0000","last_fullsized":"2023-10-10T20:19:23.493999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495844+0000","last_change":"2023-10-10T20:19:23.495784+0000","last_active":"2023-10-10T20:19:23.495844+0000","last_peered":"2023-10-10T20:19:23.495844+0000","last_clean":"2023-10-10T20:19:23.495844+0000","last_became_active":"2023-10-10T20:19:23.495346+0000","last_became_peered":"2023-10-10T20:19:23.495346+0000","last_unstale":"2023-10-10T20:19:23.495844+0000","last_undegraded":"2023-10-10T20:19:23.495844+0000","last_fullsized":"2023-10-10T20:19:23.495844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494759+0000","last_change":"2023-10-10T20:19:23.494684+0000","last_active":"2023-10-10T20:19:23.494759+0000","last_peered":"2023-10-10T20:19:23.494759+0000","last_clean":"2023-10-10T20:19:23.494759+0000","last_became_active":"2023-10-10T20:19:23.494457+0000","last_became_peered":"2023-10-10T20:19:23.494457+0000","last_unstale":"2023-10-10T20:19:23.494759+0000","last_undegraded":"2023-10-10T20:19:23.494759+0000","last_fullsized":"2023-10-10T20:19:23.494759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491717+0000","last_change":"2023-10-10T20:19:23.491294+0000","last_active":"2023-10-10T20:19:23.491717+0000","last_peered":"2023-10-10T20:19:23.491717+0000","last_clean":"2023-10-10T20:19:23.491717+0000","last_became_active":"2023-10-10T20:19:23.491078+0000","last_became_peered":"2023-10-10T20:19:23.491078+0000","last_unstale":"2023-10-10T20:19:23.491717+0000","last_undegraded":"2023-10-10T20:19:23.491717+0000","last_fullsized":"2023-10-10T20:19:23.491717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505463+0000","last_change":"2023-10-10T20:19:22.491174+0000","last_active":"2023-10-10T20:19:23.505463+0000","last_peered":"2023-10-10T20:19:23.505463+0000","last_clean":"2023-10-10T20:19:23.505463+0000","last_became_active":"2023-10-10T20:19:22.490398+0000","last_became_peered":"2023-10-10T20:19:22.490398+0000","last_unstale":"2023-10-10T20:19:23.505463+0000","last_undegraded":"2023-10-10T20:19:23.505463+0000","last_fullsized":"2023-10-10T20:19:23.505463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490418+0000","last_change":"2023-10-10T20:19:23.490364+0000","last_active":"2023-10-10T20:19:23.490418+0000","last_peered":"2023-10-10T20:19:23.490418+0000","last_clean":"2023-10-10T20:19:23.490418+0000","last_became_active":"2023-10-10T20:19:23.490129+0000","last_became_peered":"2023-10-10T20:19:23.490129+0000","last_unstale":"2023-10-10T20:19:23.490418+0000","last_undegraded":"2023-10-10T20:19:23.490418+0000","last_fullsized":"2023-10-10T20:19:23.490418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505547+0000","last_change":"2023-10-10T20:19:22.490246+0000","last_active":"2023-10-10T20:19:23.505547+0000","last_peered":"2023-10-10T20:19:23.505547+0000","last_clean":"2023-10-10T20:19:23.505547+0000","last_became_active":"2023-10-10T20:19:22.490027+0000","last_became_peered":"2023-10-10T20:19:22.490027+0000","last_unstale":"2023-10-10T20:19:23.505547+0000","last_undegraded":"2023-10-10T20:19:23.505547+0000","last_fullsized":"2023-10-10T20:19:23.505547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488264+0000","last_change":"2023-10-10T20:19:23.487706+0000","last_active":"2023-10-10T20:19:23.488264+0000","last_peered":"2023-10-10T20:19:23.488264+0000","last_clean":"2023-10-10T20:19:23.488264+0000","last_became_active":"2023-10-10T20:19:23.486658+0000","last_became_peered":"2023-10-10T20:19:23.486658+0000","last_unstale":"2023-10-10T20:19:23.488264+0000","last_undegraded":"2023-10-10T20:19:23.488264+0000","last_fullsized":"2023-10-10T20:19:23.488264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505022+0000","last_change":"2023-10-10T20:19:22.491563+0000","last_active":"2023-10-10T20:19:23.505022+0000","last_peered":"2023-10-10T20:19:23.505022+0000","last_clean":"2023-10-10T20:19:23.505022+0000","last_became_active":"2023-10-10T20:19:22.491205+0000","last_became_peered":"2023-10-10T20:19:22.491205+0000","last_unstale":"2023-10-10T20:19:23.505022+0000","last_undegraded":"2023-10-10T20:19:23.505022+0000","last_fullsized":"2023-10-10T20:19:23.505022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495250+0000","last_change":"2023-10-10T20:19:23.495186+0000","last_active":"2023-10-10T20:19:23.495250+0000","last_peered":"2023-10-10T20:19:23.495250+0000","last_clean":"2023-10-10T20:19:23.495250+0000","last_became_active":"2023-10-10T20:19:23.494935+0000","last_became_peered":"2023-10-10T20:19:23.494935+0000","last_unstale":"2023-10-10T20:19:23.495250+0000","last_undegraded":"2023-10-10T20:19:23.495250+0000","last_fullsized":"2023-10-10T20:19:23.495250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503478+0000","last_change":"2023-10-10T20:19:22.493193+0000","last_active":"2023-10-10T20:19:23.503478+0000","last_peered":"2023-10-10T20:19:23.503478+0000","last_clean":"2023-10-10T20:19:23.503478+0000","last_became_active":"2023-10-10T20:19:22.492979+0000","last_became_peered":"2023-10-10T20:19:22.492979+0000","last_unstale":"2023-10-10T20:19:23.503478+0000","last_undegraded":"2023-10-10T20:19:23.503478+0000","last_fullsized":"2023-10-10T20:19:23.503478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487270+0000","last_change":"2023-10-10T20:19:23.487131+0000","last_active":"2023-10-10T20:19:23.487270+0000","last_peered":"2023-10-10T20:19:23.487270+0000","last_clean":"2023-10-10T20:19:23.487270+0000","last_became_active":"2023-10-10T20:19:23.486894+0000","last_became_peered":"2023-10-10T20:19:23.486894+0000","last_unstale":"2023-10-10T20:19:23.487270+0000","last_undegraded":"2023-10-10T20:19:23.487270+0000","last_fullsized":"2023-10-10T20:19:23.487270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505011+0000","last_change":"2023-10-10T20:19:22.491757+0000","last_active":"2023-10-10T20:19:23.505011+0000","last_peered":"2023-10-10T20:19:23.505011+0000","last_clean":"2023-10-10T20:19:23.505011+0000","last_became_active":"2023-10-10T20:19:22.491535+0000","last_became_peered":"2023-10-10T20:19:22.491535+0000","last_unstale":"2023-10-10T20:19:23.505011+0000","last_undegraded":"2023-10-10T20:19:23.505011+0000","last_fullsized":"2023-10-10T20:19:23.505011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489517+0000","last_change":"2023-10-10T20:19:23.488600+0000","last_active":"2023-10-10T20:19:23.489517+0000","last_peered":"2023-10-10T20:19:23.489517+0000","last_clean":"2023-10-10T20:19:23.489517+0000","last_became_active":"2023-10-10T20:19:23.487824+0000","last_became_peered":"2023-10-10T20:19:23.487824+0000","last_unstale":"2023-10-10T20:19:23.489517+0000","last_undegraded":"2023-10-10T20:19:23.489517+0000","last_fullsized":"2023-10-10T20:19:23.489517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:07.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.502781+0000","last_change":"2023-10-10T20:19:22.486217+0000","last_active":"2023-10-10T20:19:23.502781+0000","last_peered":"2023-10-10T20:19:23.502781+0000","last_clean":"2023-10-10T20:19:23.502781+0000","last_became_active":"2023-10-10T20:19:22.484804+0000","last_became_peered":"2023-10-10T20:19:22.484804+0000","last_unstale":"2023-10-10T20:19:23.502781+0000","last_undegraded":"2023-10-10T20:19:23.502781+0000","last_fullsized":"2023-10-10T20:19:23.502781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485364+0000","last_change":"2023-10-10T20:19:23.485292+0000","last_active":"2023-10-10T20:19:23.485364+0000","last_peered":"2023-10-10T20:19:23.485364+0000","last_clean":"2023-10-10T20:19:23.485364+0000","last_became_active":"2023-10-10T20:19:23.485060+0000","last_became_peered":"2023-10-10T20:19:23.485060+0000","last_unstale":"2023-10-10T20:19:23.485364+0000","last_undegraded":"2023-10-10T20:19:23.485364+0000","last_fullsized":"2023-10-10T20:19:23.485364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:16.611774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482268+0000","last_change":"2023-10-10T20:19:22.489127+0000","last_active":"2023-10-10T20:19:23.482268+0000","last_peered":"2023-10-10T20:19:23.482268+0000","last_clean":"2023-10-10T20:19:23.482268+0000","last_became_active":"2023-10-10T20:19:22.488683+0000","last_became_peered":"2023-10-10T20:19:22.488683+0000","last_unstale":"2023-10-10T20:19:23.482268+0000","last_undegraded":"2023-10-10T20:19:23.482268+0000","last_fullsized":"2023-10-10T20:19:23.482268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487104+0000","last_change":"2023-10-10T20:19:23.486567+0000","last_active":"2023-10-10T20:19:23.487104+0000","last_peered":"2023-10-10T20:19:23.487104+0000","last_clean":"2023-10-10T20:19:23.487104+0000","last_became_active":"2023-10-10T20:19:23.486329+0000","last_became_peered":"2023-10-10T20:19:23.486329+0000","last_unstale":"2023-10-10T20:19:23.487104+0000","last_undegraded":"2023-10-10T20:19:23.487104+0000","last_fullsized":"2023-10-10T20:19:23.487104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483483+0000","last_change":"2023-10-10T20:19:22.485105+0000","last_active":"2023-10-10T20:19:23.483483+0000","last_peered":"2023-10-10T20:19:23.483483+0000","last_clean":"2023-10-10T20:19:23.483483+0000","last_became_active":"2023-10-10T20:19:22.484537+0000","last_became_peered":"2023-10-10T20:19:22.484537+0000","last_unstale":"2023-10-10T20:19:23.483483+0000","last_undegraded":"2023-10-10T20:19:23.483483+0000","last_fullsized":"2023-10-10T20:19:23.483483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483939+0000","last_change":"2023-10-10T20:19:23.483701+0000","last_active":"2023-10-10T20:19:23.483939+0000","last_peered":"2023-10-10T20:19:23.483939+0000","last_clean":"2023-10-10T20:19:23.483939+0000","last_became_active":"2023-10-10T20:19:23.483485+0000","last_became_peered":"2023-10-10T20:19:23.483485+0000","last_unstale":"2023-10-10T20:19:23.483939+0000","last_undegraded":"2023-10-10T20:19:23.483939+0000","last_fullsized":"2023-10-10T20:19:23.483939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486842+0000","last_change":"2023-10-10T20:19:22.486912+0000","last_active":"2023-10-10T20:19:23.486842+0000","last_peered":"2023-10-10T20:19:23.486842+0000","last_clean":"2023-10-10T20:19:23.486842+0000","last_became_active":"2023-10-10T20:19:22.486662+0000","last_became_peered":"2023-10-10T20:19:22.486662+0000","last_unstale":"2023-10-10T20:19:23.486842+0000","last_undegraded":"2023-10-10T20:19:23.486842+0000","last_fullsized":"2023-10-10T20:19:23.486842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492299+0000","last_change":"2023-10-10T20:19:23.492209+0000","last_active":"2023-10-10T20:19:23.492299+0000","last_peered":"2023-10-10T20:19:23.492299+0000","last_clean":"2023-10-10T20:19:23.492299+0000","last_became_active":"2023-10-10T20:19:23.491975+0000","last_became_peered":"2023-10-10T20:19:23.491975+0000","last_unstale":"2023-10-10T20:19:23.492299+0000","last_undegraded":"2023-10-10T20:19:23.492299+0000","last_fullsized":"2023-10-10T20:19:23.492299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485905+0000","last_change":"2023-10-10T20:19:22.490636+0000","last_active":"2023-10-10T20:19:23.485905+0000","last_peered":"2023-10-10T20:19:23.485905+0000","last_clean":"2023-10-10T20:19:23.485905+0000","last_became_active":"2023-10-10T20:19:22.490369+0000","last_became_peered":"2023-10-10T20:19:22.490369+0000","last_unstale":"2023-10-10T20:19:23.485905+0000","last_undegraded":"2023-10-10T20:19:23.485905+0000","last_fullsized":"2023-10-10T20:19:23.485905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490958+0000","last_change":"2023-10-10T20:19:23.490908+0000","last_active":"2023-10-10T20:19:23.490958+0000","last_peered":"2023-10-10T20:19:23.490958+0000","last_clean":"2023-10-10T20:19:23.490958+0000","last_became_active":"2023-10-10T20:19:23.490692+0000","last_became_peered":"2023-10-10T20:19:23.490692+0000","last_unstale":"2023-10-10T20:19:23.490958+0000","last_undegraded":"2023-10-10T20:19:23.490958+0000","last_fullsized":"2023-10-10T20:19:23.490958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503581+0000","last_change":"2023-10-10T20:19:22.487531+0000","last_active":"2023-10-10T20:19:23.503581+0000","last_peered":"2023-10-10T20:19:23.503581+0000","last_clean":"2023-10-10T20:19:23.503581+0000","last_became_active":"2023-10-10T20:19:22.487040+0000","last_became_peered":"2023-10-10T20:19:22.487040+0000","last_unstale":"2023-10-10T20:19:23.503581+0000","last_undegraded":"2023-10-10T20:19:23.503581+0000","last_fullsized":"2023-10-10T20:19:23.503581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493693+0000","last_change":"2023-10-10T20:19:23.493005+0000","last_active":"2023-10-10T20:19:23.493693+0000","last_peered":"2023-10-10T20:19:23.493693+0000","last_clean":"2023-10-10T20:19:23.493693+0000","last_became_active":"2023-10-10T20:19:23.492752+0000","last_became_peered":"2023-10-10T20:19:23.492752+0000","last_unstale":"2023-10-10T20:19:23.493693+0000","last_undegraded":"2023-10-10T20:19:23.493693+0000","last_fullsized":"2023-10-10T20:19:23.493693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.506581+0000","last_change":"2023-10-10T20:19:22.491546+0000","last_active":"2023-10-10T20:19:23.506581+0000","last_peered":"2023-10-10T20:19:23.506581+0000","last_clean":"2023-10-10T20:19:23.506581+0000","last_became_active":"2023-10-10T20:19:22.491157+0000","last_became_peered":"2023-10-10T20:19:22.491157+0000","last_unstale":"2023-10-10T20:19:23.506581+0000","last_undegraded":"2023-10-10T20:19:23.506581+0000","last_fullsized":"2023-10-10T20:19:23.506581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490504+0000","last_change":"2023-10-10T20:19:23.490357+0000","last_active":"2023-10-10T20:19:23.490504+0000","last_peered":"2023-10-10T20:19:23.490504+0000","last_clean":"2023-10-10T20:19:23.490504+0000","last_became_active":"2023-10-10T20:19:23.490094+0000","last_became_peered":"2023-10-10T20:19:23.490094+0000","last_unstale":"2023-10-10T20:19:23.490504+0000","last_undegraded":"2023-10-10T20:19:23.490504+0000","last_fullsized":"2023-10-10T20:19:23.490504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504122+0000","last_change":"2023-10-10T20:19:22.484098+0000","last_active":"2023-10-10T20:19:23.504122+0000","last_peered":"2023-10-10T20:19:23.504122+0000","last_clean":"2023-10-10T20:19:23.504122+0000","last_became_active":"2023-10-10T20:19:22.482738+0000","last_became_peered":"2023-10-10T20:19:22.482738+0000","last_unstale":"2023-10-10T20:19:23.504122+0000","last_undegraded":"2023-10-10T20:19:23.504122+0000","last_fullsized":"2023-10-10T20:19:23.504122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491923+0000","last_change":"2023-10-10T20:19:23.491873+0000","last_active":"2023-10-10T20:19:23.491923+0000","last_peered":"2023-10-10T20:19:23.491923+0000","last_clean":"2023-10-10T20:19:23.491923+0000","last_became_active":"2023-10-10T20:19:23.491532+0000","last_became_peered":"2023-10-10T20:19:23.491532+0000","last_unstale":"2023-10-10T20:19:23.491923+0000","last_undegraded":"2023-10-10T20:19:23.491923+0000","last_fullsized":"2023-10-10T20:19:23.491923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488825+0000","last_change":"2023-10-10T20:19:22.491830+0000","last_active":"2023-10-10T20:19:23.488825+0000","last_peered":"2023-10-10T20:19:23.488825+0000","last_clean":"2023-10-10T20:19:23.488825+0000","last_became_active":"2023-10-10T20:19:22.489289+0000","last_became_peered":"2023-10-10T20:19:22.489289+0000","last_unstale":"2023-10-10T20:19:23.488825+0000","last_undegraded":"2023-10-10T20:19:23.488825+0000","last_fullsized":"2023-10-10T20:19:23.488825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497717+0000","last_change":"2023-10-10T20:19:23.497580+0000","last_active":"2023-10-10T20:19:23.497717+0000","last_peered":"2023-10-10T20:19:23.497717+0000","last_clean":"2023-10-10T20:19:23.497717+0000","last_became_active":"2023-10-10T20:19:23.497043+0000","last_became_peered":"2023-10-10T20:19:23.497043+0000","last_unstale":"2023-10-10T20:19:23.497717+0000","last_undegraded":"2023-10-10T20:19:23.497717+0000","last_fullsized":"2023-10-10T20:19:23.497717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486275+0000","last_change":"2023-10-10T20:19:22.487157+0000","last_active":"2023-10-10T20:19:23.486275+0000","last_peered":"2023-10-10T20:19:23.486275+0000","last_clean":"2023-10-10T20:19:23.486275+0000","last_became_active":"2023-10-10T20:19:22.486367+0000","last_became_peered":"2023-10-10T20:19:22.486367+0000","last_unstale":"2023-10-10T20:19:23.486275+0000","last_undegraded":"2023-10-10T20:19:23.486275+0000","last_fullsized":"2023-10-10T20:19:23.486275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494474+0000","last_change":"2023-10-10T20:19:23.494326+0000","last_active":"2023-10-10T20:19:23.494474+0000","last_peered":"2023-10-10T20:19:23.494474+0000","last_clean":"2023-10-10T20:19:23.494474+0000","last_became_active":"2023-10-10T20:19:23.494043+0000","last_became_peered":"2023-10-10T20:19:23.494043+0000","last_unstale":"2023-10-10T20:19:23.494474+0000","last_undegraded":"2023-10-10T20:19:23.494474+0000","last_fullsized":"2023-10-10T20:19:23.494474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505618+0000","last_change":"2023-10-10T20:19:22.484580+0000","last_active":"2023-10-10T20:19:23.505618+0000","last_peered":"2023-10-10T20:19:23.505618+0000","last_clean":"2023-10-10T20:19:23.505618+0000","last_became_active":"2023-10-10T20:19:22.484287+0000","last_became_peered":"2023-10-10T20:19:22.484287+0000","last_unstale":"2023-10-10T20:19:23.505618+0000","last_undegraded":"2023-10-10T20:19:23.505618+0000","last_fullsized":"2023-10-10T20:19:23.505618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497360+0000","last_change":"2023-10-10T20:19:23.497292+0000","last_active":"2023-10-10T20:19:23.497360+0000","last_peered":"2023-10-10T20:19:23.497360+0000","last_clean":"2023-10-10T20:19:23.497360+0000","last_became_active":"2023-10-10T20:19:23.497063+0000","last_became_peered":"2023-10-10T20:19:23.497063+0000","last_unstale":"2023-10-10T20:19:23.497360+0000","last_undegraded":"2023-10-10T20:19:23.497360+0000","last_fullsized":"2023-10-10T20:19:23.497360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505930+0000","last_change":"2023-10-10T20:19:22.481673+0000","last_active":"2023-10-10T20:19:23.505930+0000","last_peered":"2023-10-10T20:19:23.505930+0000","last_clean":"2023-10-10T20:19:23.505930+0000","last_became_active":"2023-10-10T20:19:22.481500+0000","last_became_peered":"2023-10-10T20:19:22.481500+0000","last_unstale":"2023-10-10T20:19:23.505930+0000","last_undegraded":"2023-10-10T20:19:23.505930+0000","last_fullsized":"2023-10-10T20:19:23.505930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497808+0000","last_change":"2023-10-10T20:19:23.497718+0000","last_active":"2023-10-10T20:19:23.497808+0000","last_peered":"2023-10-10T20:19:23.497808+0000","last_clean":"2023-10-10T20:19:23.497808+0000","last_became_active":"2023-10-10T20:19:23.497352+0000","last_became_peered":"2023-10-10T20:19:23.497352+0000","last_unstale":"2023-10-10T20:19:23.497808+0000","last_undegraded":"2023-10-10T20:19:23.497808+0000","last_fullsized":"2023-10-10T20:19:23.497808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481756+0000","last_change":"2023-10-10T20:19:22.486438+0000","last_active":"2023-10-10T20:19:23.481756+0000","last_peered":"2023-10-10T20:19:23.481756+0000","last_clean":"2023-10-10T20:19:23.481756+0000","last_became_active":"2023-10-10T20:19:22.486219+0000","last_became_peered":"2023-10-10T20:19:22.486219+0000","last_unstale":"2023-10-10T20:19:23.481756+0000","last_undegraded":"2023-10-10T20:19:23.481756+0000","last_fullsized":"2023-10-10T20:19:23.481756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490783+0000","last_change":"2023-10-10T20:19:23.490700+0000","last_active":"2023-10-10T20:19:23.490783+0000","last_peered":"2023-10-10T20:19:23.490783+0000","last_clean":"2023-10-10T20:19:23.490783+0000","last_became_active":"2023-10-10T20:19:23.490468+0000","last_became_peered":"2023-10-10T20:19:23.490468+0000","last_unstale":"2023-10-10T20:19:23.490783+0000","last_undegraded":"2023-10-10T20:19:23.490783+0000","last_fullsized":"2023-10-10T20:19:23.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486018+0000","last_change":"2023-10-10T20:19:22.485041+0000","last_active":"2023-10-10T20:19:23.486018+0000","last_peered":"2023-10-10T20:19:23.486018+0000","last_clean":"2023-10-10T20:19:23.486018+0000","last_became_active":"2023-10-10T20:19:22.484798+0000","last_became_peered":"2023-10-10T20:19:22.484798+0000","last_unstale":"2023-10-10T20:19:23.486018+0000","last_undegraded":"2023-10-10T20:19:23.486018+0000","last_fullsized":"2023-10-10T20:19:23.486018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489489+0000","last_change":"2023-10-10T20:19:23.487628+0000","last_active":"2023-10-10T20:19:23.489489+0000","last_peered":"2023-10-10T20:19:23.489489+0000","last_clean":"2023-10-10T20:19:23.489489+0000","last_became_active":"2023-10-10T20:19:23.486750+0000","last_became_peered":"2023-10-10T20:19:23.486750+0000","last_unstale":"2023-10-10T20:19:23.489489+0000","last_undegraded":"2023-10-10T20:19:23.489489+0000","last_fullsized":"2023-10-10T20:19:23.489489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:28.087913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487677+0000","last_change":"2023-10-10T20:19:23.486218+0000","last_active":"2023-10-10T20:19:23.487677+0000","last_peered":"2023-10-10T20:19:23.487677+0000","last_clean":"2023-10-10T20:19:23.487677+0000","last_became_active":"2023-10-10T20:19:23.485999+0000","last_became_peered":"2023-10-10T20:19:23.485999+0000","last_unstale":"2023-10-10T20:19:23.487677+0000","last_undegraded":"2023-10-10T20:19:23.487677+0000","last_fullsized":"2023-10-10T20:19:23.487677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.484120+0000","last_change":"2023-10-10T20:19:22.489199+0000","last_active":"2023-10-10T20:19:23.484120+0000","last_peered":"2023-10-10T20:19:23.484120+0000","last_clean":"2023-10-10T20:19:23.484120+0000","last_became_active":"2023-10-10T20:19:22.488931+0000","last_became_peered":"2023-10-10T20:19:22.488931+0000","last_unstale":"2023-10-10T20:19:23.484120+0000","last_undegraded":"2023-10-10T20:19:23.484120+0000","last_fullsized":"2023-10-10T20:19:23.484120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489524+0000","last_change":"2023-10-10T20:19:23.489413+0000","last_active":"2023-10-10T20:19:23.489524+0000","last_peered":"2023-10-10T20:19:23.489524+0000","last_clean":"2023-10-10T20:19:23.489524+0000","last_became_active":"2023-10-10T20:19:23.489083+0000","last_became_peered":"2023-10-10T20:19:23.489083+0000","last_unstale":"2023-10-10T20:19:23.489524+0000","last_undegraded":"2023-10-10T20:19:23.489524+0000","last_fullsized":"2023-10-10T20:19:23.489524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:22.491979+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:22.489613+0000","last_became_peered":"2023-10-10T20:19:22.489613+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486860+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:23.486860+0000","last_peered":"2023-10-10T20:19:23.486860+0000","last_clean":"2023-10-10T20:19:23.486860+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:23.486860+0000","last_undegraded":"2023-10-10T20:19:23.486860+0000","last_fullsized":"2023-10-10T20:19:23.486860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488300+0000","last_change":"2023-10-10T20:19:23.487957+0000","last_active":"2023-10-10T20:19:23.488300+0000","last_peered":"2023-10-10T20:19:23.488300+0000","last_clean":"2023-10-10T20:19:23.488300+0000","last_became_active":"2023-10-10T20:19:23.487318+0000","last_became_peered":"2023-10-10T20:19:23.487318+0000","last_unstale":"2023-10-10T20:19:23.488300+0000","last_undegraded":"2023-10-10T20:19:23.488300+0000","last_fullsized":"2023-10-10T20:19:23.488300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486185+0000","last_change":"2023-10-10T20:19:22.492546+0000","last_active":"2023-10-10T20:19:23.486185+0000","last_peered":"2023-10-10T20:19:23.486185+0000","last_clean":"2023-10-10T20:19:23.486185+0000","last_became_active":"2023-10-10T20:19:22.492293+0000","last_became_peered":"2023-10-10T20:19:22.492293+0000","last_unstale":"2023-10-10T20:19:23.486185+0000","last_undegraded":"2023-10-10T20:19:23.486185+0000","last_fullsized":"2023-10-10T20:19:23.486185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486269+0000","last_change":"2023-10-10T20:19:22.490666+0000","last_active":"2023-10-10T20:19:23.486269+0000","last_peered":"2023-10-10T20:19:23.486269+0000","last_clean":"2023-10-10T20:19:23.486269+0000","last_became_active":"2023-10-10T20:19:22.490355+0000","last_became_peered":"2023-10-10T20:19:22.490355+0000","last_unstale":"2023-10-10T20:19:23.486269+0000","last_undegraded":"2023-10-10T20:19:23.486269+0000","last_fullsized":"2023-10-10T20:19:23.486269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487695+0000","last_change":"2023-10-10T20:19:23.486743+0000","last_active":"2023-10-10T20:19:23.487695+0000","last_peered":"2023-10-10T20:19:23.487695+0000","last_clean":"2023-10-10T20:19:23.487695+0000","last_became_active":"2023-10-10T20:19:23.486526+0000","last_became_peered":"2023-10-10T20:19:23.486526+0000","last_unstale":"2023-10-10T20:19:23.487695+0000","last_undegraded":"2023-10-10T20:19:23.487695+0000","last_fullsized":"2023-10-10T20:19:23.487695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-10T20:19:24.482879+0000","last_change":"2023-10-10T20:19:22.487657+0000","last_active":"2023-10-10T20:19:24.482879+0000","last_peered":"2023-10-10T20:19:24.482879+0000","last_clean":"2023-10-10T20:19:24.482879+0000","last_became_active":"2023-10-10T20:19:22.487329+0000","last_became_peered":"2023-10-10T20:19:22.487329+0000","last_unstale":"2023-10-10T20:19:24.482879+0000","last_undegraded":"2023-10-10T20:19:24.482879+0000","last_fullsized":"2023-10-10T20:19:24.482879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:05.420138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491172+0000","last_change":"2023-10-10T20:19:23.491123+0000","last_active":"2023-10-10T20:19:23.491172+0000","last_peered":"2023-10-10T20:19:23.491172+0000","last_clean":"2023-10-10T20:19:23.491172+0000","last_became_active":"2023-10-10T20:19:23.490946+0000","last_became_peered":"2023-10-10T20:19:23.490946+0000","last_unstale":"2023-10-10T20:19:23.491172+0000","last_undegraded":"2023-10-10T20:19:23.491172+0000","last_fullsized":"2023-10-10T20:19:23.491172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505978+0000","last_change":"2023-10-10T20:19:22.491522+0000","last_active":"2023-10-10T20:19:23.505978+0000","last_peered":"2023-10-10T20:19:23.505978+0000","last_clean":"2023-10-10T20:19:23.505978+0000","last_became_active":"2023-10-10T20:19:22.491251+0000","last_became_peered":"2023-10-10T20:19:22.491251+0000","last_unstale":"2023-10-10T20:19:23.505978+0000","last_undegraded":"2023-10-10T20:19:23.505978+0000","last_fullsized":"2023-10-10T20:19:23.505978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488247+0000","last_change":"2023-10-10T20:19:23.487572+0000","last_active":"2023-10-10T20:19:23.488247+0000","last_peered":"2023-10-10T20:19:23.488247+0000","last_clean":"2023-10-10T20:19:23.488247+0000","last_became_active":"2023-10-10T20:19:23.486362+0000","last_became_peered":"2023-10-10T20:19:23.486362+0000","last_unstale":"2023-10-10T20:19:23.488247+0000","last_undegraded":"2023-10-10T20:19:23.488247+0000","last_fullsized":"2023-10-10T20:19:23.488247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:04.867879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489482+0000","last_change":"2023-10-10T20:19:22.490501+0000","last_active":"2023-10-10T20:19:23.489482+0000","last_peered":"2023-10-10T20:19:23.489482+0000","last_clean":"2023-10-10T20:19:23.489482+0000","last_became_active":"2023-10-10T20:19:22.490208+0000","last_became_peered":"2023-10-10T20:19:22.490208+0000","last_unstale":"2023-10-10T20:19:23.489482+0000","last_undegraded":"2023-10-10T20:19:23.489482+0000","last_fullsized":"2023-10-10T20:19:23.489482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.496090+0000","last_change":"2023-10-10T20:19:23.496016+0000","last_active":"2023-10-10T20:19:23.496090+0000","last_peered":"2023-10-10T20:19:23.496090+0000","last_clean":"2023-10-10T20:19:23.496090+0000","last_became_active":"2023-10-10T20:19:23.495512+0000","last_became_peered":"2023-10-10T20:19:23.495512+0000","last_unstale":"2023-10-10T20:19:23.496090+0000","last_undegraded":"2023-10-10T20:19:23.496090+0000","last_fullsized":"2023-10-10T20:19:23.496090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:05.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481809+0000","last_change":"2023-10-10T20:19:22.490184+0000","last_active":"2023-10-10T20:19:23.481809+0000","last_peered":"2023-10-10T20:19:23.481809+0000","last_clean":"2023-10-10T20:19:23.481809+0000","last_became_active":"2023-10-10T20:19:22.490057+0000","last_became_peered":"2023-10-10T20:19:22.490057+0000","last_unstale":"2023-10-10T20:19:23.481809+0000","last_undegraded":"2023-10-10T20:19:23.481809+0000","last_fullsized":"2023-10-10T20:19:23.481809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487155+0000","last_change":"2023-10-10T20:19:23.487003+0000","last_active":"2023-10-10T20:19:23.487155+0000","last_peered":"2023-10-10T20:19:23.487155+0000","last_clean":"2023-10-10T20:19:23.487155+0000","last_became_active":"2023-10-10T20:19:23.486789+0000","last_became_peered":"2023-10-10T20:19:23.486789+0000","last_unstale":"2023-10-10T20:19:23.487155+0000","last_undegraded":"2023-10-10T20:19:23.487155+0000","last_fullsized":"2023-10-10T20:19:23.487155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482124+0000","last_change":"2023-10-10T20:19:22.491852+0000","last_active":"2023-10-10T20:19:23.482124+0000","last_peered":"2023-10-10T20:19:23.482124+0000","last_clean":"2023-10-10T20:19:23.482124+0000","last_became_active":"2023-10-10T20:19:22.491569+0000","last_became_peered":"2023-10-10T20:19:22.491569+0000","last_unstale":"2023-10-10T20:19:23.482124+0000","last_undegraded":"2023-10-10T20:19:23.482124+0000","last_fullsized":"2023-10-10T20:19:23.482124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488335+0000","last_change":"2023-10-10T20:19:23.487466+0000","last_active":"2023-10-10T20:19:23.488335+0000","last_peered":"2023-10-10T20:19:23.488335+0000","last_clean":"2023-10-10T20:19:23.488335+0000","last_became_active":"2023-10-10T20:19:23.487241+0000","last_became_peered":"2023-10-10T20:19:23.487241+0000","last_unstale":"2023-10-10T20:19:23.488335+0000","last_undegraded":"2023-10-10T20:19:23.488335+0000","last_fullsized":"2023-10-10T20:19:23.488335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503296+0000","last_change":"2023-10-10T20:19:22.490823+0000","last_active":"2023-10-10T20:19:23.503296+0000","last_peered":"2023-10-10T20:19:23.503296+0000","last_clean":"2023-10-10T20:19:23.503296+0000","last_became_active":"2023-10-10T20:19:22.490137+0000","last_became_peered":"2023-10-10T20:19:22.490137+0000","last_unstale":"2023-10-10T20:19:23.503296+0000","last_undegraded":"2023-10-10T20:19:23.503296+0000","last_fullsized":"2023-10-10T20:19:23.503296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495850+0000","last_change":"2023-10-10T20:19:23.495686+0000","last_active":"2023-10-10T20:19:23.495850+0000","last_peered":"2023-10-10T20:19:23.495850+0000","last_clean":"2023-10-10T20:19:23.495850+0000","last_became_active":"2023-10-10T20:19:23.495454+0000","last_became_peered":"2023-10-10T20:19:23.495454+0000","last_unstale":"2023-10-10T20:19:23.495850+0000","last_undegraded":"2023-10-10T20:19:23.495850+0000","last_fullsized":"2023-10-10T20:19:23.495850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483607+0000","last_change":"2023-10-10T20:19:22.485497+0000","last_active":"2023-10-10T20:19:23.483607+0000","last_peered":"2023-10-10T20:19:23.483607+0000","last_clean":"2023-10-10T20:19:23.483607+0000","last_became_active":"2023-10-10T20:19:22.485276+0000","last_became_peered":"2023-10-10T20:19:22.485276+0000","last_unstale":"2023-10-10T20:19:23.483607+0000","last_undegraded":"2023-10-10T20:19:23.483607+0000","last_fullsized":"2023-10-10T20:19:23.483607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495682+0000","last_change":"2023-10-10T20:19:23.495616+0000","last_active":"2023-10-10T20:19:23.495682+0000","last_peered":"2023-10-10T20:19:23.495682+0000","last_clean":"2023-10-10T20:19:23.495682+0000","last_became_active":"2023-10-10T20:19:23.495385+0000","last_became_peered":"2023-10-10T20:19:23.495385+0000","last_unstale":"2023-10-10T20:19:23.495682+0000","last_undegraded":"2023-10-10T20:19:23.495682+0000","last_fullsized":"2023-10-10T20:19:23.495682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504015+0000","last_change":"2023-10-10T20:19:22.483831+0000","last_active":"2023-10-10T20:19:23.504015+0000","last_peered":"2023-10-10T20:19:23.504015+0000","last_clean":"2023-10-10T20:19:23.504015+0000","last_became_active":"2023-10-10T20:19:22.483604+0000","last_became_peered":"2023-10-10T20:19:22.483604+0000","last_unstale":"2023-10-10T20:19:23.504015+0000","last_undegraded":"2023-10-10T20:19:23.504015+0000","last_fullsized":"2023-10-10T20:19:23.504015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489338+0000","last_change":"2023-10-10T20:19:23.488221+0000","last_active":"2023-10-10T20:19:23.489338+0000","last_peered":"2023-10-10T20:19:23.489338+0000","last_clean":"2023-10-10T20:19:23.489338+0000","last_became_active":"2023-10-10T20:19:23.487992+0000","last_became_peered":"2023-10-10T20:19:23.487992+0000","last_unstale":"2023-10-10T20:19:23.489338+0000","last_undegraded":"2023-10-10T20:19:23.489338+0000","last_fullsized":"2023-10-10T20:19:23.489338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504646+0000","last_change":"2023-10-10T20:19:22.488044+0000","last_active":"2023-10-10T20:19:23.504646+0000","last_peered":"2023-10-10T20:19:23.504646+0000","last_clean":"2023-10-10T20:19:23.504646+0000","last_became_active":"2023-10-10T20:19:22.487823+0000","last_became_peered":"2023-10-10T20:19:22.487823+0000","last_unstale":"2023-10-10T20:19:23.504646+0000","last_undegraded":"2023-10-10T20:19:23.504646+0000","last_fullsized":"2023-10-10T20:19:23.504646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494991+0000","last_change":"2023-10-10T20:19:23.494793+0000","last_active":"2023-10-10T20:19:23.494991+0000","last_peered":"2023-10-10T20:19:23.494991+0000","last_clean":"2023-10-10T20:19:23.494991+0000","last_became_active":"2023-10-10T20:19:23.494129+0000","last_became_peered":"2023-10-10T20:19:23.494129+0000","last_unstale":"2023-10-10T20:19:23.494991+0000","last_undegraded":"2023-10-10T20:19:23.494991+0000","last_fullsized":"2023-10-10T20:19:23.494991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493767+0000","last_change":"2023-10-10T20:19:23.489565+0000","last_active":"2023-10-10T20:19:23.493767+0000","last_peered":"2023-10-10T20:19:23.493767+0000","last_clean":"2023-10-10T20:19:23.493767+0000","last_became_active":"2023-10-10T20:19:23.489263+0000","last_became_peered":"2023-10-10T20:19:23.489263+0000","last_unstale":"2023-10-10T20:19:23.493767+0000","last_undegraded":"2023-10-10T20:19:23.493767+0000","last_fullsized":"2023-10-10T20:19:23.493767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490726+0000","last_change":"2023-10-10T20:19:23.490528+0000","last_active":"2023-10-10T20:19:23.490726+0000","last_peered":"2023-10-10T20:19:23.490726+0000","last_clean":"2023-10-10T20:19:23.490726+0000","last_became_active":"2023-10-10T20:19:23.490284+0000","last_became_peered":"2023-10-10T20:19:23.490284+0000","last_unstale":"2023-10-10T20:19:23.490726+0000","last_undegraded":"2023-10-10T20:19:23.490726+0000","last_fullsized":"2023-10-10T20:19:23.490726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490090+0000","last_change":"2023-10-10T20:19:23.489820+0000","last_active":"2023-10-10T20:19:23.490090+0000","last_peered":"2023-10-10T20:19:23.490090+0000","last_clean":"2023-10-10T20:19:23.490090+0000","last_became_active":"2023-10-10T20:19:23.489559+0000","last_became_peered":"2023-10-10T20:19:23.489559+0000","last_unstale":"2023-10-10T20:19:23.490090+0000","last_undegraded":"2023-10-10T20:19:23.490090+0000","last_fullsized":"2023-10-10T20:19:23.490090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485528+0000","last_change":"2023-10-10T20:19:23.485467+0000","last_active":"2023-10-10T20:19:23.485528+0000","last_peered":"2023-10-10T20:19:23.485528+0000","last_clean":"2023-10-10T20:19:23.485528+0000","last_became_active":"2023-10-10T20:19:23.485195+0000","last_became_peered":"2023-10-10T20:19:23.485195+0000","last_unstale":"2023-10-10T20:19:23.485528+0000","last_undegraded":"2023-10-10T20:19:23.485528+0000","last_fullsized":"2023-10-10T20:19:23.485528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487121+0000","last_change":"2023-10-10T20:19:23.486697+0000","last_active":"2023-10-10T20:19:23.487121+0000","last_peered":"2023-10-10T20:19:23.487121+0000","last_clean":"2023-10-10T20:19:23.487121+0000","last_became_active":"2023-10-10T20:19:23.486167+0000","last_became_peered":"2023-10-10T20:19:23.486167+0000","last_unstale":"2023-10-10T20:19:23.487121+0000","last_undegraded":"2023-10-10T20:19:23.487121+0000","last_fullsized":"2023-10-10T20:19:23.487121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:23.488871+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:23.488509+0000","last_became_peered":"2023-10-10T20:19:23.488509+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498221+0000","last_change":"2023-10-10T20:19:23.497323+0000","last_active":"2023-10-10T20:19:23.498221+0000","last_peered":"2023-10-10T20:19:23.498221+0000","last_clean":"2023-10-10T20:19:23.498221+0000","last_became_active":"2023-10-10T20:19:23.496823+0000","last_became_peered":"2023-10-10T20:19:23.496823+0000","last_unstale":"2023-10-10T20:19:23.498221+0000","last_undegraded":"2023-10-10T20:19:23.498221+0000","last_fullsized":"2023-10-10T20:19:23.498221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489076+0000","last_change":"2023-10-10T20:19:23.488948+0000","last_active":"2023-10-10T20:19:23.489076+0000","last_peered":"2023-10-10T20:19:23.489076+0000","last_clean":"2023-10-10T20:19:23.489076+0000","last_became_active":"2023-10-10T20:19:23.488700+0000","last_became_peered":"2023-10-10T20:19:23.488700+0000","last_unstale":"2023-10-10T20:19:23.489076+0000","last_undegraded":"2023-10-10T20:19:23.489076+0000","last_fullsized":"2023-10-10T20:19:23.489076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492260+0000","last_change":"2023-10-10T20:19:23.492008+0000","last_active":"2023-10-10T20:19:23.492260+0000","last_peered":"2023-10-10T20:19:23.492260+0000","last_clean":"2023-10-10T20:19:23.492260+0000","last_became_active":"2023-10-10T20:19:23.491368+0000","last_became_peered":"2023-10-10T20:19:23.491368+0000","last_unstale":"2023-10-10T20:19:23.492260+0000","last_undegraded":"2023-10-10T20:19:23.492260+0000","last_fullsized":"2023-10-10T20:19:23.492260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487136+0000","last_change":"2023-10-10T20:19:23.486830+0000","last_active":"2023-10-10T20:19:23.487136+0000","last_peered":"2023-10-10T20:19:23.487136+0000","last_clean":"2023-10-10T20:19:23.487136+0000","last_became_active":"2023-10-10T20:19:23.486468+0000","last_became_peered":"2023-10-10T20:19:23.486468+0000","last_unstale":"2023-10-10T20:19:23.487136+0000","last_undegraded":"2023-10-10T20:19:23.487136+0000","last_fullsized":"2023-10-10T20:19:23.487136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495811+0000","last_change":"2023-10-10T20:19:23.495638+0000","last_active":"2023-10-10T20:19:23.495811+0000","last_peered":"2023-10-10T20:19:23.495811+0000","last_clean":"2023-10-10T20:19:23.495811+0000","last_became_active":"2023-10-10T20:19:23.495020+0000","last_became_peered":"2023-10-10T20:19:23.495020+0000","last_unstale":"2023-10-10T20:19:23.495811+0000","last_undegraded":"2023-10-10T20:19:23.495811+0000","last_fullsized":"2023-10-10T20:19:23.495811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491834+0000","last_change":"2023-10-10T20:19:23.491773+0000","last_active":"2023-10-10T20:19:23.491834+0000","last_peered":"2023-10-10T20:19:23.491834+0000","last_clean":"2023-10-10T20:19:23.491834+0000","last_became_active":"2023-10-10T20:19:23.491189+0000","last_became_peered":"2023-10-10T20:19:23.491189+0000","last_unstale":"2023-10-10T20:19:23.491834+0000","last_undegraded":"2023-10-10T20:19:23.491834+0000","last_fullsized":"2023-10-10T20:19:23.491834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499362+0000","last_change":"2023-10-10T20:19:23.499299+0000","last_active":"2023-10-10T20:19:23.499362+0000","last_peered":"2023-10-10T20:19:23.499362+0000","last_clean":"2023-10-10T20:19:23.499362+0000","last_became_active":"2023-10-10T20:19:23.498950+0000","last_became_peered":"2023-10-10T20:19:23.498950+0000","last_unstale":"2023-10-10T20:19:23.499362+0000","last_undegraded":"2023-10-10T20:19:23.499362+0000","last_fullsized":"2023-10-10T20:19:23.499362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929346,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":644,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339539,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":67,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929380,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929416,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:32.451 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:19:32.452 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-10T20:19:32.452 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-10T20:19:32.453 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-10T20:19:32.453 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-10T20:19:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.600+0000 7fcab65b9700 1 Processor -- start 2023-10-10T20:19:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.600+0000 7fcab65b9700 1 -- start start 2023-10-10T20:19:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab00cd6e0 0x7fcab00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 0x7fcab015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcab015dda0 0x7fcab015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab015e850 con 0x7fcab015dda0 2023-10-10T20:19:32.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab00cd030 con 0x7fcab00cdab0 2023-10-10T20:19:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab65b9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00cb7c0 con 0x7fcab00cd6e0 2023-10-10T20:19:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcab015dda0 0x7fcab015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcab015dda0 0x7fcab015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42200/0 (socket says 172.21.15.148:42200) 2023-10-10T20:19:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 -- 172.21.15.148:0/3405455145 learned_addr learned my addr 172.21.15.148:0/3405455145 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab4db6700 1 --2- 172.21.15.148:0/3405455145 >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 0x7fcab015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab55b7700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab00cd6e0 0x7fcab00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 -- 172.21.15.148:0/3405455145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab00cd6e0 msgr2=0x7fcab00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab00cd6e0 0x7fcab00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 -- 172.21.15.148:0/3405455145 >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 msgr2=0x7fcab015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 --2- 172.21.15.148:0/3405455145 >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 0x7fcab015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 -- 172.21.15.148:0/3405455145 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab015e290 con 0x7fcab015dda0 2023-10-10T20:19:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab4db6700 1 --2- 172.21.15.148:0/3405455145 >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 0x7fcab015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:32.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.601+0000 7fcab5db8700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab015dda0 0x7fcab015cbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcab0089b50 tx=0x7fcaa8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f98d196b56ef7d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcaa7fff700 1 -- 172.21.15.148:0/3405455145 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800e070 con 0x7fcab015dda0 2023-10-10T20:19:32.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcaa7fff700 1 -- 172.21.15.148:0/3405455145 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa800ba50 con 0x7fcab015dda0 2023-10-10T20:19:32.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcaa7fff700 1 -- 172.21.15.148:0/3405455145 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80092e0 con 0x7fcab015dda0 2023-10-10T20:19:32.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 -- 172.21.15.148:0/3405455145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab015dda0 msgr2=0x7fcab015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab015dda0 0x7fcab015cbd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcab0089b50 tx=0x7fcaa8009840 comp rx=0 tx=0).stop 2023-10-10T20:19:32.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 -- 172.21.15.148:0/3405455145 shutdown_connections 2023-10-10T20:19:32.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab015dda0 0x7fcab015cbd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3405455145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab00cd6e0 0x7fcab00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3405455145 >> v2:172.21.15.161:3300/0 conn(0x7fcab00cdab0 0x7fcab015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 -- 172.21.15.148:0/3405455145 >> 172.21.15.148:0/3405455145 conn(0x7fcab0067660 msgr2=0x7fcab0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:32.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 -- 172.21.15.148:0/3405455145 shutdown_connections 2023-10-10T20:19:32.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.602+0000 7fcab65b9700 1 -- 172.21.15.148:0/3405455145 wait complete. 2023-10-10T20:19:32.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.603+0000 7fcab65b9700 1 Processor -- start 2023-10-10T20:19:32.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.603+0000 7fcab65b9700 1 -- start start 2023-10-10T20:19:32.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 0x7fcab00ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab015dda0 0x7fcab00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00c8880 con 0x7fcab00cdab0 2023-10-10T20:19:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00ca5d0 con 0x7fcab00cd6e0 2023-10-10T20:19:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab65b9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00ca330 con 0x7fcab015dda0 2023-10-10T20:19:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab4db6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab4db6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42202/0 (socket says 172.21.15.148:42202) 2023-10-10T20:19:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab4db6700 1 -- 172.21.15.148:0/3178241637 learned_addr learned my addr 172.21.15.148:0/3178241637 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab55b7700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 0x7fcab00ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.604+0000 7fcab5db8700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab015dda0 0x7fcab00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 -- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab015dda0 msgr2=0x7fcab00c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab015dda0 0x7fcab00c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 -- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 msgr2=0x7fcab00ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 0x7fcab00ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 -- 172.21.15.148:0/3178241637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8009ae0 con 0x7fcab00cdab0 2023-10-10T20:19:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab55b7700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 0x7fcab00ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab4db6700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e8d0 tx=0x7fcaa0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000ee70 con 0x7fcab00cdab0 2023-10-10T20:19:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000c5e0 con 0x7fcab00cdab0 2023-10-10T20:19:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000c980 con 0x7fcab00cdab0 2023-10-10T20:19:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab0159050 con 0x7fcab00cdab0 2023-10-10T20:19:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.605+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab00c8cd0 con 0x7fcab00cdab0 2023-10-10T20:19:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.606+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab00c89f0 con 0x7fcab00cdab0 2023-10-10T20:19:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.606+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcaa0013410 con 0x7fcab00cdab0 2023-10-10T20:19:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.607+0000 7fcaa57fa700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 0x7fca940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.607+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcaa0084320 con 0x7fcab00cdab0 2023-10-10T20:19:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.607+0000 7fcab55b7700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 0x7fca940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.607+0000 7fcab55b7700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 0x7fca940693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcab00b0b90 tx=0x7fcaac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.611+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcab00c89f0 con 0x7fcab00cdab0 2023-10-10T20:19:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.849+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcab00c89c0 con 0x7fcab00cdab0 2023-10-10T20:19:32.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.852+0000 7fcaa57fa700 1 -- 172.21.15.148:0/3178241637 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7fcaa0055020 con 0x7fcab00cdab0 2023-10-10T20:19:32.856 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:32.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 msgr2=0x7fca940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 0x7fca940693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcab00b0b90 tx=0x7fcaac008040 comp rx=0 tx=0).stop 2023-10-10T20:19:32.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 msgr2=0x7fcab0154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e8d0 tx=0x7fcaa0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:32.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 shutdown_connections 2023-10-10T20:19:32.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca94066f30 0x7fca940693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab015dda0 0x7fcab00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab00cd6e0 0x7fcab00ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 --2- 172.21.15.148:0/3178241637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab00cdab0 0x7fcab0154da0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 >> 172.21.15.148:0/3178241637 conn(0x7fcab0067660 msgr2=0x7fcab00b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:32.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 shutdown_connections 2023-10-10T20:19:32.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.855+0000 7fcab65b9700 1 -- 172.21.15.148:0/3178241637 wait complete. 2023-10-10T20:19:32.869 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":86081},{"type":"v1","addr":"172.21.15.148:6833","nonce":86081}]},"active_addr":"172.21.15.148:6833/86081","active_change":"2023-10-10T20:19:11.313822+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4038993398}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1999836899}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4037647824}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1481141478}]}]} 2023-10-10T20:19:32.872 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-10T20:19:32.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-10T20:19:32.873 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.986+0000 7f3f48443700 1 Processor -- start 2023-10-10T20:19:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.986+0000 7f3f48443700 1 -- start start 2023-10-10T20:19:32.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f48443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f4015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f48443700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3f40157c40 0x7f3f40158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f48443700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4015cbb0 con 0x7f3f4015a620 2023-10-10T20:19:32.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f48443700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4015bfa0 con 0x7f3f40157c40 2023-10-10T20:19:32.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f48443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4015c280 con 0x7f3f40157870 2023-10-10T20:19:32.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f459de700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3f4015a620 0x7f3f4015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58206/0 (socket says 172.21.15.148:58206) 2023-10-10T20:19:32.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f459de700 1 -- 172.21.15.148:0/2034320003 learned_addr learned my addr 172.21.15.148:0/2034320003 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:32.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f469e0700 1 --2- 172.21.15.148:0/2034320003 >> v2:172.21.15.161:3300/0 conn(0x7f3f40157c40 0x7f3f40158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f461df700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f4015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723386441 0 0) 0x7f3f4015cbb0 con 0x7f3f4015a620 2023-10-10T20:19:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.987+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003570 con 0x7f3f4015a620 2023-10-10T20:19:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1632010286 0 0) 0x7f3f2c003570 con 0x7f3f4015a620 2023-10-10T20:19:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 msgr2=0x7f3f4015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f4015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 >> v2:172.21.15.161:3300/0 conn(0x7f3f40157c40 msgr2=0x7f3f40158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 --2- 172.21.15.148:0/2034320003 >> v2:172.21.15.161:3300/0 conn(0x7f3f40157c40 0x7f3f40158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4015c560 con 0x7f3f4015a620 2023-10-10T20:19:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f3f30001470 con 0x7f3f4015a620 2023-10-10T20:19:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 >> v1:172.21.15.148:6789/0 conn(0x7f3f4015a620 legacy=0x7f3f4015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f2c003a60 0x7f3f2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4015bfa0 con 0x7f3f2c004810 2023-10-10T20:19:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4015c280 con 0x7f3f2c003a60 2023-10-10T20:19:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c014270 con 0x7f3f40157870 2023-10-10T20:19:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f461df700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f2c003a60 0x7f3f2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f469e0700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 -- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 msgr2=0x7f3f2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f40157870 0x7f3f2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 -- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f2c003a60 msgr2=0x7f3f2c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f2c003a60 0x7f3f2c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 -- 172.21.15.148:0/2034320003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c003570 con 0x7f3f2c004810 2023-10-10T20:19:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f459de700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c3f0 tx=0x7f3f30001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9fee8a9d4e8162 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000cc80 con 0x7f3f2c004810 2023-10-10T20:19:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3000ce00 con 0x7f3f2c004810 2023-10-10T20:19:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.988+0000 7f3f451dd700 1 -- 172.21.15.148:0/2034320003 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30005b70 con 0x7f3f2c004810 2023-10-10T20:19:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 -- 172.21.15.148:0/2034320003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c004810 msgr2=0x7f3f2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 --2- 172.21.15.148:0/2034320003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c3f0 tx=0x7f3f30001e90 comp rx=0 tx=0).stop 2023-10-10T20:19:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f469e0700 1 -- 172.21.15.148:0/2034320003 reap_dead start 2023-10-10T20:19:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 -- 172.21.15.148:0/2034320003 shutdown_connections 2023-10-10T20:19:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 -- 172.21.15.148:0/2034320003 >> 172.21.15.148:0/2034320003 conn(0x7f3f40068ce0 msgr2=0x7f3f400690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 -- 172.21.15.148:0/2034320003 shutdown_connections 2023-10-10T20:19:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.989+0000 7f3f48443700 1 -- 172.21.15.148:0/2034320003 wait complete. 2023-10-10T20:19:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.990+0000 7f3f48443700 1 Processor -- start 2023-10-10T20:19:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.990+0000 7f3f48443700 1 -- start start 2023-10-10T20:19:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f2c004810 0x7f3f40166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f400b5e50 0x7f3f40153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40153db0 con 0x7f3f40152740 2023-10-10T20:19:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4014f150 con 0x7f3f400b5e50 2023-10-10T20:19:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f48443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40166d60 con 0x7f3f2c004810 2023-10-10T20:19:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42226/0 (socket says 172.21.15.148:42226) 2023-10-10T20:19:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 -- 172.21.15.148:0/6351014 learned_addr learned my addr 172.21.15.148:0/6351014 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f461df700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f2c004810 0x7f3f40166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f459de700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f400b5e50 0x7f3f40153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 -- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f2c004810 msgr2=0x7f3f40166750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f2c004810 0x7f3f40166750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 -- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f400b5e50 msgr2=0x7f3f40153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f400b5e50 0x7f3f40153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.991+0000 7f3f469e0700 1 -- 172.21.15.148:0/6351014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40167040 con 0x7f3f40152740 2023-10-10T20:19:33.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f469e0700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f4015b630 tx=0x7f3f3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800e040 con 0x7f3f40152740 2023-10-10T20:19:33.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3800f040 con 0x7f3f40152740 2023-10-10T20:19:33.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f40167320 con 0x7f3f40152740 2023-10-10T20:19:33.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800fd70 con 0x7f3f40152740 2023-10-10T20:19:33.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.992+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f400b62c0 con 0x7f3f40152740 2023-10-10T20:19:33.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.993+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4004c3c0 con 0x7f3f40152740 2023-10-10T20:19:33.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.993+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f38015640 con 0x7f3f40152740 2023-10-10T20:19:33.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.993+0000 7f3f367fc700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 0x7f3f240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.993+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f38084cc0 con 0x7f3f40152740 2023-10-10T20:19:33.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.993+0000 7f3f461df700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 0x7f3f240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.994+0000 7f3f461df700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 0x7f3f240693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c006f00 tx=0x7f3f3c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:32.998+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f38051f30 con 0x7f3f40152740 2023-10-10T20:19:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.180+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f4015c840 con 0x7f3f40152740 2023-10-10T20:19:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.181+0000 7f3f367fc700 1 -- 172.21.15.148:0/6351014 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f3f38055550 con 0x7f3f40152740 2023-10-10T20:19:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:33.185 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:33.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.182+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 msgr2=0x7f3f240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.182+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 0x7f3f240693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c006f00 tx=0x7f3f3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.182+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 msgr2=0x7f3f40151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.182+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f4015b630 tx=0x7f3f3800b040 comp rx=0 tx=0).stop 2023-10-10T20:19:33.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 shutdown_connections 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3f24066f30 0x7f3f240693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40152740 0x7f3f40151910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3f2c004810 0x7f3f40166750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 --2- 172.21.15.148:0/6351014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f400b5e50 0x7f3f40153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 >> 172.21.15.148:0/6351014 conn(0x7f3f40068ce0 msgr2=0x7f3f40158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 shutdown_connections 2023-10-10T20:19:33.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.183+0000 7f3f48443700 1 -- 172.21.15.148:0/6351014 wait complete. 2023-10-10T20:19:33.197 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-10T20:19:33.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:33.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.309+0000 7f2378b9d700 1 Processor -- start 2023-10-10T20:19:33.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.309+0000 7f2378b9d700 1 -- start start 2023-10-10T20:19:33.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2378b9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2374157a90 0x7f23740b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2378b9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f23740ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2378b9d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740cb0b0 con 0x7f2374157e60 2023-10-10T20:19:33.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2378b9d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740b0ad0 con 0x7f2374157a90 2023-10-10T20:19:33.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2378b9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b4970 con 0x7f237406fef0 2023-10-10T20:19:33.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2372ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2374157e60 0x7f23740b0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58222/0 (socket says 172.21.15.148:58222) 2023-10-10T20:19:33.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2372ffd700 1 -- 172.21.15.148:0/2813991808 learned_addr learned my addr 172.21.15.148:0/2813991808 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:33.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f23737fe700 1 --2- 172.21.15.148:0/2813991808 >> v2:172.21.15.161:3300/0 conn(0x7f2374157a90 0x7f23740b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f2373fff700 1 --2- 172.21.15.148:0/2813991808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f23740ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394317777 0 0) 0x7f23740cb0b0 con 0x7f2374157e60 2023-10-10T20:19:33.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.310+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c003570 con 0x7f2374157e60 2023-10-10T20:19:33.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2373fff700 1 -- 172.21.15.148:0/2813991808 >> v2:172.21.15.161:3300/0 conn(0x7f2374157a90 msgr2=0x7f23740b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2373fff700 1 --2- 172.21.15.148:0/2813991808 >> v2:172.21.15.161:3300/0 conn(0x7f2374157a90 0x7f23740b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2373fff700 1 -- 172.21.15.148:0/2813991808 >> v1:172.21.15.148:6789/0 conn(0x7f2374157e60 legacy=0x7f23740b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374616276 0 0) 0x7f235c003570 con 0x7f2374157e60 2023-10-10T20:19:33.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2373fff700 1 -- 172.21.15.148:0/2813991808 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740b0830 con 0x7f237406fef0 2023-10-10T20:19:33.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2373fff700 1 --2- 172.21.15.148:0/2813991808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f23740ce950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f236c0098e0 tx=0x7f236c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a8aac0fa2235019 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368001470 con 0x7f237406fef0 2023-10-10T20:19:33.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c011420 con 0x7f237406fef0 2023-10-10T20:19:33.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f23727fc700 1 -- 172.21.15.148:0/2813991808 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0117a0 con 0x7f237406fef0 2023-10-10T20:19:33.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 -- 172.21.15.148:0/2813991808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 msgr2=0x7f23740ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 --2- 172.21.15.148:0/2813991808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f23740ce950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f236c0098e0 tx=0x7f236c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 -- 172.21.15.148:0/2813991808 shutdown_connections 2023-10-10T20:19:33.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 --2- 172.21.15.148:0/2813991808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f23740ce950 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 --2- 172.21.15.148:0/2813991808 >> v2:172.21.15.161:3300/0 conn(0x7f2374157a90 0x7f23740b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 -- 172.21.15.148:0/2813991808 >> 172.21.15.148:0/2813991808 conn(0x7f237406bc70 msgr2=0x7f237406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 -- 172.21.15.148:0/2813991808 shutdown_connections 2023-10-10T20:19:33.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.311+0000 7f2378b9d700 1 -- 172.21.15.148:0/2813991808 wait complete. 2023-10-10T20:19:33.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 Processor -- start 2023-10-10T20:19:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 -- start start 2023-10-10T20:19:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f237415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2374157e60 0x7f23741681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237415e6d0 con 0x7f2374157a90 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237415dd00 con 0x7f2374157e60 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2378b9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740cde80 con 0x7f237406fef0 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.313+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42242/0 (socket says 172.21.15.148:42242) 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 -- 172.21.15.148:0/966130274 learned_addr learned my addr 172.21.15.148:0/966130274 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2373fff700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2374157e60 0x7f23741681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f23737fe700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f237415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 -- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 msgr2=0x7f237415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f237415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 -- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2374157e60 msgr2=0x7f23741681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2374157e60 0x7f23741681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 -- 172.21.15.148:0/966130274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c009420 con 0x7f2374157a90 2023-10-10T20:19:33.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2372ffd700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f236800cbc0 tx=0x7f2368001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007150 con 0x7f2374157a90 2023-10-10T20:19:33.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680072d0 con 0x7f2374157a90 2023-10-10T20:19:33.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237415d0f0 con 0x7f2374157a90 2023-10-10T20:19:33.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007690 con 0x7f2374157a90 2023-10-10T20:19:33.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.314+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237415d6b0 con 0x7f2374157a90 2023-10-10T20:19:33.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.315+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237406e3e0 con 0x7f2374157a90 2023-10-10T20:19:33.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.316+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2368004a40 con 0x7f2374157a90 2023-10-10T20:19:33.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.316+0000 7f235b7fe700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 0x7f23600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.316+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2368083bd0 con 0x7f2374157a90 2023-10-10T20:19:33.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.316+0000 7f23737fe700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 0x7f23600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.316+0000 7f23737fe700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 0x7f23600693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f236400f570 tx=0x7f236400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.320+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2368050ec0 con 0x7f2374157a90 2023-10-10T20:19:33.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.503+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23740ca780 con 0x7f2374157a90 2023-10-10T20:19:33.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.504+0000 7f235b7fe700 1 -- 172.21.15.148:0/966130274 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2368018020 con 0x7f2374157a90 2023-10-10T20:19:33.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:33.507 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:33.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 msgr2=0x7f23600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 0x7f23600693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f236400f570 tx=0x7f236400b040 comp rx=0 tx=0).stop 2023-10-10T20:19:33.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 msgr2=0x7f23740cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f236800cbc0 tx=0x7f2368001a30 comp rx=0 tx=0).stop 2023-10-10T20:19:33.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 shutdown_connections 2023-10-10T20:19:33.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2360066f30 0x7f23600693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2374157e60 0x7f23741681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406fef0 0x7f237415e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 --2- 172.21.15.148:0/966130274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2374157a90 0x7f23740cd940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 >> 172.21.15.148:0/966130274 conn(0x7f237406bc70 msgr2=0x7f23740c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 shutdown_connections 2023-10-10T20:19:33.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.506+0000 7f2378b9d700 1 -- 172.21.15.148:0/966130274 wait complete. 2023-10-10T20:19:33.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-10T20:19:33.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-10T20:19:33.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-10T20:19:33.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-10T20:19:33.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-10T20:19:33.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-10T20:19:33.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-10T20:19:33.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-10T20:19:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.710+0000 7f3414112700 1 Processor -- start 2023-10-10T20:19:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.711+0000 7f3414112700 1 -- start start 2023-10-10T20:19:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f3414112700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c06d2e0 0x7f340c0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f3414112700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f340c06d6b0 0x7f340c0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f3414112700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0c8760 con 0x7f340c155b60 2023-10-10T20:19:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f3414112700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0c7930 con 0x7f340c06d6b0 2023-10-10T20:19:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f3414112700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0af6e0 con 0x7f340c06d2e0 2023-10-10T20:19:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f34116ad700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f340c06d6b0 0x7f340c0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f34116ad700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f340c06d6b0 0x7f340c0c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:41926/0 (socket says 172.21.15.148:41926) 2023-10-10T20:19:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.712+0000 7f34116ad700 1 -- 172.21.15.148:0/1824281855 learned_addr learned my addr 172.21.15.148:0/1824281855 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.713+0000 7f3411eae700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c06d2e0 0x7f340c0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.714+0000 7f34116ad700 1 -- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c06d2e0 msgr2=0x7f340c0c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.714+0000 7f34116ad700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c06d2e0 0x7f340c0c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.714+0000 7f34116ad700 1 -- 172.21.15.148:0/1824281855 >> v1:172.21.15.148:6789/0 conn(0x7f340c155b60 legacy=0x7f340c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.714+0000 7f34116ad700 1 -- 172.21.15.148:0/1824281855 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0c7b50 con 0x7f340c06d6b0 2023-10-10T20:19:33.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.715+0000 7f34116ad700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 0x7f340c0c73f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f340400d1a0 tx=0x7f3404007690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9277c1cd1ecc4e0a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.715+0000 7f3410eac700 1 -- 172.21.15.148:0/1824281855 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404010070 con 0x7f340c06d6b0 2023-10-10T20:19:33.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.718+0000 7f3410eac700 1 -- 172.21.15.148:0/1824281855 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404007b40 con 0x7f340c06d6b0 2023-10-10T20:19:33.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.719+0000 7f3410eac700 1 -- 172.21.15.148:0/1824281855 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400db00 con 0x7f340c06d6b0 2023-10-10T20:19:33.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.719+0000 7f3414112700 1 -- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 msgr2=0x7f340c0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.719+0000 7f3414112700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 0x7f340c0c73f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f340400d1a0 tx=0x7f3404007690 comp rx=0 tx=0).stop 2023-10-10T20:19:33.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.720+0000 7f3414112700 1 -- 172.21.15.148:0/1824281855 shutdown_connections 2023-10-10T20:19:33.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.720+0000 7f3414112700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c06d2e0 0x7f340c0c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.720+0000 7f3414112700 1 --2- 172.21.15.148:0/1824281855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 0x7f340c0c73f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.720+0000 7f3414112700 1 -- 172.21.15.148:0/1824281855 >> 172.21.15.148:0/1824281855 conn(0x7f340c070700 msgr2=0x7f340c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.720+0000 7f01bc40c700 1 Processor -- start 2023-10-10T20:19:33.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.721+0000 7f01bc40c700 1 -- start start 2023-10-10T20:19:33.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.721+0000 7f3414112700 1 -- 172.21.15.148:0/1824281855 shutdown_connections 2023-10-10T20:19:33.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.721+0000 7f3414112700 1 -- 172.21.15.148:0/1824281855 wait complete. 2023-10-10T20:19:33.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.721+0000 7fcc6e8da700 1 Processor -- start 2023-10-10T20:19:33.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.721+0000 7fcc6e8da700 1 -- start start 2023-10-10T20:19:33.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7fd99eb85700 1 Processor -- start 2023-10-10T20:19:33.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01bc40c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f01b415a080 0x7f01b40adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01bc40c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01b40af6a0 0x7f01b40ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01bc40c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b415a500 con 0x7f01b406eab0 2023-10-10T20:19:33.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01bc40c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b415afe0 con 0x7f01b40af6a0 2023-10-10T20:19:33.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01bc40c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40ae210 con 0x7f01b415a080 2023-10-10T20:19:33.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01ba1a8700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f01b406eab0 0x7f01b406ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48030/0 (socket says 172.21.15.148:48030) 2023-10-10T20:19:33.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01ba1a8700 1 -- 172.21.15.148:0/1087201519 learned_addr learned my addr 172.21.15.148:0/1087201519 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:33.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f01ba9a9700 1 --2- 172.21.15.148:0/1087201519 >> v2:172.21.15.148:3300/0 conn(0x7f01b40af6a0 0x7f01b40ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7fd99eb85700 1 -- start start 2023-10-10T20:19:33.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6e8da700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc68157270 0x7fcc680c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6e8da700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcc6806e2b0 0x7fcc6806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6e8da700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680c42b0 con 0x7fcc68156ea0 2023-10-10T20:19:33.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6e8da700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6806fac0 con 0x7fcc68157270 2023-10-10T20:19:33.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6e8da700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6806ec10 con 0x7fcc6806e2b0 2023-10-10T20:19:33.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f5b79120700 1 Processor -- start 2023-10-10T20:19:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.722+0000 7f5b79120700 1 -- start start 2023-10-10T20:19:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01ba9a9700 1 -- 172.21.15.148:0/1087201519 >> v2:172.21.15.161:3300/0 conn(0x7f01b415a080 msgr2=0x7f01b40adcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:33.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01ba9a9700 1 --2- 172.21.15.148:0/1087201519 >> v2:172.21.15.161:3300/0 conn(0x7f01b415a080 0x7f01b40adcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01ba9a9700 1 -- 172.21.15.148:0/1087201519 >> v1:172.21.15.161:6789/0 conn(0x7f01b406eab0 legacy=0x7f01b406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01ba9a9700 1 -- 172.21.15.148:0/1087201519 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40accd0 con 0x7f01b40af6a0 2023-10-10T20:19:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcc6806e2b0 0x7fcc6806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc67fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcc68156ea0 0x7fcc6806f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48032/0 (socket says 172.21.15.148:48032) 2023-10-10T20:19:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc67fff700 1 -- 172.21.15.148:0/1711073189 learned_addr learned my addr 172.21.15.148:0/1711073189 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f3414112700 1 Processor -- start 2023-10-10T20:19:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f3414112700 1 -- start start 2023-10-10T20:19:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01ba9a9700 1 --2- 172.21.15.148:0/1087201519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b40ad160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01b40c93b0 tx=0x7f01ac00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd5e273feb52fbc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99eb85700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99eb85700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99806da40 con 0x7fd9980c75c0 2023-10-10T20:19:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99eb85700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9980c8760 con 0x7fd99806d5d0 2023-10-10T20:19:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99eb85700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99815cfc0 con 0x7fd9980cc450 2023-10-10T20:19:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99c921700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc66ffd700 1 -- 172.21.15.148:0/1711073189 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950764314 0 0) 0x7fcc680c42b0 con 0x7fcc68156ea0 2023-10-10T20:19:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc66ffd700 1 -- 172.21.15.148:0/1711073189 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc50003570 con 0x7fcc68156ea0 2023-10-10T20:19:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 -- 172.21.15.148:0/1711073189 >> v2:172.21.15.148:3300/0 conn(0x7fcc68157270 msgr2=0x7fcc680c30c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 --2- 172.21.15.148:0/1711073189 >> v2:172.21.15.148:3300/0 conn(0x7fcc68157270 0x7fcc680c30c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 -- 172.21.15.148:0/1711073189 >> v1:172.21.15.161:6789/0 conn(0x7fcc68156ea0 legacy=0x7fcc6806f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 -- 172.21.15.148:0/1711073189 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680c36a0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01b91a6700 1 -- 172.21.15.148:0/1087201519 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00a2e0 con 0x7f01b40af6a0 2023-10-10T20:19:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f5b79120700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415e2b0 0x7f5b740affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99c921700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:50932/0 (socket says 172.21.15.148:50932) 2023-10-10T20:19:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fd99c921700 1 -- 172.21.15.148:0/1370607692 learned_addr learned my addr 172.21.15.148:0/1370607692 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f5b79120700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5b7415e680 0x7f5b74161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01b91a6700 1 -- 172.21.15.148:0/1087201519 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00ca90 con 0x7f01b40af6a0 2023-10-10T20:19:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f01b91a6700 1 -- 172.21.15.148:0/1087201519 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00ce50 con 0x7f01b40af6a0 2023-10-10T20:19:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7fcc6ce77700 1 --2- 172.21.15.148:0/1711073189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6806e6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc680b0620 tx=0x7fcc6000b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3be634d41dbd068 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fd99c921700 1 -- 172.21.15.148:0/1370607692 >> v1:172.21.15.161:6789/0 conn(0x7fd99806d5d0 legacy=0x7fd9980c6eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fd99c921700 1 -- 172.21.15.148:0/1370607692 >> v1:172.21.15.148:6789/0 conn(0x7fd9980c75c0 legacy=0x7fd998155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f5b79120700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b740af3f0 con 0x7f5b740b2e00 2023-10-10T20:19:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f5b79120700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b740af6d0 con 0x7f5b7415e680 2023-10-10T20:19:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.723+0000 7f5b79120700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740af9b0 con 0x7f5b7415e2b0 2023-10-10T20:19:33.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fd99c921700 1 -- 172.21.15.148:0/1370607692 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980c7b50 con 0x7fd9980cc450 2023-10-10T20:19:33.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f5b7359e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5b740b2e00 0x7f5b741624a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58240/0 (socket says 172.21.15.148:58240) 2023-10-10T20:19:33.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f5b7359e700 1 -- 172.21.15.148:0/992755811 learned_addr learned my addr 172.21.15.148:0/992755811 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:33.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f9a0baa1700 1 Processor -- start 2023-10-10T20:19:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fcc66ffd700 1 -- 172.21.15.148:0/1711073189 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000b2e0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fcc66ffd700 1 -- 172.21.15.148:0/1711073189 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6000da30 con 0x7fcc6806e2b0 2023-10-10T20:19:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fcc66ffd700 1 -- 172.21.15.148:0/1711073189 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000ddf0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7fd99c921700 1 --2- 172.21.15.148:0/1370607692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099e0 tx=0x7fd98c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391943af6ef11d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370376299 0 0) 0x7f5b740af3f0 con 0x7f5b740b2e00 2023-10-10T20:19:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b5c003570 con 0x7f5b740b2e00 2023-10-10T20:19:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 0x7f340c16f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 0x7f340c165050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c155b60 0x7f340c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c16f710 con 0x7f340c06d2e0 2023-10-10T20:19:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c21c0 con 0x7f340c06d6b0 2023-10-10T20:19:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3414112700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c24a0 con 0x7f340c155b60 2023-10-10T20:19:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3411eae700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 0x7f340c16f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3411eae700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 0x7f340c16f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42252/0 (socket says 172.21.15.148:42252) 2023-10-10T20:19:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f3411eae700 1 -- 172.21.15.148:0/4292731124 learned_addr learned my addr 172.21.15.148:0/4292731124 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f34126af700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c155b60 0x7f340c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f01bc40c700 1 -- 172.21.15.148:0/1087201519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 msgr2=0x7f01b40ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f01bc40c700 1 --2- 172.21.15.148:0/1087201519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b40ad160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01b40c93b0 tx=0x7f01ac00a890 comp rx=0 tx=0).stop 2023-10-10T20:19:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 -- 172.21.15.148:0/4292731124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c155b60 msgr2=0x7f340c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340c155b60 0x7f340c0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 -- 172.21.15.148:0/4292731124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 msgr2=0x7f340c165050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f340c06d6b0 0x7f340c165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340400be70 con 0x7f340c06d2e0 2023-10-10T20:19:33.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1711073189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 msgr2=0x7fcc6806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1711073189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6806e6d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc680b0620 tx=0x7fcc6000b990 comp rx=0 tx=0).stop 2023-10-10T20:19:33.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3411eae700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 0x7f340c16f1d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f340800ec80 tx=0x7f34080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1711073189 shutdown_connections 2023-10-10T20:19:33.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1711073189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6806e6d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1711073189 >> v2:172.21.15.148:3300/0 conn(0x7fcc68157270 0x7fcc680c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1711073189 >> 172.21.15.148:0/1711073189 conn(0x7fcc680ca4f0 msgr2=0x7fcc680ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 -- 172.21.15.148:0/1087201519 shutdown_connections 2023-10-10T20:19:33.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 --2- 172.21.15.148:0/1087201519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b40ad160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 --2- 172.21.15.148:0/1087201519 >> v2:172.21.15.161:3300/0 conn(0x7f01b415a080 0x7f01b40adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 -- 172.21.15.148:0/1087201519 >> 172.21.15.148:0/1087201519 conn(0x7f01b40cd1e0 msgr2=0x7f01b40cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 --2- 172.21.15.148:0/992755811 >> v2:172.21.15.161:3300/0 conn(0x7f5b7415e680 0x7f5b74161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2748460506 0 0) 0x7f5b5c003570 con 0x7f5b740b2e00 2023-10-10T20:19:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415e2b0 msgr2=0x7f5b740affa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415e2b0 0x7f5b740affa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 >> v2:172.21.15.161:3300/0 conn(0x7f5b7415e680 msgr2=0x7f5b74161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 --2- 172.21.15.148:0/992755811 >> v2:172.21.15.161:3300/0 conn(0x7f5b7415e680 0x7f5b74161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740b2600 con 0x7f5b740b2e00 2023-10-10T20:19:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f5b68001470 con 0x7f5b740b2e00 2023-10-10T20:19:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 >> v1:172.21.15.148:6789/0 conn(0x7f5b740b2e00 legacy=0x7f5b741624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5c003ae0 0x7f5b5c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b5c0053c0 0x7f5b5c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740af6d0 con 0x7f5b5c003ae0 2023-10-10T20:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740af9b0 con 0x7f5b5c004810 2023-10-10T20:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c00c290 con 0x7f5b5c0053c0 2023-10-10T20:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800c9d0 con 0x7f340c06d2e0 2023-10-10T20:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800cb30 con 0x7f340c06d2e0 2023-10-10T20:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408013470 con 0x7f340c06d2e0 2023-10-10T20:19:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1711073189 shutdown_connections 2023-10-10T20:19:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 -- 172.21.15.148:0/1087201519 shutdown_connections 2023-10-10T20:19:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.724+0000 7f9a0baa1700 1 -- start start 2023-10-10T20:19:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f9a0baa1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a04064650 0x7f9a040c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f9a0baa1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9a04153b50 0x7f9a04156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f9a0baa1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a040c9730 con 0x7f9a04069d00 2023-10-10T20:19:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f9a0baa1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040c8610 con 0x7f9a04064650 2023-10-10T20:19:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f9a0baa1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a04069870 con 0x7f9a04153b50 2023-10-10T20:19:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a04064650 0x7f9a040c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a04064650 0x7f9a040c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42268/0 (socket says 172.21.15.148:42268) 2023-10-10T20:19:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0903c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9a04153b50 0x7f9a04156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 -- 172.21.15.148:0/406020306 learned_addr learned my addr 172.21.15.148:0/406020306 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f1303448700 1 Processor -- start 2023-10-10T20:19:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7fd9977fe700 1 -- 172.21.15.148:0/1370607692 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00b8b0 con 0x7fd9980cc450 2023-10-10T20:19:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 -- 172.21.15.148:0/406020306 >> v2:172.21.15.161:3300/0 conn(0x7f9a04153b50 msgr2=0x7f9a04156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 --2- 172.21.15.148:0/406020306 >> v2:172.21.15.161:3300/0 conn(0x7f9a04153b50 0x7f9a04156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 -- 172.21.15.148:0/406020306 >> v1:172.21.15.148:6789/0 conn(0x7f9a04069d00 legacy=0x7f9a0406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 -- 172.21.15.148:0/406020306 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a041573e0 con 0x7f9a04064650 2023-10-10T20:19:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f9a0983d700 1 --2- 172.21.15.148:0/406020306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04064650 0x7f9a040c7670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a040b2e80 tx=0x7f99fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39065d19bd7bf54f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f1303448700 1 -- start start 2023-10-10T20:19:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f1303448700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12fc15e080 0x7f12fc0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f1303448700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0b6cc0 con 0x7f12fc0afd90 2023-10-10T20:19:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f1303448700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0b56d0 con 0x7f12fc15e450 2023-10-10T20:19:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f1303448700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc0b0230 con 0x7f12fc15e080 2023-10-10T20:19:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12fc15e080 0x7f12fc0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12fc15e080 0x7f12fc0b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42282/0 (socket says 172.21.15.148:42282) 2023-10-10T20:19:33.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f01bc40c700 1 -- 172.21.15.148:0/1087201519 wait complete. 2023-10-10T20:19:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f01bc40c700 1 Processor -- start 2023-10-10T20:19:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f9a0883b700 1 -- 172.21.15.148:0/406020306 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00e070 con 0x7f9a04064650 2023-10-10T20:19:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f9a0883b700 1 -- 172.21.15.148:0/406020306 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00ba10 con 0x7f9a04064650 2023-10-10T20:19:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f9a0883b700 1 -- 172.21.15.148:0/406020306 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0092e0 con 0x7f9a04064650 2023-10-10T20:19:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 -- 172.21.15.148:0/3729103875 learned_addr learned my addr 172.21.15.148:0/3729103875 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7fd9977fe700 1 -- 172.21.15.148:0/1370607692 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c00ba10 con 0x7fd9980cc450 2023-10-10T20:19:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7fd9977fe700 1 -- 172.21.15.148:0/1370607692 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00bdb0 con 0x7fd9980cc450 2023-10-10T20:19:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7fd99eb85700 1 -- 172.21.15.148:0/1370607692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 msgr2=0x7fd99806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7fd99eb85700 1 --2- 172.21.15.148:0/1370607692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099e0 tx=0x7fd98c009250 comp rx=0 tx=0).stop 2023-10-10T20:19:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f01bc40c700 1 -- start start 2023-10-10T20:19:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 -- 172.21.15.148:0/3729103875 >> v1:172.21.15.161:6789/0 conn(0x7f12fc15e450 legacy=0x7f12fc0b5b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 -- 172.21.15.148:0/3729103875 >> v1:172.21.15.148:6789/0 conn(0x7f12fc0afd90 legacy=0x7f12fc06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f13011e4700 1 -- 172.21.15.148:0/3729103875 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0b4da0 con 0x7f12fc15e080 2023-10-10T20:19:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1711073189 wait complete. 2023-10-10T20:19:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7fcc6e8da700 1 Processor -- start 2023-10-10T20:19:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0c3920 con 0x7f340c06d2e0 2023-10-10T20:19:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c0c3ee0 con 0x7f340c06d2e0 2023-10-10T20:19:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f340c04c3c0 con 0x7f340c06d2e0 2023-10-10T20:19:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f13011e4700 1 --2- 172.21.15.148:0/3729103875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc15e080 0x7f12fc0b6630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099e0 tx=0x7f12f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a554df9c38fd4ee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7359e700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b5c0053c0 0x7f5b5c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 -- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b5c0053c0 msgr2=0x7f5b5c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b5c0053c0 0x7f5b5c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 -- 172.21.15.148:0/992755811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5c003ae0 msgr2=0x7f5b5c003ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:33.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5c003ae0 0x7f5b5c003ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.725+0000 7f5b7259c700 1 -- 172.21.15.148:0/992755811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740b2600 con 0x7f5b5c004810 2023-10-10T20:19:33.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f5b7259c700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b5c004c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400aae0 tx=0x7f5b64008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64407231f514132 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.726+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400c600 con 0x7f5b5c004810 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400c760 con 0x7f5b5c004810 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f5b71d9b700 1 -- 172.21.15.148:0/992755811 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400cae0 con 0x7f5b5c004810 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f5b79120700 1 -- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 msgr2=0x7f5b5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f5b79120700 1 --2- 172.21.15.148:0/992755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b5c004c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400aae0 tx=0x7f5b64008040 comp rx=0 tx=0).stop 2023-10-10T20:19:33.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.727+0000 7f5b7359e700 1 -- 172.21.15.148:0/992755811 reap_dead start 2023-10-10T20:19:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f5b79120700 1 -- 172.21.15.148:0/992755811 shutdown_connections 2023-10-10T20:19:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f5b79120700 1 -- 172.21.15.148:0/992755811 >> 172.21.15.148:0/992755811 conn(0x7f5b740cd1e0 msgr2=0x7f5b740cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f5b79120700 1 -- 172.21.15.148:0/992755811 shutdown_connections 2023-10-10T20:19:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f5b79120700 1 -- 172.21.15.148:0/992755811 wait complete. 2023-10-10T20:19:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f12f3fff700 1 -- 172.21.15.148:0/3729103875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800b7e0 con 0x7f12fc15e080 2023-10-10T20:19:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f9a0baa1700 1 -- 172.21.15.148:0/406020306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04064650 msgr2=0x7f9a040c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/406020306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04064650 0x7f9a040c7670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a040b2e80 tx=0x7f99fc009840 comp rx=0 tx=0).stop 2023-10-10T20:19:33.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f12f3fff700 1 -- 172.21.15.148:0/3729103875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f800b940 con 0x7f12fc15e080 2023-10-10T20:19:33.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f12f3fff700 1 -- 172.21.15.148:0/3729103875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800bce0 con 0x7f12fc15e080 2023-10-10T20:19:33.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01b406eab0 0x7f01b40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3408013820 con 0x7f340c06d2e0 2023-10-10T20:19:33.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fcc6e8da700 1 -- start start 2023-10-10T20:19:33.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6816ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc68156ea0 0x7fcc6817a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc68157270 0x7fcc6816ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b0880 con 0x7fcc68157270 2023-10-10T20:19:33.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b0b60 con 0x7fcc6806e2b0 2023-10-10T20:19:33.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc6e8da700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6816b320 con 0x7fcc68156ea0 2023-10-10T20:19:33.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc67fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6816ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc67fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6816ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:41998/0 (socket says 172.21.15.148:41998) 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7fcc67fff700 1 -- 172.21.15.148:0/1492830719 learned_addr learned my addr 172.21.15.148:0/1492830719 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 -- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc68156ea0 msgr2=0x7fcc6817a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc677fe700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc68156ea0 0x7fcc6817a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc68156ea0 0x7fcc6817a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 -- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc68157270 msgr2=0x7fcc6816ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc68157270 0x7fcc6816ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6000bc30 con 0x7fcc6806e2b0 2023-10-10T20:19:33.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01b415a080 0x7f01b41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b416c480 con 0x7f01b40af6a0 2023-10-10T20:19:33.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b416b870 con 0x7f01b415a080 2023-10-10T20:19:33.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7f01bc40c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b416bb50 con 0x7f01b406eab0 2023-10-10T20:19:33.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba9a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01b415a080 0x7f01b41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01b406eab0 0x7f01b40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba9a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01b415a080 0x7f01b41590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:41988/0 (socket says 172.21.15.148:41988) 2023-10-10T20:19:33.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba9a9700 1 -- 172.21.15.148:0/642783124 learned_addr learned my addr 172.21.15.148:0/642783124 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01b99a7700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 -- 172.21.15.148:0/642783124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01b415a080 msgr2=0x7f01b41590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01b415a080 0x7f01b41590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 -- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 msgr2=0x7f01b4158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b40af6a0 0x7f01b4158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f01ba1a8700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac00ab30 con 0x7f01b406eab0 2023-10-10T20:19:33.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01ba1a8700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01b406eab0 0x7f01b40cb8a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b4e0 tx=0x7f01a4009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4011630 con 0x7f01b406eab0 2023-10-10T20:19:33.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4011790 con 0x7f01b406eab0 2023-10-10T20:19:33.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4011b50 con 0x7f01b406eab0 2023-10-10T20:19:33.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 -- 172.21.15.148:0/406020306 shutdown_connections 2023-10-10T20:19:33.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/406020306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04064650 0x7f9a040c7670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/406020306 >> v2:172.21.15.161:3300/0 conn(0x7f9a04153b50 0x7f9a04156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 -- 172.21.15.148:0/406020306 >> 172.21.15.148:0/406020306 conn(0x7f9a04067660 msgr2=0x7f9a04067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 -- 172.21.15.148:0/406020306 shutdown_connections 2023-10-10T20:19:33.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f9a0baa1700 1 -- 172.21.15.148:0/406020306 wait complete. 2023-10-10T20:19:33.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f1303448700 1 -- 172.21.15.148:0/3729103875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc15e080 msgr2=0x7f12fc0b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f1303448700 1 --2- 172.21.15.148:0/3729103875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc15e080 0x7f12fc0b6630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099e0 tx=0x7f12f8009250 comp rx=0 tx=0).stop 2023-10-10T20:19:33.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fd99eb85700 1 -- 172.21.15.148:0/1370607692 shutdown_connections 2023-10-10T20:19:33.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fd99eb85700 1 --2- 172.21.15.148:0/1370607692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9980cc450 0x7fd99806d090 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fd99eb85700 1 -- 172.21.15.148:0/1370607692 >> 172.21.15.148:0/1370607692 conn(0x7fd998070700 msgr2=0x7fd998070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fd99eb85700 1 -- 172.21.15.148:0/1370607692 shutdown_connections 2023-10-10T20:19:33.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.728+0000 7fd99eb85700 1 -- 172.21.15.148:0/1370607692 wait complete. 2023-10-10T20:19:33.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.729+0000 7f5b79120700 1 Processor -- start 2023-10-10T20:19:33.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f5b79120700 1 -- start start 2023-10-10T20:19:33.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b416be30 con 0x7f01b406eab0 2023-10-10T20:19:33.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40cbde0 con 0x7f01b406eab0 2023-10-10T20:19:33.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fd99eb85700 1 Processor -- start 2023-10-10T20:19:33.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc67fff700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6816ade0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c7f0 tx=0x7fcc5c007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f1303448700 1 -- 172.21.15.148:0/3729103875 shutdown_connections 2023-10-10T20:19:33.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f1303448700 1 --2- 172.21.15.148:0/3729103875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc15e080 0x7f12fc0b6630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f1303448700 1 -- 172.21.15.148:0/3729103875 >> 172.21.15.148:0/3729103875 conn(0x7f12fc068ce0 msgr2=0x7f12fc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b74151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 0x7f5b74161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b7415cd60 0x7f5b7415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7415d740 con 0x7f5b7415cd60 2023-10-10T20:19:33.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74099050 con 0x7f5b5c004810 2023-10-10T20:19:33.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b79120700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74161ec0 con 0x7f5b7415ee20 2023-10-10T20:19:33.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f9a0baa1700 1 Processor -- start 2023-10-10T20:19:33.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 0x7f5b74161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b72d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b74151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 0x7f5b74161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:50982/0 (socket says 172.21.15.148:50982) 2023-10-10T20:19:33.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b72d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b74151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42004/0 (socket says 172.21.15.148:42004) 2023-10-10T20:19:33.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 -- 172.21.15.148:0/3599271062 learned_addr learned my addr 172.21.15.148:0/3599271062 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7359e700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b7415cd60 0x7f5b7415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 -- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 msgr2=0x7f5b74151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b5c004810 0x7f5b74151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 -- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b7415cd60 msgr2=0x7f5b7415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b7415cd60 0x7f5b7415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f5b7259c700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b741621a0 con 0x7f5b7415ee20 2023-10-10T20:19:33.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01b404c3c0 con 0x7f01b406eab0 2023-10-10T20:19:33.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7fd99eb85700 1 -- start start 2023-10-10T20:19:33.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd99806d5d0 0x7fd9981675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9980c75c0 0x7fd9980c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9980cc450 0x7fd998156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998168a30 con 0x7fd9980cc450 2023-10-10T20:19:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998167e20 con 0x7fd9980c75c0 2023-10-10T20:19:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99eb85700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd998168100 con 0x7fd99806d5d0 2023-10-10T20:19:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd997fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9980c75c0 0x7fd9980c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd997fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9980c75c0 0x7fd9980c2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42012/0 (socket says 172.21.15.148:42012) 2023-10-10T20:19:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd997fff700 1 -- 172.21.15.148:0/3065347393 learned_addr learned my addr 172.21.15.148:0/3065347393 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f3401ffb700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33f0066f30 0x7f33f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3408084c50 con 0x7f340c06d2e0 2023-10-10T20:19:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f3401ffb700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7f33f006f140 0x7f33f0071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33f0071c00 con 0x7f33f006f140 2023-10-10T20:19:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3408084ff0 con 0x7f340c06d2e0 2023-10-10T20:19:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f34116ad700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33f0066f30 0x7f33f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f34126af700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7f33f006f140 0x7f33f0071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9980cc450 0x7fd998156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7f9a0baa1700 1 -- start start 2023-10-10T20:19:33.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a04064650 0x7f9a041561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 0x7f9a040c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a04153b50 0x7f9a0414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04154ca0 con 0x7f9a04069d00 2023-10-10T20:19:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04154f80 con 0x7f9a04153b50 2023-10-10T20:19:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0baa1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04155260 con 0x7f9a04064650 2023-10-10T20:19:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0903c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 0x7f9a040c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0903c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 0x7f9a040c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42332/0 (socket says 172.21.15.148:42332) 2023-10-10T20:19:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0903c700 1 -- 172.21.15.148:0/3338008802 learned_addr learned my addr 172.21.15.148:0/3338008802 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f34126af700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7f33f006f140 0x7f33f0071550 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0a03e700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a04153b50 0x7f9a0414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f9a0983d700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a04064650 0x7f9a041561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 -- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd99806d5d0 msgr2=0x7fd9981675f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd99806d5d0 0x7fd9981675f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 -- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9980c75c0 msgr2=0x7fd9980c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9980c75c0 0x7fd9980c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fd99d122700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0094f0 con 0x7fd9980cc450 2023-10-10T20:19:33.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7f34116ad700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33f0066f30 0x7f33f00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f34040076d0 tx=0x7f340401a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.730+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c005b30 con 0x7fcc6806e2b0 2023-10-10T20:19:33.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c005c90 con 0x7fcc6806e2b0 2023-10-10T20:19:33.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.731+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c007340 con 0x7fcc6806e2b0 2023-10-10T20:19:33.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6817aa50 con 0x7fcc6806e2b0 2023-10-10T20:19:33.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.732+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6817afd0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 -- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a04064650 msgr2=0x7f9a041561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a04064650 0x7f9a041561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 -- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a04153b50 msgr2=0x7f9a0414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a04153b50 0x7f9a0414ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc009ae0 con 0x7f9a04069d00 2023-10-10T20:19:33.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd99d122700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9980cc450 0x7fd998156f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c830 tx=0x7fd990001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc6804c3c0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0a03e700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a04153b50 0x7f9a0414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f5b7259c700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 0x7f5b74161570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c430 tx=0x7f5b64019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7f9a0903c700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 0x7f9a040c6590 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99f4007b10 tx=0x7f99f4006e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990012630 con 0x7fd9980cc450 2023-10-10T20:19:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990012790 con 0x7fd9980cc450 2023-10-10T20:19:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981683e0 con 0x7fd9980cc450 2023-10-10T20:19:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990012b30 con 0x7fd9980cc450 2023-10-10T20:19:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.733+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd998167b30 con 0x7fd9980cc450 2023-10-10T20:19:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== osd.0 v2:172.21.15.148:6800/86309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f33f0071c00 con 0x7f33f006f140 2023-10-10T20:19:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd99804c3c0 con 0x7fd9980cc450 2023-10-10T20:19:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01a401a400 con 0x7f01b406eab0 2023-10-10T20:19:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f01aa7fc700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0198066fb0 0x7f0198069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01a40844d0 con 0x7f01b406eab0 2023-10-10T20:19:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd99000a3b0 con 0x7fd9980cc450 2023-10-10T20:19:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f01b99a7700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0198066fb0 0x7f0198069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f01aa7fc700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f019806f140 0x7f0198071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0198071c00 con 0x7f019806f140 2023-10-10T20:19:33.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc5c01b400 con 0x7fcc6806e2b0 2023-10-10T20:19:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc54067000 0x7fcc540694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc5c083f70 con 0x7fcc6806e2b0 2023-10-10T20:19:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc677fe700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc54067000 0x7fcc540694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7fcc5406f210 0x7fcc54071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc54071cd0 con 0x7fcc5406f210 2023-10-10T20:19:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5c0865a0 con 0x7fcc6806e2b0 2023-10-10T20:19:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f01ba9a9700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f019806f140 0x7f0198071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a40516a0 con 0x7f01b406eab0 2023-10-10T20:19:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fcc677fe700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc54067000 0x7fcc540694b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc68153250 tx=0x7fcc58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd994ff9700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd97c067000 0x7fd97c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9900847e0 con 0x7fd9980cc450 2023-10-10T20:19:33.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd994ff9700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7fd97c06f190 0x7fd97c0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd97c071c50 con 0x7fd97c06f190 2023-10-10T20:19:33.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd990051ad0 con 0x7fd9980cc450 2023-10-10T20:19:33.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7fd99c921700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd97c067000 0x7fd97c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401f070 con 0x7f5b7415ee20 2023-10-10T20:19:33.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6401aba0 con 0x7f5b7415ee20 2023-10-10T20:19:33.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64023410 con 0x7f5b7415ee20 2023-10-10T20:19:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b74162480 con 0x7f5b7415ee20 2023-10-10T20:19:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b741519c0 con 0x7f5b7415ee20 2023-10-10T20:19:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7fd997fff700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7fd97c06f190 0x7fd97c0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7fd99c921700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd97c067000 0x7fd97c0694b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099e0 tx=0x7fd98c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7f1303448700 1 -- 172.21.15.148:0/3729103875 shutdown_connections 2023-10-10T20:19:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7fd997fff700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7fd97c06f190 0x7fd97c0715a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7fcc6ce77700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7fcc5406f210 0x7fcc54071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7f01ba9a9700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f019806f140 0x7f0198071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7fcc6ce77700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7fcc5406f210 0x7fcc54071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7f01b99a7700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0198066fb0 0x7f0198069460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01b0006fd0 tx=0x7f01b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== osd.5 v2:172.21.15.161:6801/86555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fd97c071c50 con 0x7fd97c06f190 2023-10-10T20:19:33.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b7404c3c0 con 0x7f5b7415ee20 2023-10-10T20:19:33.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b640113f0 con 0x7f5b7415ee20 2023-10-10T20:19:33.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7f5b5affd700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5b60066fb0 0x7f5b60069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b640854d0 con 0x7f5b7415ee20 2023-10-10T20:19:33.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.737+0000 7f5b72d9d700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5b60066fb0 0x7f5b60069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f5b5affd700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7f5b6006f1c0 0x7f5b600715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b60071c80 con 0x7f5b6006f1c0 2023-10-10T20:19:33.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b64087a70 con 0x7f5b7415ee20 2023-10-10T20:19:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== osd.2 v2:172.21.15.148:6808/86312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f0198071c00 con 0x7f019806f140 2023-10-10T20:19:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f5b7359e700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7f5b6006f1c0 0x7f5b600715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== osd.1 v2:172.21.15.148:6816/86315 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fcc54071cd0 con 0x7fcc5406f210 2023-10-10T20:19:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.738+0000 7f5b72d9d700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5b60066fb0 0x7f5b60069460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b74069480 tx=0x7f5b6c009780 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f5b7359e700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7f5b6006f1c0 0x7f5b600715d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== osd.6 v2:172.21.15.161:6805/86553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f5b60071c80 con 0x7f5b6006f1c0 2023-10-10T20:19:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.734+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4012600 con 0x7f9a04069d00 2023-10-10T20:19:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f4012760 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4012ae0 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a04155540 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.735+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a041566e0 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.736+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a040c3d70 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99f401b400 con 0x7f9a04069d00 2023-10-10T20:19:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f99e8067080 0x7f99e8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99f4084c70 con 0x7f9a04069d00 2023-10-10T20:19:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f9a0983d700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f99e8067080 0x7f99e8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f99e806f290 0x7f99e80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99e8071d50 con 0x7f99e806f290 2023-10-10T20:19:33.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f4084fa0 con 0x7f9a04069d00 2023-10-10T20:19:33.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f9a0a03e700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f99e806f290 0x7f99e80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.739+0000 7f9a0983d700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f99e8067080 0x7f99e8069530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009880 tx=0x7f99fc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.740+0000 7f9a0a03e700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f99e806f290 0x7f99e80716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.740+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== osd.3 v2:172.21.15.148:6824/86339 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f99e8071d50 con 0x7f99e806f290 2023-10-10T20:19:33.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.743+0000 7f1303448700 1 -- 172.21.15.148:0/3729103875 wait complete. 2023-10-10T20:19:33.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.744+0000 7f1303448700 1 Processor -- start 2023-10-10T20:19:33.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.745+0000 7f1303448700 1 -- start start 2023-10-10T20:19:33.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc0afd90 0x7f12fc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc15e080 0x7f12fc15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 0x7f12fc158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc160310 con 0x7f12fc0afd90 2023-10-10T20:19:33.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1588c0 con 0x7f12fc15e080 2023-10-10T20:19:33.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f1303448700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1572e0 con 0x7f12fc15e450 2023-10-10T20:19:33.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f13019e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 0x7f12fc158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f13019e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 0x7f12fc158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51016/0 (socket says 172.21.15.148:51016) 2023-10-10T20:19:33.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f13019e5700 1 -- 172.21.15.148:0/1886428588 learned_addr learned my addr 172.21.15.148:0/1886428588 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.746+0000 7f13009e3700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc15e080 0x7f12fc15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 -- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc15e080 msgr2=0x7f12fc15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc15e080 0x7f12fc15fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 -- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc0afd90 msgr2=0x7f12fc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12fc0afd90 0x7f12fc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 -- 172.21.15.148:0/1886428588 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80094f0 con 0x7f12fc15e450 2023-10-10T20:19:33.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f13019e5700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 0x7f12fc158380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f12f400c930 tx=0x7f12f40026e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4012630 con 0x7f12fc15e450 2023-10-10T20:19:33.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f40127d0 con 0x7f12fc15e450 2023-10-10T20:19:33.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.747+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4012b90 con 0x7f12fc15e450 2023-10-10T20:19:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.754+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc1575c0 con 0x7f12fc15e450 2023-10-10T20:19:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.754+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc157b80 con 0x7f12fc15e450 2023-10-10T20:19:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.755+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 --> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f01b40ae400 con 0x7f019806f140 2023-10-10T20:19:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.755+0000 7f8a6b55f700 1 Processor -- start 2023-10-10T20:19:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.755+0000 7f8a6b55f700 1 -- start start 2023-10-10T20:19:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a6b55f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a6415a330 0x7f8a64156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a6b55f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 0x7f8a64156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a6b55f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a64157c30 con 0x7f8a64156570 2023-10-10T20:19:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a6b55f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a641584f0 con 0x7f8a6415a330 2023-10-10T20:19:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a6b55f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a641573c0 con 0x7f8a6406da90 2023-10-10T20:19:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a692fb700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8a64156570 0x7f8a6406d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48048/0 (socket says 172.21.15.148:48048) 2023-10-10T20:19:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a692fb700 1 -- 172.21.15.148:0/2028952963 learned_addr learned my addr 172.21.15.148:0/2028952963 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a68afa700 1 --2- 172.21.15.148:0/2028952963 >> v2:172.21.15.148:3300/0 conn(0x7f8a6415a330 0x7f8a64156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a69afc700 1 --2- 172.21.15.148:0/2028952963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 0x7f8a64156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f8a5bfff700 1 -- 172.21.15.148:0/2028952963 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759206137 0 0) 0x7f8a64157c30 con 0x7f8a64156570 2023-10-10T20:19:33.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a5bfff700 1 -- 172.21.15.148:0/2028952963 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a50003570 con 0x7f8a64156570 2023-10-10T20:19:33.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a69afc700 1 -- 172.21.15.148:0/2028952963 >> v2:172.21.15.148:3300/0 conn(0x7f8a6415a330 msgr2=0x7f8a64156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:33.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a69afc700 1 --2- 172.21.15.148:0/2028952963 >> v2:172.21.15.148:3300/0 conn(0x7f8a6415a330 0x7f8a64156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a69afc700 1 -- 172.21.15.148:0/2028952963 >> v1:172.21.15.161:6789/0 conn(0x7f8a64156570 legacy=0x7f8a6406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a69afc700 1 -- 172.21.15.148:0/2028952963 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a64157670 con 0x7f8a6406da90 2023-10-10T20:19:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.756+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12f400a3b0 con 0x7f12fc15e450 2023-10-10T20:19:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f12f17fa700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f12e0067080 0x7f12e0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12f4084280 con 0x7f12fc15e450 2023-10-10T20:19:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a69afc700 1 --2- 172.21.15.148:0/2028952963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 0x7f8a64156e80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00af70 tx=0x7f8a5c00a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d651c76cccb3d8d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a5bfff700 1 -- 172.21.15.148:0/2028952963 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00f070 con 0x7f8a6406da90 2023-10-10T20:19:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a5bfff700 1 -- 172.21.15.148:0/2028952963 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00a2e0 con 0x7f8a6406da90 2023-10-10T20:19:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.757+0000 7f8a5bfff700 1 -- 172.21.15.148:0/2028952963 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00cb80 con 0x7f8a6406da90 2023-10-10T20:19:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 -- 172.21.15.148:0/2028952963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 msgr2=0x7f8a64156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/2028952963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 0x7f8a64156e80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00af70 tx=0x7f8a5c00a890 comp rx=0 tx=0).stop 2023-10-10T20:19:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 -- 172.21.15.148:0/2028952963 shutdown_connections 2023-10-10T20:19:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/2028952963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6406da90 0x7f8a64156e80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/2028952963 >> v2:172.21.15.148:3300/0 conn(0x7f8a6415a330 0x7f8a64156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 -- 172.21.15.148:0/2028952963 >> 172.21.15.148:0/2028952963 conn(0x7f8a64067660 msgr2=0x7f8a64067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f1303448700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f12fc157d70 0x7f12fc16edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 --> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12fc0008d0 con 0x7f12fc157d70 2023-10-10T20:19:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 -- 172.21.15.148:0/2028952963 shutdown_connections 2023-10-10T20:19:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f8a6b55f700 1 -- 172.21.15.148:0/2028952963 wait complete. 2023-10-10T20:19:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.758+0000 7f13009e3700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f12fc157d70 0x7f12fc16edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.759+0000 7f13009e3700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f12fc157d70 0x7f12fc16edd0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.759+0000 7f8a6b55f700 1 Processor -- start 2023-10-10T20:19:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.759+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== osd.4 v2:172.21.15.161:6802/86554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f12fc0008d0 con 0x7f12fc157d70 2023-10-10T20:19:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.760+0000 7f8a6b55f700 1 -- start start 2023-10-10T20:19:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a6406da90 0x7f8a640bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 0x7f8a640afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a330 0x7f8a640b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640b0690 con 0x7f8a6406da90 2023-10-10T20:19:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c7800 con 0x7f8a64156570 2023-10-10T20:19:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a6b55f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640b03f0 con 0x7f8a6415a330 2023-10-10T20:19:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a68afa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 0x7f8a640afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a68afa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 0x7f8a640afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42040/0 (socket says 172.21.15.148:42040) 2023-10-10T20:19:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a68afa700 1 -- 172.21.15.148:0/1589442357 learned_addr learned my addr 172.21.15.148:0/1589442357 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.761+0000 7f8a69afc700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a330 0x7f8a640b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 -- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a330 msgr2=0x7f8a640b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a330 0x7f8a640b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 -- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a6406da90 msgr2=0x7f8a640bfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a6406da90 0x7f8a640bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 -- 172.21.15.148:0/1589442357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c00ab30 con 0x7f8a64156570 2023-10-10T20:19:33.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a68afa700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 0x7f8a640afeb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400ec50 tx=0x7f8a540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400c850 con 0x7f8a64156570 2023-10-10T20:19:33.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400c9b0 con 0x7f8a64156570 2023-10-10T20:19:33.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640c6ed0 con 0x7f8a64156570 2023-10-10T20:19:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640c7490 con 0x7f8a64156570 2023-10-10T20:19:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 --> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f340c0c7e30 con 0x7f33f006f140 2023-10-10T20:19:33.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 --> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9980c7e30 con 0x7fd97c06f190 2023-10-10T20:19:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.762+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400cd70 con 0x7f8a64156570 2023-10-10T20:19:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3401ffb700 1 -- 172.21.15.148:0/4292731124 <== osd.0 v2:172.21.15.148:6800/86309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f340c0c7e30 con 0x7f33f006f140 2023-10-10T20:19:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f01aa7fc700 1 -- 172.21.15.148:0/642783124 <== osd.2 v2:172.21.15.148:6808/86312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f01b40ae400 con 0x7f019806f140 2023-10-10T20:19:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f13011e4700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f12e0067080 0x7f12e0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7f33f006f140 msgr2=0x7f33f0071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7f33f006f140 0x7f33f0071550 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33f0066f30 msgr2=0x7f33f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33f0066f30 0x7f33f00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f34040076d0 tx=0x7f340401a000 comp rx=0 tx=0).stop 2023-10-10T20:19:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 msgr2=0x7f340c16f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f3414112700 1 --2- 172.21.15.148:0/4292731124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f340c06d2e0 0x7f340c16f1d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f340800ec80 tx=0x7f34080091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.763+0000 7f34126af700 1 -- 172.21.15.148:0/4292731124 reap_dead start 2023-10-10T20:19:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 shutdown_connections 2023-10-10T20:19:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 >> 172.21.15.148:0/4292731124 conn(0x7f340c070700 msgr2=0x7f340c158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f13011e4700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f12e0067080 0x7f12e0069530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099e0 tx=0x7f12f8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 shutdown_connections 2023-10-10T20:19:33.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f3414112700 1 -- 172.21.15.148:0/4292731124 wait complete. 2023-10-10T20:19:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.764+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a54013680 con 0x7f8a64156570 2023-10-10T20:19:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.765+0000 7f8a597fa700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a48066fb0 0x7f8a48069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.765+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a54084830 con 0x7f8a64156570 2023-10-10T20:19:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.765+0000 7f8a692fb700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a48066fb0 0x7f8a48069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7f8a440015f0 0x7f8a44003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 --> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a44006b90 con 0x7f8a440015f0 2023-10-10T20:19:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7f8a692fb700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a48066fb0 0x7f8a48069460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a600058d0 tx=0x7f8a600079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7f8a69afc700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7f8a440015f0 0x7f8a44003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7f8a69afc700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7f8a440015f0 0x7f8a44003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.766+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 --> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc68067660 con 0x7fcc5406f210 2023-10-10T20:19:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fd994ff9700 1 -- 172.21.15.148:0/3065347393 <== osd.5 v2:172.21.15.161:6801/86555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd9980c7e30 con 0x7fd97c06f190 2023-10-10T20:19:33.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fcc4ffff700 1 -- 172.21.15.148:0/1492830719 <== osd.1 v2:172.21.15.148:6816/86315 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc68067660 con 0x7fcc5406f210 2023-10-10T20:19:33.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7fd97c06f190 msgr2=0x7fd97c0715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fd99eb85700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7fd97c06f190 0x7fd97c0715a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7fcc5406f210 msgr2=0x7fcc54071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7fcc5406f210 0x7fcc54071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f019806f140 msgr2=0x7f0198071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.767+0000 7f01bc40c700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f019806f140 0x7f0198071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd97c067000 msgr2=0x7fd97c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7fd99eb85700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd97c067000 0x7fd97c0694b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099e0 tx=0x7fd98c016000 comp rx=0 tx=0).stop 2023-10-10T20:19:33.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9980cc450 msgr2=0x7fd998156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7fd99eb85700 1 --2- 172.21.15.148:0/3065347393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9980cc450 0x7fd998156f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c830 tx=0x7fd990001690 comp rx=0 tx=0).stop 2023-10-10T20:19:33.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7fd99d122700 1 -- 172.21.15.148:0/3065347393 reap_dead start 2023-10-10T20:19:33.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0198066fb0 msgr2=0x7f0198069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0198066fb0 0x7f0198069460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01b0006fd0 tx=0x7f01b0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:33.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01b406eab0 msgr2=0x7f01b40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 --2- 172.21.15.148:0/642783124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01b406eab0 0x7f01b40cb8a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b4e0 tx=0x7f01a4009060 comp rx=0 tx=0).stop 2023-10-10T20:19:33.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01ba9a9700 1 -- 172.21.15.148:0/642783124 reap_dead start 2023-10-10T20:19:33.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 shutdown_connections 2023-10-10T20:19:33.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 >> 172.21.15.148:0/642783124 conn(0x7f01b40cd1e0 msgr2=0x7f01b40b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 --> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a0404c3c0 con 0x7f99e806f290 2023-10-10T20:19:33.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 shutdown_connections 2023-10-10T20:19:33.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f01bc40c700 1 -- 172.21.15.148:0/642783124 wait complete. 2023-10-10T20:19:33.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.768+0000 7f99f9ffb700 1 -- 172.21.15.148:0/3338008802 <== osd.3 v2:172.21.15.148:6824/86339 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a0404c3c0 con 0x7f99e806f290 2023-10-10T20:19:33.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f99e806f290 msgr2=0x7f99e80716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f99e806f290 0x7f99e80716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 --> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b74161ec0 con 0x7f5b6006f1c0 2023-10-10T20:19:33.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f99e8067080 msgr2=0x7f99e8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f99e8067080 0x7f99e8069530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009880 tx=0x7f99fc017000 comp rx=0 tx=0).stop 2023-10-10T20:19:33.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 msgr2=0x7f9a040c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 --2- 172.21.15.148:0/3338008802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a04069d00 0x7f9a040c6590 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99f4007b10 tx=0x7f99f4006e30 comp rx=0 tx=0).stop 2023-10-10T20:19:33.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0a03e700 1 -- 172.21.15.148:0/3338008802 reap_dead start 2023-10-10T20:19:33.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 shutdown_connections 2023-10-10T20:19:33.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 >> 172.21.15.148:0/3338008802 conn(0x7f9a04067660 msgr2=0x7f9a040b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 shutdown_connections 2023-10-10T20:19:33.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.769+0000 7f9a0baa1700 1 -- 172.21.15.148:0/3338008802 wait complete. 2023-10-10T20:19:33.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== osd.7 v2:172.21.15.161:6800/86574 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f8a44006b90 con 0x7f8a440015f0 2023-10-10T20:19:33.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b5affd700 1 -- 172.21.15.148:0/3599271062 <== osd.6 v2:172.21.15.161:6805/86553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b74161ec0 con 0x7f5b6006f1c0 2023-10-10T20:19:33.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7f5b6006f1c0 msgr2=0x7f5b600715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7f5b6006f1c0 0x7f5b600715d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5b60066fb0 msgr2=0x7f5b60069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5b60066fb0 0x7f5b60069460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b74069480 tx=0x7f5b6c009780 comp rx=0 tx=0).stop 2023-10-10T20:19:33.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 msgr2=0x7f5b74161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 --2- 172.21.15.148:0/3599271062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b7415ee20 0x7f5b74161570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c430 tx=0x7f5b64019000 comp rx=0 tx=0).stop 2023-10-10T20:19:33.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b7359e700 1 -- 172.21.15.148:0/3599271062 reap_dead start 2023-10-10T20:19:33.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 shutdown_connections 2023-10-10T20:19:33.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 >> 172.21.15.148:0/3599271062 conn(0x7f5b740cd1e0 msgr2=0x7f5b741596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 shutdown_connections 2023-10-10T20:19:33.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.770+0000 7f5b79120700 1 -- 172.21.15.148:0/3599271062 wait complete. 2023-10-10T20:19:33.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.774+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc54067000 msgr2=0x7fcc540694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.774+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc54067000 0x7fcc540694b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc68153250 tx=0x7fcc58008040 comp rx=0 tx=0).stop 2023-10-10T20:19:33.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.774+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 msgr2=0x7fcc6816ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.774+0000 7fcc6e8da700 1 --2- 172.21.15.148:0/1492830719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc6806e2b0 0x7fcc6816ade0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c7f0 tx=0x7fcc5c007e00 comp rx=0 tx=0).stop 2023-10-10T20:19:33.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.776+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 shutdown_connections 2023-10-10T20:19:33.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.776+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 >> 172.21.15.148:0/3065347393 conn(0x7fd998070700 msgr2=0x7fd9980b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.776+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 shutdown_connections 2023-10-10T20:19:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.776+0000 7fd99eb85700 1 -- 172.21.15.148:0/3065347393 wait complete. 2023-10-10T20:19:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.782+0000 7fcc6ce77700 1 -- 172.21.15.148:0/1492830719 reap_dead start 2023-10-10T20:19:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.782+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 shutdown_connections 2023-10-10T20:19:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.782+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 >> 172.21.15.148:0/1492830719 conn(0x7fcc680ca4f0 msgr2=0x7fcc68070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.782+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 --> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12fc0b3d60 con 0x7f12fc157d70 2023-10-10T20:19:33.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.782+0000 7f12f17fa700 1 -- 172.21.15.148:0/1886428588 <== osd.4 v2:172.21.15.161:6802/86554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f12fc0b3d60 con 0x7f12fc157d70 2023-10-10T20:19:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.784+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 shutdown_connections 2023-10-10T20:19:33.887 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-10T20:19:33.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f12fc157d70 msgr2=0x7f12fc16edd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f12fc157d70 0x7f12fc16edd0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f12e0067080 msgr2=0x7f12e0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f12e0067080 0x7f12e0069530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099e0 tx=0x7f12f8016000 comp rx=0 tx=0).stop 2023-10-10T20:19:33.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 msgr2=0x7f12fc158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 --2- 172.21.15.148:0/1886428588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc15e450 0x7f12fc158380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f12f400c930 tx=0x7f12f40026e0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f13019e5700 1 -- 172.21.15.148:0/1886428588 reap_dead start 2023-10-10T20:19:33.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 shutdown_connections 2023-10-10T20:19:33.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.786+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 >> 172.21.15.148:0/1886428588 conn(0x7f12fc068ce0 msgr2=0x7f12fc0b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.787+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 shutdown_connections 2023-10-10T20:19:33.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.787+0000 7f1303448700 1 -- 172.21.15.148:0/1886428588 wait complete. 2023-10-10T20:19:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.789+0000 7fcc6e8da700 1 -- 172.21.15.148:0/1492830719 wait complete. 2023-10-10T20:19:33.892 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-10T20:19:33.894 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-10T20:19:33.895 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-10T20:19:33.896 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.794+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 --> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a44005c20 con 0x7f8a440015f0 2023-10-10T20:19:33.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-10T20:19:33.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.795+0000 7f8a597fa700 1 -- 172.21.15.148:0/1589442357 <== osd.7 v2:172.21.15.161:6800/86574 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a44005c20 con 0x7f8a440015f0 2023-10-10T20:19:33.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.795+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7f8a440015f0 msgr2=0x7f8a44003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.795+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7f8a440015f0 0x7f8a44003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a48066fb0 msgr2=0x7f8a48069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a48066fb0 0x7f8a48069460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a600058d0 tx=0x7f8a600079d0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 msgr2=0x7f8a640afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 --2- 172.21.15.148:0/1589442357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a64156570 0x7f8a640afeb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400ec50 tx=0x7f8a540091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a69afc700 1 -- 172.21.15.148:0/1589442357 reap_dead start 2023-10-10T20:19:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 shutdown_connections 2023-10-10T20:19:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 >> 172.21.15.148:0/1589442357 conn(0x7f8a64067660 msgr2=0x7f8a64159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 shutdown_connections 2023-10-10T20:19:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:33.796+0000 7f8a6b55f700 1 -- 172.21.15.148:0/1589442357 wait complete. 2023-10-10T20:19:33.902 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-10T20:19:33.903 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-10T20:19:33.904 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:33.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-10T20:19:34.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.126+0000 7f8341965700 1 Processor -- start 2023-10-10T20:19:34.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.126+0000 7f8341965700 1 -- start start 2023-10-10T20:19:34.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f833c159dd0 0x7f833c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c15db70 0x7f833c0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f833c0cbfd0 0x7f833c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0cc590 con 0x7f833c159dd0 2023-10-10T20:19:34.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0cc870 con 0x7f833c0cbfd0 2023-10-10T20:19:34.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f8341965700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0ccb50 con 0x7f833c15db70 2023-10-10T20:19:34.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.127+0000 7fa79abd4700 1 Processor -- start 2023-10-10T20:19:34.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7fa79abd4700 1 -- start start 2023-10-10T20:19:34.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f536dce6700 1 Processor -- start 2023-10-10T20:19:34.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f536dce6700 1 -- start start 2023-10-10T20:19:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f833b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f833c0cbfd0 0x7f833c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f833b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f833c0cbfd0 0x7f833c0c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42048/0 (socket says 172.21.15.148:42048) 2023-10-10T20:19:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f833b7fe700 1 -- 172.21.15.148:0/636341096 learned_addr learned my addr 172.21.15.148:0/636341096 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f5a2cb85700 1 Processor -- start 2023-10-10T20:19:34.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 -- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c15db70 msgr2=0x7f833c0c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 --2- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c15db70 0x7f833c0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 -- 172.21.15.148:0/636341096 >> v2:172.21.15.148:3300/0 conn(0x7f833c159dd0 msgr2=0x7f833c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 --2- 172.21.15.148:0/636341096 >> v2:172.21.15.148:3300/0 conn(0x7f833c159dd0 0x7f833c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 -- 172.21.15.148:0/636341096 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0cce30 con 0x7f833c0cbfd0 2023-10-10T20:19:34.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa79abd4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa794088b90 0x7fa7940c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa79abd4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b16e0 con 0x7fa7940cc8a0 2023-10-10T20:19:34.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa79abd4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b0ad0 con 0x7fa7940887c0 2023-10-10T20:19:34.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa79abd4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940b0db0 con 0x7fa794088b90 2023-10-10T20:19:34.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f833b7fe700 1 --2- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c0cbfd0 0x7f833c0c9000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f833c0cdfb0 tx=0x7f8334009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3785e6cf91fd7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.128+0000 7f5a2cb85700 1 -- start start 2023-10-10T20:19:34.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f8339ffb700 1 -- 172.21.15.148:0/636341096 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400e070 con 0x7f833c0cbfd0 2023-10-10T20:19:34.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f8339ffb700 1 -- 172.21.15.148:0/636341096 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400ba50 con 0x7f833c0cbfd0 2023-10-10T20:19:34.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f8339ffb700 1 -- 172.21.15.148:0/636341096 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340092e0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa793fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa794088b90 0x7fa7940c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa793fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa794088b90 0x7fa7940c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42056/0 (socket says 172.21.15.148:42056) 2023-10-10T20:19:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa793fff700 1 -- 172.21.15.148:0/1671068635 learned_addr learned my addr 172.21.15.148:0/1671068635 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fa799171700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa7940cc8a0 0x7fa7940c94f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58266/0 (socket says 172.21.15.148:58266) 2023-10-10T20:19:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f3b86aaf700 1 Processor -- start 2023-10-10T20:19:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f536dce6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f536815e080 0x7f53680b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f536dce6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53680b6cc0 con 0x7f536815e450 2023-10-10T20:19:34.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f536dce6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53680b56d0 con 0x7f53680afd90 2023-10-10T20:19:34.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7f536dce6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53680b0230 con 0x7f536815e080 2023-10-10T20:19:34.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5366ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f536815e450 0x7f53680b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58282/0 (socket says 172.21.15.148:58282) 2023-10-10T20:19:34.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5366ffd700 1 -- 172.21.15.148:0/480322575 learned_addr learned my addr 172.21.15.148:0/480322575 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:34.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a2cb85700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a28070e10 0x7f5a280b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a2cb85700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a28069010 con 0x7f5a280688a0 2023-10-10T20:19:34.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a2cb85700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a280ca0f0 con 0x7f5a280c0e10 2023-10-10T20:19:34.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a2cb85700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2806fa20 con 0x7f5a28070e10 2023-10-10T20:19:34.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a26d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a28070e10 0x7f5a280b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a26d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a28070e10 0x7f5a280b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42060/0 (socket says 172.21.15.148:42060) 2023-10-10T20:19:34.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a26d9d700 1 -- 172.21.15.148:0/3401850133 learned_addr learned my addr 172.21.15.148:0/3401850133 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f5a2659c700 1 --1- 172.21.15.148:0/3401850133 >> v1:172.21.15.161:6789/0 conn(0x7f5a280c0e10 0x7f5a28070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48070/0 (socket says 172.21.15.148:48070) 2023-10-10T20:19:34.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f8341965700 1 -- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c0cbfd0 msgr2=0x7f833c0c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f8341965700 1 --2- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c0cbfd0 0x7f833c0c9000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f833c0cdfb0 tx=0x7f8334009840 comp rx=0 tx=0).stop 2023-10-10T20:19:34.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fbb203b4700 1 Processor -- start 2023-10-10T20:19:34.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.129+0000 7fbb203b4700 1 -- start start 2023-10-10T20:19:34.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f8341965700 1 -- 172.21.15.148:0/636341096 shutdown_connections 2023-10-10T20:19:34.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f8341965700 1 --2- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c0cbfd0 0x7f833c0c9000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f8341965700 1 --2- 172.21.15.148:0/636341096 >> v2:172.21.15.148:3300/0 conn(0x7f833c159dd0 0x7f833c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f8341965700 1 --2- 172.21.15.148:0/636341096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c15db70 0x7f833c0c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f8341965700 1 -- 172.21.15.148:0/636341096 >> 172.21.15.148:0/636341096 conn(0x7f833c0cd1e0 msgr2=0x7f833c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690891421 0 0) 0x7f53680b56d0 con 0x7f53680afd90 2023-10-10T20:19:34.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5350003570 con 0x7f53680afd90 2023-10-10T20:19:34.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045938579 0 0) 0x7fa7940b0ad0 con 0x7fa7940887c0 2023-10-10T20:19:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa784003570 con 0x7fa7940887c0 2023-10-10T20:19:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.130+0000 7f3b86aaf700 1 -- start start 2023-10-10T20:19:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5a26d9d700 1 -- 172.21.15.148:0/3401850133 >> v1:172.21.15.161:6789/0 conn(0x7f5a280c0e10 legacy=0x7f5a28070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5a26d9d700 1 -- 172.21.15.148:0/3401850133 >> v1:172.21.15.148:6789/0 conn(0x7f5a280688a0 legacy=0x7f5a280ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5a26d9d700 1 -- 172.21.15.148:0/3401850133 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280c9800 con 0x7f5a28070e10 2023-10-10T20:19:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53677fe700 1 --2- 172.21.15.148:0/480322575 >> v2:172.21.15.148:3300/0 conn(0x7f536815e080 0x7f53680b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb203b4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb180c7060 0x7fbb180c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb203b4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180c8930 con 0x7fbb1806b0d0 2023-10-10T20:19:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb203b4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180b2860 con 0x7fbb1806d7a0 2023-10-10T20:19:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb203b4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18156730 con 0x7fbb180c7060 2023-10-10T20:19:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b86aaf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b800bf970 0x7f3b800bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b86aaf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b8006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b86aaf700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b800c7960 con 0x7f3b8006f1a0 2023-10-10T20:19:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b86aaf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b800c7c40 con 0x7f3b800bf970 2023-10-10T20:19:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b86aaf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c7f20 con 0x7f3b8006d4c0 2023-10-10T20:19:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716689717 0 0) 0x7fa7940b16e0 con 0x7fa7940cc8a0 2023-10-10T20:19:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7940b0ad0 con 0x7fa7940cc8a0 2023-10-10T20:19:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209200013 0 0) 0x7f5350003570 con 0x7f53680afd90 2023-10-10T20:19:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 >> v2:172.21.15.148:3300/0 conn(0x7f536815e080 msgr2=0x7f53680b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 --2- 172.21.15.148:0/480322575 >> v2:172.21.15.148:3300/0 conn(0x7f536815e080 0x7f53680b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 >> v1:172.21.15.148:6789/0 conn(0x7f536815e450 legacy=0x7f53680b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680b4da0 con 0x7f53680afd90 2023-10-10T20:19:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f53600017f0 con 0x7f53680afd90 2023-10-10T20:19:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 >> v1:172.21.15.161:6789/0 conn(0x7f53680afd90 legacy=0x7f536806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5350003a60 0x7f5350003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 0x7f5350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53500053c0 0x7f535000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53680b0230 con 0x7f5350004810 2023-10-10T20:19:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535000c2f0 con 0x7f5350003a60 2023-10-10T20:19:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535000c4b0 con 0x7f53500053c0 2023-10-10T20:19:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53677fe700 1 --2- 172.21.15.148:0/480322575 >> v2:172.21.15.148:3300/0 conn(0x7f536815e080 0x7f53680b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f8341965700 1 -- 172.21.15.148:0/636341096 shutdown_connections 2023-10-10T20:19:34.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b8484b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b800bf970 0x7f3b800bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b8484b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b800bf970 0x7f3b800bfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42064/0 (socket says 172.21.15.148:42064) 2023-10-10T20:19:34.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f3b8484b700 1 -- 172.21.15.148:0/2506369469 learned_addr learned my addr 172.21.15.148:0/2506369469 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5366ffd700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 0x7f5350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f53677fe700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5350003a60 0x7f5350003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3592462344 0 0) 0x7fa784003570 con 0x7fa7940887c0 2023-10-10T20:19:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 >> v2:172.21.15.161:3300/0 conn(0x7fa794088b90 msgr2=0x7fa7940c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 --2- 172.21.15.148:0/1671068635 >> v2:172.21.15.161:3300/0 conn(0x7fa794088b90 0x7fa7940c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 >> v1:172.21.15.148:6789/0 conn(0x7fa7940cc8a0 legacy=0x7fa7940c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940b1090 con 0x7fa7940887c0 2023-10-10T20:19:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa780001470 con 0x7fa7940887c0 2023-10-10T20:19:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 >> v1:172.21.15.161:6789/0 conn(0x7fa7940887c0 legacy=0x7fa7940b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa784003c40 0x7fa784004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa784004810 0x7fa784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7840052e0 0x7fa78400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940b0db0 con 0x7fa784004810 2023-10-10T20:19:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78400c1b0 con 0x7fa7840052e0 2023-10-10T20:19:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78400c370 con 0x7fa784003c40 2023-10-10T20:19:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa784003c40 0x7fa784004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa799171700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7840052e0 0x7fa78400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5a26d9d700 1 --2- 172.21.15.148:0/3401850133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a28070e10 0x7f5a280b0900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ea0 tx=0x7f5a20009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e598eff9a11bd61 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5a2559a700 1 -- 172.21.15.148:0/3401850133 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000e070 con 0x7f5a28070e10 2023-10-10T20:19:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5a2559a700 1 -- 172.21.15.148:0/3401850133 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2000b980 con 0x7f5a28070e10 2023-10-10T20:19:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5a2559a700 1 -- 172.21.15.148:0/3401850133 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200092e0 con 0x7f5a28070e10 2023-10-10T20:19:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7f5367fff700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53500053c0 0x7f535000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 -- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53500053c0 msgr2=0x7f535000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53500053c0 0x7f535000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 -- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5350003a60 msgr2=0x7f5350003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5350003a60 0x7f5350003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 -- 172.21.15.148:0/480322575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680b4da0 con 0x7f5350004810 2023-10-10T20:19:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5367fff700 1 -- 172.21.15.148:0/480322575 reap_dead start 2023-10-10T20:19:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb1d94f700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbb1806d7a0 0x7fbb180b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48094/0 (socket says 172.21.15.148:48094) 2023-10-10T20:19:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.131+0000 7fbb1d94f700 1 -- 172.21.15.148:0/1399030667 learned_addr learned my addr 172.21.15.148:0/1399030667 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916878399 0 0) 0x7fbb180c8930 con 0x7fbb1806b0d0 2023-10-10T20:19:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb04003570 con 0x7fbb1806b0d0 2023-10-10T20:19:34.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b8484b700 1 -- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 msgr2=0x7f3b8006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b8484b700 1 --2- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b8006ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b8484b700 1 -- 172.21.15.148:0/2506369469 >> v1:172.21.15.161:6789/0 conn(0x7f3b8006f1a0 legacy=0x7f3b800ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b8484b700 1 -- 172.21.15.148:0/2506369469 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800c8200 con 0x7f3b800bf970 2023-10-10T20:19:34.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b8484b700 1 --2- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800bfd40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009f20 tx=0x7f3b78009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7a78d0dd3a9fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f8341965700 1 -- 172.21.15.148:0/636341096 wait complete. 2023-10-10T20:19:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f5366ffd700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 0x7f5350004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f535c001950 tx=0x7f535c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f0a03e4f8fbbb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 -- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7840052e0 msgr2=0x7fa78400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7840052e0 0x7fa78400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 -- 172.21.15.148:0/1671068635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa784004810 msgr2=0x7fa784004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa784004810 0x7fa784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7fa798970700 1 -- 172.21.15.148:0/1671068635 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784003570 con 0x7fa784003c40 2023-10-10T20:19:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b7f7fe700 1 -- 172.21.15.148:0/2506369469 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800e070 con 0x7f3b800bf970 2023-10-10T20:19:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b7f7fe700 1 -- 172.21.15.148:0/2506369469 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7800bb10 con 0x7f3b800bf970 2023-10-10T20:19:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f3b7f7fe700 1 -- 172.21.15.148:0/2506369469 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b780092e0 con 0x7f3b800bf970 2023-10-10T20:19:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c001d90 con 0x7f5350004810 2023-10-10T20:19:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c0048f0 con 0x7f5350004810 2023-10-10T20:19:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.132+0000 7f53667fc700 1 -- 172.21.15.148:0/480322575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c004c90 con 0x7f5350004810 2023-10-10T20:19:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 -- 172.21.15.148:0/3401850133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a28070e10 msgr2=0x7f5a280b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/3401850133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a28070e10 0x7f5a280b0900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009ea0 tx=0x7f5a20009840 comp rx=0 tx=0).stop 2023-10-10T20:19:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7fa798970700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa784003c40 0x7fa784004030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa780007ac0 tx=0x7fa780002db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdacda85c7cd9314 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 -- 172.21.15.148:0/3401850133 shutdown_connections 2023-10-10T20:19:34.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/3401850133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a28070e10 0x7f5a280b0900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 -- 172.21.15.148:0/3401850133 >> 172.21.15.148:0/3401850133 conn(0x7f5a2806a250 msgr2=0x7f5a2806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780007920 con 0x7fa784003c40 2023-10-10T20:19:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78000ca10 con 0x7fa784003c40 2023-10-10T20:19:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7fa7937fe700 1 -- 172.21.15.148:0/1671068635 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000cdd0 con 0x7fa784003c40 2023-10-10T20:19:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 -- 172.21.15.148:0/3401850133 shutdown_connections 2023-10-10T20:19:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 -- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 msgr2=0x7f3b800bfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800bfd40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009f20 tx=0x7f3b78009840 comp rx=0 tx=0).stop 2023-10-10T20:19:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f536dce6700 1 -- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 msgr2=0x7f5350004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f536dce6700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 0x7f5350004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f535c001950 tx=0x7f535c007de0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f536dce6700 1 -- 172.21.15.148:0/480322575 shutdown_connections 2023-10-10T20:19:34.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f536dce6700 1 --2- 172.21.15.148:0/480322575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5350004810 0x7f5350004c80 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f535c001950 tx=0x7f535c007de0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f536dce6700 1 -- 172.21.15.148:0/480322575 >> 172.21.15.148:0/480322575 conn(0x7f5368068ce0 msgr2=0x7f53680690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 -- 172.21.15.148:0/2506369469 shutdown_connections 2023-10-10T20:19:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800bfd40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/2506369469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b8006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f3b86aaf700 1 -- 172.21.15.148:0/2506369469 >> 172.21.15.148:0/2506369469 conn(0x7f3b80067660 msgr2=0x7f3b80067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb180c7060 0x7fbb180c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369980568 0 0) 0x7fbb180b2860 con 0x7fbb1806d7a0 2023-10-10T20:19:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb180c8930 con 0x7fbb1806d7a0 2023-10-10T20:19:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249673685 0 0) 0x7fbb04003570 con 0x7fbb1806b0d0 2023-10-10T20:19:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb180c7060 msgr2=0x7fbb180c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb180c7060 0x7fbb180c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 >> v1:172.21.15.161:6789/0 conn(0x7fbb1806d7a0 legacy=0x7fbb180b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c7d20 con 0x7fbb1806b0d0 2023-10-10T20:19:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fbb10001770 con 0x7fbb1806b0d0 2023-10-10T20:19:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 >> v1:172.21.15.148:6789/0 conn(0x7fbb1806b0d0 legacy=0x7fbb1806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04003c20 0x7fbb04003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb04004810 0x7fbb04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04005230 0x7fbb0400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18156730 con 0x7fbb04005230 2023-10-10T20:19:34.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0400c130 con 0x7fbb04004810 2023-10-10T20:19:34.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0400c2f0 con 0x7fbb04003c20 2023-10-10T20:19:34.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7f3b86aaf700 1 -- 172.21.15.148:0/2506369469 shutdown_connections 2023-10-10T20:19:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb180c7060 0x7fbb180c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7f8341965700 1 Processor -- start 2023-10-10T20:19:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7f3b86aaf700 1 -- 172.21.15.148:0/2506369469 wait complete. 2023-10-10T20:19:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04005230 0x7fbb0400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7f8341965700 1 -- start start 2023-10-10T20:19:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fa79abd4700 1 -- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa784003c40 msgr2=0x7fa784004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fa79abd4700 1 --2- 172.21.15.148:0/1671068635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa784003c40 0x7fa784004030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa780007ac0 tx=0x7fa780002db0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 -- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04003c20 msgr2=0x7fbb04003ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04003c20 0x7fbb04003ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 -- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb04004810 msgr2=0x7fbb04004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb04004810 0x7fbb04004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.134+0000 7fbb1e951700 1 -- 172.21.15.148:0/1399030667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c7d20 con 0x7fbb04005230 2023-10-10T20:19:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.133+0000 7f5a2cb85700 1 -- 172.21.15.148:0/3401850133 wait complete. 2023-10-10T20:19:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f5a2cb85700 1 Processor -- start 2023-10-10T20:19:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f5a2cb85700 1 -- start start 2023-10-10T20:19:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fbb1e951700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04005230 0x7fbb0400ba60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb180c7430 tx=0x7fbb08008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c98b6c869e9d03 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fa799171700 1 -- 172.21.15.148:0/1671068635 reap_dead start 2023-10-10T20:19:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fa79abd4700 1 -- 172.21.15.148:0/1671068635 shutdown_connections 2023-10-10T20:19:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f833c159dd0 0x7f833c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c15db70 0x7f833c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0b1fa0 con 0x7f833c159dd0 2023-10-10T20:19:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c17aaf0 con 0x7f833c15db70 2023-10-10T20:19:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f8341965700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0b1950 con 0x7f833c0cbfd0 2023-10-10T20:19:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fa79abd4700 1 -- 172.21.15.148:0/1671068635 >> 172.21.15.148:0/1671068635 conn(0x7fa794070700 msgr2=0x7fa794070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb080113f0 con 0x7fbb04005230 2023-10-10T20:19:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08011570 con 0x7fbb04005230 2023-10-10T20:19:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7fbb1d14e700 1 -- 172.21.15.148:0/1399030667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08011930 con 0x7fbb04005230 2023-10-10T20:19:34.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f833affd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f833affd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51086/0 (socket says 172.21.15.148:51086) 2023-10-10T20:19:34.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f833affd700 1 -- 172.21.15.148:0/720241626 learned_addr learned my addr 172.21.15.148:0/720241626 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f3b86aaf700 1 Processor -- start 2023-10-10T20:19:34.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.135+0000 7f833a7fc700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f833c159dd0 0x7f833c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7fa79abd4700 1 -- 172.21.15.148:0/1671068635 shutdown_connections 2023-10-10T20:19:34.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833b7fe700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c15db70 0x7f833c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 -- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c15db70 msgr2=0x7f833c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c15db70 0x7f833c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 -- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f833c159dd0 msgr2=0x7f833c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f833c159dd0 0x7f833c0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 -- 172.21.15.148:0/720241626 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009ae0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f536dce6700 1 -- 172.21.15.148:0/480322575 shutdown_connections 2023-10-10T20:19:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f833affd700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f833000b810 tx=0x7f8330009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 0x7f5a281522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a280c0e10 0x7f5a2815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2815f440 con 0x7f5a280c0e10 2023-10-10T20:19:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2815eb10 con 0x7f5a280688a0 2023-10-10T20:19:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2cb85700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2815edf0 con 0x7f5a28070e10 2023-10-10T20:19:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51098/0 (socket says 172.21.15.148:51098) 2023-10-10T20:19:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 -- 172.21.15.148:0/485776116 learned_addr learned my addr 172.21.15.148:0/485776116 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a26d9d700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a280c0e10 0x7f5a2815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7fbb203b4700 1 -- 172.21.15.148:0/1399030667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04005230 msgr2=0x7fbb0400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7fbb203b4700 1 --2- 172.21.15.148:0/1399030667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb04005230 0x7fbb0400ba60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb180c7430 tx=0x7fbb08008040 comp rx=0 tx=0).stop 2023-10-10T20:19:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000f5f0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833000f770 con 0x7f833c0cbfd0 2023-10-10T20:19:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000fb30 con 0x7f833c0cbfd0 2023-10-10T20:19:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c0b1c30 con 0x7f833c0cbfd0 2023-10-10T20:19:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c0b2970 con 0x7f833c0cbfd0 2023-10-10T20:19:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a2659c700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 0x7f5a281522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 -- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 msgr2=0x7f5a281522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 0x7f5a281522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f5a25d9b700 1 -- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a280c0e10 msgr2=0x7f5a2815b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a25d9b700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a280c0e10 0x7f5a2815b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a25d9b700 1 -- 172.21.15.148:0/485776116 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20009ae0 con 0x7f5a28070e10 2023-10-10T20:19:34.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a2659c700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 0x7f5a281522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:34.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f536dce6700 1 -- 172.21.15.148:0/480322575 wait complete. 2023-10-10T20:19:34.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7fbb1e951700 1 -- 172.21.15.148:0/1399030667 reap_dead start 2023-10-10T20:19:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.136+0000 7f3b86aaf700 1 -- start start 2023-10-10T20:19:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b800c0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 0x7f3b801490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c0960 con 0x7f3b8006f1a0 2023-10-10T20:19:34.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c4190 con 0x7f3b800bf970 2023-10-10T20:19:34.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b86aaf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800c36e0 con 0x7f3b8006d4c0 2023-10-10T20:19:34.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a25d9b700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c590 tx=0x7f5a1c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7fbb203b4700 1 -- 172.21.15.148:0/1399030667 shutdown_connections 2023-10-10T20:19:34.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7fbb203b4700 1 -- 172.21.15.148:0/1399030667 >> 172.21.15.148:0/1399030667 conn(0x7fbb18067660 msgr2=0x7fbb18067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b8484b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b800c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b7ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 0x7f3b801490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b8484b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b800c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51100/0 (socket says 172.21.15.148:51100) 2023-10-10T20:19:34.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f3b8484b700 1 -- 172.21.15.148:0/4025033101 learned_addr learned my addr 172.21.15.148:0/4025033101 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7fbb203b4700 1 -- 172.21.15.148:0/1399030667 shutdown_connections 2023-10-10T20:19:34.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8330010480 con 0x7f833c0cbfd0 2023-10-10T20:19:34.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011590 con 0x7f5a28070e10 2023-10-10T20:19:34.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c011710 con 0x7f5a28070e10 2023-10-10T20:19:34.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011ad0 con 0x7f5a28070e10 2023-10-10T20:19:34.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2815f0d0 con 0x7f5a28070e10 2023-10-10T20:19:34.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28152830 con 0x7f5a28070e10 2023-10-10T20:19:34.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f536dce6700 1 Processor -- start 2023-10-10T20:19:34.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f3b7ffff700 1 -- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 msgr2=0x7f3b800c0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f3b7ffff700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b800c0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f3b7ffff700 1 -- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 msgr2=0x7f3b800afbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:34.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f3b7ffff700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800afbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f3b7ffff700 1 -- 172.21.15.148:0/4025033101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009ae0 con 0x7f3b8006f1a0 2023-10-10T20:19:34.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.138+0000 7f536dce6700 1 -- start start 2023-10-10T20:19:34.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f8322ffd700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 0x7f83280694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7fa79abd4700 1 -- 172.21.15.148:0/1671068635 wait complete. 2023-10-10T20:19:34.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7fa79abd4700 1 Processor -- start 2023-10-10T20:19:34.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83300843d0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 0x7f536816cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f536815f8b0 0x7f536816d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5368162140 con 0x7f536815d320 2023-10-10T20:19:34.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5368161530 con 0x7f5368161010 2023-10-10T20:19:34.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f536dce6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5368161810 con 0x7f536815f8b0 2023-10-10T20:19:34.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42416/0 (socket says 172.21.15.148:42416) 2023-10-10T20:19:34.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 -- 172.21.15.148:0/1516860436 learned_addr learned my addr 172.21.15.148:0/1516860436 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f5367fff700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f536815f8b0 0x7f536816d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f5366ffd700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 0x7f536816cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.137+0000 7fbb203b4700 1 -- 172.21.15.148:0/1399030667 wait complete. 2023-10-10T20:19:34.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7fbb203b4700 1 Processor -- start 2023-10-10T20:19:34.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 -- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f536815f8b0 msgr2=0x7f536816d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f536815f8b0 0x7f536816d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 -- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 msgr2=0x7f536816cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 0x7f536816cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7f53677fe700 1 -- 172.21.15.148:0/1516860436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368161af0 con 0x7f536815d320 2023-10-10T20:19:34.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f53677fe700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f535800e880 tx=0x7f5358008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f5366ffd700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 0x7f536816cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:34.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7fbb203b4700 1 -- start start 2023-10-10T20:19:34.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f3b7ffff700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 0x7f3b801490a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800b600 tx=0x7f3b680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c0052b0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358015690 con 0x7f536815d320 2023-10-10T20:19:34.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53580157f0 con 0x7f536815d320 2023-10-10T20:19:34.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5368161dd0 con 0x7f536815d320 2023-10-10T20:19:34.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358015b70 con 0x7f536815d320 2023-10-10T20:19:34.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2815f5f0 con 0x7f5a28070e10 2023-10-10T20:19:34.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536815d8b0 con 0x7f536815d320 2023-10-10T20:19:34.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.139+0000 7fa79abd4700 1 -- start start 2023-10-10T20:19:34.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794169d70 0x7fa794080790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 0x7fa7940821b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79416a140 con 0x7fa794169d70 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794081d20 con 0x7fa794087dd0 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fa79abd4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa794081110 con 0x7fa7940826f0 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa793fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 0x7fa7940821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51120/0 (socket says 172.21.15.148:51120) 2023-10-10T20:19:34.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 -- 172.21.15.148:0/2297704695 learned_addr learned my addr 172.21.15.148:0/2297704695 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.140+0000 7fbb203b4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04005230 0x7fbb180c0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb203b4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb203b4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 0x7fbb181768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb203b4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c5950 con 0x7fbb18176480 2023-10-10T20:19:34.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb203b4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18159700 con 0x7fbb18069410 2023-10-10T20:19:34.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb203b4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb181599e0 con 0x7fbb04005230 2023-10-10T20:19:34.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb1e150700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04005230 0x7fbb180c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb1d94f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a1c01b3b0 con 0x7f5a28070e10 2023-10-10T20:19:34.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb1d94f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42120/0 (socket says 172.21.15.148:42120) 2023-10-10T20:19:34.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb1d94f700 1 -- 172.21.15.148:0/959839257 learned_addr learned my addr 172.21.15.148:0/959839257 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 -- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 msgr2=0x7fa7940821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 0x7fa7940821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 -- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794169d70 msgr2=0x7fa794080790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794169d70 0x7fa794080790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fa799171700 1 -- 172.21.15.148:0/2297704695 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940813f0 con 0x7fa7940826f0 2023-10-10T20:19:34.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f833a7fc700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 0x7f83280694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7fbb1e951700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 0x7fbb181768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53480052b0 con 0x7f536815d320 2023-10-10T20:19:34.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fa793fff700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 0x7fa7940821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:34.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f5a167fc700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 0x7f5a08069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a1c0838c0 con 0x7f5a28070e10 2023-10-10T20:19:34.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f5a2659c700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 0x7f5a08069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800f510 con 0x7f3b8006f1a0 2023-10-10T20:19:34.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6800f690 con 0x7f3b8006f1a0 2023-10-10T20:19:34.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.141+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800fa50 con 0x7f3b8006f1a0 2023-10-10T20:19:34.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f833a7fc700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 0x7f83280694b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f833c0cac60 tx=0x7f832c009120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 -- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04005230 msgr2=0x7fbb180c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04005230 0x7fbb180c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 -- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 msgr2=0x7fbb181768b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 0x7fbb181768b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 -- 172.21.15.148:0/959839257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c55e0 con 0x7fbb18069410 2023-10-10T20:19:34.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1d94f700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001eb0 tx=0x7fbb14007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb1e951700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 0x7fbb181768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:34.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b801495e0 con 0x7f3b8006f1a0 2023-10-10T20:19:34.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b800c3400 con 0x7f3b8006f1a0 2023-10-10T20:19:34.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f5a2659c700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 0x7f5a08069530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b0550 tx=0x7f5a18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14005b30 con 0x7fbb18069410 2023-10-10T20:19:34.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14005c90 con 0x7fbb18069410 2023-10-10T20:19:34.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004a60 con 0x7fbb18069410 2023-10-10T20:19:34.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fa799171700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7940b1650 tx=0x7fa78c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb18176df0 con 0x7fbb18069410 2023-10-10T20:19:34.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5358013450 con 0x7f536815d320 2023-10-10T20:19:34.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.142+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb180c08b0 con 0x7fbb18069410 2023-10-10T20:19:34.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.143+0000 7f534f7fe700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.143+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5358084f50 con 0x7f536815d320 2023-10-10T20:19:34.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.143+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b800c4320 con 0x7f3b8006f1a0 2023-10-10T20:19:34.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.143+0000 7f5366ffd700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.144+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1804c3c0 con 0x7fbb18069410 2023-10-10T20:19:34.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.144+0000 7f5366ffd700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 0x7f53540693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f536804d2d0 tx=0x7f535c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.144+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb14005030 con 0x7fbb18069410 2023-10-10T20:19:34.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.144+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b68024430 con 0x7f3b8006f1a0 2023-10-10T20:19:34.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fbb0e7fc700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 0x7fbafc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fc42af7f700 1 Processor -- start 2023-10-10T20:19:34.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7f3b7cff9700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 0x7f3b60069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb14083f10 con 0x7fbb18069410 2023-10-10T20:19:34.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7f3b8484b700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 0x7f3b60069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fbb1e150700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 0x7fbafc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7f3b8484b700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 0x7f3b60069480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009ab0 tx=0x7f3b78018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fbb1e150700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 0x7fbafc0694b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb100013c0 tx=0x7fbb10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83300551f0 con 0x7f833c0cbfd0 2023-10-10T20:19:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.145+0000 7fc42af7f700 1 -- start start 2023-10-10T20:19:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc42af7f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 0x7fc4240c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc42af7f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc42406d6b0 0x7fc4240c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc42af7f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240c8760 con 0x7fc424155b60 2023-10-10T20:19:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc42af7f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240c7930 con 0x7fc42406d6b0 2023-10-10T20:19:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc42af7f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240af6e0 con 0x7fc42406d2e0 2023-10-10T20:19:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc423fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc42406d6b0 0x7fc4240c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc423fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc42406d6b0 0x7fc4240c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42456/0 (socket says 172.21.15.148:42456) 2023-10-10T20:19:34.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc423fff700 1 -- 172.21.15.148:0/405327327 learned_addr learned my addr 172.21.15.148:0/405327327 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.146+0000 7fc428d1b700 1 --2- 172.21.15.148:0/405327327 >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 0x7fc4240c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc423fff700 1 -- 172.21.15.148:0/405327327 >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 msgr2=0x7fc4240c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:34.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc423fff700 1 --2- 172.21.15.148:0/405327327 >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 0x7fc4240c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc423fff700 1 -- 172.21.15.148:0/405327327 >> v1:172.21.15.161:6789/0 conn(0x7fc424155b60 legacy=0x7fc424155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc423fff700 1 -- 172.21.15.148:0/405327327 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240c7b50 con 0x7fc42406d6b0 2023-10-10T20:19:34.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc428d1b700 1 --2- 172.21.15.148:0/405327327 >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 0x7fc4240c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:34.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.147+0000 7fc423fff700 1 --2- 172.21.15.148:0/405327327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d6b0 0x7fc4240c73f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc424071230 tx=0x7fc414009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d1bcf6b05315a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.148+0000 7fc4237fe700 1 -- 172.21.15.148:0/405327327 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400b6e0 con 0x7fc42406d6b0 2023-10-10T20:19:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.149+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b68055ca0 con 0x7f3b8006f1a0 2023-10-10T20:19:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.149+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5358055c50 con 0x7f536815d320 2023-10-10T20:19:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb14051200 con 0x7fbb18069410 2023-10-10T20:19:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00e890 con 0x7fa7940826f0 2023-10-10T20:19:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c00ea10 con 0x7fa7940826f0 2023-10-10T20:19:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00edd0 con 0x7fa7940826f0 2023-10-10T20:19:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7940816d0 con 0x7fa7940826f0 2023-10-10T20:19:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa794080cd0 con 0x7fa7940826f0 2023-10-10T20:19:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.150+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a2815f5f0 con 0x7f5a28070e10 2023-10-10T20:19:34.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.157+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b68051fe0 con 0x7f3b8006f1a0 2023-10-10T20:19:34.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.151+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79404c3c0 con 0x7fa7940826f0 2023-10-10T20:19:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.153+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa78c014690 con 0x7fa7940826f0 2023-10-10T20:19:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.154+0000 7fa790ff9700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 0x7fa778069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.154+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa78c084610 con 0x7fa7940826f0 2023-10-10T20:19:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.154+0000 7fa798970700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 0x7fa778069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.155+0000 7fa798970700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 0x7fa778069410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa78001b040 tx=0x7fa780018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.157+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa78c0518e0 con 0x7fa7940826f0 2023-10-10T20:19:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.158+0000 7fc4237fe700 1 -- 172.21.15.148:0/405327327 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41400b840 con 0x7fc42406d6b0 2023-10-10T20:19:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.158+0000 7fc4237fe700 1 -- 172.21.15.148:0/405327327 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400bbe0 con 0x7fc42406d6b0 2023-10-10T20:19:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.160+0000 7fc42af7f700 1 -- 172.21.15.148:0/405327327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d6b0 msgr2=0x7fc4240c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.160+0000 7fc42af7f700 1 --2- 172.21.15.148:0/405327327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d6b0 0x7fc4240c73f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc424071230 tx=0x7fc414009ae0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 -- 172.21.15.148:0/405327327 shutdown_connections 2023-10-10T20:19:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 --2- 172.21.15.148:0/405327327 >> v2:172.21.15.161:3300/0 conn(0x7fc42406d2e0 0x7fc4240c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 --2- 172.21.15.148:0/405327327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d6b0 0x7fc4240c73f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 -- 172.21.15.148:0/405327327 >> 172.21.15.148:0/405327327 conn(0x7fc424070700 msgr2=0x7fc424070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 -- 172.21.15.148:0/405327327 shutdown_connections 2023-10-10T20:19:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.161+0000 7fc42af7f700 1 -- 172.21.15.148:0/405327327 wait complete. 2023-10-10T20:19:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.163+0000 7fc42af7f700 1 Processor -- start 2023-10-10T20:19:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.163+0000 7fc42af7f700 1 -- start start 2023-10-10T20:19:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d2e0 0x7fc4240b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc42406d6b0 0x7fc4240b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42416f480 con 0x7fc42406d2e0 2023-10-10T20:19:34.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240b07e0 con 0x7fc42406d6b0 2023-10-10T20:19:34.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42af7f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240b0a60 con 0x7fc424155b60 2023-10-10T20:19:34.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42951c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42951c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51144/0 (socket says 172.21.15.148:51144) 2023-10-10T20:19:34.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.164+0000 7fc42951c700 1 -- 172.21.15.148:0/1988090055 learned_addr learned my addr 172.21.15.148:0/1988090055 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 -- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc42406d6b0 msgr2=0x7fc4240b02a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc42406d6b0 0x7fc4240b02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 -- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d2e0 msgr2=0x7fc4240b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d2e0 0x7fc4240b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 -- 172.21.15.148:0/1988090055 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418009420 con 0x7fc424155b60 2023-10-10T20:19:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42951c700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c8e0 tx=0x7fc41c001710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00ce80 con 0x7fc424155b60 2023-10-10T20:19:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c00a7e0 con 0x7fc424155b60 2023-10-10T20:19:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc414009d80 con 0x7fc424155b60 2023-10-10T20:19:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.165+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240b1020 con 0x7fc424155b60 2023-10-10T20:19:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.169+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00ab60 con 0x7fc424155b60 2023-10-10T20:19:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.170+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc41c012420 con 0x7fc424155b60 2023-10-10T20:19:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.171+0000 7fc420ff9700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 0x7fc4100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.171+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc41c0848f0 con 0x7fc424155b60 2023-10-10T20:19:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.174+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42404c3c0 con 0x7fc424155b60 2023-10-10T20:19:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.174+0000 7fc428d1b700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 0x7fc4100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.177+0000 7f27173b3700 1 Processor -- start 2023-10-10T20:19:34.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.177+0000 7f27173b3700 1 -- start start 2023-10-10T20:19:34.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7fc428d1b700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 0x7fc4100693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4240b1a40 tx=0x7fc4180091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f27173b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f271015a6e0 0x7f2710068950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f27173b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f271015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f27173b3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2710067660 con 0x7f271015aff0 2023-10-10T20:19:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f27173b3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271015b460 con 0x7f271015a6e0 2023-10-10T20:19:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f27173b3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100684c0 con 0x7f271006f630 2023-10-10T20:19:34.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f271514f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f271015a6e0 0x7f2710068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.178+0000 7f2715950700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f271015aff0 0x7f2710070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48108/0 (socket says 172.21.15.148:48108) 2023-10-10T20:19:34.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f2715950700 1 -- 172.21.15.148:0/4058771156 learned_addr learned my addr 172.21.15.148:0/4058771156 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:34.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f271514f700 1 -- 172.21.15.148:0/4058771156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 msgr2=0x7f271015aab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:34.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f271514f700 1 --2- 172.21.15.148:0/4058771156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f271015aab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f271514f700 1 -- 172.21.15.148:0/4058771156 >> v1:172.21.15.161:6789/0 conn(0x7f271015aff0 legacy=0x7f2710070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f271514f700 1 -- 172.21.15.148:0/4058771156 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100678b0 con 0x7f271015a6e0 2023-10-10T20:19:34.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f271514f700 1 --2- 172.21.15.148:0/4058771156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f2710068950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27000098e0 tx=0x7f27000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488bab3c7100c600 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f2707fff700 1 -- 172.21.15.148:0/4058771156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700009e80 con 0x7f271015a6e0 2023-10-10T20:19:34.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f2707fff700 1 -- 172.21.15.148:0/4058771156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000b860 con 0x7f271015a6e0 2023-10-10T20:19:34.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.179+0000 7f2707fff700 1 -- 172.21.15.148:0/4058771156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000bc20 con 0x7f271015a6e0 2023-10-10T20:19:34.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.180+0000 7f27173b3700 1 -- 172.21.15.148:0/4058771156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 msgr2=0x7f2710068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.180+0000 7f27173b3700 1 --2- 172.21.15.148:0/4058771156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f2710068950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f27000098e0 tx=0x7f27000091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc41c051b60 con 0x7fc424155b60 2023-10-10T20:19:34.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 -- 172.21.15.148:0/4058771156 shutdown_connections 2023-10-10T20:19:34.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 --2- 172.21.15.148:0/4058771156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f2710068950 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 --2- 172.21.15.148:0/4058771156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f271015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 -- 172.21.15.148:0/4058771156 >> 172.21.15.148:0/4058771156 conn(0x7f271006a250 msgr2=0x7f271006a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 -- 172.21.15.148:0/4058771156 shutdown_connections 2023-10-10T20:19:34.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.185+0000 7f27173b3700 1 -- 172.21.15.148:0/4058771156 wait complete. 2023-10-10T20:19:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.187+0000 7f27173b3700 1 Processor -- start 2023-10-10T20:19:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.187+0000 7f27173b3700 1 -- start start 2023-10-10T20:19:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f27100b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f27100cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 0x7f27100cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710155550 con 0x7f271015a6e0 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100b3820 con 0x7f271015aff0 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f27173b3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2710154c20 con 0x7f271006f630 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f2715950700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 0x7f27100cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f2715950700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 0x7f27100cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42156/0 (socket says 172.21.15.148:42156) 2023-10-10T20:19:34.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f2715950700 1 -- 172.21.15.148:0/2052140344 learned_addr learned my addr 172.21.15.148:0/2052140344 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:34.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.189+0000 7f271514f700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f27100b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f27100cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 -- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 msgr2=0x7f27100b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f27100b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 -- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 msgr2=0x7f27100cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 0x7f27100cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.190+0000 7f271494e700 1 -- 172.21.15.148:0/2052140344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700009420 con 0x7f271015a6e0 2023-10-10T20:19:34.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f271494e700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f27100cb550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b490 tx=0x7f270c009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00f510 con 0x7f271015a6e0 2023-10-10T20:19:34.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c00f670 con 0x7f271015a6e0 2023-10-10T20:19:34.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00fa30 con 0x7f271015a6e0 2023-10-10T20:19:34.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710154f00 con 0x7f271015a6e0 2023-10-10T20:19:34.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.191+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100b14f0 con 0x7f271015a6e0 2023-10-10T20:19:34.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.195+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f270c01a400 con 0x7f271015a6e0 2023-10-10T20:19:34.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.195+0000 7f27057fa700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 0x7f26fc069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.195+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f270c083eb0 con 0x7f271015a6e0 2023-10-10T20:19:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.197+0000 7f271514f700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 0x7f26fc069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.197+0000 7f271514f700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 0x7f26fc069650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f27000098e0 tx=0x7f2700017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.197+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f40052b0 con 0x7f271015a6e0 2023-10-10T20:19:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.207+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f270c051180 con 0x7f271015a6e0 2023-10-10T20:19:34.518 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.516+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbb180c8000 con 0x7fbb18069410 2023-10-10T20:19:34.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.516+0000 7fbb0e7fc700 1 -- 172.21.15.148:0/959839257 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb14004e90 con 0x7fbb18069410 2023-10-10T20:19:34.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 msgr2=0x7fbafc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 0x7fbafc0694b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb100013c0 tx=0x7fbb10009040 comp rx=0 tx=0).stop 2023-10-10T20:19:34.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 msgr2=0x7fbb1814be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001eb0 tx=0x7fbb14007f40 comp rx=0 tx=0).stop 2023-10-10T20:19:34.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 shutdown_connections 2023-10-10T20:19:34.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbafc067000 0x7fbafc0694b0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb100013c0 tx=0x7fbb10009040 comp rx=0 tx=0).stop 2023-10-10T20:19:34.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18176480 0x7fbb181768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb04005230 0x7fbb180c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 --2- 172.21.15.148:0/959839257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb18069410 0x7fbb1814be70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 >> 172.21.15.148:0/959839257 conn(0x7fbb18067660 msgr2=0x7fbb18067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.524+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 shutdown_connections 2023-10-10T20:19:34.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.525+0000 7fbb203b4700 1 -- 172.21.15.148:0/959839257 wait complete. 2023-10-10T20:19:34.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.526+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5348005830 con 0x7f536815d320 2023-10-10T20:19:34.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.529+0000 7f534f7fe700 1 -- 172.21.15.148:0/1516860436 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53580557e0 con 0x7f536815d320 2023-10-10T20:19:34.534 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 msgr2=0x7f53540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 0x7f53540693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f536804d2d0 tx=0x7f535c015000 comp rx=0 tx=0).stop 2023-10-10T20:19:34.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 msgr2=0x7f536815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f535800e880 tx=0x7f5358008040 comp rx=0 tx=0).stop 2023-10-10T20:19:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 shutdown_connections 2023-10-10T20:19:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f536815f8b0 0x7f536816d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f536815d320 0x7f536815f370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 --2- 172.21.15.148:0/1516860436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5368161010 0x7f536816cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 >> 172.21.15.148:0/1516860436 conn(0x7f5368068ce0 msgr2=0x7f53680d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f831c005560 con 0x7f833c0cbfd0 2023-10-10T20:19:34.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.534+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 shutdown_connections 2023-10-10T20:19:34.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.535+0000 7f536dce6700 1 -- 172.21.15.148:0/1516860436 wait complete. 2023-10-10T20:19:34.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.535+0000 7f8322ffd700 1 -- 172.21.15.148:0/720241626 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8330054d80 con 0x7f833c0cbfd0 2023-10-10T20:19:34.540 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 msgr2=0x7f83280694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 0x7f83280694b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f833c0cac60 tx=0x7f832c009120 comp rx=0 tx=0).stop 2023-10-10T20:19:34.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 msgr2=0x7f833c173c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f833000b810 tx=0x7f8330009250 comp rx=0 tx=0).stop 2023-10-10T20:19:34.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 shutdown_connections 2023-10-10T20:19:34.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328067000 0x7f83280694b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c15db70 0x7f833c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f833c0cbfd0 0x7f833c173c60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 --2- 172.21.15.148:0/720241626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f833c159dd0 0x7f833c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 >> 172.21.15.148:0/720241626 conn(0x7f833c0cd1e0 msgr2=0x7f833c06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 shutdown_connections 2023-10-10T20:19:34.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f8341965700 1 -- 172.21.15.148:0/720241626 wait complete. 2023-10-10T20:19:34.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.539+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b8006aeb0 con 0x7f3b8006f1a0 2023-10-10T20:19:34.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.540+0000 7f3b7cff9700 1 -- 172.21.15.148:0/4025033101 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b68055600 con 0x7f3b8006f1a0 2023-10-10T20:19:34.545 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.541+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5a2815f5b0 con 0x7f5a28070e10 2023-10-10T20:19:34.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.542+0000 7f5a167fc700 1 -- 172.21.15.148:0/485776116 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a1c017070 con 0x7f5a28070e10 2023-10-10T20:19:34.546 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-10T20:19:34.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 msgr2=0x7f3b60069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 0x7f3b60069480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009ab0 tx=0x7f3b78018000 comp rx=0 tx=0).stop 2023-10-10T20:19:34.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 msgr2=0x7f3b801490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 0x7f3b801490a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800b600 tx=0x7f3b680091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa794081e90 con 0x7fa7940826f0 2023-10-10T20:19:34.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 shutdown_connections 2023-10-10T20:19:34.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066fd0 0x7f3b60069480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b800bf970 0x7f3b800afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b8006d4c0 0x7f3b800c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 --2- 172.21.15.148:0/4025033101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b8006f1a0 0x7f3b801490a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 >> 172.21.15.148:0/4025033101 conn(0x7f3b80067660 msgr2=0x7f3b800cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 shutdown_connections 2023-10-10T20:19:34.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f3b86aaf700 1 -- 172.21.15.148:0/4025033101 wait complete. 2023-10-10T20:19:34.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 msgr2=0x7f5a08069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 0x7f5a08069530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b0550 tx=0x7f5a18009040 comp rx=0 tx=0).stop 2023-10-10T20:19:34.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 msgr2=0x7f5a2815b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.545+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c590 tx=0x7f5a1c00c8a0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 shutdown_connections 2023-10-10T20:19:34.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a08067080 0x7f5a08069530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a280c0e10 0x7f5a2815b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a280688a0 0x7f5a281522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 --2- 172.21.15.148:0/485776116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a28070e10 0x7f5a2815b1b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 >> 172.21.15.148:0/485776116 conn(0x7f5a2806a250 msgr2=0x7f5a280ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 shutdown_connections 2023-10-10T20:19:34.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.546+0000 7f5a2cb85700 1 -- 172.21.15.148:0/485776116 wait complete. 2023-10-10T20:19:34.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.549+0000 7fa790ff9700 1 -- 172.21.15.148:0/2297704695 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa78c054f00 con 0x7fa7940826f0 2023-10-10T20:19:34.554 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.552+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 msgr2=0x7fa778069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.552+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 0x7fa778069410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa78001b040 tx=0x7fa780018000 comp rx=0 tx=0).stop 2023-10-10T20:19:34.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.553+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 msgr2=0x7fa7940c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.553+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7940b1650 tx=0x7fa78c005dc0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 shutdown_connections 2023-10-10T20:19:34.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa778066f60 0x7fa778069410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7940826f0 0x7fa7940c5dd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa794169d70 0x7fa794080790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 --2- 172.21.15.148:0/2297704695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa794087dd0 0x7fa7940821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 >> 172.21.15.148:0/2297704695 conn(0x7fa794070700 msgr2=0x7fa7940c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.556+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 shutdown_connections 2023-10-10T20:19:34.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.557+0000 7fa79abd4700 1 -- 172.21.15.148:0/2297704695 wait complete. 2023-10-10T20:19:34.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-10T20:19:34.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-10-10T20:19:34.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-10T20:19:34.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-10T20:19:34.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.573+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc42416f5f0 con 0x7fc424155b60 2023-10-10T20:19:34.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.574+0000 7fc420ff9700 1 -- 172.21.15.148:0/1988090055 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc41c055180 con 0x7fc424155b60 2023-10-10T20:19:34.576 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 msgr2=0x7fc4100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 0x7fc4100693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4240b1a40 tx=0x7fc4180091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 msgr2=0x7fc4241718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c8e0 tx=0x7fc41c001710 comp rx=0 tx=0).stop 2023-10-10T20:19:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 shutdown_connections 2023-10-10T20:19:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc410066f30 0x7fc4100693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc424155b60 0x7fc4241718b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42406d2e0 0x7fc4240b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 --2- 172.21.15.148:0/1988090055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc42406d6b0 0x7fc4240b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 >> 172.21.15.148:0/1988090055 conn(0x7fc424070700 msgr2=0x7fc424155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 shutdown_connections 2023-10-10T20:19:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.576+0000 7fc42af7f700 1 -- 172.21.15.148:0/1988090055 wait complete. 2023-10-10T20:19:34.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.579+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f26f4005560 con 0x7f271015a6e0 2023-10-10T20:19:34.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.579+0000 7f27057fa700 1 -- 172.21.15.148:0/2052140344 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f270c0547a0 con 0x7f271015a6e0 2023-10-10T20:19:34.582 INFO:teuthology.orchestra.run.smithi148.stdout:55834574853 2023-10-10T20:19:34.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-10T20:19:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 msgr2=0x7f26fc069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 0x7f26fc069650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f27000098e0 tx=0x7f2700017000 comp rx=0 tx=0).stop 2023-10-10T20:19:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 msgr2=0x7f27100cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f27100cb550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b490 tx=0x7f270c009060 comp rx=0 tx=0).stop 2023-10-10T20:19:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 shutdown_connections 2023-10-10T20:19:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f26fc0671a0 0x7f26fc069650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271015aff0 0x7f27100cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f271006f630 0x7f27100b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 --2- 172.21.15.148:0/2052140344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271015a6e0 0x7f27100cb550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 >> 172.21.15.148:0/2052140344 conn(0x7f271006a250 msgr2=0x7f27100cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 shutdown_connections 2023-10-10T20:19:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:34.582+0000 7f27173b3700 1 -- 172.21.15.148:0/2052140344 wait complete. 2023-10-10T20:19:34.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-10T20:19:34.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-10T20:19:35.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-10T20:19:35.567 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-10T20:19:35.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-10T20:19:35.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-10T20:19:35.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-10T20:19:35.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-10T20:19:35.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-10T20:19:35.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-10T20:19:35.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.743+0000 7fa8b02db700 1 Processor -- start 2023-10-10T20:19:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.743+0000 7fa8b02db700 1 -- start start 2023-10-10T20:19:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8b02db700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8a8063f40 0x7fa8a80b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8b02db700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80b2310 con 0x7fa8a80af810 2023-10-10T20:19:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8b02db700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80b1fe0 con 0x7fa8a8063ac0 2023-10-10T20:19:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8b02db700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a80afbe0 con 0x7fa8a8063f40 2023-10-10T20:19:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8ad876700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa8a8063ac0 0x7fa8a80acaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48114/0 (socket says 172.21.15.148:48114) 2023-10-10T20:19:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.745+0000 7fa8ad876700 1 -- 172.21.15.148:0/1477124646 learned_addr learned my addr 172.21.15.148:0/1477124646 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.746+0000 7fa8ae878700 1 --2- 172.21.15.148:0/1477124646 >> v2:172.21.15.161:3300/0 conn(0x7fa8a8063f40 0x7fa8a80b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.746+0000 7fa8ae878700 1 -- 172.21.15.148:0/1477124646 >> v1:172.21.15.161:6789/0 conn(0x7fa8a8063ac0 legacy=0x7fa8a80acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.746+0000 7fa8ae878700 1 -- 172.21.15.148:0/1477124646 >> v1:172.21.15.148:6789/0 conn(0x7fa8a80af810 legacy=0x7fa8a80adee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.746+0000 7fa8ae878700 1 -- 172.21.15.148:0/1477124646 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80ae730 con 0x7fa8a8063f40 2023-10-10T20:19:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.751+0000 7fa8ae878700 1 --2- 172.21.15.148:0/1477124646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a80b0160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0014a40 tx=0x7fa8a00143c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ec19be536fb8e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.751+0000 7fa8ad075700 1 -- 172.21.15.148:0/1477124646 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000fd60 con 0x7fa8a8063f40 2023-10-10T20:19:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.752+0000 7fa8ad075700 1 -- 172.21.15.148:0/1477124646 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a00110d0 con 0x7fa8a8063f40 2023-10-10T20:19:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.752+0000 7fa8ad075700 1 -- 172.21.15.148:0/1477124646 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0011470 con 0x7fa8a8063f40 2023-10-10T20:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.753+0000 7fa8b02db700 1 -- 172.21.15.148:0/1477124646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 msgr2=0x7fa8a80b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.753+0000 7fa8b02db700 1 --2- 172.21.15.148:0/1477124646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a80b0160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0014a40 tx=0x7fa8a00143c0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.753+0000 7fd114220700 1 Processor -- start 2023-10-10T20:19:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.754+0000 7fa8b02db700 1 -- 172.21.15.148:0/1477124646 shutdown_connections 2023-10-10T20:19:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.754+0000 7fa8b02db700 1 --2- 172.21.15.148:0/1477124646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a80b0160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.754+0000 7fa8b02db700 1 -- 172.21.15.148:0/1477124646 >> 172.21.15.148:0/1477124646 conn(0x7fa8a80cd1e0 msgr2=0x7fa8a80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.754+0000 7fa8b02db700 1 -- 172.21.15.148:0/1477124646 shutdown_connections 2023-10-10T20:19:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.753+0000 7fd114220700 1 -- start start 2023-10-10T20:19:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd114220700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd114220700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd10c0d18d0 0x7fd10c068cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd114220700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0cb810 con 0x7fd10c0afe40 2023-10-10T20:19:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd114220700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0b0f80 con 0x7fd10c0d18d0 2023-10-10T20:19:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd114220700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b0370 con 0x7fd10c0afa70 2023-10-10T20:19:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51156/0 (socket says 172.21.15.148:51156) 2023-10-10T20:19:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 -- 172.21.15.148:0/2438248124 learned_addr learned my addr 172.21.15.148:0/2438248124 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.754+0000 7fa8b02db700 1 -- 172.21.15.148:0/1477124646 wait complete. 2023-10-10T20:19:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 -- 172.21.15.148:0/2438248124 >> v2:172.21.15.148:3300/0 conn(0x7fd10c0d18d0 msgr2=0x7fd10c068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 --2- 172.21.15.148:0/2438248124 >> v2:172.21.15.148:3300/0 conn(0x7fd10c0d18d0 0x7fd10c068cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 -- 172.21.15.148:0/2438248124 >> v1:172.21.15.161:6789/0 conn(0x7fd10c0afe40 legacy=0x7fd10c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.755+0000 7fd111fbc700 1 -- 172.21.15.148:0/2438248124 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0b0650 con 0x7fd10c0afa70 2023-10-10T20:19:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fa8b02db700 1 Processor -- start 2023-10-10T20:19:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fd111fbc700 1 --2- 172.21.15.148:0/2438248124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1080099b0 tx=0x7fd108009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48822eea93a4e042 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fa8b02db700 1 -- start start 2023-10-10T20:19:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a81586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8a80af810 0x7fa8a816dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816bdb0 con 0x7fa8a8063ac0 2023-10-10T20:19:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816b510 con 0x7fa8a8063f40 2023-10-10T20:19:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8b02db700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a816b7f0 con 0x7fa8a80af810 2023-10-10T20:19:35.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fd110fba700 1 -- 172.21.15.148:0/2438248124 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800b860 con 0x7fd10c0afa70 2023-10-10T20:19:35.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fd110fba700 1 -- 172.21.15.148:0/2438248124 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10800b9c0 con 0x7fd10c0afa70 2023-10-10T20:19:35.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.756+0000 7fd110fba700 1 -- 172.21.15.148:0/2438248124 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800bd60 con 0x7fd10c0afa70 2023-10-10T20:19:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 -- 172.21.15.148:0/2438248124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 msgr2=0x7fd10c0cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 --2- 172.21.15.148:0/2438248124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1080099b0 tx=0x7fd108009250 comp rx=0 tx=0).stop 2023-10-10T20:19:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 -- 172.21.15.148:0/2438248124 shutdown_connections 2023-10-10T20:19:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 --2- 172.21.15.148:0/2438248124 >> v2:172.21.15.148:3300/0 conn(0x7fd10c0d18d0 0x7fd10c068cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 --2- 172.21.15.148:0/2438248124 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0afa70 0x7fd10c0cb2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fd114220700 1 -- 172.21.15.148:0/2438248124 >> 172.21.15.148:0/2438248124 conn(0x7fd10c068130 msgr2=0x7fd10c0d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8ae077700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8ae077700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42492/0 (socket says 172.21.15.148:42492) 2023-10-10T20:19:35.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.757+0000 7fa8ae077700 1 -- 172.21.15.148:0/563943167 learned_addr learned my addr 172.21.15.148:0/563943167 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fd114220700 1 -- 172.21.15.148:0/2438248124 shutdown_connections 2023-10-10T20:19:35.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 -- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8a80af810 msgr2=0x7fa8a816dde0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:35.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8a80af810 0x7fa8a816dde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 -- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 msgr2=0x7fa8a81586c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:35.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a81586c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 -- 172.21.15.148:0/563943167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0014600 con 0x7fa8a8063ac0 2023-10-10T20:19:35.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fa8ae077700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b5e0 tx=0x7fa8980090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.758+0000 7fd114220700 1 -- 172.21.15.148:0/2438248124 wait complete. 2023-10-10T20:19:35.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800f510 con 0x7fa8a8063ac0 2023-10-10T20:19:35.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89800f690 con 0x7fa8a8063ac0 2023-10-10T20:19:35.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800fa50 con 0x7fa8a8063ac0 2023-10-10T20:19:35.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a816bad0 con 0x7fa8a8063ac0 2023-10-10T20:19:35.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a816b1d0 con 0x7fa8a8063ac0 2023-10-10T20:19:35.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.759+0000 7fd114220700 1 Processor -- start 2023-10-10T20:19:35.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.760+0000 7fd114220700 1 -- start start 2023-10-10T20:19:35.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa89801a400 con 0x7fa8a8063ac0 2023-10-10T20:19:35.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c0afa70 0x7fd10c0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0b6550 0x7fd10c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b5a40 con 0x7fd10c0afa70 2023-10-10T20:19:35.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b5d20 con 0x7fd10c0d18d0 2023-10-10T20:19:35.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd114220700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b6000 con 0x7fd10c0b6550 2023-10-10T20:19:35.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1117bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1117bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42172/0 (socket says 172.21.15.148:42172) 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1117bb700 1 -- 172.21.15.148:0/31775853 learned_addr learned my addr 172.21.15.148:0/31775853 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1127bd700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0b6550 0x7fd10c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1117bb700 1 -- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0b6550 msgr2=0x7fd10c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.761+0000 7fd1117bb700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0b6550 0x7fd10c15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fd1117bb700 1 -- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c0afa70 msgr2=0x7fd10c0c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fd1117bb700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c0afa70 0x7fd10c0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fd1117bb700 1 -- 172.21.15.148:0/31775853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080094f0 con 0x7fd10c0d18d0 2023-10-10T20:19:35.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fd1117bb700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd10401db20 tx=0x7fd104001750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fa89e7fc700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 0x7fa88c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.762+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa898083ff0 con 0x7fa8a8063ac0 2023-10-10T20:19:35.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.763+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a8176f70 con 0x7fa8a8063ac0 2023-10-10T20:19:35.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.769+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400a7e0 con 0x7fd10c0d18d0 2023-10-10T20:19:35.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.769+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10400a960 con 0x7fd10c0d18d0 2023-10-10T20:19:35.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.769+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400ad20 con 0x7fd10c0d18d0 2023-10-10T20:19:35.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.769+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c15ab50 con 0x7fd10c0d18d0 2023-10-10T20:19:35.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.769+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c14cf50 con 0x7fd10c0d18d0 2023-10-10T20:19:35.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.772+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f00052e0 con 0x7fd10c0d18d0 2023-10-10T20:19:35.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.771+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8a8176f70 con 0x7fa8a8063ac0 2023-10-10T20:19:35.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.772+0000 7fa8ad876700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 0x7fa88c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.775+0000 7fa8ad876700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 0x7fa88c0694b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40058d0 tx=0x7fa8a4007a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.778+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd104003030 con 0x7fd10c0d18d0 2023-10-10T20:19:35.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.779+0000 7fd101ffb700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 0x7fd0f8069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.779+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd104095580 con 0x7fd10c0d18d0 2023-10-10T20:19:35.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.779+0000 7fd111fbc700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 0x7fd0f8069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.780+0000 7fd111fbc700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 0x7fd0f8069690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1080099b0 tx=0x7fd108017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.783+0000 7fb20c8bb700 1 Processor -- start 2023-10-10T20:19:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.783+0000 7fb20c8bb700 1 -- start start 2023-10-10T20:19:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.784+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1040627d0 con 0x7fd10c0d18d0 2023-10-10T20:19:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb20c8bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb2080afd80 0x7fb20806ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb20c8bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb20c8bb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb208067660 con 0x7fb2080b0150 2023-10-10T20:19:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb20c8bb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080c8840 con 0x7fb2080afd80 2023-10-10T20:19:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb20c8bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20815eaf0 con 0x7fb2080c9c20 2023-10-10T20:19:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51174/0 (socket says 172.21.15.148:51174) 2023-10-10T20:19:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 -- 172.21.15.148:0/1157633989 learned_addr learned my addr 172.21.15.148:0/1157633989 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 -- 172.21.15.148:0/1157633989 >> v2:172.21.15.148:3300/0 conn(0x7fb2080afd80 msgr2=0x7fb20806ec70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-10T20:19:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 --2- 172.21.15.148:0/1157633989 >> v2:172.21.15.148:3300/0 conn(0x7fb2080afd80 0x7fb20806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 -- 172.21.15.148:0/1157633989 >> v1:172.21.15.161:6789/0 conn(0x7fb2080b0150 legacy=0x7fb2080c2820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:35.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7fb206d9d700 1 -- 172.21.15.148:0/1157633989 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208067f00 con 0x7fb2080c9c20 2023-10-10T20:19:35.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.786+0000 7fb206d9d700 1 --2- 172.21.15.148:0/1157633989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb200009f20 tx=0x7fb200009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca173538caf4ab9d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.786+0000 7fb20559a700 1 -- 172.21.15.148:0/1157633989 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000090d0 con 0x7fb2080c9c20 2023-10-10T20:19:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.786+0000 7fb20559a700 1 -- 172.21.15.148:0/1157633989 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20001f070 con 0x7fb2080c9c20 2023-10-10T20:19:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.786+0000 7fb20559a700 1 -- 172.21.15.148:0/1157633989 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000ba40 con 0x7fb2080c9c20 2023-10-10T20:19:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1157633989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 msgr2=0x7fb208070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1157633989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb200009f20 tx=0x7fb200009840 comp rx=0 tx=0).stop 2023-10-10T20:19:35.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f0ef96b9700 1 Processor -- start 2023-10-10T20:19:35.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.785+0000 7f9cae443700 1 Processor -- start 2023-10-10T20:19:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.786+0000 7f9cae443700 1 -- start start 2023-10-10T20:19:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cae443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 0x7f9ca80af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cae443700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca80c6eb0 con 0x7f9ca80ccec0 2023-10-10T20:19:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cae443700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca80b1270 con 0x7f9ca80ccaf0 2023-10-10T20:19:35.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cae443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80b0660 con 0x7f9ca8088ac0 2023-10-10T20:19:35.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cac9e0700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9ca80ccec0 0x7f9ca80afe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58304/0 (socket says 172.21.15.148:58304) 2023-10-10T20:19:35.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9cac9e0700 1 -- 172.21.15.148:0/1869925895 learned_addr learned my addr 172.21.15.148:0/1869925895 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:35.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.787+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1869925895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 0x7f9ca80af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f9ca6ffd700 1 -- 172.21.15.148:0/1869925895 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214284874 0 0) 0x7f9ca80c6eb0 con 0x7f9ca80ccec0 2023-10-10T20:19:35.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f9ca6ffd700 1 -- 172.21.15.148:0/1869925895 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c90003550 con 0x7f9ca80ccec0 2023-10-10T20:19:35.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1157633989 shutdown_connections 2023-10-10T20:19:35.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1157633989 >> v2:172.21.15.148:3300/0 conn(0x7fb2080afd80 0x7fb20806ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1157633989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb208070700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1157633989 >> 172.21.15.148:0/1157633989 conn(0x7fb2080ca4f0 msgr2=0x7fb2080ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1157633989 shutdown_connections 2023-10-10T20:19:35.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1157633989 wait complete. 2023-10-10T20:19:35.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f9ca77fe700 1 -- 172.21.15.148:0/1869925895 >> v1:172.21.15.161:6789/0 conn(0x7f9ca80ccaf0 legacy=0x7f9ca80883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f9ca77fe700 1 -- 172.21.15.148:0/1869925895 >> v1:172.21.15.148:6789/0 conn(0x7f9ca80ccec0 legacy=0x7f9ca80afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f9ca77fe700 1 -- 172.21.15.148:0/1869925895 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80b0940 con 0x7f9ca8088ac0 2023-10-10T20:19:35.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.788+0000 7f0ef96b9700 1 -- start start 2023-10-10T20:19:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1869925895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 0x7f9ca80af910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d80 tx=0x7f9c980096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7675420abeb279ef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f9ca6ffd700 1 -- 172.21.15.148:0/1869925895 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800bae0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f9ca6ffd700 1 -- 172.21.15.148:0/1869925895 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800bc40 con 0x7f9ca8088ac0 2023-10-10T20:19:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7fb4db8d1700 1 Processor -- start 2023-10-10T20:19:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f0ef96b9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0ef40afe40 0x7f0ef40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f0ef96b9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40b0370 con 0x7f0ef40afa70 2023-10-10T20:19:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f0ef96b9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40b0650 con 0x7f0ef40d18d0 2023-10-10T20:19:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f0ef96b9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef40b0f80 con 0x7f0ef40afe40 2023-10-10T20:19:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7fb20c8bb700 1 Processor -- start 2023-10-10T20:19:35.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7fb20c8bb700 1 -- start start 2023-10-10T20:19:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ef40afa70 0x7f0ef40cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58318/0 (socket says 172.21.15.148:58318) 2023-10-10T20:19:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/2407048257 learned_addr learned my addr 172.21.15.148:0/2407048257 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:35.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/2407048257 >> v2:172.21.15.161:3300/0 conn(0x7f0ef40afe40 0x7f0ef40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7f9ca6ffd700 1 -- 172.21.15.148:0/1869925895 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98011430 con 0x7f9ca8088ac0 2023-10-10T20:19:35.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f9cae443700 1 -- 172.21.15.148:0/1869925895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 msgr2=0x7f9ca80af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f9cae443700 1 --2- 172.21.15.148:0/1869925895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 0x7f9ca80af910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d80 tx=0x7f9c980096d0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f9cae443700 1 -- 172.21.15.148:0/1869925895 shutdown_connections 2023-10-10T20:19:35.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f9cae443700 1 --2- 172.21.15.148:0/1869925895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca8088ac0 0x7f9ca80af910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f9cae443700 1 -- 172.21.15.148:0/1869925895 >> 172.21.15.148:0/1869925895 conn(0x7f9ca8070700 msgr2=0x7f9ca8070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805401523 0 0) 0x7f0ef40b0370 con 0x7f0ef40afa70 2023-10-10T20:19:35.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0edc003570 con 0x7f0ef40afa70 2023-10-10T20:19:35.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276886790 0 0) 0x7f0edc003570 con 0x7f0ef40afa70 2023-10-10T20:19:35.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 >> v2:172.21.15.161:3300/0 conn(0x7f0ef40afe40 msgr2=0x7f0ef40b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 --2- 172.21.15.148:0/2407048257 >> v2:172.21.15.161:3300/0 conn(0x7f0ef40afe40 0x7f0ef40b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 >> v1:172.21.15.161:6789/0 conn(0x7f0ef40d18d0 legacy=0x7f0ef4068cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-10T20:19:35.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40b0930 con 0x7f0ef40afa70 2023-10-10T20:19:35.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f0eec0017f0 con 0x7f0ef40afa70 2023-10-10T20:19:35.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 >> v1:172.21.15.148:6789/0 conn(0x7f0ef40afa70 legacy=0x7f0ef40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 0x7f0edc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc004810 0x7f0edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0edc005230 0x7f0edc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40b0f80 con 0x7f0edc004810 2023-10-10T20:19:35.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc00c200 con 0x7f0edc005230 2023-10-10T20:19:35.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc00c3c0 con 0x7f0edc003a60 2023-10-10T20:19:35.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 0x7fb2080b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 0x7fb20814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb2080c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080b3680 con 0x7fb2080afd80 2023-10-10T20:19:35.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208161f20 con 0x7fb2080b0150 2023-10-10T20:19:35.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20c8bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20815f720 con 0x7fb2080c9c20 2023-10-10T20:19:35.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.789+0000 7fb4db8d1700 1 -- start start 2023-10-10T20:19:35.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7fb4db8d1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb4d40b6990 0x7fb4d40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.790+0000 7fb4db8d1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40b5980 con 0x7fb4d4147880 2023-10-10T20:19:35.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb4db8d1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40b4d70 con 0x7fb4d40b5e10 2023-10-10T20:19:35.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb4db8d1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d40cfcc0 con 0x7fb4d40b6990 2023-10-10T20:19:35.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb4d966d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb4d4147880 0x7fb4d40b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58334/0 (socket says 172.21.15.148:58334) 2023-10-10T20:19:35.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb4d966d700 1 -- 172.21.15.148:0/1673755784 learned_addr learned my addr 172.21.15.148:0/1673755784 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:35.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7f9cae443700 1 -- 172.21.15.148:0/1869925895 shutdown_connections 2023-10-10T20:19:35.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb205d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 0x7fb20814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb2080c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb205d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 0x7fb20814e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:42184/0 (socket says 172.21.15.148:42184) 2023-10-10T20:19:35.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb205d9b700 1 -- 172.21.15.148:0/1408014375 learned_addr learned my addr 172.21.15.148:0/1408014375 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb20659c700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 0x7fb2080b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fafa156d700 1 Processor -- start 2023-10-10T20:19:35.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fafa156d700 1 -- start start 2023-10-10T20:19:35.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 -- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 msgr2=0x7fb20814e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 0x7fb20814e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 -- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 msgr2=0x7fb2080b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 0x7fb2080b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7fb206d9d700 1 -- 172.21.15.148:0/1408014375 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009ae0 con 0x7fb2080c9c20 2023-10-10T20:19:35.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.792+0000 7fb20659c700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 0x7fb2080b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:35.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.791+0000 7f9cae443700 1 -- 172.21.15.148:0/1869925895 wait complete. 2023-10-10T20:19:35.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.792+0000 7f9cae443700 1 Processor -- start 2023-10-10T20:19:35.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.792+0000 7f9cae443700 1 -- start start 2023-10-10T20:19:35.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faf9c1577d0 0x7faf9c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf9c157ba0 0x7faf9c06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c15b0d0 0x7faf9c06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15c360 con 0x7faf9c157ba0 2023-10-10T20:19:35.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15b790 con 0x7faf9c1577d0 2023-10-10T20:19:35.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fafa156d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15ba30 con 0x7faf9c15b0d0 2023-10-10T20:19:35.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf9c157ba0 0x7faf9c06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf9c157ba0 0x7faf9c06a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42508/0 (socket says 172.21.15.148:42508) 2023-10-10T20:19:35.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 -- 172.21.15.148:0/646232155 learned_addr learned my addr 172.21.15.148:0/646232155 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9affd700 1 --2- 172.21.15.148:0/646232155 >> v2:172.21.15.161:3300/0 conn(0x7faf9c1577d0 0x7faf9c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 -- 172.21.15.148:0/646232155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c15b0d0 msgr2=0x7faf9c06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/646232155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c15b0d0 0x7faf9c06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 -- 172.21.15.148:0/646232155 >> v2:172.21.15.161:3300/0 conn(0x7faf9c1577d0 msgr2=0x7faf9c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/646232155 >> v2:172.21.15.161:3300/0 conn(0x7faf9c1577d0 0x7faf9c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 -- 172.21.15.148:0/646232155 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c15bd10 con 0x7faf9c157ba0 2023-10-10T20:19:35.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 0x7f0edc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/646232155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c157ba0 0x7faf9c06a520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b6dc0 tx=0x7faf90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b23bbc981984a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc004810 0x7f0edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fb4d8e6c700 1 --2- 172.21.15.148:0/1673755784 >> v2:172.21.15.161:3300/0 conn(0x7fb4d40b6990 0x7fb4d40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7f0ef37fe700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0edc005230 0x7f0edc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7faf99ffb700 1 -- 172.21.15.148:0/646232155 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90009e70 con 0x7faf9c157ba0 2023-10-10T20:19:35.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7faf99ffb700 1 -- 172.21.15.148:0/646232155 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9000b860 con 0x7faf9c157ba0 2023-10-10T20:19:35.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7faf99ffb700 1 -- 172.21.15.148:0/646232155 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9000bc00 con 0x7faf9c157ba0 2023-10-10T20:19:35.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb4d8e6c700 1 -- 172.21.15.148:0/1673755784 >> v1:172.21.15.161:6789/0 conn(0x7fb4d40b5e10 legacy=0x7fb4d40b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb4d8e6c700 1 -- 172.21.15.148:0/1673755784 >> v1:172.21.15.148:6789/0 conn(0x7fb4d4147880 legacy=0x7fb4d40b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb4d8e6c700 1 -- 172.21.15.148:0/1673755784 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d40b5050 con 0x7fb4d40b6990 2023-10-10T20:19:35.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.793+0000 7fb206d9d700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb2080c4b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb200025000 tx=0x7fb200028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0edc005230 msgr2=0x7f0edc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0edc005230 0x7f0edc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/2407048257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc004810 msgr2=0x7f0edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc004810 0x7f0edc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/2407048257 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc003570 con 0x7f0edc003a60 2023-10-10T20:19:35.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc004810 0x7f0edc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:35.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000090d0 con 0x7fb2080c9c20 2023-10-10T20:19:35.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200009230 con 0x7fb2080c9c20 2023-10-10T20:19:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20002f680 con 0x7fb2080c9c20 2023-10-10T20:19:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 0x7f9ca817b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 0x7f9ca8167370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 0x7f9ca8169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca817c340 con 0x7f9ca8088ac0 2023-10-10T20:19:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8188b70 con 0x7f9ca80ccaf0 2023-10-10T20:19:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cae443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca817bcf0 con 0x7f9ca80ccec0 2023-10-10T20:19:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cac9e0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 0x7f9ca8169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 0x7f9ca817b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cac9e0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 0x7f9ca8169770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51222/0 (socket says 172.21.15.148:51222) 2023-10-10T20:19:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9cac9e0700 1 -- 172.21.15.148:0/1106506744 learned_addr learned my addr 172.21.15.148:0/1106506744 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 0x7f9ca8167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7fb4d8e6c700 1 --2- 172.21.15.148:0/1673755784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d40b3fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098e0 tx=0x7fb4c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6fa5b0040f444f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 -- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 msgr2=0x7f9ca8169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 0x7f9ca8169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 -- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 msgr2=0x7f9ca8167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 0x7f9ca8167370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca7fff700 1 -- 172.21.15.148:0/1106506744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9800d040 con 0x7f9ca8088ac0 2023-10-10T20:19:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 0x7f9ca8167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.794+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 0x7f0edc003e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00a950 tx=0x7f0eec0021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6dd0ded3a58b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7f9ca7fff700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 0x7f9ca817b570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c4e0 tx=0x7f9c9c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 -- 172.21.15.148:0/646232155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c157ba0 msgr2=0x7faf9c06a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 --2- 172.21.15.148:0/646232155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c157ba0 0x7faf9c06a520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b6dc0 tx=0x7faf90009250 comp rx=0 tx=0).stop 2023-10-10T20:19:35.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0070d0 con 0x7f0edc003a60 2023-10-10T20:19:35.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec007230 con 0x7f0edc003a60 2023-10-10T20:19:35.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7f0ef1ffb700 1 -- 172.21.15.148:0/2407048257 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0075f0 con 0x7f0edc003a60 2023-10-10T20:19:35.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 -- 172.21.15.148:0/646232155 shutdown_connections 2023-10-10T20:19:35.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 --2- 172.21.15.148:0/646232155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c15b0d0 0x7faf9c06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 --2- 172.21.15.148:0/646232155 >> v2:172.21.15.161:3300/0 conn(0x7faf9c1577d0 0x7faf9c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 --2- 172.21.15.148:0/646232155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c157ba0 0x7faf9c06a520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 -- 172.21.15.148:0/646232155 >> 172.21.15.148:0/646232155 conn(0x7faf9c068ce0 msgr2=0x7faf9c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7f0ef37fe700 1 -- 172.21.15.148:0/2407048257 reap_dead start 2023-10-10T20:19:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 -- 172.21.15.148:0/646232155 shutdown_connections 2023-10-10T20:19:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fafa156d700 1 -- 172.21.15.148:0/646232155 wait complete. 2023-10-10T20:19:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fb4cbfff700 1 -- 172.21.15.148:0/1673755784 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4009e80 con 0x7fb4d40b6990 2023-10-10T20:19:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fb4cbfff700 1 -- 172.21.15.148:0/1673755784 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c400b760 con 0x7fb4d40b6990 2023-10-10T20:19:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fb4cbfff700 1 -- 172.21.15.148:0/1673755784 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400bb00 con 0x7fb4d40b6990 2023-10-10T20:19:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f0ef96b9700 1 -- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 msgr2=0x7f0edc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 0x7f0edc003e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00a950 tx=0x7f0eec0021f0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2080b3420 con 0x7fb2080c9c20 2023-10-10T20:19:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.795+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20815f0d0 con 0x7fb2080c9c20 2023-10-10T20:19:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c011590 con 0x7f9ca8088ac0 2023-10-10T20:19:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c011710 con 0x7f9ca8088ac0 2023-10-10T20:19:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c011ad0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca817bfd0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca8169cb0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f0ef96b9700 1 -- 172.21.15.148:0/2407048257 shutdown_connections 2023-10-10T20:19:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/2407048257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0edc003a60 0x7f0edc003e50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7f0ef96b9700 1 -- 172.21.15.148:0/2407048257 >> 172.21.15.148:0/2407048257 conn(0x7f0ef4068130 msgr2=0x7f0ef40d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7fb4db8d1700 1 -- 172.21.15.148:0/1673755784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 msgr2=0x7fb4d40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/1673755784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d40b3fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098e0 tx=0x7fb4c40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.796+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20815f8b0 con 0x7fb2080c9c20 2023-10-10T20:19:35.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.797+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c9c01c460 con 0x7f9ca8088ac0 2023-10-10T20:19:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb20002fa70 con 0x7fb2080c9c20 2023-10-10T20:19:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb4db8d1700 1 -- 172.21.15.148:0/1673755784 shutdown_connections 2023-10-10T20:19:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/1673755784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d40b3fd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb4db8d1700 1 -- 172.21.15.148:0/1673755784 >> 172.21.15.148:0/1673755784 conn(0x7fb4d4068ce0 msgr2=0x7fb4d40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.797+0000 7fafa156d700 1 Processor -- start 2023-10-10T20:19:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7f9c8ffff700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 0x7f9c940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c9c0840f0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb1f67fc700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 0x7fb1e8069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2000972a0 con 0x7fb2080c9c20 2023-10-10T20:19:35.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fb20659c700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 0x7fb1e8069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 0x7f9c940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7f9ca77fe700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 0x7f9c940694b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d80 tx=0x7f9c98018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fb20659c700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 0x7fb1e8069480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc000f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fb4db8d1700 1 -- 172.21.15.148:0/1673755784 shutdown_connections 2023-10-10T20:19:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fb4db8d1700 1 -- 172.21.15.148:0/1673755784 wait complete. 2023-10-10T20:19:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.798+0000 7fafa156d700 1 -- start start 2023-10-10T20:19:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c157ba0 0x7faf9c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf9c15b0d0 0x7faf9c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0b1b20 con 0x7faf9c1577d0 2023-10-10T20:19:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0b0c90 con 0x7faf9c15b0d0 2023-10-10T20:19:35.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7fafa156d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0b0f10 con 0x7faf9c157ba0 2023-10-10T20:19:35.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7faf9affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7faf9affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42530/0 (socket says 172.21.15.148:42530) 2023-10-10T20:19:35.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7faf9affd700 1 -- 172.21.15.148:0/387309432 learned_addr learned my addr 172.21.15.148:0/387309432 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c157ba0 0x7faf9c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.799+0000 7faf9b7fe700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf9c15b0d0 0x7faf9c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 -- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c157ba0 msgr2=0x7faf9c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c157ba0 0x7faf9c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 -- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf9c15b0d0 msgr2=0x7faf9c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf9c15b0d0 0x7faf9c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 -- 172.21.15.148:0/387309432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900094f0 con 0x7faf9c1577d0 2023-10-10T20:19:35.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca804c3c0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7f0ef96b9700 1 -- 172.21.15.148:0/2407048257 shutdown_connections 2023-10-10T20:19:35.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7f0ef96b9700 1 -- 172.21.15.148:0/2407048257 wait complete. 2023-10-10T20:19:35.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.801+0000 7fb4db8d1700 1 Processor -- start 2023-10-10T20:19:35.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.801+0000 7fb4db8d1700 1 -- start start 2023-10-10T20:19:35.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.800+0000 7faf9affd700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00e960 tx=0x7faf8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.802+0000 7f0ef96b9700 1 Processor -- start 2023-10-10T20:19:35.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.802+0000 7f0ef96b9700 1 -- start start 2023-10-10T20:19:35.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d416e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 0x7fb4d416ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d406cc40 con 0x7fb4d4147880 2023-10-10T20:19:35.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d406d200 con 0x7fb4d40b6990 2023-10-10T20:19:35.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4db8d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d41429c0 con 0x7fb4d40b5e10 2023-10-10T20:19:35.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d9e6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 0x7fb4d416ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51250/0 (socket says 172.21.15.148:51250) 2023-10-10T20:19:35.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d9e6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 0x7fb4d416ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42540/0 (socket says 172.21.15.148:42540) 2023-10-10T20:19:35.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 -- 172.21.15.148:0/2530389603 learned_addr learned my addr 172.21.15.148:0/2530389603 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef40c9e20 0x7f0ef4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ef40cece0 0x7f0ef40cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40cf670 con 0x7f0edc003a60 2023-10-10T20:19:35.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4155150 con 0x7f0ef40c9e20 2023-10-10T20:19:35.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef96b9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4155430 con 0x7f0ef40cece0 2023-10-10T20:19:35.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42552/0 (socket says 172.21.15.148:42552) 2023-10-10T20:19:35.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/3178793108 learned_addr learned my addr 172.21.15.148:0/3178793108 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef40c9e20 0x7f0ef4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7f0ef37fe700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ef40cece0 0x7f0ef40cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 -- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 msgr2=0x7fb4d416e700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-10T20:19:35.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d416e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 -- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 msgr2=0x7fb4d416ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 0x7fb4d416ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.803+0000 7fb4d966d700 1 -- 172.21.15.148:0/2530389603 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4009420 con 0x7fb4d40b5e10 2023-10-10T20:19:35.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ef40cece0 msgr2=0x7f0ef40cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ef40cece0 0x7f0ef40cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef40c9e20 msgr2=0x7f0ef4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef40c9e20 0x7f0ef4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 -- 172.21.15.148:0/3178793108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4155710 con 0x7f0edc003a60 2023-10-10T20:19:35.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4d966d700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c790 tx=0x7fb4d0007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7f0ef2ffd700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00af40 tx=0x7f0eec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0005b30 con 0x7fb4d40b5e10 2023-10-10T20:19:35.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0005c90 con 0x7fb4d40b5e10 2023-10-10T20:19:35.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d416f260 con 0x7fb4d40b5e10 2023-10-10T20:19:35.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0007340 con 0x7fb4d40b5e10 2023-10-10T20:19:35.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.804+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d406ce90 con 0x7fb4d40b5e10 2023-10-10T20:19:35.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.805+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb200064590 con 0x7fb2080c9c20 2023-10-10T20:19:35.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.805+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c013680 con 0x7faf9c1577d0 2023-10-10T20:19:35.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec01d430 con 0x7f0edc003a60 2023-10-10T20:19:35.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec01d590 con 0x7f0edc003a60 2023-10-10T20:19:35.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec01d950 con 0x7f0edc003a60 2023-10-10T20:19:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.807+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c9c0513e0 con 0x7f9ca8088ac0 2023-10-10T20:19:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.807+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef41559f0 con 0x7f0edc003a60 2023-10-10T20:19:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.807+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef415ad60 con 0x7f0edc003a60 2023-10-10T20:19:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0137e0 con 0x7faf9c1577d0 2023-10-10T20:19:35.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c013ba0 con 0x7faf9c1577d0 2023-10-10T20:19:35.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0b11f0 con 0x7faf9c1577d0 2023-10-10T20:19:35.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c0b17b0 con 0x7faf9c1577d0 2023-10-10T20:19:35.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.806+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4d001b3b0 con 0x7fb4d40b5e10 2023-10-10T20:19:35.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.807+0000 7fb4c97fa700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 0x7fb4b8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.807+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4d0083b30 con 0x7fb4d40b5e10 2023-10-10T20:19:35.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.808+0000 7fb4d8e6c700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 0x7fb4b8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.808+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf8c00c490 con 0x7faf9c1577d0 2023-10-10T20:19:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.809+0000 7faf82ffd700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.809+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faf8c084740 con 0x7faf9c1577d0 2023-10-10T20:19:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.809+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.810+0000 7fb4d8e6c700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 0x7fb4b8069460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4146940 tx=0x7fb4c4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.810+0000 7faf9a7fc700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b6dc0 tx=0x7faf90017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.811+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d404c3c0 con 0x7fb4d40b5e10 2023-10-10T20:19:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.811+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c0052b0 con 0x7faf9c1577d0 2023-10-10T20:19:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.812+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0eec01dcc0 con 0x7f0edc003a60 2023-10-10T20:19:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.812+0000 7f0edaffd700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 0x7f0ee00694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.812+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0eec086c90 con 0x7f0edc003a60 2023-10-10T20:19:35.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.815+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4d0017080 con 0x7fb4d40b5e10 2023-10-10T20:19:35.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.816+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 0x7f0ee00694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.820+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf8c051910 con 0x7faf9c1577d0 2023-10-10T20:19:35.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.820+0000 7f0ef27fc700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 0x7f0ee00694e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4155cf0 tx=0x7f0ee8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.827+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4071330 con 0x7f0edc003a60 2023-10-10T20:19:35.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.842+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eec053ee0 con 0x7f0edc003a60 2023-10-10T20:19:35.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.870+0000 7fe75ee23700 1 Processor -- start 2023-10-10T20:19:35.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.870+0000 7fe75ee23700 1 -- start start 2023-10-10T20:19:35.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7580bfbe0 0x7fe7580ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe7580cd1d0 0x7fe7580af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75815e9c0 con 0x7fe7580bfbe0 2023-10-10T20:19:35.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7580cd620 con 0x7fe7580cd1d0 2023-10-10T20:19:35.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75ee23700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580b0380 con 0x7fe7580afc20 2023-10-10T20:19:35.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75d3c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75d3c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51260/0 (socket says 172.21.15.148:51260) 2023-10-10T20:19:35.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe75d3c0700 1 -- 172.21.15.148:0/2263996639 learned_addr learned my addr 172.21.15.148:0/2263996639 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.872+0000 7fe757fff700 1 --2- 172.21.15.148:0/2263996639 >> v2:172.21.15.161:3300/0 conn(0x7fe7580cd1d0 0x7fe7580af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe75cbbf700 1 --2- 172.21.15.148:0/2263996639 >> v2:172.21.15.148:3300/0 conn(0x7fe7580bfbe0 0x7fe7580ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 -- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 msgr2=0x7fe758162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 -- 172.21.15.148:0/2263996639 >> v2:172.21.15.148:3300/0 conn(0x7fe7580bfbe0 msgr2=0x7fe7580ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:35.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 --2- 172.21.15.148:0/2263996639 >> v2:172.21.15.148:3300/0 conn(0x7fe7580bfbe0 0x7fe7580ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 -- 172.21.15.148:0/2263996639 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7580b0660 con 0x7fe7580cd1d0 2023-10-10T20:19:35.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe757fff700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580af6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe758071230 tx=0x7fe74c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf2d952ad6b1b06 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe75d3c0700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:35.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe7577fe700 1 -- 172.21.15.148:0/2263996639 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c011630 con 0x7fe7580cd1d0 2023-10-10T20:19:35.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe7577fe700 1 -- 172.21.15.148:0/2263996639 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011790 con 0x7fe7580cd1d0 2023-10-10T20:19:35.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.873+0000 7fe7577fe700 1 -- 172.21.15.148:0/2263996639 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c011b50 con 0x7fe7580cd1d0 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.874+0000 7fe75ee23700 1 -- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 msgr2=0x7fe7580af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.874+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580af6e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe758071230 tx=0x7fe74c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 -- 172.21.15.148:0/2263996639 shutdown_connections 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe758162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2263996639 >> v2:172.21.15.148:3300/0 conn(0x7fe7580bfbe0 0x7fe7580ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2263996639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580af6e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 -- 172.21.15.148:0/2263996639 >> 172.21.15.148:0/2263996639 conn(0x7fe758070700 msgr2=0x7fe758070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:35.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 -- 172.21.15.148:0/2263996639 shutdown_connections 2023-10-10T20:19:35.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.875+0000 7fe75ee23700 1 -- 172.21.15.148:0/2263996639 wait complete. 2023-10-10T20:19:35.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.877+0000 7fe75ee23700 1 Processor -- start 2023-10-10T20:19:35.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.877+0000 7fe75ee23700 1 -- start start 2023-10-10T20:19:35.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe75815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 0x7fe7580ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580aae80 con 0x7fe7580bfbe0 2023-10-10T20:19:35.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580aa550 con 0x7fe7580cd1d0 2023-10-10T20:19:35.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.878+0000 7fe75ee23700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7580aa830 con 0x7fe7580afc20 2023-10-10T20:19:35.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.879+0000 7fe75cbbf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe75815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.879+0000 7fe75d3c0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.879+0000 7fe75cbbf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe75815ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51276/0 (socket says 172.21.15.148:51276) 2023-10-10T20:19:35.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.879+0000 7fe75cbbf700 1 -- 172.21.15.148:0/2800348972 learned_addr learned my addr 172.21.15.148:0/2800348972 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:35.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 0x7fe7580ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 -- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 msgr2=0x7fe75815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe75815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 -- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 msgr2=0x7fe7580a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:35.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:35.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.880+0000 7fe757fff700 1 -- 172.21.15.148:0/2800348972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750009ae0 con 0x7fe7580bfbe0 2023-10-10T20:19:35.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe757fff700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 0x7fe7580ab310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0093f0 tx=0x7fe74c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c011900 con 0x7fe7580bfbe0 2023-10-10T20:19:35.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011a60 con 0x7fe7580bfbe0 2023-10-10T20:19:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c025670 con 0x7fe7580bfbe0 2023-10-10T20:19:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c009420 con 0x7fe7580bfbe0 2023-10-10T20:19:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.881+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7580aa200 con 0x7fe7580bfbe0 2023-10-10T20:19:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.886+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe74c023070 con 0x7fe7580bfbe0 2023-10-10T20:19:35.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.886+0000 7fe754ff9700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 0x7fe73c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:35.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.886+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe74c0853e0 con 0x7fe7580bfbe0 2023-10-10T20:19:35.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.887+0000 7fe75cbbf700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 0x7fe73c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:35.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.887+0000 7fe75cbbf700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 0x7fe73c069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe748006fd0 tx=0x7fe748008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:35.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.888+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7340052b0 con 0x7fe7580bfbe0 2023-10-10T20:19:35.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:35.897+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe74c052630 con 0x7fe7580bfbe0 2023-10-10T20:19:36.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.155+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa8a8176f70 con 0x7fa8a8063ac0 2023-10-10T20:19:36.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.156+0000 7fa89e7fc700 1 -- 172.21.15.148:0/563943167 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa898054e50 con 0x7fa8a8063ac0 2023-10-10T20:19:36.160 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 msgr2=0x7fa88c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 0x7fa88c0694b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40058d0 tx=0x7fa8a4007a30 comp rx=0 tx=0).stop 2023-10-10T20:19:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 msgr2=0x7fa8a816ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa89800b5e0 tx=0x7fa8980090f0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 shutdown_connections 2023-10-10T20:19:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa88c067000 0x7fa88c0694b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8a80af810 0x7fa8a816dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a8063ac0 0x7fa8a816ac90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.159+0000 7fa8b02db700 1 --2- 172.21.15.148:0/563943167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a8063f40 0x7fa8a81586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.160+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 >> 172.21.15.148:0/563943167 conn(0x7fa8a80cd1e0 msgr2=0x7fa8a8069520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.160+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 shutdown_connections 2023-10-10T20:19:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.160+0000 7fa8b02db700 1 -- 172.21.15.148:0/563943167 wait complete. 2023-10-10T20:19:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.169+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd0f0005590 con 0x7fd10c0d18d0 2023-10-10T20:19:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.170+0000 7fd101ffb700 1 -- 172.21.15.148:0/31775853 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd104065df0 con 0x7fd10c0d18d0 2023-10-10T20:19:36.175 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.175+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9ca80b0c20 con 0x7f9ca8088ac0 2023-10-10T20:19:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.176+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 msgr2=0x7fd0f8069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.176+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 0x7fd0f8069690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1080099b0 tx=0x7fd108017000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.176+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 msgr2=0x7fd10c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.176+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd10401db20 tx=0x7fd104001750 comp rx=0 tx=0).stop 2023-10-10T20:19:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.176+0000 7f9c8ffff700 1 -- 172.21.15.148:0/1106506744 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c9c054a00 con 0x7f9ca8088ac0 2023-10-10T20:19:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 shutdown_connections 2023-10-10T20:19:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd0f80671e0 0x7fd0f8069690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd10c0b6550 0x7fd10c15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10c0afa70 0x7fd10c0c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 --2- 172.21.15.148:0/31775853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c0d18d0 0x7fd10c0b4f90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.177+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 >> 172.21.15.148:0/31775853 conn(0x7fd10c068130 msgr2=0x7fd10c0cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.178+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 shutdown_connections 2023-10-10T20:19:36.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.178+0000 7fd114220700 1 -- 172.21.15.148:0/31775853 wait complete. 2023-10-10T20:19:36.184 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-10T20:19:36.185 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 msgr2=0x7f9c940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 0x7f9c940694b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d80 tx=0x7f9c98018000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 msgr2=0x7f9ca817b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 0x7f9ca817b570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00c4e0 tx=0x7f9c9c007ea0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 shutdown_connections 2023-10-10T20:19:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c94067000 0x7f9c940694b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ca80ccec0 0x7f9ca8169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca8088ac0 0x7f9ca817b570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 --2- 172.21.15.148:0/1106506744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ca80ccaf0 0x7f9ca8167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.180+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 >> 172.21.15.148:0/1106506744 conn(0x7f9ca8070700 msgr2=0x7f9ca80b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.181+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 shutdown_connections 2023-10-10T20:19:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.181+0000 7f9cae443700 1 -- 172.21.15.148:0/1106506744 wait complete. 2023-10-10T20:19:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.189+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb20804c3c0 con 0x7fb2080c9c20 2023-10-10T20:19:36.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.190+0000 7fb1f67fc700 1 -- 172.21.15.148:0/1408014375 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb200067bb0 con 0x7fb2080c9c20 2023-10-10T20:19:36.193 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.194+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb4d40b5330 con 0x7fb4d40b5e10 2023-10-10T20:19:36.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.194+0000 7fb4c97fa700 1 -- 172.21.15.148:0/2530389603 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb4d0050d80 con 0x7fb4d40b5e10 2023-10-10T20:19:36.197 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-10T20:19:36.199 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.197+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 msgr2=0x7fb4b8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.197+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 0x7fb4b8069460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4146940 tx=0x7fb4c4016000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.197+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 msgr2=0x7fb4d4142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.197+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c790 tx=0x7fb4d0007ef0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 shutdown_connections 2023-10-10T20:19:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4b8066fb0 0x7fb4b8069460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d4147880 0x7fb4d416ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d40b5e10 0x7fb4d4142480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 --2- 172.21.15.148:0/2530389603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d40b6990 0x7fb4d416e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 >> 172.21.15.148:0/2530389603 conn(0x7fb4d4068ce0 msgr2=0x7fb4d40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 msgr2=0x7fb1e8069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 0x7fb1e8069480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc000f90 comp rx=0 tx=0).stop 2023-10-10T20:19:36.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 msgr2=0x7fb2080c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb2080c4b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb200025000 tx=0x7fb200028000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 shutdown_connections 2023-10-10T20:19:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1e8066fd0 0x7fb1e8069480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2080c9c20 0x7fb2080c4b30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2080afd80 0x7fb2080b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 --2- 172.21.15.148:0/1408014375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2080b0150 0x7fb20814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 >> 172.21.15.148:0/1408014375 conn(0x7fb2080ca4f0 msgr2=0x7fb208068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 shutdown_connections 2023-10-10T20:19:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.198+0000 7fb4db8d1700 1 -- 172.21.15.148:0/2530389603 wait complete. 2023-10-10T20:19:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.199+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 shutdown_connections 2023-10-10T20:19:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.199+0000 7fb20c8bb700 1 -- 172.21.15.148:0/1408014375 wait complete. 2023-10-10T20:19:36.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-10T20:19:36.208 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.208+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf7c005560 con 0x7faf9c1577d0 2023-10-10T20:19:36.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.208+0000 7faf82ffd700 1 -- 172.21.15.148:0/387309432 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf8c021090 con 0x7faf9c1577d0 2023-10-10T20:19:36.211 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-10T20:19:36.215 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 msgr2=0x7faf880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 0x7faf880693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b6dc0 tx=0x7faf90017000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 msgr2=0x7faf9c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00e960 tx=0x7faf8c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 shutdown_connections 2023-10-10T20:19:36.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf88066f30 0x7faf880693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf9c15b0d0 0x7faf9c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf9c1577d0 0x7faf9c0b1fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 --2- 172.21.15.148:0/387309432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf9c157ba0 0x7faf9c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 >> 172.21.15.148:0/387309432 conn(0x7faf9c068ce0 msgr2=0x7faf9c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 shutdown_connections 2023-10-10T20:19:36.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.212+0000 7fafa156d700 1 -- 172.21.15.148:0/387309432 wait complete. 2023-10-10T20:19:36.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-10T20:19:36.220 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-10T20:19:36.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.229+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0ef4064830 con 0x7f0edc003a60 2023-10-10T20:19:36.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.229+0000 7f0edaffd700 1 -- 172.21.15.148:0/3178793108 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0eec057500 con 0x7f0edc003a60 2023-10-10T20:19:36.232 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.233 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 msgr2=0x7f0ee00694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 0x7f0ee00694e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4155cf0 tx=0x7f0ee8008040 comp rx=0 tx=0).stop 2023-10-10T20:19:36.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 msgr2=0x7f0ef4154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00af40 tx=0x7f0eec016000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 shutdown_connections 2023-10-10T20:19:36.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ee0067030 0x7f0ee00694e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ef40cece0 0x7f0ef40cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0edc003a60 0x7f0ef4154ae0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.231+0000 7f0ef96b9700 1 --2- 172.21.15.148:0/3178793108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ef40c9e20 0x7f0ef4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.232+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 >> 172.21.15.148:0/3178793108 conn(0x7f0ef4068130 msgr2=0x7f0ef4151a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.232+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 shutdown_connections 2023-10-10T20:19:36.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.232+0000 7f0ef96b9700 1 -- 172.21.15.148:0/3178793108 wait complete. 2023-10-10T20:19:36.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.242+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe734005560 con 0x7fe7580bfbe0 2023-10-10T20:19:36.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.242+0000 7fe754ff9700 1 -- 172.21.15.148:0/2800348972 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe74c015070 con 0x7fe7580bfbe0 2023-10-10T20:19:36.245 INFO:teuthology.orchestra.run.smithi148.stdout:55834574854 2023-10-10T20:19:36.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-10T20:19:36.246 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 msgr2=0x7fe73c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 0x7fe73c069500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe748006fd0 tx=0x7fe748008040 comp rx=0 tx=0).stop 2023-10-10T20:19:36.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 msgr2=0x7fe7580ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 0x7fe7580ab310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0093f0 tx=0x7fe74c017000 comp rx=0 tx=0).stop 2023-10-10T20:19:36.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 shutdown_connections 2023-10-10T20:19:36.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe73c067050 0x7fe73c069500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7580cd1d0 0x7fe7580a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7580afc20 0x7fe75815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 --2- 172.21.15.148:0/2800348972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7580bfbe0 0x7fe7580ab310 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 >> 172.21.15.148:0/2800348972 conn(0x7fe758070700 msgr2=0x7fe7580bffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 shutdown_connections 2023-10-10T20:19:36.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.245+0000 7fe75ee23700 1 -- 172.21.15.148:0/2800348972 wait complete. 2023-10-10T20:19:36.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-10T20:19:36.259 DEBUG:teuthology.parallel:result is None 2023-10-10T20:19:36.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-10T20:19:36.260 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:36.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.411+0000 7f16734fa700 1 Processor -- start 2023-10-10T20:19:36.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.411+0000 7f16734fa700 1 -- start start 2023-10-10T20:19:36.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f16734fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f166c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f16734fa700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f166c157c40 0x7f166c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f16734fa700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c15cbb0 con 0x7f166c15a620 2023-10-10T20:19:36.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f16734fa700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c15bfa0 con 0x7f166c157c40 2023-10-10T20:19:36.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f16734fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15c280 con 0x7f166c157870 2023-10-10T20:19:36.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f1670a95700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f166c15a620 0x7f166c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58346/0 (socket says 172.21.15.148:58346) 2023-10-10T20:19:36.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f1670a95700 1 -- 172.21.15.148:0/1156095007 learned_addr learned my addr 172.21.15.148:0/1156095007 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:36.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f1671296700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f166c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> v2:172.21.15.161:3300/0 conn(0x7f166c157c40 0x7f166c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.412+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630454226 0 0) 0x7f166c15cbb0 con 0x7f166c15a620 2023-10-10T20:19:36.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1658003570 con 0x7f166c15a620 2023-10-10T20:19:36.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470570319 0 0) 0x7f1658003570 con 0x7f166c15a620 2023-10-10T20:19:36.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 msgr2=0x7f166c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:36.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f166c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 >> v2:172.21.15.161:3300/0 conn(0x7f166c157c40 msgr2=0x7f166c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:36.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 --2- 172.21.15.148:0/1156095007 >> v2:172.21.15.161:3300/0 conn(0x7f166c157c40 0x7f166c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c15c560 con 0x7f166c15a620 2023-10-10T20:19:36.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f165c001470 con 0x7f166c15a620 2023-10-10T20:19:36.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> v2:172.21.15.161:3300/0 conn(0x7f166c157c40 0x7f166c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:36.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 >> v1:172.21.15.148:6789/0 conn(0x7f166c15a620 legacy=0x7f166c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658003a60 0x7f1658003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f1658004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f16580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15bfa0 con 0x7f1658004d50 2023-10-10T20:19:36.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c15c280 con 0x7f1658003a60 2023-10-10T20:19:36.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165800ba60 con 0x7f166c157870 2023-10-10T20:19:36.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671296700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658003a60 0x7f1658003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f16580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1670a95700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f1658004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 -- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 msgr2=0x7f1658004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:36.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c157870 0x7f1658004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 -- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658003a60 msgr2=0x7f1658003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:36.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658003a60 0x7f1658003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 -- 172.21.15.148:0/1156095007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658003570 con 0x7f1658004d50 2023-10-10T20:19:36.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671296700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1658003a60 0x7f1658003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:36.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.413+0000 7f1671a97700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f16580051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f166400e3c0 tx=0x7f166400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410ff673f9d89b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166400ec10 con 0x7f1658004d50 2023-10-10T20:19:36.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400f040 con 0x7f1658004d50 2023-10-10T20:19:36.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f1663fff700 1 -- 172.21.15.148:0/1156095007 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664014480 con 0x7f1658004d50 2023-10-10T20:19:36.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 -- 172.21.15.148:0/1156095007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 msgr2=0x7f16580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 --2- 172.21.15.148:0/1156095007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f16580051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f166400e3c0 tx=0x7f166400b040 comp rx=0 tx=0).stop 2023-10-10T20:19:36.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f1671a97700 1 -- 172.21.15.148:0/1156095007 reap_dead start 2023-10-10T20:19:36.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 -- 172.21.15.148:0/1156095007 shutdown_connections 2023-10-10T20:19:36.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 -- 172.21.15.148:0/1156095007 >> 172.21.15.148:0/1156095007 conn(0x7f166c068ce0 msgr2=0x7f166c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 -- 172.21.15.148:0/1156095007 shutdown_connections 2023-10-10T20:19:36.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.414+0000 7f16734fa700 1 -- 172.21.15.148:0/1156095007 wait complete. 2023-10-10T20:19:36.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.415+0000 7f16734fa700 1 Processor -- start 2023-10-10T20:19:36.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.415+0000 7f16734fa700 1 -- start start 2023-10-10T20:19:36.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c14ebf0 0x7f166c13f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c16a6a0 0x7f166c16ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c14fd70 con 0x7f1658004d50 2023-10-10T20:19:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1540c0 con 0x7f166c14ebf0 2023-10-10T20:19:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f16734fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c14f160 con 0x7f166c16a6a0 2023-10-10T20:19:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42604/0 (socket says 172.21.15.148:42604) 2023-10-10T20:19:36.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 -- 172.21.15.148:0/970354014 learned_addr learned my addr 172.21.15.148:0/970354014 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:36.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 -- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c16a6a0 msgr2=0x7f166c16ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c16a6a0 0x7f166c16ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1670a95700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c14ebf0 0x7f166c13f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 -- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c14ebf0 msgr2=0x7f166c13f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c14ebf0 0x7f166c13f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.416+0000 7f1671296700 1 -- 172.21.15.148:0/970354014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c14f440 con 0x7f1658004d50 2023-10-10T20:19:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f1671296700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1668011d70 tx=0x7f16680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800fcd0 con 0x7f1658004d50 2023-10-10T20:19:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800fe30 con 0x7f1658004d50 2023-10-10T20:19:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c14f720 con 0x7f1658004d50 2023-10-10T20:19:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668014670 con 0x7f1658004d50 2023-10-10T20:19:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.417+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c13fb20 con 0x7f1658004d50 2023-10-10T20:19:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.418+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c04c3c0 con 0x7f1658004d50 2023-10-10T20:19:36.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.418+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1668014aa0 con 0x7f1658004d50 2023-10-10T20:19:36.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.418+0000 7f16617fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 0x7f16500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.419+0000 7f1670a95700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 0x7f16500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.419+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1668018070 con 0x7f1658004d50 2023-10-10T20:19:36.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.419+0000 7f1670a95700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 0x7f16500693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f165c000c00 tx=0x7f165c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.423+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1668052400 con 0x7f1658004d50 2023-10-10T20:19:36.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.601+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f166c14feb0 con 0x7f1650066f30 2023-10-10T20:19:36.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.609+0000 7f16617fa700 1 -- 172.21.15.148:0/970354014 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260164 (secure 0 0 0) 0x7f165c004da0 con 0x7f1650066f30 2023-10-10T20:19:36.614 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:36.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 msgr2=0x7f16500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 0x7f16500693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f165c000c00 tx=0x7f165c009040 comp rx=0 tx=0).stop 2023-10-10T20:19:36.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 msgr2=0x7f166c151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1668011d70 tx=0x7f16680091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 shutdown_connections 2023-10-10T20:19:36.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1650066f30 0x7f16500693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c16a6a0 0x7f166c16ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1658004d50 0x7f166c151710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 --2- 172.21.15.148:0/970354014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c14ebf0 0x7f166c13f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 >> 172.21.15.148:0/970354014 conn(0x7f166c068ce0 msgr2=0x7f166c15aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 shutdown_connections 2023-10-10T20:19:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.612+0000 7f16734fa700 1 -- 172.21.15.148:0/970354014 wait complete. 2023-10-10T20:19:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:36.626 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-10T20:19:35.325242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560793,"statfs":{"total":1600000000,"available":1598252722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.843045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.500183+0000","last_change":"2023-10-10T20:19:23.499934+0000","last_active":"2023-10-10T20:19:23.500183+0000","last_peered":"2023-10-10T20:19:23.500183+0000","last_clean":"2023-10-10T20:19:23.500183+0000","last_became_active":"2023-10-10T20:19:23.499640+0000","last_became_peered":"2023-10-10T20:19:23.499640+0000","last_unstale":"2023-10-10T20:19:23.500183+0000","last_undegraded":"2023-10-10T20:19:23.500183+0000","last_fullsized":"2023-10-10T20:19:23.500183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488778+0000","last_change":"2023-10-10T20:19:23.488718+0000","last_active":"2023-10-10T20:19:23.488778+0000","last_peered":"2023-10-10T20:19:23.488778+0000","last_clean":"2023-10-10T20:19:23.488778+0000","last_became_active":"2023-10-10T20:19:23.488207+0000","last_became_peered":"2023-10-10T20:19:23.488207+0000","last_unstale":"2023-10-10T20:19:23.488778+0000","last_undegraded":"2023-10-10T20:19:23.488778+0000","last_fullsized":"2023-10-10T20:19:23.488778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488280+0000","last_change":"2023-10-10T20:19:23.487828+0000","last_active":"2023-10-10T20:19:23.488280+0000","last_peered":"2023-10-10T20:19:23.488280+0000","last_clean":"2023-10-10T20:19:23.488280+0000","last_became_active":"2023-10-10T20:19:23.487010+0000","last_became_peered":"2023-10-10T20:19:23.487010+0000","last_unstale":"2023-10-10T20:19:23.488280+0000","last_undegraded":"2023-10-10T20:19:23.488280+0000","last_fullsized":"2023-10-10T20:19:23.488280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492279+0000","last_change":"2023-10-10T20:19:23.492180+0000","last_active":"2023-10-10T20:19:23.492279+0000","last_peered":"2023-10-10T20:19:23.492279+0000","last_clean":"2023-10-10T20:19:23.492279+0000","last_became_active":"2023-10-10T20:19:23.491680+0000","last_became_peered":"2023-10-10T20:19:23.491680+0000","last_unstale":"2023-10-10T20:19:23.492279+0000","last_undegraded":"2023-10-10T20:19:23.492279+0000","last_fullsized":"2023-10-10T20:19:23.492279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492487+0000","last_change":"2023-10-10T20:19:23.492418+0000","last_active":"2023-10-10T20:19:23.492487+0000","last_peered":"2023-10-10T20:19:23.492487+0000","last_clean":"2023-10-10T20:19:23.492487+0000","last_became_active":"2023-10-10T20:19:23.492183+0000","last_became_peered":"2023-10-10T20:19:23.492183+0000","last_unstale":"2023-10-10T20:19:23.492487+0000","last_undegraded":"2023-10-10T20:19:23.492487+0000","last_fullsized":"2023-10-10T20:19:23.492487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494017+0000","last_change":"2023-10-10T20:19:23.493863+0000","last_active":"2023-10-10T20:19:23.494017+0000","last_peered":"2023-10-10T20:19:23.494017+0000","last_clean":"2023-10-10T20:19:23.494017+0000","last_became_active":"2023-10-10T20:19:23.492771+0000","last_became_peered":"2023-10-10T20:19:23.492771+0000","last_unstale":"2023-10-10T20:19:23.494017+0000","last_undegraded":"2023-10-10T20:19:23.494017+0000","last_fullsized":"2023-10-10T20:19:23.494017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493478+0000","last_change":"2023-10-10T20:19:23.493170+0000","last_active":"2023-10-10T20:19:23.493478+0000","last_peered":"2023-10-10T20:19:23.493478+0000","last_clean":"2023-10-10T20:19:23.493478+0000","last_became_active":"2023-10-10T20:19:23.492278+0000","last_became_peered":"2023-10-10T20:19:23.492278+0000","last_unstale":"2023-10-10T20:19:23.493478+0000","last_undegraded":"2023-10-10T20:19:23.493478+0000","last_fullsized":"2023-10-10T20:19:23.493478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485806+0000","last_change":"2023-10-10T20:19:23.485444+0000","last_active":"2023-10-10T20:19:23.485806+0000","last_peered":"2023-10-10T20:19:23.485806+0000","last_clean":"2023-10-10T20:19:23.485806+0000","last_became_active":"2023-10-10T20:19:23.485215+0000","last_became_peered":"2023-10-10T20:19:23.485215+0000","last_unstale":"2023-10-10T20:19:23.485806+0000","last_undegraded":"2023-10-10T20:19:23.485806+0000","last_fullsized":"2023-10-10T20:19:23.485806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495009+0000","last_change":"2023-10-10T20:19:23.494933+0000","last_active":"2023-10-10T20:19:23.495009+0000","last_peered":"2023-10-10T20:19:23.495009+0000","last_clean":"2023-10-10T20:19:23.495009+0000","last_became_active":"2023-10-10T20:19:23.494414+0000","last_became_peered":"2023-10-10T20:19:23.494414+0000","last_unstale":"2023-10-10T20:19:23.495009+0000","last_undegraded":"2023-10-10T20:19:23.495009+0000","last_fullsized":"2023-10-10T20:19:23.495009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499228+0000","last_change":"2023-10-10T20:19:23.499169+0000","last_active":"2023-10-10T20:19:23.499228+0000","last_peered":"2023-10-10T20:19:23.499228+0000","last_clean":"2023-10-10T20:19:23.499228+0000","last_became_active":"2023-10-10T20:19:23.498639+0000","last_became_peered":"2023-10-10T20:19:23.498639+0000","last_unstale":"2023-10-10T20:19:23.499228+0000","last_undegraded":"2023-10-10T20:19:23.499228+0000","last_fullsized":"2023-10-10T20:19:23.499228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498244+0000","last_change":"2023-10-10T20:19:23.497453+0000","last_active":"2023-10-10T20:19:23.498244+0000","last_peered":"2023-10-10T20:19:23.498244+0000","last_clean":"2023-10-10T20:19:23.498244+0000","last_became_active":"2023-10-10T20:19:23.497111+0000","last_became_peered":"2023-10-10T20:19:23.497111+0000","last_unstale":"2023-10-10T20:19:23.498244+0000","last_undegraded":"2023-10-10T20:19:23.498244+0000","last_fullsized":"2023-10-10T20:19:23.498244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493732+0000","last_change":"2023-10-10T20:19:23.493307+0000","last_active":"2023-10-10T20:19:23.493732+0000","last_peered":"2023-10-10T20:19:23.493732+0000","last_clean":"2023-10-10T20:19:23.493732+0000","last_became_active":"2023-10-10T20:19:23.492611+0000","last_became_peered":"2023-10-10T20:19:23.492611+0000","last_unstale":"2023-10-10T20:19:23.493732+0000","last_undegraded":"2023-10-10T20:19:23.493732+0000","last_fullsized":"2023-10-10T20:19:23.493732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495418+0000","last_change":"2023-10-10T20:19:23.495368+0000","last_active":"2023-10-10T20:19:23.495418+0000","last_peered":"2023-10-10T20:19:23.495418+0000","last_clean":"2023-10-10T20:19:23.495418+0000","last_became_active":"2023-10-10T20:19:23.495214+0000","last_became_peered":"2023-10-10T20:19:23.495214+0000","last_unstale":"2023-10-10T20:19:23.495418+0000","last_undegraded":"2023-10-10T20:19:23.495418+0000","last_fullsized":"2023-10-10T20:19:23.495418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483124+0000","last_change":"2023-10-10T20:19:23.483055+0000","last_active":"2023-10-10T20:19:23.483124+0000","last_peered":"2023-10-10T20:19:23.483124+0000","last_clean":"2023-10-10T20:19:23.483124+0000","last_became_active":"2023-10-10T20:19:23.482827+0000","last_became_peered":"2023-10-10T20:19:23.482827+0000","last_unstale":"2023-10-10T20:19:23.483124+0000","last_undegraded":"2023-10-10T20:19:23.483124+0000","last_fullsized":"2023-10-10T20:19:23.483124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492517+0000","last_change":"2023-10-10T20:19:23.492454+0000","last_active":"2023-10-10T20:19:23.492517+0000","last_peered":"2023-10-10T20:19:23.492517+0000","last_clean":"2023-10-10T20:19:23.492517+0000","last_became_active":"2023-10-10T20:19:23.492262+0000","last_became_peered":"2023-10-10T20:19:23.492262+0000","last_unstale":"2023-10-10T20:19:23.492517+0000","last_undegraded":"2023-10-10T20:19:23.492517+0000","last_fullsized":"2023-10-10T20:19:23.492517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493999+0000","last_change":"2023-10-10T20:19:23.493726+0000","last_active":"2023-10-10T20:19:23.493999+0000","last_peered":"2023-10-10T20:19:23.493999+0000","last_clean":"2023-10-10T20:19:23.493999+0000","last_became_active":"2023-10-10T20:19:23.492475+0000","last_became_peered":"2023-10-10T20:19:23.492475+0000","last_unstale":"2023-10-10T20:19:23.493999+0000","last_undegraded":"2023-10-10T20:19:23.493999+0000","last_fullsized":"2023-10-10T20:19:23.493999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495844+0000","last_change":"2023-10-10T20:19:23.495784+0000","last_active":"2023-10-10T20:19:23.495844+0000","last_peered":"2023-10-10T20:19:23.495844+0000","last_clean":"2023-10-10T20:19:23.495844+0000","last_became_active":"2023-10-10T20:19:23.495346+0000","last_became_peered":"2023-10-10T20:19:23.495346+0000","last_unstale":"2023-10-10T20:19:23.495844+0000","last_undegraded":"2023-10-10T20:19:23.495844+0000","last_fullsized":"2023-10-10T20:19:23.495844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494759+0000","last_change":"2023-10-10T20:19:23.494684+0000","last_active":"2023-10-10T20:19:23.494759+0000","last_peered":"2023-10-10T20:19:23.494759+0000","last_clean":"2023-10-10T20:19:23.494759+0000","last_became_active":"2023-10-10T20:19:23.494457+0000","last_became_peered":"2023-10-10T20:19:23.494457+0000","last_unstale":"2023-10-10T20:19:23.494759+0000","last_undegraded":"2023-10-10T20:19:23.494759+0000","last_fullsized":"2023-10-10T20:19:23.494759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491717+0000","last_change":"2023-10-10T20:19:23.491294+0000","last_active":"2023-10-10T20:19:23.491717+0000","last_peered":"2023-10-10T20:19:23.491717+0000","last_clean":"2023-10-10T20:19:23.491717+0000","last_became_active":"2023-10-10T20:19:23.491078+0000","last_became_peered":"2023-10-10T20:19:23.491078+0000","last_unstale":"2023-10-10T20:19:23.491717+0000","last_undegraded":"2023-10-10T20:19:23.491717+0000","last_fullsized":"2023-10-10T20:19:23.491717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505463+0000","last_change":"2023-10-10T20:19:22.491174+0000","last_active":"2023-10-10T20:19:23.505463+0000","last_peered":"2023-10-10T20:19:23.505463+0000","last_clean":"2023-10-10T20:19:23.505463+0000","last_became_active":"2023-10-10T20:19:22.490398+0000","last_became_peered":"2023-10-10T20:19:22.490398+0000","last_unstale":"2023-10-10T20:19:23.505463+0000","last_undegraded":"2023-10-10T20:19:23.505463+0000","last_fullsized":"2023-10-10T20:19:23.505463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490418+0000","last_change":"2023-10-10T20:19:23.490364+0000","last_active":"2023-10-10T20:19:23.490418+0000","last_peered":"2023-10-10T20:19:23.490418+0000","last_clean":"2023-10-10T20:19:23.490418+0000","last_became_active":"2023-10-10T20:19:23.490129+0000","last_became_peered":"2023-10-10T20:19:23.490129+0000","last_unstale":"2023-10-10T20:19:23.490418+0000","last_undegraded":"2023-10-10T20:19:23.490418+0000","last_fullsized":"2023-10-10T20:19:23.490418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505547+0000","last_change":"2023-10-10T20:19:22.490246+0000","last_active":"2023-10-10T20:19:23.505547+0000","last_peered":"2023-10-10T20:19:23.505547+0000","last_clean":"2023-10-10T20:19:23.505547+0000","last_became_active":"2023-10-10T20:19:22.490027+0000","last_became_peered":"2023-10-10T20:19:22.490027+0000","last_unstale":"2023-10-10T20:19:23.505547+0000","last_undegraded":"2023-10-10T20:19:23.505547+0000","last_fullsized":"2023-10-10T20:19:23.505547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488264+0000","last_change":"2023-10-10T20:19:23.487706+0000","last_active":"2023-10-10T20:19:23.488264+0000","last_peered":"2023-10-10T20:19:23.488264+0000","last_clean":"2023-10-10T20:19:23.488264+0000","last_became_active":"2023-10-10T20:19:23.486658+0000","last_became_peered":"2023-10-10T20:19:23.486658+0000","last_unstale":"2023-10-10T20:19:23.488264+0000","last_undegraded":"2023-10-10T20:19:23.488264+0000","last_fullsized":"2023-10-10T20:19:23.488264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505022+0000","last_change":"2023-10-10T20:19:22.491563+0000","last_active":"2023-10-10T20:19:23.505022+0000","last_peered":"2023-10-10T20:19:23.505022+0000","last_clean":"2023-10-10T20:19:23.505022+0000","last_became_active":"2023-10-10T20:19:22.491205+0000","last_became_peered":"2023-10-10T20:19:22.491205+0000","last_unstale":"2023-10-10T20:19:23.505022+0000","last_undegraded":"2023-10-10T20:19:23.505022+0000","last_fullsized":"2023-10-10T20:19:23.505022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495250+0000","last_change":"2023-10-10T20:19:23.495186+0000","last_active":"2023-10-10T20:19:23.495250+0000","last_peered":"2023-10-10T20:19:23.495250+0000","last_clean":"2023-10-10T20:19:23.495250+0000","last_became_active":"2023-10-10T20:19:23.494935+0000","last_became_peered":"2023-10-10T20:19:23.494935+0000","last_unstale":"2023-10-10T20:19:23.495250+0000","last_undegraded":"2023-10-10T20:19:23.495250+0000","last_fullsized":"2023-10-10T20:19:23.495250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503478+0000","last_change":"2023-10-10T20:19:22.493193+0000","last_active":"2023-10-10T20:19:23.503478+0000","last_peered":"2023-10-10T20:19:23.503478+0000","last_clean":"2023-10-10T20:19:23.503478+0000","last_became_active":"2023-10-10T20:19:22.492979+0000","last_became_peered":"2023-10-10T20:19:22.492979+0000","last_unstale":"2023-10-10T20:19:23.503478+0000","last_undegraded":"2023-10-10T20:19:23.503478+0000","last_fullsized":"2023-10-10T20:19:23.503478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487270+0000","last_change":"2023-10-10T20:19:23.487131+0000","last_active":"2023-10-10T20:19:23.487270+0000","last_peered":"2023-10-10T20:19:23.487270+0000","last_clean":"2023-10-10T20:19:23.487270+0000","last_became_active":"2023-10-10T20:19:23.486894+0000","last_became_peered":"2023-10-10T20:19:23.486894+0000","last_unstale":"2023-10-10T20:19:23.487270+0000","last_undegraded":"2023-10-10T20:19:23.487270+0000","last_fullsized":"2023-10-10T20:19:23.487270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505011+0000","last_change":"2023-10-10T20:19:22.491757+0000","last_active":"2023-10-10T20:19:23.505011+0000","last_peered":"2023-10-10T20:19:23.505011+0000","last_clean":"2023-10-10T20:19:23.505011+0000","last_became_active":"2023-10-10T20:19:22.491535+0000","last_became_peered":"2023-10-10T20:19:22.491535+0000","last_unstale":"2023-10-10T20:19:23.505011+0000","last_undegraded":"2023-10-10T20:19:23.505011+0000","last_fullsized":"2023-10-10T20:19:23.505011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489517+0000","last_change":"2023-10-10T20:19:23.488600+0000","last_active":"2023-10-10T20:19:23.489517+0000","last_peered":"2023-10-10T20:19:23.489517+0000","last_clean":"2023-10-10T20:19:23.489517+0000","last_became_active":"2023-10-10T20:19:23.487824+0000","last_became_peered":"2023-10-10T20:19:23.487824+0000","last_unstale":"2023-10-10T20:19:23.489517+0000","last_undegraded":"2023-10-10T20:19:23.489517+0000","last_fullsized":"2023-10-10T20:19:23.489517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:07.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.502781+0000","last_change":"2023-10-10T20:19:22.486217+0000","last_active":"2023-10-10T20:19:23.502781+0000","last_peered":"2023-10-10T20:19:23.502781+0000","last_clean":"2023-10-10T20:19:23.502781+0000","last_became_active":"2023-10-10T20:19:22.484804+0000","last_became_peered":"2023-10-10T20:19:22.484804+0000","last_unstale":"2023-10-10T20:19:23.502781+0000","last_undegraded":"2023-10-10T20:19:23.502781+0000","last_fullsized":"2023-10-10T20:19:23.502781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485364+0000","last_change":"2023-10-10T20:19:23.485292+0000","last_active":"2023-10-10T20:19:23.485364+0000","last_peered":"2023-10-10T20:19:23.485364+0000","last_clean":"2023-10-10T20:19:23.485364+0000","last_became_active":"2023-10-10T20:19:23.485060+0000","last_became_peered":"2023-10-10T20:19:23.485060+0000","last_unstale":"2023-10-10T20:19:23.485364+0000","last_undegraded":"2023-10-10T20:19:23.485364+0000","last_fullsized":"2023-10-10T20:19:23.485364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:16.611774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482268+0000","last_change":"2023-10-10T20:19:22.489127+0000","last_active":"2023-10-10T20:19:23.482268+0000","last_peered":"2023-10-10T20:19:23.482268+0000","last_clean":"2023-10-10T20:19:23.482268+0000","last_became_active":"2023-10-10T20:19:22.488683+0000","last_became_peered":"2023-10-10T20:19:22.488683+0000","last_unstale":"2023-10-10T20:19:23.482268+0000","last_undegraded":"2023-10-10T20:19:23.482268+0000","last_fullsized":"2023-10-10T20:19:23.482268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487104+0000","last_change":"2023-10-10T20:19:23.486567+0000","last_active":"2023-10-10T20:19:23.487104+0000","last_peered":"2023-10-10T20:19:23.487104+0000","last_clean":"2023-10-10T20:19:23.487104+0000","last_became_active":"2023-10-10T20:19:23.486329+0000","last_became_peered":"2023-10-10T20:19:23.486329+0000","last_unstale":"2023-10-10T20:19:23.487104+0000","last_undegraded":"2023-10-10T20:19:23.487104+0000","last_fullsized":"2023-10-10T20:19:23.487104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483483+0000","last_change":"2023-10-10T20:19:22.485105+0000","last_active":"2023-10-10T20:19:23.483483+0000","last_peered":"2023-10-10T20:19:23.483483+0000","last_clean":"2023-10-10T20:19:23.483483+0000","last_became_active":"2023-10-10T20:19:22.484537+0000","last_became_peered":"2023-10-10T20:19:22.484537+0000","last_unstale":"2023-10-10T20:19:23.483483+0000","last_undegraded":"2023-10-10T20:19:23.483483+0000","last_fullsized":"2023-10-10T20:19:23.483483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483939+0000","last_change":"2023-10-10T20:19:23.483701+0000","last_active":"2023-10-10T20:19:23.483939+0000","last_peered":"2023-10-10T20:19:23.483939+0000","last_clean":"2023-10-10T20:19:23.483939+0000","last_became_active":"2023-10-10T20:19:23.483485+0000","last_became_peered":"2023-10-10T20:19:23.483485+0000","last_unstale":"2023-10-10T20:19:23.483939+0000","last_undegraded":"2023-10-10T20:19:23.483939+0000","last_fullsized":"2023-10-10T20:19:23.483939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486842+0000","last_change":"2023-10-10T20:19:22.486912+0000","last_active":"2023-10-10T20:19:23.486842+0000","last_peered":"2023-10-10T20:19:23.486842+0000","last_clean":"2023-10-10T20:19:23.486842+0000","last_became_active":"2023-10-10T20:19:22.486662+0000","last_became_peered":"2023-10-10T20:19:22.486662+0000","last_unstale":"2023-10-10T20:19:23.486842+0000","last_undegraded":"2023-10-10T20:19:23.486842+0000","last_fullsized":"2023-10-10T20:19:23.486842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492299+0000","last_change":"2023-10-10T20:19:23.492209+0000","last_active":"2023-10-10T20:19:23.492299+0000","last_peered":"2023-10-10T20:19:23.492299+0000","last_clean":"2023-10-10T20:19:23.492299+0000","last_became_active":"2023-10-10T20:19:23.491975+0000","last_became_peered":"2023-10-10T20:19:23.491975+0000","last_unstale":"2023-10-10T20:19:23.492299+0000","last_undegraded":"2023-10-10T20:19:23.492299+0000","last_fullsized":"2023-10-10T20:19:23.492299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485905+0000","last_change":"2023-10-10T20:19:22.490636+0000","last_active":"2023-10-10T20:19:23.485905+0000","last_peered":"2023-10-10T20:19:23.485905+0000","last_clean":"2023-10-10T20:19:23.485905+0000","last_became_active":"2023-10-10T20:19:22.490369+0000","last_became_peered":"2023-10-10T20:19:22.490369+0000","last_unstale":"2023-10-10T20:19:23.485905+0000","last_undegraded":"2023-10-10T20:19:23.485905+0000","last_fullsized":"2023-10-10T20:19:23.485905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490958+0000","last_change":"2023-10-10T20:19:23.490908+0000","last_active":"2023-10-10T20:19:23.490958+0000","last_peered":"2023-10-10T20:19:23.490958+0000","last_clean":"2023-10-10T20:19:23.490958+0000","last_became_active":"2023-10-10T20:19:23.490692+0000","last_became_peered":"2023-10-10T20:19:23.490692+0000","last_unstale":"2023-10-10T20:19:23.490958+0000","last_undegraded":"2023-10-10T20:19:23.490958+0000","last_fullsized":"2023-10-10T20:19:23.490958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503581+0000","last_change":"2023-10-10T20:19:22.487531+0000","last_active":"2023-10-10T20:19:23.503581+0000","last_peered":"2023-10-10T20:19:23.503581+0000","last_clean":"2023-10-10T20:19:23.503581+0000","last_became_active":"2023-10-10T20:19:22.487040+0000","last_became_peered":"2023-10-10T20:19:22.487040+0000","last_unstale":"2023-10-10T20:19:23.503581+0000","last_undegraded":"2023-10-10T20:19:23.503581+0000","last_fullsized":"2023-10-10T20:19:23.503581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493693+0000","last_change":"2023-10-10T20:19:23.493005+0000","last_active":"2023-10-10T20:19:23.493693+0000","last_peered":"2023-10-10T20:19:23.493693+0000","last_clean":"2023-10-10T20:19:23.493693+0000","last_became_active":"2023-10-10T20:19:23.492752+0000","last_became_peered":"2023-10-10T20:19:23.492752+0000","last_unstale":"2023-10-10T20:19:23.493693+0000","last_undegraded":"2023-10-10T20:19:23.493693+0000","last_fullsized":"2023-10-10T20:19:23.493693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.506581+0000","last_change":"2023-10-10T20:19:22.491546+0000","last_active":"2023-10-10T20:19:23.506581+0000","last_peered":"2023-10-10T20:19:23.506581+0000","last_clean":"2023-10-10T20:19:23.506581+0000","last_became_active":"2023-10-10T20:19:22.491157+0000","last_became_peered":"2023-10-10T20:19:22.491157+0000","last_unstale":"2023-10-10T20:19:23.506581+0000","last_undegraded":"2023-10-10T20:19:23.506581+0000","last_fullsized":"2023-10-10T20:19:23.506581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490504+0000","last_change":"2023-10-10T20:19:23.490357+0000","last_active":"2023-10-10T20:19:23.490504+0000","last_peered":"2023-10-10T20:19:23.490504+0000","last_clean":"2023-10-10T20:19:23.490504+0000","last_became_active":"2023-10-10T20:19:23.490094+0000","last_became_peered":"2023-10-10T20:19:23.490094+0000","last_unstale":"2023-10-10T20:19:23.490504+0000","last_undegraded":"2023-10-10T20:19:23.490504+0000","last_fullsized":"2023-10-10T20:19:23.490504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504122+0000","last_change":"2023-10-10T20:19:22.484098+0000","last_active":"2023-10-10T20:19:23.504122+0000","last_peered":"2023-10-10T20:19:23.504122+0000","last_clean":"2023-10-10T20:19:23.504122+0000","last_became_active":"2023-10-10T20:19:22.482738+0000","last_became_peered":"2023-10-10T20:19:22.482738+0000","last_unstale":"2023-10-10T20:19:23.504122+0000","last_undegraded":"2023-10-10T20:19:23.504122+0000","last_fullsized":"2023-10-10T20:19:23.504122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491923+0000","last_change":"2023-10-10T20:19:23.491873+0000","last_active":"2023-10-10T20:19:23.491923+0000","last_peered":"2023-10-10T20:19:23.491923+0000","last_clean":"2023-10-10T20:19:23.491923+0000","last_became_active":"2023-10-10T20:19:23.491532+0000","last_became_peered":"2023-10-10T20:19:23.491532+0000","last_unstale":"2023-10-10T20:19:23.491923+0000","last_undegraded":"2023-10-10T20:19:23.491923+0000","last_fullsized":"2023-10-10T20:19:23.491923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488825+0000","last_change":"2023-10-10T20:19:22.491830+0000","last_active":"2023-10-10T20:19:23.488825+0000","last_peered":"2023-10-10T20:19:23.488825+0000","last_clean":"2023-10-10T20:19:23.488825+0000","last_became_active":"2023-10-10T20:19:22.489289+0000","last_became_peered":"2023-10-10T20:19:22.489289+0000","last_unstale":"2023-10-10T20:19:23.488825+0000","last_undegraded":"2023-10-10T20:19:23.488825+0000","last_fullsized":"2023-10-10T20:19:23.488825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497717+0000","last_change":"2023-10-10T20:19:23.497580+0000","last_active":"2023-10-10T20:19:23.497717+0000","last_peered":"2023-10-10T20:19:23.497717+0000","last_clean":"2023-10-10T20:19:23.497717+0000","last_became_active":"2023-10-10T20:19:23.497043+0000","last_became_peered":"2023-10-10T20:19:23.497043+0000","last_unstale":"2023-10-10T20:19:23.497717+0000","last_undegraded":"2023-10-10T20:19:23.497717+0000","last_fullsized":"2023-10-10T20:19:23.497717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486275+0000","last_change":"2023-10-10T20:19:22.487157+0000","last_active":"2023-10-10T20:19:23.486275+0000","last_peered":"2023-10-10T20:19:23.486275+0000","last_clean":"2023-10-10T20:19:23.486275+0000","last_became_active":"2023-10-10T20:19:22.486367+0000","last_became_peered":"2023-10-10T20:19:22.486367+0000","last_unstale":"2023-10-10T20:19:23.486275+0000","last_undegraded":"2023-10-10T20:19:23.486275+0000","last_fullsized":"2023-10-10T20:19:23.486275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494474+0000","last_change":"2023-10-10T20:19:23.494326+0000","last_active":"2023-10-10T20:19:23.494474+0000","last_peered":"2023-10-10T20:19:23.494474+0000","last_clean":"2023-10-10T20:19:23.494474+0000","last_became_active":"2023-10-10T20:19:23.494043+0000","last_became_peered":"2023-10-10T20:19:23.494043+0000","last_unstale":"2023-10-10T20:19:23.494474+0000","last_undegraded":"2023-10-10T20:19:23.494474+0000","last_fullsized":"2023-10-10T20:19:23.494474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505618+0000","last_change":"2023-10-10T20:19:22.484580+0000","last_active":"2023-10-10T20:19:23.505618+0000","last_peered":"2023-10-10T20:19:23.505618+0000","last_clean":"2023-10-10T20:19:23.505618+0000","last_became_active":"2023-10-10T20:19:22.484287+0000","last_became_peered":"2023-10-10T20:19:22.484287+0000","last_unstale":"2023-10-10T20:19:23.505618+0000","last_undegraded":"2023-10-10T20:19:23.505618+0000","last_fullsized":"2023-10-10T20:19:23.505618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497360+0000","last_change":"2023-10-10T20:19:23.497292+0000","last_active":"2023-10-10T20:19:23.497360+0000","last_peered":"2023-10-10T20:19:23.497360+0000","last_clean":"2023-10-10T20:19:23.497360+0000","last_became_active":"2023-10-10T20:19:23.497063+0000","last_became_peered":"2023-10-10T20:19:23.497063+0000","last_unstale":"2023-10-10T20:19:23.497360+0000","last_undegraded":"2023-10-10T20:19:23.497360+0000","last_fullsized":"2023-10-10T20:19:23.497360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505930+0000","last_change":"2023-10-10T20:19:22.481673+0000","last_active":"2023-10-10T20:19:23.505930+0000","last_peered":"2023-10-10T20:19:23.505930+0000","last_clean":"2023-10-10T20:19:23.505930+0000","last_became_active":"2023-10-10T20:19:22.481500+0000","last_became_peered":"2023-10-10T20:19:22.481500+0000","last_unstale":"2023-10-10T20:19:23.505930+0000","last_undegraded":"2023-10-10T20:19:23.505930+0000","last_fullsized":"2023-10-10T20:19:23.505930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497808+0000","last_change":"2023-10-10T20:19:23.497718+0000","last_active":"2023-10-10T20:19:23.497808+0000","last_peered":"2023-10-10T20:19:23.497808+0000","last_clean":"2023-10-10T20:19:23.497808+0000","last_became_active":"2023-10-10T20:19:23.497352+0000","last_became_peered":"2023-10-10T20:19:23.497352+0000","last_unstale":"2023-10-10T20:19:23.497808+0000","last_undegraded":"2023-10-10T20:19:23.497808+0000","last_fullsized":"2023-10-10T20:19:23.497808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481756+0000","last_change":"2023-10-10T20:19:22.486438+0000","last_active":"2023-10-10T20:19:23.481756+0000","last_peered":"2023-10-10T20:19:23.481756+0000","last_clean":"2023-10-10T20:19:23.481756+0000","last_became_active":"2023-10-10T20:19:22.486219+0000","last_became_peered":"2023-10-10T20:19:22.486219+0000","last_unstale":"2023-10-10T20:19:23.481756+0000","last_undegraded":"2023-10-10T20:19:23.481756+0000","last_fullsized":"2023-10-10T20:19:23.481756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490783+0000","last_change":"2023-10-10T20:19:23.490700+0000","last_active":"2023-10-10T20:19:23.490783+0000","last_peered":"2023-10-10T20:19:23.490783+0000","last_clean":"2023-10-10T20:19:23.490783+0000","last_became_active":"2023-10-10T20:19:23.490468+0000","last_became_peered":"2023-10-10T20:19:23.490468+0000","last_unstale":"2023-10-10T20:19:23.490783+0000","last_undegraded":"2023-10-10T20:19:23.490783+0000","last_fullsized":"2023-10-10T20:19:23.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486018+0000","last_change":"2023-10-10T20:19:22.485041+0000","last_active":"2023-10-10T20:19:23.486018+0000","last_peered":"2023-10-10T20:19:23.486018+0000","last_clean":"2023-10-10T20:19:23.486018+0000","last_became_active":"2023-10-10T20:19:22.484798+0000","last_became_peered":"2023-10-10T20:19:22.484798+0000","last_unstale":"2023-10-10T20:19:23.486018+0000","last_undegraded":"2023-10-10T20:19:23.486018+0000","last_fullsized":"2023-10-10T20:19:23.486018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489489+0000","last_change":"2023-10-10T20:19:23.487628+0000","last_active":"2023-10-10T20:19:23.489489+0000","last_peered":"2023-10-10T20:19:23.489489+0000","last_clean":"2023-10-10T20:19:23.489489+0000","last_became_active":"2023-10-10T20:19:23.486750+0000","last_became_peered":"2023-10-10T20:19:23.486750+0000","last_unstale":"2023-10-10T20:19:23.489489+0000","last_undegraded":"2023-10-10T20:19:23.489489+0000","last_fullsized":"2023-10-10T20:19:23.489489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:28.087913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487677+0000","last_change":"2023-10-10T20:19:23.486218+0000","last_active":"2023-10-10T20:19:23.487677+0000","last_peered":"2023-10-10T20:19:23.487677+0000","last_clean":"2023-10-10T20:19:23.487677+0000","last_became_active":"2023-10-10T20:19:23.485999+0000","last_became_peered":"2023-10-10T20:19:23.485999+0000","last_unstale":"2023-10-10T20:19:23.487677+0000","last_undegraded":"2023-10-10T20:19:23.487677+0000","last_fullsized":"2023-10-10T20:19:23.487677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.484120+0000","last_change":"2023-10-10T20:19:22.489199+0000","last_active":"2023-10-10T20:19:23.484120+0000","last_peered":"2023-10-10T20:19:23.484120+0000","last_clean":"2023-10-10T20:19:23.484120+0000","last_became_active":"2023-10-10T20:19:22.488931+0000","last_became_peered":"2023-10-10T20:19:22.488931+0000","last_unstale":"2023-10-10T20:19:23.484120+0000","last_undegraded":"2023-10-10T20:19:23.484120+0000","last_fullsized":"2023-10-10T20:19:23.484120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489524+0000","last_change":"2023-10-10T20:19:23.489413+0000","last_active":"2023-10-10T20:19:23.489524+0000","last_peered":"2023-10-10T20:19:23.489524+0000","last_clean":"2023-10-10T20:19:23.489524+0000","last_became_active":"2023-10-10T20:19:23.489083+0000","last_became_peered":"2023-10-10T20:19:23.489083+0000","last_unstale":"2023-10-10T20:19:23.489524+0000","last_undegraded":"2023-10-10T20:19:23.489524+0000","last_fullsized":"2023-10-10T20:19:23.489524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:22.491979+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:22.489613+0000","last_became_peered":"2023-10-10T20:19:22.489613+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486860+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:23.486860+0000","last_peered":"2023-10-10T20:19:23.486860+0000","last_clean":"2023-10-10T20:19:23.486860+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:23.486860+0000","last_undegraded":"2023-10-10T20:19:23.486860+0000","last_fullsized":"2023-10-10T20:19:23.486860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488300+0000","last_change":"2023-10-10T20:19:23.487957+0000","last_active":"2023-10-10T20:19:23.488300+0000","last_peered":"2023-10-10T20:19:23.488300+0000","last_clean":"2023-10-10T20:19:23.488300+0000","last_became_active":"2023-10-10T20:19:23.487318+0000","last_became_peered":"2023-10-10T20:19:23.487318+0000","last_unstale":"2023-10-10T20:19:23.488300+0000","last_undegraded":"2023-10-10T20:19:23.488300+0000","last_fullsized":"2023-10-10T20:19:23.488300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486185+0000","last_change":"2023-10-10T20:19:22.492546+0000","last_active":"2023-10-10T20:19:23.486185+0000","last_peered":"2023-10-10T20:19:23.486185+0000","last_clean":"2023-10-10T20:19:23.486185+0000","last_became_active":"2023-10-10T20:19:22.492293+0000","last_became_peered":"2023-10-10T20:19:22.492293+0000","last_unstale":"2023-10-10T20:19:23.486185+0000","last_undegraded":"2023-10-10T20:19:23.486185+0000","last_fullsized":"2023-10-10T20:19:23.486185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486269+0000","last_change":"2023-10-10T20:19:22.490666+0000","last_active":"2023-10-10T20:19:23.486269+0000","last_peered":"2023-10-10T20:19:23.486269+0000","last_clean":"2023-10-10T20:19:23.486269+0000","last_became_active":"2023-10-10T20:19:22.490355+0000","last_became_peered":"2023-10-10T20:19:22.490355+0000","last_unstale":"2023-10-10T20:19:23.486269+0000","last_undegraded":"2023-10-10T20:19:23.486269+0000","last_fullsized":"2023-10-10T20:19:23.486269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487695+0000","last_change":"2023-10-10T20:19:23.486743+0000","last_active":"2023-10-10T20:19:23.487695+0000","last_peered":"2023-10-10T20:19:23.487695+0000","last_clean":"2023-10-10T20:19:23.487695+0000","last_became_active":"2023-10-10T20:19:23.486526+0000","last_became_peered":"2023-10-10T20:19:23.486526+0000","last_unstale":"2023-10-10T20:19:23.487695+0000","last_undegraded":"2023-10-10T20:19:23.487695+0000","last_fullsized":"2023-10-10T20:19:23.487695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-10T20:19:24.482879+0000","last_change":"2023-10-10T20:19:22.487657+0000","last_active":"2023-10-10T20:19:24.482879+0000","last_peered":"2023-10-10T20:19:24.482879+0000","last_clean":"2023-10-10T20:19:24.482879+0000","last_became_active":"2023-10-10T20:19:22.487329+0000","last_became_peered":"2023-10-10T20:19:22.487329+0000","last_unstale":"2023-10-10T20:19:24.482879+0000","last_undegraded":"2023-10-10T20:19:24.482879+0000","last_fullsized":"2023-10-10T20:19:24.482879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:05.420138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491172+0000","last_change":"2023-10-10T20:19:23.491123+0000","last_active":"2023-10-10T20:19:23.491172+0000","last_peered":"2023-10-10T20:19:23.491172+0000","last_clean":"2023-10-10T20:19:23.491172+0000","last_became_active":"2023-10-10T20:19:23.490946+0000","last_became_peered":"2023-10-10T20:19:23.490946+0000","last_unstale":"2023-10-10T20:19:23.491172+0000","last_undegraded":"2023-10-10T20:19:23.491172+0000","last_fullsized":"2023-10-10T20:19:23.491172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505978+0000","last_change":"2023-10-10T20:19:22.491522+0000","last_active":"2023-10-10T20:19:23.505978+0000","last_peered":"2023-10-10T20:19:23.505978+0000","last_clean":"2023-10-10T20:19:23.505978+0000","last_became_active":"2023-10-10T20:19:22.491251+0000","last_became_peered":"2023-10-10T20:19:22.491251+0000","last_unstale":"2023-10-10T20:19:23.505978+0000","last_undegraded":"2023-10-10T20:19:23.505978+0000","last_fullsized":"2023-10-10T20:19:23.505978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488247+0000","last_change":"2023-10-10T20:19:23.487572+0000","last_active":"2023-10-10T20:19:23.488247+0000","last_peered":"2023-10-10T20:19:23.488247+0000","last_clean":"2023-10-10T20:19:23.488247+0000","last_became_active":"2023-10-10T20:19:23.486362+0000","last_became_peered":"2023-10-10T20:19:23.486362+0000","last_unstale":"2023-10-10T20:19:23.488247+0000","last_undegraded":"2023-10-10T20:19:23.488247+0000","last_fullsized":"2023-10-10T20:19:23.488247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:04.867879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489482+0000","last_change":"2023-10-10T20:19:22.490501+0000","last_active":"2023-10-10T20:19:23.489482+0000","last_peered":"2023-10-10T20:19:23.489482+0000","last_clean":"2023-10-10T20:19:23.489482+0000","last_became_active":"2023-10-10T20:19:22.490208+0000","last_became_peered":"2023-10-10T20:19:22.490208+0000","last_unstale":"2023-10-10T20:19:23.489482+0000","last_undegraded":"2023-10-10T20:19:23.489482+0000","last_fullsized":"2023-10-10T20:19:23.489482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.496090+0000","last_change":"2023-10-10T20:19:23.496016+0000","last_active":"2023-10-10T20:19:23.496090+0000","last_peered":"2023-10-10T20:19:23.496090+0000","last_clean":"2023-10-10T20:19:23.496090+0000","last_became_active":"2023-10-10T20:19:23.495512+0000","last_became_peered":"2023-10-10T20:19:23.495512+0000","last_unstale":"2023-10-10T20:19:23.496090+0000","last_undegraded":"2023-10-10T20:19:23.496090+0000","last_fullsized":"2023-10-10T20:19:23.496090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:05.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481809+0000","last_change":"2023-10-10T20:19:22.490184+0000","last_active":"2023-10-10T20:19:23.481809+0000","last_peered":"2023-10-10T20:19:23.481809+0000","last_clean":"2023-10-10T20:19:23.481809+0000","last_became_active":"2023-10-10T20:19:22.490057+0000","last_became_peered":"2023-10-10T20:19:22.490057+0000","last_unstale":"2023-10-10T20:19:23.481809+0000","last_undegraded":"2023-10-10T20:19:23.481809+0000","last_fullsized":"2023-10-10T20:19:23.481809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487155+0000","last_change":"2023-10-10T20:19:23.487003+0000","last_active":"2023-10-10T20:19:23.487155+0000","last_peered":"2023-10-10T20:19:23.487155+0000","last_clean":"2023-10-10T20:19:23.487155+0000","last_became_active":"2023-10-10T20:19:23.486789+0000","last_became_peered":"2023-10-10T20:19:23.486789+0000","last_unstale":"2023-10-10T20:19:23.487155+0000","last_undegraded":"2023-10-10T20:19:23.487155+0000","last_fullsized":"2023-10-10T20:19:23.487155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482124+0000","last_change":"2023-10-10T20:19:22.491852+0000","last_active":"2023-10-10T20:19:23.482124+0000","last_peered":"2023-10-10T20:19:23.482124+0000","last_clean":"2023-10-10T20:19:23.482124+0000","last_became_active":"2023-10-10T20:19:22.491569+0000","last_became_peered":"2023-10-10T20:19:22.491569+0000","last_unstale":"2023-10-10T20:19:23.482124+0000","last_undegraded":"2023-10-10T20:19:23.482124+0000","last_fullsized":"2023-10-10T20:19:23.482124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488335+0000","last_change":"2023-10-10T20:19:23.487466+0000","last_active":"2023-10-10T20:19:23.488335+0000","last_peered":"2023-10-10T20:19:23.488335+0000","last_clean":"2023-10-10T20:19:23.488335+0000","last_became_active":"2023-10-10T20:19:23.487241+0000","last_became_peered":"2023-10-10T20:19:23.487241+0000","last_unstale":"2023-10-10T20:19:23.488335+0000","last_undegraded":"2023-10-10T20:19:23.488335+0000","last_fullsized":"2023-10-10T20:19:23.488335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503296+0000","last_change":"2023-10-10T20:19:22.490823+0000","last_active":"2023-10-10T20:19:23.503296+0000","last_peered":"2023-10-10T20:19:23.503296+0000","last_clean":"2023-10-10T20:19:23.503296+0000","last_became_active":"2023-10-10T20:19:22.490137+0000","last_became_peered":"2023-10-10T20:19:22.490137+0000","last_unstale":"2023-10-10T20:19:23.503296+0000","last_undegraded":"2023-10-10T20:19:23.503296+0000","last_fullsized":"2023-10-10T20:19:23.503296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495850+0000","last_change":"2023-10-10T20:19:23.495686+0000","last_active":"2023-10-10T20:19:23.495850+0000","last_peered":"2023-10-10T20:19:23.495850+0000","last_clean":"2023-10-10T20:19:23.495850+0000","last_became_active":"2023-10-10T20:19:23.495454+0000","last_became_peered":"2023-10-10T20:19:23.495454+0000","last_unstale":"2023-10-10T20:19:23.495850+0000","last_undegraded":"2023-10-10T20:19:23.495850+0000","last_fullsized":"2023-10-10T20:19:23.495850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483607+0000","last_change":"2023-10-10T20:19:22.485497+0000","last_active":"2023-10-10T20:19:23.483607+0000","last_peered":"2023-10-10T20:19:23.483607+0000","last_clean":"2023-10-10T20:19:23.483607+0000","last_became_active":"2023-10-10T20:19:22.485276+0000","last_became_peered":"2023-10-10T20:19:22.485276+0000","last_unstale":"2023-10-10T20:19:23.483607+0000","last_undegraded":"2023-10-10T20:19:23.483607+0000","last_fullsized":"2023-10-10T20:19:23.483607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495682+0000","last_change":"2023-10-10T20:19:23.495616+0000","last_active":"2023-10-10T20:19:23.495682+0000","last_peered":"2023-10-10T20:19:23.495682+0000","last_clean":"2023-10-10T20:19:23.495682+0000","last_became_active":"2023-10-10T20:19:23.495385+0000","last_became_peered":"2023-10-10T20:19:23.495385+0000","last_unstale":"2023-10-10T20:19:23.495682+0000","last_undegraded":"2023-10-10T20:19:23.495682+0000","last_fullsized":"2023-10-10T20:19:23.495682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504015+0000","last_change":"2023-10-10T20:19:22.483831+0000","last_active":"2023-10-10T20:19:23.504015+0000","last_peered":"2023-10-10T20:19:23.504015+0000","last_clean":"2023-10-10T20:19:23.504015+0000","last_became_active":"2023-10-10T20:19:22.483604+0000","last_became_peered":"2023-10-10T20:19:22.483604+0000","last_unstale":"2023-10-10T20:19:23.504015+0000","last_undegraded":"2023-10-10T20:19:23.504015+0000","last_fullsized":"2023-10-10T20:19:23.504015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489338+0000","last_change":"2023-10-10T20:19:23.488221+0000","last_active":"2023-10-10T20:19:23.489338+0000","last_peered":"2023-10-10T20:19:23.489338+0000","last_clean":"2023-10-10T20:19:23.489338+0000","last_became_active":"2023-10-10T20:19:23.487992+0000","last_became_peered":"2023-10-10T20:19:23.487992+0000","last_unstale":"2023-10-10T20:19:23.489338+0000","last_undegraded":"2023-10-10T20:19:23.489338+0000","last_fullsized":"2023-10-10T20:19:23.489338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504646+0000","last_change":"2023-10-10T20:19:22.488044+0000","last_active":"2023-10-10T20:19:23.504646+0000","last_peered":"2023-10-10T20:19:23.504646+0000","last_clean":"2023-10-10T20:19:23.504646+0000","last_became_active":"2023-10-10T20:19:22.487823+0000","last_became_peered":"2023-10-10T20:19:22.487823+0000","last_unstale":"2023-10-10T20:19:23.504646+0000","last_undegraded":"2023-10-10T20:19:23.504646+0000","last_fullsized":"2023-10-10T20:19:23.504646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494991+0000","last_change":"2023-10-10T20:19:23.494793+0000","last_active":"2023-10-10T20:19:23.494991+0000","last_peered":"2023-10-10T20:19:23.494991+0000","last_clean":"2023-10-10T20:19:23.494991+0000","last_became_active":"2023-10-10T20:19:23.494129+0000","last_became_peered":"2023-10-10T20:19:23.494129+0000","last_unstale":"2023-10-10T20:19:23.494991+0000","last_undegraded":"2023-10-10T20:19:23.494991+0000","last_fullsized":"2023-10-10T20:19:23.494991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493767+0000","last_change":"2023-10-10T20:19:23.489565+0000","last_active":"2023-10-10T20:19:23.493767+0000","last_peered":"2023-10-10T20:19:23.493767+0000","last_clean":"2023-10-10T20:19:23.493767+0000","last_became_active":"2023-10-10T20:19:23.489263+0000","last_became_peered":"2023-10-10T20:19:23.489263+0000","last_unstale":"2023-10-10T20:19:23.493767+0000","last_undegraded":"2023-10-10T20:19:23.493767+0000","last_fullsized":"2023-10-10T20:19:23.493767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490726+0000","last_change":"2023-10-10T20:19:23.490528+0000","last_active":"2023-10-10T20:19:23.490726+0000","last_peered":"2023-10-10T20:19:23.490726+0000","last_clean":"2023-10-10T20:19:23.490726+0000","last_became_active":"2023-10-10T20:19:23.490284+0000","last_became_peered":"2023-10-10T20:19:23.490284+0000","last_unstale":"2023-10-10T20:19:23.490726+0000","last_undegraded":"2023-10-10T20:19:23.490726+0000","last_fullsized":"2023-10-10T20:19:23.490726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490090+0000","last_change":"2023-10-10T20:19:23.489820+0000","last_active":"2023-10-10T20:19:23.490090+0000","last_peered":"2023-10-10T20:19:23.490090+0000","last_clean":"2023-10-10T20:19:23.490090+0000","last_became_active":"2023-10-10T20:19:23.489559+0000","last_became_peered":"2023-10-10T20:19:23.489559+0000","last_unstale":"2023-10-10T20:19:23.490090+0000","last_undegraded":"2023-10-10T20:19:23.490090+0000","last_fullsized":"2023-10-10T20:19:23.490090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485528+0000","last_change":"2023-10-10T20:19:23.485467+0000","last_active":"2023-10-10T20:19:23.485528+0000","last_peered":"2023-10-10T20:19:23.485528+0000","last_clean":"2023-10-10T20:19:23.485528+0000","last_became_active":"2023-10-10T20:19:23.485195+0000","last_became_peered":"2023-10-10T20:19:23.485195+0000","last_unstale":"2023-10-10T20:19:23.485528+0000","last_undegraded":"2023-10-10T20:19:23.485528+0000","last_fullsized":"2023-10-10T20:19:23.485528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487121+0000","last_change":"2023-10-10T20:19:23.486697+0000","last_active":"2023-10-10T20:19:23.487121+0000","last_peered":"2023-10-10T20:19:23.487121+0000","last_clean":"2023-10-10T20:19:23.487121+0000","last_became_active":"2023-10-10T20:19:23.486167+0000","last_became_peered":"2023-10-10T20:19:23.486167+0000","last_unstale":"2023-10-10T20:19:23.487121+0000","last_undegraded":"2023-10-10T20:19:23.487121+0000","last_fullsized":"2023-10-10T20:19:23.487121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:23.488871+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:23.488509+0000","last_became_peered":"2023-10-10T20:19:23.488509+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498221+0000","last_change":"2023-10-10T20:19:23.497323+0000","last_active":"2023-10-10T20:19:23.498221+0000","last_peered":"2023-10-10T20:19:23.498221+0000","last_clean":"2023-10-10T20:19:23.498221+0000","last_became_active":"2023-10-10T20:19:23.496823+0000","last_became_peered":"2023-10-10T20:19:23.496823+0000","last_unstale":"2023-10-10T20:19:23.498221+0000","last_undegraded":"2023-10-10T20:19:23.498221+0000","last_fullsized":"2023-10-10T20:19:23.498221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489076+0000","last_change":"2023-10-10T20:19:23.488948+0000","last_active":"2023-10-10T20:19:23.489076+0000","last_peered":"2023-10-10T20:19:23.489076+0000","last_clean":"2023-10-10T20:19:23.489076+0000","last_became_active":"2023-10-10T20:19:23.488700+0000","last_became_peered":"2023-10-10T20:19:23.488700+0000","last_unstale":"2023-10-10T20:19:23.489076+0000","last_undegraded":"2023-10-10T20:19:23.489076+0000","last_fullsized":"2023-10-10T20:19:23.489076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492260+0000","last_change":"2023-10-10T20:19:23.492008+0000","last_active":"2023-10-10T20:19:23.492260+0000","last_peered":"2023-10-10T20:19:23.492260+0000","last_clean":"2023-10-10T20:19:23.492260+0000","last_became_active":"2023-10-10T20:19:23.491368+0000","last_became_peered":"2023-10-10T20:19:23.491368+0000","last_unstale":"2023-10-10T20:19:23.492260+0000","last_undegraded":"2023-10-10T20:19:23.492260+0000","last_fullsized":"2023-10-10T20:19:23.492260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487136+0000","last_change":"2023-10-10T20:19:23.486830+0000","last_active":"2023-10-10T20:19:23.487136+0000","last_peered":"2023-10-10T20:19:23.487136+0000","last_clean":"2023-10-10T20:19:23.487136+0000","last_became_active":"2023-10-10T20:19:23.486468+0000","last_became_peered":"2023-10-10T20:19:23.486468+0000","last_unstale":"2023-10-10T20:19:23.487136+0000","last_undegraded":"2023-10-10T20:19:23.487136+0000","last_fullsized":"2023-10-10T20:19:23.487136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495811+0000","last_change":"2023-10-10T20:19:23.495638+0000","last_active":"2023-10-10T20:19:23.495811+0000","last_peered":"2023-10-10T20:19:23.495811+0000","last_clean":"2023-10-10T20:19:23.495811+0000","last_became_active":"2023-10-10T20:19:23.495020+0000","last_became_peered":"2023-10-10T20:19:23.495020+0000","last_unstale":"2023-10-10T20:19:23.495811+0000","last_undegraded":"2023-10-10T20:19:23.495811+0000","last_fullsized":"2023-10-10T20:19:23.495811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491834+0000","last_change":"2023-10-10T20:19:23.491773+0000","last_active":"2023-10-10T20:19:23.491834+0000","last_peered":"2023-10-10T20:19:23.491834+0000","last_clean":"2023-10-10T20:19:23.491834+0000","last_became_active":"2023-10-10T20:19:23.491189+0000","last_became_peered":"2023-10-10T20:19:23.491189+0000","last_unstale":"2023-10-10T20:19:23.491834+0000","last_undegraded":"2023-10-10T20:19:23.491834+0000","last_fullsized":"2023-10-10T20:19:23.491834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499362+0000","last_change":"2023-10-10T20:19:23.499299+0000","last_active":"2023-10-10T20:19:23.499362+0000","last_peered":"2023-10-10T20:19:23.499362+0000","last_clean":"2023-10-10T20:19:23.499362+0000","last_became_active":"2023-10-10T20:19:23.498950+0000","last_became_peered":"2023-10-10T20:19:23.498950+0000","last_unstale":"2023-10-10T20:19:23.499362+0000","last_undegraded":"2023-10-10T20:19:23.499362+0000","last_fullsized":"2023-10-10T20:19:23.499362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929346,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":644,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339539,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":67,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929380,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929416,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:36.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:19:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.780+0000 7fb33ffff700 1 Processor -- start 2023-10-10T20:19:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.780+0000 7fb33ffff700 1 -- start start 2023-10-10T20:19:36.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb34015cbd0 0x7fb34015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400cb7c0 con 0x7fb3400cd6e0 2023-10-10T20:19:36.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33ffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34015e290 con 0x7fb3400cdab0 2023-10-10T20:19:36.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33ffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34015e850 con 0x7fb34015cbd0 2023-10-10T20:19:36.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33effd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb3400cd6e0 0x7fb3400cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58354/0 (socket says 172.21.15.148:58354) 2023-10-10T20:19:36.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33effd700 1 -- 172.21.15.148:0/3564868245 learned_addr learned my addr 172.21.15.148:0/3564868245 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:36.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33f7fe700 1 --2- 172.21.15.148:0/3564868245 >> v2:172.21.15.148:3300/0 conn(0x7fb34015cbd0 0x7fb34015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33e7fc700 1 -- 172.21.15.148:0/3564868245 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341992933 0 0) 0x7fb3400cb7c0 con 0x7fb3400cd6e0 2023-10-10T20:19:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33e7fc700 1 -- 172.21.15.148:0/3564868245 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c003570 con 0x7fb3400cd6e0 2023-10-10T20:19:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33f7fe700 1 -- 172.21.15.148:0/3564868245 >> v1:172.21.15.161:6789/0 conn(0x7fb3400cdab0 legacy=0x7fb34015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33f7fe700 1 -- 172.21.15.148:0/3564868245 >> v1:172.21.15.148:6789/0 conn(0x7fb3400cd6e0 legacy=0x7fb3400cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33f7fe700 1 -- 172.21.15.148:0/3564868245 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34015e570 con 0x7fb34015cbd0 2023-10-10T20:19:36.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.781+0000 7fb33f7fe700 1 --2- 172.21.15.148:0/3564868245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34015cbd0 0x7fb34015d050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e60 tx=0x7fb338009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579110fb0ba0b9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33e7fc700 1 -- 172.21.15.148:0/3564868245 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800e070 con 0x7fb34015cbd0 2023-10-10T20:19:36.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33e7fc700 1 -- 172.21.15.148:0/3564868245 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800bb60 con 0x7fb34015cbd0 2023-10-10T20:19:36.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33e7fc700 1 -- 172.21.15.148:0/3564868245 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338012430 con 0x7fb34015cbd0 2023-10-10T20:19:36.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 -- 172.21.15.148:0/3564868245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34015cbd0 msgr2=0x7fb34015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 --2- 172.21.15.148:0/3564868245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34015cbd0 0x7fb34015d050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e60 tx=0x7fb338009780 comp rx=0 tx=0).stop 2023-10-10T20:19:36.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 -- 172.21.15.148:0/3564868245 shutdown_connections 2023-10-10T20:19:36.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 --2- 172.21.15.148:0/3564868245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb34015cbd0 0x7fb34015d050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 -- 172.21.15.148:0/3564868245 >> 172.21.15.148:0/3564868245 conn(0x7fb340067660 msgr2=0x7fb340067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 -- 172.21.15.148:0/3564868245 shutdown_connections 2023-10-10T20:19:36.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.782+0000 7fb33ffff700 1 -- 172.21.15.148:0/3564868245 wait complete. 2023-10-10T20:19:36.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.783+0000 7fb33ffff700 1 Processor -- start 2023-10-10T20:19:36.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.783+0000 7fb33ffff700 1 -- start start 2023-10-10T20:19:36.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3400cdab0 0x7fb340155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb34015cbd0 0x7fb3400afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400cc360 con 0x7fb3400cd6e0 2023-10-10T20:19:36.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c7d20 con 0x7fb3400cdab0 2023-10-10T20:19:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34015c480 con 0x7fb34015cbd0 2023-10-10T20:19:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42608/0 (socket says 172.21.15.148:42608) 2023-10-10T20:19:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 -- 172.21.15.148:0/574651901 learned_addr learned my addr 172.21.15.148:0/574651901 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb3367fc700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3400cdab0 0x7fb340155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33f7fe700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb34015cbd0 0x7fb3400afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 -- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb34015cbd0 msgr2=0x7fb3400afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb34015cbd0 0x7fb3400afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 -- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3400cdab0 msgr2=0x7fb340155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3400cdab0 0x7fb340155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.784+0000 7fb33effd700 1 -- 172.21.15.148:0/574651901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338009a20 con 0x7fb3400cd6e0 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb33effd700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c5c0 tx=0x7fb330007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330005b30 con 0x7fb3400cd6e0 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330005cb0 con 0x7fb3400cd6e0 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34015b870 con 0x7fb3400cd6e0 2023-10-10T20:19:36.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330007320 con 0x7fb3400cd6e0 2023-10-10T20:19:36.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.785+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34015be30 con 0x7fb3400cd6e0 2023-10-10T20:19:36.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.786+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb340089280 con 0x7fb3400cd6e0 2023-10-10T20:19:36.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.786+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb33001b3b0 con 0x7fb3400cd6e0 2023-10-10T20:19:36.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.787+0000 7fb3377fe700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 0x7fb3200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:36.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.787+0000 7fb3367fc700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 0x7fb3200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:36.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.787+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb330083ae0 con 0x7fb3400cd6e0 2023-10-10T20:19:36.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.787+0000 7fb3367fc700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 0x7fb3200693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb3400b5a40 tx=0x7fb328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:36.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.791+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb330050d50 con 0x7fb3400cd6e0 2023-10-10T20:19:36.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.969+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb34004bb80 con 0x7fb320066f30 2023-10-10T20:19:36.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.976+0000 7fb3377fe700 1 -- 172.21.15.148:0/574651901 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260164 (secure 0 0 0) 0x7fb34004bb80 con 0x7fb320066f30 2023-10-10T20:19:36.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:36.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 msgr2=0x7fb3200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 0x7fb3200693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb3400b5a40 tx=0x7fb328009040 comp rx=0 tx=0).stop 2023-10-10T20:19:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 msgr2=0x7fb34015b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c5c0 tx=0x7fb330007e00 comp rx=0 tx=0).stop 2023-10-10T20:19:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 shutdown_connections 2023-10-10T20:19:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb320066f30 0x7fb3200693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb34015cbd0 0x7fb3400afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3400cd6e0 0x7fb34015b220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 --2- 172.21.15.148:0/574651901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3400cdab0 0x7fb340155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:36.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 >> 172.21.15.148:0/574651901 conn(0x7fb340067660 msgr2=0x7fb3400b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:36.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 shutdown_connections 2023-10-10T20:19:36.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:36.979+0000 7fb33ffff700 1 -- 172.21.15.148:0/574651901 wait complete. 2023-10-10T20:19:36.986 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:19:36.993 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-10T20:19:35.325242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560793,"statfs":{"total":1600000000,"available":1598252722,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.843045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.500183+0000","last_change":"2023-10-10T20:19:23.499934+0000","last_active":"2023-10-10T20:19:23.500183+0000","last_peered":"2023-10-10T20:19:23.500183+0000","last_clean":"2023-10-10T20:19:23.500183+0000","last_became_active":"2023-10-10T20:19:23.499640+0000","last_became_peered":"2023-10-10T20:19:23.499640+0000","last_unstale":"2023-10-10T20:19:23.500183+0000","last_undegraded":"2023-10-10T20:19:23.500183+0000","last_fullsized":"2023-10-10T20:19:23.500183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488778+0000","last_change":"2023-10-10T20:19:23.488718+0000","last_active":"2023-10-10T20:19:23.488778+0000","last_peered":"2023-10-10T20:19:23.488778+0000","last_clean":"2023-10-10T20:19:23.488778+0000","last_became_active":"2023-10-10T20:19:23.488207+0000","last_became_peered":"2023-10-10T20:19:23.488207+0000","last_unstale":"2023-10-10T20:19:23.488778+0000","last_undegraded":"2023-10-10T20:19:23.488778+0000","last_fullsized":"2023-10-10T20:19:23.488778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488280+0000","last_change":"2023-10-10T20:19:23.487828+0000","last_active":"2023-10-10T20:19:23.488280+0000","last_peered":"2023-10-10T20:19:23.488280+0000","last_clean":"2023-10-10T20:19:23.488280+0000","last_became_active":"2023-10-10T20:19:23.487010+0000","last_became_peered":"2023-10-10T20:19:23.487010+0000","last_unstale":"2023-10-10T20:19:23.488280+0000","last_undegraded":"2023-10-10T20:19:23.488280+0000","last_fullsized":"2023-10-10T20:19:23.488280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492279+0000","last_change":"2023-10-10T20:19:23.492180+0000","last_active":"2023-10-10T20:19:23.492279+0000","last_peered":"2023-10-10T20:19:23.492279+0000","last_clean":"2023-10-10T20:19:23.492279+0000","last_became_active":"2023-10-10T20:19:23.491680+0000","last_became_peered":"2023-10-10T20:19:23.491680+0000","last_unstale":"2023-10-10T20:19:23.492279+0000","last_undegraded":"2023-10-10T20:19:23.492279+0000","last_fullsized":"2023-10-10T20:19:23.492279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492487+0000","last_change":"2023-10-10T20:19:23.492418+0000","last_active":"2023-10-10T20:19:23.492487+0000","last_peered":"2023-10-10T20:19:23.492487+0000","last_clean":"2023-10-10T20:19:23.492487+0000","last_became_active":"2023-10-10T20:19:23.492183+0000","last_became_peered":"2023-10-10T20:19:23.492183+0000","last_unstale":"2023-10-10T20:19:23.492487+0000","last_undegraded":"2023-10-10T20:19:23.492487+0000","last_fullsized":"2023-10-10T20:19:23.492487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494017+0000","last_change":"2023-10-10T20:19:23.493863+0000","last_active":"2023-10-10T20:19:23.494017+0000","last_peered":"2023-10-10T20:19:23.494017+0000","last_clean":"2023-10-10T20:19:23.494017+0000","last_became_active":"2023-10-10T20:19:23.492771+0000","last_became_peered":"2023-10-10T20:19:23.492771+0000","last_unstale":"2023-10-10T20:19:23.494017+0000","last_undegraded":"2023-10-10T20:19:23.494017+0000","last_fullsized":"2023-10-10T20:19:23.494017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493478+0000","last_change":"2023-10-10T20:19:23.493170+0000","last_active":"2023-10-10T20:19:23.493478+0000","last_peered":"2023-10-10T20:19:23.493478+0000","last_clean":"2023-10-10T20:19:23.493478+0000","last_became_active":"2023-10-10T20:19:23.492278+0000","last_became_peered":"2023-10-10T20:19:23.492278+0000","last_unstale":"2023-10-10T20:19:23.493478+0000","last_undegraded":"2023-10-10T20:19:23.493478+0000","last_fullsized":"2023-10-10T20:19:23.493478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:26:39.436516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485806+0000","last_change":"2023-10-10T20:19:23.485444+0000","last_active":"2023-10-10T20:19:23.485806+0000","last_peered":"2023-10-10T20:19:23.485806+0000","last_clean":"2023-10-10T20:19:23.485806+0000","last_became_active":"2023-10-10T20:19:23.485215+0000","last_became_peered":"2023-10-10T20:19:23.485215+0000","last_unstale":"2023-10-10T20:19:23.485806+0000","last_undegraded":"2023-10-10T20:19:23.485806+0000","last_fullsized":"2023-10-10T20:19:23.485806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495009+0000","last_change":"2023-10-10T20:19:23.494933+0000","last_active":"2023-10-10T20:19:23.495009+0000","last_peered":"2023-10-10T20:19:23.495009+0000","last_clean":"2023-10-10T20:19:23.495009+0000","last_became_active":"2023-10-10T20:19:23.494414+0000","last_became_peered":"2023-10-10T20:19:23.494414+0000","last_unstale":"2023-10-10T20:19:23.495009+0000","last_undegraded":"2023-10-10T20:19:23.495009+0000","last_fullsized":"2023-10-10T20:19:23.495009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499228+0000","last_change":"2023-10-10T20:19:23.499169+0000","last_active":"2023-10-10T20:19:23.499228+0000","last_peered":"2023-10-10T20:19:23.499228+0000","last_clean":"2023-10-10T20:19:23.499228+0000","last_became_active":"2023-10-10T20:19:23.498639+0000","last_became_peered":"2023-10-10T20:19:23.498639+0000","last_unstale":"2023-10-10T20:19:23.499228+0000","last_undegraded":"2023-10-10T20:19:23.499228+0000","last_fullsized":"2023-10-10T20:19:23.499228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498244+0000","last_change":"2023-10-10T20:19:23.497453+0000","last_active":"2023-10-10T20:19:23.498244+0000","last_peered":"2023-10-10T20:19:23.498244+0000","last_clean":"2023-10-10T20:19:23.498244+0000","last_became_active":"2023-10-10T20:19:23.497111+0000","last_became_peered":"2023-10-10T20:19:23.497111+0000","last_unstale":"2023-10-10T20:19:23.498244+0000","last_undegraded":"2023-10-10T20:19:23.498244+0000","last_fullsized":"2023-10-10T20:19:23.498244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493732+0000","last_change":"2023-10-10T20:19:23.493307+0000","last_active":"2023-10-10T20:19:23.493732+0000","last_peered":"2023-10-10T20:19:23.493732+0000","last_clean":"2023-10-10T20:19:23.493732+0000","last_became_active":"2023-10-10T20:19:23.492611+0000","last_became_peered":"2023-10-10T20:19:23.492611+0000","last_unstale":"2023-10-10T20:19:23.493732+0000","last_undegraded":"2023-10-10T20:19:23.493732+0000","last_fullsized":"2023-10-10T20:19:23.493732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495418+0000","last_change":"2023-10-10T20:19:23.495368+0000","last_active":"2023-10-10T20:19:23.495418+0000","last_peered":"2023-10-10T20:19:23.495418+0000","last_clean":"2023-10-10T20:19:23.495418+0000","last_became_active":"2023-10-10T20:19:23.495214+0000","last_became_peered":"2023-10-10T20:19:23.495214+0000","last_unstale":"2023-10-10T20:19:23.495418+0000","last_undegraded":"2023-10-10T20:19:23.495418+0000","last_fullsized":"2023-10-10T20:19:23.495418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483124+0000","last_change":"2023-10-10T20:19:23.483055+0000","last_active":"2023-10-10T20:19:23.483124+0000","last_peered":"2023-10-10T20:19:23.483124+0000","last_clean":"2023-10-10T20:19:23.483124+0000","last_became_active":"2023-10-10T20:19:23.482827+0000","last_became_peered":"2023-10-10T20:19:23.482827+0000","last_unstale":"2023-10-10T20:19:23.483124+0000","last_undegraded":"2023-10-10T20:19:23.483124+0000","last_fullsized":"2023-10-10T20:19:23.483124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492517+0000","last_change":"2023-10-10T20:19:23.492454+0000","last_active":"2023-10-10T20:19:23.492517+0000","last_peered":"2023-10-10T20:19:23.492517+0000","last_clean":"2023-10-10T20:19:23.492517+0000","last_became_active":"2023-10-10T20:19:23.492262+0000","last_became_peered":"2023-10-10T20:19:23.492262+0000","last_unstale":"2023-10-10T20:19:23.492517+0000","last_undegraded":"2023-10-10T20:19:23.492517+0000","last_fullsized":"2023-10-10T20:19:23.492517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493999+0000","last_change":"2023-10-10T20:19:23.493726+0000","last_active":"2023-10-10T20:19:23.493999+0000","last_peered":"2023-10-10T20:19:23.493999+0000","last_clean":"2023-10-10T20:19:23.493999+0000","last_became_active":"2023-10-10T20:19:23.492475+0000","last_became_peered":"2023-10-10T20:19:23.492475+0000","last_unstale":"2023-10-10T20:19:23.493999+0000","last_undegraded":"2023-10-10T20:19:23.493999+0000","last_fullsized":"2023-10-10T20:19:23.493999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495844+0000","last_change":"2023-10-10T20:19:23.495784+0000","last_active":"2023-10-10T20:19:23.495844+0000","last_peered":"2023-10-10T20:19:23.495844+0000","last_clean":"2023-10-10T20:19:23.495844+0000","last_became_active":"2023-10-10T20:19:23.495346+0000","last_became_peered":"2023-10-10T20:19:23.495346+0000","last_unstale":"2023-10-10T20:19:23.495844+0000","last_undegraded":"2023-10-10T20:19:23.495844+0000","last_fullsized":"2023-10-10T20:19:23.495844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494759+0000","last_change":"2023-10-10T20:19:23.494684+0000","last_active":"2023-10-10T20:19:23.494759+0000","last_peered":"2023-10-10T20:19:23.494759+0000","last_clean":"2023-10-10T20:19:23.494759+0000","last_became_active":"2023-10-10T20:19:23.494457+0000","last_became_peered":"2023-10-10T20:19:23.494457+0000","last_unstale":"2023-10-10T20:19:23.494759+0000","last_undegraded":"2023-10-10T20:19:23.494759+0000","last_fullsized":"2023-10-10T20:19:23.494759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:25.117130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491717+0000","last_change":"2023-10-10T20:19:23.491294+0000","last_active":"2023-10-10T20:19:23.491717+0000","last_peered":"2023-10-10T20:19:23.491717+0000","last_clean":"2023-10-10T20:19:23.491717+0000","last_became_active":"2023-10-10T20:19:23.491078+0000","last_became_peered":"2023-10-10T20:19:23.491078+0000","last_unstale":"2023-10-10T20:19:23.491717+0000","last_undegraded":"2023-10-10T20:19:23.491717+0000","last_fullsized":"2023-10-10T20:19:23.491717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505463+0000","last_change":"2023-10-10T20:19:22.491174+0000","last_active":"2023-10-10T20:19:23.505463+0000","last_peered":"2023-10-10T20:19:23.505463+0000","last_clean":"2023-10-10T20:19:23.505463+0000","last_became_active":"2023-10-10T20:19:22.490398+0000","last_became_peered":"2023-10-10T20:19:22.490398+0000","last_unstale":"2023-10-10T20:19:23.505463+0000","last_undegraded":"2023-10-10T20:19:23.505463+0000","last_fullsized":"2023-10-10T20:19:23.505463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:26:58.128615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490418+0000","last_change":"2023-10-10T20:19:23.490364+0000","last_active":"2023-10-10T20:19:23.490418+0000","last_peered":"2023-10-10T20:19:23.490418+0000","last_clean":"2023-10-10T20:19:23.490418+0000","last_became_active":"2023-10-10T20:19:23.490129+0000","last_became_peered":"2023-10-10T20:19:23.490129+0000","last_unstale":"2023-10-10T20:19:23.490418+0000","last_undegraded":"2023-10-10T20:19:23.490418+0000","last_fullsized":"2023-10-10T20:19:23.490418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505547+0000","last_change":"2023-10-10T20:19:22.490246+0000","last_active":"2023-10-10T20:19:23.505547+0000","last_peered":"2023-10-10T20:19:23.505547+0000","last_clean":"2023-10-10T20:19:23.505547+0000","last_became_active":"2023-10-10T20:19:22.490027+0000","last_became_peered":"2023-10-10T20:19:22.490027+0000","last_unstale":"2023-10-10T20:19:23.505547+0000","last_undegraded":"2023-10-10T20:19:23.505547+0000","last_fullsized":"2023-10-10T20:19:23.505547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488264+0000","last_change":"2023-10-10T20:19:23.487706+0000","last_active":"2023-10-10T20:19:23.488264+0000","last_peered":"2023-10-10T20:19:23.488264+0000","last_clean":"2023-10-10T20:19:23.488264+0000","last_became_active":"2023-10-10T20:19:23.486658+0000","last_became_peered":"2023-10-10T20:19:23.486658+0000","last_unstale":"2023-10-10T20:19:23.488264+0000","last_undegraded":"2023-10-10T20:19:23.488264+0000","last_fullsized":"2023-10-10T20:19:23.488264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505022+0000","last_change":"2023-10-10T20:19:22.491563+0000","last_active":"2023-10-10T20:19:23.505022+0000","last_peered":"2023-10-10T20:19:23.505022+0000","last_clean":"2023-10-10T20:19:23.505022+0000","last_became_active":"2023-10-10T20:19:22.491205+0000","last_became_peered":"2023-10-10T20:19:22.491205+0000","last_unstale":"2023-10-10T20:19:23.505022+0000","last_undegraded":"2023-10-10T20:19:23.505022+0000","last_fullsized":"2023-10-10T20:19:23.505022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495250+0000","last_change":"2023-10-10T20:19:23.495186+0000","last_active":"2023-10-10T20:19:23.495250+0000","last_peered":"2023-10-10T20:19:23.495250+0000","last_clean":"2023-10-10T20:19:23.495250+0000","last_became_active":"2023-10-10T20:19:23.494935+0000","last_became_peered":"2023-10-10T20:19:23.494935+0000","last_unstale":"2023-10-10T20:19:23.495250+0000","last_undegraded":"2023-10-10T20:19:23.495250+0000","last_fullsized":"2023-10-10T20:19:23.495250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503478+0000","last_change":"2023-10-10T20:19:22.493193+0000","last_active":"2023-10-10T20:19:23.503478+0000","last_peered":"2023-10-10T20:19:23.503478+0000","last_clean":"2023-10-10T20:19:23.503478+0000","last_became_active":"2023-10-10T20:19:22.492979+0000","last_became_peered":"2023-10-10T20:19:22.492979+0000","last_unstale":"2023-10-10T20:19:23.503478+0000","last_undegraded":"2023-10-10T20:19:23.503478+0000","last_fullsized":"2023-10-10T20:19:23.503478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487270+0000","last_change":"2023-10-10T20:19:23.487131+0000","last_active":"2023-10-10T20:19:23.487270+0000","last_peered":"2023-10-10T20:19:23.487270+0000","last_clean":"2023-10-10T20:19:23.487270+0000","last_became_active":"2023-10-10T20:19:23.486894+0000","last_became_peered":"2023-10-10T20:19:23.486894+0000","last_unstale":"2023-10-10T20:19:23.487270+0000","last_undegraded":"2023-10-10T20:19:23.487270+0000","last_fullsized":"2023-10-10T20:19:23.487270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505011+0000","last_change":"2023-10-10T20:19:22.491757+0000","last_active":"2023-10-10T20:19:23.505011+0000","last_peered":"2023-10-10T20:19:23.505011+0000","last_clean":"2023-10-10T20:19:23.505011+0000","last_became_active":"2023-10-10T20:19:22.491535+0000","last_became_peered":"2023-10-10T20:19:22.491535+0000","last_unstale":"2023-10-10T20:19:23.505011+0000","last_undegraded":"2023-10-10T20:19:23.505011+0000","last_fullsized":"2023-10-10T20:19:23.505011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489517+0000","last_change":"2023-10-10T20:19:23.488600+0000","last_active":"2023-10-10T20:19:23.489517+0000","last_peered":"2023-10-10T20:19:23.489517+0000","last_clean":"2023-10-10T20:19:23.489517+0000","last_became_active":"2023-10-10T20:19:23.487824+0000","last_became_peered":"2023-10-10T20:19:23.487824+0000","last_unstale":"2023-10-10T20:19:23.489517+0000","last_undegraded":"2023-10-10T20:19:23.489517+0000","last_fullsized":"2023-10-10T20:19:23.489517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:07.656571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.502781+0000","last_change":"2023-10-10T20:19:22.486217+0000","last_active":"2023-10-10T20:19:23.502781+0000","last_peered":"2023-10-10T20:19:23.502781+0000","last_clean":"2023-10-10T20:19:23.502781+0000","last_became_active":"2023-10-10T20:19:22.484804+0000","last_became_peered":"2023-10-10T20:19:22.484804+0000","last_unstale":"2023-10-10T20:19:23.502781+0000","last_undegraded":"2023-10-10T20:19:23.502781+0000","last_fullsized":"2023-10-10T20:19:23.502781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485364+0000","last_change":"2023-10-10T20:19:23.485292+0000","last_active":"2023-10-10T20:19:23.485364+0000","last_peered":"2023-10-10T20:19:23.485364+0000","last_clean":"2023-10-10T20:19:23.485364+0000","last_became_active":"2023-10-10T20:19:23.485060+0000","last_became_peered":"2023-10-10T20:19:23.485060+0000","last_unstale":"2023-10-10T20:19:23.485364+0000","last_undegraded":"2023-10-10T20:19:23.485364+0000","last_fullsized":"2023-10-10T20:19:23.485364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:16.611774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482268+0000","last_change":"2023-10-10T20:19:22.489127+0000","last_active":"2023-10-10T20:19:23.482268+0000","last_peered":"2023-10-10T20:19:23.482268+0000","last_clean":"2023-10-10T20:19:23.482268+0000","last_became_active":"2023-10-10T20:19:22.488683+0000","last_became_peered":"2023-10-10T20:19:22.488683+0000","last_unstale":"2023-10-10T20:19:23.482268+0000","last_undegraded":"2023-10-10T20:19:23.482268+0000","last_fullsized":"2023-10-10T20:19:23.482268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487104+0000","last_change":"2023-10-10T20:19:23.486567+0000","last_active":"2023-10-10T20:19:23.487104+0000","last_peered":"2023-10-10T20:19:23.487104+0000","last_clean":"2023-10-10T20:19:23.487104+0000","last_became_active":"2023-10-10T20:19:23.486329+0000","last_became_peered":"2023-10-10T20:19:23.486329+0000","last_unstale":"2023-10-10T20:19:23.487104+0000","last_undegraded":"2023-10-10T20:19:23.487104+0000","last_fullsized":"2023-10-10T20:19:23.487104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483483+0000","last_change":"2023-10-10T20:19:22.485105+0000","last_active":"2023-10-10T20:19:23.483483+0000","last_peered":"2023-10-10T20:19:23.483483+0000","last_clean":"2023-10-10T20:19:23.483483+0000","last_became_active":"2023-10-10T20:19:22.484537+0000","last_became_peered":"2023-10-10T20:19:22.484537+0000","last_unstale":"2023-10-10T20:19:23.483483+0000","last_undegraded":"2023-10-10T20:19:23.483483+0000","last_fullsized":"2023-10-10T20:19:23.483483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483939+0000","last_change":"2023-10-10T20:19:23.483701+0000","last_active":"2023-10-10T20:19:23.483939+0000","last_peered":"2023-10-10T20:19:23.483939+0000","last_clean":"2023-10-10T20:19:23.483939+0000","last_became_active":"2023-10-10T20:19:23.483485+0000","last_became_peered":"2023-10-10T20:19:23.483485+0000","last_unstale":"2023-10-10T20:19:23.483939+0000","last_undegraded":"2023-10-10T20:19:23.483939+0000","last_fullsized":"2023-10-10T20:19:23.483939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486842+0000","last_change":"2023-10-10T20:19:22.486912+0000","last_active":"2023-10-10T20:19:23.486842+0000","last_peered":"2023-10-10T20:19:23.486842+0000","last_clean":"2023-10-10T20:19:23.486842+0000","last_became_active":"2023-10-10T20:19:22.486662+0000","last_became_peered":"2023-10-10T20:19:22.486662+0000","last_unstale":"2023-10-10T20:19:23.486842+0000","last_undegraded":"2023-10-10T20:19:23.486842+0000","last_fullsized":"2023-10-10T20:19:23.486842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492299+0000","last_change":"2023-10-10T20:19:23.492209+0000","last_active":"2023-10-10T20:19:23.492299+0000","last_peered":"2023-10-10T20:19:23.492299+0000","last_clean":"2023-10-10T20:19:23.492299+0000","last_became_active":"2023-10-10T20:19:23.491975+0000","last_became_peered":"2023-10-10T20:19:23.491975+0000","last_unstale":"2023-10-10T20:19:23.492299+0000","last_undegraded":"2023-10-10T20:19:23.492299+0000","last_fullsized":"2023-10-10T20:19:23.492299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:05.772320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485905+0000","last_change":"2023-10-10T20:19:22.490636+0000","last_active":"2023-10-10T20:19:23.485905+0000","last_peered":"2023-10-10T20:19:23.485905+0000","last_clean":"2023-10-10T20:19:23.485905+0000","last_became_active":"2023-10-10T20:19:22.490369+0000","last_became_peered":"2023-10-10T20:19:22.490369+0000","last_unstale":"2023-10-10T20:19:23.485905+0000","last_undegraded":"2023-10-10T20:19:23.485905+0000","last_fullsized":"2023-10-10T20:19:23.485905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490958+0000","last_change":"2023-10-10T20:19:23.490908+0000","last_active":"2023-10-10T20:19:23.490958+0000","last_peered":"2023-10-10T20:19:23.490958+0000","last_clean":"2023-10-10T20:19:23.490958+0000","last_became_active":"2023-10-10T20:19:23.490692+0000","last_became_peered":"2023-10-10T20:19:23.490692+0000","last_unstale":"2023-10-10T20:19:23.490958+0000","last_undegraded":"2023-10-10T20:19:23.490958+0000","last_fullsized":"2023-10-10T20:19:23.490958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503581+0000","last_change":"2023-10-10T20:19:22.487531+0000","last_active":"2023-10-10T20:19:23.503581+0000","last_peered":"2023-10-10T20:19:23.503581+0000","last_clean":"2023-10-10T20:19:23.503581+0000","last_became_active":"2023-10-10T20:19:22.487040+0000","last_became_peered":"2023-10-10T20:19:22.487040+0000","last_unstale":"2023-10-10T20:19:23.503581+0000","last_undegraded":"2023-10-10T20:19:23.503581+0000","last_fullsized":"2023-10-10T20:19:23.503581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493693+0000","last_change":"2023-10-10T20:19:23.493005+0000","last_active":"2023-10-10T20:19:23.493693+0000","last_peered":"2023-10-10T20:19:23.493693+0000","last_clean":"2023-10-10T20:19:23.493693+0000","last_became_active":"2023-10-10T20:19:23.492752+0000","last_became_peered":"2023-10-10T20:19:23.492752+0000","last_unstale":"2023-10-10T20:19:23.493693+0000","last_undegraded":"2023-10-10T20:19:23.493693+0000","last_fullsized":"2023-10-10T20:19:23.493693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.506581+0000","last_change":"2023-10-10T20:19:22.491546+0000","last_active":"2023-10-10T20:19:23.506581+0000","last_peered":"2023-10-10T20:19:23.506581+0000","last_clean":"2023-10-10T20:19:23.506581+0000","last_became_active":"2023-10-10T20:19:22.491157+0000","last_became_peered":"2023-10-10T20:19:22.491157+0000","last_unstale":"2023-10-10T20:19:23.506581+0000","last_undegraded":"2023-10-10T20:19:23.506581+0000","last_fullsized":"2023-10-10T20:19:23.506581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490504+0000","last_change":"2023-10-10T20:19:23.490357+0000","last_active":"2023-10-10T20:19:23.490504+0000","last_peered":"2023-10-10T20:19:23.490504+0000","last_clean":"2023-10-10T20:19:23.490504+0000","last_became_active":"2023-10-10T20:19:23.490094+0000","last_became_peered":"2023-10-10T20:19:23.490094+0000","last_unstale":"2023-10-10T20:19:23.490504+0000","last_undegraded":"2023-10-10T20:19:23.490504+0000","last_fullsized":"2023-10-10T20:19:23.490504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504122+0000","last_change":"2023-10-10T20:19:22.484098+0000","last_active":"2023-10-10T20:19:23.504122+0000","last_peered":"2023-10-10T20:19:23.504122+0000","last_clean":"2023-10-10T20:19:23.504122+0000","last_became_active":"2023-10-10T20:19:22.482738+0000","last_became_peered":"2023-10-10T20:19:22.482738+0000","last_unstale":"2023-10-10T20:19:23.504122+0000","last_undegraded":"2023-10-10T20:19:23.504122+0000","last_fullsized":"2023-10-10T20:19:23.504122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:40:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491923+0000","last_change":"2023-10-10T20:19:23.491873+0000","last_active":"2023-10-10T20:19:23.491923+0000","last_peered":"2023-10-10T20:19:23.491923+0000","last_clean":"2023-10-10T20:19:23.491923+0000","last_became_active":"2023-10-10T20:19:23.491532+0000","last_became_peered":"2023-10-10T20:19:23.491532+0000","last_unstale":"2023-10-10T20:19:23.491923+0000","last_undegraded":"2023-10-10T20:19:23.491923+0000","last_fullsized":"2023-10-10T20:19:23.491923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488825+0000","last_change":"2023-10-10T20:19:22.491830+0000","last_active":"2023-10-10T20:19:23.488825+0000","last_peered":"2023-10-10T20:19:23.488825+0000","last_clean":"2023-10-10T20:19:23.488825+0000","last_became_active":"2023-10-10T20:19:22.489289+0000","last_became_peered":"2023-10-10T20:19:22.489289+0000","last_unstale":"2023-10-10T20:19:23.488825+0000","last_undegraded":"2023-10-10T20:19:23.488825+0000","last_fullsized":"2023-10-10T20:19:23.488825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497717+0000","last_change":"2023-10-10T20:19:23.497580+0000","last_active":"2023-10-10T20:19:23.497717+0000","last_peered":"2023-10-10T20:19:23.497717+0000","last_clean":"2023-10-10T20:19:23.497717+0000","last_became_active":"2023-10-10T20:19:23.497043+0000","last_became_peered":"2023-10-10T20:19:23.497043+0000","last_unstale":"2023-10-10T20:19:23.497717+0000","last_undegraded":"2023-10-10T20:19:23.497717+0000","last_fullsized":"2023-10-10T20:19:23.497717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:18.031834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486275+0000","last_change":"2023-10-10T20:19:22.487157+0000","last_active":"2023-10-10T20:19:23.486275+0000","last_peered":"2023-10-10T20:19:23.486275+0000","last_clean":"2023-10-10T20:19:23.486275+0000","last_became_active":"2023-10-10T20:19:22.486367+0000","last_became_peered":"2023-10-10T20:19:22.486367+0000","last_unstale":"2023-10-10T20:19:23.486275+0000","last_undegraded":"2023-10-10T20:19:23.486275+0000","last_fullsized":"2023-10-10T20:19:23.486275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494474+0000","last_change":"2023-10-10T20:19:23.494326+0000","last_active":"2023-10-10T20:19:23.494474+0000","last_peered":"2023-10-10T20:19:23.494474+0000","last_clean":"2023-10-10T20:19:23.494474+0000","last_became_active":"2023-10-10T20:19:23.494043+0000","last_became_peered":"2023-10-10T20:19:23.494043+0000","last_unstale":"2023-10-10T20:19:23.494474+0000","last_undegraded":"2023-10-10T20:19:23.494474+0000","last_fullsized":"2023-10-10T20:19:23.494474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505618+0000","last_change":"2023-10-10T20:19:22.484580+0000","last_active":"2023-10-10T20:19:23.505618+0000","last_peered":"2023-10-10T20:19:23.505618+0000","last_clean":"2023-10-10T20:19:23.505618+0000","last_became_active":"2023-10-10T20:19:22.484287+0000","last_became_peered":"2023-10-10T20:19:22.484287+0000","last_unstale":"2023-10-10T20:19:23.505618+0000","last_undegraded":"2023-10-10T20:19:23.505618+0000","last_fullsized":"2023-10-10T20:19:23.505618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497360+0000","last_change":"2023-10-10T20:19:23.497292+0000","last_active":"2023-10-10T20:19:23.497360+0000","last_peered":"2023-10-10T20:19:23.497360+0000","last_clean":"2023-10-10T20:19:23.497360+0000","last_became_active":"2023-10-10T20:19:23.497063+0000","last_became_peered":"2023-10-10T20:19:23.497063+0000","last_unstale":"2023-10-10T20:19:23.497360+0000","last_undegraded":"2023-10-10T20:19:23.497360+0000","last_fullsized":"2023-10-10T20:19:23.497360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:44:31.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505930+0000","last_change":"2023-10-10T20:19:22.481673+0000","last_active":"2023-10-10T20:19:23.505930+0000","last_peered":"2023-10-10T20:19:23.505930+0000","last_clean":"2023-10-10T20:19:23.505930+0000","last_became_active":"2023-10-10T20:19:22.481500+0000","last_became_peered":"2023-10-10T20:19:22.481500+0000","last_unstale":"2023-10-10T20:19:23.505930+0000","last_undegraded":"2023-10-10T20:19:23.505930+0000","last_fullsized":"2023-10-10T20:19:23.505930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:33.728394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.497808+0000","last_change":"2023-10-10T20:19:23.497718+0000","last_active":"2023-10-10T20:19:23.497808+0000","last_peered":"2023-10-10T20:19:23.497808+0000","last_clean":"2023-10-10T20:19:23.497808+0000","last_became_active":"2023-10-10T20:19:23.497352+0000","last_became_peered":"2023-10-10T20:19:23.497352+0000","last_unstale":"2023-10-10T20:19:23.497808+0000","last_undegraded":"2023-10-10T20:19:23.497808+0000","last_fullsized":"2023-10-10T20:19:23.497808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.704243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481756+0000","last_change":"2023-10-10T20:19:22.486438+0000","last_active":"2023-10-10T20:19:23.481756+0000","last_peered":"2023-10-10T20:19:23.481756+0000","last_clean":"2023-10-10T20:19:23.481756+0000","last_became_active":"2023-10-10T20:19:22.486219+0000","last_became_peered":"2023-10-10T20:19:22.486219+0000","last_unstale":"2023-10-10T20:19:23.481756+0000","last_undegraded":"2023-10-10T20:19:23.481756+0000","last_fullsized":"2023-10-10T20:19:23.481756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490783+0000","last_change":"2023-10-10T20:19:23.490700+0000","last_active":"2023-10-10T20:19:23.490783+0000","last_peered":"2023-10-10T20:19:23.490783+0000","last_clean":"2023-10-10T20:19:23.490783+0000","last_became_active":"2023-10-10T20:19:23.490468+0000","last_became_peered":"2023-10-10T20:19:23.490468+0000","last_unstale":"2023-10-10T20:19:23.490783+0000","last_undegraded":"2023-10-10T20:19:23.490783+0000","last_fullsized":"2023-10-10T20:19:23.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486018+0000","last_change":"2023-10-10T20:19:22.485041+0000","last_active":"2023-10-10T20:19:23.486018+0000","last_peered":"2023-10-10T20:19:23.486018+0000","last_clean":"2023-10-10T20:19:23.486018+0000","last_became_active":"2023-10-10T20:19:22.484798+0000","last_became_peered":"2023-10-10T20:19:22.484798+0000","last_unstale":"2023-10-10T20:19:23.486018+0000","last_undegraded":"2023-10-10T20:19:23.486018+0000","last_fullsized":"2023-10-10T20:19:23.486018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489489+0000","last_change":"2023-10-10T20:19:23.487628+0000","last_active":"2023-10-10T20:19:23.489489+0000","last_peered":"2023-10-10T20:19:23.489489+0000","last_clean":"2023-10-10T20:19:23.489489+0000","last_became_active":"2023-10-10T20:19:23.486750+0000","last_became_peered":"2023-10-10T20:19:23.486750+0000","last_unstale":"2023-10-10T20:19:23.489489+0000","last_undegraded":"2023-10-10T20:19:23.489489+0000","last_fullsized":"2023-10-10T20:19:23.489489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:28.087913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487677+0000","last_change":"2023-10-10T20:19:23.486218+0000","last_active":"2023-10-10T20:19:23.487677+0000","last_peered":"2023-10-10T20:19:23.487677+0000","last_clean":"2023-10-10T20:19:23.487677+0000","last_became_active":"2023-10-10T20:19:23.485999+0000","last_became_peered":"2023-10-10T20:19:23.485999+0000","last_unstale":"2023-10-10T20:19:23.487677+0000","last_undegraded":"2023-10-10T20:19:23.487677+0000","last_fullsized":"2023-10-10T20:19:23.487677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.484120+0000","last_change":"2023-10-10T20:19:22.489199+0000","last_active":"2023-10-10T20:19:23.484120+0000","last_peered":"2023-10-10T20:19:23.484120+0000","last_clean":"2023-10-10T20:19:23.484120+0000","last_became_active":"2023-10-10T20:19:22.488931+0000","last_became_peered":"2023-10-10T20:19:22.488931+0000","last_unstale":"2023-10-10T20:19:23.484120+0000","last_undegraded":"2023-10-10T20:19:23.484120+0000","last_fullsized":"2023-10-10T20:19:23.484120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489524+0000","last_change":"2023-10-10T20:19:23.489413+0000","last_active":"2023-10-10T20:19:23.489524+0000","last_peered":"2023-10-10T20:19:23.489524+0000","last_clean":"2023-10-10T20:19:23.489524+0000","last_became_active":"2023-10-10T20:19:23.489083+0000","last_became_peered":"2023-10-10T20:19:23.489083+0000","last_unstale":"2023-10-10T20:19:23.489524+0000","last_undegraded":"2023-10-10T20:19:23.489524+0000","last_fullsized":"2023-10-10T20:19:23.489524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:22.491979+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:22.489613+0000","last_became_peered":"2023-10-10T20:19:22.489613+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:07.507653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486860+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:19:23.486860+0000","last_peered":"2023-10-10T20:19:23.486860+0000","last_clean":"2023-10-10T20:19:23.486860+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:19:23.486860+0000","last_undegraded":"2023-10-10T20:19:23.486860+0000","last_fullsized":"2023-10-10T20:19:23.486860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488300+0000","last_change":"2023-10-10T20:19:23.487957+0000","last_active":"2023-10-10T20:19:23.488300+0000","last_peered":"2023-10-10T20:19:23.488300+0000","last_clean":"2023-10-10T20:19:23.488300+0000","last_became_active":"2023-10-10T20:19:23.487318+0000","last_became_peered":"2023-10-10T20:19:23.487318+0000","last_unstale":"2023-10-10T20:19:23.488300+0000","last_undegraded":"2023-10-10T20:19:23.488300+0000","last_fullsized":"2023-10-10T20:19:23.488300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486185+0000","last_change":"2023-10-10T20:19:22.492546+0000","last_active":"2023-10-10T20:19:23.486185+0000","last_peered":"2023-10-10T20:19:23.486185+0000","last_clean":"2023-10-10T20:19:23.486185+0000","last_became_active":"2023-10-10T20:19:22.492293+0000","last_became_peered":"2023-10-10T20:19:22.492293+0000","last_unstale":"2023-10-10T20:19:23.486185+0000","last_undegraded":"2023-10-10T20:19:23.486185+0000","last_fullsized":"2023-10-10T20:19:23.486185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.486269+0000","last_change":"2023-10-10T20:19:22.490666+0000","last_active":"2023-10-10T20:19:23.486269+0000","last_peered":"2023-10-10T20:19:23.486269+0000","last_clean":"2023-10-10T20:19:23.486269+0000","last_became_active":"2023-10-10T20:19:22.490355+0000","last_became_peered":"2023-10-10T20:19:22.490355+0000","last_unstale":"2023-10-10T20:19:23.486269+0000","last_undegraded":"2023-10-10T20:19:23.486269+0000","last_fullsized":"2023-10-10T20:19:23.486269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:04.994838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487695+0000","last_change":"2023-10-10T20:19:23.486743+0000","last_active":"2023-10-10T20:19:23.487695+0000","last_peered":"2023-10-10T20:19:23.487695+0000","last_clean":"2023-10-10T20:19:23.487695+0000","last_became_active":"2023-10-10T20:19:23.486526+0000","last_became_peered":"2023-10-10T20:19:23.486526+0000","last_unstale":"2023-10-10T20:19:23.487695+0000","last_undegraded":"2023-10-10T20:19:23.487695+0000","last_fullsized":"2023-10-10T20:19:23.487695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:38.338508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-10T20:19:24.482879+0000","last_change":"2023-10-10T20:19:22.487657+0000","last_active":"2023-10-10T20:19:24.482879+0000","last_peered":"2023-10-10T20:19:24.482879+0000","last_clean":"2023-10-10T20:19:24.482879+0000","last_became_active":"2023-10-10T20:19:22.487329+0000","last_became_peered":"2023-10-10T20:19:22.487329+0000","last_unstale":"2023-10-10T20:19:24.482879+0000","last_undegraded":"2023-10-10T20:19:24.482879+0000","last_fullsized":"2023-10-10T20:19:24.482879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:05.420138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491172+0000","last_change":"2023-10-10T20:19:23.491123+0000","last_active":"2023-10-10T20:19:23.491172+0000","last_peered":"2023-10-10T20:19:23.491172+0000","last_clean":"2023-10-10T20:19:23.491172+0000","last_became_active":"2023-10-10T20:19:23.490946+0000","last_became_peered":"2023-10-10T20:19:23.490946+0000","last_unstale":"2023-10-10T20:19:23.491172+0000","last_undegraded":"2023-10-10T20:19:23.491172+0000","last_fullsized":"2023-10-10T20:19:23.491172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.505978+0000","last_change":"2023-10-10T20:19:22.491522+0000","last_active":"2023-10-10T20:19:23.505978+0000","last_peered":"2023-10-10T20:19:23.505978+0000","last_clean":"2023-10-10T20:19:23.505978+0000","last_became_active":"2023-10-10T20:19:22.491251+0000","last_became_peered":"2023-10-10T20:19:22.491251+0000","last_unstale":"2023-10-10T20:19:23.505978+0000","last_undegraded":"2023-10-10T20:19:23.505978+0000","last_fullsized":"2023-10-10T20:19:23.505978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:06.652741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488247+0000","last_change":"2023-10-10T20:19:23.487572+0000","last_active":"2023-10-10T20:19:23.488247+0000","last_peered":"2023-10-10T20:19:23.488247+0000","last_clean":"2023-10-10T20:19:23.488247+0000","last_became_active":"2023-10-10T20:19:23.486362+0000","last_became_peered":"2023-10-10T20:19:23.486362+0000","last_unstale":"2023-10-10T20:19:23.488247+0000","last_undegraded":"2023-10-10T20:19:23.488247+0000","last_fullsized":"2023-10-10T20:19:23.488247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:04.867879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489482+0000","last_change":"2023-10-10T20:19:22.490501+0000","last_active":"2023-10-10T20:19:23.489482+0000","last_peered":"2023-10-10T20:19:23.489482+0000","last_clean":"2023-10-10T20:19:23.489482+0000","last_became_active":"2023-10-10T20:19:22.490208+0000","last_became_peered":"2023-10-10T20:19:22.490208+0000","last_unstale":"2023-10-10T20:19:23.489482+0000","last_undegraded":"2023-10-10T20:19:23.489482+0000","last_fullsized":"2023-10-10T20:19:23.489482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.496090+0000","last_change":"2023-10-10T20:19:23.496016+0000","last_active":"2023-10-10T20:19:23.496090+0000","last_peered":"2023-10-10T20:19:23.496090+0000","last_clean":"2023-10-10T20:19:23.496090+0000","last_became_active":"2023-10-10T20:19:23.495512+0000","last_became_peered":"2023-10-10T20:19:23.495512+0000","last_unstale":"2023-10-10T20:19:23.496090+0000","last_undegraded":"2023-10-10T20:19:23.496090+0000","last_fullsized":"2023-10-10T20:19:23.496090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:19:05.998668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.481809+0000","last_change":"2023-10-10T20:19:22.490184+0000","last_active":"2023-10-10T20:19:23.481809+0000","last_peered":"2023-10-10T20:19:23.481809+0000","last_clean":"2023-10-10T20:19:23.481809+0000","last_became_active":"2023-10-10T20:19:22.490057+0000","last_became_peered":"2023-10-10T20:19:22.490057+0000","last_unstale":"2023-10-10T20:19:23.481809+0000","last_undegraded":"2023-10-10T20:19:23.481809+0000","last_fullsized":"2023-10-10T20:19:23.481809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:13.393866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487155+0000","last_change":"2023-10-10T20:19:23.487003+0000","last_active":"2023-10-10T20:19:23.487155+0000","last_peered":"2023-10-10T20:19:23.487155+0000","last_clean":"2023-10-10T20:19:23.487155+0000","last_became_active":"2023-10-10T20:19:23.486789+0000","last_became_peered":"2023-10-10T20:19:23.486789+0000","last_unstale":"2023-10-10T20:19:23.487155+0000","last_undegraded":"2023-10-10T20:19:23.487155+0000","last_fullsized":"2023-10-10T20:19:23.487155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:05.523817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.482124+0000","last_change":"2023-10-10T20:19:22.491852+0000","last_active":"2023-10-10T20:19:23.482124+0000","last_peered":"2023-10-10T20:19:23.482124+0000","last_clean":"2023-10-10T20:19:23.482124+0000","last_became_active":"2023-10-10T20:19:22.491569+0000","last_became_peered":"2023-10-10T20:19:22.491569+0000","last_unstale":"2023-10-10T20:19:23.482124+0000","last_undegraded":"2023-10-10T20:19:23.482124+0000","last_fullsized":"2023-10-10T20:19:23.482124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:47:24.113300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488335+0000","last_change":"2023-10-10T20:19:23.487466+0000","last_active":"2023-10-10T20:19:23.488335+0000","last_peered":"2023-10-10T20:19:23.488335+0000","last_clean":"2023-10-10T20:19:23.488335+0000","last_became_active":"2023-10-10T20:19:23.487241+0000","last_became_peered":"2023-10-10T20:19:23.487241+0000","last_unstale":"2023-10-10T20:19:23.488335+0000","last_undegraded":"2023-10-10T20:19:23.488335+0000","last_fullsized":"2023-10-10T20:19:23.488335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.503296+0000","last_change":"2023-10-10T20:19:22.490823+0000","last_active":"2023-10-10T20:19:23.503296+0000","last_peered":"2023-10-10T20:19:23.503296+0000","last_clean":"2023-10-10T20:19:23.503296+0000","last_became_active":"2023-10-10T20:19:22.490137+0000","last_became_peered":"2023-10-10T20:19:22.490137+0000","last_unstale":"2023-10-10T20:19:23.503296+0000","last_undegraded":"2023-10-10T20:19:23.503296+0000","last_fullsized":"2023-10-10T20:19:23.503296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:12.258553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495850+0000","last_change":"2023-10-10T20:19:23.495686+0000","last_active":"2023-10-10T20:19:23.495850+0000","last_peered":"2023-10-10T20:19:23.495850+0000","last_clean":"2023-10-10T20:19:23.495850+0000","last_became_active":"2023-10-10T20:19:23.495454+0000","last_became_peered":"2023-10-10T20:19:23.495454+0000","last_unstale":"2023-10-10T20:19:23.495850+0000","last_undegraded":"2023-10-10T20:19:23.495850+0000","last_fullsized":"2023-10-10T20:19:23.495850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:20.869423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.483607+0000","last_change":"2023-10-10T20:19:22.485497+0000","last_active":"2023-10-10T20:19:23.483607+0000","last_peered":"2023-10-10T20:19:23.483607+0000","last_clean":"2023-10-10T20:19:23.483607+0000","last_became_active":"2023-10-10T20:19:22.485276+0000","last_became_peered":"2023-10-10T20:19:22.485276+0000","last_unstale":"2023-10-10T20:19:23.483607+0000","last_undegraded":"2023-10-10T20:19:23.483607+0000","last_fullsized":"2023-10-10T20:19:23.483607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:48:00.391386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495682+0000","last_change":"2023-10-10T20:19:23.495616+0000","last_active":"2023-10-10T20:19:23.495682+0000","last_peered":"2023-10-10T20:19:23.495682+0000","last_clean":"2023-10-10T20:19:23.495682+0000","last_became_active":"2023-10-10T20:19:23.495385+0000","last_became_peered":"2023-10-10T20:19:23.495385+0000","last_unstale":"2023-10-10T20:19:23.495682+0000","last_undegraded":"2023-10-10T20:19:23.495682+0000","last_fullsized":"2023-10-10T20:19:23.495682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:23:34.998561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504015+0000","last_change":"2023-10-10T20:19:22.483831+0000","last_active":"2023-10-10T20:19:23.504015+0000","last_peered":"2023-10-10T20:19:23.504015+0000","last_clean":"2023-10-10T20:19:23.504015+0000","last_became_active":"2023-10-10T20:19:22.483604+0000","last_became_peered":"2023-10-10T20:19:22.483604+0000","last_unstale":"2023-10-10T20:19:23.504015+0000","last_undegraded":"2023-10-10T20:19:23.504015+0000","last_fullsized":"2023-10-10T20:19:23.504015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:30:27.084083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489338+0000","last_change":"2023-10-10T20:19:23.488221+0000","last_active":"2023-10-10T20:19:23.489338+0000","last_peered":"2023-10-10T20:19:23.489338+0000","last_clean":"2023-10-10T20:19:23.489338+0000","last_became_active":"2023-10-10T20:19:23.487992+0000","last_became_peered":"2023-10-10T20:19:23.487992+0000","last_unstale":"2023-10-10T20:19:23.489338+0000","last_undegraded":"2023-10-10T20:19:23.489338+0000","last_fullsized":"2023-10-10T20:19:23.489338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:31:30.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.504646+0000","last_change":"2023-10-10T20:19:22.488044+0000","last_active":"2023-10-10T20:19:23.504646+0000","last_peered":"2023-10-10T20:19:23.504646+0000","last_clean":"2023-10-10T20:19:23.504646+0000","last_became_active":"2023-10-10T20:19:22.487823+0000","last_became_peered":"2023-10-10T20:19:22.487823+0000","last_unstale":"2023-10-10T20:19:23.504646+0000","last_undegraded":"2023-10-10T20:19:23.504646+0000","last_fullsized":"2023-10-10T20:19:23.504646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:21.468881+0000","last_clean_scrub_stamp":"2023-10-10T20:19:21.468881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:14:42.873543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.494991+0000","last_change":"2023-10-10T20:19:23.494793+0000","last_active":"2023-10-10T20:19:23.494991+0000","last_peered":"2023-10-10T20:19:23.494991+0000","last_clean":"2023-10-10T20:19:23.494991+0000","last_became_active":"2023-10-10T20:19:23.494129+0000","last_became_peered":"2023-10-10T20:19:23.494129+0000","last_unstale":"2023-10-10T20:19:23.494991+0000","last_undegraded":"2023-10-10T20:19:23.494991+0000","last_fullsized":"2023-10-10T20:19:23.494991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.493767+0000","last_change":"2023-10-10T20:19:23.489565+0000","last_active":"2023-10-10T20:19:23.493767+0000","last_peered":"2023-10-10T20:19:23.493767+0000","last_clean":"2023-10-10T20:19:23.493767+0000","last_became_active":"2023-10-10T20:19:23.489263+0000","last_became_peered":"2023-10-10T20:19:23.489263+0000","last_unstale":"2023-10-10T20:19:23.493767+0000","last_undegraded":"2023-10-10T20:19:23.493767+0000","last_fullsized":"2023-10-10T20:19:23.493767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:01:28.658287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490726+0000","last_change":"2023-10-10T20:19:23.490528+0000","last_active":"2023-10-10T20:19:23.490726+0000","last_peered":"2023-10-10T20:19:23.490726+0000","last_clean":"2023-10-10T20:19:23.490726+0000","last_became_active":"2023-10-10T20:19:23.490284+0000","last_became_peered":"2023-10-10T20:19:23.490284+0000","last_unstale":"2023-10-10T20:19:23.490726+0000","last_undegraded":"2023-10-10T20:19:23.490726+0000","last_fullsized":"2023-10-10T20:19:23.490726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:14.397696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.490090+0000","last_change":"2023-10-10T20:19:23.489820+0000","last_active":"2023-10-10T20:19:23.490090+0000","last_peered":"2023-10-10T20:19:23.490090+0000","last_clean":"2023-10-10T20:19:23.490090+0000","last_became_active":"2023-10-10T20:19:23.489559+0000","last_became_peered":"2023-10-10T20:19:23.489559+0000","last_unstale":"2023-10-10T20:19:23.490090+0000","last_undegraded":"2023-10-10T20:19:23.490090+0000","last_fullsized":"2023-10-10T20:19:23.490090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:38.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.485528+0000","last_change":"2023-10-10T20:19:23.485467+0000","last_active":"2023-10-10T20:19:23.485528+0000","last_peered":"2023-10-10T20:19:23.485528+0000","last_clean":"2023-10-10T20:19:23.485528+0000","last_became_active":"2023-10-10T20:19:23.485195+0000","last_became_peered":"2023-10-10T20:19:23.485195+0000","last_unstale":"2023-10-10T20:19:23.485528+0000","last_undegraded":"2023-10-10T20:19:23.485528+0000","last_fullsized":"2023-10-10T20:19:23.485528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487121+0000","last_change":"2023-10-10T20:19:23.486697+0000","last_active":"2023-10-10T20:19:23.487121+0000","last_peered":"2023-10-10T20:19:23.487121+0000","last_clean":"2023-10-10T20:19:23.487121+0000","last_became_active":"2023-10-10T20:19:23.486167+0000","last_became_peered":"2023-10-10T20:19:23.486167+0000","last_unstale":"2023-10-10T20:19:23.487121+0000","last_undegraded":"2023-10-10T20:19:23.487121+0000","last_fullsized":"2023-10-10T20:19:23.487121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.488923+0000","last_change":"2023-10-10T20:19:23.488871+0000","last_active":"2023-10-10T20:19:23.488923+0000","last_peered":"2023-10-10T20:19:23.488923+0000","last_clean":"2023-10-10T20:19:23.488923+0000","last_became_active":"2023-10-10T20:19:23.488509+0000","last_became_peered":"2023-10-10T20:19:23.488509+0000","last_unstale":"2023-10-10T20:19:23.488923+0000","last_undegraded":"2023-10-10T20:19:23.488923+0000","last_fullsized":"2023-10-10T20:19:23.488923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:31.161105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.498221+0000","last_change":"2023-10-10T20:19:23.497323+0000","last_active":"2023-10-10T20:19:23.498221+0000","last_peered":"2023-10-10T20:19:23.498221+0000","last_clean":"2023-10-10T20:19:23.498221+0000","last_became_active":"2023-10-10T20:19:23.496823+0000","last_became_peered":"2023-10-10T20:19:23.496823+0000","last_unstale":"2023-10-10T20:19:23.498221+0000","last_undegraded":"2023-10-10T20:19:23.498221+0000","last_fullsized":"2023-10-10T20:19:23.498221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:45:00.258334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.489076+0000","last_change":"2023-10-10T20:19:23.488948+0000","last_active":"2023-10-10T20:19:23.489076+0000","last_peered":"2023-10-10T20:19:23.489076+0000","last_clean":"2023-10-10T20:19:23.489076+0000","last_became_active":"2023-10-10T20:19:23.488700+0000","last_became_peered":"2023-10-10T20:19:23.488700+0000","last_unstale":"2023-10-10T20:19:23.489076+0000","last_undegraded":"2023-10-10T20:19:23.489076+0000","last_fullsized":"2023-10-10T20:19:23.489076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:49.690601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.492260+0000","last_change":"2023-10-10T20:19:23.492008+0000","last_active":"2023-10-10T20:19:23.492260+0000","last_peered":"2023-10-10T20:19:23.492260+0000","last_clean":"2023-10-10T20:19:23.492260+0000","last_became_active":"2023-10-10T20:19:23.491368+0000","last_became_peered":"2023-10-10T20:19:23.491368+0000","last_unstale":"2023-10-10T20:19:23.492260+0000","last_undegraded":"2023-10-10T20:19:23.492260+0000","last_fullsized":"2023-10-10T20:19:23.492260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:37:12.239518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.487136+0000","last_change":"2023-10-10T20:19:23.486830+0000","last_active":"2023-10-10T20:19:23.487136+0000","last_peered":"2023-10-10T20:19:23.487136+0000","last_clean":"2023-10-10T20:19:23.487136+0000","last_became_active":"2023-10-10T20:19:23.486468+0000","last_became_peered":"2023-10-10T20:19:23.486468+0000","last_unstale":"2023-10-10T20:19:23.487136+0000","last_undegraded":"2023-10-10T20:19:23.487136+0000","last_fullsized":"2023-10-10T20:19:23.487136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:39.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.495811+0000","last_change":"2023-10-10T20:19:23.495638+0000","last_active":"2023-10-10T20:19:23.495811+0000","last_peered":"2023-10-10T20:19:23.495811+0000","last_clean":"2023-10-10T20:19:23.495811+0000","last_became_active":"2023-10-10T20:19:23.495020+0000","last_became_peered":"2023-10-10T20:19:23.495020+0000","last_unstale":"2023-10-10T20:19:23.495811+0000","last_undegraded":"2023-10-10T20:19:23.495811+0000","last_fullsized":"2023-10-10T20:19:23.495811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:58.338847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.491834+0000","last_change":"2023-10-10T20:19:23.491773+0000","last_active":"2023-10-10T20:19:23.491834+0000","last_peered":"2023-10-10T20:19:23.491834+0000","last_clean":"2023-10-10T20:19:23.491834+0000","last_became_active":"2023-10-10T20:19:23.491189+0000","last_became_peered":"2023-10-10T20:19:23.491189+0000","last_unstale":"2023-10-10T20:19:23.491834+0000","last_undegraded":"2023-10-10T20:19:23.491834+0000","last_fullsized":"2023-10-10T20:19:23.491834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-10T20:19:23.499362+0000","last_change":"2023-10-10T20:19:23.499299+0000","last_active":"2023-10-10T20:19:23.499362+0000","last_peered":"2023-10-10T20:19:23.499362+0000","last_clean":"2023-10-10T20:19:23.499362+0000","last_became_active":"2023-10-10T20:19:23.498950+0000","last_became_peered":"2023-10-10T20:19:23.498950+0000","last_unstale":"2023-10-10T20:19:23.499362+0000","last_undegraded":"2023-10-10T20:19:23.499362+0000","last_fullsized":"2023-10-10T20:19:23.499362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:22.472711+0000","last_clean_scrub_stamp":"2023-10-10T20:19:22.472711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:43:46.707133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929346,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":644,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339539,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":67,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199927970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929380,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929416,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:19:36.999 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-10T20:19:36.999 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-10T20:19:36.999 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-10T20:19:37.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-10T20:19:37.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.146+0000 7f6b27d5c700 1 Processor -- start 2023-10-10T20:19:37.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.146+0000 7f6b27d5c700 1 -- start start 2023-10-10T20:19:37.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b27d5c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b2006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b27d5c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2015c360 con 0x7f6b2015b0d0 2023-10-10T20:19:37.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b27d5c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2015b790 con 0x7f6b201577d0 2023-10-10T20:19:37.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b27d5c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015ba30 con 0x7f6b20157ba0 2023-10-10T20:19:37.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b262f9700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6b2015b0d0 0x7f6b2015c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58356/0 (socket says 172.21.15.148:58356) 2023-10-10T20:19:37.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.147+0000 7f6b262f9700 1 -- 172.21.15.148:0/1434745542 learned_addr learned my addr 172.21.15.148:0/1434745542 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b252f7700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b2006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227239626 0 0) 0x7f6b2015c360 con 0x7f6b2015b0d0 2023-10-10T20:19:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b0c003570 con 0x7f6b2015b0d0 2023-10-10T20:19:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877220217 0 0) 0x7f6b2015b790 con 0x7f6b201577d0 2023-10-10T20:19:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2015c360 con 0x7f6b201577d0 2023-10-10T20:19:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3796952442 0 0) 0x7f6b0c003570 con 0x7f6b2015b0d0 2023-10-10T20:19:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 msgr2=0x7f6b2006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b2006a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 >> v1:172.21.15.161:6789/0 conn(0x7f6b201577d0 legacy=0x7f6b200cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2015bd10 con 0x7f6b2015b0d0 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f6b18001470 con 0x7f6b2015b0d0 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 >> v1:172.21.15.148:6789/0 conn(0x7f6b2015b0d0 legacy=0x7f6b2015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0c003c20 0x7f6b0c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c004d50 0x7f6b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015ba30 con 0x7f6b0c003c20 2023-10-10T20:19:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c00bb70 con 0x7f6b0c004d50 2023-10-10T20:19:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c00bd30 con 0x7f6b20157ba0 2023-10-10T20:19:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0c003c20 0x7f6b0c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b252f7700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b262f9700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c004d50 0x7f6b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 -- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 msgr2=0x7f6b0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b20157ba0 0x7f6b0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 -- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c004d50 msgr2=0x7f6b0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c004d50 0x7f6b0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 -- 172.21.15.148:0/1434745542 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c003570 con 0x7f6b0c003c20 2023-10-10T20:19:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.148+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0c003c20 0x7f6b0c004010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002120 tx=0x7f6b1c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fdecdef3a18f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c001ab0 con 0x7f6b0c003c20 2023-10-10T20:19:37.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c00c980 con 0x7f6b0c003c20 2023-10-10T20:19:37.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b24af6700 1 -- 172.21.15.148:0/1434745542 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00cd20 con 0x7f6b0c003c20 2023-10-10T20:19:37.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1434745542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0c003c20 msgr2=0x7f6b0c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1434745542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0c003c20 0x7f6b0c004010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002120 tx=0x7f6b1c0017a0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b262f9700 1 -- 172.21.15.148:0/1434745542 reap_dead start 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1434745542 shutdown_connections 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1434745542 >> 172.21.15.148:0/1434745542 conn(0x7f6b20068ce0 msgr2=0x7f6b200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1434745542 shutdown_connections 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.149+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1434745542 wait complete. 2023-10-10T20:19:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.150+0000 7f6b27d5c700 1 Processor -- start 2023-10-10T20:19:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.150+0000 7f6b27d5c700 1 -- start start 2023-10-10T20:19:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c003c20 0x7f6b2016b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b2014e610 0x7f6b20150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2015ddb0 con 0x7f6b2013a0a0 2023-10-10T20:19:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2014ea40 con 0x7f6b0c003c20 2023-10-10T20:19:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b27d5c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2013a510 con 0x7f6b2014e610 2023-10-10T20:19:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42622/0 (socket says 172.21.15.148:42622) 2023-10-10T20:19:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 -- 172.21.15.148:0/1568114595 learned_addr learned my addr 172.21.15.148:0/1568114595 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c003c20 0x7f6b2016b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 -- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b2014e610 msgr2=0x7f6b20150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b2014e610 0x7f6b20150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 -- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c003c20 msgr2=0x7f6b2016b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c003c20 0x7f6b2016b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.151+0000 7f6b252f7700 1 -- 172.21.15.148:0/1568114595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b201510a0 con 0x7f6b2013a0a0 2023-10-10T20:19:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b252f7700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b10011d40 tx=0x7f6b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000fac0 con 0x7f6b2013a0a0 2023-10-10T20:19:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000fc20 con 0x7f6b2013a0a0 2023-10-10T20:19:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b20151380 con 0x7f6b2013a0a0 2023-10-10T20:19:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10014440 con 0x7f6b2013a0a0 2023-10-10T20:19:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.152+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2015da40 con 0x7f6b2013a0a0 2023-10-10T20:19:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.153+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2004c3c0 con 0x7f6b2013a0a0 2023-10-10T20:19:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.153+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b10014870 con 0x7f6b2013a0a0 2023-10-10T20:19:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.154+0000 7f6b15ffb700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 0x7f6b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.154+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 0x7f6b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.154+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b10018030 con 0x7f6b2013a0a0 2023-10-10T20:19:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.154+0000 7f6b25af8700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 0x7f6b040693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000f80 tx=0x7f6b1c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.158+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b10052350 con 0x7f6b2013a0a0 2023-10-10T20:19:37.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.380+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6b2015bff0 con 0x7f6b2013a0a0 2023-10-10T20:19:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.381+0000 7f6b15ffb700 1 -- 172.21.15.148:0/1568114595 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6b10055970 con 0x7f6b2013a0a0 2023-10-10T20:19:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:37.383 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-10T20:19:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 msgr2=0x7f6b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 0x7f6b040693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000f80 tx=0x7f6b1c016000 comp rx=0 tx=0).stop 2023-10-10T20:19:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 msgr2=0x7f6b2014e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b10011d40 tx=0x7f6b100091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 shutdown_connections 2023-10-10T20:19:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6b04066f30 0x7f6b040693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b2014e610 0x7f6b20150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b0c003c20 0x7f6b2016b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 --2- 172.21.15.148:0/1568114595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b2013a0a0 0x7f6b2014e0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 >> 172.21.15.148:0/1568114595 conn(0x7f6b20068ce0 msgr2=0x7f6b2014d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 shutdown_connections 2023-10-10T20:19:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.383+0000 7f6b27d5c700 1 -- 172.21.15.148:0/1568114595 wait complete. 2023-10-10T20:19:37.397 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-10T20:19:37.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:37.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.510+0000 7f12b8b60700 1 Processor -- start 2023-10-10T20:19:37.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.510+0000 7f12b8b60700 1 -- start start 2023-10-10T20:19:37.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b8b60700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156e20 0x7f12b4067b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b8b60700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12b40680b0 0x7f12b40adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b8b60700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40c87d0 con 0x7f12b4156a50 2023-10-10T20:19:37.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b8b60700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b4068d10 con 0x7f12b40680b0 2023-10-10T20:19:37.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b8b60700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b406e650 con 0x7f12b4156e20 2023-10-10T20:19:37.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b2d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12b40680b0 0x7f12b40adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b2d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f12b40680b0 0x7f12b40adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42632/0 (socket says 172.21.15.148:42632) 2023-10-10T20:19:37.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.511+0000 7f12b2d9d700 1 -- 172.21.15.148:0/680966715 learned_addr learned my addr 172.21.15.148:0/680966715 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12a9d9b700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156e20 0x7f12b4067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b2d9d700 1 -- 172.21.15.148:0/680966715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156e20 msgr2=0x7f12b4067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b2d9d700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156e20 0x7f12b4067b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b2d9d700 1 -- 172.21.15.148:0/680966715 >> v1:172.21.15.161:6789/0 conn(0x7f12b4156a50 legacy=0x7f12b40c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b2d9d700 1 -- 172.21.15.148:0/680966715 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4068750 con 0x7f12b40680b0 2023-10-10T20:19:37.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b2d9d700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b40680b0 0x7f12b40adab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009f20 tx=0x7f12ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80061a4321362f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b1d9b700 1 -- 172.21.15.148:0/680966715 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00e070 con 0x7f12b40680b0 2023-10-10T20:19:37.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b1d9b700 1 -- 172.21.15.148:0/680966715 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac00ba80 con 0x7f12b40680b0 2023-10-10T20:19:37.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b1d9b700 1 -- 172.21.15.148:0/680966715 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0092e0 con 0x7f12b40680b0 2023-10-10T20:19:37.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 -- 172.21.15.148:0/680966715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b40680b0 msgr2=0x7f12b40adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b40680b0 0x7f12b40adab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009f20 tx=0x7f12ac009840 comp rx=0 tx=0).stop 2023-10-10T20:19:37.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 -- 172.21.15.148:0/680966715 shutdown_connections 2023-10-10T20:19:37.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b40680b0 0x7f12b40adab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 --2- 172.21.15.148:0/680966715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156e20 0x7f12b4067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.512+0000 7f12b8b60700 1 -- 172.21.15.148:0/680966715 >> 172.21.15.148:0/680966715 conn(0x7f12b406a250 msgr2=0x7f12b406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:37.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.513+0000 7f12b8b60700 1 -- 172.21.15.148:0/680966715 shutdown_connections 2023-10-10T20:19:37.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.513+0000 7f12b8b60700 1 -- 172.21.15.148:0/680966715 wait complete. 2023-10-10T20:19:37.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.514+0000 7f12b8b60700 1 Processor -- start 2023-10-10T20:19:37.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.514+0000 7f12b8b60700 1 -- start start 2023-10-10T20:19:37.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12b40680b0 0x7f12b40aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156a50 0x7f12b40c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b0ba0 con 0x7f12b4156e20 2023-10-10T20:19:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40ab160 con 0x7f12b40680b0 2023-10-10T20:19:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40ab440 con 0x7f12b4156a50 2023-10-10T20:19:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42644/0 (socket says 172.21.15.148:42644) 2023-10-10T20:19:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12a9d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156a50 0x7f12b40c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 -- 172.21.15.148:0/3394007127 learned_addr learned my addr 172.21.15.148:0/3394007127 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:37.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b259c700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12b40680b0 0x7f12b40aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 -- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156a50 msgr2=0x7f12b40c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156a50 0x7f12b40c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 -- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12b40680b0 msgr2=0x7f12b40aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12b40680b0 0x7f12b40aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 -- 172.21.15.148:0/3394007127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac009ae0 con 0x7f12b4156e20 2023-10-10T20:19:37.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b2d9d700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12b40cd880 tx=0x7f12ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0032f0 con 0x7f12b4156e20 2023-10-10T20:19:37.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac008d20 con 0x7f12b4156e20 2023-10-10T20:19:37.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac025690 con 0x7f12b4156e20 2023-10-10T20:19:37.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b40ab720 con 0x7f12b4156e20 2023-10-10T20:19:37.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.515+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b40c2320 con 0x7f12b4156e20 2023-10-10T20:19:37.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.516+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b40b0900 con 0x7f12b4156e20 2023-10-10T20:19:37.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.516+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12ac003040 con 0x7f12b4156e20 2023-10-10T20:19:37.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.517+0000 7f12aaffd700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 0x7f12940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.517+0000 7f12b259c700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 0x7f12940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.517+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12ac0860b0 con 0x7f12b4156e20 2023-10-10T20:19:37.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.517+0000 7f12b259c700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 0x7f12940693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12a40039e0 tx=0x7f12a4007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.521+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12b40b0900 con 0x7f12b4156e20 2023-10-10T20:19:37.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.710+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b40b0ce0 con 0x7f12b4156e20 2023-10-10T20:19:37.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.711+0000 7f12aaffd700 1 -- 172.21.15.148:0/3394007127 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f12b40b0ce0 con 0x7f12b4156e20 2023-10-10T20:19:37.713 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:37.714 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 msgr2=0x7f12940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 0x7f12940693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12a40039e0 tx=0x7f12a4007ec0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 msgr2=0x7f12b40708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12b40cd880 tx=0x7f12ac018000 comp rx=0 tx=0).stop 2023-10-10T20:19:37.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 shutdown_connections 2023-10-10T20:19:37.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1294066f30 0x7f12940693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b4156e20 0x7f12b40708c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12b40680b0 0x7f12b40aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 --2- 172.21.15.148:0/3394007127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12b4156a50 0x7f12b40c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 >> 172.21.15.148:0/3394007127 conn(0x7f12b406a250 msgr2=0x7f12b40afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:37.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 shutdown_connections 2023-10-10T20:19:37.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.713+0000 7f12b8b60700 1 -- 172.21.15.148:0/3394007127 wait complete. 2023-10-10T20:19:37.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:37.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:37.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.844+0000 7f86ef4d3700 1 Processor -- start 2023-10-10T20:19:37.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.844+0000 7f86ef4d3700 1 -- start start 2023-10-10T20:19:37.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ef4d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86e80b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ef4d3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80b1850 con 0x7f86e80ce9e0 2023-10-10T20:19:37.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ef4d3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80b20f0 con 0x7f86e80b0460 2023-10-10T20:19:37.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ef4d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80cf550 con 0x7f86e80adc80 2023-10-10T20:19:37.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86eecd2700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f86e80ce9e0 0x7f86e80cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58372/0 (socket says 172.21.15.148:58372) 2023-10-10T20:19:37.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86eecd2700 1 -- 172.21.15.148:0/2495405324 learned_addr learned my addr 172.21.15.148:0/2495405324 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ee4d1700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86e80b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853584165 0 0) 0x7f86e80b1850 con 0x7f86e80ce9e0 2023-10-10T20:19:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d4003570 con 0x7f86e80ce9e0 2023-10-10T20:19:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409495966 0 0) 0x7f86d4003570 con 0x7f86e80ce9e0 2023-10-10T20:19:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 msgr2=0x7f86e80b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86e80b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 >> v1:172.21.15.161:6789/0 conn(0x7f86e80b0460 legacy=0x7f86e80b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80cf750 con 0x7f86e80ce9e0 2023-10-10T20:19:37.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.845+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f86e00017f0 con 0x7f86e80ce9e0 2023-10-10T20:19:37.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 >> v1:172.21.15.148:6789/0 conn(0x7f86e80ce9e0 legacy=0x7f86e80cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86d4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86d4004d50 0x7f86d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80cf550 con 0x7f86d4004010 2023-10-10T20:19:37.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d400bbf0 con 0x7f86d4004d50 2023-10-10T20:19:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d400bdb0 con 0x7f86e80adc80 2023-10-10T20:19:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ee4d1700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86d4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86eecd2700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86d4004d50 0x7f86d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 -- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 msgr2=0x7f86d4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80adc80 0x7f86d4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 -- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86d4004d50 msgr2=0x7f86d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:37.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86d4004d50 0x7f86d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 -- 172.21.15.148:0/2495405324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4003570 con 0x7f86d4004010 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86edcd0700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86d4004810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86d8001880 tx=0x7f86d8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d9a6feaf5f2e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8012420 con 0x7f86d4004010 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8012580 con 0x7f86d4004010 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.846+0000 7f86ed4cf700 1 -- 172.21.15.148:0/2495405324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8012920 con 0x7f86d4004010 2023-10-10T20:19:37.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 -- 172.21.15.148:0/2495405324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 msgr2=0x7f86d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/2495405324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86d4004810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86d8001880 tx=0x7f86d8007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86eecd2700 1 -- 172.21.15.148:0/2495405324 reap_dead start 2023-10-10T20:19:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 -- 172.21.15.148:0/2495405324 shutdown_connections 2023-10-10T20:19:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 -- 172.21.15.148:0/2495405324 >> 172.21.15.148:0/2495405324 conn(0x7f86e806c700 msgr2=0x7f86e8069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:37.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 -- 172.21.15.148:0/2495405324 shutdown_connections 2023-10-10T20:19:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.847+0000 7f86ef4d3700 1 -- 172.21.15.148:0/2495405324 wait complete. 2023-10-10T20:19:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.848+0000 7f86ef4d3700 1 Processor -- start 2023-10-10T20:19:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.848+0000 7f86ef4d3700 1 -- start start 2023-10-10T20:19:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e80aa8d0 0x7f86e80aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80ac8f0 0x7f86e80acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80ab280 con 0x7f86d4004010 2023-10-10T20:19:37.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8146390 con 0x7f86e80aa8d0 2023-10-10T20:19:37.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8146670 con 0x7f86e80ac8f0 2023-10-10T20:19:37.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42660/0 (socket says 172.21.15.148:42660) 2023-10-10T20:19:37.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 -- 172.21.15.148:0/1773391982 learned_addr learned my addr 172.21.15.148:0/1773391982 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:37.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86eecd2700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80ac8f0 0x7f86e80acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86edcd0700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e80aa8d0 0x7f86e80aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 -- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80ac8f0 msgr2=0x7f86e80acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80ac8f0 0x7f86e80acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 -- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e80aa8d0 msgr2=0x7f86e80aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:37.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e80aa8d0 0x7f86e80aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:37.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 -- 172.21.15.148:0/1773391982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8146950 con 0x7f86d4004010 2023-10-10T20:19:37.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ee4d1700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86e4011d40 tx=0x7f86e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400fac0 con 0x7f86d4004010 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e400fc20 con 0x7f86d4004010 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e8146c30 con 0x7f86d4004010 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4014440 con 0x7f86d4004010 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.849+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e8147430 con 0x7f86d4004010 2023-10-10T20:19:37.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.850+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e804c3c0 con 0x7f86d4004010 2023-10-10T20:19:37.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.851+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86e4014870 con 0x7f86d4004010 2023-10-10T20:19:37.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.851+0000 7f86de7fc700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 0x7f86cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:37.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.851+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86e4018030 con 0x7f86d4004010 2023-10-10T20:19:37.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.851+0000 7f86edcd0700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 0x7f86cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:37.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.852+0000 7f86edcd0700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 0x7f86cc0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d8008fd0 tx=0x7f86d8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:37.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:37.855+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86e4052280 con 0x7f86d4004010 2023-10-10T20:19:38.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.036+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86e80b1aa0 con 0x7f86d4004010 2023-10-10T20:19:38.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.037+0000 7f86de7fc700 1 -- 172.21.15.148:0/1773391982 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86e40558a0 con 0x7f86d4004010 2023-10-10T20:19:38.039 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:38.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 msgr2=0x7f86cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 0x7f86cc0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d8008fd0 tx=0x7f86d8015000 comp rx=0 tx=0).stop 2023-10-10T20:19:38.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 msgr2=0x7f86e814a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86e4011d40 tx=0x7f86e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 shutdown_connections 2023-10-10T20:19:38.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86cc066f30 0x7f86cc0693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86e80ac8f0 0x7f86e80acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86d4004010 0x7f86e814a400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 --2- 172.21.15.148:0/1773391982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e80aa8d0 0x7f86e80aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 >> 172.21.15.148:0/1773391982 conn(0x7f86e806c700 msgr2=0x7f86e81572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 shutdown_connections 2023-10-10T20:19:38.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.039+0000 7f86ef4d3700 1 -- 172.21.15.148:0/1773391982 wait complete. 2023-10-10T20:19:38.053 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:38.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.167+0000 7f8f289a7700 1 Processor -- start 2023-10-10T20:19:38.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.167+0000 7f8f289a7700 1 -- start start 2023-10-10T20:19:38.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f289a7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f24159ec0 0x7f8f240af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f289a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f240afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f289a7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240b0790 con 0x7f8f2415a290 2023-10-10T20:19:38.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f289a7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f240ce1f0 con 0x7f8f24159ec0 2023-10-10T20:19:38.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f289a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240b0b20 con 0x7f8f2406acb0 2023-10-10T20:19:38.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f21d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8f2415a290 0x7f8f2406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58378/0 (socket says 172.21.15.148:58378) 2023-10-10T20:19:38.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f21d9b700 1 -- 172.21.15.148:0/1858107138 learned_addr learned my addr 172.21.15.148:0/1858107138 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:38.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2259c700 1 --2- 172.21.15.148:0/1858107138 >> v2:172.21.15.161:3300/0 conn(0x7f8f24159ec0 0x7f8f240af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f240afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071837539 0 0) 0x7f8f240b0790 con 0x7f8f2415a290 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f10003570 con 0x7f8f2415a290 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2529640468 0 0) 0x7f8f10003570 con 0x7f8f2415a290 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 msgr2=0x7f8f240afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f240afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 >> v2:172.21.15.161:3300/0 conn(0x7f8f24159ec0 msgr2=0x7f8f240af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 --2- 172.21.15.148:0/1858107138 >> v2:172.21.15.161:3300/0 conn(0x7f8f24159ec0 0x7f8f240af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240b04b0 con 0x7f8f2415a290 2023-10-10T20:19:38.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f8f0c001470 con 0x7f8f2415a290 2023-10-10T20:19:38.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 >> v1:172.21.15.148:6789/0 conn(0x7f8f2415a290 legacy=0x7f8f2406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f10003a60 0x7f8f10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240ce1f0 con 0x7f8f10004d50 2023-10-10T20:19:38.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240b0b20 con 0x7f8f10003a60 2023-10-10T20:19:38.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.168+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1000ba60 con 0x7f8f2406acb0 2023-10-10T20:19:38.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f2259c700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f10003a60 0x7f8f10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f21d9b700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 -- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 msgr2=0x7f8f10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f2406acb0 0x7f8f10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 -- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f10003a60 msgr2=0x7f8f10003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f10003a60 0x7f8f10003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 -- 172.21.15.148:0/1858107138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f10003570 con 0x7f8f10004d50 2023-10-10T20:19:38.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f100051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cbc50 tx=0x7f8f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0860a165debd463 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400e680 con 0x7f8f10004d50 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1400e7e0 con 0x7f8f10004d50 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.169+0000 7f8f2159a700 1 -- 172.21.15.148:0/1858107138 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400eba0 con 0x7f8f10004d50 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 -- 172.21.15.148:0/1858107138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 msgr2=0x7f8f100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 --2- 172.21.15.148:0/1858107138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f100051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cbc50 tx=0x7f8f140091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f22d9d700 1 -- 172.21.15.148:0/1858107138 reap_dead start 2023-10-10T20:19:38.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 -- 172.21.15.148:0/1858107138 shutdown_connections 2023-10-10T20:19:38.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 -- 172.21.15.148:0/1858107138 >> 172.21.15.148:0/1858107138 conn(0x7f8f240d1880 msgr2=0x7f8f240cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 -- 172.21.15.148:0/1858107138 shutdown_connections 2023-10-10T20:19:38.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.170+0000 7f8f289a7700 1 -- 172.21.15.148:0/1858107138 wait complete. 2023-10-10T20:19:38.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.171+0000 7f8f289a7700 1 Processor -- start 2023-10-10T20:19:38.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.171+0000 7f8f289a7700 1 -- start start 2023-10-10T20:19:38.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.171+0000 7f8f289a7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.171+0000 7f8f289a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f240afdf0 0x7f8f240ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f241555b0 0x7f8f24155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2415cfc0 con 0x7f8f10004d50 2023-10-10T20:19:38.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240b5310 con 0x7f8f240afdf0 2023-10-10T20:19:38.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24155f60 con 0x7f8f241555b0 2023-10-10T20:19:38.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42668/0 (socket says 172.21.15.148:42668) 2023-10-10T20:19:38.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 -- 172.21.15.148:0/2893307384 learned_addr learned my addr 172.21.15.148:0/2893307384 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:38.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f22d9d700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f241555b0 0x7f8f24155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f21d9b700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f240afdf0 0x7f8f240ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 -- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f241555b0 msgr2=0x7f8f24155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f241555b0 0x7f8f24155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 -- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f240afdf0 msgr2=0x7f8f240ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f240afdf0 0x7f8f240ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 -- 172.21.15.148:0/2893307384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2415ca00 con 0x7f8f10004d50 2023-10-10T20:19:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f2259c700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c011e80 tx=0x7f8f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00e040 con 0x7f8f10004d50 2023-10-10T20:19:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c00f040 con 0x7f8f10004d50 2023-10-10T20:19:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00fe30 con 0x7f8f10004d50 2023-10-10T20:19:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2415cce0 con 0x7f8f10004d50 2023-10-10T20:19:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.172+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f240cade0 con 0x7f8f10004d50 2023-10-10T20:19:38.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.173+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240b5160 con 0x7f8f10004d50 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.173+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f1c015640 con 0x7f8f10004d50 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.174+0000 7f8f1a7fc700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.174+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f1c085130 con 0x7f8f10004d50 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.174+0000 7f8f21d9b700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.174+0000 7f8f21d9b700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 0x7f8f040693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f2416d4a0 tx=0x7f8f0c001e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.178+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f1c055e30 con 0x7f8f10004d50 2023-10-10T20:19:38.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.358+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f2404c3c0 con 0x7f8f10004d50 2023-10-10T20:19:38.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.358+0000 7f8f1a7fc700 1 -- 172.21.15.148:0/2893307384 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f1c0559c0 con 0x7f8f10004d50 2023-10-10T20:19:38.361 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.360+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 msgr2=0x7f8f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.360+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 0x7f8f040693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f2416d4a0 tx=0x7f8f0c001e90 comp rx=0 tx=0).stop 2023-10-10T20:19:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.360+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 msgr2=0x7f8f240c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.360+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c011e80 tx=0x7f8f1c00b040 comp rx=0 tx=0).stop 2023-10-10T20:19:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 shutdown_connections 2023-10-10T20:19:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f241555b0 0x7f8f24155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f10004d50 0x7f8f240c7e60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 --2- 172.21.15.148:0/2893307384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f240afdf0 0x7f8f240ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 >> 172.21.15.148:0/2893307384 conn(0x7f8f240d1880 msgr2=0x7f8f240cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 shutdown_connections 2023-10-10T20:19:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.361+0000 7f8f289a7700 1 -- 172.21.15.148:0/2893307384 wait complete. 2023-10-10T20:19:38.374 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:38.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 Processor -- start 2023-10-10T20:19:38.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 -- start start 2023-10-10T20:19:38.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a9806a5f0 0x7f5a980cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980686f0 con 0x7f5a98068b80 2023-10-10T20:19:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a98067ae0 con 0x7f5a9806a220 2023-10-10T20:19:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.490+0000 7f5a9c846700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a98068fc0 con 0x7f5a9806a5f0 2023-10-10T20:19:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9759e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5a98068b80 0x7f5a981563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58384/0 (socket says 172.21.15.148:58384) 2023-10-10T20:19:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9759e700 1 -- 172.21.15.148:0/2072309867 learned_addr learned my addr 172.21.15.148:0/2072309867 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:38.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9659c700 1 --2- 172.21.15.148:0/2072309867 >> v2:172.21.15.161:3300/0 conn(0x7f5a9806a5f0 0x7f5a980cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a95d9b700 1 -- 172.21.15.148:0/2072309867 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632990347 0 0) 0x7f5a980686f0 con 0x7f5a98068b80 2023-10-10T20:19:38.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a95d9b700 1 -- 172.21.15.148:0/2072309867 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a84003570 con 0x7f5a98068b80 2023-10-10T20:19:38.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9659c700 1 -- 172.21.15.148:0/2072309867 >> v1:172.21.15.161:6789/0 conn(0x7f5a9806a220 legacy=0x7f5a980cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9659c700 1 -- 172.21.15.148:0/2072309867 >> v1:172.21.15.148:6789/0 conn(0x7f5a98068b80 legacy=0x7f5a981563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9659c700 1 -- 172.21.15.148:0/2072309867 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98067dc0 con 0x7f5a9806a5f0 2023-10-10T20:19:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a9659c700 1 --2- 172.21.15.148:0/2072309867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a9806a5f0 0x7f5a980cc590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a800098e0 tx=0x7f5a800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e538c8e6d13bc363 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a95d9b700 1 -- 172.21.15.148:0/2072309867 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000b640 con 0x7f5a9806a5f0 2023-10-10T20:19:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a95d9b700 1 -- 172.21.15.148:0/2072309867 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8000b7a0 con 0x7f5a9806a5f0 2023-10-10T20:19:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.491+0000 7f5a95d9b700 1 -- 172.21.15.148:0/2072309867 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000bb20 con 0x7f5a9806a5f0 2023-10-10T20:19:38.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 -- 172.21.15.148:0/2072309867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a9806a5f0 msgr2=0x7f5a980cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2072309867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a9806a5f0 0x7f5a980cc590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a800098e0 tx=0x7f5a800091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 -- 172.21.15.148:0/2072309867 shutdown_connections 2023-10-10T20:19:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2072309867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a9806a5f0 0x7f5a980cc590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 -- 172.21.15.148:0/2072309867 >> 172.21.15.148:0/2072309867 conn(0x7f5a980cd940 msgr2=0x7f5a980cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 -- 172.21.15.148:0/2072309867 shutdown_connections 2023-10-10T20:19:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.492+0000 7f5a9c846700 1 -- 172.21.15.148:0/2072309867 wait complete. 2023-10-10T20:19:38.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.493+0000 7f5a9c846700 1 Processor -- start 2023-10-10T20:19:38.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.493+0000 7f5a9c846700 1 -- start start 2023-10-10T20:19:38.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a98068b80 0x7f5a98169150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a9806a5f0 0x7f5a98158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98168fe0 con 0x7f5a9806a220 2023-10-10T20:19:38.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98151020 con 0x7f5a98068b80 2023-10-10T20:19:38.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9c846700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981683d0 con 0x7f5a9806a5f0 2023-10-10T20:19:38.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42676/0 (socket says 172.21.15.148:42676) 2023-10-10T20:19:38.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 -- 172.21.15.148:0/2547835235 learned_addr learned my addr 172.21.15.148:0/2547835235 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:38.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9759e700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a9806a5f0 0x7f5a98158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a96d9d700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a98068b80 0x7f5a98169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 -- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a9806a5f0 msgr2=0x7f5a98158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a9806a5f0 0x7f5a98158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 -- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a98068b80 msgr2=0x7f5a98169150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a98068b80 0x7f5a98169150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 -- 172.21.15.148:0/2547835235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80009420 con 0x7f5a9806a220 2023-10-10T20:19:38.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a9659c700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009f80 tx=0x7f5a80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.494+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000bb20 con 0x7f5a9806a220 2023-10-10T20:19:38.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.495+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a80000cc0 con 0x7f5a9806a220 2023-10-10T20:19:38.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.495+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80024690 con 0x7f5a9806a220 2023-10-10T20:19:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.495+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a981686b0 con 0x7f5a9806a220 2023-10-10T20:19:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.495+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a98168c70 con 0x7f5a9806a220 2023-10-10T20:19:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.495+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a98153740 con 0x7f5a9806a220 2023-10-10T20:19:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.496+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a80024ac0 con 0x7f5a9806a220 2023-10-10T20:19:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.496+0000 7f5a8effd700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.496+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a80085f60 con 0x7f5a9806a220 2023-10-10T20:19:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.496+0000 7f5a96d9d700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.497+0000 7f5a96d9d700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 0x7f5a780693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a88007fd0 tx=0x7f5a88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.501+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a98153740 con 0x7f5a9806a220 2023-10-10T20:19:38.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.683+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a98153740 con 0x7f5a9806a220 2023-10-10T20:19:38.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.683+0000 7f5a8effd700 1 -- 172.21.15.148:0/2547835235 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a98153740 con 0x7f5a9806a220 2023-10-10T20:19:38.686 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:38.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 msgr2=0x7f5a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 0x7f5a780693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a88007fd0 tx=0x7f5a88009040 comp rx=0 tx=0).stop 2023-10-10T20:19:38.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 msgr2=0x7f5a980710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009f80 tx=0x7f5a80017000 comp rx=0 tx=0).stop 2023-10-10T20:19:38.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 shutdown_connections 2023-10-10T20:19:38.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a78066f30 0x7f5a780693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a9806a5f0 0x7f5a98158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a98068b80 0x7f5a98169150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 --2- 172.21.15.148:0/2547835235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a9806a220 0x7f5a980710c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.685+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 >> 172.21.15.148:0/2547835235 conn(0x7f5a980cd940 msgr2=0x7f5a980ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.686+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 shutdown_connections 2023-10-10T20:19:38.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.686+0000 7f5a9c846700 1 -- 172.21.15.148:0/2547835235 wait complete. 2023-10-10T20:19:38.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:38.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 Processor -- start 2023-10-10T20:19:38.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 -- start start 2023-10-10T20:19:38.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3400157870 0x7f340015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f340015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340015cbb0 con 0x7f340015d580 2023-10-10T20:19:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340015bfa0 con 0x7f3400157870 2023-10-10T20:19:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f34050da700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340015c280 con 0x7f340015a620 2023-10-10T20:19:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.813+0000 7f33ff59e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f340015d580 0x7f3400158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58396/0 (socket says 172.21.15.148:58396) 2023-10-10T20:19:38.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33ff59e700 1 -- 172.21.15.148:0/173720179 learned_addr learned my addr 172.21.15.148:0/173720179 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:38.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357471144 0 0) 0x7f340015cbb0 con 0x7f340015d580 2023-10-10T20:19:38.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33ec003570 con 0x7f340015d580 2023-10-10T20:19:38.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f340015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fed9d700 1 --2- 172.21.15.148:0/173720179 >> v2:172.21.15.161:3300/0 conn(0x7f3400157870 0x7f340015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2100626128 0 0) 0x7f33ec003570 con 0x7f340015d580 2023-10-10T20:19:38.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 msgr2=0x7f340015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f340015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 >> v2:172.21.15.161:3300/0 conn(0x7f3400157870 msgr2=0x7f340015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 --2- 172.21.15.148:0/173720179 >> v2:172.21.15.161:3300/0 conn(0x7f3400157870 0x7f340015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340015c560 con 0x7f340015d580 2023-10-10T20:19:38.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f33f80017f0 con 0x7f340015d580 2023-10-10T20:19:38.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 >> v1:172.21.15.148:6789/0 conn(0x7f340015d580 legacy=0x7f3400158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f33ec003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec003fe0 0x7f33ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33ec004d50 0x7f33ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340015bfa0 con 0x7f33ec003fe0 2023-10-10T20:19:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340015c280 con 0x7f33ec004d50 2023-10-10T20:19:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec00ba90 con 0x7f340015a620 2023-10-10T20:19:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec003fe0 0x7f33ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33fed9d700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f33ec003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33ff59e700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33ec004d50 0x7f33ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 -- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 msgr2=0x7f33ec003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f340015a620 0x7f33ec003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 -- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33ec004d50 msgr2=0x7f33ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:38.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33ec004d50 0x7f33ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33f659c700 1 -- 172.21.15.148:0/173720179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec003570 con 0x7f33ec003fe0 2023-10-10T20:19:38.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.814+0000 7f33ff59e700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33ec004d50 0x7f33ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:38.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f33f659c700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec003fe0 0x7f33ec004810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f340015aa20 tx=0x7f33e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d3f0409b4ef04 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800e6b0 con 0x7f33ec003fe0 2023-10-10T20:19:38.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800e810 con 0x7f33ec003fe0 2023-10-10T20:19:38.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f33fe59c700 1 -- 172.21.15.148:0/173720179 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800ebd0 con 0x7f33ec003fe0 2023-10-10T20:19:38.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 -- 172.21.15.148:0/173720179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec003fe0 msgr2=0x7f33ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 --2- 172.21.15.148:0/173720179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ec003fe0 0x7f33ec004810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f340015aa20 tx=0x7f33e80091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f33ff59e700 1 -- 172.21.15.148:0/173720179 reap_dead start 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 -- 172.21.15.148:0/173720179 shutdown_connections 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 -- 172.21.15.148:0/173720179 >> 172.21.15.148:0/173720179 conn(0x7f3400068ce0 msgr2=0x7f34000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 -- 172.21.15.148:0/173720179 shutdown_connections 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.815+0000 7f34050da700 1 -- 172.21.15.148:0/173720179 wait complete. 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.816+0000 7f34050da700 1 Processor -- start 2023-10-10T20:19:38.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.816+0000 7f34050da700 1 -- start start 2023-10-10T20:19:38.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f33ec003fe0 0x7f340014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 0x7f34000b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b1710 con 0x7f3400159090 2023-10-10T20:19:38.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b24b0 con 0x7f34000b1b00 2023-10-10T20:19:38.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f34050da700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b1280 con 0x7f33ec003fe0 2023-10-10T20:19:38.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f33f659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f33f659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42700/0 (socket says 172.21.15.148:42700) 2023-10-10T20:19:38.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f33f659c700 1 -- 172.21.15.148:0/577717399 learned_addr learned my addr 172.21.15.148:0/577717399 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:38.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f33fed9d700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f33ec003fe0 0x7f340014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.817+0000 7f33ff59e700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 0x7f34000b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 -- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f33ec003fe0 msgr2=0x7f340014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f33ec003fe0 0x7f340014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 -- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 msgr2=0x7f34000b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 0x7f34000b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 -- 172.21.15.148:0/577717399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000b06b0 con 0x7f3400159090 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33ff59e700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 0x7f34000b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f659c700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f33e800e4b0 tx=0x7f33e801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8022070 con 0x7f3400159090 2023-10-10T20:19:38.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800e980 con 0x7f3400159090 2023-10-10T20:19:38.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34000b0950 con 0x7f3400159090 2023-10-10T20:19:38.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8027470 con 0x7f3400159090 2023-10-10T20:19:38.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.818+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34000b0f10 con 0x7f3400159090 2023-10-10T20:19:38.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.819+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340004c3c0 con 0x7f3400159090 2023-10-10T20:19:38.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.819+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33e801c450 con 0x7f3400159090 2023-10-10T20:19:38.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.820+0000 7f33f77fe700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 0x7f33e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:38.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.820+0000 7f33fed9d700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 0x7f33e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:38.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.820+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f33e80860b0 con 0x7f3400159090 2023-10-10T20:19:38.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.820+0000 7f33fed9d700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 0x7f33e00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006f00 tx=0x7f33f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:38.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:38.824+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33e8053320 con 0x7f3400159090 2023-10-10T20:19:39.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.050+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34000b13c0 con 0x7f3400159090 2023-10-10T20:19:39.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.051+0000 7f33f77fe700 1 -- 172.21.15.148:0/577717399 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f33e8017090 con 0x7f3400159090 2023-10-10T20:19:39.053 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:39.053 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:39.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.052+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 msgr2=0x7f33e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 0x7f33e00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006f00 tx=0x7f33f0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:39.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 msgr2=0x7f3400159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f33e800e4b0 tx=0x7f33e801a000 comp rx=0 tx=0).stop 2023-10-10T20:19:39.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 shutdown_connections 2023-10-10T20:19:39.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f33e0066f30 0x7f33e00693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34000b1b00 0x7f34000b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f33ec003fe0 0x7f340014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 --2- 172.21.15.148:0/577717399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3400159090 0x7f3400159500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 >> 172.21.15.148:0/577717399 conn(0x7f3400068ce0 msgr2=0x7f340014c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 shutdown_connections 2023-10-10T20:19:39.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.053+0000 7f34050da700 1 -- 172.21.15.148:0/577717399 wait complete. 2023-10-10T20:19:39.057 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:39.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.179+0000 7f90c1efc700 1 Processor -- start 2023-10-10T20:19:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.179+0000 7f90c1efc700 1 -- start start 2023-10-10T20:19:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90c1efc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90bc0cf1c0 0x7f90bc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90c1efc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc0cca20 con 0x7f90bc15a0c0 2023-10-10T20:19:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90c1efc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc15ea40 con 0x7f90bc159cf0 2023-10-10T20:19:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90c1efc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc067660 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90baffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90bc0cf1c0 0x7f90bc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90baffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90bc0cf1c0 0x7f90bc15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42710/0 (socket says 172.21.15.148:42710) 2023-10-10T20:19:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90bbfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f90bc15a0c0 0x7f90bc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58408/0 (socket says 172.21.15.148:58408) 2023-10-10T20:19:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90baffd700 1 -- 172.21.15.148:0/4173272381 learned_addr learned my addr 172.21.15.148:0/4173272381 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.180+0000 7f90bb7fe700 1 --1- 172.21.15.148:0/4173272381 >> v1:172.21.15.161:6789/0 conn(0x7f90bc159cf0 0x7f90bc0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48226/0 (socket says 172.21.15.148:48226) 2023-10-10T20:19:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90baffd700 1 -- 172.21.15.148:0/4173272381 >> v1:172.21.15.161:6789/0 conn(0x7f90bc159cf0 legacy=0x7f90bc0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90baffd700 1 -- 172.21.15.148:0/4173272381 >> v1:172.21.15.148:6789/0 conn(0x7f90bc15a0c0 legacy=0x7f90bc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90ba7fc700 1 -- 172.21.15.148:0/4173272381 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242460333 0 0) 0x7f90ac001490 con 0x7f90bc159cf0 2023-10-10T20:19:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90baffd700 1 -- 172.21.15.148:0/4173272381 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc067f00 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90baffd700 1 --2- 172.21.15.148:0/4173272381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc15e500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f90b00099b0 tx=0x7f90b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45087f764daf8784 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90ba7fc700 1 -- 172.21.15.148:0/4173272381 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000b760 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90ba7fc700 1 -- 172.21.15.148:0/4173272381 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b000b8c0 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90ba7fc700 1 -- 172.21.15.148:0/4173272381 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000bc60 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90c1efc700 1 -- 172.21.15.148:0/4173272381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc0cf1c0 msgr2=0x7f90bc15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.181+0000 7f90c1efc700 1 --2- 172.21.15.148:0/4173272381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc15e500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f90b00099b0 tx=0x7f90b0009250 comp rx=0 tx=0).stop 2023-10-10T20:19:39.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.182+0000 7f90c1efc700 1 -- 172.21.15.148:0/4173272381 shutdown_connections 2023-10-10T20:19:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.182+0000 7f90c1efc700 1 --2- 172.21.15.148:0/4173272381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc15e500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.182+0000 7f90c1efc700 1 -- 172.21.15.148:0/4173272381 >> 172.21.15.148:0/4173272381 conn(0x7f90bc06c6e0 msgr2=0x7f90bc069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.182+0000 7f90c1efc700 1 -- 172.21.15.148:0/4173272381 shutdown_connections 2023-10-10T20:19:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.182+0000 7f90c1efc700 1 -- 172.21.15.148:0/4173272381 wait complete. 2023-10-10T20:19:39.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.183+0000 7f90c1efc700 1 Processor -- start 2023-10-10T20:19:39.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.183+0000 7f90c1efc700 1 -- start start 2023-10-10T20:19:39.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.183+0000 7f90c1efc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.183+0000 7f90c1efc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90bc15a0c0 0x7f90bc0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c7a30 con 0x7f90bc159cf0 2023-10-10T20:19:39.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c7d10 con 0x7f90bc0cf1c0 2023-10-10T20:19:39.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c7ff0 con 0x7f90bc15a0c0 2023-10-10T20:19:39.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42714/0 (socket says 172.21.15.148:42714) 2023-10-10T20:19:39.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 -- 172.21.15.148:0/2213448282 learned_addr learned my addr 172.21.15.148:0/2213448282 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:39.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90bbfff700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90bc15a0c0 0x7f90bc0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90bb7fe700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 -- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90bc15a0c0 msgr2=0x7f90bc0688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90bc15a0c0 0x7f90bc0688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 -- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 msgr2=0x7f90bc158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 -- 172.21.15.148:0/2213448282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00094f0 con 0x7f90bc159cf0 2023-10-10T20:19:39.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90bb7fe700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:39.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90baffd700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009290 tx=0x7f90b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b001c670 con 0x7f90bc159cf0 2023-10-10T20:19:39.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b001c7f0 con 0x7f90bc159cf0 2023-10-10T20:19:39.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc0c82d0 con 0x7f90bc159cf0 2023-10-10T20:19:39.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b001cbb0 con 0x7f90bc159cf0 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.184+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc15cb30 con 0x7f90bc159cf0 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.185+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc04c3c0 con 0x7f90bc159cf0 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.185+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90b001a450 con 0x7f90bc159cf0 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.186+0000 7f90a37fe700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.186+0000 7f90bb7fe700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.186+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90b0086ae0 con 0x7f90bc159cf0 2023-10-10T20:19:39.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.186+0000 7f90bb7fe700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 0x7f90a80693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00d350 tx=0x7f90ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.190+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90b0053d50 con 0x7f90bc159cf0 2023-10-10T20:19:39.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.377+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90bc04c3c0 con 0x7f90bc159cf0 2023-10-10T20:19:39.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.378+0000 7f90a37fe700 1 -- 172.21.15.148:0/2213448282 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f90b0057370 con 0x7f90bc159cf0 2023-10-10T20:19:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:39.380 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:39.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 msgr2=0x7f90a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 0x7f90a80693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00d350 tx=0x7f90ac009040 comp rx=0 tx=0).stop 2023-10-10T20:19:39.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 msgr2=0x7f90bc068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009290 tx=0x7f90b0016000 comp rx=0 tx=0).stop 2023-10-10T20:19:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 shutdown_connections 2023-10-10T20:19:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f90a8066f30 0x7f90a80693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90bc15a0c0 0x7f90bc0688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90bc0cf1c0 0x7f90bc158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 --2- 172.21.15.148:0/2213448282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90bc159cf0 0x7f90bc068390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 >> 172.21.15.148:0/2213448282 conn(0x7f90bc06c6e0 msgr2=0x7f90bc158ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 shutdown_connections 2023-10-10T20:19:39.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.380+0000 7f90c1efc700 1 -- 172.21.15.148:0/2213448282 wait complete. 2023-10-10T20:19:39.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:39.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:39.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.512+0000 7fe993685700 1 Processor -- start 2023-10-10T20:19:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.512+0000 7fe993685700 1 -- start start 2023-10-10T20:19:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe993685700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe98c070ad0 0x7fe98c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe993685700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c067700 con 0x7fe98c1584d0 2023-10-10T20:19:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe993685700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c158940 con 0x7fe98c070700 2023-10-10T20:19:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe993685700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c158c20 con 0x7fe98c070ad0 2023-10-10T20:19:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe991c22700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe98c1584d0 0x7fe98c067be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58410/0 (socket says 172.21.15.148:58410) 2023-10-10T20:19:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe990c20700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe98c070ad0 0x7fe98c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe991c22700 1 -- 172.21.15.148:0/841990108 learned_addr learned my addr 172.21.15.148:0/841990108 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe990c20700 1 -- 172.21.15.148:0/841990108 >> v1:172.21.15.161:6789/0 conn(0x7fe98c070700 legacy=0x7fe98c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe990c20700 1 -- 172.21.15.148:0/841990108 >> v1:172.21.15.148:6789/0 conn(0x7fe98c1584d0 legacy=0x7fe98c067be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe990c20700 1 -- 172.21.15.148:0/841990108 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c158f00 con 0x7fe98c070ad0 2023-10-10T20:19:39.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe983fff700 1 -- 172.21.15.148:0/841990108 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431339141 0 0) 0x7fe98c067700 con 0x7fe98c1584d0 2023-10-10T20:19:39.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe990c20700 1 --2- 172.21.15.148:0/841990108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070ad0 0x7fe98c157f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098e0 tx=0x7fe97c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28484514203de066 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.513+0000 7fe983fff700 1 -- 172.21.15.148:0/841990108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c009e80 con 0x7fe98c070ad0 2023-10-10T20:19:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe983fff700 1 -- 172.21.15.148:0/841990108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c00b860 con 0x7fe98c070ad0 2023-10-10T20:19:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe983fff700 1 -- 172.21.15.148:0/841990108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00bbe0 con 0x7fe98c070ad0 2023-10-10T20:19:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 -- 172.21.15.148:0/841990108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070ad0 msgr2=0x7fe98c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 --2- 172.21.15.148:0/841990108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070ad0 0x7fe98c157f90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098e0 tx=0x7fe97c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 -- 172.21.15.148:0/841990108 shutdown_connections 2023-10-10T20:19:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 --2- 172.21.15.148:0/841990108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070ad0 0x7fe98c157f90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 -- 172.21.15.148:0/841990108 >> 172.21.15.148:0/841990108 conn(0x7fe98c0ce4d0 msgr2=0x7fe98c0ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 -- 172.21.15.148:0/841990108 shutdown_connections 2023-10-10T20:19:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.514+0000 7fe993685700 1 -- 172.21.15.148:0/841990108 wait complete. 2023-10-10T20:19:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.515+0000 7fe993685700 1 Processor -- start 2023-10-10T20:19:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.515+0000 7fe993685700 1 -- start start 2023-10-10T20:19:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 0x7fe98c1694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 0x7fe98c154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b1110 con 0x7fe98c070700 2023-10-10T20:19:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b13f0 con 0x7fe98c070ad0 2023-10-10T20:19:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe993685700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b16d0 con 0x7fe98c1584d0 2023-10-10T20:19:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42724/0 (socket says 172.21.15.148:42724) 2023-10-10T20:19:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 -- 172.21.15.148:0/440204154 learned_addr learned my addr 172.21.15.148:0/440204154 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe990c20700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 0x7fe98c1694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991c22700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 0x7fe98c154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 -- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 msgr2=0x7fe98c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 0x7fe98c154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 -- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 msgr2=0x7fe98c1694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 0x7fe98c1694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991421700 1 -- 172.21.15.148:0/440204154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c009420 con 0x7fe98c070700 2023-10-10T20:19:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe990c20700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 0x7fe98c1694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.516+0000 7fe991c22700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 0x7fe98c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe991421700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c590 tx=0x7fe98800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988011590 con 0x7fe98c070700 2023-10-10T20:19:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880116f0 con 0x7fe98c070700 2023-10-10T20:19:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988011ab0 con 0x7fe98c070700 2023-10-10T20:19:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c0b1a20 con 0x7fe98c070700 2023-10-10T20:19:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.517+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c0b0dc0 con 0x7fe98c070700 2023-10-10T20:19:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.518+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c04c3c0 con 0x7fe98c070700 2023-10-10T20:19:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.518+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe98801b3b0 con 0x7fe98c070700 2023-10-10T20:19:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.518+0000 7fe9817fa700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 0x7fe9700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.518+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe988083ae0 con 0x7fe98c070700 2023-10-10T20:19:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.518+0000 7fe990c20700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 0x7fe9700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.519+0000 7fe990c20700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 0x7fe9700693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0093f0 tx=0x7fe97c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.523+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe988050d50 con 0x7fe98c070700 2023-10-10T20:19:39.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.704+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe98c1591e0 con 0x7fe98c070700 2023-10-10T20:19:39.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.705+0000 7fe9817fa700 1 -- 172.21.15.148:0/440204154 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe988017020 con 0x7fe98c070700 2023-10-10T20:19:39.707 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 msgr2=0x7fe9700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 0x7fe9700693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0093f0 tx=0x7fe97c017000 comp rx=0 tx=0).stop 2023-10-10T20:19:39.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 msgr2=0x7fe98c0b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c590 tx=0x7fe98800c950 comp rx=0 tx=0).stop 2023-10-10T20:19:39.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 shutdown_connections 2023-10-10T20:19:39.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe970066f30 0x7fe9700693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe98c1584d0 0x7fe98c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe98c070700 0x7fe98c0b0880 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 --2- 172.21.15.148:0/440204154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe98c070ad0 0x7fe98c1694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 >> 172.21.15.148:0/440204154 conn(0x7fe98c0ce4d0 msgr2=0x7fe98c155be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 shutdown_connections 2023-10-10T20:19:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.707+0000 7fe993685700 1 -- 172.21.15.148:0/440204154 wait complete. 2023-10-10T20:19:39.721 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:39.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.836+0000 7f7f00f5f700 1 Processor -- start 2023-10-10T20:19:39.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.836+0000 7f7f00f5f700 1 -- start start 2023-10-10T20:19:39.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7f00f5f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7efc15e140 0x7f7efc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7f00f5f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0cd790 con 0x7f7efc0cc680 2023-10-10T20:19:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7f00f5f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0ccb80 con 0x7f7efc159de0 2023-10-10T20:19:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7f00f5f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0ce160 con 0x7f7efc15e140 2023-10-10T20:19:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7efad9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7efc0cc680 0x7f7efc0ca820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58414/0 (socket says 172.21.15.148:58414) 2023-10-10T20:19:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7efad9d700 1 -- 172.21.15.148:0/4201023574 learned_addr learned my addr 172.21.15.148:0/4201023574 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7ef1d9b700 1 --2- 172.21.15.148:0/4201023574 >> v2:172.21.15.161:3300/0 conn(0x7f7efc15e140 0x7f7efc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068796048 0 0) 0x7f7efc0cd790 con 0x7f7efc0cc680 2023-10-10T20:19:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee8003570 con 0x7f7efc0cc680 2023-10-10T20:19:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180244465 0 0) 0x7f7efc0ccb80 con 0x7f7efc159de0 2023-10-10T20:19:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.837+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7efc0cd790 con 0x7f7efc159de0 2023-10-10T20:19:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224500437 0 0) 0x7f7ee8003570 con 0x7f7efc0cc680 2023-10-10T20:19:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 >> v2:172.21.15.161:3300/0 conn(0x7f7efc15e140 msgr2=0x7f7efc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 --2- 172.21.15.148:0/4201023574 >> v2:172.21.15.161:3300/0 conn(0x7f7efc15e140 0x7f7efc0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 >> v1:172.21.15.161:6789/0 conn(0x7f7efc159de0 legacy=0x7f7efc15da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0cce60 con 0x7f7efc0cc680 2023-10-10T20:19:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7ef40017f0 con 0x7f7efc0cc680 2023-10-10T20:19:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 >> v1:172.21.15.148:6789/0 conn(0x7f7efc0cc680 legacy=0x7f7efc0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee8003bf0 0x7f7ee8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ee8004810 0x7f7ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 0x7f7ee800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee800c080 con 0x7f7ee80051c0 2023-10-10T20:19:39.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee800c1f0 con 0x7f7ee8003bf0 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee800c3b0 con 0x7f7ee8004810 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 0x7f7ee800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efa59c700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee8003bf0 0x7f7ee8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef1d9b700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ee8004810 0x7f7ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 -- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ee8004810 msgr2=0x7f7ee8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ee8004810 0x7f7ee8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 -- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee8003bf0 msgr2=0x7f7ee8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:39.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee8003bf0 0x7f7ee8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 -- 172.21.15.148:0/4201023574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8003570 con 0x7f7ee80051c0 2023-10-10T20:19:39.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 -- 172.21.15.148:0/4201023574 reap_dead start 2023-10-10T20:19:39.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7efad9d700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 0x7f7ee800ba60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400a8d0 tx=0x7f7ef4002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f816d72266b4a816 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40070d0 con 0x7f7ee80051c0 2023-10-10T20:19:39.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4007270 con 0x7f7ee80051c0 2023-10-10T20:19:39.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.838+0000 7f7ef9d9b700 1 -- 172.21.15.148:0/4201023574 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4007630 con 0x7f7ee80051c0 2023-10-10T20:19:39.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 -- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 msgr2=0x7f7ee800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 0x7f7ee800ba60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400a8d0 tx=0x7f7ef4002190 comp rx=0 tx=0).stop 2023-10-10T20:19:39.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 -- 172.21.15.148:0/4201023574 shutdown_connections 2023-10-10T20:19:39.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/4201023574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee80051c0 0x7f7ee800ba60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 -- 172.21.15.148:0/4201023574 >> 172.21.15.148:0/4201023574 conn(0x7f7efc0cb5c0 msgr2=0x7f7efc0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:39.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 -- 172.21.15.148:0/4201023574 shutdown_connections 2023-10-10T20:19:39.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.839+0000 7f7f00f5f700 1 -- 172.21.15.148:0/4201023574 wait complete. 2023-10-10T20:19:39.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.840+0000 7f7f00f5f700 1 Processor -- start 2023-10-10T20:19:39.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.840+0000 7f7f00f5f700 1 -- start start 2023-10-10T20:19:39.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee80051c0 0x7f7efc156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 0x7f7efc0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc15c760 con 0x7f7efc15b0e0 2023-10-10T20:19:39.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0c5200 con 0x7f7ee80051c0 2023-10-10T20:19:39.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0c54e0 con 0x7f7efc15c1f0 2023-10-10T20:19:39.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42738/0 (socket says 172.21.15.148:42738) 2023-10-10T20:19:39.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 -- 172.21.15.148:0/292825369 learned_addr learned my addr 172.21.15.148:0/292825369 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:39.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efa59c700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee80051c0 0x7f7efc156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7ef1d9b700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 0x7f7efc0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 -- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 msgr2=0x7f7efc0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 0x7f7efc0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 -- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee80051c0 msgr2=0x7f7efc156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:39.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee80051c0 0x7f7efc156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:39.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 -- 172.21.15.148:0/292825369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0c57c0 con 0x7f7efc15b0e0 2023-10-10T20:19:39.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7ef1d9b700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 0x7f7efc0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:39.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7efad9d700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0c96d0 tx=0x7f7ef4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401e430 con 0x7f7efc15b0e0 2023-10-10T20:19:39.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef401e590 con 0x7f7efc15b0e0 2023-10-10T20:19:39.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc1578b0 con 0x7f7efc15b0e0 2023-10-10T20:19:39.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401e950 con 0x7f7efc15b0e0 2023-10-10T20:19:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.841+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc15ba90 con 0x7f7efc15b0e0 2023-10-10T20:19:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.842+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc04c3c0 con 0x7f7efc15b0e0 2023-10-10T20:19:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.843+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ef401ed00 con 0x7f7efc15b0e0 2023-10-10T20:19:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.843+0000 7f7ef2ffd700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 0x7f7edc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.843+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ef4086d90 con 0x7f7efc15b0e0 2023-10-10T20:19:39.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.843+0000 7f7efa59c700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 0x7f7edc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:39.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.843+0000 7f7efa59c700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 0x7f7edc069410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000f60 tx=0x7f7eec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:39.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:39.847+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ef4054000 con 0x7f7efc15b0e0 2023-10-10T20:19:40.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.029+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7efc0c9400 con 0x7f7efc15b0e0 2023-10-10T20:19:40.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.029+0000 7f7ef2ffd700 1 -- 172.21.15.148:0/292825369 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ef4057620 con 0x7f7efc15b0e0 2023-10-10T20:19:40.032 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.031+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 msgr2=0x7f7edc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.031+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 0x7f7edc069410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000f60 tx=0x7f7eec009040 comp rx=0 tx=0).stop 2023-10-10T20:19:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.031+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 msgr2=0x7f7efc15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.031+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0c96d0 tx=0x7f7ef4016000 comp rx=0 tx=0).stop 2023-10-10T20:19:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 shutdown_connections 2023-10-10T20:19:40.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7edc066f60 0x7f7edc069410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7efc15b0e0 0x7f7efc15b550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ee80051c0 0x7f7efc156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 --2- 172.21.15.148:0/292825369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7efc15c1f0 0x7f7efc0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 >> 172.21.15.148:0/292825369 conn(0x7f7efc0cb5c0 msgr2=0x7f7efc0b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 shutdown_connections 2023-10-10T20:19:40.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.032+0000 7f7f00f5f700 1 -- 172.21.15.148:0/292825369 wait complete. 2023-10-10T20:19:40.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:40.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.160+0000 7f498c005700 1 Processor -- start 2023-10-10T20:19:40.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.160+0000 7f498c005700 1 -- start start 2023-10-10T20:19:40.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f498c005700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 0x7f49840af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f498c005700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f498415a290 0x7f49840afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f498c005700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840b0790 con 0x7f498406a5a0 2023-10-10T20:19:40.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f498c005700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840ce1f0 con 0x7f498415a290 2023-10-10T20:19:40.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f498c005700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840b0b20 con 0x7f4984159ec0 2023-10-10T20:19:40.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f49895a0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f498415a290 0x7f49840afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f49895a0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f498415a290 0x7f49840afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42740/0 (socket says 172.21.15.148:42740) 2023-10-10T20:19:40.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.161+0000 7f49895a0700 1 -- 172.21.15.148:0/3478776591 learned_addr learned my addr 172.21.15.148:0/3478776591 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:40.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f4989da1700 1 --2- 172.21.15.148:0/3478776591 >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 0x7f49840af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f49895a0700 1 -- 172.21.15.148:0/3478776591 >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 msgr2=0x7f49840af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:40.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f49895a0700 1 --2- 172.21.15.148:0/3478776591 >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 0x7f49840af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f49895a0700 1 -- 172.21.15.148:0/3478776591 >> v1:172.21.15.161:6789/0 conn(0x7f498406a5a0 legacy=0x7f498415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f49895a0700 1 -- 172.21.15.148:0/3478776591 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840b04b0 con 0x7f498415a290 2023-10-10T20:19:40.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f4989da1700 1 --2- 172.21.15.148:0/3478776591 >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 0x7f49840af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:40.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f49895a0700 1 --2- 172.21.15.148:0/3478776591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498415a290 0x7f49840afe80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49840898e0 tx=0x7f4974009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc2ae1c856144a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f4988d9f700 1 -- 172.21.15.148:0/3478776591 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400b860 con 0x7f498415a290 2023-10-10T20:19:40.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f4988d9f700 1 -- 172.21.15.148:0/3478776591 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400b9c0 con 0x7f498415a290 2023-10-10T20:19:40.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f4988d9f700 1 -- 172.21.15.148:0/3478776591 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400bd60 con 0x7f498415a290 2023-10-10T20:19:40.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 -- 172.21.15.148:0/3478776591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498415a290 msgr2=0x7f49840afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 --2- 172.21.15.148:0/3478776591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498415a290 0x7f49840afe80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49840898e0 tx=0x7f4974009250 comp rx=0 tx=0).stop 2023-10-10T20:19:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 -- 172.21.15.148:0/3478776591 shutdown_connections 2023-10-10T20:19:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 --2- 172.21.15.148:0/3478776591 >> v2:172.21.15.161:3300/0 conn(0x7f4984159ec0 0x7f49840af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 --2- 172.21.15.148:0/3478776591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498415a290 0x7f49840afe80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.162+0000 7f498c005700 1 -- 172.21.15.148:0/3478776591 >> 172.21.15.148:0/3478776591 conn(0x7f49840d1880 msgr2=0x7f49840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.163+0000 7f498c005700 1 -- 172.21.15.148:0/3478776591 shutdown_connections 2023-10-10T20:19:40.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.163+0000 7f498c005700 1 -- 172.21.15.148:0/3478776591 wait complete. 2023-10-10T20:19:40.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 Processor -- start 2023-10-10T20:19:40.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 -- start start 2023-10-10T20:19:40.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4984159ec0 0x7f498416d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f498415a290 0x7f498415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b4200 con 0x7f498406a5a0 2023-10-10T20:19:40.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b3bf0 con 0x7f498415a290 2023-10-10T20:19:40.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f498c005700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498415b8e0 con 0x7f4984159ec0 2023-10-10T20:19:40.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.164+0000 7f4989da1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42756/0 (socket says 172.21.15.148:42756) 2023-10-10T20:19:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 -- 172.21.15.148:0/1283888295 learned_addr learned my addr 172.21.15.148:0/1283888295 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f49895a0700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4984159ec0 0x7f498416d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f498a5a2700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f498415a290 0x7f498415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 -- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4984159ec0 msgr2=0x7f498416d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4984159ec0 0x7f498416d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 -- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f498415a290 msgr2=0x7f498415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f498415a290 0x7f498415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 -- 172.21.15.148:0/1283888295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740094f0 con 0x7f498406a5a0 2023-10-10T20:19:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4989da1700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f498000d1d0 tx=0x7f4980009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000dd10 con 0x7f498406a5a0 2023-10-10T20:19:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498000f040 con 0x7f498406a5a0 2023-10-10T20:19:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840b3e90 con 0x7f498406a5a0 2023-10-10T20:19:40.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980015670 con 0x7f498406a5a0 2023-10-10T20:19:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.165+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498415bbc0 con 0x7f498406a5a0 2023-10-10T20:19:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.166+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49840b44e0 con 0x7f498406a5a0 2023-10-10T20:19:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.166+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4980015a60 con 0x7f498406a5a0 2023-10-10T20:19:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.167+0000 7f4979ffb700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 0x7f49680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.167+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4980084880 con 0x7f498406a5a0 2023-10-10T20:19:40.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.167+0000 7f49895a0700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 0x7f49680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.167+0000 7f49895a0700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 0x7f49680693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49840898e0 tx=0x7f4974017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.171+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f498004e5a0 con 0x7f498406a5a0 2023-10-10T20:19:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.357+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f498404c3c0 con 0x7f498406a5a0 2023-10-10T20:19:40.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.358+0000 7f4979ffb700 1 -- 172.21.15.148:0/1283888295 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f498001a0a0 con 0x7f498406a5a0 2023-10-10T20:19:40.360 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:40.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 msgr2=0x7f49680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 0x7f49680693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f49840898e0 tx=0x7f4974017000 comp rx=0 tx=0).stop 2023-10-10T20:19:40.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 msgr2=0x7f498415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f498000d1d0 tx=0x7f4980009a90 comp rx=0 tx=0).stop 2023-10-10T20:19:40.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 shutdown_connections 2023-10-10T20:19:40.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4968066f30 0x7f49680693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f498415a290 0x7f498415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f498406a5a0 0x7f498415c820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 --2- 172.21.15.148:0/1283888295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4984159ec0 0x7f498416d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 >> 172.21.15.148:0/1283888295 conn(0x7f49840d1880 msgr2=0x7f4984154360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 shutdown_connections 2023-10-10T20:19:40.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.360+0000 7f498c005700 1 -- 172.21.15.148:0/1283888295 wait complete. 2023-10-10T20:19:40.374 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:40.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.487+0000 7feec0820700 1 Processor -- start 2023-10-10T20:19:40.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.487+0000 7feec0820700 1 -- start start 2023-10-10T20:19:40.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 0x7feebc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feebc156d00 0x7feebc067b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc0680b0 0x7feebc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc06f180 con 0x7feebc156d00 2023-10-10T20:19:40.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0c82d0 con 0x7feebc156930 2023-10-10T20:19:40.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feec0820700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc068550 con 0x7feebc0680b0 2023-10-10T20:19:40.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feebc156d00 0x7feebc067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feebc156d00 0x7feebc067b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42770/0 (socket says 172.21.15.148:42770) 2023-10-10T20:19:40.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 -- 172.21.15.148:0/1848194562 learned_addr learned my addr 172.21.15.148:0/1848194562 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:40.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feebad9d700 1 --2- 172.21.15.148:0/1848194562 >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 0x7feebc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feebb59e700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc0680b0 0x7feebc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 -- 172.21.15.148:0/1848194562 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc0680b0 msgr2=0x7feebc06a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:40.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc0680b0 0x7feebc06a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 -- 172.21.15.148:0/1848194562 >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 msgr2=0x7feebc0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:40.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 --2- 172.21.15.148:0/1848194562 >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 0x7feebc0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feeb259c700 1 -- 172.21.15.148:0/1848194562 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc068830 con 0x7feebc156d00 2023-10-10T20:19:40.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.488+0000 7feebad9d700 1 --2- 172.21.15.148:0/1848194562 >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 0x7feebc0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:40.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feeb259c700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc156d00 0x7feebc067b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feebc06b370 tx=0x7feea400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b300a887147427f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feeba59c700 1 -- 172.21.15.148:0/1848194562 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400dc50 con 0x7feebc156d00 2023-10-10T20:19:40.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feeba59c700 1 -- 172.21.15.148:0/1848194562 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400ddb0 con 0x7feebc156d00 2023-10-10T20:19:40.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feeba59c700 1 -- 172.21.15.148:0/1848194562 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40196d0 con 0x7feebc156d00 2023-10-10T20:19:40.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 -- 172.21.15.148:0/1848194562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc156d00 msgr2=0x7feebc067b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc156d00 0x7feebc067b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feebc06b370 tx=0x7feea400b4f0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 -- 172.21.15.148:0/1848194562 shutdown_connections 2023-10-10T20:19:40.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc0680b0 0x7feebc06a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 --2- 172.21.15.148:0/1848194562 >> v2:172.21.15.161:3300/0 conn(0x7feebc156930 0x7feebc0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 --2- 172.21.15.148:0/1848194562 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc156d00 0x7feebc067b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 -- 172.21.15.148:0/1848194562 >> 172.21.15.148:0/1848194562 conn(0x7feebc0cd8f0 msgr2=0x7feebc0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 -- 172.21.15.148:0/1848194562 shutdown_connections 2023-10-10T20:19:40.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.489+0000 7feec0820700 1 -- 172.21.15.148:0/1848194562 wait complete. 2023-10-10T20:19:40.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 Processor -- start 2023-10-10T20:19:40.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 -- start start 2023-10-10T20:19:40.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 0x7feebc0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 0x7feebc0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc1588c0 con 0x7feebc0680b0 2023-10-10T20:19:40.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c77a0 con 0x7feebc156930 2023-10-10T20:19:40.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feec0820700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc167640 con 0x7feebc156d00 2023-10-10T20:19:40.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feebad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feebad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42774/0 (socket says 172.21.15.148:42774) 2023-10-10T20:19:40.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.491+0000 7feebad9d700 1 -- 172.21.15.148:0/2550050439 learned_addr learned my addr 172.21.15.148:0/2550050439 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:40.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebb59e700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 0x7feebc0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feeb259c700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 0x7feebc0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 -- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 msgr2=0x7feebc0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 0x7feebc0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 -- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 msgr2=0x7feebc0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 0x7feebc0c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 -- 172.21.15.148:0/2550050439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea400b790 con 0x7feebc0680b0 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebb59e700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 0x7feebc0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feeb259c700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 0x7feebc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feebad9d700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c830 tx=0x7feeac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011600 con 0x7feebc0680b0 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac011760 con 0x7feebc0680b0 2023-10-10T20:19:40.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc166d30 con 0x7feebc0680b0 2023-10-10T20:19:40.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac011b20 con 0x7feebc0680b0 2023-10-10T20:19:40.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.492+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc166ff0 con 0x7feebc0680b0 2023-10-10T20:19:40.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.493+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc06d640 con 0x7feebc0680b0 2023-10-10T20:19:40.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.493+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeac00a3e0 con 0x7feebc0680b0 2023-10-10T20:19:40.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.494+0000 7feeb37fe700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.494+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feeac084630 con 0x7feebc0680b0 2023-10-10T20:19:40.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.494+0000 7feeb259c700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.494+0000 7feeb259c700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feebc06b370 tx=0x7feea401b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.498+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeac0518a0 con 0x7feebc0680b0 2023-10-10T20:19:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.722+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feebc068b10 con 0x7feebc0680b0 2023-10-10T20:19:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.722+0000 7feeb37fe700 1 -- 172.21.15.148:0/2550050439 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7feeac054ec0 con 0x7feebc0680b0 2023-10-10T20:19:40.724 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:40.725 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 msgr2=0x7fee9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feebc06b370 tx=0x7feea401b000 comp rx=0 tx=0).stop 2023-10-10T20:19:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 msgr2=0x7feebc167ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c830 tx=0x7feeac008040 comp rx=0 tx=0).stop 2023-10-10T20:19:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 shutdown_connections 2023-10-10T20:19:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feebc156d00 0x7feebc0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0680b0 0x7feebc167ad0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 --2- 172.21.15.148:0/2550050439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feebc156930 0x7feebc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 >> 172.21.15.148:0/2550050439 conn(0x7feebc0cd8f0 msgr2=0x7feebc0cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 shutdown_connections 2023-10-10T20:19:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.724+0000 7feec0820700 1 -- 172.21.15.148:0/2550050439 wait complete. 2023-10-10T20:19:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:40.738 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:40.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 Processor -- start 2023-10-10T20:19:40.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 -- start start 2023-10-10T20:19:40.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54e015d580 0x7f54e0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e015cbb0 con 0x7f54e0157870 2023-10-10T20:19:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e015bdb0 con 0x7f54e015d580 2023-10-10T20:19:40.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.851+0000 7f54e6494700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e015bfa0 con 0x7f54e015a620 2023-10-10T20:19:40.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54e015d580 0x7f54e0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54dffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f54e0157870 0x7f54e015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48248/0 (socket says 172.21.15.148:48248) 2023-10-10T20:19:40.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:51518/0 (socket says 172.21.15.148:51518) 2023-10-10T20:19:40.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54dffff700 1 -- 172.21.15.148:0/3478040576 learned_addr learned my addr 172.21.15.148:0/3478040576 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:40.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54deffd700 1 -- 172.21.15.148:0/3478040576 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710129295 0 0) 0x7f54e015cbb0 con 0x7f54e0157870 2023-10-10T20:19:40.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54deffd700 1 -- 172.21.15.148:0/3478040576 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c8003570 con 0x7f54e0157870 2023-10-10T20:19:40.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 -- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 msgr2=0x7f54e015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:40.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 -- 172.21.15.148:0/3478040576 >> v1:172.21.15.161:6789/0 conn(0x7f54e0157870 legacy=0x7f54e015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 -- 172.21.15.148:0/3478040576 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e015c280 con 0x7f54e015d580 2023-10-10T20:19:40.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54df7fe700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:40.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54e4a31700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e015d580 0x7f54e0158110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009e60 tx=0x7f54d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79f42f4834c989cc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54deffd700 1 -- 172.21.15.148:0/3478040576 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d800e070 con 0x7f54e015d580 2023-10-10T20:19:40.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54deffd700 1 -- 172.21.15.148:0/3478040576 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d800bab0 con 0x7f54e015d580 2023-10-10T20:19:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.852+0000 7f54deffd700 1 -- 172.21.15.148:0/3478040576 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d800be70 con 0x7f54e015d580 2023-10-10T20:19:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 -- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e015d580 msgr2=0x7f54e0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e015d580 0x7f54e0158110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009e60 tx=0x7f54d8009780 comp rx=0 tx=0).stop 2023-10-10T20:19:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 -- 172.21.15.148:0/3478040576 shutdown_connections 2023-10-10T20:19:40.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e015d580 0x7f54e0158110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 --2- 172.21.15.148:0/3478040576 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 -- 172.21.15.148:0/3478040576 >> 172.21.15.148:0/3478040576 conn(0x7f54e0068ce0 msgr2=0x7f54e00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:40.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 -- 172.21.15.148:0/3478040576 shutdown_connections 2023-10-10T20:19:40.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.853+0000 7f54e6494700 1 -- 172.21.15.148:0/3478040576 wait complete. 2023-10-10T20:19:40.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.854+0000 7f54e6494700 1 Processor -- start 2023-10-10T20:19:40.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.854+0000 7f54e6494700 1 -- start start 2023-10-10T20:19:40.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e0157870 0x7f54e016d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e0154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0158af0 con 0x7f54e015d580 2023-10-10T20:19:40.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e016bbd0 con 0x7f54e0157870 2023-10-10T20:19:40.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e6494700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0157ee0 con 0x7f54e015a620 2023-10-10T20:19:40.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42780/0 (socket says 172.21.15.148:42780) 2023-10-10T20:19:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 -- 172.21.15.148:0/1555251480 learned_addr learned my addr 172.21.15.148:0/1555251480 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54dffff700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e0157870 0x7f54e016d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54df7fe700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e0154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 -- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 msgr2=0x7f54e0154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e0154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 -- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e0157870 msgr2=0x7f54e016d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:40.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e0157870 0x7f54e016d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:40.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.855+0000 7f54e4a31700 1 -- 172.21.15.148:0/1555251480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d4009420 con 0x7f54e015d580 2023-10-10T20:19:40.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54e4a31700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54d8013000 tx=0x7f54d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8012e10 con 0x7f54e015d580 2023-10-10T20:19:40.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d8027470 con 0x7f54e015d580 2023-10-10T20:19:40.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8027830 con 0x7f54e015d580 2023-10-10T20:19:40.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d8009a20 con 0x7f54e015d580 2023-10-10T20:19:40.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.856+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e01584a0 con 0x7f54e015d580 2023-10-10T20:19:40.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.857+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e004c3c0 con 0x7f54e015d580 2023-10-10T20:19:40.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.857+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54d8024070 con 0x7f54e015d580 2023-10-10T20:19:40.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.857+0000 7f54c7fff700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 0x7f54cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:40.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.857+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54d8086f50 con 0x7f54e015d580 2023-10-10T20:19:40.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.857+0000 7f54dffff700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 0x7f54cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:40.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.858+0000 7f54dffff700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 0x7f54cc0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54d0007fd0 tx=0x7f54d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:40.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:40.862+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54d80541c0 con 0x7f54e015d580 2023-10-10T20:19:41.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.045+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54e015c560 con 0x7f54e015d580 2023-10-10T20:19:41.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.046+0000 7f54c7fff700 1 -- 172.21.15.148:0/1555251480 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f54d802c090 con 0x7f54e015d580 2023-10-10T20:19:41.048 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:41.049 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:41.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 msgr2=0x7f54cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 0x7f54cc0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54d0007fd0 tx=0x7f54d0009040 comp rx=0 tx=0).stop 2023-10-10T20:19:41.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 msgr2=0x7f54e0154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54d8013000 tx=0x7f54d8019000 comp rx=0 tx=0).stop 2023-10-10T20:19:41.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 shutdown_connections 2023-10-10T20:19:41.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54cc066f30 0x7f54cc0693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e015d580 0x7f54e0154c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54e0157870 0x7f54e016d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 --2- 172.21.15.148:0/1555251480 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54e015a620 0x7f54e0154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 >> 172.21.15.148:0/1555251480 conn(0x7f54e0068ce0 msgr2=0x7f54e00b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 shutdown_connections 2023-10-10T20:19:41.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.048+0000 7f54e6494700 1 -- 172.21.15.148:0/1555251480 wait complete. 2023-10-10T20:19:41.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:41.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.175+0000 7fa54f833700 1 Processor -- start 2023-10-10T20:19:41.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.175+0000 7fa54f833700 1 -- start start 2023-10-10T20:19:41.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54f833700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 0x7fa5480c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54f833700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa54806e2b0 0x7fa54806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54f833700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480c42b0 con 0x7fa548156ea0 2023-10-10T20:19:41.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54f833700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54806fac0 con 0x7fa54806e2b0 2023-10-10T20:19:41.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54f833700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54806ec10 con 0x7fa548157270 2023-10-10T20:19:41.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54ddd0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa54806e2b0 0x7fa54806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54ddd0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa54806e2b0 0x7fa54806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42784/0 (socket says 172.21.15.148:42784) 2023-10-10T20:19:41.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.176+0000 7fa54ddd0700 1 -- 172.21.15.148:0/4008325237 learned_addr learned my addr 172.21.15.148:0/4008325237 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54cdce700 1 --2- 172.21.15.148:0/4008325237 >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 0x7fa5480c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54ddd0700 1 -- 172.21.15.148:0/4008325237 >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 msgr2=0x7fa5480c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54ddd0700 1 --2- 172.21.15.148:0/4008325237 >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 0x7fa5480c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54ddd0700 1 -- 172.21.15.148:0/4008325237 >> v1:172.21.15.161:6789/0 conn(0x7fa548156ea0 legacy=0x7fa54806f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54ddd0700 1 -- 172.21.15.148:0/4008325237 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480c36a0 con 0x7fa54806e2b0 2023-10-10T20:19:41.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54cdce700 1 --2- 172.21.15.148:0/4008325237 >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 0x7fa5480c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:41.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54ddd0700 1 --2- 172.21.15.148:0/4008325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa54806e6d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa54806dd30 tx=0x7fa540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f0730c7a64716b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa53ffff700 1 -- 172.21.15.148:0/4008325237 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000b860 con 0x7fa54806e2b0 2023-10-10T20:19:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa53ffff700 1 -- 172.21.15.148:0/4008325237 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54000b9c0 con 0x7fa54806e2b0 2023-10-10T20:19:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa53ffff700 1 -- 172.21.15.148:0/4008325237 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000bd60 con 0x7fa54806e2b0 2023-10-10T20:19:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 -- 172.21.15.148:0/4008325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 msgr2=0x7fa54806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 --2- 172.21.15.148:0/4008325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa54806e6d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa54806dd30 tx=0x7fa540009250 comp rx=0 tx=0).stop 2023-10-10T20:19:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 -- 172.21.15.148:0/4008325237 shutdown_connections 2023-10-10T20:19:41.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 --2- 172.21.15.148:0/4008325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa54806e6d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 --2- 172.21.15.148:0/4008325237 >> v2:172.21.15.161:3300/0 conn(0x7fa548157270 0x7fa5480c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.177+0000 7fa54f833700 1 -- 172.21.15.148:0/4008325237 >> 172.21.15.148:0/4008325237 conn(0x7fa5480ca4f0 msgr2=0x7fa5480ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.178+0000 7fa54f833700 1 -- 172.21.15.148:0/4008325237 shutdown_connections 2023-10-10T20:19:41.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.178+0000 7fa54f833700 1 -- 172.21.15.148:0/4008325237 wait complete. 2023-10-10T20:19:41.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.179+0000 7fa54f833700 1 Processor -- start 2023-10-10T20:19:41.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.179+0000 7fa54f833700 1 -- start start 2023-10-10T20:19:41.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.179+0000 7fa54f833700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.179+0000 7fa54f833700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa548156ea0 0x7fa5480c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.179+0000 7fa54f833700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa548157270 0x7fa5480cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54f833700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54816f190 con 0x7fa54806e2b0 2023-10-10T20:19:41.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54f833700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b37a0 con 0x7fa548157270 2023-10-10T20:19:41.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54f833700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54816e510 con 0x7fa548156ea0 2023-10-10T20:19:41.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42800/0 (socket says 172.21.15.148:42800) 2023-10-10T20:19:41.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 -- 172.21.15.148:0/211750601 learned_addr learned my addr 172.21.15.148:0/211750601 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54ddd0700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa548157270 0x7fa5480cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54cdce700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa548156ea0 0x7fa5480c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 -- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa548156ea0 msgr2=0x7fa5480c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa548156ea0 0x7fa5480c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 -- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa548157270 msgr2=0x7fa5480cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa548157270 0x7fa5480cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 -- 172.21.15.148:0/211750601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400094f0 con 0x7fa54806e2b0 2023-10-10T20:19:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54d5cf700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa54400cb10 tx=0x7fa5440017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400ce20 con 0x7fa54806e2b0 2023-10-10T20:19:41.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54400a7e0 con 0x7fa54806e2b0 2023-10-10T20:19:41.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54816e7f0 con 0x7fa54806e2b0 2023-10-10T20:19:41.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400aba0 con 0x7fa54806e2b0 2023-10-10T20:19:41.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.180+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54816edb0 con 0x7fa54806e2b0 2023-10-10T20:19:41.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.181+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54816f2d0 con 0x7fa54806e2b0 2023-10-10T20:19:41.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.182+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa544003030 con 0x7fa54806e2b0 2023-10-10T20:19:41.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.182+0000 7fa53d7fa700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 0x7fa52c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.182+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5440844c0 con 0x7fa54806e2b0 2023-10-10T20:19:41.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.182+0000 7fa54cdce700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 0x7fa52c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.182+0000 7fa54cdce700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 0x7fa52c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa538006f00 tx=0x7fa538008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.186+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa54816f2d0 con 0x7fa54806e2b0 2023-10-10T20:19:41.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.411+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa54816f2d0 con 0x7fa54806e2b0 2023-10-10T20:19:41.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.412+0000 7fa53d7fa700 1 -- 172.21.15.148:0/211750601 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7fa54816f2d0 con 0x7fa54806e2b0 2023-10-10T20:19:41.414 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:41.415 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:41.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 msgr2=0x7fa52c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 0x7fa52c0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa538006f00 tx=0x7fa538008040 comp rx=0 tx=0).stop 2023-10-10T20:19:41.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 msgr2=0x7fa5480b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa54400cb10 tx=0x7fa5440017b0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 shutdown_connections 2023-10-10T20:19:41.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa52c066f30 0x7fa52c0693e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa548157270 0x7fa5480cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54806e2b0 0x7fa5480b3260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 --2- 172.21.15.148:0/211750601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa548156ea0 0x7fa5480c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 >> 172.21.15.148:0/211750601 conn(0x7fa5480ca4f0 msgr2=0x7fa548070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 shutdown_connections 2023-10-10T20:19:41.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.414+0000 7fa54f833700 1 -- 172.21.15.148:0/211750601 wait complete. 2023-10-10T20:19:41.419 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:41.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:19:22.857181+0000', 'modified': '2023-10-10T20:19:26.509765+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/2566190456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 2566190456}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2566190456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:19:41.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:19:41.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-10T20:19:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.947+0000 7ff0d143d700 1 Processor -- start 2023-10-10T20:19:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.947+0000 7ff0d143d700 1 -- start start 2023-10-10T20:19:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0d143d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0cc15a620 0x7ff0cc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0d143d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff0cc15d580 0x7ff0cc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0d143d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc15cbb0 con 0x7ff0cc157870 2023-10-10T20:19:41.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0d143d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc15bdb0 con 0x7ff0cc15a620 2023-10-10T20:19:41.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0d143d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc15bfa0 con 0x7ff0cc15d580 2023-10-10T20:19:41.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0cc15a620 0x7ff0cc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0caffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0cc157870 0x7ff0cc15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58424/0 (socket says 172.21.15.148:58424) 2023-10-10T20:19:41.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0caffd700 1 -- 172.21.15.148:0/1718659472 learned_addr learned my addr 172.21.15.148:0/1718659472 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:41.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1718659472 >> v2:172.21.15.148:3300/0 conn(0x7ff0cc15a620 0x7ff0cc15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42806/0 (socket says 172.21.15.148:42806) 2023-10-10T20:19:41.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1718659472 >> v2:172.21.15.161:3300/0 conn(0x7ff0cc15d580 0x7ff0cc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1718659472 >> v2:172.21.15.161:3300/0 conn(0x7ff0cc15d580 msgr2=0x7ff0cc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1718659472 >> v2:172.21.15.161:3300/0 conn(0x7ff0cc15d580 0x7ff0cc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1718659472 >> v1:172.21.15.148:6789/0 conn(0x7ff0cc157870 legacy=0x7ff0cc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1718659472 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc15c280 con 0x7ff0cc15a620 2023-10-10T20:19:41.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0c9ffb700 1 -- 172.21.15.148:0/1718659472 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669612754 0 0) 0x7ff0cc15cbb0 con 0x7ff0cc157870 2023-10-10T20:19:41.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1718659472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc15d040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc06b4e0 tx=0x7ff0c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e5a8e68fd11d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0c9ffb700 1 -- 172.21.15.148:0/1718659472 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011630 con 0x7ff0cc15a620 2023-10-10T20:19:41.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0c9ffb700 1 -- 172.21.15.148:0/1718659472 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0011790 con 0x7ff0cc15a620 2023-10-10T20:19:41.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.948+0000 7ff0c9ffb700 1 -- 172.21.15.148:0/1718659472 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011b30 con 0x7ff0cc15a620 2023-10-10T20:19:41.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 -- 172.21.15.148:0/1718659472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 msgr2=0x7ff0cc15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1718659472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc15d040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc06b4e0 tx=0x7ff0c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 -- 172.21.15.148:0/1718659472 shutdown_connections 2023-10-10T20:19:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1718659472 >> v2:172.21.15.161:3300/0 conn(0x7ff0cc15d580 0x7ff0cc158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1718659472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc15d040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 -- 172.21.15.148:0/1718659472 >> 172.21.15.148:0/1718659472 conn(0x7ff0cc068ce0 msgr2=0x7ff0cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 -- 172.21.15.148:0/1718659472 shutdown_connections 2023-10-10T20:19:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.949+0000 7ff0d143d700 1 -- 172.21.15.148:0/1718659472 wait complete. 2023-10-10T20:19:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.950+0000 7ff0d143d700 1 Processor -- start 2023-10-10T20:19:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.950+0000 7ff0d143d700 1 -- start start 2023-10-10T20:19:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 0x7ff0cc0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 0x7ff0cc0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc151030 con 0x7ff0cc15a620 2023-10-10T20:19:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc151310 con 0x7ff0cc157870 2023-10-10T20:19:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0d143d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1515f0 con 0x7ff0cc15d580 2023-10-10T20:19:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42818/0 (socket says 172.21.15.148:42818) 2023-10-10T20:19:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1085834686 learned_addr learned my addr 172.21.15.148:0/1085834686 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 0x7ff0cc0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 0x7ff0cc0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 msgr2=0x7ff0cc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 0x7ff0cc0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 msgr2=0x7ff0cc0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 0x7ff0cc0cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/1085834686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0009420 con 0x7ff0cc15a620 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 0x7ff0cc0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.951+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 0x7ff0cc0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0013000 tx=0x7ff0c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1085834686 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011b10 con 0x7ff0cc15a620 2023-10-10T20:19:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1085834686 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00253f0 con 0x7ff0cc15a620 2023-10-10T20:19:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc1518d0 con 0x7ff0cc15a620 2023-10-10T20:19:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1085834686 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0025790 con 0x7ff0cc15a620 2023-10-10T20:19:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.952+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0d01d0 con 0x7ff0cc15a620 2023-10-10T20:19:41.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.953+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1085834686 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0c0025b80 con 0x7ff0cc15a620 2023-10-10T20:19:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.954+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 0x7ff0b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.954+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1085834686 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0c0085e40 con 0x7ff0cc15a620 2023-10-10T20:19:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.954+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 0x7ff0b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.954+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 0x7ff0b80693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4001840 tx=0x7ff0c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.956+0000 7ff0da736b80 1 Processor -- start 2023-10-10T20:19:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.956+0000 7ff0da736b80 1 -- start start 2023-10-10T20:19:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.956+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6d1a50 0x56225f6d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.956+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.956+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0da736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f628a70 con 0x56225f71bd00 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f626270 con 0x56225f6faf60 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56225f622060 con 0x56225f6d1a50 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42832/0 (socket says 172.21.15.148:42832) 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/4017876598 learned_addr learned my addr 172.21.15.148:0/4017876598 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0caffd700 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6d1a50 0x56225f6d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6d1a50 msgr2=0x56225f6d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6d1a50 0x56225f6d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 msgr2=0x56225f6cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 -- 172.21.15.148:0/4017876598 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225f4bc270 con 0x56225f71bd00 2023-10-10T20:19:41.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0084330 tx=0x7ff0c0003040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d949ea31a24ec5a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0abfff700 1 -- 172.21.15.148:0/4017876598 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c005c0f0 con 0x56225f71bd00 2023-10-10T20:19:41.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0abfff700 1 -- 172.21.15.148:0/4017876598 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0057470 con 0x56225f71bd00 2023-10-10T20:19:41.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.957+0000 7ff0abfff700 1 -- 172.21.15.148:0/4017876598 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0056d70 con 0x56225f71bd00 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- 172.21.15.148:0/4017876598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 msgr2=0x56225f6faa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0084330 tx=0x7ff0c0003040 comp rx=0 tx=0).stop 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- 172.21.15.148:0/4017876598 shutdown_connections 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6d1a50 0x56225f6d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 --2- 172.21.15.148:0/4017876598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f6faa20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- 172.21.15.148:0/4017876598 >> 172.21.15.148:0/4017876598 conn(0x56225f6d3370 msgr2=0x56225f6d3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- 172.21.15.148:0/4017876598 shutdown_connections 2023-10-10T20:19:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- 172.21.15.148:0/4017876598 wait complete. 2023-10-10T20:19:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.958+0000 7ff0da736b80 1 -- start start 2023-10-10T20:19:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 Processor -- start 2023-10-10T20:19:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f718650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f711e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f5cd890 con 0x56225f6d1a50 2023-10-10T20:19:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f5e8ea0 con 0x56225f6faf60 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.959+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56225f4a4e30 con 0x56225f71bd00 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 10 client.0 _ll_register_callbacks cb 0x56225f695110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42840/0 (socket says 172.21.15.148:42840) 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 -- 172.21.15.148:0/3995314626 learned_addr learned my addr 172.21.15.148:0/3995314626 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f718650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f711e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 -- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 msgr2=0x56225f711e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f711e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 -- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 msgr2=0x56225f718650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f718650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 -- 172.21.15.148:0/3995314626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0d01d0 con 0x56225f6d1a50 2023-10-10T20:19:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f718650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0caffd700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4005fd0 tx=0x7ff0c4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3168133789ebaa85 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:19:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400dde0 con 0x56225f6d1a50 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0c4020a90 con 0x56225f6d1a50 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56225f6c6b70 con 0x56225f6d1a50 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c4021430 con 0x56225f6d1a50 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56225f45efe0 con 0x56225f6d1a50 2023-10-10T20:19:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56225f6c6b70 con 0x56225f6d1a50 2023-10-10T20:19:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff0c4020420 con 0x56225f6d1a50 2023-10-10T20:19:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-10-10T20:19:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:19:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 20 client.4662 populate_metadata read hostname 'smithi148' 2023-10-10T20:19:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.960+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f72bea0 0x56225f72c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.961+0000 7ff0da736b80 4 client.4662 mds_command: new command op to 4273 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:19:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.961+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 --> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56225f6a3aa0 con 0x56225f72bea0 2023-10-10T20:19:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.961+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f72bea0 0x56225f72c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.961+0000 7ff0a97fa700 10 client.4662 ms_handle_connect on v2:172.21.15.161:6833/2566190456 2023-10-10T20:19:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.961+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f72bea0 0x56225f72c270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c95ec1e1b2a4bb4 server_cookie=1b653ee33a72f604 in_seq=0 out_seq=0 2023-10-10T20:19:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0a97fa700 1 -- 172.21.15.148:0/3995314626 <== mds.0 v2:172.21.15.161:6833/2566190456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56225f6a3aa0 con 0x56225f72bea0 2023-10-10T20:19:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0a97fa700 10 client.4662 handle_command_reply: tid=0 2023-10-10T20:19:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 client.4662 shutdown 2023-10-10T20:19:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 msgr2=0x56225f727d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4005fd0 tx=0x7ff0c4016040 comp rx=0 tx=0).stop 2023-10-10T20:19:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 shutdown_connections 2023-10-10T20:19:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f72bea0 0x56225f72c270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f711e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f727d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 --2- 172.21.15.148:0/3995314626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f718650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0a97fa700 0 client.4662 ms_handle_reset on v2:172.21.15.161:6833/2566190456 2023-10-10T20:19:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 >> 172.21.15.148:0/3995314626 conn(0x56225f6d3370 msgr2=0x56225f6ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 shutdown_connections 2023-10-10T20:19:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 1 -- 172.21.15.148:0/3995314626 wait complete. 2023-10-10T20:19:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.962+0000 7ff0da736b80 20 client.4662 trim_cache size 0 max 16384 2023-10-10T20:19:41.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 Processor -- start 2023-10-10T20:19:41.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 -- start start 2023-10-10T20:19:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f71a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f6cc5a0 con 0x56225f6d1a50 2023-10-10T20:19:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f72b8e0 con 0x56225f6faf60 2023-10-10T20:19:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56225f6d2870 con 0x56225f71bd00 2023-10-10T20:19:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42854/0 (socket says 172.21.15.148:42854) 2023-10-10T20:19:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.979+0000 7ff0caffd700 1 -- 172.21.15.148:0/1622177224 learned_addr learned my addr 172.21.15.148:0/1622177224 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:41.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 -- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 msgr2=0x56225f71a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f71a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 -- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 msgr2=0x56225f6cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 -- 172.21.15.148:0/1622177224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225f5f0780 con 0x56225f6d1a50 2023-10-10T20:19:41.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4020f70 tx=0x7ff0c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7abad95d1a1ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0b0ff9700 1 -- 172.21.15.148:0/1622177224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4001460 con 0x56225f6d1a50 2023-10-10T20:19:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0b0ff9700 1 -- 172.21.15.148:0/1622177224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c401a050 con 0x56225f6d1a50 2023-10-10T20:19:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0b0ff9700 1 -- 172.21.15.148:0/1622177224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4020b80 con 0x56225f6d1a50 2023-10-10T20:19:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 -- 172.21.15.148:0/1622177224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 msgr2=0x56225f701980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:41.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4020f70 tx=0x7ff0c4013000 comp rx=0 tx=0).stop 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 -- 172.21.15.148:0/1622177224 shutdown_connections 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f71bd00 0x56225f71a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f6d1a50 0x56225f701980 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1622177224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6faf60 0x56225f6cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 -- 172.21.15.148:0/1622177224 >> 172.21.15.148:0/1622177224 conn(0x56225f6d3370 msgr2=0x56225f7322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:41.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 -- 172.21.15.148:0/1622177224 shutdown_connections 2023-10-10T20:19:41.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.980+0000 7ff0da736b80 1 -- 172.21.15.148:0/1622177224 wait complete. 2023-10-10T20:19:41.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.981+0000 7ff0da736b80 1 -- start start 2023-10-10T20:19:41.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.981+0000 7ff0da736b80 1 Processor -- start 2023-10-10T20:19:41.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 0x56225f77e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6faf60 0x56225f717420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f77f350 con 0x56225f71bd00 2023-10-10T20:19:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225f71adc0 con 0x56225f6d1a50 2023-10-10T20:19:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56225f6ea150 con 0x56225f6faf60 2023-10-10T20:19:41.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0da736b80 10 client.0 _ll_register_callbacks cb 0x56225f695110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:19:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42864/0 (socket says 172.21.15.148:42864) 2023-10-10T20:19:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 -- 172.21.15.148:0/1425681654 learned_addr learned my addr 172.21.15.148:0/1425681654 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0ca7fc700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6faf60 0x56225f717420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 0x56225f77e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 -- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6faf60 msgr2=0x56225f717420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6faf60 0x56225f717420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 -- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 msgr2=0x56225f77e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 0x56225f77e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 -- 172.21.15.148:0/1425681654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56225f45efe0 con 0x56225f71bd00 2023-10-10T20:19:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 0x56225f77e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0cb7fe700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc16a690 tx=0x7ff0bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c66d2fbd3ce508 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.982+0000 7ff0a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:19:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00e040 con 0x56225f71bd00 2023-10-10T20:19:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0bc012ca0 con 0x56225f71bd00 2023-10-10T20:19:42.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0c0057470 con 0x56225f71bd00 2023-10-10T20:19:42.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0bc012e60 con 0x56225f71bd00 2023-10-10T20:19:42.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 10 client.4668 fetch_fsmap learned FSMap version 8 2023-10-10T20:19:42.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56225f5d7470 con 0x56225f71bd00 2023-10-10T20:19:42.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0057470 con 0x56225f71bd00 2023-10-10T20:19:42.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff0bc014040 con 0x56225f71bd00 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 20 client.4668 populate_metadata read hostname 'smithi148' 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f6eaa80 0x56225f6eae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 4 client.4668 mds_command: new command op to 4273 tid=0[{"prefix": "status"}] 2023-10-10T20:19:42.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 --> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56225f5c4e00 con 0x56225f6eaa80 2023-10-10T20:19:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.983+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f6eaa80 0x56225f6eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0a97fa700 10 client.4668 ms_handle_connect on v2:172.21.15.161:6833/2566190456 2023-10-10T20:19:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0caffd700 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f6eaa80 0x56225f6eae50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd91896cc56c2f8 server_cookie=48c2be2b2f9c4aa9 in_seq=0 out_seq=0 2023-10-10T20:19:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0a97fa700 1 -- 172.21.15.148:0/1425681654 <== mds.0 v2:172.21.15.161:6833/2566190456 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56225f5c4e00 con 0x56225f6eaa80 2023-10-10T20:19:42.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0a97fa700 10 client.4668 handle_command_reply: tid=0 2023-10-10T20:19:42.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 client.4668 shutdown 2023-10-10T20:19:42.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 msgr2=0x56225f717960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc16a690 tx=0x7ff0bc00b040 comp rx=0 tx=0).stop 2023-10-10T20:19:42.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 shutdown_connections 2023-10-10T20:19:42.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:6833/2566190456,v1:172.21.15.161:6835/2566190456] conn(0x56225f6eaa80 0x56225f6eae50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56225f71bd00 0x56225f717960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56225f6d1a50 0x56225f77e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 --2- 172.21.15.148:0/1425681654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56225f6faf60 0x56225f717420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0a97fa700 0 client.4668 ms_handle_reset on v2:172.21.15.161:6833/2566190456 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 >> 172.21.15.148:0/1425681654 conn(0x56225f6d3370 msgr2=0x56225f6cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:42.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 shutdown_connections 2023-10-10T20:19:42.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 1 -- 172.21.15.148:0/1425681654 wait complete. 2023-10-10T20:19:42.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0da736b80 20 client.4668 trim_cache size 0 max 16384 2023-10-10T20:19:42.008 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:19:42.009 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:19:42.009 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:19:42.009 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4273, 2023-10-10T20:19:42.009 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:19:42.009 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 18.491900300000001, 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 8, 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 20, 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 20, 2023-10-10T20:19:42.010 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 25.201343420000001 2023-10-10T20:19:42.011 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:19:42.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 msgr2=0x7ff0b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 0x7ff0b80693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4001840 tx=0x7ff0c4009040 comp rx=0 tx=0).stop 2023-10-10T20:19:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 msgr2=0x7ff0cc14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.984+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0013000 tx=0x7ff0c0017000 comp rx=0 tx=0).stop 2023-10-10T20:19:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 shutdown_connections 2023-10-10T20:19:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0b8066f30 0x7ff0b80693e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0cc15d580 0x7ff0cc0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc157870 0x7ff0cc0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 --2- 172.21.15.148:0/1085834686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc15a620 0x7ff0cc14bbe0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0013000 tx=0x7ff0c0017000 comp rx=0 tx=0).stop 2023-10-10T20:19:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 >> 172.21.15.148:0/1085834686 conn(0x7ff0cc068ce0 msgr2=0x7ff0cc15fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 shutdown_connections 2023-10-10T20:19:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:41.985+0000 7ff0d143d700 1 -- 172.21.15.148:0/1085834686 wait complete. 2023-10-10T20:19:42.015 INFO:teuthology.run_tasks:Running task kclient... 2023-10-10T20:19:42.037 INFO:tasks.kclient:Mounting kernel clients... 2023-10-10T20:19:42.037 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-10T20:19:42.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.038 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:19:42.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.054 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-10-10T20:19:42.120 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2023-10-10T20:19:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:19:42.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.122 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:19:42.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-10-10T20:19:42.241 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2023-10-10T20:19:42.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:19:42.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.243 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:19:42.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-10-10T20:19:42.334 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2023-10-10T20:19:42.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:19:42.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.337 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:19:42.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.394 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-10-10T20:19:42.461 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2023-10-10T20:19:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:19:42.463 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-10T20:19:42.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:19:42.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.577+0000 7f5d17fff700 1 Processor -- start 2023-10-10T20:19:42.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.577+0000 7f5d17fff700 1 -- start start 2023-10-10T20:19:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d17fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d18156980 0x7f5d180ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d17fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d1806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d17fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d180ad5c0 con 0x7f5d180cd420 2023-10-10T20:19:42.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d17fff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d180ac9b0 con 0x7f5d18156980 2023-10-10T20:19:42.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d17fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180acc90 con 0x7f5d180cbf10 2023-10-10T20:19:42.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d18156980 0x7f5d180ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d167fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5d180cd420 0x7f5d180cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51220/0 (socket says 172.21.15.148:51220) 2023-10-10T20:19:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d18156980 0x7f5d180ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46912/0 (socket says 172.21.15.148:46912) 2023-10-10T20:19:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d167fc700 1 -- 172.21.15.148:0/1402358072 learned_addr learned my addr 172.21.15.148:0/1402358072 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:42.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d177fe700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d1806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 -- 172.21.15.148:0/1402358072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 msgr2=0x7f5d1806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:42.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d1806d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 -- 172.21.15.148:0/1402358072 >> v1:172.21.15.148:6789/0 conn(0x7f5d180cd420 legacy=0x7f5d180cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d15ffb700 1 -- 172.21.15.148:0/1402358072 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353111950 0 0) 0x7f5d180ad5c0 con 0x7f5d180cd420 2023-10-10T20:19:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.578+0000 7f5d16ffd700 1 -- 172.21.15.148:0/1402358072 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d180acf70 con 0x7f5d18156980 2023-10-10T20:19:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d16ffd700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d180ac0d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e60 tx=0x7f5d08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4a2747138c139d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d15ffb700 1 -- 172.21.15.148:0/1402358072 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800e070 con 0x7f5d18156980 2023-10-10T20:19:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d15ffb700 1 -- 172.21.15.148:0/1402358072 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800bc30 con 0x7f5d18156980 2023-10-10T20:19:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d15ffb700 1 -- 172.21.15.148:0/1402358072 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08012430 con 0x7f5d18156980 2023-10-10T20:19:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 -- 172.21.15.148:0/1402358072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 msgr2=0x7f5d180ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d180ac0d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e60 tx=0x7f5d08009780 comp rx=0 tx=0).stop 2023-10-10T20:19:42.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 -- 172.21.15.148:0/1402358072 shutdown_connections 2023-10-10T20:19:42.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d1806d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 --2- 172.21.15.148:0/1402358072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d180ac0d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 -- 172.21.15.148:0/1402358072 >> 172.21.15.148:0/1402358072 conn(0x7f5d180680e0 msgr2=0x7f5d180cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:42.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 -- 172.21.15.148:0/1402358072 shutdown_connections 2023-10-10T20:19:42.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.579+0000 7f5d17fff700 1 -- 172.21.15.148:0/1402358072 wait complete. 2023-10-10T20:19:42.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 Processor -- start 2023-10-10T20:19:42.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 -- start start 2023-10-10T20:19:42.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d180b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d180cd420 0x7f5d181660b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18156eb0 con 0x7f5d18156980 2023-10-10T20:19:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18157190 con 0x7f5d180cd420 2023-10-10T20:19:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d17fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180b0c60 con 0x7f5d180cbf10 2023-10-10T20:19:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46916/0 (socket says 172.21.15.148:46916) 2023-10-10T20:19:42.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d177fe700 1 -- 172.21.15.148:0/3300958000 learned_addr learned my addr 172.21.15.148:0/3300958000 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:42.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d16ffd700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d180b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.581+0000 7f5d167fc700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d180cd420 0x7f5d181660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 -- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 msgr2=0x7f5d180b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d180b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 -- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d180cd420 msgr2=0x7f5d181660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d180cd420 0x7f5d181660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 -- 172.21.15.148:0/3300958000 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08009a20 con 0x7f5d18156980 2023-10-10T20:19:42.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d177fe700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d890 tx=0x7f5d10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:42.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100125f0 con 0x7f5d18156980 2023-10-10T20:19:42.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10012750 con 0x7f5d18156980 2023-10-10T20:19:42.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10012b10 con 0x7f5d18156980 2023-10-10T20:19:42.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d180b0f40 con 0x7f5d18156980 2023-10-10T20:19:42.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.582+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d180b07e0 con 0x7f5d18156980 2023-10-10T20:19:42.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.583+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d180b0a00 con 0x7f5d18156980 2023-10-10T20:19:42.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.583+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d10013480 con 0x7f5d18156980 2023-10-10T20:19:42.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.584+0000 7f5d0effd700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 0x7f5cf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:42.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.584+0000 7f5d16ffd700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 0x7f5cf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:42.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.584+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d10084820 con 0x7f5d18156980 2023-10-10T20:19:42.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.584+0000 7f5d16ffd700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 0x7f5cf80693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e60 tx=0x7f5d08019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:42.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.588+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d10051a90 con 0x7f5d18156980 2023-10-10T20:19:42.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.813+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d18157300 con 0x7f5d18156980 2023-10-10T20:19:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.814+0000 7f5d0effd700 1 -- 172.21.15.148:0/3300958000 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d100550b0 con 0x7f5d18156980 2023-10-10T20:19:42.818 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:19:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 msgr2=0x7f5cf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 0x7f5cf80693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e60 tx=0x7f5d08019000 comp rx=0 tx=0).stop 2023-10-10T20:19:42.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 msgr2=0x7f5d181665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:42.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d890 tx=0x7f5d10008040 comp rx=0 tx=0).stop 2023-10-10T20:19:42.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 shutdown_connections 2023-10-10T20:19:42.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cf8066f30 0x7f5cf80693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d18156980 0x7f5d181665f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d180cbf10 0x7f5d180b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 --2- 172.21.15.148:0/3300958000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d180cd420 0x7f5d181660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:42.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 >> 172.21.15.148:0/3300958000 conn(0x7f5d180680e0 msgr2=0x7f5d180cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:42.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 shutdown_connections 2023-10-10T20:19:42.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:42.816+0000 7f5d17fff700 1 -- 172.21.15.148:0/3300958000 wait complete. 2023-10-10T20:19:42.830 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:19:42.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:19:42.830 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:19:42.830 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:19:42.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:19:42.831 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:19:42.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:19:42.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:19:42.832 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:19:42.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.832 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:19:42.847 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:42.847 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:42.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:42.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:42.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:42.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:42.848 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:42.849 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:42.849 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:19:42.849 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:42.849 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:19:42.850 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:42.851 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:19:42.851 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42528sec preferred_lft 42528sec 2023-10-10T20:19:42.851 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:19:42.851 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:42.852 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-10T20:19:42.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:42.853 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:42.853 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:42.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2023-10-10T20:19:42.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2023-10-10T20:19:42.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2023-10-10T20:19:42.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-10T20:19:42.854 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:43.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.015 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:19:43.055 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:19:43.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.057 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:19:43.134 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:19:43.134 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:19:43.134 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:19:43.134 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:19:43.135 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:19:43.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.136 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:43.136 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:43.136 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:19:43.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:19:43.137 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:19:43.137 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:43.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.388 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:19:43.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.403 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2023-10-10T20:19:43.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.457 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:43.458 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:43.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:19:43.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-10T20:19:43.458 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:43.546 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-10T20:19:43.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.547 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:43.548 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:43.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-10T20:19:43.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-10T20:19:43.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-10T20:19:43.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-10T20:19:43.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-10T20:19:43.549 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:43.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:43.731 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:43.731 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:43.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:19:43.731 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2023-10-10T20:19:43.732 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:43.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:43.825 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:19:43.839 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:19:43.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:43.840 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:19:43.894 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:19:43.894 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:19:43.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:43.986 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:43.986 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:44.075 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:44.075 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:44.076 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:44.076 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:44.076 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:44.076 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:44.076 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:44.077 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:44.097 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:44.098 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:44.098 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:44.098 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:44.098 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:44.098 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:19:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:19:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:19:44.100 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:19:44.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:19:44.127 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-10T20:19:44.127 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:19:44.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.279+0000 7f0ab94d2700 1 Processor -- start 2023-10-10T20:19:44.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.279+0000 7f0ab94d2700 1 -- start start 2023-10-10T20:19:44.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab94d2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0ab40cf060 0x7f0ab40b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab94d2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab4067800 con 0x7f0ab4159be0 2023-10-10T20:19:44.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab94d2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab4067ae0 con 0x7f0ab4159fb0 2023-10-10T20:19:44.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab94d2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40701f0 con 0x7f0ab40cf060 2023-10-10T20:19:44.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ab4159be0 0x7f0ab4068db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51226/0 (socket says 172.21.15.148:51226) 2023-10-10T20:19:44.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab3fff700 1 -- 172.21.15.148:0/775925658 learned_addr learned my addr 172.21.15.148:0/775925658 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:44.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab8cd1700 1 --2- 172.21.15.148:0/775925658 >> v2:172.21.15.161:3300/0 conn(0x7f0ab40cf060 0x7f0ab40b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910872140 0 0) 0x7f0ab4067800 con 0x7f0ab4159be0 2023-10-10T20:19:44.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.280+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a9c003570 con 0x7f0ab4159be0 2023-10-10T20:19:44.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843952642 0 0) 0x7f0a9c003570 con 0x7f0ab4159be0 2023-10-10T20:19:44.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 >> v2:172.21.15.161:3300/0 conn(0x7f0ab40cf060 msgr2=0x7f0ab40b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:44.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 --2- 172.21.15.148:0/775925658 >> v2:172.21.15.161:3300/0 conn(0x7f0ab40cf060 0x7f0ab40b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 >> v1:172.21.15.161:6789/0 conn(0x7f0ab4159fb0 legacy=0x7f0ab40ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4067dc0 con 0x7f0ab4159be0 2023-10-10T20:19:44.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f0aa4001470 con 0x7f0ab4159be0 2023-10-10T20:19:44.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 >> v1:172.21.15.148:6789/0 conn(0x7f0ab4159be0 legacy=0x7f0ab4068db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0a9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c004810 0x7f0a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a9c0051c0 0x7f0a9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40701f0 con 0x7f0a9c003a60 2023-10-10T20:19:44.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c00c0d0 con 0x7f0a9c004810 2023-10-10T20:19:44.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c00c290 con 0x7f0a9c0051c0 2023-10-10T20:19:44.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0a9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab8cd1700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a9c0051c0 0x7f0a9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab37fe700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c004810 0x7f0a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 -- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a9c0051c0 msgr2=0x7f0a9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:44.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a9c0051c0 0x7f0a9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 -- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c004810 msgr2=0x7f0a9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:44.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a9c004810 0x7f0a9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab8cd1700 1 -- 172.21.15.148:0/775925658 reap_dead start 2023-10-10T20:19:44.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 -- 172.21.15.148:0/775925658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c003570 con 0x7f0a9c003a60 2023-10-10T20:19:44.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0a9c003e70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c3c0 tx=0x7f0aa4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e3fe87ab28a8c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:44.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400cc10 con 0x7f0a9c003a60 2023-10-10T20:19:44.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa400cd70 con 0x7f0a9c003a60 2023-10-10T20:19:44.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.281+0000 7f0ab2ffd700 1 -- 172.21.15.148:0/775925658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4005b50 con 0x7f0a9c003a60 2023-10-10T20:19:44.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 -- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 msgr2=0x7f0a9c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0a9c003e70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c3c0 tx=0x7f0aa4001e50 comp rx=0 tx=0).stop 2023-10-10T20:19:44.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 -- 172.21.15.148:0/775925658 shutdown_connections 2023-10-10T20:19:44.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/775925658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0a9c003e70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 -- 172.21.15.148:0/775925658 >> 172.21.15.148:0/775925658 conn(0x7f0ab406bc70 msgr2=0x7f0ab406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:44.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 -- 172.21.15.148:0/775925658 shutdown_connections 2023-10-10T20:19:44.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.282+0000 7f0ab94d2700 1 -- 172.21.15.148:0/775925658 wait complete. 2023-10-10T20:19:44.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.283+0000 7f0ab94d2700 1 Processor -- start 2023-10-10T20:19:44.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.283+0000 7f0ab94d2700 1 -- start start 2023-10-10T20:19:44.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 0x7f0ab40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 0x7f0ab416d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40ce9e0 con 0x7f0a9c003a60 2023-10-10T20:19:44.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c4b60 con 0x7f0ab40c4ff0 2023-10-10T20:19:44.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab94d2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c4270 con 0x7f0ab40ce540 2023-10-10T20:19:44.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46934/0 (socket says 172.21.15.148:46934) 2023-10-10T20:19:44.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab3fff700 1 -- 172.21.15.148:0/2794461461 learned_addr learned my addr 172.21.15.148:0/2794461461 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:44.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab8cd1700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 0x7f0ab416d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab37fe700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 0x7f0ab40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.284+0000 7f0ab3fff700 1 -- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 msgr2=0x7f0ab416d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 0x7f0ab416d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab3fff700 1 -- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 msgr2=0x7f0ab40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 0x7f0ab40ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab3fff700 1 -- 172.21.15.148:0/2794461461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab416e300 con 0x7f0a9c003a60 2023-10-10T20:19:44.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab8cd1700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 0x7f0ab416d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:44.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab37fe700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 0x7f0ab40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:44.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab3fff700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c0a0 tx=0x7f0aa4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:44.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4001500 con 0x7f0a9c003a60 2023-10-10T20:19:44.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40070d0 con 0x7f0a9c003a60 2023-10-10T20:19:44.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40c4510 con 0x7f0a9c003a60 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4007490 con 0x7f0a9c003a60 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.285+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41575c0 con 0x7f0a9c003a60 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.286+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab406e3e0 con 0x7f0a9c003a60 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.286+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0aa4028460 con 0x7f0a9c003a60 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.287+0000 7f0a9bfff700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.287+0000 7f0ab37fe700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:44.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.287+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0aa4085b60 con 0x7f0a9c003a60 2023-10-10T20:19:44.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.287+0000 7f0ab37fe700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 0x7f0aa00693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40c5420 tx=0x7f0aa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:44.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.291+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aa4052e80 con 0x7f0a9c003a60 2023-10-10T20:19:44.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.515+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0ab404bb80 con 0x7f0a9c003a60 2023-10-10T20:19:44.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.516+0000 7f0a9bfff700 1 -- 172.21.15.148:0/2794461461 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0aa4017070 con 0x7f0a9c003a60 2023-10-10T20:19:44.519 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:19:44.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 msgr2=0x7f0aa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 0x7f0aa00693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40c5420 tx=0x7f0aa8009040 comp rx=0 tx=0).stop 2023-10-10T20:19:44.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 msgr2=0x7f0ab4157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:44.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c0a0 tx=0x7f0aa4015000 comp rx=0 tx=0).stop 2023-10-10T20:19:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 shutdown_connections 2023-10-10T20:19:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0aa0066f30 0x7f0aa00693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab40ce540 0x7f0ab416d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a9c003a60 0x7f0ab4157080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 --2- 172.21.15.148:0/2794461461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab40c4ff0 0x7f0ab40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:44.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 >> 172.21.15.148:0/2794461461 conn(0x7f0ab406bc70 msgr2=0x7f0ab415a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:44.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 shutdown_connections 2023-10-10T20:19:44.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:44.518+0000 7f0ab94d2700 1 -- 172.21.15.148:0/2794461461 wait complete. 2023-10-10T20:19:44.532 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:19:44.532 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:19:44.532 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:19:44.533 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-10T20:19:44.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-10T20:19:44.533 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:19:44.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:19:44.534 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:19:44.534 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-10T20:19:44.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.535 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:19:44.550 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:44.550 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:44.550 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:44.551 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.551 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:44.551 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.551 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:44.551 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:44.552 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:19:44.552 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:44.552 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:44.552 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:19:44.552 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:44.553 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:44.553 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:19:44.553 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:44.553 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:19:44.553 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42527sec preferred_lft 42527sec 2023-10-10T20:19:44.554 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:19:44.554 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.554 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:44.554 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:44.555 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:19:44.555 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.555 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:19:44.555 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.555 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:19:44.556 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:19:44.556 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link tentative 2023-10-10T20:19:44.556 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.557 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:19:44.614 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:19:44.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.615 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:19:44.673 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:19:44.674 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:19:44.674 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:19:44.674 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:19:44.674 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:19:44.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.675 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:44.676 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:44.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:19:44.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:19:44.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:19:44.677 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:44.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.774 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:19:44.789 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:19:44.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.790 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2023-10-10T20:19:44.844 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-10T20:19:44.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.845 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:44.845 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:44.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:19:44.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-10T20:19:44.846 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:44.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-10T20:19:44.970 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.971 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.972 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:19:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7e:fe:5a:a3:e5:20 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:19:44.972 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:19:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7cfe:5aff:fea3:e520/64 scope link tentative 2023-10-10T20:19:44.973 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:44.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:44.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-10T20:19:45.055 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:45.056 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:45.056 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:45.056 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:45.056 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:45.056 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:45.057 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:19:45.057 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7e:fe:5a:a3:e5:20 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:19:45.057 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:19:45.057 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:45.057 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7cfe:5aff:fea3:e520/64 scope link tentative 2023-10-10T20:19:45.058 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:45.065 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-10T20:19:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:45.066 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:45.066 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:45.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-10T20:19:45.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-10T20:19:45.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-10T20:19:45.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-10T20:19:45.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-10T20:19:45.068 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:45.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:45.286 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:19:45.286 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:19:45.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2023-10-10T20:19:45.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2023-10-10T20:19:45.287 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:19:45.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:45.367 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-10T20:19:45.381 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-10T20:19:45.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:45.382 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-10T20:19:45.436 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-10T20:19:45.437 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:19:45.437 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:45.517 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:45.517 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:45.536 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:45.536 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:45.537 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:45.537 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 2@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:45.537 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:45.538 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:45.538 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:45.538 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:45.559 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:45.559 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:45.560 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:45.560 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 2@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:45.560 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:45.561 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:45.561 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:45.561 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:45.561 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:19:45.562 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:19:45.562 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:19:45.564 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:19:45.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:45.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-10T20:19:45.628 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-10T20:19:45.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:19:45.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.781+0000 7f8ef9f9d700 1 Processor -- start 2023-10-10T20:19:45.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.781+0000 7f8ef9f9d700 1 -- start start 2023-10-10T20:19:45.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef9f9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8ef415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef9f9d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40cb7c0 con 0x7f8ef415dda0 2023-10-10T20:19:45.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef9f9d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef415e290 con 0x7f8ef40cd6e0 2023-10-10T20:19:45.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef9f9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef415e850 con 0x7f8ef40cdab0 2023-10-10T20:19:45.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef979c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ef415dda0 0x7f8ef415cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51234/0 (socket says 172.21.15.148:51234) 2023-10-10T20:19:45.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.782+0000 7f8ef979c700 1 -- 172.21.15.148:0/3494123921 learned_addr learned my addr 172.21.15.148:0/3494123921 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:45.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eebfff700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8ef415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662808021 0 0) 0x7f8ef40cb7c0 con 0x7f8ef415dda0 2023-10-10T20:19:45.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8edc003570 con 0x7f8ef415dda0 2023-10-10T20:19:45.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471724648 0 0) 0x7f8ef415e290 con 0x7f8ef40cd6e0 2023-10-10T20:19:45.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef40cb7c0 con 0x7f8ef40cd6e0 2023-10-10T20:19:45.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647158025 0 0) 0x7f8edc003570 con 0x7f8ef415dda0 2023-10-10T20:19:45.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 msgr2=0x7f8ef415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:45.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8ef415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:45.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 >> v1:172.21.15.161:6789/0 conn(0x7f8ef40cd6e0 legacy=0x7f8ef40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef415e570 con 0x7f8ef415dda0 2023-10-10T20:19:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f8eec0017f0 con 0x7f8ef415dda0 2023-10-10T20:19:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eebfff700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8ef415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 >> v1:172.21.15.148:6789/0 conn(0x7f8ef415dda0 legacy=0x7f8ef415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8edc003c20 0x7f8edc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8edc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8edc004d50 0x7f8edc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef415e850 con 0x7f8edc004d50 2023-10-10T20:19:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc00bb40 con 0x7f8edc003c20 2023-10-10T20:19:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc00bd00 con 0x7f8ef40cdab0 2023-10-10T20:19:45.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8edc004d50 0x7f8edc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef8f9b700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8edc003c20 0x7f8edc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8eebfff700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8edc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 -- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 msgr2=0x7f8edc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:45.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef40cdab0 0x7f8edc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:45.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 -- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8edc003c20 msgr2=0x7f8edc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:45.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8edc003c20 0x7f8edc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:45.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.783+0000 7f8ef979c700 1 -- 172.21.15.148:0/3494123921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4009420 con 0x7f8edc004d50 2023-10-10T20:19:45.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef979c700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8edc004d50 0x7f8edc0051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00a8d0 tx=0x7f8eec002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c7e39d03426b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:45.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0070d0 con 0x7f8edc004d50 2023-10-10T20:19:45.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec007230 con 0x7f8edc004d50 2023-10-10T20:19:45.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8eeb7fe700 1 -- 172.21.15.148:0/3494123921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0075f0 con 0x7f8edc004d50 2023-10-10T20:19:45.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/3494123921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8edc004d50 msgr2=0x7f8edc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:45.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/3494123921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8edc004d50 0x7f8edc0051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00a8d0 tx=0x7f8eec002190 comp rx=0 tx=0).stop 2023-10-10T20:19:45.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef979c700 1 -- 172.21.15.148:0/3494123921 reap_dead start 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/3494123921 shutdown_connections 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/3494123921 >> 172.21.15.148:0/3494123921 conn(0x7f8ef4067660 msgr2=0x7f8ef4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/3494123921 shutdown_connections 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.784+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/3494123921 wait complete. 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.785+0000 7f8ef9f9d700 1 Processor -- start 2023-10-10T20:19:45.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 -- start start 2023-10-10T20:19:45.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef4155d40 0x7f8ef40ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 0x7f8ef416a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40cc360 con 0x7f8ef40ca910 2023-10-10T20:19:45.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b5a60 con 0x7f8ef40b55f0 2023-10-10T20:19:45.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef9f9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c9f40 con 0x7f8ef4155d40 2023-10-10T20:19:45.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8eebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8eebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46952/0 (socket says 172.21.15.148:46952) 2023-10-10T20:19:45.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8eebfff700 1 -- 172.21.15.148:0/1824849701 learned_addr learned my addr 172.21.15.148:0/1824849701 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:45.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.786+0000 7f8ef8f9b700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef4155d40 0x7f8ef40ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ef979c700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 0x7f8ef416a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 -- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef4155d40 msgr2=0x7f8ef40ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:45.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef4155d40 0x7f8ef40ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:45.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 -- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 msgr2=0x7f8ef416a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:45.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 0x7f8ef416a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:45.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 -- 172.21.15.148:0/1824849701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40c9330 con 0x7f8ef40ca910 2023-10-10T20:19:45.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ef979c700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 0x7f8ef416a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:45.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8eebfff700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4011da0 tx=0x7f8ee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:45.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400fcd0 con 0x7f8ef40ca910 2023-10-10T20:19:45.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee400fe30 con 0x7f8ef40ca910 2023-10-10T20:19:45.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef40c9610 con 0x7f8ef40ca910 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee401d6d0 con 0x7f8ef40ca910 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.787+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef40c9bd0 con 0x7f8ef40ca910 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.788+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4089280 con 0x7f8ef40ca910 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.788+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ee401da40 con 0x7f8ef40ca910 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.789+0000 7f8ee8ff9700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 0x7f8ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:45.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.789+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ee4018070 con 0x7f8ef40ca910 2023-10-10T20:19:45.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.789+0000 7f8ef8f9b700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 0x7f8ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:45.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.789+0000 7f8ef8f9b700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 0x7f8ee00693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007e40 tx=0x7f8ef0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:45.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:45.793+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ee40526b0 con 0x7f8ef40ca910 2023-10-10T20:19:46.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.018+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8ef40ca080 con 0x7f8ef40ca910 2023-10-10T20:19:46.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.019+0000 7f8ee8ff9700 1 -- 172.21.15.148:0/1824849701 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8ee4055cd0 con 0x7f8ef40ca910 2023-10-10T20:19:46.021 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:19:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 msgr2=0x7f8ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 0x7f8ee00693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007e40 tx=0x7f8ef0007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 msgr2=0x7f8ef40b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4011da0 tx=0x7f8ee40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 shutdown_connections 2023-10-10T20:19:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ee0066f30 0x7f8ee00693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef40b55f0 0x7f8ef416a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ef4155d40 0x7f8ef40ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 --2- 172.21.15.148:0/1824849701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ef40ca910 0x7f8ef40b50b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 >> 172.21.15.148:0/1824849701 conn(0x7f8ef4067660 msgr2=0x7f8ef40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 shutdown_connections 2023-10-10T20:19:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:46.021+0000 7f8ef9f9d700 1 -- 172.21.15.148:0/1824849701 wait complete. 2023-10-10T20:19:46.034 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:19:46.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:19:46.035 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:19:46.035 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:19:46.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:19:46.036 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:19:46.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:19:46.037 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:19:46.037 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:19:46.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.038 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:19:46.056 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:46.057 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:46.057 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:46.057 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:46.058 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:46.058 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:46.058 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:46.059 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:46.059 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:19:46.060 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:46.060 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:46.060 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:19:46.060 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:46.061 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:46.061 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:19:46.061 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:46.062 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:19:46.062 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42521sec preferred_lft 42521sec 2023-10-10T20:19:46.062 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:19:46.063 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:46.063 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-10T20:19:46.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.065 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:46.066 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:46.066 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2023-10-10T20:19:46.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2023-10-10T20:19:46.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2023-10-10T20:19:46.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-10T20:19:46.069 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:46.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.230 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:19:46.271 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:19:46.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.272 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:19:46.349 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:19:46.349 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:19:46.350 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:19:46.350 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:19:46.350 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:19:46.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.351 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:46.351 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:46.351 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:19:46.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:19:46.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:19:46.352 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:46.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.622 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:19:46.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.637 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2023-10-10T20:19:46.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.691 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:46.692 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:46.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:19:46.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-10T20:19:46.692 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:46.780 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-10T20:19:46.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.781 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:46.781 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:46.781 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-10T20:19:46.781 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-10T20:19:46.782 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-10T20:19:46.782 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-10T20:19:46.782 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-10T20:19:46.783 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:46.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:46.996 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:46.996 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:46.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:19:46.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2023-10-10T20:19:46.997 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:47.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:47.090 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:19:47.105 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:19:47.105 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:47.105 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:19:47.159 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:19:47.160 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:19:47.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:47.250 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:47.251 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:47.339 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:47.339 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:47.339 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:47.339 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:47.340 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:47.340 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:47.340 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:47.340 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:47.359 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:47.360 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:47.360 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:47.360 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:47.361 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:47.361 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:47.361 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:47.362 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:47.362 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:19:47.363 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:19:47.363 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:19:47.365 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:19:47.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:47.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:19:47.391 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-10T20:19:47.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:19:47.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.517+0000 7f7220270700 1 Processor -- start 2023-10-10T20:19:47.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.517+0000 7f7220270700 1 -- start start 2023-10-10T20:19:47.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.518+0000 7f7220270700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f721806f5c0 0x7f721815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.518+0000 7f7220270700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7218068fb0 con 0x7f721815b020 2023-10-10T20:19:47.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.518+0000 7f7220270700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7218067800 con 0x7f721815a710 2023-10-10T20:19:47.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.518+0000 7f7220270700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721815b410 con 0x7f721806f5c0 2023-10-10T20:19:47.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721e80d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f721815b020 0x7f7218070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51244/0 (socket says 172.21.15.148:51244) 2023-10-10T20:19:47.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721e80d700 1 -- 172.21.15.148:0/2478386957 learned_addr learned my addr 172.21.15.148:0/2478386957 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:47.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d80b700 1 --2- 172.21.15.148:0/2478386957 >> v2:172.21.15.161:3300/0 conn(0x7f721806f5c0 0x7f721815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105653822 0 0) 0x7f7218068fb0 con 0x7f721815b020 2023-10-10T20:19:47.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7204003570 con 0x7f721815b020 2023-10-10T20:19:47.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589837143 0 0) 0x7f7218067800 con 0x7f721815a710 2023-10-10T20:19:47.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7218068fb0 con 0x7f721815a710 2023-10-10T20:19:47.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091252975 0 0) 0x7f7204003570 con 0x7f721815b020 2023-10-10T20:19:47.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 >> v2:172.21.15.161:3300/0 conn(0x7f721806f5c0 msgr2=0x7f721815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:47.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 --2- 172.21.15.148:0/2478386957 >> v2:172.21.15.161:3300/0 conn(0x7f721806f5c0 0x7f721815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 >> v1:172.21.15.161:6789/0 conn(0x7f721815a710 legacy=0x7f72180688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218067ae0 con 0x7f721815b020 2023-10-10T20:19:47.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7210001470 con 0x7f721815b020 2023-10-10T20:19:47.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 >> v1:172.21.15.148:6789/0 conn(0x7f721815b020 legacy=0x7f7218070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7204004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7204004810 0x7f7204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7204005210 0x7f720400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721815b410 con 0x7f7204003c40 2023-10-10T20:19:47.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720400c130 con 0x7f7204004810 2023-10-10T20:19:47.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720400c2c0 con 0x7f7204005210 2023-10-10T20:19:47.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721e00c700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7204004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721d80b700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7204004810 0x7f7204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.519+0000 7f721e80d700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7204005210 0x7f720400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 -- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7204005210 msgr2=0x7f720400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:47.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7204005210 0x7f720400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 -- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7204004810 msgr2=0x7f7204004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7204004810 0x7f7204004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e80d700 1 -- 172.21.15.148:0/2478386957 reap_dead start 2023-10-10T20:19:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 -- 172.21.15.148:0/2478386957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204003570 con 0x7f7204003c40 2023-10-10T20:19:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721d80b700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7204004810 0x7f7204004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721e00c700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7204004030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f72140020f0 tx=0x7f72140017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dbf76a3532c77a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214001ab0 con 0x7f7204003c40 2023-10-10T20:19:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400c830 con 0x7f7204003c40 2023-10-10T20:19:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f721d00a700 1 -- 172.21.15.148:0/2478386957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400cbd0 con 0x7f7204003c40 2023-10-10T20:19:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f7220270700 1 -- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 msgr2=0x7f7204004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.520+0000 7f7220270700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7204004030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f72140020f0 tx=0x7f72140017a0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.521+0000 7f7220270700 1 -- 172.21.15.148:0/2478386957 shutdown_connections 2023-10-10T20:19:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.521+0000 7f7220270700 1 --2- 172.21.15.148:0/2478386957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7204004030 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.521+0000 7f7220270700 1 -- 172.21.15.148:0/2478386957 >> 172.21.15.148:0/2478386957 conn(0x7f721806a250 msgr2=0x7f721806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.521+0000 7f7220270700 1 -- 172.21.15.148:0/2478386957 shutdown_connections 2023-10-10T20:19:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.521+0000 7f7220270700 1 -- 172.21.15.148:0/2478386957 wait complete. 2023-10-10T20:19:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 Processor -- start 2023-10-10T20:19:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 -- start start 2023-10-10T20:19:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72180ae060 0x7f72180b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72180ae4d0 0x7f72180b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721816b430 con 0x7f7204003c40 2023-10-10T20:19:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72180b32b0 con 0x7f72180ae4d0 2023-10-10T20:19:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.522+0000 7f7220270700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72180b23a0 con 0x7f72180ae060 2023-10-10T20:19:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46958/0 (socket says 172.21.15.148:46958) 2023-10-10T20:19:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 -- 172.21.15.148:0/3938212563 learned_addr learned my addr 172.21.15.148:0/3938212563 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721d80b700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72180ae060 0x7f72180b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e80d700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72180ae4d0 0x7f72180b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 -- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72180ae060 msgr2=0x7f72180b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72180ae060 0x7f72180b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 -- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72180ae4d0 msgr2=0x7f72180b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72180ae4d0 0x7f72180b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 -- 172.21.15.148:0/3938212563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218153bd0 con 0x7f7204003c40 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f721e00c700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f721400c6a0 tx=0x7f7214016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214001ab0 con 0x7f7204003c40 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400cbb0 con 0x7f7204003c40 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72180b1a70 con 0x7f7204003c40 2023-10-10T20:19:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214006ba0 con 0x7f7204003c40 2023-10-10T20:19:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.523+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72180b2030 con 0x7f7204003c40 2023-10-10T20:19:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.524+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721804c3c0 con 0x7f7204003c40 2023-10-10T20:19:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.524+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7214024020 con 0x7f7204003c40 2023-10-10T20:19:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.525+0000 7f720e7fc700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 0x7f71fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.525+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72140864c0 con 0x7f7204003c40 2023-10-10T20:19:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.525+0000 7f721d80b700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 0x7f71fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.525+0000 7f721d80b700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 0x7f71fc0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7208009720 tx=0x7f72080091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.529+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7214053730 con 0x7f7204003c40 2023-10-10T20:19:47.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.758+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7218154480 con 0x7f7204003c40 2023-10-10T20:19:47.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.758+0000 7f720e7fc700 1 -- 172.21.15.148:0/3938212563 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f721402c020 con 0x7f7204003c40 2023-10-10T20:19:47.761 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:19:47.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.760+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 msgr2=0x7f71fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.760+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 0x7f71fc0693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7208009720 tx=0x7f72080091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.760+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 msgr2=0x7f7218153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:47.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.760+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f721400c6a0 tx=0x7f7214016000 comp rx=0 tx=0).stop 2023-10-10T20:19:47.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.760+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 shutdown_connections 2023-10-10T20:19:47.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f71fc066f30 0x7f71fc0693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72180ae4d0 0x7f72180b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7204003c40 0x7f7218153690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 --2- 172.21.15.148:0/3938212563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72180ae060 0x7f72180b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:47.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 >> 172.21.15.148:0/3938212563 conn(0x7f721806a250 msgr2=0x7f72180cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:47.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 shutdown_connections 2023-10-10T20:19:47.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:47.761+0000 7f7220270700 1 -- 172.21.15.148:0/3938212563 wait complete. 2023-10-10T20:19:47.774 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-10T20:19:47.774 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:19:47.775 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:19:47.775 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-10T20:19:47.775 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-10T20:19:47.775 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-10T20:19:47.776 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-10T20:19:47.776 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:19:47.776 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-10T20:19:47.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:47.777 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:19:47.792 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:47.793 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:47.793 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:47.793 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.793 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:47.794 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.794 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:47.794 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:47.794 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:19:47.794 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:47.795 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:47.795 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:19:47.795 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:19:47.795 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:47.795 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:19:47.796 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:47.796 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:19:47.796 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42520sec preferred_lft 42520sec 2023-10-10T20:19:47.796 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:19:47.796 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:19:47.797 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.798 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:19:47.798 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:19:47.798 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link tentative 2023-10-10T20:19:47.798 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:47.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:47.799 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:19:47.856 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:19:47.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:47.858 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:19:47.916 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:19:47.916 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:19:47.916 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:19:47.916 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:19:47.917 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:19:47.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:47.917 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:47.918 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:47.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:19:47.918 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:19:47.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:19:47.919 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:48.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.017 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:19:48.032 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:19:48.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.034 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2023-10-10T20:19:48.087 INFO:teuthology.orchestra.run.smithi161.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-10T20:19:48.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.088 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:48.088 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:48.088 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:19:48.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-10T20:19:48.089 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:48.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.176 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-10T20:19:48.210 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:48.210 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.211 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:19:48.211 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 06:dd:43:8a:c3:2b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:19:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:19:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.212 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::4dd:43ff:fe8a:c32b/64 scope link tentative 2023-10-10T20:19:48.213 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.222 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-10T20:19:48.297 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:19:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:19:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:19:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:19:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.298 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:19:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 06:dd:43:8a:c3:2b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-10T20:19:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-10T20:19:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::4dd:43ff:fe8a:c32b/64 scope link tentative 2023-10-10T20:19:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:19:48.308 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-10T20:19:48.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.309 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:48.309 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:48.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-10T20:19:48.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-10T20:19:48.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-10T20:19:48.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-10T20:19:48.311 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-10T20:19:48.311 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:48.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.517 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:19:48.517 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:19:48.517 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.1 up 2023-10-10T20:19:48.518 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.1 master ceph-brx 2023-10-10T20:19:48.518 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:19:48.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:48.598 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-10T20:19:48.613 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-10T20:19:48.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:19:48.614 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-10T20:19:48.667 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-10T20:19:48.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:19:48.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:48.749 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:48.749 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:48.767 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:48.768 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:48.768 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:48.768 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 3@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:48.768 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:48.768 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:48.769 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:48.769 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:48.788 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:19:48.789 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:19:48.789 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:19:48.789 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 3@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:19:48.789 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:19:48.790 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:19:48.790 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:19:48.790 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:19:48.790 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:19:48.791 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:19:48.791 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:19:48.792 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:19:48.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:48.793 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-10T20:19:48.856 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-10T20:19:48.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:48.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.992+0000 7f5337789700 1 Processor -- start 2023-10-10T20:19:48.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 -- start start 2023-10-10T20:19:48.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5330157870 0x7f533015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:48.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 0x7f533015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:48.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533015cbb0 con 0x7f533015d580 2023-10-10T20:19:48.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533015bfa0 con 0x7f5330157870 2023-10-10T20:19:48.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.993+0000 7f5337789700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533015c280 con 0x7f533015a620 2023-10-10T20:19:48.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5330157870 0x7f533015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:48.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5330157870 0x7f533015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46968/0 (socket says 172.21.15.148:46968) 2023-10-10T20:19:48.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335d26700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f533015d580 0x7f5330158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51256/0 (socket says 172.21.15.148:51256) 2023-10-10T20:19:48.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 -- 172.21.15.148:0/2553668059 learned_addr learned my addr 172.21.15.148:0/2553668059 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5334d24700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 0x7f533015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 -- 172.21.15.148:0/2553668059 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 msgr2=0x7f533015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 0x7f533015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 -- 172.21.15.148:0/2553668059 >> v1:172.21.15.148:6789/0 conn(0x7f533015d580 legacy=0x7f5330158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 -- 172.21.15.148:0/2553668059 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533015c560 con 0x7f5330157870 2023-10-10T20:19:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5327fff700 1 -- 172.21.15.148:0/2553668059 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393536071 0 0) 0x7f533015cbb0 con 0x7f533015d580 2023-10-10T20:19:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5334d24700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 0x7f533015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5335525700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f533015b6a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f532c009f20 tx=0x7f532c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c21357e602b714 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5327fff700 1 -- 172.21.15.148:0/2553668059 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00e070 con 0x7f5330157870 2023-10-10T20:19:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5327fff700 1 -- 172.21.15.148:0/2553668059 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c00ba80 con 0x7f5330157870 2023-10-10T20:19:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.994+0000 7f5327fff700 1 -- 172.21.15.148:0/2553668059 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0092e0 con 0x7f5330157870 2023-10-10T20:19:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 -- 172.21.15.148:0/2553668059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 msgr2=0x7f533015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f533015b6a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f532c009f20 tx=0x7f532c009840 comp rx=0 tx=0).stop 2023-10-10T20:19:49.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 -- 172.21.15.148:0/2553668059 shutdown_connections 2023-10-10T20:19:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f533015b6a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 --2- 172.21.15.148:0/2553668059 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015a620 0x7f533015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 -- 172.21.15.148:0/2553668059 >> 172.21.15.148:0/2553668059 conn(0x7f5330068ce0 msgr2=0x7f53300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 -- 172.21.15.148:0/2553668059 shutdown_connections 2023-10-10T20:19:49.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.995+0000 7f5337789700 1 -- 172.21.15.148:0/2553668059 wait complete. 2023-10-10T20:19:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.996+0000 7f5337789700 1 Processor -- start 2023-10-10T20:19:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.996+0000 7f5337789700 1 -- start start 2023-10-10T20:19:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f533015a620 0x7f533016af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015d580 0x7f533015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5330147dd0 con 0x7f5330157870 2023-10-10T20:19:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53300b1230 con 0x7f533015a620 2023-10-10T20:19:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5337789700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533016b480 con 0x7f533015d580 2023-10-10T20:19:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46980/0 (socket says 172.21.15.148:46980) 2023-10-10T20:19:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 -- 172.21.15.148:0/2551643469 learned_addr learned my addr 172.21.15.148:0/2551643469 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5334d24700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f533015a620 0x7f533016af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335d26700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015d580 0x7f533015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 -- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015d580 msgr2=0x7f533015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015d580 0x7f533015dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 -- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f533015a620 msgr2=0x7f533016af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f533015a620 0x7f533016af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 -- 172.21.15.148:0/2551643469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009ae0 con 0x7f5330157870 2023-10-10T20:19:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f5335525700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f532c00b6c0 tx=0x7f532c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.997+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c008a00 con 0x7f5330157870 2023-10-10T20:19:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.998+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c008b60 con 0x7f5330157870 2023-10-10T20:19:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.998+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c01c410 con 0x7f5330157870 2023-10-10T20:19:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.998+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53300c60d0 con 0x7f5330157870 2023-10-10T20:19:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.998+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53300c6690 con 0x7f5330157870 2023-10-10T20:19:49.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.999+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53300b13e0 con 0x7f5330157870 2023-10-10T20:19:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.999+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f532c003040 con 0x7f5330157870 2023-10-10T20:19:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.999+0000 7f53257fa700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 0x7f53140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.999+0000 7f5334d24700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 0x7f53140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:48.999+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f532c085b80 con 0x7f5330157870 2023-10-10T20:19:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.000+0000 7f5334d24700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 0x7f53140693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5320006f00 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.004+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f532c052df0 con 0x7f5330157870 2023-10-10T20:19:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.187+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f533004c3c0 con 0x7f5330157870 2023-10-10T20:19:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.188+0000 7f53257fa700 1 -- 172.21.15.148:0/2551643469 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f532c056410 con 0x7f5330157870 2023-10-10T20:19:49.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:49.190 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 msgr2=0x7f53140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 0x7f53140693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5320006f00 tx=0x7f5320008040 comp rx=0 tx=0).stop 2023-10-10T20:19:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 msgr2=0x7f5330147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f532c00b6c0 tx=0x7f532c018000 comp rx=0 tx=0).stop 2023-10-10T20:19:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 shutdown_connections 2023-10-10T20:19:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5314066f30 0x7f53140693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f533015d580 0x7f533015dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5330157870 0x7f5330147890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 --2- 172.21.15.148:0/2551643469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f533015a620 0x7f533016af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 >> 172.21.15.148:0/2551643469 conn(0x7f5330068ce0 msgr2=0x7f533015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 shutdown_connections 2023-10-10T20:19:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.190+0000 7f5337789700 1 -- 172.21.15.148:0/2551643469 wait complete. 2023-10-10T20:19:49.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:49.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:49.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.320+0000 7f05cc172700 1 Processor -- start 2023-10-10T20:19:49.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.320+0000 7f05cc172700 1 -- start start 2023-10-10T20:19:49.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05cc172700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f05c415a290 0x7f05c406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05cc172700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05c406aae0 0x7f05c40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05cc172700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c40b0790 con 0x7f05c4159ec0 2023-10-10T20:19:49.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05cc172700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c406af20 con 0x7f05c406aae0 2023-10-10T20:19:49.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05cc172700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40ce1f0 con 0x7f05c415a290 2023-10-10T20:19:49.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05c9f0e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f05c4159ec0 0x7f05c40af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51270/0 (socket says 172.21.15.148:51270) 2023-10-10T20:19:49.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05ca70f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05c406aae0 0x7f05c40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05c9f0e700 1 -- 172.21.15.148:0/2611650079 learned_addr learned my addr 172.21.15.148:0/2611650079 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:49.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05c970d700 1 --2- 172.21.15.148:0/2611650079 >> v2:172.21.15.161:3300/0 conn(0x7f05c415a290 0x7f05c406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.321+0000 7f05c8f0c700 1 -- 172.21.15.148:0/2611650079 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084298831 0 0) 0x7f05c40b0790 con 0x7f05c4159ec0 2023-10-10T20:19:49.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05c8f0c700 1 -- 172.21.15.148:0/2611650079 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b0003570 con 0x7f05c4159ec0 2023-10-10T20:19:49.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05ca70f700 1 -- 172.21.15.148:0/2611650079 >> v2:172.21.15.161:3300/0 conn(0x7f05c415a290 msgr2=0x7f05c406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:49.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05ca70f700 1 --2- 172.21.15.148:0/2611650079 >> v2:172.21.15.161:3300/0 conn(0x7f05c415a290 0x7f05c406a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05ca70f700 1 -- 172.21.15.148:0/2611650079 >> v1:172.21.15.148:6789/0 conn(0x7f05c4159ec0 legacy=0x7f05c40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05ca70f700 1 -- 172.21.15.148:0/2611650079 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40b0b20 con 0x7f05c406aae0 2023-10-10T20:19:49.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05ca70f700 1 --2- 172.21.15.148:0/2611650079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c406aae0 0x7f05c40cf590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05c4089910 tx=0x7f05bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce1240ece88db15 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05c8f0c700 1 -- 172.21.15.148:0/2611650079 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00e070 con 0x7f05c406aae0 2023-10-10T20:19:49.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05c8f0c700 1 -- 172.21.15.148:0/2611650079 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc00bab0 con 0x7f05c406aae0 2023-10-10T20:19:49.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05c8f0c700 1 -- 172.21.15.148:0/2611650079 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00be70 con 0x7f05c406aae0 2023-10-10T20:19:49.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 -- 172.21.15.148:0/2611650079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c406aae0 msgr2=0x7f05c40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 --2- 172.21.15.148:0/2611650079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c406aae0 0x7f05c40cf590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05c4089910 tx=0x7f05bc009780 comp rx=0 tx=0).stop 2023-10-10T20:19:49.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 -- 172.21.15.148:0/2611650079 shutdown_connections 2023-10-10T20:19:49.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 --2- 172.21.15.148:0/2611650079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c406aae0 0x7f05c40cf590 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 --2- 172.21.15.148:0/2611650079 >> v2:172.21.15.161:3300/0 conn(0x7f05c415a290 0x7f05c406a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 -- 172.21.15.148:0/2611650079 >> 172.21.15.148:0/2611650079 conn(0x7f05c40d1880 msgr2=0x7f05c40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 -- 172.21.15.148:0/2611650079 shutdown_connections 2023-10-10T20:19:49.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.322+0000 7f05cc172700 1 -- 172.21.15.148:0/2611650079 wait complete. 2023-10-10T20:19:49.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 Processor -- start 2023-10-10T20:19:49.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 -- start start 2023-10-10T20:19:49.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c406aae0 0x7f05c40ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c4159ec0 0x7f05c415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40b4030 con 0x7f05c415a290 2023-10-10T20:19:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40b4310 con 0x7f05c406aae0 2023-10-10T20:19:49.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05cc172700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40b45f0 con 0x7f05c4159ec0 2023-10-10T20:19:49.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05ca70f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05ca70f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46986/0 (socket says 172.21.15.148:46986) 2023-10-10T20:19:49.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05ca70f700 1 -- 172.21.15.148:0/245332251 learned_addr learned my addr 172.21.15.148:0/245332251 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05c970d700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c4159ec0 0x7f05c415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.324+0000 7f05c9f0e700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c406aae0 0x7f05c40ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 -- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c4159ec0 msgr2=0x7f05c415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c4159ec0 0x7f05c415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 -- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c406aae0 msgr2=0x7f05c40ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c406aae0 0x7f05c40ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 -- 172.21.15.148:0/245332251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009a20 con 0x7f05c415a290 2023-10-10T20:19:49.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05ca70f700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05c4071eb0 tx=0x7f05bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc012d00 con 0x7f05c415a290 2023-10-10T20:19:49.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc012e80 con 0x7f05c415a290 2023-10-10T20:19:49.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c40b48d0 con 0x7f05c415a290 2023-10-10T20:19:49.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc0276c0 con 0x7f05c415a290 2023-10-10T20:19:49.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.325+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c40c8920 con 0x7f05c415a290 2023-10-10T20:19:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.326+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c404c3c0 con 0x7f05c415a290 2023-10-10T20:19:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.327+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05bc024070 con 0x7f05c415a290 2023-10-10T20:19:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.327+0000 7f05b9ffb700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.327+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05bc086a40 con 0x7f05c415a290 2023-10-10T20:19:49.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.327+0000 7f05c9f0e700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.327+0000 7f05c9f0e700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 0x7f05a80693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05b4007fd0 tx=0x7f05b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.331+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05bc088e60 con 0x7f05c415a290 2023-10-10T20:19:49.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.513+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05c40b04b0 con 0x7f05c415a290 2023-10-10T20:19:49.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.513+0000 7f05b9ffb700 1 -- 172.21.15.148:0/245332251 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05bc02c0a0 con 0x7f05c415a290 2023-10-10T20:19:49.515 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 msgr2=0x7f05a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 0x7f05a80693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05b4007fd0 tx=0x7f05b4009040 comp rx=0 tx=0).stop 2023-10-10T20:19:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 msgr2=0x7f05c415df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f05c4071eb0 tx=0x7f05bc019000 comp rx=0 tx=0).stop 2023-10-10T20:19:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 shutdown_connections 2023-10-10T20:19:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f05a8066f30 0x7f05a80693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05c415a290 0x7f05c415df00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c406aae0 0x7f05c40ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 --2- 172.21.15.148:0/245332251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c4159ec0 0x7f05c415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 >> 172.21.15.148:0/245332251 conn(0x7f05c40d1880 msgr2=0x7f05c40d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 shutdown_connections 2023-10-10T20:19:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.515+0000 7f05cc172700 1 -- 172.21.15.148:0/245332251 wait complete. 2023-10-10T20:19:49.529 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.645+0000 7fe15af14700 1 Processor -- start 2023-10-10T20:19:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.645+0000 7fe15af14700 1 -- start start 2023-10-10T20:19:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe15af14700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1540c9830 0x7fe1540afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe15af14700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe1540b12c0 0x7fe15415c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe15af14700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540b0e30 con 0x7fe15415afe0 2023-10-10T20:19:49.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe15af14700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540af670 con 0x7fe1540c9830 2023-10-10T20:19:49.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe15af14700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15415c870 con 0x7fe1540b12c0 2023-10-10T20:19:49.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe153fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1540c9830 0x7fe1540afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe153fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1540c9830 0x7fe1540afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47002/0 (socket says 172.21.15.148:47002) 2023-10-10T20:19:49.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe153fff700 1 -- 172.21.15.148:0/3995671862 learned_addr learned my addr 172.21.15.148:0/3995671862 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.646+0000 7fe158cb0700 1 --1- 172.21.15.148:0/3995671862 >> v1:172.21.15.148:6789/0 conn(0x7fe15415afe0 0x7fe1540c9120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51278/0 (socket says 172.21.15.148:51278) 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe1594b1700 1 --2- 172.21.15.148:0/3995671862 >> v2:172.21.15.161:3300/0 conn(0x7fe1540b12c0 0x7fe15415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe153fff700 1 -- 172.21.15.148:0/3995671862 >> v2:172.21.15.161:3300/0 conn(0x7fe1540b12c0 msgr2=0x7fe15415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe153fff700 1 --2- 172.21.15.148:0/3995671862 >> v2:172.21.15.161:3300/0 conn(0x7fe1540b12c0 0x7fe15415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe153fff700 1 -- 172.21.15.148:0/3995671862 >> v1:172.21.15.148:6789/0 conn(0x7fe15415afe0 legacy=0x7fe1540c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe1537fe700 1 -- 172.21.15.148:0/3995671862 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250124093 0 0) 0x7fe1540b0e30 con 0x7fe15415afe0 2023-10-10T20:19:49.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe153fff700 1 -- 172.21.15.148:0/3995671862 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540b0220 con 0x7fe1540c9830 2023-10-10T20:19:49.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe153fff700 1 --2- 172.21.15.148:0/3995671862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe1540afb60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1540b4be0 tx=0x7fe148009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e10f4485dfc9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe1537fe700 1 -- 172.21.15.148:0/3995671862 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800b860 con 0x7fe1540c9830 2023-10-10T20:19:49.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe1537fe700 1 -- 172.21.15.148:0/3995671862 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14800b9c0 con 0x7fe1540c9830 2023-10-10T20:19:49.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe1537fe700 1 -- 172.21.15.148:0/3995671862 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800bd60 con 0x7fe1540c9830 2023-10-10T20:19:49.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 -- 172.21.15.148:0/3995671862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 msgr2=0x7fe1540afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 --2- 172.21.15.148:0/3995671862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe1540afb60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1540b4be0 tx=0x7fe148009250 comp rx=0 tx=0).stop 2023-10-10T20:19:49.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 -- 172.21.15.148:0/3995671862 shutdown_connections 2023-10-10T20:19:49.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 --2- 172.21.15.148:0/3995671862 >> v2:172.21.15.161:3300/0 conn(0x7fe1540b12c0 0x7fe15415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 --2- 172.21.15.148:0/3995671862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe1540afb60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.647+0000 7fe15af14700 1 -- 172.21.15.148:0/3995671862 >> 172.21.15.148:0/3995671862 conn(0x7fe15406f3e0 msgr2=0x7fe15406f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.648+0000 7fe15af14700 1 -- 172.21.15.148:0/3995671862 shutdown_connections 2023-10-10T20:19:49.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.648+0000 7fe15af14700 1 -- 172.21.15.148:0/3995671862 wait complete. 2023-10-10T20:19:49.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 Processor -- start 2023-10-10T20:19:49.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 -- start start 2023-10-10T20:19:49.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 0x7fe1540b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 0x7fe1540b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b2f70 con 0x7fe1540c9830 2023-10-10T20:19:49.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b2360 con 0x7fe15415afe0 2023-10-10T20:19:49.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.649+0000 7fe15af14700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b2640 con 0x7fe1540b12c0 2023-10-10T20:19:49.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47006/0 (socket says 172.21.15.148:47006) 2023-10-10T20:19:49.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 -- 172.21.15.148:0/725975126 learned_addr learned my addr 172.21.15.148:0/725975126 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe1594b1700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 0x7fe1540b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe158cb0700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 0x7fe1540b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 -- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 msgr2=0x7fe1540b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 0x7fe1540b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 -- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 msgr2=0x7fe1540b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 0x7fe1540b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 -- 172.21.15.148:0/725975126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480094f0 con 0x7fe1540c9830 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe158cb0700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 0x7fe1540b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:49.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe1594b1700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 0x7fe1540b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:49.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe153fff700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe148006f50 tx=0x7fe148017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148024440 con 0x7fe1540c9830 2023-10-10T20:19:49.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480245c0 con 0x7fe1540c9830 2023-10-10T20:19:49.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1540b2920 con 0x7fe1540c9830 2023-10-10T20:19:49.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148024980 con 0x7fe1540c9830 2023-10-10T20:19:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.650+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1541517b0 con 0x7fe1540c9830 2023-10-10T20:19:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.651+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15404c3c0 con 0x7fe1540c9830 2023-10-10T20:19:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.651+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe14801a420 con 0x7fe1540c9830 2023-10-10T20:19:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.652+0000 7fe150ff9700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.652+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe148086480 con 0x7fe1540c9830 2023-10-10T20:19:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.652+0000 7fe158cb0700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.652+0000 7fe158cb0700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 0x7fe1400693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe144007fd0 tx=0x7fe144007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.656+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe148053770 con 0x7fe1540c9830 2023-10-10T20:19:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.839+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1540b0500 con 0x7fe1540c9830 2023-10-10T20:19:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.839+0000 7fe150ff9700 1 -- 172.21.15.148:0/725975126 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe148056d90 con 0x7fe1540c9830 2023-10-10T20:19:49.842 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 msgr2=0x7fe1400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 0x7fe1400693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe144007fd0 tx=0x7fe144007e30 comp rx=0 tx=0).stop 2023-10-10T20:19:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 msgr2=0x7fe154151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe148006f50 tx=0x7fe148017000 comp rx=0 tx=0).stop 2023-10-10T20:19:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 shutdown_connections 2023-10-10T20:19:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe15415afe0 0x7fe1540b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1540b12c0 0x7fe1540b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 --2- 172.21.15.148:0/725975126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1540c9830 0x7fe154151270 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 >> 172.21.15.148:0/725975126 conn(0x7fe15406f3e0 msgr2=0x7fe1541520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 shutdown_connections 2023-10-10T20:19:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.841+0000 7fe15af14700 1 -- 172.21.15.148:0/725975126 wait complete. 2023-10-10T20:19:49.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.970+0000 7fd388928700 1 Processor -- start 2023-10-10T20:19:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.970+0000 7fd388928700 1 -- start start 2023-10-10T20:19:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.970+0000 7fd388928700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.970+0000 7fd388928700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd388928700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3840b4d70 con 0x7fd38406a830 2023-10-10T20:19:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd388928700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38406ac70 con 0x7fd3840b6580 2023-10-10T20:19:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd388928700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840cd8d0 con 0x7fd3840cce30 2023-10-10T20:19:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd37a59c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:35668/0 (socket says 172.21.15.148:35668) 2023-10-10T20:19:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 -- 172.21.15.148:0/3641055222 learned_addr learned my addr 172.21.15.148:0/3641055222 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd37a59c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:40162/0 (socket says 172.21.15.148:40162) 2023-10-10T20:19:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd38359e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd38406a830 0x7fd38415d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:49668/0 (socket says 172.21.15.148:49668) 2023-10-10T20:19:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 -- 172.21.15.148:0/3641055222 >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 msgr2=0x7fd38406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 --2- 172.21.15.148:0/3641055222 >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 -- 172.21.15.148:0/3641055222 >> v1:172.21.15.161:6789/0 conn(0x7fd38406a830 legacy=0x7fd38415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 -- 172.21.15.148:0/3641055222 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840b4440 con 0x7fd3840cce30 2023-10-10T20:19:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd37a59c700 1 --2- 172.21.15.148:0/3641055222 >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:49.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd382d9d700 1 --2- 172.21.15.148:0/3641055222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd3740099b0 tx=0x7fd374009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af523b24c20b8713 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd38259c700 1 -- 172.21.15.148:0/3641055222 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374009ea0 con 0x7fd3840cce30 2023-10-10T20:19:49.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd38259c700 1 -- 172.21.15.148:0/3641055222 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400b860 con 0x7fd3840cce30 2023-10-10T20:19:49.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.971+0000 7fd38259c700 1 -- 172.21.15.148:0/3641055222 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bc00 con 0x7fd3840cce30 2023-10-10T20:19:49.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 -- 172.21.15.148:0/3641055222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 msgr2=0x7fd38415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 --2- 172.21.15.148:0/3641055222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd3740099b0 tx=0x7fd374009250 comp rx=0 tx=0).stop 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 -- 172.21.15.148:0/3641055222 shutdown_connections 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 --2- 172.21.15.148:0/3641055222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840cce30 0x7fd38415aca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 --2- 172.21.15.148:0/3641055222 >> v2:172.21.15.161:3300/0 conn(0x7fd3840b6580 0x7fd38406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 -- 172.21.15.148:0/3641055222 >> 172.21.15.148:0/3641055222 conn(0x7fd384068cb0 msgr2=0x7fd3840690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:49.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 -- 172.21.15.148:0/3641055222 shutdown_connections 2023-10-10T20:19:49.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.972+0000 7fd388928700 1 -- 172.21.15.148:0/3641055222 wait complete. 2023-10-10T20:19:49.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.973+0000 7fd388928700 1 Processor -- start 2023-10-10T20:19:49.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.973+0000 7fd388928700 1 -- start start 2023-10-10T20:19:49.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840b6580 0x7fd38414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 0x7fd3840d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840b0be0 con 0x7fd38406a830 2023-10-10T20:19:49.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd384151b80 con 0x7fd3840cce30 2023-10-10T20:19:49.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd388928700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd384151e60 con 0x7fd3840b6580 2023-10-10T20:19:49.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47018/0 (socket says 172.21.15.148:47018) 2023-10-10T20:19:49.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 -- 172.21.15.148:0/345147536 learned_addr learned my addr 172.21.15.148:0/345147536 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:49.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd38359e700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 0x7fd3840d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd37a59c700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840b6580 0x7fd38414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 -- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840b6580 msgr2=0x7fd38414b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840b6580 0x7fd38414b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 -- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 msgr2=0x7fd3840d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 0x7fd3840d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd382d9d700 1 -- 172.21.15.148:0/345147536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c009420 con 0x7fd38406a830 2023-10-10T20:19:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.974+0000 7fd38359e700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 0x7fd3840d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd382d9d700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3840cda10 tx=0x7fd374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bbe0 con 0x7fd38406a830 2023-10-10T20:19:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374011420 con 0x7fd38406a830 2023-10-10T20:19:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3740094f0 con 0x7fd38406a830 2023-10-10T20:19:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3740117c0 con 0x7fd38406a830 2023-10-10T20:19:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.975+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd384152420 con 0x7fd38406a830 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.976+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3840b0d50 con 0x7fd38406a830 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.976+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd374011bf0 con 0x7fd38406a830 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.977+0000 7fd37b7fe700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 0x7fd364069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.977+0000 7fd37a59c700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 0x7fd364069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.977+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3740859e0 con 0x7fd38406a830 2023-10-10T20:19:49.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.977+0000 7fd37a59c700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 0x7fd364069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0093f0 tx=0x7fd36c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:49.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:49.981+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd374052cd0 con 0x7fd38406a830 2023-10-10T20:19:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.162+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3840b4720 con 0x7fd38406a830 2023-10-10T20:19:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.163+0000 7fd37b7fe700 1 -- 172.21.15.148:0/345147536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3740562f0 con 0x7fd38406a830 2023-10-10T20:19:50.165 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.164+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 msgr2=0x7fd364069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.164+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 0x7fd364069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0093f0 tx=0x7fd36c008040 comp rx=0 tx=0).stop 2023-10-10T20:19:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 msgr2=0x7fd3841512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3840cda10 tx=0x7fd374017000 comp rx=0 tx=0).stop 2023-10-10T20:19:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 shutdown_connections 2023-10-10T20:19:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd364066f60 0x7fd364069410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3840cce30 0x7fd3840d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd38406a830 0x7fd3841512e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 --2- 172.21.15.148:0/345147536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3840b6580 0x7fd38414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 >> 172.21.15.148:0/345147536 conn(0x7fd384068cb0 msgr2=0x7fd384150420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 shutdown_connections 2023-10-10T20:19:50.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.165+0000 7fd388928700 1 -- 172.21.15.148:0/345147536 wait complete. 2023-10-10T20:19:50.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.291+0000 7fc09f3ce700 1 Processor -- start 2023-10-10T20:19:50.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.291+0000 7fc09f3ce700 1 -- start start 2023-10-10T20:19:50.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09f3ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 0x7fc09806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09f3ce700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc09815b0d0 0x7fc09806aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09f3ce700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09815c360 con 0x7fc0981577d0 2023-10-10T20:19:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09f3ce700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09815b790 con 0x7fc09815b0d0 2023-10-10T20:19:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09f3ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09815d800 con 0x7fc098157ba0 2023-10-10T20:19:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09d96b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc09815b0d0 0x7fc09806aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09d16a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc0981577d0 0x7fc0980cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51288/0 (socket says 172.21.15.148:51288) 2023-10-10T20:19:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09d16a700 1 -- 172.21.15.148:0/2856115217 learned_addr learned my addr 172.21.15.148:0/2856115217 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09c969700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 0x7fc09806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09d96b700 1 -- 172.21.15.148:0/2856115217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 msgr2=0x7fc09806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.292+0000 7fc09d96b700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 0x7fc09806a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09d96b700 1 -- 172.21.15.148:0/2856115217 >> v1:172.21.15.148:6789/0 conn(0x7fc0981577d0 legacy=0x7fc0980cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09d96b700 1 -- 172.21.15.148:0/2856115217 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09815ba30 con 0x7fc09815b0d0 2023-10-10T20:19:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc08ffff700 1 -- 172.21.15.148:0/2856115217 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763199537 0 0) 0x7fc09815c360 con 0x7fc0981577d0 2023-10-10T20:19:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09c969700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 0x7fc09806a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09d96b700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc09815b0d0 0x7fc09806aa60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0900098e0 tx=0x7fc0900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9510d737350200fe server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc08ffff700 1 -- 172.21.15.148:0/2856115217 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090009e80 con 0x7fc09815b0d0 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc08ffff700 1 -- 172.21.15.148:0/2856115217 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09000b860 con 0x7fc09815b0d0 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc08ffff700 1 -- 172.21.15.148:0/2856115217 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000bbe0 con 0x7fc09815b0d0 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 -- 172.21.15.148:0/2856115217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc09815b0d0 msgr2=0x7fc09806aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc09815b0d0 0x7fc09806aa60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0900098e0 tx=0x7fc0900091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 -- 172.21.15.148:0/2856115217 shutdown_connections 2023-10-10T20:19:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc09815b0d0 0x7fc09806aa60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/2856115217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc098157ba0 0x7fc09806a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 -- 172.21.15.148:0/2856115217 >> 172.21.15.148:0/2856115217 conn(0x7fc098068ce0 msgr2=0x7fc0980690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.293+0000 7fc09f3ce700 1 -- 172.21.15.148:0/2856115217 shutdown_connections 2023-10-10T20:19:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.294+0000 7fc09f3ce700 1 -- 172.21.15.148:0/2856115217 wait complete. 2023-10-10T20:19:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 Processor -- start 2023-10-10T20:19:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 -- start start 2023-10-10T20:19:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 0x7fc0980b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 0x7fc09815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09815dc30 con 0x7fc098157ba0 2023-10-10T20:19:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09816b100 con 0x7fc09815b0d0 2023-10-10T20:19:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.295+0000 7fc09f3ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09816a600 con 0x7fc0981577d0 2023-10-10T20:19:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47050/0 (socket says 172.21.15.148:47050) 2023-10-10T20:19:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 -- 172.21.15.148:0/1774709988 learned_addr learned my addr 172.21.15.148:0/1774709988 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09d16a700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 0x7fc0980b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09d96b700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 0x7fc09815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 -- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 msgr2=0x7fc0980b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 0x7fc0980b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 -- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 msgr2=0x7fc09815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 0x7fc09815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 -- 172.21.15.148:0/1774709988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090009420 con 0x7fc098157ba0 2023-10-10T20:19:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09d16a700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 0x7fc0980b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09d96b700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 0x7fc09815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09c969700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c830 tx=0x7fc088008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088011600 con 0x7fc098157ba0 2023-10-10T20:19:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc088011760 con 0x7fc098157ba0 2023-10-10T20:19:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088011b20 con 0x7fc098157ba0 2023-10-10T20:19:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09816a860 con 0x7fc098157ba0 2023-10-10T20:19:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.296+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09816ae20 con 0x7fc098157ba0 2023-10-10T20:19:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.297+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09815dda0 con 0x7fc098157ba0 2023-10-10T20:19:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.297+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc08800a3e0 con 0x7fc098157ba0 2023-10-10T20:19:50.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.298+0000 7fc08d7fa700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 0x7fc07c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.298+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc0880849f0 con 0x7fc098157ba0 2023-10-10T20:19:50.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.298+0000 7fc09d16a700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 0x7fc07c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.298+0000 7fc09d16a700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 0x7fc07c0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e280 tx=0x7fc094009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.302+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc09815dda0 con 0x7fc098157ba0 2023-10-10T20:19:50.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.486+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc09815dd70 con 0x7fc098157ba0 2023-10-10T20:19:50.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.487+0000 7fc08d7fa700 1 -- 172.21.15.148:0/1774709988 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc088015090 con 0x7fc098157ba0 2023-10-10T20:19:50.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:50.489 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:50.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 msgr2=0x7fc07c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 0x7fc07c0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e280 tx=0x7fc094009040 comp rx=0 tx=0).stop 2023-10-10T20:19:50.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 msgr2=0x7fc09816a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c830 tx=0x7fc088008040 comp rx=0 tx=0).stop 2023-10-10T20:19:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 shutdown_connections 2023-10-10T20:19:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc07c066f30 0x7fc07c0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09815b0d0 0x7fc09815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0981577d0 0x7fc0980b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 --2- 172.21.15.148:0/1774709988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc098157ba0 0x7fc09816a0c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 >> 172.21.15.148:0/1774709988 conn(0x7fc098068ce0 msgr2=0x7fc0980d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 shutdown_connections 2023-10-10T20:19:50.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.489+0000 7fc09f3ce700 1 -- 172.21.15.148:0/1774709988 wait complete. 2023-10-10T20:19:50.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:50.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:50.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.620+0000 7fd6d101a700 1 Processor -- start 2023-10-10T20:19:50.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.620+0000 7fd6d101a700 1 -- start start 2023-10-10T20:19:50.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6cc15d190 0x7fd6cc06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b46b0 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b54f0 con 0x7fd6cc15d190 2023-10-10T20:19:50.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6d101a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b48e0 con 0x7fd6cc15d560 2023-10-10T20:19:50.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47052/0 (socket says 172.21.15.148:47052) 2023-10-10T20:19:50.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 -- 172.21.15.148:0/1480695406 learned_addr learned my addr 172.21.15.148:0/1480695406 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6cad9d700 1 --2- 172.21.15.148:0/1480695406 >> v2:172.21.15.161:3300/0 conn(0x7fd6cc15d190 0x7fd6cc06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6cb59e700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 -- 172.21.15.148:0/1480695406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 msgr2=0x7fd6cc15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 -- 172.21.15.148:0/1480695406 >> v2:172.21.15.161:3300/0 conn(0x7fd6cc15d190 msgr2=0x7fd6cc06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:50.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1480695406 >> v2:172.21.15.161:3300/0 conn(0x7fd6cc15d190 0x7fd6cc06a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.621+0000 7fd6c259c700 1 -- 172.21.15.148:0/1480695406 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0b4bc0 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b67d0 tx=0x7fd6b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603c2bbf1bc90d2b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6ca59c700 1 -- 172.21.15.148:0/1480695406 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400b860 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6ca59c700 1 -- 172.21.15.148:0/1480695406 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b400b9c0 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6ca59c700 1 -- 172.21.15.148:0/1480695406 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400bd60 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 -- 172.21.15.148:0/1480695406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc0b68e0 msgr2=0x7fd6cc06aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0b67d0 tx=0x7fd6b4009250 comp rx=0 tx=0).stop 2023-10-10T20:19:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 -- 172.21.15.148:0/1480695406 shutdown_connections 2023-10-10T20:19:50.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1480695406 >> v2:172.21.15.161:3300/0 conn(0x7fd6cc15d190 0x7fd6cc06a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1480695406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc06aa60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 -- 172.21.15.148:0/1480695406 >> 172.21.15.148:0/1480695406 conn(0x7fd6cc068ce0 msgr2=0x7fd6cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 -- 172.21.15.148:0/1480695406 shutdown_connections 2023-10-10T20:19:50.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.622+0000 7fd6d101a700 1 -- 172.21.15.148:0/1480695406 wait complete. 2023-10-10T20:19:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.623+0000 7fd6d101a700 1 Processor -- start 2023-10-10T20:19:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 -- start start 2023-10-10T20:19:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc1462d0 con 0x7fd6cc15d190 2023-10-10T20:19:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc149c30 con 0x7fd6cc0b68e0 2023-10-10T20:19:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6d101a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc149f10 con 0x7fd6cc15d560 2023-10-10T20:19:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47066/0 (socket says 172.21.15.148:47066) 2023-10-10T20:19:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6c259c700 1 -- 172.21.15.148:0/1647726228 learned_addr learned my addr 172.21.15.148:0/1647726228 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6cad9d700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.624+0000 7fd6cb59e700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 -- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 msgr2=0x7fd6cc0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 -- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 msgr2=0x7fd6cc145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 -- 172.21.15.148:0/1647726228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40094f0 con 0x7fd6cc15d190 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6cad9d700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:50.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c259c700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06b530 tx=0x7fd6b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400bcd0 con 0x7fd6cc15d190 2023-10-10T20:19:50.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b401a630 con 0x7fd6cc15d190 2023-10-10T20:19:50.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc14a1f0 con 0x7fd6cc15d190 2023-10-10T20:19:50.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b401a9f0 con 0x7fd6cc15d190 2023-10-10T20:19:50.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.625+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc14d7c0 con 0x7fd6cc15d190 2023-10-10T20:19:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.626+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc14d9b0 con 0x7fd6cc15d190 2023-10-10T20:19:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.626+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6b4024440 con 0x7fd6cc15d190 2023-10-10T20:19:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.627+0000 7fd6c37fe700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 0x7fd6ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.627+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6b4085f50 con 0x7fd6cc15d190 2023-10-10T20:19:50.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.627+0000 7fd6cad9d700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 0x7fd6ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.627+0000 7fd6cad9d700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 0x7fd6ac0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc006fd0 tx=0x7fd6bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.631+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6cc14d9b0 con 0x7fd6cc15d190 2023-10-10T20:19:50.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.812+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6cc14d9b0 con 0x7fd6cc15d190 2023-10-10T20:19:50.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.813+0000 7fd6c37fe700 1 -- 172.21.15.148:0/1647726228 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6cc14d9b0 con 0x7fd6cc15d190 2023-10-10T20:19:50.815 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:50.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 msgr2=0x7fd6ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 0x7fd6ac0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc006fd0 tx=0x7fd6bc008040 comp rx=0 tx=0).stop 2023-10-10T20:19:50.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 msgr2=0x7fd6cc149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06b530 tx=0x7fd6b4017000 comp rx=0 tx=0).stop 2023-10-10T20:19:50.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 shutdown_connections 2023-10-10T20:19:50.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd6ac066f30 0x7fd6ac0693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc15d560 0x7fd6cc0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc0b68e0 0x7fd6cc145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 --2- 172.21.15.148:0/1647726228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6cc15d190 0x7fd6cc149270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 >> 172.21.15.148:0/1647726228 conn(0x7fd6cc068ce0 msgr2=0x7fd6cc14fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 shutdown_connections 2023-10-10T20:19:50.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.815+0000 7fd6d101a700 1 -- 172.21.15.148:0/1647726228 wait complete. 2023-10-10T20:19:50.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:50.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.943+0000 7fd8d885d700 1 Processor -- start 2023-10-10T20:19:50.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.943+0000 7fd8d885d700 1 -- start start 2023-10-10T20:19:50.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d885d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d4159ec0 0x7fd8d40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d885d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8d415a290 0x7fd8d40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d885d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d40b0790 con 0x7fd8d406a5a0 2023-10-10T20:19:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d885d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40ce1f0 con 0x7fd8d415a290 2023-10-10T20:19:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d885d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40b0b20 con 0x7fd8d4159ec0 2023-10-10T20:19:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8d415a290 0x7fd8d40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8d415a290 0x7fd8d40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47078/0 (socket says 172.21.15.148:47078) 2023-10-10T20:19:50.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 -- 172.21.15.148:0/797463796 learned_addr learned my addr 172.21.15.148:0/797463796 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:50.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8d259c700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d4159ec0 0x7fd8d40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 -- 172.21.15.148:0/797463796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d4159ec0 msgr2=0x7fd8d40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:50.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d4159ec0 0x7fd8d40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 -- 172.21.15.148:0/797463796 >> v1:172.21.15.161:6789/0 conn(0x7fd8d406a5a0 legacy=0x7fd8d415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 -- 172.21.15.148:0/797463796 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40b04b0 con 0x7fd8d415a290 2023-10-10T20:19:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.944+0000 7fd8c9d9b700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d40afe80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0099b0 tx=0x7fd8bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a534a781b2f3aee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d1d9b700 1 -- 172.21.15.148:0/797463796 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00b860 con 0x7fd8d415a290 2023-10-10T20:19:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d1d9b700 1 -- 172.21.15.148:0/797463796 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc00b9c0 con 0x7fd8d415a290 2023-10-10T20:19:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d1d9b700 1 -- 172.21.15.148:0/797463796 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00bd60 con 0x7fd8d415a290 2023-10-10T20:19:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 -- 172.21.15.148:0/797463796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 msgr2=0x7fd8d40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d40afe80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0099b0 tx=0x7fd8bc009250 comp rx=0 tx=0).stop 2023-10-10T20:19:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 -- 172.21.15.148:0/797463796 shutdown_connections 2023-10-10T20:19:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d4159ec0 0x7fd8d40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 --2- 172.21.15.148:0/797463796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d40afe80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 -- 172.21.15.148:0/797463796 >> 172.21.15.148:0/797463796 conn(0x7fd8d40d1880 msgr2=0x7fd8d40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 -- 172.21.15.148:0/797463796 shutdown_connections 2023-10-10T20:19:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.945+0000 7fd8d885d700 1 -- 172.21.15.148:0/797463796 wait complete. 2023-10-10T20:19:50.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.946+0000 7fd8d885d700 1 Processor -- start 2023-10-10T20:19:50.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.946+0000 7fd8d885d700 1 -- start start 2023-10-10T20:19:50.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d406a5a0 0x7fd8d415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 0x7fd8d415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d415ec10 con 0x7fd8d415a290 2023-10-10T20:19:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d415eef0 con 0x7fd8d4159ec0 2023-10-10T20:19:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d885d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d415f1d0 con 0x7fd8d406a5a0 2023-10-10T20:19:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47088/0 (socket says 172.21.15.148:47088) 2023-10-10T20:19:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 -- 172.21.15.148:0/2156596723 learned_addr learned my addr 172.21.15.148:0/2156596723 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d259c700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d406a5a0 0x7fd8d415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8c9d9b700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 0x7fd8d415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 -- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d406a5a0 msgr2=0x7fd8d415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d406a5a0 0x7fd8d415e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 -- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 msgr2=0x7fd8d415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.947+0000 7fd8d2d9d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 0x7fd8d415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8d2d9d700 1 -- 172.21.15.148:0/2156596723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0094f0 con 0x7fd8d415a290 2023-10-10T20:19:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8c9d9b700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 0x7fd8d415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8d2d9d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c840 tx=0x7fd8cc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc012630 con 0x7fd8d415a290 2023-10-10T20:19:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc0127b0 con 0x7fd8d415a290 2023-10-10T20:19:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc012b70 con 0x7fd8d415a290 2023-10-10T20:19:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d415f4b0 con 0x7fd8d415a290 2023-10-10T20:19:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.948+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d415e870 con 0x7fd8d415a290 2023-10-10T20:19:50.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.949+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d40c9070 con 0x7fd8d415a290 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.949+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8cc003030 con 0x7fd8d415a290 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.949+0000 7fd8caffd700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 0x7fd8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.950+0000 7fd8d259c700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 0x7fd8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.950+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8cc0844a0 con 0x7fd8d415a290 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.950+0000 7fd8d259c700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 0x7fd8b40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4006fd0 tx=0x7fd8c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:50.954+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8cc051710 con 0x7fd8d415a290 2023-10-10T20:19:51.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.135+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8d404c3c0 con 0x7fd8d415a290 2023-10-10T20:19:51.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.135+0000 7fd8caffd700 1 -- 172.21.15.148:0/2156596723 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8cc054d30 con 0x7fd8d415a290 2023-10-10T20:19:51.138 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:51.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 msgr2=0x7fd8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 0x7fd8b40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4006fd0 tx=0x7fd8c4008040 comp rx=0 tx=0).stop 2023-10-10T20:19:51.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 msgr2=0x7fd8d4156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00c840 tx=0x7fd8cc001670 comp rx=0 tx=0).stop 2023-10-10T20:19:51.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 shutdown_connections 2023-10-10T20:19:51.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8b4066f30 0x7fd8b40693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8d415a290 0x7fd8d4156d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8d406a5a0 0x7fd8d415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 --2- 172.21.15.148:0/2156596723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8d4159ec0 0x7fd8d415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 >> 172.21.15.148:0/2156596723 conn(0x7fd8d40d1880 msgr2=0x7fd8d40b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 shutdown_connections 2023-10-10T20:19:51.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.137+0000 7fd8d885d700 1 -- 172.21.15.148:0/2156596723 wait complete. 2023-10-10T20:19:51.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:51.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.266+0000 7f6664daf700 1 Processor -- start 2023-10-10T20:19:51.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.266+0000 7f6664daf700 1 -- start start 2023-10-10T20:19:51.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f6664daf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666015dcc0 0x7f666015e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f6664daf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f666015e5d0 0x7f666015b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f6664daf700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666015b260 con 0x7f66600cd900 2023-10-10T20:19:51.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f6664daf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666009af70 con 0x7f666015dcc0 2023-10-10T20:19:51.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f6664daf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600c7830 con 0x7f666015e5d0 2023-10-10T20:19:51.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.267+0000 7f665e59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666015dcc0 0x7f666015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666015dcc0 0x7f666015e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47090/0 (socket says 172.21.15.148:47090) 2023-10-10T20:19:51.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 -- 172.21.15.148:0/3155990044 learned_addr learned my addr 172.21.15.148:0/3155990044 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:51.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665dd9b700 1 --2- 172.21.15.148:0/3155990044 >> v2:172.21.15.161:3300/0 conn(0x7f666015e5d0 0x7f666015b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 -- 172.21.15.148:0/3155990044 >> v2:172.21.15.161:3300/0 conn(0x7f666015e5d0 msgr2=0x7f666015b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 --2- 172.21.15.148:0/3155990044 >> v2:172.21.15.161:3300/0 conn(0x7f666015e5d0 0x7f666015b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 -- 172.21.15.148:0/3155990044 >> v1:172.21.15.161:6789/0 conn(0x7f66600cd900 legacy=0x7f66600c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 -- 172.21.15.148:0/3155990044 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600c7b10 con 0x7f666015dcc0 2023-10-10T20:19:51.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665e59c700 1 --2- 172.21.15.148:0/3155990044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015dcc0 0x7f666015e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6660071860 tx=0x7f6650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b5e470d87d9c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665d59a700 1 -- 172.21.15.148:0/3155990044 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650009e70 con 0x7f666015dcc0 2023-10-10T20:19:51.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665d59a700 1 -- 172.21.15.148:0/3155990044 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665000b860 con 0x7f666015dcc0 2023-10-10T20:19:51.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.268+0000 7f665d59a700 1 -- 172.21.15.148:0/3155990044 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000bc00 con 0x7f666015dcc0 2023-10-10T20:19:51.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 -- 172.21.15.148:0/3155990044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015dcc0 msgr2=0x7f666015e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 --2- 172.21.15.148:0/3155990044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015dcc0 0x7f666015e090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6660071860 tx=0x7f6650009250 comp rx=0 tx=0).stop 2023-10-10T20:19:51.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 -- 172.21.15.148:0/3155990044 shutdown_connections 2023-10-10T20:19:51.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 --2- 172.21.15.148:0/3155990044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015dcc0 0x7f666015e090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 --2- 172.21.15.148:0/3155990044 >> v2:172.21.15.161:3300/0 conn(0x7f666015e5d0 0x7f666015b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 -- 172.21.15.148:0/3155990044 >> 172.21.15.148:0/3155990044 conn(0x7f6660068170 msgr2=0x7f66600cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 -- 172.21.15.148:0/3155990044 shutdown_connections 2023-10-10T20:19:51.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.269+0000 7f6664daf700 1 -- 172.21.15.148:0/3155990044 wait complete. 2023-10-10T20:19:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.270+0000 7f6664daf700 1 Processor -- start 2023-10-10T20:19:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.270+0000 7f6664daf700 1 -- start start 2023-10-10T20:19:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66600cd900 0x7f6660098b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666015dcc0 0x7f666009a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660099190 con 0x7f666015e5d0 2023-10-10T20:19:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660099470 con 0x7f66600cd900 2023-10-10T20:19:51.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6660099750 con 0x7f666015dcc0 2023-10-10T20:19:51.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47104/0 (socket says 172.21.15.148:47104) 2023-10-10T20:19:51.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 -- 172.21.15.148:0/1137578892 learned_addr learned my addr 172.21.15.148:0/1137578892 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:51.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665e59c700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66600cd900 0x7f6660098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665dd9b700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666015dcc0 0x7f666009a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 -- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666015dcc0 msgr2=0x7f666009a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666015dcc0 0x7f666009a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 -- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66600cd900 msgr2=0x7f6660098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66600cd900 0x7f6660098b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 -- 172.21.15.148:0/1137578892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66500094f0 con 0x7f666015e5d0 2023-10-10T20:19:51.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f665ed9d700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f665800c8e0 tx=0x7f6658001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658012630 con 0x7f666015e5d0 2023-10-10T20:19:51.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66580127b0 con 0x7f666015e5d0 2023-10-10T20:19:51.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658012b70 con 0x7f666015e5d0 2023-10-10T20:19:51.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6660099a30 con 0x7f666015e5d0 2023-10-10T20:19:51.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.271+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66600b1450 con 0x7f666015e5d0 2023-10-10T20:19:51.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.272+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660099f50 con 0x7f666015e5d0 2023-10-10T20:19:51.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.273+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6658003030 con 0x7f666015e5d0 2023-10-10T20:19:51.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.273+0000 7f664e7fc700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 0x7f66400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.273+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6658084380 con 0x7f666015e5d0 2023-10-10T20:19:51.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.273+0000 7f665e59c700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 0x7f66400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.273+0000 7f665e59c700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 0x7f66400693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6660071860 tx=0x7f6650017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.277+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6660099f50 con 0x7f666015e5d0 2023-10-10T20:19:51.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.460+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6660099f50 con 0x7f666015e5d0 2023-10-10T20:19:51.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.460+0000 7f664e7fc700 1 -- 172.21.15.148:0/1137578892 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6660099f50 con 0x7f666015e5d0 2023-10-10T20:19:51.462 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:51.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 msgr2=0x7f66400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 0x7f66400693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6660071860 tx=0x7f6650017000 comp rx=0 tx=0).stop 2023-10-10T20:19:51.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 msgr2=0x7f66600b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f665800c8e0 tx=0x7f6658001710 comp rx=0 tx=0).stop 2023-10-10T20:19:51.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 shutdown_connections 2023-10-10T20:19:51.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6640066f30 0x7f66400693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666015e5d0 0x7f66600b0f10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66600cd900 0x7f6660098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.462+0000 7f6664daf700 1 --2- 172.21.15.148:0/1137578892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666015dcc0 0x7f666009a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.463+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 >> 172.21.15.148:0/1137578892 conn(0x7f6660068170 msgr2=0x7f666015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.463+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 shutdown_connections 2023-10-10T20:19:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.463+0000 7f6664daf700 1 -- 172.21.15.148:0/1137578892 wait complete. 2023-10-10T20:19:51.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:51.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.589+0000 7f859e232700 1 Processor -- start 2023-10-10T20:19:51.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.589+0000 7f859e232700 1 -- start start 2023-10-10T20:19:51.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f859e232700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f859815a620 0x7f859815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f859e232700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 0x7f8598158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f859e232700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859815cbb0 con 0x7f8598157870 2023-10-10T20:19:51.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f859e232700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859815bdb0 con 0x7f859815a620 2023-10-10T20:19:51.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f859e232700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859815bfa0 con 0x7f859815d580 2023-10-10T20:19:51.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f85977fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8598157870 0x7f859815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51300/0 (socket says 172.21.15.148:51300) 2023-10-10T20:19:51.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f85977fe700 1 -- 172.21.15.148:0/2317086887 learned_addr learned my addr 172.21.15.148:0/2317086887 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:51.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f8596ffd700 1 --2- 172.21.15.148:0/2317086887 >> v2:172.21.15.148:3300/0 conn(0x7f859815a620 0x7f859815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.590+0000 7f8597fff700 1 --2- 172.21.15.148:0/2317086887 >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 0x7f8598158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f85967fc700 1 -- 172.21.15.148:0/2317086887 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201569411 0 0) 0x7f859815cbb0 con 0x7f8598157870 2023-10-10T20:19:51.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f85967fc700 1 -- 172.21.15.148:0/2317086887 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8580003570 con 0x7f8598157870 2023-10-10T20:19:51.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8596ffd700 1 -- 172.21.15.148:0/2317086887 >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 msgr2=0x7f8598158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:51.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8596ffd700 1 --2- 172.21.15.148:0/2317086887 >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 0x7f8598158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8596ffd700 1 -- 172.21.15.148:0/2317086887 >> v1:172.21.15.148:6789/0 conn(0x7f8598157870 legacy=0x7f859815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8596ffd700 1 -- 172.21.15.148:0/2317086887 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859815c280 con 0x7f859815a620 2023-10-10T20:19:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8597fff700 1 --2- 172.21.15.148:0/2317086887 >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 0x7f8598158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f8596ffd700 1 --2- 172.21.15.148:0/2317086887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859815a620 0x7f859815d040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f859806b4e0 tx=0x7f858c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca95e2e7c2736db5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f85967fc700 1 -- 172.21.15.148:0/2317086887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c011630 con 0x7f859815a620 2023-10-10T20:19:51.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f85967fc700 1 -- 172.21.15.148:0/2317086887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c011790 con 0x7f859815a620 2023-10-10T20:19:51.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f85967fc700 1 -- 172.21.15.148:0/2317086887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c011b30 con 0x7f859815a620 2023-10-10T20:19:51.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 -- 172.21.15.148:0/2317086887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859815a620 msgr2=0x7f859815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 --2- 172.21.15.148:0/2317086887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859815a620 0x7f859815d040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f859806b4e0 tx=0x7f858c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 -- 172.21.15.148:0/2317086887 shutdown_connections 2023-10-10T20:19:51.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 --2- 172.21.15.148:0/2317086887 >> v2:172.21.15.161:3300/0 conn(0x7f859815d580 0x7f8598158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 --2- 172.21.15.148:0/2317086887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859815a620 0x7f859815d040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 -- 172.21.15.148:0/2317086887 >> 172.21.15.148:0/2317086887 conn(0x7f8598068ce0 msgr2=0x7f85980690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 -- 172.21.15.148:0/2317086887 shutdown_connections 2023-10-10T20:19:51.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.591+0000 7f859e232700 1 -- 172.21.15.148:0/2317086887 wait complete. 2023-10-10T20:19:51.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 Processor -- start 2023-10-10T20:19:51.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 -- start start 2023-10-10T20:19:51.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f859815a620 0x7f859816ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 0x7f859816b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859816c040 con 0x7f8598157870 2023-10-10T20:19:51.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859816bcf0 con 0x7f859815d580 2023-10-10T20:19:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.593+0000 7f859e232700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85981514c0 con 0x7f859815a620 2023-10-10T20:19:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47130/0 (socket says 172.21.15.148:47130) 2023-10-10T20:19:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 -- 172.21.15.148:0/1108712088 learned_addr learned my addr 172.21.15.148:0/1108712088 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:51.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f8597fff700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 0x7f859816b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f8596ffd700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f859815a620 0x7f859816ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 -- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f859815a620 msgr2=0x7f859816ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f859815a620 0x7f859816ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 -- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 msgr2=0x7f859816b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 0x7f859816b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 -- 172.21.15.148:0/1108712088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c009420 con 0x7f8598157870 2023-10-10T20:19:51.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f8597fff700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 0x7f859816b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f85977fe700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f858800cd30 tx=0x7f8588001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800ab70 con 0x7f8598157870 2023-10-10T20:19:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858800acd0 con 0x7f8598157870 2023-10-10T20:19:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85980b6430 con 0x7f8598157870 2023-10-10T20:19:51.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588007340 con 0x7f8598157870 2023-10-10T20:19:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.594+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85980b69f0 con 0x7f8598157870 2023-10-10T20:19:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.595+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859816c180 con 0x7f8598157870 2023-10-10T20:19:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.595+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8588007770 con 0x7f8598157870 2023-10-10T20:19:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.596+0000 7f857f7fe700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.596+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f85880849b0 con 0x7f8598157870 2023-10-10T20:19:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.596+0000 7f8596ffd700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.596+0000 7f8596ffd700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 0x7f85840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f859815aa20 tx=0x7f858c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.600+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f859816c180 con 0x7f8598157870 2023-10-10T20:19:51.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.785+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f859816c180 con 0x7f8598157870 2023-10-10T20:19:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.785+0000 7f857f7fe700 1 -- 172.21.15.148:0/1108712088 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f8588051ca0 con 0x7f8598157870 2023-10-10T20:19:51.788 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:51.788 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:51.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 msgr2=0x7f85840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 0x7f85840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f859815aa20 tx=0x7f858c017000 comp rx=0 tx=0).stop 2023-10-10T20:19:51.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 msgr2=0x7f85980b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f858800cd30 tx=0x7f8588001b10 comp rx=0 tx=0).stop 2023-10-10T20:19:51.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 shutdown_connections 2023-10-10T20:19:51.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f859815d580 0x7f859816b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598157870 0x7f85980b5b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 --2- 172.21.15.148:0/1108712088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f859815a620 0x7f859816ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 >> 172.21.15.148:0/1108712088 conn(0x7f8598068ce0 msgr2=0x7f8598155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.787+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 shutdown_connections 2023-10-10T20:19:51.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.788+0000 7f859e232700 1 -- 172.21.15.148:0/1108712088 wait complete. 2023-10-10T20:19:51.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:51.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:51.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.919+0000 7f43c02b0700 1 Processor -- start 2023-10-10T20:19:51.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.919+0000 7f43c02b0700 1 -- start start 2023-10-10T20:19:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43c02b0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43b815b9a0 0x7f43b80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43c02b0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b806d370 0x7f43b806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43c02b0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b815c8d0 con 0x7f43b8152300 2023-10-10T20:19:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43c02b0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b80b1400 con 0x7f43b806d370 2023-10-10T20:19:51.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43c02b0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b815be20 con 0x7f43b815b9a0 2023-10-10T20:19:51.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43be84d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b806d370 0x7f43b806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43be84d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b806d370 0x7f43b806d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47136/0 (socket says 172.21.15.148:47136) 2023-10-10T20:19:51.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43be04c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f43b8152300 0x7f43b8158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51302/0 (socket says 172.21.15.148:51302) 2023-10-10T20:19:51.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43be84d700 1 -- 172.21.15.148:0/1919820474 learned_addr learned my addr 172.21.15.148:0/1919820474 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:51.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.920+0000 7f43bd84b700 1 --2- 172.21.15.148:0/1919820474 >> v2:172.21.15.161:3300/0 conn(0x7f43b815b9a0 0x7f43b80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43be84d700 1 -- 172.21.15.148:0/1919820474 >> v2:172.21.15.161:3300/0 conn(0x7f43b815b9a0 msgr2=0x7f43b80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:51.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43be84d700 1 --2- 172.21.15.148:0/1919820474 >> v2:172.21.15.161:3300/0 conn(0x7f43b815b9a0 0x7f43b80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43be84d700 1 -- 172.21.15.148:0/1919820474 >> v1:172.21.15.148:6789/0 conn(0x7f43b8152300 legacy=0x7f43b8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43bd04a700 1 -- 172.21.15.148:0/1919820474 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495603871 0 0) 0x7f43b815c8d0 con 0x7f43b8152300 2023-10-10T20:19:51.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43be84d700 1 -- 172.21.15.148:0/1919820474 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80c9370 con 0x7f43b806d370 2023-10-10T20:19:51.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43be84d700 1 --2- 172.21.15.148:0/1919820474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b806d370 0x7f43b806d790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43b806e8d0 tx=0x7f43b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7250b4f65b5899 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43bd04a700 1 -- 172.21.15.148:0/1919820474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000e070 con 0x7f43b806d370 2023-10-10T20:19:51.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43bd04a700 1 -- 172.21.15.148:0/1919820474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000ba20 con 0x7f43b806d370 2023-10-10T20:19:51.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43bd04a700 1 -- 172.21.15.148:0/1919820474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00092e0 con 0x7f43b806d370 2023-10-10T20:19:51.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 -- 172.21.15.148:0/1919820474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b806d370 msgr2=0x7f43b806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 --2- 172.21.15.148:0/1919820474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b806d370 0x7f43b806d790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43b806e8d0 tx=0x7f43b0009840 comp rx=0 tx=0).stop 2023-10-10T20:19:51.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 -- 172.21.15.148:0/1919820474 shutdown_connections 2023-10-10T20:19:51.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 --2- 172.21.15.148:0/1919820474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b806d370 0x7f43b806d790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 --2- 172.21.15.148:0/1919820474 >> v2:172.21.15.161:3300/0 conn(0x7f43b815b9a0 0x7f43b80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.921+0000 7f43c02b0700 1 -- 172.21.15.148:0/1919820474 >> 172.21.15.148:0/1919820474 conn(0x7f43b8069870 msgr2=0x7f43b8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:51.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.922+0000 7f43c02b0700 1 -- 172.21.15.148:0/1919820474 shutdown_connections 2023-10-10T20:19:51.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.922+0000 7f43c02b0700 1 -- 172.21.15.148:0/1919820474 wait complete. 2023-10-10T20:19:51.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 Processor -- start 2023-10-10T20:19:51.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 -- start start 2023-10-10T20:19:51.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 0x7f43b8151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 0x7f43b8168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81535e0 con 0x7f43b8152300 2023-10-10T20:19:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8153020 con 0x7f43b806d370 2023-10-10T20:19:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43c02b0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8153300 con 0x7f43b815b9a0 2023-10-10T20:19:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43bd84b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43bd84b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47142/0 (socket says 172.21.15.148:47142) 2023-10-10T20:19:51.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.923+0000 7f43bd84b700 1 -- 172.21.15.148:0/3224946788 learned_addr learned my addr 172.21.15.148:0/3224946788 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:51.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43be84d700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 0x7f43b8168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43be04c700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 0x7f43b8151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 -- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 msgr2=0x7f43b8168c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 0x7f43b8168c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 -- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 msgr2=0x7f43b8151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:51.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 0x7f43b8151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:51.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 -- 172.21.15.148:0/3224946788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009ae0 con 0x7f43b8152300 2023-10-10T20:19:51.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43be84d700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 0x7f43b8168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43be04c700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 0x7f43b8151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43bd84b700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f43a800e930 tx=0x7f43a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800eed0 con 0x7f43b8152300 2023-10-10T20:19:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800c850 con 0x7f43b8152300 2023-10-10T20:19:51.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800cbf0 con 0x7f43b8152300 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b8155c70 con 0x7f43b8152300 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.924+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b8151b80 con 0x7f43b8152300 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.925+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b804c3c0 con 0x7f43b8152300 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.925+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43a8013470 con 0x7f43b8152300 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.925+0000 7f43ae7fc700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 0x7f439c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:51.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.926+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43a80847a0 con 0x7f43b8152300 2023-10-10T20:19:51.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.926+0000 7f43be04c700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 0x7f439c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:51.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.926+0000 7f43be04c700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 0x7f439c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43b40018d0 tx=0x7f43b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:51.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:51.930+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43a8051a10 con 0x7f43b8152300 2023-10-10T20:19:52.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.109+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43b80cdf10 con 0x7f43b8152300 2023-10-10T20:19:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.110+0000 7f43ae7fc700 1 -- 172.21.15.148:0/3224946788 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43a8055030 con 0x7f43b8152300 2023-10-10T20:19:52.112 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:52.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 msgr2=0x7f439c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 0x7f439c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43b40018d0 tx=0x7f43b4009040 comp rx=0 tx=0).stop 2023-10-10T20:19:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 msgr2=0x7f43b81526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f43a800e930 tx=0x7f43a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 shutdown_connections 2023-10-10T20:19:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f439c066f30 0x7f439c0693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b815b9a0 0x7f43b8168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b806d370 0x7f43b8151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 --2- 172.21.15.148:0/3224946788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b8152300 0x7f43b81526d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 >> 172.21.15.148:0/3224946788 conn(0x7f43b8069870 msgr2=0x7f43b80b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 shutdown_connections 2023-10-10T20:19:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.112+0000 7f43c02b0700 1 -- 172.21.15.148:0/3224946788 wait complete. 2023-10-10T20:19:52.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:52.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.242+0000 7f3317e19700 1 Processor -- start 2023-10-10T20:19:52.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.242+0000 7f3317e19700 1 -- start start 2023-10-10T20:19:52.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3317e19700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3310159ec0 0x7f33100af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3317e19700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100b0790 con 0x7f331015a290 2023-10-10T20:19:52.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3317e19700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100ce1f0 con 0x7f331006acb0 2023-10-10T20:19:52.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3317e19700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100b0b20 con 0x7f3310159ec0 2023-10-10T20:19:52.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3315bb5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3310159ec0 0x7f33100af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f33153b4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f331015a290 0x7f331006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51308/0 (socket says 172.21.15.148:51308) 2023-10-10T20:19:52.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3315bb5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3310159ec0 0x7f33100af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47158/0 (socket says 172.21.15.148:47158) 2023-10-10T20:19:52.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f33153b4700 1 -- 172.21.15.148:0/1747531756 learned_addr learned my addr 172.21.15.148:0/1747531756 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:52.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3315bb5700 1 -- 172.21.15.148:0/1747531756 >> v1:172.21.15.161:6789/0 conn(0x7f331006acb0 legacy=0x7f33100cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3315bb5700 1 -- 172.21.15.148:0/1747531756 >> v1:172.21.15.148:6789/0 conn(0x7f331015a290 legacy=0x7f331006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3315bb5700 1 -- 172.21.15.148:0/1747531756 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100b04b0 con 0x7f3310159ec0 2023-10-10T20:19:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.243+0000 7f3314bb3700 1 -- 172.21.15.148:0/1747531756 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323400967 0 0) 0x7f33100ce1f0 con 0x7f331006acb0 2023-10-10T20:19:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3315bb5700 1 --2- 172.21.15.148:0/1747531756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3310159ec0 0x7f33100af940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3300009e60 tx=0x7f3300009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b631f1fcb9473236 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3314bb3700 1 -- 172.21.15.148:0/1747531756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000e070 con 0x7f3310159ec0 2023-10-10T20:19:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3314bb3700 1 -- 172.21.15.148:0/1747531756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000bb30 con 0x7f3310159ec0 2023-10-10T20:19:52.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3314bb3700 1 -- 172.21.15.148:0/1747531756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000bed0 con 0x7f3310159ec0 2023-10-10T20:19:52.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 -- 172.21.15.148:0/1747531756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3310159ec0 msgr2=0x7f33100af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 --2- 172.21.15.148:0/1747531756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3310159ec0 0x7f33100af940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3300009e60 tx=0x7f3300009780 comp rx=0 tx=0).stop 2023-10-10T20:19:52.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 -- 172.21.15.148:0/1747531756 shutdown_connections 2023-10-10T20:19:52.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 --2- 172.21.15.148:0/1747531756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3310159ec0 0x7f33100af940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 -- 172.21.15.148:0/1747531756 >> 172.21.15.148:0/1747531756 conn(0x7f33100d1880 msgr2=0x7f33100cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 -- 172.21.15.148:0/1747531756 shutdown_connections 2023-10-10T20:19:52.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.244+0000 7f3317e19700 1 -- 172.21.15.148:0/1747531756 wait complete. 2023-10-10T20:19:52.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.245+0000 7f3317e19700 1 Processor -- start 2023-10-10T20:19:52.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.245+0000 7f3317e19700 1 -- start start 2023-10-10T20:19:52.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 0x7f33100c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 0x7f33100c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100c9880 con 0x7f331006acb0 2023-10-10T20:19:52.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331016d770 con 0x7f331015a290 2023-10-10T20:19:52.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3317e19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100c8c70 con 0x7f3310159ec0 2023-10-10T20:19:52.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3315bb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3315bb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47168/0 (socket says 172.21.15.148:47168) 2023-10-10T20:19:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f3315bb5700 1 -- 172.21.15.148:0/2109642780 learned_addr learned my addr 172.21.15.148:0/2109642780 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f33153b4700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 0x7f33100c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.246+0000 7f33163b6700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 0x7f33100c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 -- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 msgr2=0x7f33100c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 0x7f33100c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 -- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 msgr2=0x7f33100c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 0x7f33100c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 -- 172.21.15.148:0/2109642780 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300009a20 con 0x7f331006acb0 2023-10-10T20:19:52.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f33153b4700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 0x7f33100c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:52.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f33163b6700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 0x7f33100c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:52.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3315bb5700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f330000b6c0 tx=0x7f3300018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300026460 con 0x7f331006acb0 2023-10-10T20:19:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000265e0 con 0x7f331006acb0 2023-10-10T20:19:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000269a0 con 0x7f331006acb0 2023-10-10T20:19:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33100c8f50 con 0x7f331006acb0 2023-10-10T20:19:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.247+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33100c9510 con 0x7f331006acb0 2023-10-10T20:19:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.248+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331004c3c0 con 0x7f331006acb0 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.248+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3300026d50 con 0x7f331006acb0 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.249+0000 7f330dffb700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 0x7f32fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.249+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3300086af0 con 0x7f331006acb0 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.249+0000 7f33153b4700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 0x7f32fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.249+0000 7f33153b4700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 0x7f32fc0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33040058d0 tx=0x7f3304009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.253+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3300053d60 con 0x7f331006acb0 2023-10-10T20:19:52.307 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:19:52.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:19:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.442+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33100ceab0 con 0x7f331006acb0 2023-10-10T20:19:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.443+0000 7f330dffb700 1 -- 172.21.15.148:0/2109642780 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3300057380 con 0x7f331006acb0 2023-10-10T20:19:52.446 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 msgr2=0x7f32fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 0x7f32fc0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f33040058d0 tx=0x7f3304009040 comp rx=0 tx=0).stop 2023-10-10T20:19:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 msgr2=0x7f331016d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f330000b6c0 tx=0x7f3300018000 comp rx=0 tx=0).stop 2023-10-10T20:19:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 shutdown_connections 2023-10-10T20:19:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32fc066f30 0x7f32fc0693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331015a290 0x7f33100c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f331006acb0 0x7f331016d230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 --2- 172.21.15.148:0/2109642780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3310159ec0 0x7f33100c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 >> 172.21.15.148:0/2109642780 conn(0x7f33100d1880 msgr2=0x7f331006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 shutdown_connections 2023-10-10T20:19:52.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.445+0000 7f3317e19700 1 -- 172.21.15.148:0/2109642780 wait complete. 2023-10-10T20:19:52.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:52.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.573+0000 7fb6fae90700 1 Processor -- start 2023-10-10T20:19:52.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 -- start start 2023-10-10T20:19:52.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6f40cb5d0 0x7fb6f40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 0x7fb6f40af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f40af8e0 con 0x7fb6f40cb9a0 2023-10-10T20:19:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f40c9960 con 0x7fb6f40cb5d0 2023-10-10T20:19:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.574+0000 7fb6fae90700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40c3610 con 0x7fb6f40c5180 2023-10-10T20:19:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6f40cb5d0 0x7fb6f40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6f40cb9a0 0x7fb6f40c4a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56954/0 (socket says 172.21.15.148:56954) 2023-10-10T20:19:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f3fff700 1 -- 172.21.15.148:0/1316368341 learned_addr learned my addr 172.21.15.148:0/1316368341 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f942d700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 0x7fb6f40af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 -- 172.21.15.148:0/1316368341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 msgr2=0x7fb6f40af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 0x7fb6f40af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 -- 172.21.15.148:0/1316368341 >> v1:172.21.15.148:6789/0 conn(0x7fb6f40cb9a0 legacy=0x7fb6f40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 -- 172.21.15.148:0/1316368341 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40af6b0 con 0x7fb6f40cb5d0 2023-10-10T20:19:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f37fe700 1 -- 172.21.15.148:0/1316368341 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001028414 0 0) 0x7fb6f40af8e0 con 0x7fb6f40cb9a0 2023-10-10T20:19:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f942d700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 0x7fb6f40af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f8c2c700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40cb5d0 0x7fb6f40b0cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009e60 tx=0x7fb6e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0da76e80a026d5f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f37fe700 1 -- 172.21.15.148:0/1316368341 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400e070 con 0x7fb6f40cb5d0 2023-10-10T20:19:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f37fe700 1 -- 172.21.15.148:0/1316368341 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400bcc0 con 0x7fb6f40cb5d0 2023-10-10T20:19:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.575+0000 7fb6f37fe700 1 -- 172.21.15.148:0/1316368341 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4012680 con 0x7fb6f40cb5d0 2023-10-10T20:19:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 -- 172.21.15.148:0/1316368341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40cb5d0 msgr2=0x7fb6f40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40cb5d0 0x7fb6f40b0cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009e60 tx=0x7fb6e4009780 comp rx=0 tx=0).stop 2023-10-10T20:19:52.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 -- 172.21.15.148:0/1316368341 shutdown_connections 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40c5180 0x7fb6f40af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 --2- 172.21.15.148:0/1316368341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40cb5d0 0x7fb6f40b0cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 -- 172.21.15.148:0/1316368341 >> 172.21.15.148:0/1316368341 conn(0x7fb6f406f3e0 msgr2=0x7fb6f406f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 -- 172.21.15.148:0/1316368341 shutdown_connections 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.576+0000 7fb6fae90700 1 -- 172.21.15.148:0/1316368341 wait complete. 2023-10-10T20:19:52.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.577+0000 7fb6fae90700 1 Processor -- start 2023-10-10T20:19:52.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.577+0000 7fb6fae90700 1 -- start start 2023-10-10T20:19:52.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f40c5180 0x7fb6f415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40cb9a0 0x7fb6f40c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40c8f00 con 0x7fb6f40c8a80 2023-10-10T20:19:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4068840 con 0x7fb6f40c5180 2023-10-10T20:19:52.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40ac220 con 0x7fb6f40cb9a0 2023-10-10T20:19:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59434/0 (socket says 172.21.15.148:59434) 2023-10-10T20:19:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 -- 172.21.15.148:0/3448380241 learned_addr learned my addr 172.21.15.148:0/3448380241 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f3fff700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40cb9a0 0x7fb6f40c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f8c2c700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f40c5180 0x7fb6f415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 -- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40cb9a0 msgr2=0x7fb6f40c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40cb9a0 0x7fb6f40c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 -- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f40c5180 msgr2=0x7fb6f415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f40c5180 0x7fb6f415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 -- 172.21.15.148:0/3448380241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4009a20 con 0x7fb6f40c8a80 2023-10-10T20:19:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f942d700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c880 tx=0x7fb6ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec011600 con 0x7fb6f40c8a80 2023-10-10T20:19:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec011780 con 0x7fb6f40c8a80 2023-10-10T20:19:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f40ac500 con 0x7fb6f40c8a80 2023-10-10T20:19:52.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec011b40 con 0x7fb6f40c8a80 2023-10-10T20:19:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.578+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f415d1e0 con 0x7fb6f40c8a80 2023-10-10T20:19:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.579+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f40689b0 con 0x7fb6f40c8a80 2023-10-10T20:19:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.580+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6ec0193b0 con 0x7fb6f40c8a80 2023-10-10T20:19:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.580+0000 7fb6f0ff9700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 0x7fb6e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.580+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6ec0844f0 con 0x7fb6f40c8a80 2023-10-10T20:19:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.580+0000 7fb6f8c2c700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 0x7fb6e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.581+0000 7fb6f8c2c700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 0x7fb6e00693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6f406b510 tx=0x7fb6e4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.584+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6ec051760 con 0x7fb6f40c8a80 2023-10-10T20:19:52.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.768+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6f404c3c0 con 0x7fb6f40c8a80 2023-10-10T20:19:52.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.768+0000 7fb6f0ff9700 1 -- 172.21.15.148:0/3448380241 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6ec054d80 con 0x7fb6f40c8a80 2023-10-10T20:19:52.770 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 msgr2=0x7fb6e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 0x7fb6e00693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6f406b510 tx=0x7fb6e4019000 comp rx=0 tx=0).stop 2023-10-10T20:19:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 msgr2=0x7fb6f40ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c880 tx=0x7fb6ec008040 comp rx=0 tx=0).stop 2023-10-10T20:19:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 shutdown_connections 2023-10-10T20:19:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6e0066f30 0x7fb6e00693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f40c8a80 0x7fb6f40ab4f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f40c5180 0x7fb6f415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 --2- 172.21.15.148:0/3448380241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f40cb9a0 0x7fb6f40c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 >> 172.21.15.148:0/3448380241 conn(0x7fb6f406f3e0 msgr2=0x7fb6f40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 shutdown_connections 2023-10-10T20:19:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.770+0000 7fb6fae90700 1 -- 172.21.15.148:0/3448380241 wait complete. 2023-10-10T20:19:52.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:52.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.899+0000 7fa5bf59e700 1 Processor -- start 2023-10-10T20:19:52.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.899+0000 7fa5bf59e700 1 -- start start 2023-10-10T20:19:52.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bf59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5c0157a90 0x7fa5c00b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bf59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c00cb0b0 con 0x7fa5c0157e60 2023-10-10T20:19:52.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bf59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c00b0830 con 0x7fa5c006fef0 2023-10-10T20:19:52.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bf59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00b0ad0 con 0x7fa5c0157a90 2023-10-10T20:19:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5b5d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa5c0157e60 0x7fa5c00b0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56970/0 (socket says 172.21.15.148:56970) 2023-10-10T20:19:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5b5d9b700 1 -- 172.21.15.148:0/2620956052 learned_addr learned my addr 172.21.15.148:0/2620956052 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5be59c700 1 --2- 172.21.15.148:0/2620956052 >> v2:172.21.15.161:3300/0 conn(0x7fa5c0157a90 0x7fa5c00b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bdd9b700 1 -- 172.21.15.148:0/2620956052 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760899729 0 0) 0x7fa5c00cb0b0 con 0x7fa5c0157e60 2023-10-10T20:19:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5bdd9b700 1 -- 172.21.15.148:0/2620956052 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac003570 con 0x7fa5c0157e60 2023-10-10T20:19:52.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5be59c700 1 -- 172.21.15.148:0/2620956052 >> v1:172.21.15.161:6789/0 conn(0x7fa5c006fef0 legacy=0x7fa5c00ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5be59c700 1 -- 172.21.15.148:0/2620956052 >> v1:172.21.15.148:6789/0 conn(0x7fa5c0157e60 legacy=0x7fa5c00b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.900+0000 7fa5be59c700 1 -- 172.21.15.148:0/2620956052 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00ca780 con 0x7fa5c0157a90 2023-10-10T20:19:52.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5be59c700 1 --2- 172.21.15.148:0/2620956052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c0157a90 0x7fa5c00b1790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00098e0 tx=0x7fa5b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aeb8cb9711a4972 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bdd9b700 1 -- 172.21.15.148:0/2620956052 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000b640 con 0x7fa5c0157a90 2023-10-10T20:19:52.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bdd9b700 1 -- 172.21.15.148:0/2620956052 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000b7a0 con 0x7fa5c0157a90 2023-10-10T20:19:52.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bdd9b700 1 -- 172.21.15.148:0/2620956052 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000bb40 con 0x7fa5c0157a90 2023-10-10T20:19:52.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 -- 172.21.15.148:0/2620956052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c0157a90 msgr2=0x7fa5c00b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/2620956052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c0157a90 0x7fa5c00b1790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00098e0 tx=0x7fa5b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 -- 172.21.15.148:0/2620956052 shutdown_connections 2023-10-10T20:19:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/2620956052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c0157a90 0x7fa5c00b1790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 -- 172.21.15.148:0/2620956052 >> 172.21.15.148:0/2620956052 conn(0x7fa5c006bc70 msgr2=0x7fa5c006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 -- 172.21.15.148:0/2620956052 shutdown_connections 2023-10-10T20:19:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.901+0000 7fa5bf59e700 1 -- 172.21.15.148:0/2620956052 wait complete. 2023-10-10T20:19:52.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 Processor -- start 2023-10-10T20:19:52.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 -- start start 2023-10-10T20:19:52.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c006fef0 0x7fa5c00c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5c0157a90 0x7fa5c00c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0159b30 con 0x7fa5c0157e60 2023-10-10T20:19:52.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0159500 con 0x7fa5c006fef0 2023-10-10T20:19:52.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.903+0000 7fa5bf59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00c9990 con 0x7fa5c0157a90 2023-10-10T20:19:52.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59448/0 (socket says 172.21.15.148:59448) 2023-10-10T20:19:52.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 -- 172.21.15.148:0/3429209969 learned_addr learned my addr 172.21.15.148:0/3429209969 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:52.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5b5d9b700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5c0157a90 0x7fa5c00c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5be59c700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c006fef0 0x7fa5c00c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 -- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5c0157a90 msgr2=0x7fa5c00c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5c0157a90 0x7fa5c00c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 -- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c006fef0 msgr2=0x7fa5c00c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:52.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c006fef0 0x7fa5c00c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:52.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 -- 172.21.15.148:0/3429209969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009420 con 0x7fa5c0157e60 2023-10-10T20:19:52.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bed9d700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c840 tx=0x7fa5b8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8012630 con 0x7fa5c0157e60 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b80127b0 con 0x7fa5c0157e60 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c00c9c70 con 0x7fa5c0157e60 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8012b70 con 0x7fa5c0157e60 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.904+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c00c9670 con 0x7fa5c0157e60 2023-10-10T20:19:52.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.905+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c0159ca0 con 0x7fa5c0157e60 2023-10-10T20:19:52.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.906+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5b8003030 con 0x7fa5c0157e60 2023-10-10T20:19:52.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.906+0000 7fa5b6ffd700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 0x7fa5a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:52.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.906+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5b80844a0 con 0x7fa5c0157e60 2023-10-10T20:19:52.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.906+0000 7fa5be59c700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 0x7fa5a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:52.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.906+0000 7fa5be59c700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 0x7fa5a00693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0003080 tx=0x7fa5b0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:52.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:52.910+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b8051710 con 0x7fa5c0157e60 2023-10-10T20:19:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.093+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c00b3fd0 con 0x7fa5c0157e60 2023-10-10T20:19:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.094+0000 7fa5b6ffd700 1 -- 172.21.15.148:0/3429209969 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa5b8054d30 con 0x7fa5c0157e60 2023-10-10T20:19:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:53.096 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 msgr2=0x7fa5a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 0x7fa5a00693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0003080 tx=0x7fa5b0017000 comp rx=0 tx=0).stop 2023-10-10T20:19:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 msgr2=0x7fa5c0147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c840 tx=0x7fa5b8001670 comp rx=0 tx=0).stop 2023-10-10T20:19:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 shutdown_connections 2023-10-10T20:19:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5a0066f30 0x7fa5a00693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c0157e60 0x7fa5c0147cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c006fef0 0x7fa5c00c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 --2- 172.21.15.148:0/3429209969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5c0157a90 0x7fa5c00c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 >> 172.21.15.148:0/3429209969 conn(0x7fa5c006bc70 msgr2=0x7fa5c00ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 shutdown_connections 2023-10-10T20:19:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.096+0000 7fa5bf59e700 1 -- 172.21.15.148:0/3429209969 wait complete. 2023-10-10T20:19:53.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:53.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:53.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.226+0000 7f11cd506700 1 Processor -- start 2023-10-10T20:19:53.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.226+0000 7f11cd506700 1 -- start start 2023-10-10T20:19:53.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11cd506700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11c8070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11cd506700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11c806e2e0 0x7f11c8070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11cd506700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c806e8a0 con 0x7f11c81413b0 2023-10-10T20:19:53.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11cd506700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c806eb80 con 0x7f11c806e2e0 2023-10-10T20:19:53.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11cd506700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806ee60 con 0x7f11c8141780 2023-10-10T20:19:53.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11c6ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f11c81413b0 0x7f11c8068a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56980/0 (socket says 172.21.15.148:56980) 2023-10-10T20:19:53.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.227+0000 7f11c6ffd700 1 -- 172.21.15.148:0/1594002873 learned_addr learned my addr 172.21.15.148:0/1594002873 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:53.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079218501 0 0) 0x7f11c806e8a0 con 0x7f11c81413b0 2023-10-10T20:19:53.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c77fe700 1 --2- 172.21.15.148:0/1594002873 >> v2:172.21.15.161:3300/0 conn(0x7f11c806e2e0 0x7f11c8070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b4003570 con 0x7f11c81413b0 2023-10-10T20:19:53.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11be7fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11c8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779951006 0 0) 0x7f11b4003570 con 0x7f11c81413b0 2023-10-10T20:19:53.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 msgr2=0x7f11c8070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11c8070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 >> v2:172.21.15.161:3300/0 conn(0x7f11c806e2e0 msgr2=0x7f11c8070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 --2- 172.21.15.148:0/1594002873 >> v2:172.21.15.161:3300/0 conn(0x7f11c806e2e0 0x7f11c8070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c806f140 con 0x7f11c81413b0 2023-10-10T20:19:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f11b8001470 con 0x7f11c81413b0 2023-10-10T20:19:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 >> v1:172.21.15.148:6789/0 conn(0x7f11c81413b0 legacy=0x7f11c8068a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11b4004810 0x7f11b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806eb80 con 0x7f11b4003a60 2023-10-10T20:19:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c806ee60 con 0x7f11b4004810 2023-10-10T20:19:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b400ba60 con 0x7f11c8141780 2023-10-10T20:19:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11c77fe700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.228+0000 7f11be7fc700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11b4004810 0x7f11b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 -- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 msgr2=0x7f11b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c8141780 0x7f11b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 -- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11b4004810 msgr2=0x7f11b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11b4004810 0x7f11b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 -- 172.21.15.148:0/1594002873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4003570 con 0x7f11b4003a60 2023-10-10T20:19:53.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11b4003e70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c3f0 tx=0x7f11b8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8006f24490245ec server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800cc80 con 0x7f11b4003a60 2023-10-10T20:19:53.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b800ce00 con 0x7f11b4003a60 2023-10-10T20:19:53.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c67fc700 1 -- 172.21.15.148:0/1594002873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005b70 con 0x7f11b4003a60 2023-10-10T20:19:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 -- 172.21.15.148:0/1594002873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 msgr2=0x7f11b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 --2- 172.21.15.148:0/1594002873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11b4003e70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11b800c3f0 tx=0x7f11b8001e90 comp rx=0 tx=0).stop 2023-10-10T20:19:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11c77fe700 1 -- 172.21.15.148:0/1594002873 reap_dead start 2023-10-10T20:19:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 -- 172.21.15.148:0/1594002873 shutdown_connections 2023-10-10T20:19:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 -- 172.21.15.148:0/1594002873 >> 172.21.15.148:0/1594002873 conn(0x7f11c80ce570 msgr2=0x7f11c80ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 -- 172.21.15.148:0/1594002873 shutdown_connections 2023-10-10T20:19:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.229+0000 7f11cd506700 1 -- 172.21.15.148:0/1594002873 wait complete. 2023-10-10T20:19:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 Processor -- start 2023-10-10T20:19:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 -- start start 2023-10-10T20:19:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11c80a6460 0x7f11c80a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c80c7020 0x7f11c80c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80c79e0 con 0x7f11b4003a60 2023-10-10T20:19:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80a7be0 con 0x7f11c80a6460 2023-10-10T20:19:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.231+0000 7f11cd506700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c80b00d0 con 0x7f11c80c7020 2023-10-10T20:19:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59466/0 (socket says 172.21.15.148:59466) 2023-10-10T20:19:53.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 -- 172.21.15.148:0/308671516 learned_addr learned my addr 172.21.15.148:0/308671516 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:53.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c77fe700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c80c7020 0x7f11c80c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11be7fc700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11c80a6460 0x7f11c80a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 -- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c80c7020 msgr2=0x7f11c80c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c80c7020 0x7f11c80c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 -- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11c80a6460 msgr2=0x7f11c80a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11c80a6460 0x7f11c80a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 -- 172.21.15.148:0/308671516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80a6fd0 con 0x7f11b4003a60 2023-10-10T20:19:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11c6ffd700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11b4004700 tx=0x7f11b8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800cc80 con 0x7f11b4003a60 2023-10-10T20:19:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8001430 con 0x7f11b4003a60 2023-10-10T20:19:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c80a72b0 con 0x7f11b4003a60 2023-10-10T20:19:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8007510 con 0x7f11b4003a60 2023-10-10T20:19:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.232+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c80a7870 con 0x7f11b4003a60 2023-10-10T20:19:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.233+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c804c3c0 con 0x7f11b4003a60 2023-10-10T20:19:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.233+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11b8019030 con 0x7f11b4003a60 2023-10-10T20:19:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.234+0000 7f11bf7fe700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.234+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11b8085b10 con 0x7f11b4003a60 2023-10-10T20:19:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.234+0000 7f11be7fc700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.234+0000 7f11be7fc700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 0x7f11a80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11c8141c00 tx=0x7f11b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.238+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11b8052d80 con 0x7f11b4003a60 2023-10-10T20:19:53.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.423+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11c806f420 con 0x7f11b4003a60 2023-10-10T20:19:53.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.423+0000 7f11bf7fe700 1 -- 172.21.15.148:0/308671516 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11b80563a0 con 0x7f11b4003a60 2023-10-10T20:19:53.426 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:53.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 msgr2=0x7f11a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 0x7f11a80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11c8141c00 tx=0x7f11b0008040 comp rx=0 tx=0).stop 2023-10-10T20:19:53.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 msgr2=0x7f11c813f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11b4004700 tx=0x7f11b8015000 comp rx=0 tx=0).stop 2023-10-10T20:19:53.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 shutdown_connections 2023-10-10T20:19:53.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f11a8066f30 0x7f11a80693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11c80c7020 0x7f11c80c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11b4003a60 0x7f11c813f600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 --2- 172.21.15.148:0/308671516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11c80a6460 0x7f11c80a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 >> 172.21.15.148:0/308671516 conn(0x7f11c80ce570 msgr2=0x7f11c8140550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 shutdown_connections 2023-10-10T20:19:53.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.426+0000 7f11cd506700 1 -- 172.21.15.148:0/308671516 wait complete. 2023-10-10T20:19:53.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:53.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.555+0000 7f622925e700 1 Processor -- start 2023-10-10T20:19:53.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.555+0000 7f622925e700 1 -- start start 2023-10-10T20:19:53.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62240cce30 0x7f622415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240b6580 0x7f622406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622406a830 0x7f622415d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62240b4d70 con 0x7f622406a830 2023-10-10T20:19:53.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622406ac70 con 0x7f62240cce30 2023-10-10T20:19:53.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f622925e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62240cd8d0 con 0x7f62240b6580 2023-10-10T20:19:53.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622406a830 0x7f622415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622406a830 0x7f622415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59474/0 (socket says 172.21.15.148:59474) 2023-10-10T20:19:53.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 -- 172.21.15.148:0/3812464658 learned_addr learned my addr 172.21.15.148:0/3812464658 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:53.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f62237fe700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240b6580 0x7f622406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6223fff700 1 --2- 172.21.15.148:0/3812464658 >> v2:172.21.15.161:3300/0 conn(0x7f62240cce30 0x7f622415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 -- 172.21.15.148:0/3812464658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240b6580 msgr2=0x7f622406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240b6580 0x7f622406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 -- 172.21.15.148:0/3812464658 >> v2:172.21.15.161:3300/0 conn(0x7f62240cce30 msgr2=0x7f622415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 --2- 172.21.15.148:0/3812464658 >> v2:172.21.15.161:3300/0 conn(0x7f62240cce30 0x7f622415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 -- 172.21.15.148:0/3812464658 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62240b4440 con 0x7f622406a830 2023-10-10T20:19:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6228a5d700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406a830 0x7f622415d880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62240b3770 tx=0x7f621c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7945732d9805584a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.556+0000 7f6222ffd700 1 -- 172.21.15.148:0/3812464658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00e070 con 0x7f622406a830 2023-10-10T20:19:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f6222ffd700 1 -- 172.21.15.148:0/3812464658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c00b970 con 0x7f622406a830 2023-10-10T20:19:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f6222ffd700 1 -- 172.21.15.148:0/3812464658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0092e0 con 0x7f622406a830 2023-10-10T20:19:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 -- 172.21.15.148:0/3812464658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406a830 msgr2=0x7f622415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406a830 0x7f622415d880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62240b3770 tx=0x7f621c009840 comp rx=0 tx=0).stop 2023-10-10T20:19:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 -- 172.21.15.148:0/3812464658 shutdown_connections 2023-10-10T20:19:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622406a830 0x7f622415d880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 --2- 172.21.15.148:0/3812464658 >> v2:172.21.15.161:3300/0 conn(0x7f62240cce30 0x7f622415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 --2- 172.21.15.148:0/3812464658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240b6580 0x7f622406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 -- 172.21.15.148:0/3812464658 >> 172.21.15.148:0/3812464658 conn(0x7f6224068cb0 msgr2=0x7f62240690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 -- 172.21.15.148:0/3812464658 shutdown_connections 2023-10-10T20:19:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.557+0000 7f622925e700 1 -- 172.21.15.148:0/3812464658 wait complete. 2023-10-10T20:19:53.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.558+0000 7f622925e700 1 Processor -- start 2023-10-10T20:19:53.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.558+0000 7f622925e700 1 -- start start 2023-10-10T20:19:53.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 0x7f6224154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 0x7f62240b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62241535c0 con 0x7f62240b6580 2023-10-10T20:19:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6224152940 con 0x7f622406a830 2023-10-10T20:19:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f622925e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6224152c20 con 0x7f62240cce30 2023-10-10T20:19:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59482/0 (socket says 172.21.15.148:59482) 2023-10-10T20:19:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 -- 172.21.15.148:0/3622359641 learned_addr learned my addr 172.21.15.148:0/3622359641 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f6228a5d700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 0x7f62240b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f6223fff700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 0x7f6224154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 -- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 msgr2=0x7f62240b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 0x7f62240b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 -- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 msgr2=0x7f6224154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 0x7f6224154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f62237fe700 1 -- 172.21.15.148:0/3622359641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009ae0 con 0x7f62240b6580 2023-10-10T20:19:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f6223fff700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 0x7f6224154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.559+0000 7f6228a5d700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 0x7f62240b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f62237fe700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f621800ec50 tx=0x7f62180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800c9d0 con 0x7f62240b6580 2023-10-10T20:19:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800cb30 con 0x7f62240b6580 2023-10-10T20:19:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6224152f00 con 0x7f62240b6580 2023-10-10T20:19:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218013470 con 0x7f62240b6580 2023-10-10T20:19:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.560+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62240b27b0 con 0x7f62240b6580 2023-10-10T20:19:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.561+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6224153700 con 0x7f62240b6580 2023-10-10T20:19:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.561+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6218013820 con 0x7f62240b6580 2023-10-10T20:19:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.561+0000 7f620bfff700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 0x7f62100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.561+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6218084800 con 0x7f62240b6580 2023-10-10T20:19:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.562+0000 7f6223fff700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 0x7f62100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.562+0000 7f6223fff700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 0x7f62100693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6214009ce0 tx=0x7f6214008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.566+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6224153700 con 0x7f62240b6580 2023-10-10T20:19:53.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.749+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6224153700 con 0x7f62240b6580 2023-10-10T20:19:53.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.749+0000 7f620bfff700 1 -- 172.21.15.148:0/3622359641 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6224153700 con 0x7f62240b6580 2023-10-10T20:19:53.752 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:53.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 msgr2=0x7f62100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 0x7f62100693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6214009ce0 tx=0x7f6214008040 comp rx=0 tx=0).stop 2023-10-10T20:19:53.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 msgr2=0x7f622414f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f621800ec50 tx=0x7f62180091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 shutdown_connections 2023-10-10T20:19:53.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6210066f30 0x7f62100693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62240cce30 0x7f62240b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622406a830 0x7f6224154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 --2- 172.21.15.148:0/3622359641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62240b6580 0x7f622414f880 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.751+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 >> 172.21.15.148:0/3622359641 conn(0x7f6224068cb0 msgr2=0x7f62240cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.752+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 shutdown_connections 2023-10-10T20:19:53.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.752+0000 7f622925e700 1 -- 172.21.15.148:0/3622359641 wait complete. 2023-10-10T20:19:53.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:53.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.881+0000 7f9dd93e0700 1 Processor -- start 2023-10-10T20:19:53.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.881+0000 7f9dd93e0700 1 -- start start 2023-10-10T20:19:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd93e0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dd4159e60 0x7f9dd415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd93e0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd415ea90 con 0x7f9dd40697c0 2023-10-10T20:19:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd93e0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd40b3100 con 0x7f9dd415a230 2023-10-10T20:19:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd93e0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd40d1cf0 con 0x7f9dd4159e60 2023-10-10T20:19:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dd4159e60 0x7f9dd415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9dd40697c0 0x7f9dd40af730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56984/0 (socket says 172.21.15.148:56984) 2023-10-10T20:19:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.882+0000 7f9dd37fe700 1 -- 172.21.15.148:0/1509531010 learned_addr learned my addr 172.21.15.148:0/1509531010 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/1509531010 >> v1:172.21.15.161:6789/0 conn(0x7f9dd415a230 legacy=0x7f9dd40690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/1509531010 >> v1:172.21.15.148:6789/0 conn(0x7f9dd40697c0 legacy=0x7f9dd40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/1509531010 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd40d10e0 con 0x7f9dd4159e60 2023-10-10T20:19:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd27fc700 1 -- 172.21.15.148:0/1509531010 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720560135 0 0) 0x7f9dd415ea90 con 0x7f9dd40697c0 2023-10-10T20:19:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd2ffd700 1 --2- 172.21.15.148:0/1509531010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd4159e60 0x7f9dd415e550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40098e0 tx=0x7f9dc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe1b6a477cd559 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd27fc700 1 -- 172.21.15.148:0/1509531010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4009e80 con 0x7f9dd4159e60 2023-10-10T20:19:53.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd27fc700 1 -- 172.21.15.148:0/1509531010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400b820 con 0x7f9dd4159e60 2023-10-10T20:19:53.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd27fc700 1 -- 172.21.15.148:0/1509531010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400bba0 con 0x7f9dd4159e60 2023-10-10T20:19:53.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd93e0700 1 -- 172.21.15.148:0/1509531010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd4159e60 msgr2=0x7f9dd415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.883+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/1509531010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd4159e60 0x7f9dd415e550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40098e0 tx=0x7f9dc40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.884+0000 7f9dd93e0700 1 -- 172.21.15.148:0/1509531010 shutdown_connections 2023-10-10T20:19:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.884+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/1509531010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd4159e60 0x7f9dd415e550 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.884+0000 7f9dd93e0700 1 -- 172.21.15.148:0/1509531010 >> 172.21.15.148:0/1509531010 conn(0x7f9dd4070700 msgr2=0x7f9dd4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.884+0000 7f9dd93e0700 1 -- 172.21.15.148:0/1509531010 shutdown_connections 2023-10-10T20:19:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.884+0000 7f9dd93e0700 1 -- 172.21.15.148:0/1509531010 wait complete. 2023-10-10T20:19:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 Processor -- start 2023-10-10T20:19:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 -- start start 2023-10-10T20:19:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9dd4159e60 0x7f9dd415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 0x7f9dd415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.885+0000 7f9dd93e0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415dcd0 con 0x7f9dd40697c0 2023-10-10T20:19:53.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd93e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415dfb0 con 0x7f9dd415a230 2023-10-10T20:19:53.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd93e0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415e290 con 0x7f9dd4159e60 2023-10-10T20:19:53.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59496/0 (socket says 172.21.15.148:59496) 2023-10-10T20:19:53.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/4048450508 learned_addr learned my addr 172.21.15.148:0/4048450508 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:53.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9dd4159e60 msgr2=0x7f9dd415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd37fe700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 0x7f9dd415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dca7fc700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9dd4159e60 0x7f9dd415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9dd4159e60 0x7f9dd415aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 msgr2=0x7f9dd415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:53.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 0x7f9dd415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:53.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 -- 172.21.15.148:0/4048450508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009420 con 0x7f9dd40697c0 2023-10-10T20:19:53.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd37fe700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 0x7f9dd415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:53.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd2ffd700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40030c0 tx=0x7f9dc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400ba90 con 0x7f9dd40697c0 2023-10-10T20:19:53.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4020070 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd40b46f0 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc401b6a0 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.886+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd415d1e0 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.887+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd404c3c0 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.887+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9dc4022030 con 0x7f9dd40697c0 2023-10-10T20:19:53.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.888+0000 7f9dcb7fe700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 0x7f9db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:53.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.888+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9dc4086610 con 0x7f9dd40697c0 2023-10-10T20:19:53.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.888+0000 7f9dca7fc700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 0x7f9db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:53.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.888+0000 7f9dca7fc700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 0x7f9db40693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40c5980 tx=0x7f9dbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:53.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:53.892+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dc4053880 con 0x7f9dd40697c0 2023-10-10T20:19:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.074+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9dd404c3c0 con 0x7f9dd40697c0 2023-10-10T20:19:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.074+0000 7f9dcb7fe700 1 -- 172.21.15.148:0/4048450508 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9dc4056ea0 con 0x7f9dd40697c0 2023-10-10T20:19:54.077 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:54.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 msgr2=0x7f9db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 0x7f9db40693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40c5980 tx=0x7f9dbc009040 comp rx=0 tx=0).stop 2023-10-10T20:19:54.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 msgr2=0x7f9dd414d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40030c0 tx=0x7f9dc4016000 comp rx=0 tx=0).stop 2023-10-10T20:19:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 shutdown_connections 2023-10-10T20:19:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9db4066f30 0x7f9db40693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dd415a230 0x7f9dd415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dd40697c0 0x7f9dd414d320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 --2- 172.21.15.148:0/4048450508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9dd4159e60 0x7f9dd415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 >> 172.21.15.148:0/4048450508 conn(0x7f9dd4070700 msgr2=0x7f9dd4069030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 shutdown_connections 2023-10-10T20:19:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.076+0000 7f9dd93e0700 1 -- 172.21.15.148:0/4048450508 wait complete. 2023-10-10T20:19:54.102 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-10T20:19:54.103 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-10-10T20:19:54.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-10T20:19:54.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.216+0000 7fde2bd7e700 1 Processor -- start 2023-10-10T20:19:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2bd7e700 1 -- start start 2023-10-10T20:19:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2bd7e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde240cfe20 0x7fde240ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2bd7e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240c5d80 con 0x7fde240d1c30 2023-10-10T20:19:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2bd7e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240c6060 con 0x7fde2415a2b0 2023-10-10T20:19:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2bd7e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde24068470 con 0x7fde240cfe20 2023-10-10T20:19:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2a31b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde240cfe20 0x7fde240ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.217+0000 7fde2a31b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fde240cfe20 0x7fde240ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59500/0 (socket says 172.21.15.148:59500) 2023-10-10T20:19:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde2a31b700 1 -- 172.21.15.148:0/2965113307 learned_addr learned my addr 172.21.15.148:0/2965113307 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde2a31b700 1 -- 172.21.15.148:0/2965113307 >> v1:172.21.15.161:6789/0 conn(0x7fde2415a2b0 legacy=0x7fde240b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde2a31b700 1 -- 172.21.15.148:0/2965113307 >> v1:172.21.15.148:6789/0 conn(0x7fde240d1c30 legacy=0x7fde240cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde28b18700 1 -- 172.21.15.148:0/2965113307 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954779804 0 0) 0x7fde240c5d80 con 0x7fde240d1c30 2023-10-10T20:19:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde2a31b700 1 -- 172.21.15.148:0/2965113307 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240c6340 con 0x7fde240cfe20 2023-10-10T20:19:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde2a31b700 1 --2- 172.21.15.148:0/2965113307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde240ccdd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0099b0 tx=0x7fde1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc2b28991a5b00d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde28b18700 1 -- 172.21.15.148:0/2965113307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00b820 con 0x7fde240cfe20 2023-10-10T20:19:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde28b18700 1 -- 172.21.15.148:0/2965113307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c00b980 con 0x7fde240cfe20 2023-10-10T20:19:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.218+0000 7fde28b18700 1 -- 172.21.15.148:0/2965113307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00bd20 con 0x7fde240cfe20 2023-10-10T20:19:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 -- 172.21.15.148:0/2965113307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 msgr2=0x7fde240ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/2965113307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde240ccdd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0099b0 tx=0x7fde1c009250 comp rx=0 tx=0).stop 2023-10-10T20:19:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 -- 172.21.15.148:0/2965113307 shutdown_connections 2023-10-10T20:19:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/2965113307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde240ccdd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 -- 172.21.15.148:0/2965113307 >> 172.21.15.148:0/2965113307 conn(0x7fde24067e40 msgr2=0x7fde240d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 -- 172.21.15.148:0/2965113307 shutdown_connections 2023-10-10T20:19:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.219+0000 7fde2bd7e700 1 -- 172.21.15.148:0/2965113307 wait complete. 2023-10-10T20:19:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.220+0000 7fde2bd7e700 1 Processor -- start 2023-10-10T20:19:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.220+0000 7fde2bd7e700 1 -- start start 2023-10-10T20:19:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde240d1c30 0x7fde2415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 0x7fde2415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415aed0 con 0x7fde240cfe20 2023-10-10T20:19:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415b1b0 con 0x7fde2415a2b0 2023-10-10T20:19:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415b490 con 0x7fde240d1c30 2023-10-10T20:19:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59516/0 (socket says 172.21.15.148:59516) 2023-10-10T20:19:54.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 -- 172.21.15.148:0/3464431446 learned_addr learned my addr 172.21.15.148:0/3464431446 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:54.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2a31b700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 0x7fde2415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29319700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde240d1c30 0x7fde2415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 -- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde240d1c30 msgr2=0x7fde2415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde240d1c30 0x7fde2415bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 -- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 msgr2=0x7fde2415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 0x7fde2415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 -- 172.21.15.148:0/3464431446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0094f0 con 0x7fde240cfe20 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2a31b700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 0x7fde2415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde29b1a700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fde2000bc30 tx=0x7fde20009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000c9b0 con 0x7fde240cfe20 2023-10-10T20:19:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2000cb50 con 0x7fde240cfe20 2023-10-10T20:19:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2415b770 con 0x7fde240cfe20 2023-10-10T20:19:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20014410 con 0x7fde240cfe20 2023-10-10T20:19:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.221+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2415abc0 con 0x7fde240cfe20 2023-10-10T20:19:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.222+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2404c3c0 con 0x7fde240cfe20 2023-10-10T20:19:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.223+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde20014800 con 0x7fde240cfe20 2023-10-10T20:19:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.223+0000 7fde19ffb700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.223+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde200846c0 con 0x7fde240cfe20 2023-10-10T20:19:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.223+0000 7fde29319700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.223+0000 7fde29319700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde24171700 tx=0x7fde14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.227+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde2004c5a0 con 0x7fde240cfe20 2023-10-10T20:19:54.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.486+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fde24064a00 con 0x7fde240cfe20 2023-10-10T20:19:54.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.617+0000 7fde19ffb700 1 -- 172.21.15.148:0/3464431446 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v46) v1 ==== 133+0+0 (secure 0 0 0) 0x7fde20051930 con 0x7fde240cfe20 2023-10-10T20:19:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 msgr2=0x7fde080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 0x7fde080693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde24171700 tx=0x7fde14009040 comp rx=0 tx=0).stop 2023-10-10T20:19:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 msgr2=0x7fde2409e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fde2000bc30 tx=0x7fde20009820 comp rx=0 tx=0).stop 2023-10-10T20:19:54.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 shutdown_connections 2023-10-10T20:19:54.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde08066f30 0x7fde080693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde2415a2b0 0x7fde2415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde240cfe20 0x7fde2409e930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 --2- 172.21.15.148:0/3464431446 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde240d1c30 0x7fde2415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 >> 172.21.15.148:0/3464431446 conn(0x7fde24067e40 msgr2=0x7fde240b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:54.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 shutdown_connections 2023-10-10T20:19:54.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.620+0000 7fde2bd7e700 1 -- 172.21.15.148:0/3464431446 wait complete. 2023-10-10T20:19:54.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-10T20:19:54.764 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_objectstore":"memstore"} 2023-10-10T20:19:54.764 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-10T20:19:54.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:19:54.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.880+0000 7ff60b287700 1 Processor -- start 2023-10-10T20:19:54.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.880+0000 7ff60b287700 1 -- start start 2023-10-10T20:19:54.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff60b287700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 0x7ff604070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff60b287700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 0x7ff604070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff60b287700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60406e8a0 con 0x7ff6041413b0 2023-10-10T20:19:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff60b287700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60406eb80 con 0x7ff60406e2e0 2023-10-10T20:19:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff60b287700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60406ee60 con 0x7ff604141780 2023-10-10T20:19:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff608822700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 0x7ff604070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff609824700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 0x7ff604070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff609023700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff6041413b0 0x7ff604068a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43536/0 (socket says 172.21.15.148:43536) 2023-10-10T20:19:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.881+0000 7ff609023700 1 -- 172.21.15.148:0/2898806478 learned_addr learned my addr 172.21.15.148:0/2898806478 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:54.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff608822700 1 -- 172.21.15.148:0/2898806478 >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 msgr2=0x7ff604070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:54.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff608822700 1 --2- 172.21.15.148:0/2898806478 >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 0x7ff604070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff608822700 1 -- 172.21.15.148:0/2898806478 >> v1:172.21.15.161:6789/0 conn(0x7ff6041413b0 legacy=0x7ff604068a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff608822700 1 -- 172.21.15.148:0/2898806478 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60406f140 con 0x7ff604141780 2023-10-10T20:19:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff609824700 1 --2- 172.21.15.148:0/2898806478 >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 0x7ff604070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff608822700 1 --2- 172.21.15.148:0/2898806478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 0x7ff604070700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40098e0 tx=0x7ff5f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da10174cc496936d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff5fbfff700 1 -- 172.21.15.148:0/2898806478 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400b640 con 0x7ff604141780 2023-10-10T20:19:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff5fbfff700 1 -- 172.21.15.148:0/2898806478 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f400b7a0 con 0x7ff604141780 2023-10-10T20:19:54.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff5fbfff700 1 -- 172.21.15.148:0/2898806478 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400bb40 con 0x7ff604141780 2023-10-10T20:19:54.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff60b287700 1 -- 172.21.15.148:0/2898806478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 msgr2=0x7ff604070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.882+0000 7ff60b287700 1 --2- 172.21.15.148:0/2898806478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 0x7ff604070700 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40098e0 tx=0x7ff5f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 -- 172.21.15.148:0/2898806478 shutdown_connections 2023-10-10T20:19:54.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 --2- 172.21.15.148:0/2898806478 >> v2:172.21.15.161:3300/0 conn(0x7ff60406e2e0 0x7ff604070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 --2- 172.21.15.148:0/2898806478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff604141780 0x7ff604070700 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 -- 172.21.15.148:0/2898806478 >> 172.21.15.148:0/2898806478 conn(0x7ff6040ce570 msgr2=0x7ff6040ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:54.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 -- 172.21.15.148:0/2898806478 shutdown_connections 2023-10-10T20:19:54.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.883+0000 7ff60b287700 1 -- 172.21.15.148:0/2898806478 wait complete. 2023-10-10T20:19:54.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.884+0000 7ff60b287700 1 Processor -- start 2023-10-10T20:19:54.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.884+0000 7ff60b287700 1 -- start start 2023-10-10T20:19:54.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6041413b0 0x7ff6040c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff604141780 0x7ff6040c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c83f0 con 0x7ff60406e2e0 2023-10-10T20:19:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c77e0 con 0x7ff604141780 2023-10-10T20:19:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c7ac0 con 0x7ff6041413b0 2023-10-10T20:19:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59526/0 (socket says 172.21.15.148:59526) 2023-10-10T20:19:54.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 -- 172.21.15.148:0/464115214 learned_addr learned my addr 172.21.15.148:0/464115214 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:54.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609824700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff604141780 0x7ff6040c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff608822700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6041413b0 0x7ff6040c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 -- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6041413b0 msgr2=0x7ff6040c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6041413b0 0x7ff6040c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 -- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff604141780 msgr2=0x7ff6040c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:54.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff604141780 0x7ff6040c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:54.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 -- 172.21.15.148:0/464115214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600009a20 con 0x7ff60406e2e0 2023-10-10T20:19:54.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff609023700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c7c0 tx=0x7ff5fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc005b30 con 0x7ff60406e2e0 2023-10-10T20:19:54.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc005c90 con 0x7ff60406e2e0 2023-10-10T20:19:54.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f4009420 con 0x7ff60406e2e0 2023-10-10T20:19:54.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc004a80 con 0x7ff60406e2e0 2023-10-10T20:19:54.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.885+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6040c8080 con 0x7ff60406e2e0 2023-10-10T20:19:54.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.886+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60404c3c0 con 0x7ff60406e2e0 2023-10-10T20:19:54.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.887+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5fc0070d0 con 0x7ff60406e2e0 2023-10-10T20:19:54.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.887+0000 7ff5f97fa700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 0x7ff5e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:54.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.887+0000 7ff608822700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 0x7ff5e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:54.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.887+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5fc083840 con 0x7ff60406e2e0 2023-10-10T20:19:54.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.888+0000 7ff608822700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 0x7ff5e80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff604141c00 tx=0x7ff5f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:54.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:54.892+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5fc04b5a0 con 0x7ff60406e2e0 2023-10-10T20:19:55.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.074+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff60406f420 con 0x7ff60406e2e0 2023-10-10T20:19:55.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.077+0000 7ff5f97fa700 1 -- 172.21.15.148:0/464115214 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff5fc050b70 con 0x7ff60406e2e0 2023-10-10T20:19:55.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.079+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 msgr2=0x7ff5e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.079+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 0x7ff5e80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff604141c00 tx=0x7ff5f4017000 comp rx=0 tx=0).stop 2023-10-10T20:19:55.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 msgr2=0x7ff60416af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c7c0 tx=0x7ff5fc007f70 comp rx=0 tx=0).stop 2023-10-10T20:19:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 shutdown_connections 2023-10-10T20:19:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff5e8066f30 0x7ff5e80693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff604141780 0x7ff6040c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60406e2e0 0x7ff60416af80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 --2- 172.21.15.148:0/464115214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6041413b0 0x7ff6040c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 >> 172.21.15.148:0/464115214 conn(0x7ff6040ce570 msgr2=0x7ff6040c40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 shutdown_connections 2023-10-10T20:19:55.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.080+0000 7ff60b287700 1 -- 172.21.15.148:0/464115214 wait complete. 2023-10-10T20:19:55.094 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:19:55.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:19:55.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:19:55.173 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:19:55.173 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:19:55.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:19:55.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-10T20:19:55.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:19:55.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:55.210 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:19:55.265 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:19:55.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:19:55.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-10T20:19:55.310 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:19:55.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:55.311 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:19:55.328 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-10T20:19:55.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:19:55.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-10T20:19:55.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:19:55.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:55.365 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-10T20:19:55.419 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-10T20:19:55.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:19:55.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-10T20:19:55.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:19:55.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:19:55.462 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-10T20:19:55.517 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:19:55.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:55.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.631+0000 7f95a005a700 1 Processor -- start 2023-10-10T20:19:55.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.631+0000 7f95a005a700 1 -- start start 2023-10-10T20:19:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f95a005a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95980c9200 0x7f95980b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f95a005a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95980caa60 con 0x7f95980cba50 2023-10-10T20:19:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f95a005a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95980cf330 con 0x7f95980cc720 2023-10-10T20:19:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f95a005a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95980cf570 con 0x7f95980c9200 2023-10-10T20:19:55.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f959e5f7700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f95980cba50 0x7f95980cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57002/0 (socket says 172.21.15.148:57002) 2023-10-10T20:19:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f959e5f7700 1 -- 172.21.15.148:0/2892048190 learned_addr learned my addr 172.21.15.148:0/2892048190 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f959d5f5700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95980c9200 0x7f95980b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308203255 0 0) 0x7f95980caa60 con 0x7f95980cba50 2023-10-10T20:19:55.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.632+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9584003570 con 0x7f95980cba50 2023-10-10T20:19:55.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944432236 0 0) 0x7f9594001110 con 0x7f95980cc720 2023-10-10T20:19:55.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95980caa60 con 0x7f95980cc720 2023-10-10T20:19:55.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378933577 0 0) 0x7f9584003570 con 0x7f95980cba50 2023-10-10T20:19:55.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95980c9200 msgr2=0x7f95980b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:55.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95980c9200 0x7f95980b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 >> v1:172.21.15.161:6789/0 conn(0x7f95980cc720 legacy=0x7f95980cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980cb5c0 con 0x7f95980cba50 2023-10-10T20:19:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9590001770 con 0x7f95980cba50 2023-10-10T20:19:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 >> v1:172.21.15.148:6789/0 conn(0x7f95980cba50 legacy=0x7f95980cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9584003c20 0x7f9584004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584004810 0x7f9584004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95840051c0 0x7f958400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95980cf570 con 0x7f95840051c0 2023-10-10T20:19:55.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958400c080 con 0x7f9584003c20 2023-10-10T20:19:55.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958400c240 con 0x7f9584004810 2023-10-10T20:19:55.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959d5f5700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95980c9200 0x7f95980b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:55.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95840051c0 0x7f958400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959ddf6700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9584003c20 0x7f9584004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959d5f5700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584004810 0x7f9584004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 -- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584004810 msgr2=0x7f9584004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584004810 0x7f9584004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 -- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9584003c20 msgr2=0x7f9584004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9584003c20 0x7f9584004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 -- 172.21.15.148:0/2892048190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584003570 con 0x7f95840051c0 2023-10-10T20:19:55.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959d5f5700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584004810 0x7f9584004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:55.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959e5f7700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95840051c0 0x7f958400ba60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f959000a770 tx=0x7f95980cb530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51962cdb6e5ea43c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:55.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900070d0 con 0x7f95840051c0 2023-10-10T20:19:55.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590007270 con 0x7f95840051c0 2023-10-10T20:19:55.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.633+0000 7f959cdf4700 1 -- 172.21.15.148:0/2892048190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590007630 con 0x7f95840051c0 2023-10-10T20:19:55.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 -- 172.21.15.148:0/2892048190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95840051c0 msgr2=0x7f958400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 --2- 172.21.15.148:0/2892048190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95840051c0 0x7f958400ba60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f959000a770 tx=0x7f95980cb530 comp rx=0 tx=0).stop 2023-10-10T20:19:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f959e5f7700 1 -- 172.21.15.148:0/2892048190 reap_dead start 2023-10-10T20:19:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 -- 172.21.15.148:0/2892048190 shutdown_connections 2023-10-10T20:19:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 -- 172.21.15.148:0/2892048190 >> 172.21.15.148:0/2892048190 conn(0x7f959806c6e0 msgr2=0x7f9598069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 -- 172.21.15.148:0/2892048190 shutdown_connections 2023-10-10T20:19:55.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.634+0000 7f95a005a700 1 -- 172.21.15.148:0/2892048190 wait complete. 2023-10-10T20:19:55.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.635+0000 7f95a005a700 1 Processor -- start 2023-10-10T20:19:55.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.635+0000 7f95a005a700 1 -- start start 2023-10-10T20:19:55.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959815a020 0x7f95980c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959815a3f0 0x7f9598068230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9598068770 con 0x7f959814f2e0 2023-10-10T20:19:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959814fcc0 con 0x7f959815a3f0 2023-10-10T20:19:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f95a005a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95980c7c70 con 0x7f959815a020 2023-10-10T20:19:55.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59542/0 (socket says 172.21.15.148:59542) 2023-10-10T20:19:55.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 -- 172.21.15.148:0/4122725401 learned_addr learned my addr 172.21.15.148:0/4122725401 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:55.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959d5f5700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959815a3f0 0x7f9598068230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959ddf6700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959815a020 0x7f95980c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 -- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959815a020 msgr2=0x7f95980c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959815a020 0x7f95980c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 -- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959815a3f0 msgr2=0x7f9598068230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959815a3f0 0x7f9598068230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 -- 172.21.15.148:0/4122725401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980c7f50 con 0x7f959814f2e0 2023-10-10T20:19:55.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.636+0000 7f959e5f7700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9590004c10 tx=0x7f9590016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:55.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.637+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590023070 con 0x7f959814f2e0 2023-10-10T20:19:55.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.637+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959000ad90 con 0x7f959814f2e0 2023-10-10T20:19:55.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.637+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95980c8230 con 0x7f959814f2e0 2023-10-10T20:19:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.637+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900070d0 con 0x7f959814f2e0 2023-10-10T20:19:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.637+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95980c87f0 con 0x7f959814f2e0 2023-10-10T20:19:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.638+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95980c89e0 con 0x7f959814f2e0 2023-10-10T20:19:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.638+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9590028410 con 0x7f959814f2e0 2023-10-10T20:19:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.638+0000 7f958dffb700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.639+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95900858c0 con 0x7f959814f2e0 2023-10-10T20:19:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.639+0000 7f959ddf6700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.639+0000 7f959ddf6700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 0x7f957c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f9594007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.643+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9590052bf0 con 0x7f959814f2e0 2023-10-10T20:19:55.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.870+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f959804c3c0 con 0x7f959814f2e0 2023-10-10T20:19:55.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.871+0000 7f958dffb700 1 -- 172.21.15.148:0/4122725401 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f95900075f0 con 0x7f959814f2e0 2023-10-10T20:19:55.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:55.873 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 msgr2=0x7f957c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 0x7f957c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f9594007d10 comp rx=0 tx=0).stop 2023-10-10T20:19:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 msgr2=0x7f959814f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9590004c10 tx=0x7f9590016000 comp rx=0 tx=0).stop 2023-10-10T20:19:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 shutdown_connections 2023-10-10T20:19:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f959814f2e0 0x7f959814f780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959815a020 0x7f95980c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 --2- 172.21.15.148:0/4122725401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959815a3f0 0x7f9598068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 >> 172.21.15.148:0/4122725401 conn(0x7f959806c6e0 msgr2=0x7f9598150640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 shutdown_connections 2023-10-10T20:19:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:55.873+0000 7f95a005a700 1 -- 172.21.15.148:0/4122725401 wait complete. 2023-10-10T20:19:55.878 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:55.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:56.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.000+0000 7f2340d78700 1 Processor -- start 2023-10-10T20:19:56.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.000+0000 7f2340d78700 1 -- start start 2023-10-10T20:19:56.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f2340d78700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233c143520 0x7f233c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f2340d78700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 0x7f233c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f2340d78700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0d0870 con 0x7f233c1438f0 2023-10-10T20:19:56.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f2340d78700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233c0cdfb0 con 0x7f233c143520 2023-10-10T20:19:56.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f2340d78700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233c0cfc60 con 0x7f233c06ab50 2023-10-10T20:19:56.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233c143520 0x7f233c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233c143520 0x7f233c0cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59558/0 (socket says 172.21.15.148:59558) 2023-10-10T20:19:56.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 -- 172.21.15.148:0/3836280132 learned_addr learned my addr 172.21.15.148:0/3836280132 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:56.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233bfff700 1 --2- 172.21.15.148:0/3836280132 >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 0x7f233c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 -- 172.21.15.148:0/3836280132 >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 msgr2=0x7f233c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:56.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 --2- 172.21.15.148:0/3836280132 >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 0x7f233c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 -- 172.21.15.148:0/3836280132 >> v1:172.21.15.161:6789/0 conn(0x7f233c1438f0 legacy=0x7f233c06a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:56.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233b7fe700 1 -- 172.21.15.148:0/3836280132 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0cff40 con 0x7f233c143520 2023-10-10T20:19:56.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.001+0000 7f233bfff700 1 --2- 172.21.15.148:0/3836280132 >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 0x7f233c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:56.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f233b7fe700 1 --2- 172.21.15.148:0/3836280132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c143520 0x7f233c0cf580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f233c0afca0 tx=0x7f232c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49548dab76f107e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f233a7fc700 1 -- 172.21.15.148:0/3836280132 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00b820 con 0x7f233c143520 2023-10-10T20:19:56.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f233a7fc700 1 -- 172.21.15.148:0/3836280132 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00b980 con 0x7f233c143520 2023-10-10T20:19:56.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f233a7fc700 1 -- 172.21.15.148:0/3836280132 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00bd20 con 0x7f233c143520 2023-10-10T20:19:56.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 -- 172.21.15.148:0/3836280132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c143520 msgr2=0x7f233c0cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 --2- 172.21.15.148:0/3836280132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c143520 0x7f233c0cf580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f233c0afca0 tx=0x7f232c009250 comp rx=0 tx=0).stop 2023-10-10T20:19:56.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 -- 172.21.15.148:0/3836280132 shutdown_connections 2023-10-10T20:19:56.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 --2- 172.21.15.148:0/3836280132 >> v2:172.21.15.161:3300/0 conn(0x7f233c06ab50 0x7f233c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 --2- 172.21.15.148:0/3836280132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c143520 0x7f233c0cf580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 -- 172.21.15.148:0/3836280132 >> 172.21.15.148:0/3836280132 conn(0x7f233c069750 msgr2=0x7f233c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 -- 172.21.15.148:0/3836280132 shutdown_connections 2023-10-10T20:19:56.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.002+0000 7f2340d78700 1 -- 172.21.15.148:0/3836280132 wait complete. 2023-10-10T20:19:56.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 Processor -- start 2023-10-10T20:19:56.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 -- start start 2023-10-10T20:19:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f233c06ab50 0x7f233c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f233c143520 0x7f233c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c13d300 con 0x7f233c1438f0 2023-10-10T20:19:56.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c13d580 con 0x7f233c06ab50 2023-10-10T20:19:56.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f2340d78700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c13d860 con 0x7f233c143520 2023-10-10T20:19:56.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f233bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f233bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59572/0 (socket says 172.21.15.148:59572) 2023-10-10T20:19:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.004+0000 7f233bfff700 1 -- 172.21.15.148:0/1435009675 learned_addr learned my addr 172.21.15.148:0/1435009675 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 -- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f233c143520 msgr2=0x7f233c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f233c143520 0x7f233c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 -- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f233c06ab50 msgr2=0x7f233c158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233b7fe700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f233c06ab50 0x7f233c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f233c06ab50 0x7f233c158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 -- 172.21.15.148:0/1435009675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0094f0 con 0x7f233c1438f0 2023-10-10T20:19:56.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f233bfff700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f233400cfb0 tx=0x7f23340078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334015440 con 0x7f233c1438f0 2023-10-10T20:19:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23340155a0 con 0x7f233c1438f0 2023-10-10T20:19:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c13db40 con 0x7f233c1438f0 2023-10-10T20:19:56.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334015960 con 0x7f233c1438f0 2023-10-10T20:19:56.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.005+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c158dc0 con 0x7f233c1438f0 2023-10-10T20:19:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.006+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c13e340 con 0x7f233c1438f0 2023-10-10T20:19:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.006+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2334003020 con 0x7f233c1438f0 2023-10-10T20:19:56.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.007+0000 7f23237fe700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 0x7f23280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.007+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2334084d10 con 0x7f233c1438f0 2023-10-10T20:19:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.007+0000 7f233b7fe700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 0x7f23280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.007+0000 7f233b7fe700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 0x7f23280693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f233c0afca0 tx=0x7f232c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.011+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f233404d5a0 con 0x7f233c1438f0 2023-10-10T20:19:56.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.196+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f233c089030 con 0x7f233c1438f0 2023-10-10T20:19:56.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.196+0000 7f23237fe700 1 -- 172.21.15.148:0/1435009675 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f233c13e340 con 0x7f233c1438f0 2023-10-10T20:19:56.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:56.200 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:56.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.198+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 msgr2=0x7f23280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.198+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 0x7f23280693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f233c0afca0 tx=0x7f232c016000 comp rx=0 tx=0).stop 2023-10-10T20:19:56.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 msgr2=0x7f233c15d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f233400cfb0 tx=0x7f23340078c0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 shutdown_connections 2023-10-10T20:19:56.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2328066f30 0x7f23280693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233c1438f0 0x7f233c15d0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f233c06ab50 0x7f233c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 --2- 172.21.15.148:0/1435009675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f233c143520 0x7f233c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 >> 172.21.15.148:0/1435009675 conn(0x7f233c069750 msgr2=0x7f233c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 shutdown_connections 2023-10-10T20:19:56.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.199+0000 7f2340d78700 1 -- 172.21.15.148:0/1435009675 wait complete. 2023-10-10T20:19:56.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:19:21.398623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:19:21.823720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:19:56.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:19:56.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.329+0000 7fe515dfc700 1 Processor -- start 2023-10-10T20:19:56.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.329+0000 7fe515dfc700 1 -- start start 2023-10-10T20:19:56.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe515dfc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 0x7fe51015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe515dfc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe51006aa00 0x7fe5100b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe515dfc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5100b4d70 con 0x7fe5100b6580 2023-10-10T20:19:56.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe515dfc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100cd8d0 con 0x7fe51006aa00 2023-10-10T20:19:56.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe515dfc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100b4440 con 0x7fe5100cce30 2023-10-10T20:19:56.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe514dfa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 0x7fe51015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe5155fb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe51006aa00 0x7fe5100b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe50ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe5100b6580 0x7fe51006a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43566/0 (socket says 172.21.15.148:43566) 2023-10-10T20:19:56.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.330+0000 7fe50ffff700 1 -- 172.21.15.148:0/2107128457 learned_addr learned my addr 172.21.15.148:0/2107128457 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:56.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe5155fb700 1 -- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 msgr2=0x7fe51015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:56.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe5155fb700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 0x7fe51015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe5155fb700 1 -- 172.21.15.148:0/2107128457 >> v1:172.21.15.161:6789/0 conn(0x7fe5100b6580 legacy=0x7fe51006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe5155fb700 1 -- 172.21.15.148:0/2107128457 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100b4720 con 0x7fe51006aa00 2023-10-10T20:19:56.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe514dfa700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 0x7fe51015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:56.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe50f7fe700 1 -- 172.21.15.148:0/2107128457 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607938178 0 0) 0x7fe5100b4d70 con 0x7fe5100b6580 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe5155fb700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe51006aa00 0x7fe5100b5200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80098e0 tx=0x7fe4f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1336054835efb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe50f7fe700 1 -- 172.21.15.148:0/2107128457 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800b640 con 0x7fe51006aa00 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe50f7fe700 1 -- 172.21.15.148:0/2107128457 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f800b7a0 con 0x7fe51006aa00 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.331+0000 7fe50f7fe700 1 -- 172.21.15.148:0/2107128457 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800bb40 con 0x7fe51006aa00 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 -- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe51006aa00 msgr2=0x7fe5100b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe51006aa00 0x7fe5100b5200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80098e0 tx=0x7fe4f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 -- 172.21.15.148:0/2107128457 shutdown_connections 2023-10-10T20:19:56.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe51006aa00 0x7fe5100b5200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 --2- 172.21.15.148:0/2107128457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100cce30 0x7fe51015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 -- 172.21.15.148:0/2107128457 >> 172.21.15.148:0/2107128457 conn(0x7fe510068cb0 msgr2=0x7fe5100690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 -- 172.21.15.148:0/2107128457 shutdown_connections 2023-10-10T20:19:56.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.332+0000 7fe515dfc700 1 -- 172.21.15.148:0/2107128457 wait complete. 2023-10-10T20:19:56.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.333+0000 7fe515dfc700 1 Processor -- start 2023-10-10T20:19:56.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.333+0000 7fe515dfc700 1 -- start start 2023-10-10T20:19:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100b6580 0x7fe510151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100cce30 0x7fe5101535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510152310 con 0x7fe51006aa00 2023-10-10T20:19:56.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510153b10 con 0x7fe5100cce30 2023-10-10T20:19:56.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe510153140 con 0x7fe5100b6580 2023-10-10T20:19:56.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59576/0 (socket says 172.21.15.148:59576) 2023-10-10T20:19:56.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 -- 172.21.15.148:0/314815904 learned_addr learned my addr 172.21.15.148:0/314815904 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:56.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe50ffff700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100b6580 0x7fe510151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe5155fb700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100cce30 0x7fe5101535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 -- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100b6580 msgr2=0x7fe510151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100b6580 0x7fe510151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 -- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100cce30 msgr2=0x7fe5101535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100cce30 0x7fe5101535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 -- 172.21.15.148:0/314815904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508009a20 con 0x7fe51006aa00 2023-10-10T20:19:56.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe514dfa700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c0f0 tx=0x7fe50800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800cc30 con 0x7fe51006aa00 2023-10-10T20:19:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800f040 con 0x7fe51006aa00 2023-10-10T20:19:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f8009420 con 0x7fe51006aa00 2023-10-10T20:19:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508014480 con 0x7fe51006aa00 2023-10-10T20:19:56.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.334+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510152810 con 0x7fe51006aa00 2023-10-10T20:19:56.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.335+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe510089550 con 0x7fe51006aa00 2023-10-10T20:19:56.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.336+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5080147f0 con 0x7fe51006aa00 2023-10-10T20:19:56.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.336+0000 7fe50cff9700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 0x7fe4f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.336+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5080851a0 con 0x7fe51006aa00 2023-10-10T20:19:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.336+0000 7fe50ffff700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 0x7fe4f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.336+0000 7fe50ffff700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 0x7fe4f00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe51014eea0 tx=0x7fe500007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.340+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe508052410 con 0x7fe51006aa00 2023-10-10T20:19:56.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.522+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5100b4a00 con 0x7fe51006aa00 2023-10-10T20:19:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.522+0000 7fe50cff9700 1 -- 172.21.15.148:0/314815904 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe508055a30 con 0x7fe51006aa00 2023-10-10T20:19:56.525 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:19:56.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.524+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 msgr2=0x7fe4f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.524+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 0x7fe4f00693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe51014eea0 tx=0x7fe500007d30 comp rx=0 tx=0).stop 2023-10-10T20:19:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.524+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 msgr2=0x7fe510154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.524+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c0f0 tx=0x7fe50800b040 comp rx=0 tx=0).stop 2023-10-10T20:19:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 shutdown_connections 2023-10-10T20:19:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4f0066f30 0x7fe4f00693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5100cce30 0x7fe5101535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe51006aa00 0x7fe510154140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 --2- 172.21.15.148:0/314815904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5100b6580 0x7fe510151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 >> 172.21.15.148:0/314815904 conn(0x7fe510068cb0 msgr2=0x7fe5100d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 shutdown_connections 2023-10-10T20:19:56.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.525+0000 7fe515dfc700 1 -- 172.21.15.148:0/314815904 wait complete. 2023-10-10T20:19:56.539 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:19:56.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.656+0000 7fa59967a700 1 Processor -- start 2023-10-10T20:19:56.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa59967a700 1 -- start start 2023-10-10T20:19:56.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa59967a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5940cb280 0x7fa594156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa59967a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5940cc440 con 0x7fa5940c71d0 2023-10-10T20:19:56.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa59967a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5940cb830 con 0x7fa594156160 2023-10-10T20:19:56.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa59967a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940c31a0 con 0x7fa5940cb280 2023-10-10T20:19:56.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa5927fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5940cb280 0x7fa594156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.657+0000 7fa5937fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa5940c71d0 0x7fa59406d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57018/0 (socket says 172.21.15.148:57018) 2023-10-10T20:19:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5927fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5940cb280 0x7fa594156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59592/0 (socket says 172.21.15.148:59592) 2023-10-10T20:19:56.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5937fe700 1 -- 172.21.15.148:0/4098018864 learned_addr learned my addr 172.21.15.148:0/4098018864 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5927fc700 1 -- 172.21.15.148:0/4098018864 >> v1:172.21.15.161:6789/0 conn(0x7fa594156160 legacy=0x7fa5940c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5927fc700 1 -- 172.21.15.148:0/4098018864 >> v1:172.21.15.148:6789/0 conn(0x7fa5940c71d0 legacy=0x7fa59406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5927fc700 1 -- 172.21.15.148:0/4098018864 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940cbb10 con 0x7fa5940cb280 2023-10-10T20:19:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa591ffb700 1 -- 172.21.15.148:0/4098018864 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110619424 0 0) 0x7fa5940cc440 con 0x7fa5940c71d0 2023-10-10T20:19:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa5927fc700 1 --2- 172.21.15.148:0/4098018864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940cb280 0x7fa594156530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa58800bbc0 tx=0x7fa58800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29754b44016c3b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa591ffb700 1 -- 172.21.15.148:0/4098018864 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800db90 con 0x7fa5940cb280 2023-10-10T20:19:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa591ffb700 1 -- 172.21.15.148:0/4098018864 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58800dcf0 con 0x7fa5940cb280 2023-10-10T20:19:56.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.658+0000 7fa591ffb700 1 -- 172.21.15.148:0/4098018864 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880196b0 con 0x7fa5940cb280 2023-10-10T20:19:56.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 -- 172.21.15.148:0/4098018864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940cb280 msgr2=0x7fa594156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 --2- 172.21.15.148:0/4098018864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940cb280 0x7fa594156530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa58800bbc0 tx=0x7fa58800b430 comp rx=0 tx=0).stop 2023-10-10T20:19:56.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 -- 172.21.15.148:0/4098018864 shutdown_connections 2023-10-10T20:19:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 --2- 172.21.15.148:0/4098018864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940cb280 0x7fa594156530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 -- 172.21.15.148:0/4098018864 >> 172.21.15.148:0/4098018864 conn(0x7fa594067660 msgr2=0x7fa594067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 -- 172.21.15.148:0/4098018864 shutdown_connections 2023-10-10T20:19:56.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.659+0000 7fa59967a700 1 -- 172.21.15.148:0/4098018864 wait complete. 2023-10-10T20:19:56.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.660+0000 7fa59967a700 1 Processor -- start 2023-10-10T20:19:56.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.660+0000 7fa59967a700 1 -- start start 2023-10-10T20:19:56.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 0x7fa5940c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa594156160 0x7fa5940c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940ac9e0 con 0x7fa5940c71d0 2023-10-10T20:19:56.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c50c0 con 0x7fa5940cb280 2023-10-10T20:19:56.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c53a0 con 0x7fa594156160 2023-10-10T20:19:56.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59606/0 (socket says 172.21.15.148:59606) 2023-10-10T20:19:56.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 -- 172.21.15.148:0/2253143011 learned_addr learned my addr 172.21.15.148:0/2253143011 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:56.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa5927fc700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 0x7fa5940c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa5937fe700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa594156160 0x7fa5940c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 -- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa594156160 msgr2=0x7fa5940c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa594156160 0x7fa5940c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 -- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 msgr2=0x7fa5940c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 0x7fa5940c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 -- 172.21.15.148:0/2253143011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58800b6d0 con 0x7fa5940c71d0 2023-10-10T20:19:56.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa5927fc700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 0x7fa5940c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:56.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa592ffd700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c4e0 tx=0x7fa58c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011590 con 0x7fa5940c71d0 2023-10-10T20:19:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c011710 con 0x7fa5940c71d0 2023-10-10T20:19:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5940acc30 con 0x7fa5940c71d0 2023-10-10T20:19:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011ad0 con 0x7fa5940c71d0 2023-10-10T20:19:56.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.661+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa594166ae0 con 0x7fa5940c71d0 2023-10-10T20:19:56.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.662+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59404c3c0 con 0x7fa5940c71d0 2023-10-10T20:19:56.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.663+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa58c01b3b0 con 0x7fa5940c71d0 2023-10-10T20:19:56.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.663+0000 7fa57affd700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.663+0000 7fa5927fc700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.663+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa58c083a70 con 0x7fa5940c71d0 2023-10-10T20:19:56.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.664+0000 7fa5927fc700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c59d0 tx=0x7fa58801b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.668+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa58c050ce0 con 0x7fa5940c71d0 2023-10-10T20:19:56.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.849+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5940cbdf0 con 0x7fa5940c71d0 2023-10-10T20:19:56.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.849+0000 7fa57affd700 1 -- 172.21.15.148:0/2253143011 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa58c017020 con 0x7fa5940c71d0 2023-10-10T20:19:56.853 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:19:56.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 msgr2=0x7fa5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 0x7fa5800693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c59d0 tx=0x7fa58801b000 comp rx=0 tx=0).stop 2023-10-10T20:19:56.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 msgr2=0x7fa594154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c4e0 tx=0x7fa58c00c8a0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 shutdown_connections 2023-10-10T20:19:56.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa580066f30 0x7fa5800693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa594156160 0x7fa5940c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5940c71d0 0x7fa594154850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 --2- 172.21.15.148:0/2253143011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5940cb280 0x7fa5940c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 >> 172.21.15.148:0/2253143011 conn(0x7fa594067660 msgr2=0x7fa594069370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 shutdown_connections 2023-10-10T20:19:56.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.852+0000 7fa59967a700 1 -- 172.21.15.148:0/2253143011 wait complete. 2023-10-10T20:19:56.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:19:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.981+0000 7fc54d04c700 1 Processor -- start 2023-10-10T20:19:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.981+0000 7fc54d04c700 1 -- start start 2023-10-10T20:19:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54d04c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc54815a290 0x7fc54806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54d04c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5480b0790 con 0x7fc54806aae0 2023-10-10T20:19:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54d04c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54806af20 con 0x7fc548159ec0 2023-10-10T20:19:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54d04c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5480ce1f0 con 0x7fc54815a290 2023-10-10T20:19:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc54815a290 0x7fc54806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54759e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc54806aae0 0x7fc5480cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57034/0 (socket says 172.21.15.148:57034) 2023-10-10T20:19:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc54815a290 0x7fc54806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59614/0 (socket says 172.21.15.148:59614) 2023-10-10T20:19:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54759e700 1 -- 172.21.15.148:0/1540819771 learned_addr learned my addr 172.21.15.148:0/1540819771 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:56.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 -- 172.21.15.148:0/1540819771 >> v1:172.21.15.161:6789/0 conn(0x7fc548159ec0 legacy=0x7fc5480af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 -- 172.21.15.148:0/1540819771 >> v1:172.21.15.148:6789/0 conn(0x7fc54806aae0 legacy=0x7fc5480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 -- 172.21.15.148:0/1540819771 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480b0b20 con 0x7fc54815a290 2023-10-10T20:19:56.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54659c700 1 -- 172.21.15.148:0/1540819771 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161650857 0 0) 0x7fc5480b0790 con 0x7fc54806aae0 2023-10-10T20:19:56.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc53ffff700 1 --2- 172.21.15.148:0/1540819771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54815a290 0x7fc54806a5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098e0 tx=0x7fc5300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6b5d8e3dea0901 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:56.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54659c700 1 -- 172.21.15.148:0/1540819771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530009e80 con 0x7fc54815a290 2023-10-10T20:19:56.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54659c700 1 -- 172.21.15.148:0/1540819771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53000b860 con 0x7fc54815a290 2023-10-10T20:19:56.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.982+0000 7fc54659c700 1 -- 172.21.15.148:0/1540819771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53000bbe0 con 0x7fc54815a290 2023-10-10T20:19:56.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 -- 172.21.15.148:0/1540819771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54815a290 msgr2=0x7fc54806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:56.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 --2- 172.21.15.148:0/1540819771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54815a290 0x7fc54806a5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098e0 tx=0x7fc5300091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 -- 172.21.15.148:0/1540819771 shutdown_connections 2023-10-10T20:19:56.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 --2- 172.21.15.148:0/1540819771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54815a290 0x7fc54806a5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:56.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 -- 172.21.15.148:0/1540819771 >> 172.21.15.148:0/1540819771 conn(0x7fc5480d1880 msgr2=0x7fc5480cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:56.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 -- 172.21.15.148:0/1540819771 shutdown_connections 2023-10-10T20:19:56.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.983+0000 7fc54d04c700 1 -- 172.21.15.148:0/1540819771 wait complete. 2023-10-10T20:19:56.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.984+0000 7fc54d04c700 1 Processor -- start 2023-10-10T20:19:56.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.984+0000 7fc54d04c700 1 -- start start 2023-10-10T20:19:56.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 0x7fc54816d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 0x7fc54816deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54816e910 con 0x7fc54806aae0 2023-10-10T20:19:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54816e3f0 con 0x7fc54815a290 2023-10-10T20:19:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54816a330 con 0x7fc548159ec0 2023-10-10T20:19:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59630/0 (socket says 172.21.15.148:59630) 2023-10-10T20:19:56.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 -- 172.21.15.148:0/2527271470 learned_addr learned my addr 172.21.15.148:0/2527271470 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:56.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54759e700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 0x7fc54816deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc53ffff700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 0x7fc54816d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:56.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 -- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 msgr2=0x7fc54816d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 0x7fc54816d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 -- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 msgr2=0x7fc54816deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 0x7fc54816deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 -- 172.21.15.148:0/2527271470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530009420 con 0x7fc54806aae0 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54759e700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 0x7fc54816deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc53ffff700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 0x7fc54816d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc546d9d700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc53800c590 tx=0x7fc53800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538011590 con 0x7fc54806aae0 2023-10-10T20:19:57.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5380116f0 con 0x7fc54806aae0 2023-10-10T20:19:57.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538011ab0 con 0x7fc54806aae0 2023-10-10T20:19:57.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.985+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54816a580 con 0x7fc54806aae0 2023-10-10T20:19:57.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.986+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54816ab40 con 0x7fc54806aae0 2023-10-10T20:19:57.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.986+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54804c3c0 con 0x7fc54806aae0 2023-10-10T20:19:57.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.987+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc53801b3b0 con 0x7fc54806aae0 2023-10-10T20:19:57.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.987+0000 7fc53effd700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.987+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc538083a50 con 0x7fc54806aae0 2023-10-10T20:19:57.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.987+0000 7fc53ffff700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.988+0000 7fc53ffff700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 0x7fc5280693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5300093f0 tx=0x7fc530017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:56.991+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc538050cc0 con 0x7fc54806aae0 2023-10-10T20:19:57.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.173+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5480b04b0 con 0x7fc54806aae0 2023-10-10T20:19:57.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.174+0000 7fc53effd700 1 -- 172.21.15.148:0/2527271470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc538017070 con 0x7fc54806aae0 2023-10-10T20:19:57.176 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:19:57.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 msgr2=0x7fc5280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 0x7fc5280693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5300093f0 tx=0x7fc530017000 comp rx=0 tx=0).stop 2023-10-10T20:19:57.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 msgr2=0x7fc5480b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc53800c590 tx=0x7fc53800c950 comp rx=0 tx=0).stop 2023-10-10T20:19:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 shutdown_connections 2023-10-10T20:19:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc528066f30 0x7fc5280693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc54815a290 0x7fc54816deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc54806aae0 0x7fc5480b0f50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 --2- 172.21.15.148:0/2527271470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc548159ec0 0x7fc54816d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 >> 172.21.15.148:0/2527271470 conn(0x7fc5480d1880 msgr2=0x7fc5481579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:57.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 shutdown_connections 2023-10-10T20:19:57.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.176+0000 7fc54d04c700 1 -- 172.21.15.148:0/2527271470 wait complete. 2023-10-10T20:19:57.190 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:57.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.302+0000 7f2dfad0e700 1 Processor -- start 2023-10-10T20:19:57.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.302+0000 7f2dfad0e700 1 -- start start 2023-10-10T20:19:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2dfad0e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2df40cdab0 0x7f2df415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2dfad0e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40cb7c0 con 0x7f2df40cd6e0 2023-10-10T20:19:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2dfad0e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df415e290 con 0x7f2df415dda0 2023-10-10T20:19:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2dfad0e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df415e850 con 0x7f2df40cdab0 2023-10-10T20:19:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df9d0c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2df40cd6e0 0x7f2df40cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57044/0 (socket says 172.21.15.148:57044) 2023-10-10T20:19:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df9d0c700 1 -- 172.21.15.148:0/1672405811 learned_addr learned my addr 172.21.15.148:0/1672405811 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237272695 0 0) 0x7f2df40cb7c0 con 0x7f2df40cd6e0 2023-10-10T20:19:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df950b700 1 --2- 172.21.15.148:0/1672405811 >> v2:172.21.15.161:3300/0 conn(0x7f2df40cdab0 0x7f2df415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ddc003570 con 0x7f2df40cd6e0 2023-10-10T20:19:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401112406 0 0) 0x7f2ddc003570 con 0x7f2df40cd6e0 2023-10-10T20:19:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 >> v2:172.21.15.161:3300/0 conn(0x7f2df40cdab0 msgr2=0x7f2df415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 --2- 172.21.15.148:0/1672405811 >> v2:172.21.15.161:3300/0 conn(0x7f2df40cdab0 0x7f2df415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 >> v1:172.21.15.161:6789/0 conn(0x7f2df415dda0 legacy=0x7f2df415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df415e570 con 0x7f2df40cd6e0 2023-10-10T20:19:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f2dec001470 con 0x7f2df40cd6e0 2023-10-10T20:19:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 >> v1:172.21.15.148:6789/0 conn(0x7f2df40cd6e0 legacy=0x7f2df40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.303+0000 7f2df8d0a700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415e850 con 0x7f2ddc004810 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc00c0d0 con 0x7f2ddc003a60 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc00c290 con 0x7f2ddc0051c0 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df9d0c700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2dfa50d700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 -- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ddc0051c0 msgr2=0x7f2ddc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:57.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 -- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ddc003a60 msgr2=0x7f2ddc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:57.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 -- 172.21.15.148:0/1672405811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc003570 con 0x7f2ddc004810 2023-10-10T20:19:57.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2dfa50d700 1 -- 172.21.15.148:0/1672405811 reap_dead start 2023-10-10T20:19:57.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df950b700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2ddc004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e210 tx=0x7f2de800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ce6ec15a5971e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800eac0 con 0x7f2ddc004810 2023-10-10T20:19:57.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de800f040 con 0x7f2ddc004810 2023-10-10T20:19:57.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.304+0000 7f2df8d0a700 1 -- 172.21.15.148:0/1672405811 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800ee70 con 0x7f2ddc004810 2023-10-10T20:19:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 -- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 msgr2=0x7f2ddc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2ddc004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e210 tx=0x7f2de800b040 comp rx=0 tx=0).stop 2023-10-10T20:19:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 -- 172.21.15.148:0/1672405811 shutdown_connections 2023-10-10T20:19:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/1672405811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 -- 172.21.15.148:0/1672405811 >> 172.21.15.148:0/1672405811 conn(0x7f2df4067660 msgr2=0x7f2df4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 -- 172.21.15.148:0/1672405811 shutdown_connections 2023-10-10T20:19:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.305+0000 7f2dfad0e700 1 -- 172.21.15.148:0/1672405811 wait complete. 2023-10-10T20:19:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.306+0000 7f2dfad0e700 1 Processor -- start 2023-10-10T20:19:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.306+0000 7f2dfad0e700 1 -- start start 2023-10-10T20:19:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df40b5a80 0x7f2df4153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2df40cec80 0x7f2df40cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cc360 con 0x7f2ddc004810 2023-10-10T20:19:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cf640 con 0x7f2df40b5a80 2023-10-10T20:19:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df416e5a0 con 0x7f2df40cec80 2023-10-10T20:19:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59658/0 (socket says 172.21.15.148:59658) 2023-10-10T20:19:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 -- 172.21.15.148:0/847579322 learned_addr learned my addr 172.21.15.148:0/847579322 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df950b700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df40b5a80 0x7f2df4153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfa50d700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2df40cec80 0x7f2df40cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 -- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2df40cec80 msgr2=0x7f2df40cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2df40cec80 0x7f2df40cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 -- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df40b5a80 msgr2=0x7f2df4153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df40b5a80 0x7f2df4153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 -- 172.21.15.148:0/847579322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df416dc70 con 0x7f2ddc004810 2023-10-10T20:19:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df9d0c700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007370 tx=0x7f2dec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00f6c0 con 0x7f2ddc004810 2023-10-10T20:19:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00f820 con 0x7f2ddc004810 2023-10-10T20:19:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df416df50 con 0x7f2ddc004810 2023-10-10T20:19:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.307+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00fbe0 con 0x7f2ddc004810 2023-10-10T20:19:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.308+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df40b5ef0 con 0x7f2ddc004810 2023-10-10T20:19:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.308+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4089280 con 0x7f2ddc004810 2023-10-10T20:19:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.309+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2dec01e3b0 con 0x7f2ddc004810 2023-10-10T20:19:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.309+0000 7f2df1ffb700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 0x7f2de00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.309+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2dec084380 con 0x7f2ddc004810 2023-10-10T20:19:57.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.309+0000 7f2df950b700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 0x7f2de00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.310+0000 7f2df950b700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 0x7f2de00693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de800afd0 tx=0x7f2de801a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.314+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dec051550 con 0x7f2ddc004810 2023-10-10T20:19:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.537+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2df404bb80 con 0x7f2ddc004810 2023-10-10T20:19:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.537+0000 7f2df1ffb700 1 -- 172.21.15.148:0/847579322 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f2dec054b70 con 0x7f2ddc004810 2023-10-10T20:19:57.540 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:57.540 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:57.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 msgr2=0x7f2de00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 0x7f2de00693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de800afd0 tx=0x7f2de801a000 comp rx=0 tx=0).stop 2023-10-10T20:19:57.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 msgr2=0x7f2df40b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2dec007370 tx=0x7f2dec001e50 comp rx=0 tx=0).stop 2023-10-10T20:19:57.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 shutdown_connections 2023-10-10T20:19:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2de0066f30 0x7f2de00693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2df40cec80 0x7f2df40cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ddc004810 0x7f2df40b6150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 --2- 172.21.15.148:0/847579322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df40b5a80 0x7f2df4153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 >> 172.21.15.148:0/847579322 conn(0x7f2df4067660 msgr2=0x7f2df4157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 shutdown_connections 2023-10-10T20:19:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.539+0000 7f2dfad0e700 1 -- 172.21.15.148:0/847579322 wait complete. 2023-10-10T20:19:57.545 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:57.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:57.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.668+0000 7f2ba0809700 1 Processor -- start 2023-10-10T20:19:57.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.668+0000 7f2ba0809700 1 -- start start 2023-10-10T20:19:57.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2ba0809700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b9c159c60 0x7f2b9c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2ba0809700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c068de0 con 0x7f2b9c15a030 2023-10-10T20:19:57.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2ba0809700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0affc0 con 0x7f2b9c06a060 2023-10-10T20:19:57.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2ba0809700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c15b370 con 0x7f2b9c159c60 2023-10-10T20:19:57.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9b59e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2b9c15a030 0x7f2b9c0b0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57058/0 (socket says 172.21.15.148:57058) 2023-10-10T20:19:57.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9b59e700 1 -- 172.21.15.148:0/869078150 learned_addr learned my addr 172.21.15.148:0/869078150 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9ad9d700 1 --2- 172.21.15.148:0/869078150 >> v2:172.21.15.161:3300/0 conn(0x7f2b9c159c60 0x7f2b9c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b99d9b700 1 -- 172.21.15.148:0/869078150 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334034685 0 0) 0x7f2b9c068de0 con 0x7f2b9c15a030 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b99d9b700 1 -- 172.21.15.148:0/869078150 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b84003570 con 0x7f2b9c15a030 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9ad9d700 1 -- 172.21.15.148:0/869078150 >> v1:172.21.15.161:6789/0 conn(0x7f2b9c06a060 legacy=0x7f2b9c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9ad9d700 1 -- 172.21.15.148:0/869078150 >> v1:172.21.15.148:6789/0 conn(0x7f2b9c15a030 legacy=0x7f2b9c0b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9ad9d700 1 -- 172.21.15.148:0/869078150 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0695f0 con 0x7f2b9c159c60 2023-10-10T20:19:57.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.669+0000 7f2b9ad9d700 1 --2- 172.21.15.148:0/869078150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c159c60 0x7f2b9c15ae30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0098e0 tx=0x7f2b8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905c98381cab07cf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2b99d9b700 1 -- 172.21.15.148:0/869078150 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c00b640 con 0x7f2b9c159c60 2023-10-10T20:19:57.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2b99d9b700 1 -- 172.21.15.148:0/869078150 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c00b7a0 con 0x7f2b9c159c60 2023-10-10T20:19:57.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2b99d9b700 1 -- 172.21.15.148:0/869078150 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c00bb40 con 0x7f2b9c159c60 2023-10-10T20:19:57.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 -- 172.21.15.148:0/869078150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c159c60 msgr2=0x7f2b9c15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 --2- 172.21.15.148:0/869078150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c159c60 0x7f2b9c15ae30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0098e0 tx=0x7f2b8c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 -- 172.21.15.148:0/869078150 shutdown_connections 2023-10-10T20:19:57.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 --2- 172.21.15.148:0/869078150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c159c60 0x7f2b9c15ae30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 -- 172.21.15.148:0/869078150 >> 172.21.15.148:0/869078150 conn(0x7f2b9c06c740 msgr2=0x7f2b9c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:57.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 -- 172.21.15.148:0/869078150 shutdown_connections 2023-10-10T20:19:57.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.670+0000 7f2ba0809700 1 -- 172.21.15.148:0/869078150 wait complete. 2023-10-10T20:19:57.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.671+0000 7f2ba0809700 1 Processor -- start 2023-10-10T20:19:57.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 -- start start 2023-10-10T20:19:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c06a060 0x7f2b9c148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9c159c60 0x7f2b9c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c14c6f0 con 0x7f2b9c15a030 2023-10-10T20:19:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0c48e0 con 0x7f2b9c06a060 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2ba0809700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c149720 con 0x7f2b9c159c60 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2b9b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2b9b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59670/0 (socket says 172.21.15.148:59670) 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.672+0000 7f2b9b59e700 1 -- 172.21.15.148:0/1595506307 learned_addr learned my addr 172.21.15.148:0/1595506307 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9ad9d700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c06a060 0x7f2b9c148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9a59c700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9c159c60 0x7f2b9c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 -- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9c159c60 msgr2=0x7f2b9c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9c159c60 0x7f2b9c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 -- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c06a060 msgr2=0x7f2b9c148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c06a060 0x7f2b9c148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 -- 172.21.15.148:0/1595506307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009420 con 0x7f2b9c15a030 2023-10-10T20:19:57.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b9b59e700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b94001460 tx=0x7f2b94001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400ce80 con 0x7f2b9c15a030 2023-10-10T20:19:57.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400ab70 con 0x7f2b9c15a030 2023-10-10T20:19:57.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c0c3cd0 con 0x7f2b9c15a030 2023-10-10T20:19:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940070f0 con 0x7f2b9c15a030 2023-10-10T20:19:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.673+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c0c4290 con 0x7f2b9c15a030 2023-10-10T20:19:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.674+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c06df20 con 0x7f2b9c15a030 2023-10-10T20:19:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.674+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b94007520 con 0x7f2b9c15a030 2023-10-10T20:19:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.675+0000 7f2b82ffd700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.675+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b94084af0 con 0x7f2b9c15a030 2023-10-10T20:19:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.675+0000 7f2b9ad9d700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.675+0000 7f2b9ad9d700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 0x7f2b880693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0093f0 tx=0x7f2b8c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.679+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b94051cd0 con 0x7f2b9c15a030 2023-10-10T20:19:57.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.863+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b9c04bb80 con 0x7f2b9c15a030 2023-10-10T20:19:57.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.863+0000 7f2b82ffd700 1 -- 172.21.15.148:0/1595506307 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2b940552f0 con 0x7f2b9c15a030 2023-10-10T20:19:57.866 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:57.867 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:57.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 msgr2=0x7f2b880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 0x7f2b880693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0093f0 tx=0x7f2b8c017000 comp rx=0 tx=0).stop 2023-10-10T20:19:57.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 msgr2=0x7f2b9c0c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:57.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b94001460 tx=0x7f2b94001ae0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 shutdown_connections 2023-10-10T20:19:57.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b9c15a030 0x7f2b9c0c65a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b9c06a060 0x7f2b9c148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 --2- 172.21.15.148:0/1595506307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9c159c60 0x7f2b9c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 >> 172.21.15.148:0/1595506307 conn(0x7f2b9c06c740 msgr2=0x7f2b9c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 shutdown_connections 2023-10-10T20:19:57.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.866+0000 7f2ba0809700 1 -- 172.21.15.148:0/1595506307 wait complete. 2023-10-10T20:19:57.880 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:19:57.881 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:57.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.994+0000 7fb5fffff700 1 Processor -- start 2023-10-10T20:19:57.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.994+0000 7fb5fffff700 1 -- start start 2023-10-10T20:19:58.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6001438f0 0x7fb60006a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 0x7fb60006a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000d0870 con 0x7fb6001438f0 2023-10-10T20:19:58.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000cdfb0 con 0x7fb6000ccf00 2023-10-10T20:19:58.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000cfc60 con 0x7fb600143520 2023-10-10T20:19:58.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6001438f0 0x7fb60006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6001438f0 0x7fb60006a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59674/0 (socket says 172.21.15.148:59674) 2023-10-10T20:19:58.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 -- 172.21.15.148:0/1189762489 learned_addr learned my addr 172.21.15.148:0/1189762489 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:58.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5ff7fe700 1 --2- 172.21.15.148:0/1189762489 >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 0x7fb60006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5feffd700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 -- 172.21.15.148:0/1189762489 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 msgr2=0x7fb6000cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 -- 172.21.15.148:0/1189762489 >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 msgr2=0x7fb60006a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 --2- 172.21.15.148:0/1189762489 >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 0x7fb60006a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5fe7fc700 1 -- 172.21.15.148:0/1189762489 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000cff40 con 0x7fb6001438f0 2023-10-10T20:19:58.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.995+0000 7fb5ff7fe700 1 --2- 172.21.15.148:0/1189762489 >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 0x7fb60006a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:58.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fe7fc700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6001438f0 0x7fb60006a440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b6cb0 tx=0x7fb5f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c069c50dc01dd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fdffb700 1 -- 172.21.15.148:0/1189762489 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400b860 con 0x7fb6001438f0 2023-10-10T20:19:58.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fdffb700 1 -- 172.21.15.148:0/1189762489 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400b9c0 con 0x7fb6001438f0 2023-10-10T20:19:58.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fdffb700 1 -- 172.21.15.148:0/1189762489 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400bd60 con 0x7fb6001438f0 2023-10-10T20:19:58.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 -- 172.21.15.148:0/1189762489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6001438f0 msgr2=0x7fb60006a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6001438f0 0x7fb60006a440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b6cb0 tx=0x7fb5f4009250 comp rx=0 tx=0).stop 2023-10-10T20:19:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 -- 172.21.15.148:0/1189762489 shutdown_connections 2023-10-10T20:19:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 --2- 172.21.15.148:0/1189762489 >> v2:172.21.15.161:3300/0 conn(0x7fb6000ccf00 0x7fb60006a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 --2- 172.21.15.148:0/1189762489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6001438f0 0x7fb60006a440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 -- 172.21.15.148:0/1189762489 >> 172.21.15.148:0/1189762489 conn(0x7fb600069750 msgr2=0x7fb6000b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 -- 172.21.15.148:0/1189762489 shutdown_connections 2023-10-10T20:19:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.996+0000 7fb5fffff700 1 -- 172.21.15.148:0/1189762489 wait complete. 2023-10-10T20:19:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.997+0000 7fb5fffff700 1 Processor -- start 2023-10-10T20:19:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.997+0000 7fb5fffff700 1 -- start start 2023-10-10T20:19:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6001438f0 0x7fb60016e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000cc7d0 con 0x7fb6000ccf00 2023-10-10T20:19:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60013c720 con 0x7fb6001438f0 2023-10-10T20:19:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5fffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60016db80 con 0x7fb600143520 2023-10-10T20:19:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59688/0 (socket says 172.21.15.148:59688) 2023-10-10T20:19:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 -- 172.21.15.148:0/2954125532 learned_addr learned my addr 172.21.15.148:0/2954125532 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 -- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 msgr2=0x7fb6000cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 -- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6001438f0 msgr2=0x7fb60016e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6001438f0 0x7fb60016e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.998+0000 7fb5feffd700 1 -- 172.21.15.148:0/2954125532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40094f0 con 0x7fb6000ccf00 2023-10-10T20:19:58.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5feffd700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e190 tx=0x7fb5f0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000edd0 con 0x7fb6000ccf00 2023-10-10T20:19:58.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0014420 con 0x7fb6000ccf00 2023-10-10T20:19:58.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60013c160 con 0x7fb6000ccf00 2023-10-10T20:19:58.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00147e0 con 0x7fb6000ccf00 2023-10-10T20:19:58.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:57.999+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60016de20 con 0x7fb6000ccf00 2023-10-10T20:19:58.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.000+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600089030 con 0x7fb6000ccf00 2023-10-10T20:19:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.000+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5f0014b50 con 0x7fb6000ccf00 2023-10-10T20:19:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.000+0000 7fb5e6ffd700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 0x7fb5ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.001+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb5f0084cb0 con 0x7fb6000ccf00 2023-10-10T20:19:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.001+0000 7fb5fe7fc700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 0x7fb5ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.001+0000 7fb5fe7fc700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 0x7fb5ec0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b6cb0 tx=0x7fb5f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.005+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5f00559b0 con 0x7fb6000ccf00 2023-10-10T20:19:58.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.231+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6000d0220 con 0x7fb6000ccf00 2023-10-10T20:19:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.231+0000 7fb5e6ffd700 1 -- 172.21.15.148:0/2954125532 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7fb5f0055540 con 0x7fb6000ccf00 2023-10-10T20:19:58.234 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:58.235 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 msgr2=0x7fb5ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 0x7fb5ec0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b6cb0 tx=0x7fb5f4017000 comp rx=0 tx=0).stop 2023-10-10T20:19:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 msgr2=0x7fb60013bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e190 tx=0x7fb5f0009900 comp rx=0 tx=0).stop 2023-10-10T20:19:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 shutdown_connections 2023-10-10T20:19:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5ec066f30 0x7fb5ec0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6001438f0 0x7fb60016e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6000ccf00 0x7fb60013bb10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 --2- 172.21.15.148:0/2954125532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb600143520 0x7fb6000cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 >> 172.21.15.148:0/2954125532 conn(0x7fb600069750 msgr2=0x7fb6000b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 shutdown_connections 2023-10-10T20:19:58.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.233+0000 7fb5fffff700 1 -- 172.21.15.148:0/2954125532 wait complete. 2023-10-10T20:19:58.240 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:58.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:19:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.363+0000 7f68edcaa700 1 Processor -- start 2023-10-10T20:19:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.363+0000 7f68edcaa700 1 -- start start 2023-10-10T20:19:58.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68edcaa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68e80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68edcaa700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80cb7c0 con 0x7f68e815cbd0 2023-10-10T20:19:58.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68edcaa700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e815e290 con 0x7f68e80cdab0 2023-10-10T20:19:58.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68edcaa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815e850 con 0x7f68e80cd6e0 2023-10-10T20:19:58.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68ed4a9700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68e815cbd0 0x7f68e815d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57060/0 (socket says 172.21.15.148:57060) 2023-10-10T20:19:58.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68ed4a9700 1 -- 172.21.15.148:0/381977429 learned_addr learned my addr 172.21.15.148:0/381977429 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:58.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68ecca8700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68e80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405227793 0 0) 0x7f68e80cb7c0 con 0x7f68e815cbd0 2023-10-10T20:19:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.364+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d0003570 con 0x7f68e815cbd0 2023-10-10T20:19:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175748768 0 0) 0x7f68d0003570 con 0x7f68e815cbd0 2023-10-10T20:19:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 msgr2=0x7f68e80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68e80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 >> v1:172.21.15.161:6789/0 conn(0x7f68e80cdab0 legacy=0x7f68e815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e815e570 con 0x7f68e815cbd0 2023-10-10T20:19:58.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f68e00017f0 con 0x7f68e815cbd0 2023-10-10T20:19:58.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 >> v1:172.21.15.148:6789/0 conn(0x7f68e815cbd0 legacy=0x7f68e815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d0003a90 0x7f68d0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815e850 con 0x7f68d0004d50 2023-10-10T20:19:58.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d000bb70 con 0x7f68d0003a90 2023-10-10T20:19:58.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d000bd30 con 0x7f68e80cd6e0 2023-10-10T20:19:58.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ecca8700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d0003a90 0x7f68d0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68dffff700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 -- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 msgr2=0x7f68d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80cd6e0 0x7f68d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 -- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d0003a90 msgr2=0x7f68d0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d0003a90 0x7f68d0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 -- 172.21.15.148:0/381977429 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0003570 con 0x7f68d0004d50 2023-10-10T20:19:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68ed4a9700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68d00051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68e000a930 tx=0x7f68e00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af524765c85dd1e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00070d0 con 0x7f68d0004d50 2023-10-10T20:19:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0007230 con 0x7f68d0004d50 2023-10-10T20:19:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.365+0000 7f68df7fe700 1 -- 172.21.15.148:0/381977429 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00075d0 con 0x7f68d0004d50 2023-10-10T20:19:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 -- 172.21.15.148:0/381977429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 msgr2=0x7f68d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 --2- 172.21.15.148:0/381977429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68d00051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68e000a930 tx=0x7f68e00021d0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68ed4a9700 1 -- 172.21.15.148:0/381977429 reap_dead start 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 -- 172.21.15.148:0/381977429 shutdown_connections 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 -- 172.21.15.148:0/381977429 >> 172.21.15.148:0/381977429 conn(0x7f68e8067660 msgr2=0x7f68e8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 -- 172.21.15.148:0/381977429 shutdown_connections 2023-10-10T20:19:58.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.366+0000 7f68edcaa700 1 -- 172.21.15.148:0/381977429 wait complete. 2023-10-10T20:19:58.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.367+0000 7f68edcaa700 1 Processor -- start 2023-10-10T20:19:58.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.367+0000 7f68edcaa700 1 -- start start 2023-10-10T20:19:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e815b390 0x7f68e80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80d0870 0x7f68e816b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80ca9c0 con 0x7f68d0004d50 2023-10-10T20:19:58.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80d0cf0 con 0x7f68e815b390 2023-10-10T20:19:58.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68edcaa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815a630 con 0x7f68e80d0870 2023-10-10T20:19:58.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59714/0 (socket says 172.21.15.148:59714) 2023-10-10T20:19:58.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 -- 172.21.15.148:0/3291553037 learned_addr learned my addr 172.21.15.148:0/3291553037 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:58.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ed4a9700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80d0870 0x7f68e816b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68dffff700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e815b390 0x7f68e80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 -- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80d0870 msgr2=0x7f68e816b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80d0870 0x7f68e816b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 -- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e815b390 msgr2=0x7f68e80d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e815b390 0x7f68e80d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 -- 172.21.15.148:0/3291553037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e816b430 con 0x7f68d0004d50 2023-10-10T20:19:58.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.368+0000 7f68ecca8700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f68e4011b10 tx=0x7f68e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.369+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400f9f0 con 0x7f68d0004d50 2023-10-10T20:19:58.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.369+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400fb50 con 0x7f68d0004d50 2023-10-10T20:19:58.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.369+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e815a8b0 con 0x7f68d0004d50 2023-10-10T20:19:58.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.369+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400fed0 con 0x7f68d0004d50 2023-10-10T20:19:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.369+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e815ae70 con 0x7f68d0004d50 2023-10-10T20:19:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.370+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e815b090 con 0x7f68d0004d50 2023-10-10T20:19:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.370+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68e4014650 con 0x7f68d0004d50 2023-10-10T20:19:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.370+0000 7f68dcff9700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 0x7f68d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.370+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68e4018030 con 0x7f68d0004d50 2023-10-10T20:19:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.370+0000 7f68dffff700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 0x7f68d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.371+0000 7f68dffff700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 0x7f68d40693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f68d8003b00 tx=0x7f68d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.375+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e4052290 con 0x7f68d0004d50 2023-10-10T20:19:58.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.596+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68e804bb80 con 0x7f68d0004d50 2023-10-10T20:19:58.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.597+0000 7f68dcff9700 1 -- 172.21.15.148:0/3291553037 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f68e40558b0 con 0x7f68d0004d50 2023-10-10T20:19:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:58.599 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2962317695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2962317695},{"type":"v1","addr":"172.21.15.148:6837","nonce":2962317695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2883114708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2883114708},{"type":"v1","addr":"172.21.15.161:6834","nonce":2883114708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/509058992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":509058992},{"type":"v1","addr":"172.21.15.148:6836","nonce":509058992}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:19:22.857181+0000","modified":"2023-10-10T20:19:26.509765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2566190456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2566190456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2566190456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-10T20:19:58.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 msgr2=0x7f68d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 0x7f68d40693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f68d8003b00 tx=0x7f68d8009040 comp rx=0 tx=0).stop 2023-10-10T20:19:58.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 msgr2=0x7f68e80ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f68e4011b10 tx=0x7f68e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 shutdown_connections 2023-10-10T20:19:58.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f68d4066f30 0x7f68d40693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68e80d0870 0x7f68e816b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d0004d50 0x7f68e80ca480 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 --2- 172.21.15.148:0/3291553037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68e815b390 0x7f68e80d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 >> 172.21.15.148:0/3291553037 conn(0x7f68e8067660 msgr2=0x7f68e80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 shutdown_connections 2023-10-10T20:19:58.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.599+0000 7f68edcaa700 1 -- 172.21.15.148:0/3291553037 wait complete. 2023-10-10T20:19:58.604 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 8 2023-10-10T20:19:58.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:19:58.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.727+0000 7fa47a5cb700 1 Processor -- start 2023-10-10T20:19:58.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.727+0000 7fa47a5cb700 1 -- start start 2023-10-10T20:19:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa47a5cb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa4740c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa47a5cb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa4740abeb0 0x7fa4740c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa47a5cb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740c9cb0 con 0x7fa4740c8400 2023-10-10T20:19:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa47a5cb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4740c90a0 con 0x7fa4740abeb0 2023-10-10T20:19:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa47a5cb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740c9380 con 0x7fa4740abae0 2023-10-10T20:19:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa478b68700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4740c8400 0x7fa4740c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57064/0 (socket says 172.21.15.148:57064) 2023-10-10T20:19:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa478b68700 1 -- 172.21.15.148:0/1042503421 learned_addr learned my addr 172.21.15.148:0/1042503421 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa473fff700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa4740c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa4737fe700 1 --2- 172.21.15.148:0/1042503421 >> v2:172.21.15.161:3300/0 conn(0x7fa4740abeb0 0x7fa4740c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005883654 0 0) 0x7fa4740c9cb0 con 0x7fa4740c8400 2023-10-10T20:19:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa45c003570 con 0x7fa4740c8400 2023-10-10T20:19:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940187098 0 0) 0x7fa45c003570 con 0x7fa4740c8400 2023-10-10T20:19:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 msgr2=0x7fa4740c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa4740c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 >> v2:172.21.15.161:3300/0 conn(0x7fa4740abeb0 msgr2=0x7fa4740c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 --2- 172.21.15.148:0/1042503421 >> v2:172.21.15.161:3300/0 conn(0x7fa4740abeb0 0x7fa4740c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740c9660 con 0x7fa4740c8400 2023-10-10T20:19:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa46c0017f0 con 0x7fa4740c8400 2023-10-10T20:19:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 >> v1:172.21.15.148:6789/0 conn(0x7fa4740c8400 legacy=0x7fa4740c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c003a60 0x7fa45c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa45c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45c004d50 0x7fa45c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740c90a0 con 0x7fa45c004d50 2023-10-10T20:19:58.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740c9380 con 0x7fa45c003a60 2023-10-10T20:19:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.728+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c00ba60 con 0x7fa4740abae0 2023-10-10T20:19:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45c004d50 0x7fa45c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa473fff700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c003a60 0x7fa45c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa4737fe700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa45c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 -- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 msgr2=0x7fa45c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4740abae0 0x7fa45c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 -- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c003a60 msgr2=0x7fa45c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45c003a60 0x7fa45c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 -- 172.21.15.148:0/1042503421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c003570 con 0x7fa45c004d50 2023-10-10T20:19:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa478b68700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45c004d50 0x7fa45c0051c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00a970 tx=0x7fa46c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785d2f38fa1b952f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012440 con 0x7fa45c004d50 2023-10-10T20:19:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c0125c0 con 0x7fa45c004d50 2023-10-10T20:19:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa472ffd700 1 -- 172.21.15.148:0/1042503421 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012980 con 0x7fa45c004d50 2023-10-10T20:19:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1042503421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45c004d50 msgr2=0x7fa45c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.729+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1042503421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45c004d50 0x7fa45c0051c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00a970 tx=0x7fa46c002210 comp rx=0 tx=0).stop 2023-10-10T20:19:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.730+0000 7fa478b68700 1 -- 172.21.15.148:0/1042503421 reap_dead start 2023-10-10T20:19:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.730+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1042503421 shutdown_connections 2023-10-10T20:19:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.730+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1042503421 >> 172.21.15.148:0/1042503421 conn(0x7fa474067660 msgr2=0x7fa474067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.730+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1042503421 shutdown_connections 2023-10-10T20:19:58.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.730+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1042503421 wait complete. 2023-10-10T20:19:58.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.731+0000 7fa47a5cb700 1 Processor -- start 2023-10-10T20:19:58.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.731+0000 7fa47a5cb700 1 -- start start 2023-10-10T20:19:58.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 0x7fa4740af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 0x7fa4740b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740b0ab0 con 0x7fa4740b1480 2023-10-10T20:19:58.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740afc00 con 0x7fa4741481e0 2023-10-10T20:19:58.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740afea0 con 0x7fa474147e10 2023-10-10T20:19:58.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59728/0 (socket says 172.21.15.148:59728) 2023-10-10T20:19:58.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 -- 172.21.15.148:0/1676105494 learned_addr learned my addr 172.21.15.148:0/1676105494 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:58.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa473fff700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 0x7fa4740af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa4737fe700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 0x7fa4740b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 -- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 msgr2=0x7fa4740af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 0x7fa4740af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 -- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 msgr2=0x7fa4740b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 0x7fa4740b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 -- 172.21.15.148:0/1676105494 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740b0180 con 0x7fa4740b1480 2023-10-10T20:19:58.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa4737fe700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 0x7fa4740b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:58.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa473fff700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 0x7fa4740af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:58.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa478b68700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00a000 tx=0x7fa46c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c023070 con 0x7fa4740b1480 2023-10-10T20:19:58.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c001360 con 0x7fa4740b1480 2023-10-10T20:19:58.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4740b0460 con 0x7fa4740b1480 2023-10-10T20:19:58.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012440 con 0x7fa4740b1480 2023-10-10T20:19:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.732+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa474147310 con 0x7fa4740b1480 2023-10-10T20:19:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.733+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4740b0900 con 0x7fa4740b1480 2023-10-10T20:19:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.734+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa46c019030 con 0x7fa4740b1480 2023-10-10T20:19:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.734+0000 7fa45bfff700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.734+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa46c085790 con 0x7fa4740b1480 2023-10-10T20:19:58.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.734+0000 7fa473fff700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:58.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.734+0000 7fa473fff700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 0x7fa4600693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa464006f00 tx=0x7fa464008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:58.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.738+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa46c052a00 con 0x7fa4740b1480 2023-10-10T20:19:58.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.922+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4740ab6d0 con 0x7fa4740b1480 2023-10-10T20:19:58.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.923+0000 7fa45bfff700 1 -- 172.21.15.148:0/1676105494 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa46c017090 con 0x7fa4740b1480 2023-10-10T20:19:58.925 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:19:58.925 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":20,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:19:24.478064+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:19:21.398623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-10T20:19:21.823720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:19:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 msgr2=0x7fa4600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 0x7fa4600693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa464006f00 tx=0x7fa464008040 comp rx=0 tx=0).stop 2023-10-10T20:19:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 msgr2=0x7fa4741542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00a000 tx=0x7fa46c015000 comp rx=0 tx=0).stop 2023-10-10T20:19:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 shutdown_connections 2023-10-10T20:19:58.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa460066f30 0x7fa4600693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b1480 0x7fa4741542a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa474147e10 0x7fa4740af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 --2- 172.21.15.148:0/1676105494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4741481e0 0x7fa4740b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:58.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.925+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 >> 172.21.15.148:0/1676105494 conn(0x7fa474067660 msgr2=0x7fa4740c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:58.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.926+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 shutdown_connections 2023-10-10T20:19:58.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:58.926+0000 7fa47a5cb700 1 -- 172.21.15.148:0/1676105494 wait complete. 2023-10-10T20:19:58.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:19:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.053+0000 7ff1b7663700 1 Processor -- start 2023-10-10T20:19:59.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.055+0000 7ff1b7663700 1 -- start start 2023-10-10T20:19:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff1b0152010 0x7ff1b015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b006c010 con 0x7ff1b01523e0 2023-10-10T20:19:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b006bd10 con 0x7ff1b0158e50 2023-10-10T20:19:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b015c060 con 0x7ff1b0152010 2023-10-10T20:19:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b4bfe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1b01523e0 0x7ff1b006c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57078/0 (socket says 172.21.15.148:57078) 2023-10-10T20:19:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b4bfe700 1 -- 172.21.15.148:0/113141896 learned_addr learned my addr 172.21.15.148:0/113141896 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:59.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/113141896 >> v2:172.21.15.161:3300/0 conn(0x7ff1b0152010 0x7ff1b015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146526611 0 0) 0x7ff1b006c010 con 0x7ff1b01523e0 2023-10-10T20:19:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19c003570 con 0x7ff1b01523e0 2023-10-10T20:19:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048777636 0 0) 0x7ff1b006bd10 con 0x7ff1b0158e50 2023-10-10T20:19:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b006c010 con 0x7ff1b0158e50 2023-10-10T20:19:59.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b53ff700 1 -- 172.21.15.148:0/113141896 >> v1:172.21.15.161:6789/0 conn(0x7ff1b0158e50 legacy=0x7ff1b00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b53ff700 1 -- 172.21.15.148:0/113141896 >> v1:172.21.15.148:6789/0 conn(0x7ff1b01523e0 legacy=0x7ff1b006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b53ff700 1 -- 172.21.15.148:0/113141896 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b015c340 con 0x7ff1b0152010 2023-10-10T20:19:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/113141896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b015b810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098e0 tx=0x7ff1ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2aeb8018ed53de server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac011630 con 0x7ff1b0152010 2023-10-10T20:19:59.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac011790 con 0x7ff1b0152010 2023-10-10T20:19:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1a7fff700 1 -- 172.21.15.148:0/113141896 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac011b30 con 0x7ff1b0152010 2023-10-10T20:19:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- 172.21.15.148:0/113141896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 msgr2=0x7ff1b015b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 --2- 172.21.15.148:0/113141896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b015b810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098e0 tx=0x7ff1ac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- 172.21.15.148:0/113141896 shutdown_connections 2023-10-10T20:19:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 --2- 172.21.15.148:0/113141896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b015b810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- 172.21.15.148:0/113141896 >> 172.21.15.148:0/113141896 conn(0x7ff1b0069870 msgr2=0x7ff1b0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- 172.21.15.148:0/113141896 shutdown_connections 2023-10-10T20:19:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.056+0000 7ff1b7663700 1 -- 172.21.15.148:0/113141896 wait complete. 2023-10-10T20:19:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.057+0000 7ff1b7663700 1 Processor -- start 2023-10-10T20:19:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.057+0000 7ff1b7663700 1 -- start start 2023-10-10T20:19:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 0x7ff1b0067f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c7310 con 0x7ff1b0158e50 2023-10-10T20:19:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b014ff60 con 0x7ff1b0152010 2023-10-10T20:19:59.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b7663700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00b48a0 con 0x7ff1b01523e0 2023-10-10T20:19:59.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b5c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b5c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59738/0 (socket says 172.21.15.148:59738) 2023-10-10T20:19:59.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.058+0000 7ff1b5c00700 1 -- 172.21.15.148:0/2233002004 learned_addr learned my addr 172.21.15.148:0/2233002004 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:59.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b4bfe700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 0x7ff1b0067f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 -- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 msgr2=0x7ff1b0067f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 0x7ff1b0067f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 -- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 msgr2=0x7ff1b00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b00b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 -- 172.21.15.148:0/2233002004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac009420 con 0x7ff1b0158e50 2023-10-10T20:19:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b4bfe700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 0x7ff1b0067f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:19:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b5c00700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c910 tx=0x7ff1a80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800a7e0 con 0x7ff1b0158e50 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800a960 con 0x7ff1b0158e50 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b00c74f0 con 0x7ff1b0158e50 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800ad20 con 0x7ff1b0158e50 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.059+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b00c7b30 con 0x7ff1b0158e50 2023-10-10T20:19:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.060+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b004c3c0 con 0x7ff1b0158e50 2023-10-10T20:19:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.060+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1a8012630 con 0x7ff1b0158e50 2023-10-10T20:19:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.061+0000 7ff1a57fa700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.061+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1a80849f0 con 0x7ff1b0158e50 2023-10-10T20:19:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.061+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.061+0000 7ff1b53ff700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0093f0 tx=0x7ff1ac017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.065+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1a8051d10 con 0x7ff1b0158e50 2023-10-10T20:19:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.286+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff1b006d1b0 con 0x7ff1b0158e50 2023-10-10T20:19:59.289 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:19:59.287+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:19:59.295 INFO:tasks.ceph.mds.d.smithi161.stderr: -1646> 2023-10-10T20:19:27.782+0000 7f890359e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:19:59.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.293+0000 7ff1a57fa700 1 -- 172.21.15.148:0/2233002004 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff1a8055330 con 0x7ff1b0158e50 2023-10-10T20:19:59.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 msgr2=0x7ff1940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0093f0 tx=0x7ff1ac017000 comp rx=0 tx=0).stop 2023-10-10T20:19:59.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 msgr2=0x7ff1b016fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c910 tx=0x7ff1a80027a0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 shutdown_connections 2023-10-10T20:19:59.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b0158e50 0x7ff1b016fa50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1b0152010 0x7ff1b00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 --2- 172.21.15.148:0/2233002004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1b01523e0 0x7ff1b0067f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.295+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 >> 172.21.15.148:0/2233002004 conn(0x7ff1b0069870 msgr2=0x7ff1b0150760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:59.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.296+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 shutdown_connections 2023-10-10T20:19:59.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.296+0000 7ff1b7663700 1 -- 172.21.15.148:0/2233002004 wait complete. 2023-10-10T20:19:59.300 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:19:59.310 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:19:59.319 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:19:59.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.424+0000 7f14beabc700 1 Processor -- start 2023-10-10T20:19:59.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.424+0000 7f14beabc700 1 -- start start 2023-10-10T20:19:59.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14beabc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 0x7f14b815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14beabc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14b80b6580 0x7f14b806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14beabc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80b4d70 con 0x7f14b806a830 2023-10-10T20:19:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14beabc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b806ac70 con 0x7f14b80b6580 2023-10-10T20:19:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14beabc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80cd8d0 con 0x7f14b80cce30 2023-10-10T20:19:59.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14bdaba700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 0x7f14b815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14bd2b9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14b80b6580 0x7f14b806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14be2bb700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f14b806a830 0x7f14b815d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43616/0 (socket says 172.21.15.148:43616) 2023-10-10T20:19:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14bdaba700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 0x7f14b815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:46192/0 (socket says 172.21.15.148:46192) 2023-10-10T20:19:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14be2bb700 1 -- 172.21.15.148:0/2034993030 learned_addr learned my addr 172.21.15.148:0/2034993030 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:19:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.425+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/2034993030 >> v2:172.21.15.161:3300/0 conn(0x7f14b80b6580 0x7f14b806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:36608/0 (socket says 172.21.15.148:36608) 2023-10-10T20:19:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bd2b9700 1 -- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 msgr2=0x7f14b815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:19:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 0x7f14b815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bd2b9700 1 -- 172.21.15.148:0/2034993030 >> v1:172.21.15.161:6789/0 conn(0x7f14b806a830 legacy=0x7f14b815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bd2b9700 1 -- 172.21.15.148:0/2034993030 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80b4440 con 0x7f14b80b6580 2023-10-10T20:19:59.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bcab8700 1 -- 172.21.15.148:0/2034993030 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889032157 0 0) 0x7f14b80b4d70 con 0x7f14b806a830 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80b6580 0x7f14b806a2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14a80098e0 tx=0x7f14a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3087e67d923b8ddb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bcab8700 1 -- 172.21.15.148:0/2034993030 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800b640 con 0x7f14b80b6580 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bcab8700 1 -- 172.21.15.148:0/2034993030 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800b7a0 con 0x7f14b80b6580 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14bcab8700 1 -- 172.21.15.148:0/2034993030 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800bb40 con 0x7f14b80b6580 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14beabc700 1 -- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80b6580 msgr2=0x7f14b806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14beabc700 1 --2- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80b6580 0x7f14b806a2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14a80098e0 tx=0x7f14a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.426+0000 7f14beabc700 1 -- 172.21.15.148:0/2034993030 shutdown_connections 2023-10-10T20:19:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.427+0000 7f14beabc700 1 --2- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b80cce30 0x7f14b815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.427+0000 7f14beabc700 1 --2- 172.21.15.148:0/2034993030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80b6580 0x7f14b806a2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.427+0000 7f14beabc700 1 -- 172.21.15.148:0/2034993030 >> 172.21.15.148:0/2034993030 conn(0x7f14b8068cb0 msgr2=0x7f14b80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:19:59.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.427+0000 7f14beabc700 1 -- 172.21.15.148:0/2034993030 shutdown_connections 2023-10-10T20:19:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.427+0000 7f14beabc700 1 -- 172.21.15.148:0/2034993030 wait complete. 2023-10-10T20:19:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 Processor -- start 2023-10-10T20:19:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 -- start start 2023-10-10T20:19:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 0x7f14b816afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80cce30 0x7f14b813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b81527f0 con 0x7f14b80b6580 2023-10-10T20:19:59.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b813c670 con 0x7f14b80cce30 2023-10-10T20:19:59.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.428+0000 7f14beabc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8153630 con 0x7f14b806a830 2023-10-10T20:19:59.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59748/0 (socket says 172.21.15.148:59748) 2023-10-10T20:19:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 -- 172.21.15.148:0/3310869389 learned_addr learned my addr 172.21.15.148:0/3310869389 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:19:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bdaba700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 0x7f14b816afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14be2bb700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80cce30 0x7f14b813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 -- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 msgr2=0x7f14b816afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 0x7f14b816afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 -- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80cce30 msgr2=0x7f14b813c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:19:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80cce30 0x7f14b813c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:19:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 -- 172.21.15.148:0/3310869389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009420 con 0x7f14b80b6580 2023-10-10T20:19:59.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bdaba700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 0x7f14b816afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:19:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14bd2b9700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14a8013000 tx=0x7f14a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800bb20 con 0x7f14b80b6580 2023-10-10T20:19:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8000c90 con 0x7f14b80b6580 2023-10-10T20:19:59.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8152a20 con 0x7f14b80b6580 2023-10-10T20:19:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011630 con 0x7f14b80b6580 2023-10-10T20:19:59.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.429+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b8152fe0 con 0x7f14b80b6580 2023-10-10T20:19:59.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.430+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b8065050 con 0x7f14b80b6580 2023-10-10T20:19:59.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.431+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14a80119a0 con 0x7f14b80b6580 2023-10-10T20:19:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.431+0000 7f14adffb700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:19:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.431+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f14a8085820 con 0x7f14b80b6580 2023-10-10T20:19:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.431+0000 7f14bdaba700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:19:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.431+0000 7f14bdaba700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 0x7f149c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b0a50 tx=0x7f14b40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:19:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.435+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14a8052a40 con 0x7f14b80b6580 2023-10-10T20:19:59.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:19:59.664+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f14b80895b0 con 0x7f14b80b6580 2023-10-10T20:20:00.000 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:19:59.999+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2023-10-10T20:20:00.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.292+0000 7f14adffb700 1 -- 172.21.15.148:0/3310869389 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f14a8056060 con 0x7f14b80b6580 2023-10-10T20:20:00.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 msgr2=0x7f149c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 0x7f149c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b0a50 tx=0x7f14b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 msgr2=0x7f14b81522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14a8013000 tx=0x7f14a8017000 comp rx=0 tx=0).stop 2023-10-10T20:20:00.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 shutdown_connections 2023-10-10T20:20:00.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f149c066f30 0x7f149c0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b80cce30 0x7f14b813c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b806a830 0x7f14b816afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 --2- 172.21.15.148:0/3310869389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b80b6580 0x7f14b81522b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 >> 172.21.15.148:0/3310869389 conn(0x7f14b8068cb0 msgr2=0x7f14b8150610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:00.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 shutdown_connections 2023-10-10T20:20:00.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.295+0000 7f14beabc700 1 -- 172.21.15.148:0/3310869389 wait complete. 2023-10-10T20:20:00.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:20:00.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.428+0000 7f48f114d700 1 Processor -- start 2023-10-10T20:20:00.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.428+0000 7f48f114d700 1 -- start start 2023-10-10T20:20:00.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48f114d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48ec0af8f0 0x7f48ec0cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48f114d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 0x7f48ec0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48f114d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0c6f20 con 0x7f48ec156210 2023-10-10T20:20:00.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48f114d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0c7200 con 0x7f48ec0af8f0 2023-10-10T20:20:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48f114d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c74e0 con 0x7f48ec06dad0 2023-10-10T20:20:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48e259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48ec0af8f0 0x7f48ec0cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48e259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48ec0af8f0 0x7f48ec0cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59752/0 (socket says 172.21.15.148:59752) 2023-10-10T20:20:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48e259c700 1 -- 172.21.15.148:0/1498709239 learned_addr learned my addr 172.21.15.148:0/1498709239 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.429+0000 7f48eb59e700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 0x7f48ec0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48e259c700 1 -- 172.21.15.148:0/1498709239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 msgr2=0x7f48ec0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48e259c700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 0x7f48ec0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48e259c700 1 -- 172.21.15.148:0/1498709239 >> v1:172.21.15.161:6789/0 conn(0x7f48ec156210 legacy=0x7f48ec06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48e259c700 1 -- 172.21.15.148:0/1498709239 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0c77c0 con 0x7f48ec0af8f0 2023-10-10T20:20:00.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48eb59e700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 0x7f48ec0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:00.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48e259c700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec0af8f0 0x7f48ec0cb770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099b0 tx=0x7f48d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5551a3ad70941c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:00.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48ea59c700 1 -- 172.21.15.148:0/1498709239 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400b860 con 0x7f48ec0af8f0 2023-10-10T20:20:00.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48ea59c700 1 -- 172.21.15.148:0/1498709239 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400b9c0 con 0x7f48ec0af8f0 2023-10-10T20:20:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48ea59c700 1 -- 172.21.15.148:0/1498709239 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400bd60 con 0x7f48ec0af8f0 2023-10-10T20:20:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 -- 172.21.15.148:0/1498709239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec0af8f0 msgr2=0x7f48ec0cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec0af8f0 0x7f48ec0cb770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48d40099b0 tx=0x7f48d4009250 comp rx=0 tx=0).stop 2023-10-10T20:20:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 -- 172.21.15.148:0/1498709239 shutdown_connections 2023-10-10T20:20:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec06dad0 0x7f48ec0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 --2- 172.21.15.148:0/1498709239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec0af8f0 0x7f48ec0cb770 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.430+0000 7f48f114d700 1 -- 172.21.15.148:0/1498709239 >> 172.21.15.148:0/1498709239 conn(0x7f48ec0cd450 msgr2=0x7f48ec0c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:00.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.431+0000 7f48f114d700 1 -- 172.21.15.148:0/1498709239 shutdown_connections 2023-10-10T20:20:00.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.431+0000 7f48f114d700 1 -- 172.21.15.148:0/1498709239 wait complete. 2023-10-10T20:20:00.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 Processor -- start 2023-10-10T20:20:00.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 -- start start 2023-10-10T20:20:00.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48ec0af8f0 0x7f48ec14b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec156210 0x7f48ec157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec157da0 con 0x7f48ec06dad0 2023-10-10T20:20:00.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0ad9b0 con 0x7f48ec0af8f0 2023-10-10T20:20:00.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.432+0000 7f48f114d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0ad080 con 0x7f48ec156210 2023-10-10T20:20:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59760/0 (socket says 172.21.15.148:59760) 2023-10-10T20:20:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 -- 172.21.15.148:0/3984043533 learned_addr learned my addr 172.21.15.148:0/3984043533 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48e259c700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48ec0af8f0 0x7f48ec14b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48eb59e700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec156210 0x7f48ec157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 -- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec156210 msgr2=0x7f48ec157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec156210 0x7f48ec157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 -- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48ec0af8f0 msgr2=0x7f48ec14b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48ec0af8f0 0x7f48ec14b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:00.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 -- 172.21.15.148:0/3984043533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40094f0 con 0x7f48ec06dad0 2023-10-10T20:20:00.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48ead9d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c590 tx=0x7f48dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:00.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011590 con 0x7f48ec06dad0 2023-10-10T20:20:00.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc011710 con 0x7f48ec06dad0 2023-10-10T20:20:00.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011ad0 con 0x7f48ec06dad0 2023-10-10T20:20:00.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0ad360 con 0x7f48ec06dad0 2023-10-10T20:20:00.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.433+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec1594a0 con 0x7f48ec06dad0 2023-10-10T20:20:00.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.434+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec159690 con 0x7f48ec06dad0 2023-10-10T20:20:00.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.434+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48dc01b3b0 con 0x7f48ec06dad0 2023-10-10T20:20:00.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.435+0000 7f48e37fe700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 0x7f48cc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:00.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.435+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f48dc083990 con 0x7f48ec06dad0 2023-10-10T20:20:00.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.435+0000 7f48e259c700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 0x7f48cc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:00.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.435+0000 7f48e259c700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 0x7f48cc0693b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48ec06d180 tx=0x7f48d4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:00.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.439+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48ec159690 con 0x7f48ec06dad0 2023-10-10T20:20:00.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:00.621+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48ec159690 con 0x7f48ec06dad0 2023-10-10T20:20:01.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.478+0000 7f48e37fe700 1 -- 172.21.15.148:0/3984043533 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f48ec159690 con 0x7f48ec06dad0 2023-10-10T20:20:01.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 msgr2=0x7f48cc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:01.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 0x7f48cc0693b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48ec06d180 tx=0x7f48d4017000 comp rx=0 tx=0).stop 2023-10-10T20:20:01.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 msgr2=0x7f48ec0c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:01.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c590 tx=0x7f48dc00c8a0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 shutdown_connections 2023-10-10T20:20:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48cc066f00 0x7f48cc0693b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48ec156210 0x7f48ec157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48ec06dad0 0x7f48ec0c1870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 --2- 172.21.15.148:0/3984043533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48ec0af8f0 0x7f48ec14b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 >> 172.21.15.148:0/3984043533 conn(0x7f48ec0cd450 msgr2=0x7f48ec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:01.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 shutdown_connections 2023-10-10T20:20:01.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.481+0000 7f48f114d700 1 -- 172.21.15.148:0/3984043533 wait complete. 2023-10-10T20:20:01.487 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:20:01.496 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:20:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.619+0000 7fa148447700 1 Processor -- start 2023-10-10T20:20:01.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.619+0000 7fa148447700 1 -- start start 2023-10-10T20:20:01.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa148447700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1400b6a10 0x7fa14015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa148447700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1400b46f0 0x7fa1400b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa148447700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400b5090 con 0x7fa140159ef0 2023-10-10T20:20:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa148447700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400b17b0 con 0x7fa1400b46f0 2023-10-10T20:20:01.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa148447700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400b1a60 con 0x7fa1400b6a10 2023-10-10T20:20:01.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa1469e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1400b46f0 0x7fa1400b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa1469e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1400b46f0 0x7fa1400b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59774/0 (socket says 172.21.15.148:59774) 2023-10-10T20:20:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa1469e4700 1 -- 172.21.15.148:0/2214728126 learned_addr learned my addr 172.21.15.148:0/2214728126 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:01.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.620+0000 7fa1459e2700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1400b6a10 0x7fa14015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1469e4700 1 -- 172.21.15.148:0/2214728126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1400b6a10 msgr2=0x7fa14015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1469e4700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1400b6a10 0x7fa14015a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1469e4700 1 -- 172.21.15.148:0/2214728126 >> v1:172.21.15.161:6789/0 conn(0x7fa140159ef0 legacy=0x7fa1400b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:01.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1469e4700 1 -- 172.21.15.148:0/2214728126 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400b1d40 con 0x7fa1400b46f0 2023-10-10T20:20:01.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1469e4700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b46f0 0x7fa1400b4b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa138009f20 tx=0x7fa138009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ddb6997a8907a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:01.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1451e1700 1 -- 172.21.15.148:0/2214728126 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800e070 con 0x7fa1400b46f0 2023-10-10T20:20:01.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1451e1700 1 -- 172.21.15.148:0/2214728126 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800ba40 con 0x7fa1400b46f0 2023-10-10T20:20:01.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa1451e1700 1 -- 172.21.15.148:0/2214728126 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380092e0 con 0x7fa1400b46f0 2023-10-10T20:20:01.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 -- 172.21.15.148:0/2214728126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b46f0 msgr2=0x7fa1400b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:01.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b46f0 0x7fa1400b4b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa138009f20 tx=0x7fa138009840 comp rx=0 tx=0).stop 2023-10-10T20:20:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 -- 172.21.15.148:0/2214728126 shutdown_connections 2023-10-10T20:20:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b46f0 0x7fa1400b4b50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 --2- 172.21.15.148:0/2214728126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1400b6a10 0x7fa14015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.621+0000 7fa148447700 1 -- 172.21.15.148:0/2214728126 >> 172.21.15.148:0/2214728126 conn(0x7fa14009a0a0 msgr2=0x7fa140067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.622+0000 7fa148447700 1 -- 172.21.15.148:0/2214728126 shutdown_connections 2023-10-10T20:20:01.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.622+0000 7fa148447700 1 -- 172.21.15.148:0/2214728126 wait complete. 2023-10-10T20:20:01.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 Processor -- start 2023-10-10T20:20:01.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 -- start start 2023-10-10T20:20:01.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1400b46f0 0x7fa140147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa140159ef0 0x7fa1400aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400abf90 con 0x7fa1400b6a10 2023-10-10T20:20:01.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400af470 con 0x7fa1400b46f0 2023-10-10T20:20:01.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa148447700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400af750 con 0x7fa140159ef0 2023-10-10T20:20:01.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa1459e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa1459e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59776/0 (socket says 172.21.15.148:59776) 2023-10-10T20:20:01.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa1459e2700 1 -- 172.21.15.148:0/3622805377 learned_addr learned my addr 172.21.15.148:0/3622805377 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:01.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa1469e4700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa140159ef0 0x7fa1400aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.623+0000 7fa1461e3700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1400b46f0 0x7fa140147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 -- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa140159ef0 msgr2=0x7fa1400aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:01.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa140159ef0 0x7fa1400aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 -- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1400b46f0 msgr2=0x7fa140147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1400b46f0 0x7fa140147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 -- 172.21.15.148:0/3622805377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138009ae0 con 0x7fa1400b6a10 2023-10-10T20:20:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1459e2700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c7d0 tx=0x7fa130008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130011600 con 0x7fa1400b6a10 2023-10-10T20:20:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa130011760 con 0x7fa1400b6a10 2023-10-10T20:20:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1400afa30 con 0x7fa1400b6a10 2023-10-10T20:20:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130011b20 con 0x7fa1400b6a10 2023-10-10T20:20:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.624+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1400af130 con 0x7fa1400b6a10 2023-10-10T20:20:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.625+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1400ac100 con 0x7fa1400b6a10 2023-10-10T20:20:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.625+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1300193f0 con 0x7fa1400b6a10 2023-10-10T20:20:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.626+0000 7fa1367fc700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 0x7fa1240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.626+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa130084670 con 0x7fa1400b6a10 2023-10-10T20:20:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.626+0000 7fa1461e3700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 0x7fa1240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.626+0000 7fa1461e3700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 0x7fa1240693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa13c003a70 tx=0x7fa13c007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:01.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.630+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa130051a40 con 0x7fa1400b6a10 2023-10-10T20:20:01.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:01.817+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1400abd30 con 0x7fa1400b6a10 2023-10-10T20:20:02.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.640+0000 7fa1367fc700 1 -- 172.21.15.148:0/3622805377 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa130055060 con 0x7fa1400b6a10 2023-10-10T20:20:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 msgr2=0x7fa1240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 0x7fa1240693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa13c003a70 tx=0x7fa13c007ef0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 msgr2=0x7fa1400ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c7d0 tx=0x7fa130008040 comp rx=0 tx=0).stop 2023-10-10T20:20:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 shutdown_connections 2023-10-10T20:20:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa124066f30 0x7fa1240693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa140159ef0 0x7fa1400aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1400b46f0 0x7fa140147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 --2- 172.21.15.148:0/3622805377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1400b6a10 0x7fa1400ae6b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 >> 172.21.15.148:0/3622805377 conn(0x7fa14009a0a0 msgr2=0x7fa140158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 shutdown_connections 2023-10-10T20:20:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.643+0000 7fa148447700 1 -- 172.21.15.148:0/3622805377 wait complete. 2023-10-10T20:20:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:20:02.658 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:20:02.659 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:20:02.659 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:20:02.659 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:02.659 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:20:02.660 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:20:02.660 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:20:02.660 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:02.661 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:20:02.661 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:20:02.661 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:20:02.661 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:02.662 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:20:02.662 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:20:02.662 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:20:02.662 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:02.668 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:20:02.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:20:02.669 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:20:02.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:20:02.671 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:20:02.671 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:20:02.672 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:20:02.672 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:20:02.711 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:20:02.712 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:02.714 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:20:02.714 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:02.715 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:20:02.717 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:20:02.717 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:02.718 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:02.719 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:20:02.786 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:20:02.793 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:20:02.801 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:20:02.802 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:20:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.849+0000 7f62dcee3700 1 Processor -- start 2023-10-10T20:20:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.849+0000 7f62dcee3700 1 -- start start 2023-10-10T20:20:02.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62dcee3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 0x7f62d80af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62dcee3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 0x7f62d815a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62dcee3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d815d8d0 con 0x7f62d80d15c0 2023-10-10T20:20:02.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62dcee3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d815e170 con 0x7f62d815e600 2023-10-10T20:20:02.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62dcee3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b00c0 con 0x7f62d80d0160 2023-10-10T20:20:02.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d6ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f62d80d15c0 0x7f62d80cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34878/0 (socket says 172.21.15.148:34878) 2023-10-10T20:20:02.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d6ffd700 1 -- 172.21.15.148:0/220464679 learned_addr learned my addr 172.21.15.148:0/220464679 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:02.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d7fff700 1 --2- 172.21.15.148:0/220464679 >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 0x7f62d815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d77fe700 1 --2- 172.21.15.148:0/220464679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 0x7f62d80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441747393 0 0) 0x7f62d815d8d0 con 0x7f62d80d15c0 2023-10-10T20:20:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c0003570 con 0x7f62d80d15c0 2023-10-10T20:20:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1570922469 0 0) 0x7f62c0003570 con 0x7f62d80d15c0 2023-10-10T20:20:02.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d77fe700 1 -- 172.21.15.148:0/220464679 >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 msgr2=0x7f62d815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:02.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.850+0000 7f62d77fe700 1 --2- 172.21.15.148:0/220464679 >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 0x7f62d815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d77fe700 1 -- 172.21.15.148:0/220464679 >> v1:172.21.15.148:6789/0 conn(0x7f62d80d15c0 legacy=0x7f62d80cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d77fe700 1 -- 172.21.15.148:0/220464679 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d815db20 con 0x7f62d80d0160 2023-10-10T20:20:02.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d7fff700 1 --2- 172.21.15.148:0/220464679 >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 0x7f62d815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:02.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d77fe700 1 --2- 172.21.15.148:0/220464679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 0x7f62d80af750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009e60 tx=0x7f62c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910f165d26d06e38 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:02.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800e070 con 0x7f62d80d0160 2023-10-10T20:20:02.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c800b7a0 con 0x7f62d80d0160 2023-10-10T20:20:02.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62d67fc700 1 -- 172.21.15.148:0/220464679 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc001470 con 0x7f62d80d0160 2023-10-10T20:20:02.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 -- 172.21.15.148:0/220464679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 msgr2=0x7f62d80af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 --2- 172.21.15.148:0/220464679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 0x7f62d80af750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009e60 tx=0x7f62c8009780 comp rx=0 tx=0).stop 2023-10-10T20:20:02.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 -- 172.21.15.148:0/220464679 shutdown_connections 2023-10-10T20:20:02.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 --2- 172.21.15.148:0/220464679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d0160 0x7f62d80af750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 --2- 172.21.15.148:0/220464679 >> v2:172.21.15.161:3300/0 conn(0x7f62d815e600 0x7f62d815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 -- 172.21.15.148:0/220464679 >> 172.21.15.148:0/220464679 conn(0x7f62d80695d0 msgr2=0x7f62d80d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:02.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 -- 172.21.15.148:0/220464679 shutdown_connections 2023-10-10T20:20:02.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.851+0000 7f62dcee3700 1 -- 172.21.15.148:0/220464679 wait complete. 2023-10-10T20:20:02.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 Processor -- start 2023-10-10T20:20:02.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 -- start start 2023-10-10T20:20:02.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 0x7f62d80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 0x7f62d80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8151fd0 con 0x7f62d80d0160 2023-10-10T20:20:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b5430 con 0x7f62d815e600 2023-10-10T20:20:02.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.853+0000 7f62dcee3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b4820 con 0x7f62d80d15c0 2023-10-10T20:20:02.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45318/0 (socket says 172.21.15.148:45318) 2023-10-10T20:20:02.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 -- 172.21.15.148:0/2260901395 learned_addr learned my addr 172.21.15.148:0/2260901395 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:02.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d7fff700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 0x7f62d80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d6ffd700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 0x7f62d80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 -- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 msgr2=0x7f62d80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 0x7f62d80b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 -- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 msgr2=0x7f62d80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:02.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 0x7f62d80b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:02.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 -- 172.21.15.148:0/2260901395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0009420 con 0x7f62d80d0160 2023-10-10T20:20:02.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d7fff700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 0x7f62d80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:02.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d6ffd700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 0x7f62d80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:02.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62d77fe700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62d815b450 tx=0x7f62c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:02.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8012af0 con 0x7f62d80d0160 2023-10-10T20:20:02.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8012c70 con 0x7f62d80d0160 2023-10-10T20:20:02.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c8009a20 con 0x7f62d80d0160 2023-10-10T20:20:02.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8027470 con 0x7f62d80d0160 2023-10-10T20:20:02.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.854+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80b4de0 con 0x7f62d80d0160 2023-10-10T20:20:02.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.855+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d804bb80 con 0x7f62d80d0160 2023-10-10T20:20:02.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.856+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62c8024070 con 0x7f62d80d0160 2023-10-10T20:20:02.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.856+0000 7f62bf7fe700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 0x7f62c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:02.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.856+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f62c80866a0 con 0x7f62d80d0160 2023-10-10T20:20:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.856+0000 7f62d6ffd700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 0x7f62c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.856+0000 7f62d6ffd700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 0x7f62c40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001620 tx=0x7f62cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:02.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:02.860+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62c8053c10 con 0x7f62d80d0160 2023-10-10T20:20:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.059+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f62d80ce4f0 con 0x7f62d80d0160 2023-10-10T20:20:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.653+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2260901395 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7f62c8057230 con 0x7f62d80d0160 2023-10-10T20:20:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 msgr2=0x7f62c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 0x7f62c40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001620 tx=0x7f62cc009040 comp rx=0 tx=0).stop 2023-10-10T20:20:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 msgr2=0x7f62d8151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62d815b450 tx=0x7f62c8018000 comp rx=0 tx=0).stop 2023-10-10T20:20:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 shutdown_connections 2023-10-10T20:20:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62c4066f30 0x7f62c40693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d815e600 0x7f62d80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62d80d0160 0x7f62d8151a90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 --2- 172.21.15.148:0/2260901395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62d80d15c0 0x7f62d80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 >> 172.21.15.148:0/2260901395 conn(0x7f62d80695d0 msgr2=0x7f62d80af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 shutdown_connections 2023-10-10T20:20:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.655+0000 7f62dcee3700 1 -- 172.21.15.148:0/2260901395 wait complete. 2023-10-10T20:20:03.660 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:20:03.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:20:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.785+0000 7fee569b1700 1 Processor -- start 2023-10-10T20:20:03.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.785+0000 7fee569b1700 1 -- start start 2023-10-10T20:20:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee569b1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee500b1220 0x7fee500c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee569b1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500cb4f0 0x7fee500cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee569b1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5015de60 con 0x7fee500b0e50 2023-10-10T20:20:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee569b1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee500cbed0 con 0x7fee500b1220 2023-10-10T20:20:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee569b1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015e140 con 0x7fee500cb4f0 2023-10-10T20:20:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee4f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee500b1220 0x7fee500c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee4f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee500b1220 0x7fee500c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45332/0 (socket says 172.21.15.148:45332) 2023-10-10T20:20:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.786+0000 7fee4f7fe700 1 -- 172.21.15.148:0/603314742 learned_addr learned my addr 172.21.15.148:0/603314742 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee54f4e700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500cb4f0 0x7fee500cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4f7fe700 1 -- 172.21.15.148:0/603314742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500cb4f0 msgr2=0x7fee500cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4f7fe700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500cb4f0 0x7fee500cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4f7fe700 1 -- 172.21.15.148:0/603314742 >> v1:172.21.15.161:6789/0 conn(0x7fee500b0e50 legacy=0x7fee5006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4f7fe700 1 -- 172.21.15.148:0/603314742 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5015e420 con 0x7fee500b1220 2023-10-10T20:20:03.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4f7fe700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500c9180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee440099b0 tx=0x7fee44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a8521fdcaf8805 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:03.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4effd700 1 -- 172.21.15.148:0/603314742 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4400b860 con 0x7fee500b1220 2023-10-10T20:20:03.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4effd700 1 -- 172.21.15.148:0/603314742 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400b9c0 con 0x7fee500b1220 2023-10-10T20:20:03.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee4effd700 1 -- 172.21.15.148:0/603314742 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4400bd80 con 0x7fee500b1220 2023-10-10T20:20:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 -- 172.21.15.148:0/603314742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 msgr2=0x7fee500c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500c9180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee440099b0 tx=0x7fee44009250 comp rx=0 tx=0).stop 2023-10-10T20:20:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 -- 172.21.15.148:0/603314742 shutdown_connections 2023-10-10T20:20:03.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500cb4f0 0x7fee500cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 --2- 172.21.15.148:0/603314742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500c9180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 -- 172.21.15.148:0/603314742 >> 172.21.15.148:0/603314742 conn(0x7fee50069870 msgr2=0x7fee50069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:03.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.787+0000 7fee569b1700 1 -- 172.21.15.148:0/603314742 shutdown_connections 2023-10-10T20:20:03.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.788+0000 7fee569b1700 1 -- 172.21.15.148:0/603314742 wait complete. 2023-10-10T20:20:03.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 Processor -- start 2023-10-10T20:20:03.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 -- start start 2023-10-10T20:20:03.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500b0e50 0x7fee500b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500cb4f0 0x7fee50151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5015c1e0 con 0x7fee500b1220 2023-10-10T20:20:03.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee501526e0 con 0x7fee500cb4f0 2023-10-10T20:20:03.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.789+0000 7fee569b1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee50151bd0 con 0x7fee500b0e50 2023-10-10T20:20:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45344/0 (socket says 172.21.15.148:45344) 2023-10-10T20:20:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 -- 172.21.15.148:0/3178901615 learned_addr learned my addr 172.21.15.148:0/3178901615 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:03.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee54f4e700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500cb4f0 0x7fee50151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4ffff700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500b0e50 0x7fee500b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 -- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500b0e50 msgr2=0x7fee500b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500b0e50 0x7fee500b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 -- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500cb4f0 msgr2=0x7fee50151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:03.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500cb4f0 0x7fee50151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:03.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 -- 172.21.15.148:0/3178901615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee440094f0 con 0x7fee500b1220 2023-10-10T20:20:03.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee4f7fe700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b3c0 tx=0x7fee44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:03.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4401f070 con 0x7fee500b1220 2023-10-10T20:20:03.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee44006b10 con 0x7fee500b1220 2023-10-10T20:20:03.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4401a650 con 0x7fee500b1220 2023-10-10T20:20:03.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee50152960 con 0x7fee500b1220 2023-10-10T20:20:03.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.790+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee500b3250 con 0x7fee500b1220 2023-10-10T20:20:03.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.791+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5004c3c0 con 0x7fee500b1220 2023-10-10T20:20:03.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.791+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee4401aa40 con 0x7fee500b1220 2023-10-10T20:20:03.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.792+0000 7fee37fff700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 0x7fee3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:03.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.792+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fee440856a0 con 0x7fee500b1220 2023-10-10T20:20:03.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.792+0000 7fee4ffff700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 0x7fee3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:03.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.792+0000 7fee4ffff700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 0x7fee3c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee400037c0 tx=0x7fee40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:03.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:03.796+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee44052ce0 con 0x7fee500b1220 2023-10-10T20:20:04.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.051+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fee501583a0 con 0x7fee500b1220 2023-10-10T20:20:04.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.051+0000 7fee37fff700 1 -- 172.21.15.148:0/3178901615 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fee44015090 con 0x7fee500b1220 2023-10-10T20:20:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:04.053 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:20:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.053 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:20:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:20:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:04.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.055 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:20:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:20:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.056 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:20:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:20:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.058 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:20:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:20:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.060 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:04.061 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:20:04.062 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:20:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.064 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.065 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.065 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.066 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:20:04.066 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:20:04.066 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.066 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.066 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.067 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.067 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:20:04.068 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:20:04.068 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.068 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.068 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.068 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.069 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:20:04.070 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:20:04.070 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.070 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.071 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.072 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.072 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.073 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:20:04.074 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:20:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:20:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:20:04.076 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:20:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:20:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.078 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:20:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:20:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.080 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:20:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:20:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.082 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:04.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:20:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:20:04.084 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:04.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:20:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:20:04.086 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mds", 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlbf3yKhAAJa2ChBVQnHvzcOPlLkOK4g==", 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.088 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mds" 2023-10-10T20:20:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mgr", 2023-10-10T20:20:04.089 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlyQfzKhAAzaImcUj9kHv77bsV+EN24A==", 2023-10-10T20:20:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-10T20:20:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-osd", 2023-10-10T20:20:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlmBDzKhAALkQhlbKU8MOGb6rP6UpBRQ==", 2023-10-10T20:20:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-osd" 2023-10-10T20:20:04.091 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd", 2023-10-10T20:20:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlBxjzKhAAL3Iui3XEkOjMOgnC6s81ig==", 2023-10-10T20:20:04.093 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-10T20:20:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.094 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-10T20:20:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlgB/zKhAA7TeXXhL2C9mdkaZZU/hfQA==", 2023-10-10T20:20:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-10T20:20:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rgw", 2023-10-10T20:20:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC5sSVlNSfzKhAAVqv1e2M94Br/WYEod7nmGw==", 2023-10-10T20:20:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.096 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:20:04.097 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:20:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:20:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:20:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:04.099 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:20:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:04.100 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:04.101 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:04.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 msgr2=0x7fee3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 0x7fee3c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee400037c0 tx=0x7fee40009040 comp rx=0 tx=0).stop 2023-10-10T20:20:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 msgr2=0x7fee500b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b3c0 tx=0x7fee44017000 comp rx=0 tx=0).stop 2023-10-10T20:20:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 shutdown_connections 2023-10-10T20:20:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee3c066f30 0x7fee3c0693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee500cb4f0 0x7fee50151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee500b0e50 0x7fee500b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 --2- 172.21.15.148:0/3178901615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee500b1220 0x7fee500b28b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.053+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 >> 172.21.15.148:0/3178901615 conn(0x7fee50069870 msgr2=0x7fee50158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:04.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.054+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 shutdown_connections 2023-10-10T20:20:04.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.054+0000 7fee569b1700 1 -- 172.21.15.148:0/3178901615 wait complete. 2023-10-10T20:20:04.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-10T20:20:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.220+0000 7f1dd96a2700 1 Processor -- start 2023-10-10T20:20:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.220+0000 7f1dd96a2700 1 -- start start 2023-10-10T20:20:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd96a2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1dd4159d50 0x7f1dd40cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd96a2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 0x7f1dd40af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd96a2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd4069d90 con 0x7f1dd415a120 2023-10-10T20:20:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd96a2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd40ccdd0 con 0x7f1dd4159d50 2023-10-10T20:20:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd96a2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4069740 con 0x7f1dd406daa0 2023-10-10T20:20:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd27fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1dd415a120 0x7f1dd40aee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34884/0 (socket says 172.21.15.148:34884) 2023-10-10T20:20:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd27fc700 1 -- 172.21.15.148:0/3405133348 learned_addr learned my addr 172.21.15.148:0/3405133348 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3405133348 >> v2:172.21.15.161:3300/0 conn(0x7f1dd4159d50 0x7f1dd40cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.221+0000 7f1dd37fe700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 0x7f1dd40af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd1ffb700 1 -- 172.21.15.148:0/3405133348 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239747731 0 0) 0x7f1dd4069d90 con 0x7f1dd415a120 2023-10-10T20:20:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd1ffb700 1 -- 172.21.15.148:0/3405133348 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc0003570 con 0x7f1dd415a120 2023-10-10T20:20:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd2ffd700 1 -- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 msgr2=0x7f1dd40af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 0x7f1dd40af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd2ffd700 1 -- 172.21.15.148:0/3405133348 >> v1:172.21.15.148:6789/0 conn(0x7f1dd415a120 legacy=0x7f1dd40aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd2ffd700 1 -- 172.21.15.148:0/3405133348 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4069a20 con 0x7f1dd4159d50 2023-10-10T20:20:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd37fe700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 0x7f1dd40af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd4159d50 0x7f1dd40cc7f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e60 tx=0x7f1dc4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770975bb3ffcbd0c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd1ffb700 1 -- 172.21.15.148:0/3405133348 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400e070 con 0x7f1dd4159d50 2023-10-10T20:20:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd1ffb700 1 -- 172.21.15.148:0/3405133348 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc400bab0 con 0x7f1dd4159d50 2023-10-10T20:20:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.222+0000 7f1dd1ffb700 1 -- 172.21.15.148:0/3405133348 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400be70 con 0x7f1dd4159d50 2023-10-10T20:20:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd4159d50 msgr2=0x7f1dd40cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd4159d50 0x7f1dd40cc7f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e60 tx=0x7f1dc4009780 comp rx=0 tx=0).stop 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3405133348 shutdown_connections 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd406daa0 0x7f1dd40af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3405133348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd4159d50 0x7f1dd40cc7f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3405133348 >> 172.21.15.148:0/3405133348 conn(0x7f1dd40cd200 msgr2=0x7f1dd40cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3405133348 shutdown_connections 2023-10-10T20:20:04.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.223+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3405133348 wait complete. 2023-10-10T20:20:04.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.224+0000 7f1dd96a2700 1 Processor -- start 2023-10-10T20:20:04.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.224+0000 7f1dd96a2700 1 -- start start 2023-10-10T20:20:04.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd406daa0 0x7f1dd409a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd4159d50 0x7f1dd40c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd415d390 con 0x7f1dd415a120 2023-10-10T20:20:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd415cf00 con 0x7f1dd406daa0 2023-10-10T20:20:04.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd409a9f0 con 0x7f1dd4159d50 2023-10-10T20:20:04.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45350/0 (socket says 172.21.15.148:45350) 2023-10-10T20:20:04.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 -- 172.21.15.148:0/3194918001 learned_addr learned my addr 172.21.15.148:0/3194918001 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:04.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd27fc700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd4159d50 0x7f1dd40c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd406daa0 0x7f1dd409a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 -- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd4159d50 msgr2=0x7f1dd40c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd4159d50 0x7f1dd40c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 -- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd406daa0 msgr2=0x7f1dd409a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd406daa0 0x7f1dd409a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 -- 172.21.15.148:0/3194918001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009420 con 0x7f1dd415a120 2023-10-10T20:20:04.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd37fe700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00ecb0 tx=0x7f1dcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00c850 con 0x7f1dd415a120 2023-10-10T20:20:04.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00c9b0 con 0x7f1dd415a120 2023-10-10T20:20:04.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4009a20 con 0x7f1dd415a120 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00cd70 con 0x7f1dd415a120 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.225+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd415c5d0 con 0x7f1dd415a120 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.226+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd415d090 con 0x7f1dd415a120 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.227+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dcc013680 con 0x7f1dd415a120 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.227+0000 7f1dcaffd700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.227+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1dcc0841a0 con 0x7f1dd415a120 2023-10-10T20:20:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.227+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.228+0000 7f1dd2ffd700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40097c0 tx=0x7f1dc4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.232+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dcc051760 con 0x7f1dd415a120 2023-10-10T20:20:04.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.486+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1dd404c3c0 con 0x7f1dd415a120 2023-10-10T20:20:04.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.490+0000 7f1dcaffd700 1 -- 172.21.15.148:0/3194918001 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1dcc054d80 con 0x7f1dd415a120 2023-10-10T20:20:04.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 msgr2=0x7f1db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40097c0 tx=0x7f1dc4019000 comp rx=0 tx=0).stop 2023-10-10T20:20:04.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 msgr2=0x7f1dd40cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00ecb0 tx=0x7f1dcc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 shutdown_connections 2023-10-10T20:20:04.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dd415a120 0x7f1dd40cad30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd406daa0 0x7f1dd409a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 --2- 172.21.15.148:0/3194918001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dd4159d50 0x7f1dd40c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 >> 172.21.15.148:0/3194918001 conn(0x7f1dd40cd200 msgr2=0x7f1dd4154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 shutdown_connections 2023-10-10T20:20:04.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.492+0000 7f1dd96a2700 1 -- 172.21.15.148:0/3194918001 wait complete. 2023-10-10T20:20:04.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-10T20:20:04.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.620+0000 7f9f0cefb700 1 Processor -- start 2023-10-10T20:20:04.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.620+0000 7f9f0cefb700 1 -- start start 2023-10-10T20:20:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f0cefb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f08159da0 0x7f9f080b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f0cefb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0815ea70 con 0x7f9f0815a170 2023-10-10T20:20:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f0cefb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080b3990 con 0x7f9f08069210 2023-10-10T20:20:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f0cefb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080b0b70 con 0x7f9f08159da0 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f077fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f08159da0 0x7f9f080b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f06ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9f0815a170 0x7f9f08068b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34886/0 (socket says 172.21.15.148:34886) 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f06ffd700 1 -- 172.21.15.148:0/4253505720 learned_addr learned my addr 172.21.15.148:0/4253505720 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f077fe700 1 -- 172.21.15.148:0/4253505720 >> v1:172.21.15.161:6789/0 conn(0x7f9f08069210 legacy=0x7f9f080af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f077fe700 1 -- 172.21.15.148:0/4253505720 >> v1:172.21.15.148:6789/0 conn(0x7f9f0815a170 legacy=0x7f9f08068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.621+0000 7f9f077fe700 1 -- 172.21.15.148:0/4253505720 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0815def0 con 0x7f9f08159da0 2023-10-10T20:20:04.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f077fe700 1 --2- 172.21.15.148:0/4253505720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f080b2da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009e60 tx=0x7f9ef8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8eff9b2425e727 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f067fc700 1 -- 172.21.15.148:0/4253505720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800e070 con 0x7f9f08159da0 2023-10-10T20:20:04.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f067fc700 1 -- 172.21.15.148:0/4253505720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800bc30 con 0x7f9f08159da0 2023-10-10T20:20:04.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f067fc700 1 -- 172.21.15.148:0/4253505720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8012430 con 0x7f9f08159da0 2023-10-10T20:20:04.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4253505720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 msgr2=0x7f9f080b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4253505720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f080b2da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009e60 tx=0x7f9ef8009780 comp rx=0 tx=0).stop 2023-10-10T20:20:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4253505720 shutdown_connections 2023-10-10T20:20:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4253505720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f080b2da0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4253505720 >> 172.21.15.148:0/4253505720 conn(0x7f9f080d0bc0 msgr2=0x7f9f080d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:04.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4253505720 shutdown_connections 2023-10-10T20:20:04.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.622+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4253505720 wait complete. 2023-10-10T20:20:04.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.623+0000 7f9f0cefb700 1 Processor -- start 2023-10-10T20:20:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 -- start start 2023-10-10T20:20:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 0x7f9f080c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f0815a170 0x7f9f080b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08155a10 con 0x7f9f08159da0 2023-10-10T20:20:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080c4620 con 0x7f9f08069210 2023-10-10T20:20:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f0cefb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080c7750 con 0x7f9f0815a170 2023-10-10T20:20:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f06ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f06ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45374/0 (socket says 172.21.15.148:45374) 2023-10-10T20:20:04.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.624+0000 7f9f06ffd700 1 -- 172.21.15.148:0/4191357485 learned_addr learned my addr 172.21.15.148:0/4191357485 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:04.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f07fff700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f0815a170 0x7f9f080b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f077fe700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 0x7f9f080c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 -- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f0815a170 msgr2=0x7f9f080b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f0815a170 0x7f9f080b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 -- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 msgr2=0x7f9f080c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 0x7f9f080c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 -- 172.21.15.148:0/4191357485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8009a20 con 0x7f9f08159da0 2023-10-10T20:20:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f077fe700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 0x7f9f080c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:04.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f06ffd700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c560 tx=0x7f9efc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc005b30 con 0x7f9f08159da0 2023-10-10T20:20:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc005cb0 con 0x7f9f08159da0 2023-10-10T20:20:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc007320 con 0x7f9f08159da0 2023-10-10T20:20:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f080c4870 con 0x7f9f08159da0 2023-10-10T20:20:04.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.625+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f080c4e30 con 0x7f9f08159da0 2023-10-10T20:20:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.626+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08089480 con 0x7f9f08159da0 2023-10-10T20:20:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.626+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9efc01b3b0 con 0x7f9f08159da0 2023-10-10T20:20:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.627+0000 7f9eef7fe700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 0x7f9ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.627+0000 7f9f077fe700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 0x7f9ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:04.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.627+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9efc083720 con 0x7f9f08159da0 2023-10-10T20:20:04.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.627+0000 7f9f077fe700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 0x7f9ef40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099a0 tx=0x7f9ef8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:04.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.631+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9efc050ce0 con 0x7f9f08159da0 2023-10-10T20:20:04.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.892+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9f0815e1d0 con 0x7f9f08159da0 2023-10-10T20:20:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.898+0000 7f9eef7fe700 1 -- 172.21.15.148:0/4191357485 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9efc054300 con 0x7f9f08159da0 2023-10-10T20:20:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 msgr2=0x7f9ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 0x7f9ef40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80099a0 tx=0x7f9ef8018000 comp rx=0 tx=0).stop 2023-10-10T20:20:04.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 msgr2=0x7f9f081554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:04.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c560 tx=0x7f9efc001020 comp rx=0 tx=0).stop 2023-10-10T20:20:04.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 shutdown_connections 2023-10-10T20:20:04.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ef4066f30 0x7f9ef40693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f0815a170 0x7f9f080b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f08069210 0x7f9f080c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 --2- 172.21.15.148:0/4191357485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f08159da0 0x7f9f081554d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 >> 172.21.15.148:0/4191357485 conn(0x7f9f080d0bc0 msgr2=0x7f9f080d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 shutdown_connections 2023-10-10T20:20:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:04.900+0000 7f9f0cefb700 1 -- 172.21.15.148:0/4191357485 wait complete. 2023-10-10T20:20:04.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-10T20:20:05.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.028+0000 7f470b418700 1 Processor -- start 2023-10-10T20:20:05.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.028+0000 7f470b418700 1 -- start start 2023-10-10T20:20:05.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f470b418700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f470415a620 0x7f470415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f470b418700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f470b418700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470415cbb0 con 0x7f4704157870 2023-10-10T20:20:05.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f470b418700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470415bdb0 con 0x7f470415a620 2023-10-10T20:20:05.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f470b418700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470415bfa0 con 0x7f470415d580 2023-10-10T20:20:05.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f47089b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f470415a620 0x7f470415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f47089b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f470415a620 0x7f470415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45382/0 (socket says 172.21.15.148:45382) 2023-10-10T20:20:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f47089b3700 1 -- 172.21.15.148:0/4248045868 learned_addr learned my addr 172.21.15.148:0/4248045868 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.029+0000 7f47099b5700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47089b3700 1 -- 172.21.15.148:0/4248045868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 msgr2=0x7f4704158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47089b3700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47089b3700 1 -- 172.21.15.148:0/4248045868 >> v1:172.21.15.161:6789/0 conn(0x7f4704157870 legacy=0x7f470415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47089b3700 1 -- 172.21.15.148:0/4248045868 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470415c280 con 0x7f470415a620 2023-10-10T20:20:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47099b5700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f47089b3700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470415a620 0x7f470415d040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46f40099b0 tx=0x7f46f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7b5cad08181d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f46fbfff700 1 -- 172.21.15.148:0/4248045868 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400b860 con 0x7f470415a620 2023-10-10T20:20:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f46fbfff700 1 -- 172.21.15.148:0/4248045868 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f400b9c0 con 0x7f470415a620 2023-10-10T20:20:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f46fbfff700 1 -- 172.21.15.148:0/4248045868 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400bd60 con 0x7f470415a620 2023-10-10T20:20:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 -- 172.21.15.148:0/4248045868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470415a620 msgr2=0x7f470415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470415a620 0x7f470415d040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46f40099b0 tx=0x7f46f4009250 comp rx=0 tx=0).stop 2023-10-10T20:20:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 -- 172.21.15.148:0/4248045868 shutdown_connections 2023-10-10T20:20:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 --2- 172.21.15.148:0/4248045868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470415a620 0x7f470415d040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.030+0000 7f470b418700 1 -- 172.21.15.148:0/4248045868 >> 172.21.15.148:0/4248045868 conn(0x7f4704068ce0 msgr2=0x7f47040690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.031+0000 7f470b418700 1 -- 172.21.15.148:0/4248045868 shutdown_connections 2023-10-10T20:20:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.031+0000 7f470b418700 1 -- 172.21.15.148:0/4248045868 wait complete. 2023-10-10T20:20:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 Processor -- start 2023-10-10T20:20:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 -- start start 2023-10-10T20:20:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 0x7f47041548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470414dfb0 con 0x7f4704157870 2023-10-10T20:20:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470414d680 con 0x7f470415a620 2023-10-10T20:20:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.032+0000 7f470b418700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470414d960 con 0x7f470415d580 2023-10-10T20:20:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45388/0 (socket says 172.21.15.148:45388) 2023-10-10T20:20:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 -- 172.21.15.148:0/3790673309 learned_addr learned my addr 172.21.15.148:0/3790673309 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47099b5700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47089b3700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 0x7f47041548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 -- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 msgr2=0x7f4704154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 -- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 msgr2=0x7f47041548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 0x7f47041548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 -- 172.21.15.148:0/3790673309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40094f0 con 0x7f4704157870 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47089b3700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 0x7f47041548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f47091b4700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f470000c590 tx=0x7f470000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700011590 con 0x7f4704157870 2023-10-10T20:20:05.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4700011710 con 0x7f4704157870 2023-10-10T20:20:05.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470414dc40 con 0x7f4704157870 2023-10-10T20:20:05.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700011ad0 con 0x7f4704157870 2023-10-10T20:20:05.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.033+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47040d1480 con 0x7f4704157870 2023-10-10T20:20:05.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.034+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470414e340 con 0x7f4704157870 2023-10-10T20:20:05.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.035+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f470001b3b0 con 0x7f4704157870 2023-10-10T20:20:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.035+0000 7f46f97fa700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.035+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f47000837a0 con 0x7f4704157870 2023-10-10T20:20:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.035+0000 7f47089b3700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.035+0000 7f47089b3700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 0x7f46e80693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f40099b0 tx=0x7f46f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.039+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f470414e340 con 0x7f4704157870 2023-10-10T20:20:05.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.296+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f470414ddd0 con 0x7f4704157870 2023-10-10T20:20:05.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.302+0000 7f46f97fa700 1 -- 172.21.15.148:0/3790673309 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4700017020 con 0x7f4704157870 2023-10-10T20:20:05.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 msgr2=0x7f46e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 0x7f46e80693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f46f40099b0 tx=0x7f46f4017000 comp rx=0 tx=0).stop 2023-10-10T20:20:05.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 msgr2=0x7f47040d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f470000c590 tx=0x7f470000c8a0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 shutdown_connections 2023-10-10T20:20:05.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46e8066f30 0x7f46e80693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f470415d580 0x7f4704154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4704157870 0x7f47040d0f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 --2- 172.21.15.148:0/3790673309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f470415a620 0x7f47041548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 >> 172.21.15.148:0/3790673309 conn(0x7f4704068ce0 msgr2=0x7f470414fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:05.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 shutdown_connections 2023-10-10T20:20:05.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.304+0000 7f470b418700 1 -- 172.21.15.148:0/3790673309 wait complete. 2023-10-10T20:20:05.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-10T20:20:05.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.431+0000 7f4bd0a35700 1 Processor -- start 2023-10-10T20:20:05.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.431+0000 7f4bd0a35700 1 -- start start 2023-10-10T20:20:05.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bd0a35700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bcc157870 0x7f4bcc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bd0a35700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4bcc157c40 0x7f4bcc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bd0a35700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc15cbb0 con 0x7f4bcc15a620 2023-10-10T20:20:05.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bd0a35700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc15bfa0 con 0x7f4bcc157870 2023-10-10T20:20:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bd0a35700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc15c280 con 0x7f4bcc157c40 2023-10-10T20:20:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bca59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bcc157870 0x7f4bcc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bc1d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bcc15a620 0x7f4bcc15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34888/0 (socket says 172.21.15.148:34888) 2023-10-10T20:20:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bc1d9b700 1 -- 172.21.15.148:0/979268540 learned_addr learned my addr 172.21.15.148:0/979268540 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bcad9d700 1 --2- 172.21.15.148:0/979268540 >> v2:172.21.15.161:3300/0 conn(0x7f4bcc157c40 0x7f4bcc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bca59c700 1 -- 172.21.15.148:0/979268540 >> v2:172.21.15.161:3300/0 conn(0x7f4bcc157c40 msgr2=0x7f4bcc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:05.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bca59c700 1 --2- 172.21.15.148:0/979268540 >> v2:172.21.15.161:3300/0 conn(0x7f4bcc157c40 0x7f4bcc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bca59c700 1 -- 172.21.15.148:0/979268540 >> v1:172.21.15.148:6789/0 conn(0x7f4bcc15a620 legacy=0x7f4bcc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bca59c700 1 -- 172.21.15.148:0/979268540 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc15c560 con 0x7f4bcc157870 2023-10-10T20:20:05.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.432+0000 7f4bc9d9b700 1 -- 172.21.15.148:0/979268540 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787397934 0 0) 0x7f4bcc15cbb0 con 0x7f4bcc15a620 2023-10-10T20:20:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bca59c700 1 --2- 172.21.15.148:0/979268540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc157870 0x7f4bcc15b6a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc06b4e0 tx=0x7f4bbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b97670f73821d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bc9d9b700 1 -- 172.21.15.148:0/979268540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00b6b0 con 0x7f4bcc157870 2023-10-10T20:20:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bc9d9b700 1 -- 172.21.15.148:0/979268540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc00b810 con 0x7f4bcc157870 2023-10-10T20:20:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bc9d9b700 1 -- 172.21.15.148:0/979268540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00bbb0 con 0x7f4bcc157870 2023-10-10T20:20:05.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 -- 172.21.15.148:0/979268540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc157870 msgr2=0x7f4bcc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/979268540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc157870 0x7f4bcc15b6a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc06b4e0 tx=0x7f4bbc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 -- 172.21.15.148:0/979268540 shutdown_connections 2023-10-10T20:20:05.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/979268540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc157870 0x7f4bcc15b6a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/979268540 >> v2:172.21.15.161:3300/0 conn(0x7f4bcc157c40 0x7f4bcc158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 -- 172.21.15.148:0/979268540 >> 172.21.15.148:0/979268540 conn(0x7f4bcc068ce0 msgr2=0x7f4bcc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:05.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 -- 172.21.15.148:0/979268540 shutdown_connections 2023-10-10T20:20:05.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.433+0000 7f4bd0a35700 1 -- 172.21.15.148:0/979268540 wait complete. 2023-10-10T20:20:05.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.434+0000 7f4bd0a35700 1 Processor -- start 2023-10-10T20:20:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 -- start start 2023-10-10T20:20:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4bcc157870 0x7f4bcc1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bcc157c40 0x7f4bcc15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc15e770 con 0x7f4bcc15a620 2023-10-10T20:20:05.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc15db60 con 0x7f4bcc157c40 2023-10-10T20:20:05.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bd0a35700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc15de40 con 0x7f4bcc157870 2023-10-10T20:20:05.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bcad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bcad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45406/0 (socket says 172.21.15.148:45406) 2023-10-10T20:20:05.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bcad9d700 1 -- 172.21.15.148:0/270811781 learned_addr learned my addr 172.21.15.148:0/270811781 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:05.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bca59c700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4bcc157870 0x7f4bcc1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.435+0000 7f4bc1d9b700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bcc157c40 0x7f4bcc15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 -- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4bcc157870 msgr2=0x7f4bcc1489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4bcc157870 0x7f4bcc1489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 -- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bcc157c40 msgr2=0x7f4bcc15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bcc157c40 0x7f4bcc15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 -- 172.21.15.148:0/270811781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009420 con 0x7f4bcc15a620 2023-10-10T20:20:05.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bcad9d700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007f30 tx=0x7f4bc40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400eb90 con 0x7f4bcc15a620 2023-10-10T20:20:05.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc400ecf0 con 0x7f4bcc15a620 2023-10-10T20:20:05.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc15e120 con 0x7f4bcc15a620 2023-10-10T20:20:05.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4014670 con 0x7f4bcc15a620 2023-10-10T20:20:05.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.436+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc15ec00 con 0x7f4bcc15a620 2023-10-10T20:20:05.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.437+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc15e8e0 con 0x7f4bcc15a620 2023-10-10T20:20:05.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.437+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bc4014a20 con 0x7f4bcc15a620 2023-10-10T20:20:05.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.437+0000 7f4bc2ffd700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 0x7f4bac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.438+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4bc40848a0 con 0x7f4bcc15a620 2023-10-10T20:20:05.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.438+0000 7f4bca59c700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 0x7f4bac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.438+0000 7f4bca59c700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 0x7f4bac0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0093f0 tx=0x7f4bbc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.442+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bc4051dc0 con 0x7f4bcc15a620 2023-10-10T20:20:05.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.702+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f4bcc04c3c0 con 0x7f4bcc15a620 2023-10-10T20:20:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.705+0000 7f4bc2ffd700 1 -- 172.21.15.148:0/270811781 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4bc400ee60 con 0x7f4bcc15a620 2023-10-10T20:20:05.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 msgr2=0x7f4bac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 0x7f4bac0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0093f0 tx=0x7f4bbc017000 comp rx=0 tx=0).stop 2023-10-10T20:20:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 msgr2=0x7f4bcc0c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007f30 tx=0x7f4bc40098e0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 shutdown_connections 2023-10-10T20:20:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4bac066f30 0x7f4bac0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bcc15a620 0x7f4bcc0c66e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4bcc157870 0x7f4bcc1489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 --2- 172.21.15.148:0/270811781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bcc157c40 0x7f4bcc15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 >> 172.21.15.148:0/270811781 conn(0x7f4bcc068ce0 msgr2=0x7f4bcc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 shutdown_connections 2023-10-10T20:20:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.707+0000 7f4bd0a35700 1 -- 172.21.15.148:0/270811781 wait complete. 2023-10-10T20:20:05.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-10T20:20:05.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.834+0000 7f39e5b9a700 1 Processor -- start 2023-10-10T20:20:05.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.834+0000 7f39e5b9a700 1 -- start start 2023-10-10T20:20:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5b9a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39e0159a20 0x7f39e00abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5b9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5b9a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00cb3a0 con 0x7f39e015d8f0 2023-10-10T20:20:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5b9a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e015e770 con 0x7f39e0159a20 2023-10-10T20:20:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5b9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00aba00 con 0x7f39e00cabc0 2023-10-10T20:20:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e4b98700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39e0159a20 0x7f39e00abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5399700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f39e015d8f0 0x7f39e0069300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34896/0 (socket says 172.21.15.148:34896) 2023-10-10T20:20:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e5399700 1 -- 172.21.15.148:0/2452103212 learned_addr learned my addr 172.21.15.148:0/2452103212 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39d7fff700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e4b98700 1 -- 172.21.15.148:0/2452103212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 msgr2=0x7f39e0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e4b98700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e4b98700 1 -- 172.21.15.148:0/2452103212 >> v1:172.21.15.148:6789/0 conn(0x7f39e015d8f0 legacy=0x7f39e0069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39e4b98700 1 -- 172.21.15.148:0/2452103212 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00cb0c0 con 0x7f39e0159a20 2023-10-10T20:20:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.835+0000 7f39d77fe700 1 -- 172.21.15.148:0/2452103212 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314257584 0 0) 0x7f39e00cb3a0 con 0x7f39e015d8f0 2023-10-10T20:20:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e4b98700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00abe90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009e60 tx=0x7f39dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018f14d8ab213b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39d77fe700 1 -- 172.21.15.148:0/2452103212 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00e070 con 0x7f39e0159a20 2023-10-10T20:20:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39d77fe700 1 -- 172.21.15.148:0/2452103212 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc00bb60 con 0x7f39e0159a20 2023-10-10T20:20:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39d77fe700 1 -- 172.21.15.148:0/2452103212 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc012430 con 0x7f39e0159a20 2023-10-10T20:20:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 -- 172.21.15.148:0/2452103212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 msgr2=0x7f39e00abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00abe90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009e60 tx=0x7f39dc009780 comp rx=0 tx=0).stop 2023-10-10T20:20:05.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 -- 172.21.15.148:0/2452103212 shutdown_connections 2023-10-10T20:20:05.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00abe90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/2452103212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 -- 172.21.15.148:0/2452103212 >> 172.21.15.148:0/2452103212 conn(0x7f39e00ccff0 msgr2=0x7f39e00cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:05.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 -- 172.21.15.148:0/2452103212 shutdown_connections 2023-10-10T20:20:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.836+0000 7f39e5b9a700 1 -- 172.21.15.148:0/2452103212 wait complete. 2023-10-10T20:20:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.837+0000 7f39e5b9a700 1 Processor -- start 2023-10-10T20:20:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 -- start start 2023-10-10T20:20:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e016ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e015d8f0 0x7f39e00ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e016b3a0 con 0x7f39e0159a20 2023-10-10T20:20:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00aea00 con 0x7f39e015d8f0 2023-10-10T20:20:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39e5b9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00acb40 con 0x7f39e00cabc0 2023-10-10T20:20:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39d7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39d7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45416/0 (socket says 172.21.15.148:45416) 2023-10-10T20:20:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.838+0000 7f39d7fff700 1 -- 172.21.15.148:0/102059640 learned_addr learned my addr 172.21.15.148:0/102059640 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39e5399700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e015d8f0 0x7f39e00ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39e4b98700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e016ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 -- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 msgr2=0x7f39e016ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e016ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 -- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e015d8f0 msgr2=0x7f39e00ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e015d8f0 0x7f39e00ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 -- 172.21.15.148:0/102059640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc009a20 con 0x7f39e0159a20 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d7fff700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ec80 tx=0x7f39d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000c9d0 con 0x7f39e0159a20 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d000cb30 con 0x7f39e0159a20 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e00b0d60 con 0x7f39e0159a20 2023-10-10T20:20:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0013470 con 0x7f39e0159a20 2023-10-10T20:20:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.839+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e00aecc0 con 0x7f39e0159a20 2023-10-10T20:20:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.840+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e00b0f80 con 0x7f39e0159a20 2023-10-10T20:20:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.840+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39d0013860 con 0x7f39e0159a20 2023-10-10T20:20:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.841+0000 7f39d4ff9700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 0x7f39cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:05.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.841+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f39d0084530 con 0x7f39e0159a20 2023-10-10T20:20:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.841+0000 7f39e4b98700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 0x7f39cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.841+0000 7f39e4b98700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 0x7f39cc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097c0 tx=0x7f39dc019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:05.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:05.845+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39e00b0f80 con 0x7f39e0159a20 2023-10-10T20:20:06.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.102+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f39e00ab3b0 con 0x7f39e0159a20 2023-10-10T20:20:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.105+0000 7f39d4ff9700 1 -- 172.21.15.148:0/102059640 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f39d0051af0 con 0x7f39e0159a20 2023-10-10T20:20:06.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.107+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 msgr2=0x7f39cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.107+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 0x7f39cc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097c0 tx=0x7f39dc019000 comp rx=0 tx=0).stop 2023-10-10T20:20:06.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.107+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 msgr2=0x7f39e00ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.107+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ec80 tx=0x7f39d00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 shutdown_connections 2023-10-10T20:20:06.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f39cc066f30 0x7f39cc0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e015d8f0 0x7f39e00ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39e00cabc0 0x7f39e016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 --2- 172.21.15.148:0/102059640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39e0159a20 0x7f39e00ae4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 >> 172.21.15.148:0/102059640 conn(0x7f39e00ccff0 msgr2=0x7f39e0156730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 shutdown_connections 2023-10-10T20:20:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.108+0000 7f39e5b9a700 1 -- 172.21.15.148:0/102059640 wait complete. 2023-10-10T20:20:06.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-10T20:20:06.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.235+0000 7f8a6a6cf700 1 Processor -- start 2023-10-10T20:20:06.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.235+0000 7f8a6a6cf700 1 -- start start 2023-10-10T20:20:06.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a6a6cf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a640d1c30 0x7f8a640ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a6a6cf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a640cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a6a6cf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a640c5d80 con 0x7f8a6415a2b0 2023-10-10T20:20:06.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a6a6cf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a64068470 con 0x7f8a640d1c30 2023-10-10T20:20:06.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a6a6cf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c6060 con 0x7f8a6415a680 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a640d1c30 0x7f8a640ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a63fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8a6415a2b0 0x7f8a640b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34912/0 (socket says 172.21.15.148:34912) 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a640d1c30 0x7f8a640ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45432/0 (socket says 172.21.15.148:45432) 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a63fff700 1 -- 172.21.15.148:0/373743736 learned_addr learned my addr 172.21.15.148:0/373743736 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a68c6c700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a640cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 -- 172.21.15.148:0/373743736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 msgr2=0x7f8a640cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:06.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a640cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 -- 172.21.15.148:0/373743736 >> v1:172.21.15.148:6789/0 conn(0x7f8a6415a2b0 legacy=0x7f8a640b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a62ffd700 1 -- 172.21.15.148:0/373743736 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063591939 0 0) 0x7f8a640c5d80 con 0x7f8a6415a2b0 2023-10-10T20:20:06.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 -- 172.21.15.148:0/373743736 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640c6340 con 0x7f8a640d1c30 2023-10-10T20:20:06.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a68c6c700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a640cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:06.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.236+0000 7f8a637fe700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a640ccdd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a580098e0 tx=0x7f8a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60949b8df6bc264 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a62ffd700 1 -- 172.21.15.148:0/373743736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58009e80 con 0x7f8a640d1c30 2023-10-10T20:20:06.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a62ffd700 1 -- 172.21.15.148:0/373743736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5800b860 con 0x7f8a640d1c30 2023-10-10T20:20:06.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a62ffd700 1 -- 172.21.15.148:0/373743736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800bbe0 con 0x7f8a640d1c30 2023-10-10T20:20:06.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/373743736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 msgr2=0x7f8a640ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a640ccdd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a580098e0 tx=0x7f8a580091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/373743736 shutdown_connections 2023-10-10T20:20:06.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a640cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/373743736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a640ccdd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/373743736 >> 172.21.15.148:0/373743736 conn(0x7f8a64067e40 msgr2=0x7f8a640d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/373743736 shutdown_connections 2023-10-10T20:20:06.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.237+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/373743736 wait complete. 2023-10-10T20:20:06.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.238+0000 7f8a6a6cf700 1 Processor -- start 2023-10-10T20:20:06.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.238+0000 7f8a6a6cf700 1 -- start start 2023-10-10T20:20:06.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 0x7f8a64155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a64158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6416e3f0 con 0x7f8a640d1c30 2023-10-10T20:20:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a64155ec0 con 0x7f8a6415a2b0 2023-10-10T20:20:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a6a6cf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6415ddc0 con 0x7f8a6415a680 2023-10-10T20:20:06.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a63fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a63fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45444/0 (socket says 172.21.15.148:45444) 2023-10-10T20:20:06.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a63fff700 1 -- 172.21.15.148:0/1031180706 learned_addr learned my addr 172.21.15.148:0/1031180706 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:06.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a68c6c700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a64158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.239+0000 7f8a637fe700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 0x7f8a64155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 -- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 msgr2=0x7f8a64158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a64158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 -- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 msgr2=0x7f8a64155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 0x7f8a64155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 -- 172.21.15.148:0/1031180706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58009420 con 0x7f8a640d1c30 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a637fe700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 0x7f8a64155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a63fff700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c5e0 tx=0x7f8a54007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540072e0 con 0x7f8a640d1c30 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a54007440 con 0x7f8a640d1c30 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6416d630 con 0x7f8a640d1c30 2023-10-10T20:20:06.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540057f0 con 0x7f8a640d1c30 2023-10-10T20:20:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.240+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6416dbf0 con 0x7f8a640d1c30 2023-10-10T20:20:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.241+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6404c3c0 con 0x7f8a640d1c30 2023-10-10T20:20:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.241+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a54005bc0 con 0x7f8a640d1c30 2023-10-10T20:20:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.241+0000 7f8a4bfff700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 0x7f8a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.242+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8a54083770 con 0x7f8a640d1c30 2023-10-10T20:20:06.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.242+0000 7f8a637fe700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 0x7f8a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.242+0000 7f8a637fe700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 0x7f8a500693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d2000 tx=0x7f8a58017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.246+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a54050d30 con 0x7f8a640d1c30 2023-10-10T20:20:06.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.502+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8a64064a00 con 0x7f8a640d1c30 2023-10-10T20:20:06.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.505+0000 7f8a4bfff700 1 -- 172.21.15.148:0/1031180706 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8a54017020 con 0x7f8a640d1c30 2023-10-10T20:20:06.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.507+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 msgr2=0x7f8a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 0x7f8a500693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d2000 tx=0x7f8a58017000 comp rx=0 tx=0).stop 2023-10-10T20:20:06.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 msgr2=0x7f8a6415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c5e0 tx=0x7f8a54007e80 comp rx=0 tx=0).stop 2023-10-10T20:20:06.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 shutdown_connections 2023-10-10T20:20:06.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8a50066f30 0x7f8a500693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a6415a680 0x7f8a64158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a640d1c30 0x7f8a6415d6e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 --2- 172.21.15.148:0/1031180706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6415a2b0 0x7f8a64155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 >> 172.21.15.148:0/1031180706 conn(0x7f8a64067e40 msgr2=0x7f8a64069140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 shutdown_connections 2023-10-10T20:20:06.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.508+0000 7f8a6a6cf700 1 -- 172.21.15.148:0/1031180706 wait complete. 2023-10-10T20:20:06.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.635+0000 7fd9aa74b700 1 Processor -- start 2023-10-10T20:20:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.635+0000 7fd9aa74b700 1 -- start start 2023-10-10T20:20:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9aa74b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 0x7fd9a40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9aa74b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9a415cbd0 0x7fd9a415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9aa74b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a40cb7c0 con 0x7fd9a40cdab0 2023-10-10T20:20:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9aa74b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a415e850 con 0x7fd9a415cbd0 2023-10-10T20:20:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9aa74b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40cd030 con 0x7fd9a40cd6e0 2023-10-10T20:20:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9a415cbd0 0x7fd9a415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9a415cbd0 0x7fd9a415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45446/0 (socket says 172.21.15.148:45446) 2023-10-10T20:20:06.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 -- 172.21.15.148:0/1187717946 learned_addr learned my addr 172.21.15.148:0/1187717946 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:06.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9749700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 0x7fd9a40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 -- 172.21.15.148:0/1187717946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 msgr2=0x7fd9a40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:06.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 0x7fd9a40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 -- 172.21.15.148:0/1187717946 >> v1:172.21.15.161:6789/0 conn(0x7fd9a40cdab0 legacy=0x7fd9a415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 -- 172.21.15.148:0/1187717946 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a415e290 con 0x7fd9a415cbd0 2023-10-10T20:20:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9749700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 0x7fd9a40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.636+0000 7fd9a9f4a700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a415cbd0 0x7fd9a415d020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009f20 tx=0x7fd99c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f146b38eec55ddf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd99bfff700 1 -- 172.21.15.148:0/1187717946 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00e070 con 0x7fd9a415cbd0 2023-10-10T20:20:06.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd99bfff700 1 -- 172.21.15.148:0/1187717946 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c00ba80 con 0x7fd9a415cbd0 2023-10-10T20:20:06.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd99bfff700 1 -- 172.21.15.148:0/1187717946 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0092e0 con 0x7fd9a415cbd0 2023-10-10T20:20:06.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 -- 172.21.15.148:0/1187717946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a415cbd0 msgr2=0x7fd9a415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a415cbd0 0x7fd9a415d020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009f20 tx=0x7fd99c009840 comp rx=0 tx=0).stop 2023-10-10T20:20:06.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 -- 172.21.15.148:0/1187717946 shutdown_connections 2023-10-10T20:20:06.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a415cbd0 0x7fd9a415d020 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/1187717946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a40cd6e0 0x7fd9a40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 -- 172.21.15.148:0/1187717946 >> 172.21.15.148:0/1187717946 conn(0x7fd9a4067660 msgr2=0x7fd9a4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 -- 172.21.15.148:0/1187717946 shutdown_connections 2023-10-10T20:20:06.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.637+0000 7fd9aa74b700 1 -- 172.21.15.148:0/1187717946 wait complete. 2023-10-10T20:20:06.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.638+0000 7fd9aa74b700 1 Processor -- start 2023-10-10T20:20:06.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.638+0000 7fd9aa74b700 1 -- start start 2023-10-10T20:20:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9a40cd6e0 0x7fd9a414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a415cbd0 0x7fd9a416ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a414cd90 con 0x7fd9a40cdab0 2023-10-10T20:20:06.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40b5390 con 0x7fd9a40cd6e0 2023-10-10T20:20:06.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9aa74b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40c8490 con 0x7fd9a415cbd0 2023-10-10T20:20:06.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9a8f48700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9a8f48700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45454/0 (socket says 172.21.15.148:45454) 2023-10-10T20:20:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9a8f48700 1 -- 172.21.15.148:0/980400715 learned_addr learned my addr 172.21.15.148:0/980400715 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9a9749700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9a40cd6e0 0x7fd9a414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.639+0000 7fd9a9f4a700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a415cbd0 0x7fd9a416ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 -- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a415cbd0 msgr2=0x7fd9a416ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a415cbd0 0x7fd9a416ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 -- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9a40cd6e0 msgr2=0x7fd9a414c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9a40cd6e0 0x7fd9a414c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 -- 172.21.15.148:0/980400715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009ae0 con 0x7fd9a40cdab0 2023-10-10T20:20:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9a8f48700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c5c0 tx=0x7fd99400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994011590 con 0x7fd9a40cdab0 2023-10-10T20:20:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd994011710 con 0x7fd9a40cdab0 2023-10-10T20:20:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a416f330 con 0x7fd9a40cdab0 2023-10-10T20:20:06.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994011ad0 con 0x7fd9a40cdab0 2023-10-10T20:20:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.640+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a40b4d40 con 0x7fd9a40cdab0 2023-10-10T20:20:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.641+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a4089280 con 0x7fd9a40cdab0 2023-10-10T20:20:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.641+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd99401b3b0 con 0x7fd9a40cdab0 2023-10-10T20:20:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.642+0000 7fd9997fa700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 0x7fd9880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.642+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd994083690 con 0x7fd9a40cdab0 2023-10-10T20:20:06.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.642+0000 7fd9a9749700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 0x7fd9880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.642+0000 7fd9a9749700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 0x7fd9880693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40cad90 tx=0x7fd9a0006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.646+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd994050c50 con 0x7fd9a40cdab0 2023-10-10T20:20:06.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.831+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9a415e5d0 con 0x7fd9a40cdab0 2023-10-10T20:20:06.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.832+0000 7fd9997fa700 1 -- 172.21.15.148:0/980400715 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12182 (secure 0 0 0) 0x7fd994017020 con 0x7fd9a40cdab0 2023-10-10T20:20:06.833 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:06.834 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:03.657239+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:06.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 msgr2=0x7fd9880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 0x7fd9880693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40cad90 tx=0x7fd9a0006f90 comp rx=0 tx=0).stop 2023-10-10T20:20:06.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 msgr2=0x7fd9a416e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c5c0 tx=0x7fd99400c8d0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 shutdown_connections 2023-10-10T20:20:06.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd988066f30 0x7fd9880693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9a415cbd0 0x7fd9a416ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9a40cd6e0 0x7fd9a414c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 --2- 172.21.15.148:0/980400715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9a40cdab0 0x7fd9a416e750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 >> 172.21.15.148:0/980400715 conn(0x7fd9a4067660 msgr2=0x7fd9a40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 shutdown_connections 2023-10-10T20:20:06.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.834+0000 7fd9aa74b700 1 -- 172.21.15.148:0/980400715 wait complete. 2023-10-10T20:20:06.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:20:06.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:20:06.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.964+0000 7f4279b45700 1 Processor -- start 2023-10-10T20:20:06.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.964+0000 7f4279b45700 1 -- start start 2023-10-10T20:20:06.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4279b45700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4274159ec0 0x7f42740af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4279b45700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f427406acb0 0x7f42740afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:06.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4279b45700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42740b0790 con 0x7f427415a290 2023-10-10T20:20:06.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4279b45700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740ce1f0 con 0x7f427406acb0 2023-10-10T20:20:06.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4279b45700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740b0b20 con 0x7f4274159ec0 2023-10-10T20:20:06.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f427406acb0 0x7f42740afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4272ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f427415a290 0x7f427406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34918/0 (socket says 172.21.15.148:34918) 2023-10-10T20:20:06.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4272ffd700 1 -- 172.21.15.148:0/3507427182 learned_addr learned my addr 172.21.15.148:0/3507427182 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:06.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f42737fe700 1 --2- 172.21.15.148:0/3507427182 >> v2:172.21.15.161:3300/0 conn(0x7f4274159ec0 0x7f42740af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:06.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 -- 172.21.15.148:0/3507427182 >> v2:172.21.15.161:3300/0 conn(0x7f4274159ec0 msgr2=0x7f42740af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:06.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 --2- 172.21.15.148:0/3507427182 >> v2:172.21.15.161:3300/0 conn(0x7f4274159ec0 0x7f42740af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 -- 172.21.15.148:0/3507427182 >> v1:172.21.15.148:6789/0 conn(0x7f427415a290 legacy=0x7f427406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 -- 172.21.15.148:0/3507427182 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740b04b0 con 0x7f427406acb0 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f42727fc700 1 -- 172.21.15.148:0/3507427182 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223155633 0 0) 0x7f42740b0790 con 0x7f427415a290 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.965+0000 7f4273fff700 1 --2- 172.21.15.148:0/3507427182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f42740afe80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42740898e0 tx=0x7f426c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9049a5dbe52be524 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f42727fc700 1 -- 172.21.15.148:0/3507427182 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00b6b0 con 0x7f427406acb0 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f42727fc700 1 -- 172.21.15.148:0/3507427182 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c00b810 con 0x7f427406acb0 2023-10-10T20:20:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f42727fc700 1 -- 172.21.15.148:0/3507427182 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00bbb0 con 0x7f427406acb0 2023-10-10T20:20:06.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 -- 172.21.15.148:0/3507427182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 msgr2=0x7f42740afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:06.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 --2- 172.21.15.148:0/3507427182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f42740afe80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42740898e0 tx=0x7f426c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 -- 172.21.15.148:0/3507427182 shutdown_connections 2023-10-10T20:20:06.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 --2- 172.21.15.148:0/3507427182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f42740afe80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 --2- 172.21.15.148:0/3507427182 >> v2:172.21.15.161:3300/0 conn(0x7f4274159ec0 0x7f42740af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 -- 172.21.15.148:0/3507427182 >> 172.21.15.148:0/3507427182 conn(0x7f42740d1880 msgr2=0x7f42740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 -- 172.21.15.148:0/3507427182 shutdown_connections 2023-10-10T20:20:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.966+0000 7f4279b45700 1 -- 172.21.15.148:0/3507427182 wait complete. 2023-10-10T20:20:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.967+0000 7f4279b45700 1 Processor -- start 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.967+0000 7f4279b45700 1 -- start start 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 0x7f42740b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f427415a290 0x7f42740b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c8070 con 0x7f427406acb0 2023-10-10T20:20:07.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4274155ec0 con 0x7f4274159ec0 2023-10-10T20:20:07.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f4279b45700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740cb1f0 con 0x7f427415a290 2023-10-10T20:20:07.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f42737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f42737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45468/0 (socket says 172.21.15.148:45468) 2023-10-10T20:20:07.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.968+0000 7f42737fe700 1 -- 172.21.15.148:0/3476923998 learned_addr learned my addr 172.21.15.148:0/3476923998 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:07.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f4272ffd700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 0x7f42740b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f4273fff700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f427415a290 0x7f42740b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 -- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f427415a290 msgr2=0x7f42740b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f427415a290 0x7f42740b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 -- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 msgr2=0x7f42740b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 0x7f42740b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 -- 172.21.15.148:0/3476923998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009420 con 0x7f427406acb0 2023-10-10T20:20:07.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f4272ffd700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 0x7f42740b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:07.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f42737fe700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f426400e190 tx=0x7f4264009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400ee30 con 0x7f427406acb0 2023-10-10T20:20:07.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4264014420 con 0x7f427406acb0 2023-10-10T20:20:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42740cb4d0 con 0x7f427406acb0 2023-10-10T20:20:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42640147e0 con 0x7f427406acb0 2023-10-10T20:20:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.969+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42740b6650 con 0x7f427406acb0 2023-10-10T20:20:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.970+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42740c8350 con 0x7f427406acb0 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.970+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4264014b90 con 0x7f427406acb0 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.971+0000 7f425b7fe700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 0x7f42600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.971+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4264084a20 con 0x7f427406acb0 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.971+0000 7f4272ffd700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 0x7f42600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.971+0000 7f4272ffd700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 0x7f42600693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f427416b120 tx=0x7f4268009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:06.975+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4264051fe0 con 0x7f427406acb0 2023-10-10T20:20:07.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.159+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f427404c3c0 con 0x7f427406acb0 2023-10-10T20:20:07.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.160+0000 7f425b7fe700 1 -- 172.21.15.148:0/3476923998 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4264009a60 con 0x7f427406acb0 2023-10-10T20:20:07.161 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:20:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 msgr2=0x7f42600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 0x7f42600693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f427416b120 tx=0x7f4268009040 comp rx=0 tx=0).stop 2023-10-10T20:20:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 msgr2=0x7f4274155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f426400e190 tx=0x7f4264009920 comp rx=0 tx=0).stop 2023-10-10T20:20:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 shutdown_connections 2023-10-10T20:20:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4260066f30 0x7f42600693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f427415a290 0x7f42740b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f427406acb0 0x7f4274155980 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.161+0000 7f4279b45700 1 --2- 172.21.15.148:0/3476923998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4274159ec0 0x7f42740b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.162+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 >> 172.21.15.148:0/3476923998 conn(0x7f42740d1880 msgr2=0x7f4274156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.162+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 shutdown_connections 2023-10-10T20:20:07.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.162+0000 7f4279b45700 1 -- 172.21.15.148:0/3476923998 wait complete. 2023-10-10T20:20:07.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:20:07.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.289+0000 7f9c5da89700 1 Processor -- start 2023-10-10T20:20:07.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.289+0000 7f9c5da89700 1 -- start start 2023-10-10T20:20:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c5da89700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c58157c40 0x7f9c58158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c5da89700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5815cbb0 con 0x7f9c58157870 2023-10-10T20:20:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c5da89700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5815bfa0 con 0x7f9c5815a620 2023-10-10T20:20:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c5da89700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5815bdb0 con 0x7f9c58157c40 2023-10-10T20:20:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c57fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c58157c40 0x7f9c58158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c577fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9c58157870 0x7f9c5815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34920/0 (socket says 172.21.15.148:34920) 2023-10-10T20:20:07.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.290+0000 7f9c577fe700 1 -- 172.21.15.148:0/1076327328 learned_addr learned my addr 172.21.15.148:0/1076327328 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:07.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c57fff700 1 -- 172.21.15.148:0/1076327328 >> v1:172.21.15.161:6789/0 conn(0x7f9c5815a620 legacy=0x7f9c5815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c57fff700 1 -- 172.21.15.148:0/1076327328 >> v1:172.21.15.148:6789/0 conn(0x7f9c58157870 legacy=0x7f9c5815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c57fff700 1 -- 172.21.15.148:0/1076327328 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5815c280 con 0x7f9c58157c40 2023-10-10T20:20:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c567fc700 1 -- 172.21.15.148:0/1076327328 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196858178 0 0) 0x7f9c5815cbb0 con 0x7f9c58157870 2023-10-10T20:20:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c57fff700 1 --2- 172.21.15.148:0/1076327328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c58157c40 0x7f9c58158110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098e0 tx=0x7f9c500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733e56ab36608e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c567fc700 1 -- 172.21.15.148:0/1076327328 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50009e80 con 0x7f9c58157c40 2023-10-10T20:20:07.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c567fc700 1 -- 172.21.15.148:0/1076327328 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5000b860 con 0x7f9c58157c40 2023-10-10T20:20:07.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c567fc700 1 -- 172.21.15.148:0/1076327328 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000bbe0 con 0x7f9c58157c40 2023-10-10T20:20:07.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 -- 172.21.15.148:0/1076327328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c58157c40 msgr2=0x7f9c58158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 --2- 172.21.15.148:0/1076327328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c58157c40 0x7f9c58158110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098e0 tx=0x7f9c500091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 -- 172.21.15.148:0/1076327328 shutdown_connections 2023-10-10T20:20:07.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 --2- 172.21.15.148:0/1076327328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c58157c40 0x7f9c58158110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 -- 172.21.15.148:0/1076327328 >> 172.21.15.148:0/1076327328 conn(0x7f9c58068ce0 msgr2=0x7f9c580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:07.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 -- 172.21.15.148:0/1076327328 shutdown_connections 2023-10-10T20:20:07.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.291+0000 7f9c5da89700 1 -- 172.21.15.148:0/1076327328 wait complete. 2023-10-10T20:20:07.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 Processor -- start 2023-10-10T20:20:07.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 -- start start 2023-10-10T20:20:07.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c58157870 0x7f9c58138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 0x7f9c58150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b3530 con 0x7f9c5815a620 2023-10-10T20:20:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b3250 con 0x7f9c58157870 2023-10-10T20:20:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.293+0000 7f9c5da89700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5814e840 con 0x7f9c58157c40 2023-10-10T20:20:07.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45480/0 (socket says 172.21.15.148:45480) 2023-10-10T20:20:07.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c58157870 0x7f9c58138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 -- 172.21.15.148:0/4138474312 learned_addr learned my addr 172.21.15.148:0/4138474312 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:07.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c56ffd700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 0x7f9c58150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 -- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 msgr2=0x7f9c58150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 0x7f9c58150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 -- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c58157870 msgr2=0x7f9c58138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c58157870 0x7f9c58138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 -- 172.21.15.148:0/4138474312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009420 con 0x7f9c5815a620 2023-10-10T20:20:07.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c56ffd700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 0x7f9c58150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:07.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c57fff700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c50013000 tx=0x7f9c50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5001f070 con 0x7f9c5815a620 2023-10-10T20:20:07.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50011420 con 0x7f9c5815a620 2023-10-10T20:20:07.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500117e0 con 0x7f9c5815a620 2023-10-10T20:20:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5814eb20 con 0x7f9c5815a620 2023-10-10T20:20:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.294+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c58139380 con 0x7f9c5815a620 2023-10-10T20:20:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.295+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58139570 con 0x7f9c5815a620 2023-10-10T20:20:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.295+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c50011b50 con 0x7f9c5815a620 2023-10-10T20:20:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.296+0000 7f9c3f7fe700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.296+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9c500858b0 con 0x7f9c5815a620 2023-10-10T20:20:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.296+0000 7f9c577fe700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.296+0000 7f9c577fe700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 0x7f9c440693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c48002250 tx=0x7f9c48001a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.300+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c50052e70 con 0x7f9c5815a620 2023-10-10T20:20:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.487+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9c5804c3c0 con 0x7f9c5815a620 2023-10-10T20:20:07.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.487+0000 7f9c3f7fe700 1 -- 172.21.15.148:0/4138474312 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9c50028060 con 0x7f9c5815a620 2023-10-10T20:20:07.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:07.490 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:20:07.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:07.491 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:20:07.491 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:20:07.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:07.492 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:20:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 msgr2=0x7f9c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 0x7f9c440693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c48002250 tx=0x7f9c48001a00 comp rx=0 tx=0).stop 2023-10-10T20:20:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 msgr2=0x7f9c58151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c50013000 tx=0x7f9c50017000 comp rx=0 tx=0).stop 2023-10-10T20:20:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 shutdown_connections 2023-10-10T20:20:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9c44066f30 0x7f9c440693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5815a620 0x7f9c58151480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c58157870 0x7f9c58138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 --2- 172.21.15.148:0/4138474312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c58157c40 0x7f9c58150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 >> 172.21.15.148:0/4138474312 conn(0x7f9c58068ce0 msgr2=0x7f9c580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 shutdown_connections 2023-10-10T20:20:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.489+0000 7f9c5da89700 1 -- 172.21.15.148:0/4138474312 wait complete. 2023-10-10T20:20:07.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:20:07.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:20:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.620+0000 7f8dd86e5700 1 Processor -- start 2023-10-10T20:20:07.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.620+0000 7f8dd86e5700 1 -- start start 2023-10-10T20:20:07.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8dd00b6a10 0x7f8dd00b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8dd00b15e0 0x7f8dd00b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd00b1a60 con 0x7f8dd00b15e0 2023-10-10T20:20:07.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd00b1d40 con 0x7f8dd00b6a10 2023-10-10T20:20:07.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd86e5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd00b2020 con 0x7f8dd0159ef0 2023-10-10T20:20:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6c82700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8dd00b15e0 0x7f8dd00b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6c82700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8dd00b15e0 0x7f8dd00b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45492/0 (socket says 172.21.15.148:45492) 2023-10-10T20:20:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6c82700 1 -- 172.21.15.148:0/92385490 learned_addr learned my addr 172.21.15.148:0/92385490 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd5c80700 1 --2- 172.21.15.148:0/92385490 >> v2:172.21.15.161:3300/0 conn(0x7f8dd00b6a10 0x7f8dd00b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6481700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6c82700 1 -- 172.21.15.148:0/92385490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 msgr2=0x7f8dd015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:07.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.621+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd015a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd6c82700 1 -- 172.21.15.148:0/92385490 >> v2:172.21.15.161:3300/0 conn(0x7f8dd00b6a10 msgr2=0x7f8dd00b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:07.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/92385490 >> v2:172.21.15.161:3300/0 conn(0x7f8dd00b6a10 0x7f8dd00b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd6c82700 1 -- 172.21.15.148:0/92385490 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00b2300 con 0x7f8dd00b15e0 2023-10-10T20:20:07.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd00b15e0 0x7f8dd00b46f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00894b0 tx=0x7f8dc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6569c9f0f2ec62ef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd547f700 1 -- 172.21.15.148:0/92385490 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800e070 con 0x7f8dd00b15e0 2023-10-10T20:20:07.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd547f700 1 -- 172.21.15.148:0/92385490 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800ba50 con 0x7f8dd00b15e0 2023-10-10T20:20:07.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd547f700 1 -- 172.21.15.148:0/92385490 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80092e0 con 0x7f8dd00b15e0 2023-10-10T20:20:07.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 -- 172.21.15.148:0/92385490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd00b15e0 msgr2=0x7f8dd00b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd00b15e0 0x7f8dd00b46f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00894b0 tx=0x7f8dc8009840 comp rx=0 tx=0).stop 2023-10-10T20:20:07.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 -- 172.21.15.148:0/92385490 shutdown_connections 2023-10-10T20:20:07.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd00b15e0 0x7f8dd00b46f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/92385490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/92385490 >> v2:172.21.15.161:3300/0 conn(0x7f8dd00b6a10 0x7f8dd00b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.622+0000 7f8dd86e5700 1 -- 172.21.15.148:0/92385490 >> 172.21.15.148:0/92385490 conn(0x7f8dd009a0a0 msgr2=0x7f8dd0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:07.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.623+0000 7f8dd86e5700 1 -- 172.21.15.148:0/92385490 shutdown_connections 2023-10-10T20:20:07.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.623+0000 7f8dd86e5700 1 -- 172.21.15.148:0/92385490 wait complete. 2023-10-10T20:20:07.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 Processor -- start 2023-10-10T20:20:07.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 -- start start 2023-10-10T20:20:07.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 0x7f8dd009e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd00ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd009d190 con 0x7f8dd009cc20 2023-10-10T20:20:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd009d470 con 0x7f8dd00b6a10 2023-10-10T20:20:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd86e5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd009d750 con 0x7f8dd0159ef0 2023-10-10T20:20:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd6c82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd6c82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45506/0 (socket says 172.21.15.148:45506) 2023-10-10T20:20:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd6c82700 1 -- 172.21.15.148:0/867506277 learned_addr learned my addr 172.21.15.148:0/867506277 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.624+0000 7f8dd6481700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 0x7f8dd009e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd5c80700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd00ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 -- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 msgr2=0x7f8dd00ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd00ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 -- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 msgr2=0x7f8dd009e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:07.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 0x7f8dd009e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:07.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 -- 172.21.15.148:0/867506277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8009ae0 con 0x7f8dd009cc20 2023-10-10T20:20:07.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6481700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 0x7f8dd009e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:07.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd5c80700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd00ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:07.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd6c82700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8012000 tx=0x7f8dc8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80032f0 con 0x7f8dd009cc20 2023-10-10T20:20:07.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8008e80 con 0x7f8dd009cc20 2023-10-10T20:20:07.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80266b0 con 0x7f8dd009cc20 2023-10-10T20:20:07.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd009da30 con 0x7f8dd009cc20 2023-10-10T20:20:07.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.625+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd009e770 con 0x7f8dd009cc20 2023-10-10T20:20:07.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.626+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd004c3c0 con 0x7f8dd009cc20 2023-10-10T20:20:07.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.626+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8dc8003040 con 0x7f8dd009cc20 2023-10-10T20:20:07.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.627+0000 7f8dc67fc700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 0x7f8db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.627+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8dc8085cc0 con 0x7f8dd009cc20 2023-10-10T20:20:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.627+0000 7f8dd6481700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 0x7f8db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.627+0000 7f8dd6481700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 0x7f8db40693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00aa80 tx=0x7f8dcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.631+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8dc8053330 con 0x7f8dd009cc20 2023-10-10T20:20:07.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:07.813+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8dd0071e20 con 0x7f8dd009cc20 2023-10-10T20:20:08.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.670+0000 7f8dc67fc700 1 -- 172.21.15.148:0/867506277 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8dc8009940 con 0x7f8dd009cc20 2023-10-10T20:20:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 msgr2=0x7f8db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 0x7f8db40693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00aa80 tx=0x7f8dcc008040 comp rx=0 tx=0).stop 2023-10-10T20:20:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 msgr2=0x7f8dd015c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8012000 tx=0x7f8dc8018000 comp rx=0 tx=0).stop 2023-10-10T20:20:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 shutdown_connections 2023-10-10T20:20:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8db4066f30 0x7f8db40693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8dd009cc20 0x7f8dd015c730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00b6a10 0x7f8dd009e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 --2- 172.21.15.148:0/867506277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd0159ef0 0x7f8dd00ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.674+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 >> 172.21.15.148:0/867506277 conn(0x7f8dd009a0a0 msgr2=0x7f8dd0158060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.675+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 shutdown_connections 2023-10-10T20:20:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.675+0000 7f8dd86e5700 1 -- 172.21.15.148:0/867506277 wait complete. 2023-10-10T20:20:08.678 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:20:08.688 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:20:08.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.809+0000 7fd7a759e700 1 Processor -- start 2023-10-10T20:20:08.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.809+0000 7fd7a759e700 1 -- start start 2023-10-10T20:20:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd7a759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a806aa00 0x7fd7a80b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd7a759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a806b0a0 con 0x7fd7a81542a0 2023-10-10T20:20:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd7a759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a815ad20 con 0x7fd7a80cd630 2023-10-10T20:20:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd7a759e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a815b2e0 con 0x7fd7a806aa00 2023-10-10T20:20:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd79dd9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a806aa00 0x7fd7a80b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:08.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd79dd9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a806aa00 0x7fd7a80b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45508/0 (socket says 172.21.15.148:45508) 2023-10-10T20:20:08.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd7a659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd7a81542a0 0x7fd7a806a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34934/0 (socket says 172.21.15.148:34934) 2023-10-10T20:20:08.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.810+0000 7fd79dd9b700 1 -- 172.21.15.148:0/1514043859 learned_addr learned my addr 172.21.15.148:0/1514043859 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:08.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd79dd9b700 1 -- 172.21.15.148:0/1514043859 >> v1:172.21.15.161:6789/0 conn(0x7fd7a80cd630 legacy=0x7fd7a815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:08.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd79dd9b700 1 -- 172.21.15.148:0/1514043859 >> v1:172.21.15.148:6789/0 conn(0x7fd7a81542a0 legacy=0x7fd7a806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd79dd9b700 1 -- 172.21.15.148:0/1514043859 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a815b000 con 0x7fd7a806aa00 2023-10-10T20:20:08.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a5d9b700 1 -- 172.21.15.148:0/1514043859 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668775010 0 0) 0x7fd7a806b0a0 con 0x7fd7a81542a0 2023-10-10T20:20:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd79dd9b700 1 --2- 172.21.15.148:0/1514043859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a806aa00 0x7fd7a80b65a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7900099b0 tx=0x7fd790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1847b881b86172f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a5d9b700 1 -- 172.21.15.148:0/1514043859 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000b820 con 0x7fd7a806aa00 2023-10-10T20:20:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a5d9b700 1 -- 172.21.15.148:0/1514043859 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000b980 con 0x7fd7a806aa00 2023-10-10T20:20:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a5d9b700 1 -- 172.21.15.148:0/1514043859 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000bd20 con 0x7fd7a806aa00 2023-10-10T20:20:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 -- 172.21.15.148:0/1514043859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a806aa00 msgr2=0x7fd7a80b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1514043859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a806aa00 0x7fd7a80b65a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7900099b0 tx=0x7fd790009250 comp rx=0 tx=0).stop 2023-10-10T20:20:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 -- 172.21.15.148:0/1514043859 shutdown_connections 2023-10-10T20:20:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1514043859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a806aa00 0x7fd7a80b65a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 -- 172.21.15.148:0/1514043859 >> 172.21.15.148:0/1514043859 conn(0x7fd7a8068cb0 msgr2=0x7fd7a80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 -- 172.21.15.148:0/1514043859 shutdown_connections 2023-10-10T20:20:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.811+0000 7fd7a759e700 1 -- 172.21.15.148:0/1514043859 wait complete. 2023-10-10T20:20:08.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 Processor -- start 2023-10-10T20:20:08.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 -- start start 2023-10-10T20:20:08.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a806aa00 0x7fd7a8156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:08.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a80cd630 0x7fd7a8151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80c5f90 con 0x7fd7a81542a0 2023-10-10T20:20:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a8155dc0 con 0x7fd7a80cd630 2023-10-10T20:20:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a816b010 con 0x7fd7a806aa00 2023-10-10T20:20:08.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.813+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45522/0 (socket says 172.21.15.148:45522) 2023-10-10T20:20:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 -- 172.21.15.148:0/1353538529 learned_addr learned my addr 172.21.15.148:0/1353538529 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd79dd9b700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a80cd630 0x7fd7a8151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a659c700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a806aa00 0x7fd7a8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 -- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a806aa00 msgr2=0x7fd7a8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a806aa00 0x7fd7a8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 -- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a80cd630 msgr2=0x7fd7a8151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a80cd630 0x7fd7a8151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 -- 172.21.15.148:0/1353538529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900094f0 con 0x7fd7a81542a0 2023-10-10T20:20:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a6d9d700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c930 tx=0x7fd7a00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0012630 con 0x7fd7a81542a0 2023-10-10T20:20:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a00127b0 con 0x7fd7a81542a0 2023-10-10T20:20:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a8155770 con 0x7fd7a81542a0 2023-10-10T20:20:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0012b70 con 0x7fd7a81542a0 2023-10-10T20:20:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.814+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a8151840 con 0x7fd7a81542a0 2023-10-10T20:20:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.815+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a8155f70 con 0x7fd7a81542a0 2023-10-10T20:20:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.815+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7a001a3b0 con 0x7fd7a81542a0 2023-10-10T20:20:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.816+0000 7fd79effd700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 0x7fd7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.816+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fd7a0083f40 con 0x7fd7a81542a0 2023-10-10T20:20:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.816+0000 7fd7a659c700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 0x7fd7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:08.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.816+0000 7fd7a659c700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 0x7fd7880693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7980012a0 tx=0x7fd798009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:08.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:08.820+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7a8155f70 con 0x7fd7a81542a0 2023-10-10T20:20:09.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.001+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7a8155f70 con 0x7fd7a81542a0 2023-10-10T20:20:09.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.673+0000 7fd79effd700 1 -- 172.21.15.148:0/1353538529 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd7a8155f70 con 0x7fd7a81542a0 2023-10-10T20:20:09.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.679+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 msgr2=0x7fd7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.679+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 0x7fd7880693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7980012a0 tx=0x7fd798009040 comp rx=0 tx=0).stop 2023-10-10T20:20:09.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.679+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 msgr2=0x7fd7a80b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.679+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c930 tx=0x7fd7a00026e0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 shutdown_connections 2023-10-10T20:20:09.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd788066f30 0x7fd7880693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a81542a0 0x7fd7a80b5540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a806aa00 0x7fd7a8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 --2- 172.21.15.148:0/1353538529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a80cd630 0x7fd7a8151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.682+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 >> 172.21.15.148:0/1353538529 conn(0x7fd7a8068cb0 msgr2=0x7fd7a80cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:09.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.684+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 shutdown_connections 2023-10-10T20:20:09.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.684+0000 7fd7a759e700 1 -- 172.21.15.148:0/1353538529 wait complete. 2023-10-10T20:20:09.686 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:20:09.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:20:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.817+0000 7f934c7bf700 1 Processor -- start 2023-10-10T20:20:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.817+0000 7f934c7bf700 1 -- start start 2023-10-10T20:20:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934c7bf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9344159de0 0x7f93440cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934c7bf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f934415e140 0x7f93440ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934c7bf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440cd790 con 0x7f93440cc680 2023-10-10T20:20:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934c7bf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440ce160 con 0x7f9344159de0 2023-10-10T20:20:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934c7bf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440cb440 con 0x7f934415e140 2023-10-10T20:20:09.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9344159de0 0x7f93440cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9344159de0 0x7f93440cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45534/0 (socket says 172.21.15.148:45534) 2023-10-10T20:20:09.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 -- 172.21.15.148:0/2741246460 learned_addr learned my addr 172.21.15.148:0/2741246460 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:09.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f9349d5a700 1 --1- 172.21.15.148:0/2741246460 >> v1:172.21.15.148:6789/0 conn(0x7f93440cc680 0x7f934415da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34936/0 (socket says 172.21.15.148:34936) 2023-10-10T20:20:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934ad5c700 1 --2- 172.21.15.148:0/2741246460 >> v2:172.21.15.161:3300/0 conn(0x7f934415e140 0x7f93440ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 -- 172.21.15.148:0/2741246460 >> v2:172.21.15.161:3300/0 conn(0x7f934415e140 msgr2=0x7f93440ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 --2- 172.21.15.148:0/2741246460 >> v2:172.21.15.161:3300/0 conn(0x7f934415e140 0x7f93440ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 -- 172.21.15.148:0/2741246460 >> v1:172.21.15.148:6789/0 conn(0x7f93440cc680 legacy=0x7f934415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f9349559700 1 -- 172.21.15.148:0/2741246460 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792004017 0 0) 0x7f93440cd790 con 0x7f93440cc680 2023-10-10T20:20:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 -- 172.21.15.148:0/2741246460 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440cad60 con 0x7f9344159de0 2023-10-10T20:20:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f934a55b700 1 --2- 172.21.15.148:0/2741246460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9344159de0 0x7f93440cdc20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93440cc5f0 tx=0x7f9340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f769a86715bb32b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f9349559700 1 -- 172.21.15.148:0/2741246460 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000b860 con 0x7f9344159de0 2023-10-10T20:20:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f9349559700 1 -- 172.21.15.148:0/2741246460 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934000b9c0 con 0x7f9344159de0 2023-10-10T20:20:09.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.818+0000 7f9349559700 1 -- 172.21.15.148:0/2741246460 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000bd60 con 0x7f9344159de0 2023-10-10T20:20:09.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 -- 172.21.15.148:0/2741246460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9344159de0 msgr2=0x7f93440cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 --2- 172.21.15.148:0/2741246460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9344159de0 0x7f93440cdc20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93440cc5f0 tx=0x7f9340009250 comp rx=0 tx=0).stop 2023-10-10T20:20:09.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 -- 172.21.15.148:0/2741246460 shutdown_connections 2023-10-10T20:20:09.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 --2- 172.21.15.148:0/2741246460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9344159de0 0x7f93440cdc20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 --2- 172.21.15.148:0/2741246460 >> v2:172.21.15.161:3300/0 conn(0x7f934415e140 0x7f93440ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 -- 172.21.15.148:0/2741246460 >> 172.21.15.148:0/2741246460 conn(0x7f93440cb5c0 msgr2=0x7f93440cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 -- 172.21.15.148:0/2741246460 shutdown_connections 2023-10-10T20:20:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.819+0000 7f934c7bf700 1 -- 172.21.15.148:0/2741246460 wait complete. 2023-10-10T20:20:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.820+0000 7f934c7bf700 1 Processor -- start 2023-10-10T20:20:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.820+0000 7f934c7bf700 1 -- start start 2023-10-10T20:20:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93440cc680 0x7f93440c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9344159de0 0x7f93440c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344144a40 con 0x7f934415e140 2023-10-10T20:20:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934415cfc0 con 0x7f93440cc680 2023-10-10T20:20:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934c7bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934415d2a0 con 0x7f9344159de0 2023-10-10T20:20:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45536/0 (socket says 172.21.15.148:45536) 2023-10-10T20:20:09.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 -- 172.21.15.148:0/1148064324 learned_addr learned my addr 172.21.15.148:0/1148064324 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:09.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934a55b700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93440cc680 0x7f93440c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f9349d5a700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9344159de0 0x7f93440c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 -- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9344159de0 msgr2=0x7f93440c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9344159de0 0x7f93440c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 -- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93440cc680 msgr2=0x7f93440c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93440cc680 0x7f93440c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 -- 172.21.15.148:0/1148064324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400094f0 con 0x7f934415e140 2023-10-10T20:20:09.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.821+0000 7f934ad5c700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f933c007f30 tx=0x7f933c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.822+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00eb90 con 0x7f934415e140 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.822+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00ecf0 con 0x7f934415e140 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.822+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93440b2080 con 0x7f934415e140 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.822+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c014670 con 0x7f934415e140 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.822+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440b2640 con 0x7f934415e140 2023-10-10T20:20:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.823+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93440b2860 con 0x7f934415e140 2023-10-10T20:20:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.823+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f933c014a20 con 0x7f934415e140 2023-10-10T20:20:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.824+0000 7f933a7fc700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 0x7f93280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.824+0000 7f934a55b700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 0x7f93280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.824+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f933c084b40 con 0x7f934415e140 2023-10-10T20:20:09.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.824+0000 7f934a55b700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 0x7f93280693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f93440cc5f0 tx=0x7f9340017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:09.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:09.828+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f933c0558a0 con 0x7f934415e140 2023-10-10T20:20:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.052+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f934404c3c0 con 0x7f934415e140 2023-10-10T20:20:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.695+0000 7f933a7fc700 1 -- 172.21.15.148:0/1148064324 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f933c009a20 con 0x7f934415e140 2023-10-10T20:20:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.699+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 msgr2=0x7f93280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.699+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 0x7f93280693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f93440cc5f0 tx=0x7f9340017000 comp rx=0 tx=0).stop 2023-10-10T20:20:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.699+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 msgr2=0x7f93440c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.699+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f933c007f30 tx=0x7f933c0098e0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 shutdown_connections 2023-10-10T20:20:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9328066f30 0x7f93280693e0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f93440cc5f0 tx=0x7f9340017000 comp rx=0 tx=0).stop 2023-10-10T20:20:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f934415e140 0x7f93440c62c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93440cc680 0x7f93440c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 --2- 172.21.15.148:0/1148064324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9344159de0 0x7f93440c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.700+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 >> 172.21.15.148:0/1148064324 conn(0x7f93440cb5c0 msgr2=0x7f934406d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.701+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 shutdown_connections 2023-10-10T20:20:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.701+0000 7f934c7bf700 1 -- 172.21.15.148:0/1148064324 wait complete. 2023-10-10T20:20:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 4 and data pool 5 2023-10-10T20:20:10.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.829+0000 7f920b40c700 1 Processor -- start 2023-10-10T20:20:10.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.829+0000 7f920b40c700 1 -- start start 2023-10-10T20:20:10.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920b40c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f92040cdab0 0x7f920415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920b40c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040cb7c0 con 0x7f92040cd6e0 2023-10-10T20:20:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920b40c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415e290 con 0x7f920415dda0 2023-10-10T20:20:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920b40c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920415e850 con 0x7f92040cdab0 2023-10-10T20:20:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920a40a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f92040cd6e0 0x7f92040cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34950/0 (socket says 172.21.15.148:34950) 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f920a40a700 1 -- 172.21.15.148:0/4071550809 learned_addr learned my addr 172.21.15.148:0/4071550809 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209c09700 1 --2- 172.21.15.148:0/4071550809 >> v2:172.21.15.161:3300/0 conn(0x7f92040cdab0 0x7f920415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18242125 0 0) 0x7f92040cb7c0 con 0x7f92040cd6e0 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f0003570 con 0x7f92040cd6e0 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805419290 0 0) 0x7f920415e290 con 0x7f920415dda0 2023-10-10T20:20:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92040cb7c0 con 0x7f920415dda0 2023-10-10T20:20:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061157321 0 0) 0x7f91f0003570 con 0x7f92040cd6e0 2023-10-10T20:20:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 >> v2:172.21.15.161:3300/0 conn(0x7f92040cdab0 msgr2=0x7f920415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 --2- 172.21.15.148:0/4071550809 >> v2:172.21.15.161:3300/0 conn(0x7f92040cdab0 0x7f920415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 >> v1:172.21.15.161:6789/0 conn(0x7f920415dda0 legacy=0x7f920415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920415e570 con 0x7f92040cd6e0 2023-10-10T20:20:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.830+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9200001470 con 0x7f92040cd6e0 2023-10-10T20:20:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 >> v1:172.21.15.148:6789/0 conn(0x7f92040cd6e0 legacy=0x7f92040cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 0x7f91f0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f0004810 0x7f91f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0005210 0x7f91f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920415e850 con 0x7f91f0003c40 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000c130 con 0x7f91f0004810 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000c2c0 con 0x7f91f0005210 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 0x7f91f0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209c09700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f0004810 0x7f91f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920ac0b700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0005210 0x7f91f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 -- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0005210 msgr2=0x7f91f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0005210 0x7f91f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 -- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f0004810 msgr2=0x7f91f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f0004810 0x7f91f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920ac0b700 1 -- 172.21.15.148:0/4071550809 reap_dead start 2023-10-10T20:20:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 -- 172.21.15.148:0/4071550809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0003570 con 0x7f91f0003c40 2023-10-10T20:20:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f920a40a700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 0x7f91f0004030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9200007fd0 tx=0x7f9200001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9808c52eef727414 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000cad0 con 0x7f91f0003c40 2023-10-10T20:20:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920000cc50 con 0x7f91f0003c40 2023-10-10T20:20:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.831+0000 7f9209408700 1 -- 172.21.15.148:0/4071550809 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200005920 con 0x7f91f0003c40 2023-10-10T20:20:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 -- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 msgr2=0x7f91f0004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 0x7f91f0004030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9200007fd0 tx=0x7f9200001d40 comp rx=0 tx=0).stop 2023-10-10T20:20:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 -- 172.21.15.148:0/4071550809 shutdown_connections 2023-10-10T20:20:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 --2- 172.21.15.148:0/4071550809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91f0003c40 0x7f91f0004030 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 -- 172.21.15.148:0/4071550809 >> 172.21.15.148:0/4071550809 conn(0x7f9204067660 msgr2=0x7f9204067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 -- 172.21.15.148:0/4071550809 shutdown_connections 2023-10-10T20:20:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.832+0000 7f920b40c700 1 -- 172.21.15.148:0/4071550809 wait complete. 2023-10-10T20:20:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.833+0000 7f920b40c700 1 Processor -- start 2023-10-10T20:20:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.833+0000 7f920b40c700 1 -- start start 2023-10-10T20:20:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0003c40 0x7f920416ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92040b2020 0x7f9204156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040cc360 con 0x7f92040b1bb0 2023-10-10T20:20:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040c8d80 con 0x7f92040b2020 2023-10-10T20:20:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920416b320 con 0x7f91f0003c40 2023-10-10T20:20:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45554/0 (socket says 172.21.15.148:45554) 2023-10-10T20:20:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 -- 172.21.15.148:0/3390137815 learned_addr learned my addr 172.21.15.148:0/3390137815 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920ac0b700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92040b2020 0x7f9204156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920a40a700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0003c40 0x7f920416ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 -- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0003c40 msgr2=0x7f920416ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0003c40 0x7f920416ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 -- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92040b2020 msgr2=0x7f9204156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92040b2020 0x7f9204156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 -- 172.21.15.148:0/3390137815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040c8100 con 0x7f92040b1bb0 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f9209c09700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f4006ac0 tx=0x7f91f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400f8e0 con 0x7f92040b1bb0 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400fa40 con 0x7f92040b1bb0 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400fdc0 con 0x7f92040b1bb0 2023-10-10T20:20:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92040c83e0 con 0x7f92040b1bb0 2023-10-10T20:20:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.834+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92040c89a0 con 0x7f92040b1bb0 2023-10-10T20:20:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.835+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9204089280 con 0x7f92040b1bb0 2023-10-10T20:20:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.836+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91f4014650 con 0x7f92040b1bb0 2023-10-10T20:20:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.836+0000 7f91fa7fc700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.836+0000 7f920a40a700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.836+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91f4018030 con 0x7f92040b1bb0 2023-10-10T20:20:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.836+0000 7f920a40a700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 0x7f91e80693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9200007fd0 tx=0x7f9200015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:10.840+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91f40520f0 con 0x7f92040b1bb0 2023-10-10T20:20:11.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.023+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f920404bb80 con 0x7f92040b1bb0 2023-10-10T20:20:11.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.024+0000 7f91fa7fc700 1 -- 172.21.15.148:0/3390137815 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f91f4022090 con 0x7f92040b1bb0 2023-10-10T20:20:11.025 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:11.026 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":30,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:10.053577+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:11.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 msgr2=0x7f91e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 0x7f91e80693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9200007fd0 tx=0x7f9200015000 comp rx=0 tx=0).stop 2023-10-10T20:20:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 msgr2=0x7f9204155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f4006ac0 tx=0x7f91f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 shutdown_connections 2023-10-10T20:20:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92040b2020 0x7f9204156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91f0003c40 0x7f920416ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 --2- 172.21.15.148:0/3390137815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92040b1bb0 0x7f9204155c30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 >> 172.21.15.148:0/3390137815 conn(0x7f9204067660 msgr2=0x7f92040cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 shutdown_connections 2023-10-10T20:20:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.026+0000 7f920b40c700 1 -- 172.21.15.148:0/3390137815 wait complete. 2023-10-10T20:20:11.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:11.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.156+0000 7fd778e94700 1 Processor -- start 2023-10-10T20:20:11.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.156+0000 7fd778e94700 1 -- start start 2023-10-10T20:20:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd778e94700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd774157870 0x7fd77415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd778e94700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd77415a620 0x7fd77415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd778e94700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77415cbb0 con 0x7fd77415d580 2023-10-10T20:20:11.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd778e94700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77415bfa0 con 0x7fd774157870 2023-10-10T20:20:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd778e94700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77415c280 con 0x7fd77415a620 2023-10-10T20:20:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd774157870 0x7fd77415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd774157870 0x7fd77415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45562/0 (socket says 172.21.15.148:45562) 2023-10-10T20:20:11.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 -- 172.21.15.148:0/2671900448 learned_addr learned my addr 172.21.15.148:0/2671900448 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd772d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd77415d580 0x7fd774158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34958/0 (socket says 172.21.15.148:34958) 2023-10-10T20:20:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd769d9b700 1 --2- 172.21.15.148:0/2671900448 >> v2:172.21.15.161:3300/0 conn(0x7fd77415a620 0x7fd77415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 -- 172.21.15.148:0/2671900448 >> v2:172.21.15.161:3300/0 conn(0x7fd77415a620 msgr2=0x7fd77415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 --2- 172.21.15.148:0/2671900448 >> v2:172.21.15.161:3300/0 conn(0x7fd77415a620 0x7fd77415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.157+0000 7fd77259c700 1 -- 172.21.15.148:0/2671900448 >> v1:172.21.15.148:6789/0 conn(0x7fd77415d580 legacy=0x7fd774158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd77259c700 1 -- 172.21.15.148:0/2671900448 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77415c560 con 0x7fd774157870 2023-10-10T20:20:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd771d9b700 1 -- 172.21.15.148:0/2671900448 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577170790 0 0) 0x7fd77415cbb0 con 0x7fd77415d580 2023-10-10T20:20:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd77259c700 1 --2- 172.21.15.148:0/2671900448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd774157870 0x7fd77415b6a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd77415cb20 tx=0x7fd764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06bedafd9d6d918 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd771d9b700 1 -- 172.21.15.148:0/2671900448 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76400b860 con 0x7fd774157870 2023-10-10T20:20:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd771d9b700 1 -- 172.21.15.148:0/2671900448 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76400b9c0 con 0x7fd774157870 2023-10-10T20:20:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd771d9b700 1 -- 172.21.15.148:0/2671900448 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76400bd60 con 0x7fd774157870 2023-10-10T20:20:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 -- 172.21.15.148:0/2671900448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd774157870 msgr2=0x7fd77415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 --2- 172.21.15.148:0/2671900448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd774157870 0x7fd77415b6a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd77415cb20 tx=0x7fd764009250 comp rx=0 tx=0).stop 2023-10-10T20:20:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 -- 172.21.15.148:0/2671900448 shutdown_connections 2023-10-10T20:20:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 --2- 172.21.15.148:0/2671900448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd774157870 0x7fd77415b6a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 --2- 172.21.15.148:0/2671900448 >> v2:172.21.15.161:3300/0 conn(0x7fd77415a620 0x7fd77415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 -- 172.21.15.148:0/2671900448 >> 172.21.15.148:0/2671900448 conn(0x7fd774068ce0 msgr2=0x7fd7740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 -- 172.21.15.148:0/2671900448 shutdown_connections 2023-10-10T20:20:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.158+0000 7fd778e94700 1 -- 172.21.15.148:0/2671900448 wait complete. 2023-10-10T20:20:11.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 Processor -- start 2023-10-10T20:20:11.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 -- start start 2023-10-10T20:20:11.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 0x7fd7740b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd77415d580 0x7fd7740b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b1920 con 0x7fd77415a620 2023-10-10T20:20:11.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b0d10 con 0x7fd774157870 2023-10-10T20:20:11.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd778e94700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b0ff0 con 0x7fd77415d580 2023-10-10T20:20:11.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd769d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd769d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45578/0 (socket says 172.21.15.148:45578) 2023-10-10T20:20:11.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd769d9b700 1 -- 172.21.15.148:0/3887291652 learned_addr learned my addr 172.21.15.148:0/3887291652 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:11.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.160+0000 7fd772d9d700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd77415d580 0x7fd7740b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd77259c700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 0x7fd7740b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 -- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd77415d580 msgr2=0x7fd7740b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd77415d580 0x7fd7740b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 -- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 msgr2=0x7fd7740b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 0x7fd7740b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 -- 172.21.15.148:0/3887291652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640094f0 con 0x7fd77415a620 2023-10-10T20:20:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd77259c700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 0x7fd7740b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd769d9b700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d860 tx=0x7fd75c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0125f0 con 0x7fd77415a620 2023-10-10T20:20:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c012750 con 0x7fd77415a620 2023-10-10T20:20:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740b12d0 con 0x7fd77415a620 2023-10-10T20:20:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012af0 con 0x7fd77415a620 2023-10-10T20:20:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.161+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740b22f0 con 0x7fd77415a620 2023-10-10T20:20:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.162+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740b1ad0 con 0x7fd77415a620 2023-10-10T20:20:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.162+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd75c01e460 con 0x7fd77415a620 2023-10-10T20:20:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.163+0000 7fd76affd700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 0x7fd7540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.163+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd75c084b20 con 0x7fd77415a620 2023-10-10T20:20:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.163+0000 7fd77259c700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 0x7fd7540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.163+0000 7fd77259c700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 0x7fd7540693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd77415cb20 tx=0x7fd764017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.167+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd75c051d90 con 0x7fd77415a620 2023-10-10T20:20:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.351+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7740b1770 con 0x7fd77415a620 2023-10-10T20:20:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.352+0000 7fd76affd700 1 -- 172.21.15.148:0/3887291652 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fd75c0553b0 con 0x7fd77415a620 2023-10-10T20:20:11.354 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:11.354 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":30,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:10.053577+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:11.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 msgr2=0x7fd7540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 0x7fd7540693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd77415cb20 tx=0x7fd764017000 comp rx=0 tx=0).stop 2023-10-10T20:20:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 msgr2=0x7fd7740d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d860 tx=0x7fd75c008040 comp rx=0 tx=0).stop 2023-10-10T20:20:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 shutdown_connections 2023-10-10T20:20:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd754066f30 0x7fd7540693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd77415d580 0x7fd7740b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd774157870 0x7fd7740b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 --2- 172.21.15.148:0/3887291652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd77415a620 0x7fd7740d1130 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 >> 172.21.15.148:0/3887291652 conn(0x7fd774068ce0 msgr2=0x7fd774159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:11.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 shutdown_connections 2023-10-10T20:20:11.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.354+0000 7fd778e94700 1 -- 172.21.15.148:0/3887291652 wait complete. 2023-10-10T20:20:11.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:20:11.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.487+0000 7f56462d3700 1 Processor -- start 2023-10-10T20:20:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.487+0000 7f56462d3700 1 -- start start 2023-10-10T20:20:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f56462d3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5640159e60 0x7f564015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f56462d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56400697c0 0x7f56400af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f56462d3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564015ea90 con 0x7f564015a230 2023-10-10T20:20:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f56462d3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56400d1cf0 con 0x7f5640159e60 2023-10-10T20:20:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f56462d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400b3100 con 0x7f56400697c0 2023-10-10T20:20:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5640159e60 0x7f564015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5640159e60 0x7f564015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45590/0 (socket says 172.21.15.148:45590) 2023-10-10T20:20:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 -- 172.21.15.148:0/1952972597 learned_addr learned my addr 172.21.15.148:0/1952972597 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 -- 172.21.15.148:0/1952972597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56400697c0 msgr2=0x7f56400af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 --2- 172.21.15.148:0/1952972597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56400697c0 0x7f56400af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 -- 172.21.15.148:0/1952972597 >> v1:172.21.15.161:6789/0 conn(0x7f564015a230 legacy=0x7f56400690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 -- 172.21.15.148:0/1952972597 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400d10e0 con 0x7f5640159e60 2023-10-10T20:20:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.488+0000 7f563ffff700 1 --2- 172.21.15.148:0/1952972597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5640159e60 0x7f564015e550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56300099b0 tx=0x7f5630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789acd0813ac2044 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f563effd700 1 -- 172.21.15.148:0/1952972597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563000b820 con 0x7f5640159e60 2023-10-10T20:20:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f563effd700 1 -- 172.21.15.148:0/1952972597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563000b980 con 0x7f5640159e60 2023-10-10T20:20:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f563effd700 1 -- 172.21.15.148:0/1952972597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563000bd20 con 0x7f5640159e60 2023-10-10T20:20:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 -- 172.21.15.148:0/1952972597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5640159e60 msgr2=0x7f564015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 --2- 172.21.15.148:0/1952972597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5640159e60 0x7f564015e550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56300099b0 tx=0x7f5630009250 comp rx=0 tx=0).stop 2023-10-10T20:20:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 -- 172.21.15.148:0/1952972597 shutdown_connections 2023-10-10T20:20:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 --2- 172.21.15.148:0/1952972597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5640159e60 0x7f564015e550 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 --2- 172.21.15.148:0/1952972597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56400697c0 0x7f56400af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 -- 172.21.15.148:0/1952972597 >> 172.21.15.148:0/1952972597 conn(0x7f5640070700 msgr2=0x7f5640070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 -- 172.21.15.148:0/1952972597 shutdown_connections 2023-10-10T20:20:11.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.489+0000 7f56462d3700 1 -- 172.21.15.148:0/1952972597 wait complete. 2023-10-10T20:20:11.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.490+0000 7f56462d3700 1 Processor -- start 2023-10-10T20:20:11.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 -- start start 2023-10-10T20:20:11.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564015a230 0x7f564015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f564015d7f0 0x7f56400c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564015b0a0 con 0x7f56400697c0 2023-10-10T20:20:11.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564015c070 con 0x7f564015a230 2023-10-10T20:20:11.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f56462d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564015b2b0 con 0x7f564015d7f0 2023-10-10T20:20:11.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f563ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f563ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45596/0 (socket says 172.21.15.148:45596) 2023-10-10T20:20:11.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f563ffff700 1 -- 172.21.15.148:0/3995659297 learned_addr learned my addr 172.21.15.148:0/3995659297 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:11.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f563f7fe700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564015a230 0x7f564015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.491+0000 7f5644870700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f564015d7f0 0x7f56400c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 -- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f564015d7f0 msgr2=0x7f56400c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f564015d7f0 0x7f56400c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 -- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564015a230 msgr2=0x7f564015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:11.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564015a230 0x7f564015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:11.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 -- 172.21.15.148:0/3995659297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300094f0 con 0x7f56400697c0 2023-10-10T20:20:11.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f563ffff700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5630009290 tx=0x7f5630016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630026480 con 0x7f56400697c0 2023-10-10T20:20:11.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563001f070 con 0x7f56400697c0 2023-10-10T20:20:11.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564015b590 con 0x7f56400697c0 2023-10-10T20:20:11.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630026870 con 0x7f56400697c0 2023-10-10T20:20:11.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.492+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564015bb50 con 0x7f56400697c0 2023-10-10T20:20:11.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.493+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564004c3c0 con 0x7f56400697c0 2023-10-10T20:20:11.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.493+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f563001a440 con 0x7f56400697c0 2023-10-10T20:20:11.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.494+0000 7f5627fff700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:11.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.494+0000 7f563f7fe700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:11.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.494+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5630021070 con 0x7f56400697c0 2023-10-10T20:20:11.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.494+0000 7f563f7fe700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 0x7f562c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5634006fd0 tx=0x7f5634008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:11.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.498+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5630053380 con 0x7f56400697c0 2023-10-10T20:20:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:11.725+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56400cb590 con 0x7f56400697c0 2023-10-10T20:20:12.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.716+0000 7f5627fff700 1 -- 172.21.15.148:0/3995659297 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56300569a0 con 0x7f56400697c0 2023-10-10T20:20:12.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 msgr2=0x7f562c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 0x7f562c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5634006fd0 tx=0x7f5634008040 comp rx=0 tx=0).stop 2023-10-10T20:20:12.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 msgr2=0x7f5640158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5630009290 tx=0x7f5630016000 comp rx=0 tx=0).stop 2023-10-10T20:20:12.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 shutdown_connections 2023-10-10T20:20:12.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f562c066f30 0x7f562c0693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f564015d7f0 0x7f56400c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56400697c0 0x7f5640158450 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 --2- 172.21.15.148:0/3995659297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564015a230 0x7f564015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 >> 172.21.15.148:0/3995659297 conn(0x7f5640070700 msgr2=0x7f56401592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:12.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 shutdown_connections 2023-10-10T20:20:12.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.719+0000 7f56462d3700 1 -- 172.21.15.148:0/3995659297 wait complete. 2023-10-10T20:20:12.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:20:12.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.858+0000 7f342a26a700 1 Processor -- start 2023-10-10T20:20:12.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.858+0000 7f342a26a700 1 -- start start 2023-10-10T20:20:12.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f342a26a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3424159ec0 0x7f34240af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f342a26a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f342406acb0 0x7f34240afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f342a26a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240b0790 con 0x7f342415a290 2023-10-10T20:20:12.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f342a26a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240ce1f0 con 0x7f342406acb0 2023-10-10T20:20:12.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f342a26a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240b0b20 con 0x7f3424159ec0 2023-10-10T20:20:12.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f3428807700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f342406acb0 0x7f34240afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:12.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f34237fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f342415a290 0x7f342406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59998/0 (socket says 172.21.15.148:59998) 2023-10-10T20:20:12.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.859+0000 7f34237fe700 1 -- 172.21.15.148:0/184952751 learned_addr learned my addr 172.21.15.148:0/184952751 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:12.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3423fff700 1 --2- 172.21.15.148:0/184952751 >> v2:172.21.15.161:3300/0 conn(0x7f3424159ec0 0x7f34240af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:12.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3428807700 1 -- 172.21.15.148:0/184952751 >> v2:172.21.15.161:3300/0 conn(0x7f3424159ec0 msgr2=0x7f34240af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:12.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3428807700 1 --2- 172.21.15.148:0/184952751 >> v2:172.21.15.161:3300/0 conn(0x7f3424159ec0 0x7f34240af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3422ffd700 1 -- 172.21.15.148:0/184952751 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612397249 0 0) 0x7f34240b0790 con 0x7f342415a290 2023-10-10T20:20:12.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3428807700 1 -- 172.21.15.148:0/184952751 >> v1:172.21.15.148:6789/0 conn(0x7f342415a290 legacy=0x7f342406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3428807700 1 -- 172.21.15.148:0/184952751 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240b04b0 con 0x7f342406acb0 2023-10-10T20:20:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3428807700 1 --2- 172.21.15.148:0/184952751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240afe80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f34240898e0 tx=0x7f341c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d465a8e16ad1a00a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3422ffd700 1 -- 172.21.15.148:0/184952751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00e070 con 0x7f342406acb0 2023-10-10T20:20:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3422ffd700 1 -- 172.21.15.148:0/184952751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00bc30 con 0x7f342406acb0 2023-10-10T20:20:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f3422ffd700 1 -- 172.21.15.148:0/184952751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c012430 con 0x7f342406acb0 2023-10-10T20:20:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 -- 172.21.15.148:0/184952751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 msgr2=0x7f34240afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 --2- 172.21.15.148:0/184952751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240afe80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f34240898e0 tx=0x7f341c009780 comp rx=0 tx=0).stop 2023-10-10T20:20:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 -- 172.21.15.148:0/184952751 shutdown_connections 2023-10-10T20:20:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 --2- 172.21.15.148:0/184952751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240afe80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 --2- 172.21.15.148:0/184952751 >> v2:172.21.15.161:3300/0 conn(0x7f3424159ec0 0x7f34240af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.860+0000 7f342a26a700 1 -- 172.21.15.148:0/184952751 >> 172.21.15.148:0/184952751 conn(0x7f34240d1880 msgr2=0x7f34240cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:12.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.861+0000 7f342a26a700 1 -- 172.21.15.148:0/184952751 shutdown_connections 2023-10-10T20:20:12.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.861+0000 7f342a26a700 1 -- 172.21.15.148:0/184952751 wait complete. 2023-10-10T20:20:12.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.862+0000 7f342a26a700 1 Processor -- start 2023-10-10T20:20:12.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.862+0000 7f342a26a700 1 -- start start 2023-10-10T20:20:12.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.862+0000 7f342a26a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.862+0000 7f342a26a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3424159ec0 0x7f34240c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.862+0000 7f342a26a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f342415a290 0x7f34240b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f342a26a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cb3b0 con 0x7f342406acb0 2023-10-10T20:20:12.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f342a26a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cb840 con 0x7f3424159ec0 2023-10-10T20:20:12.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f342a26a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240ca7a0 con 0x7f342415a290 2023-10-10T20:20:12.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:12.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47310/0 (socket says 172.21.15.148:47310) 2023-10-10T20:20:12.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 -- 172.21.15.148:0/927150828 learned_addr learned my addr 172.21.15.148:0/927150828 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:12.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f34237fe700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3424159ec0 0x7f34240c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:12.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 -- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f342415a290 msgr2=0x7f34240b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f342415a290 0x7f34240b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 -- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3424159ec0 msgr2=0x7f34240c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3424159ec0 0x7f34240c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 -- 172.21.15.148:0/927150828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c009a20 con 0x7f342406acb0 2023-10-10T20:20:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f3423fff700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f341400c7d0 tx=0x7f3414008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:12.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011600 con 0x7f342406acb0 2023-10-10T20:20:12.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414011760 con 0x7f342406acb0 2023-10-10T20:20:12.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240caa80 con 0x7f342406acb0 2023-10-10T20:20:12.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414011b20 con 0x7f342406acb0 2023-10-10T20:20:12.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.863+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240cb040 con 0x7f342406acb0 2023-10-10T20:20:12.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.864+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342404c3c0 con 0x7f342406acb0 2023-10-10T20:20:12.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.865+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34140193f0 con 0x7f342406acb0 2023-10-10T20:20:12.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.865+0000 7f340bfff700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:12.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.865+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3414084a50 con 0x7f342406acb0 2023-10-10T20:20:12.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.865+0000 7f34237fe700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:12.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.865+0000 7f34237fe700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 0x7f34100693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34241598e0 tx=0x7f3418009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:12.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:12.869+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3414051cc0 con 0x7f342406acb0 2023-10-10T20:20:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.095+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f34240ceab0 con 0x7f342406acb0 2023-10-10T20:20:13.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.720+0000 7f340bfff700 1 -- 172.21.15.148:0/927150828 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34140552e0 con 0x7f342406acb0 2023-10-10T20:20:13.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 msgr2=0x7f34100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 0x7f34100693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34241598e0 tx=0x7f3418009040 comp rx=0 tx=0).stop 2023-10-10T20:20:13.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 msgr2=0x7f34240c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f341400c7d0 tx=0x7f3414008040 comp rx=0 tx=0).stop 2023-10-10T20:20:13.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 shutdown_connections 2023-10-10T20:20:13.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f342415a290 0x7f34240b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f342406acb0 0x7f34240c7e60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 --2- 172.21.15.148:0/927150828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3424159ec0 0x7f34240c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.722+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 >> 172.21.15.148:0/927150828 conn(0x7f34240d1880 msgr2=0x7f3424156310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:13.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.723+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 shutdown_connections 2023-10-10T20:20:13.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.723+0000 7f342a26a700 1 -- 172.21.15.148:0/927150828 wait complete. 2023-10-10T20:20:13.738 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:13.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.852+0000 7f10a6663700 1 Processor -- start 2023-10-10T20:20:13.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.852+0000 7f10a6663700 1 -- start start 2023-10-10T20:20:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a6663700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10a0157870 0x7f10a015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a6663700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a015cbb0 con 0x7f10a0157c40 2023-10-10T20:20:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a6663700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a015bfa0 con 0x7f10a015a620 2023-10-10T20:20:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a6663700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a015c280 con 0x7f10a0157870 2023-10-10T20:20:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f109ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10a0157870 0x7f10a015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a4c00700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f10a0157c40 0x7f10a0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60012/0 (socket says 172.21.15.148:60012) 2023-10-10T20:20:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f109ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10a0157870 0x7f10a015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47316/0 (socket says 172.21.15.148:47316) 2023-10-10T20:20:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f10a4c00700 1 -- 172.21.15.148:0/1343378873 learned_addr learned my addr 172.21.15.148:0/1343378873 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f109ffff700 1 -- 172.21.15.148:0/1343378873 >> v1:172.21.15.161:6789/0 conn(0x7f10a015a620 legacy=0x7f10a015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.853+0000 7f109ffff700 1 -- 172.21.15.148:0/1343378873 >> v1:172.21.15.148:6789/0 conn(0x7f10a0157c40 legacy=0x7f10a0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109ffff700 1 -- 172.21.15.148:0/1343378873 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a015c560 con 0x7f10a0157870 2023-10-10T20:20:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109effd700 1 -- 172.21.15.148:0/1343378873 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968330797 0 0) 0x7f10a015cbb0 con 0x7f10a0157c40 2023-10-10T20:20:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109ffff700 1 --2- 172.21.15.148:0/1343378873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a0157870 0x7f10a015b6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10900098e0 tx=0x7f10900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dc265e1b468cc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109effd700 1 -- 172.21.15.148:0/1343378873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090009e80 con 0x7f10a0157870 2023-10-10T20:20:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109effd700 1 -- 172.21.15.148:0/1343378873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000b820 con 0x7f10a0157870 2023-10-10T20:20:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f109effd700 1 -- 172.21.15.148:0/1343378873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000bba0 con 0x7f10a0157870 2023-10-10T20:20:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f10a6663700 1 -- 172.21.15.148:0/1343378873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a0157870 msgr2=0x7f10a015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f10a6663700 1 --2- 172.21.15.148:0/1343378873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a0157870 0x7f10a015b6a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f10900098e0 tx=0x7f10900091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f10a6663700 1 -- 172.21.15.148:0/1343378873 shutdown_connections 2023-10-10T20:20:13.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f10a6663700 1 --2- 172.21.15.148:0/1343378873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a0157870 0x7f10a015b6a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.854+0000 7f10a6663700 1 -- 172.21.15.148:0/1343378873 >> 172.21.15.148:0/1343378873 conn(0x7f10a0068ce0 msgr2=0x7f10a00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:13.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.855+0000 7f10a6663700 1 -- 172.21.15.148:0/1343378873 shutdown_connections 2023-10-10T20:20:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.855+0000 7f10a6663700 1 -- 172.21.15.148:0/1343378873 wait complete. 2023-10-10T20:20:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 Processor -- start 2023-10-10T20:20:13.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 -- start start 2023-10-10T20:20:13.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 0x7f10a0148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:13.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10a0157c40 0x7f10a00b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:13.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:13.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b2080 con 0x7f10a015a620 2023-10-10T20:20:13.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00a77b0 con 0x7f10a0157870 2023-10-10T20:20:13.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.856+0000 7f10a6663700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00a7a90 con 0x7f10a0157c40 2023-10-10T20:20:13.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:13.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47322/0 (socket says 172.21.15.148:47322) 2023-10-10T20:20:13.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 -- 172.21.15.148:0/3403098263 learned_addr learned my addr 172.21.15.148:0/3403098263 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f109ffff700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 0x7f10a0148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f109f7fe700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10a0157c40 0x7f10a00b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 -- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10a0157c40 msgr2=0x7f10a00b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10a0157c40 0x7f10a00b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 -- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 msgr2=0x7f10a0148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 0x7f10a0148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 -- 172.21.15.148:0/3403098263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009420 con 0x7f10a015a620 2023-10-10T20:20:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f109ffff700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 0x7f10a0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a4c00700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f109800ccc0 tx=0x7f10980019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800ce70 con 0x7f10a015a620 2023-10-10T20:20:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109800ab70 con 0x7f10a015a620 2023-10-10T20:20:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a00a7d70 con 0x7f10a015a620 2023-10-10T20:20:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10980070f0 con 0x7f10a015a620 2023-10-10T20:20:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.857+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a0148e50 con 0x7f10a015a620 2023-10-10T20:20:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.858+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a004c3c0 con 0x7f10a015a620 2023-10-10T20:20:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.859+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10980074a0 con 0x7f10a015a620 2023-10-10T20:20:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.859+0000 7f1087fff700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.859+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1098084920 con 0x7f10a015a620 2023-10-10T20:20:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.859+0000 7f109ffff700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.860+0000 7f109ffff700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 0x7f108c0693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10900030c0 tx=0x7f1090016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:13.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:13.863+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1098051c10 con 0x7f10a015a620 2023-10-10T20:20:14.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.085+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a015c840 con 0x7f10a015a620 2023-10-10T20:20:14.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.085+0000 7f1087fff700 1 -- 172.21.15.148:0/3403098263 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f1098055230 con 0x7f10a015a620 2023-10-10T20:20:14.088 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:14.088 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:14.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 msgr2=0x7f108c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 0x7f108c0693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10900030c0 tx=0x7f1090016000 comp rx=0 tx=0).stop 2023-10-10T20:20:14.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 msgr2=0x7f10a00b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f109800ccc0 tx=0x7f10980019e0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 shutdown_connections 2023-10-10T20:20:14.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f108c066f30 0x7f108c0693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10a015a620 0x7f10a00b2b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0157870 0x7f10a0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 --2- 172.21.15.148:0/3403098263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10a0157c40 0x7f10a00b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 >> 172.21.15.148:0/3403098263 conn(0x7f10a0068ce0 msgr2=0x7f10a00d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 shutdown_connections 2023-10-10T20:20:14.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.087+0000 7f10a6663700 1 -- 172.21.15.148:0/3403098263 wait complete. 2023-10-10T20:20:14.092 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:14.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:14.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.214+0000 7f44b1205700 1 Processor -- start 2023-10-10T20:20:14.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.214+0000 7f44b1205700 1 -- start start 2023-10-10T20:20:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44b1205700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44ac157870 0x7f44ac15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44b1205700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 0x7f44ac158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44b1205700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ac15cbb0 con 0x7f44ac15a620 2023-10-10T20:20:14.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44b1205700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ac15bfa0 con 0x7f44ac157870 2023-10-10T20:20:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44b1205700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ac15c280 con 0x7f44ac157c40 2023-10-10T20:20:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44ac157870 0x7f44ac15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44ac157870 0x7f44ac15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47332/0 (socket says 172.21.15.148:47332) 2023-10-10T20:20:14.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 -- 172.21.15.148:0/2454289938 learned_addr learned my addr 172.21.15.148:0/2454289938 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44ab59e700 1 --2- 172.21.15.148:0/2454289938 >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 0x7f44ac158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 -- 172.21.15.148:0/2454289938 >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 msgr2=0x7f44ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 --2- 172.21.15.148:0/2454289938 >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 0x7f44ac158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 -- 172.21.15.148:0/2454289938 >> v1:172.21.15.161:6789/0 conn(0x7f44ac15a620 legacy=0x7f44ac15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44aad9d700 1 -- 172.21.15.148:0/2454289938 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac15c560 con 0x7f44ac157870 2023-10-10T20:20:14.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.215+0000 7f44ab59e700 1 --2- 172.21.15.148:0/2454289938 >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 0x7f44ac158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:14.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44aad9d700 1 --2- 172.21.15.148:0/2454289938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157870 0x7f44ac15b6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06b4e0 tx=0x7f449c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd202742bf65c39e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44aa59c700 1 -- 172.21.15.148:0/2454289938 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00b860 con 0x7f44ac157870 2023-10-10T20:20:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44aa59c700 1 -- 172.21.15.148:0/2454289938 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00b9c0 con 0x7f44ac157870 2023-10-10T20:20:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44aa59c700 1 -- 172.21.15.148:0/2454289938 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00bd60 con 0x7f44ac157870 2023-10-10T20:20:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 -- 172.21.15.148:0/2454289938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157870 msgr2=0x7f44ac15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 --2- 172.21.15.148:0/2454289938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157870 0x7f44ac15b6a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06b4e0 tx=0x7f449c009250 comp rx=0 tx=0).stop 2023-10-10T20:20:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 -- 172.21.15.148:0/2454289938 shutdown_connections 2023-10-10T20:20:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 --2- 172.21.15.148:0/2454289938 >> v2:172.21.15.161:3300/0 conn(0x7f44ac157c40 0x7f44ac158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 --2- 172.21.15.148:0/2454289938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157870 0x7f44ac15b6a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 -- 172.21.15.148:0/2454289938 >> 172.21.15.148:0/2454289938 conn(0x7f44ac068ce0 msgr2=0x7f44ac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 -- 172.21.15.148:0/2454289938 shutdown_connections 2023-10-10T20:20:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.216+0000 7f44b1205700 1 -- 172.21.15.148:0/2454289938 wait complete. 2023-10-10T20:20:14.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.217+0000 7f44b1205700 1 Processor -- start 2023-10-10T20:20:14.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.217+0000 7f44b1205700 1 -- start start 2023-10-10T20:20:14.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44ac157870 0x7f44ac16b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac15a620 0x7f44ac16db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac150f90 con 0x7f44ac157c40 2023-10-10T20:20:14.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac154790 con 0x7f44ac15a620 2023-10-10T20:20:14.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44b1205700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac155030 con 0x7f44ac157870 2023-10-10T20:20:14.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44a3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44a3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47340/0 (socket says 172.21.15.148:47340) 2023-10-10T20:20:14.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44a3fff700 1 -- 172.21.15.148:0/1683720845 learned_addr learned my addr 172.21.15.148:0/1683720845 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:14.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44aad9d700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44ac157870 0x7f44ac16b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.218+0000 7f44ab59e700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac15a620 0x7f44ac16db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 -- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44ac157870 msgr2=0x7f44ac16b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44ac157870 0x7f44ac16b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 -- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac15a620 msgr2=0x7f44ac16db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac15a620 0x7f44ac16db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 -- 172.21.15.148:0/1683720845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0094f0 con 0x7f44ac157c40 2023-10-10T20:20:14.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a3fff700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f449400c5c0 tx=0x7f4494007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494011590 con 0x7f44ac157c40 2023-10-10T20:20:14.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494011710 con 0x7f44ac157c40 2023-10-10T20:20:14.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac1543d0 con 0x7f44ac157c40 2023-10-10T20:20:14.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494011ad0 con 0x7f44ac157c40 2023-10-10T20:20:14.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.219+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac154cc0 con 0x7f44ac157c40 2023-10-10T20:20:14.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.220+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ac04c3c0 con 0x7f44ac157c40 2023-10-10T20:20:14.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.220+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f449401b3b0 con 0x7f44ac157c40 2023-10-10T20:20:14.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.221+0000 7f44a2ffd700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.221+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4494083860 con 0x7f44ac157c40 2023-10-10T20:20:14.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.221+0000 7f44aad9d700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.221+0000 7f44aad9d700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 0x7f448c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06b4e0 tx=0x7f449c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.225+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4494050ad0 con 0x7f44ac157c40 2023-10-10T20:20:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.409+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44ac155170 con 0x7f44ac157c40 2023-10-10T20:20:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.410+0000 7f44a2ffd700 1 -- 172.21.15.148:0/1683720845 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f4494017070 con 0x7f44ac157c40 2023-10-10T20:20:14.412 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:14.412 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":31,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:11.472571+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 msgr2=0x7f448c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 0x7f448c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06b4e0 tx=0x7f449c017000 comp rx=0 tx=0).stop 2023-10-10T20:20:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 msgr2=0x7f44ac153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f449400c5c0 tx=0x7f4494007f50 comp rx=0 tx=0).stop 2023-10-10T20:20:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 shutdown_connections 2023-10-10T20:20:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44ac15a620 0x7f44ac16db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44ac157870 0x7f44ac16b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 --2- 172.21.15.148:0/1683720845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ac157c40 0x7f44ac153980 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 >> 172.21.15.148:0/1683720845 conn(0x7f44ac068ce0 msgr2=0x7f44ac0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 shutdown_connections 2023-10-10T20:20:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.412+0000 7f44b1205700 1 -- 172.21.15.148:0/1683720845 wait complete. 2023-10-10T20:20:14.426 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:20:14.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:20:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.539+0000 7f0440cb2700 1 Processor -- start 2023-10-10T20:20:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.539+0000 7f0440cb2700 1 -- start start 2023-10-10T20:20:14.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f043c157870 0x7f043c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c15d580 0x7f043c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c15cbb0 con 0x7f043c15d580 2023-10-10T20:20:14.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c15bfa0 con 0x7f043c157870 2023-10-10T20:20:14.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0440cb2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c15c280 con 0x7f043c15a620 2023-10-10T20:20:14.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c15d580 0x7f043c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c15d580 0x7f043c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47356/0 (socket says 172.21.15.148:47356) 2023-10-10T20:20:14.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 -- 172.21.15.148:0/3833162316 learned_addr learned my addr 172.21.15.148:0/3833162316 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:14.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043a59c700 1 --2- 172.21.15.148:0/3833162316 >> v2:172.21.15.161:3300/0 conn(0x7f043c157870 0x7f043c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f0431d9b700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 -- 172.21.15.148:0/3833162316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 msgr2=0x7f043c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:14.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 -- 172.21.15.148:0/3833162316 >> v2:172.21.15.161:3300/0 conn(0x7f043c157870 msgr2=0x7f043c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:14.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 --2- 172.21.15.148:0/3833162316 >> v2:172.21.15.161:3300/0 conn(0x7f043c157870 0x7f043c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.540+0000 7f043ad9d700 1 -- 172.21.15.148:0/3833162316 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c15c560 con 0x7f043c15d580 2023-10-10T20:20:14.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f043ad9d700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c15d580 0x7f043c158110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f043c069d80 tx=0x7f0434009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d701ec2622692204 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0439d9b700 1 -- 172.21.15.148:0/3833162316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043400e070 con 0x7f043c15d580 2023-10-10T20:20:14.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0439d9b700 1 -- 172.21.15.148:0/3833162316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043400b970 con 0x7f043c15d580 2023-10-10T20:20:14.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0439d9b700 1 -- 172.21.15.148:0/3833162316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340092e0 con 0x7f043c15d580 2023-10-10T20:20:14.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 -- 172.21.15.148:0/3833162316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c15d580 msgr2=0x7f043c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c15d580 0x7f043c158110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f043c069d80 tx=0x7f0434009840 comp rx=0 tx=0).stop 2023-10-10T20:20:14.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 -- 172.21.15.148:0/3833162316 shutdown_connections 2023-10-10T20:20:14.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c15d580 0x7f043c158110 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3833162316 >> v2:172.21.15.161:3300/0 conn(0x7f043c157870 0x7f043c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3833162316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 -- 172.21.15.148:0/3833162316 >> 172.21.15.148:0/3833162316 conn(0x7f043c068ce0 msgr2=0x7f043c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 -- 172.21.15.148:0/3833162316 shutdown_connections 2023-10-10T20:20:14.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.541+0000 7f0440cb2700 1 -- 172.21.15.148:0/3833162316 wait complete. 2023-10-10T20:20:14.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.542+0000 7f0440cb2700 1 Processor -- start 2023-10-10T20:20:14.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 -- start start 2023-10-10T20:20:14.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c15d580 0x7f043c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c14b7a0 con 0x7f043c157870 2023-10-10T20:20:14.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c14b9b0 con 0x7f043c15d580 2023-10-10T20:20:14.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f0440cb2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c14bc90 con 0x7f043c15a620 2023-10-10T20:20:14.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f043a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f043a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47364/0 (socket says 172.21.15.148:47364) 2023-10-10T20:20:14.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.543+0000 7f043a59c700 1 -- 172.21.15.148:0/3307605431 learned_addr learned my addr 172.21.15.148:0/3307605431 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:14.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0431d9b700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043ad9d700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c15d580 0x7f043c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 -- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 msgr2=0x7f043c14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c14b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 -- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c15d580 msgr2=0x7f043c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c15d580 0x7f043c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 -- 172.21.15.148:0/3307605431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009ae0 con 0x7f043c157870 2023-10-10T20:20:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f043a59c700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f042c00ec80 tx=0x7f042c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00c9d0 con 0x7f043c157870 2023-10-10T20:20:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00cb30 con 0x7f043c157870 2023-10-10T20:20:14.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c14bf70 con 0x7f043c157870 2023-10-10T20:20:14.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c013470 con 0x7f043c157870 2023-10-10T20:20:14.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.544+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c0b18a0 con 0x7f043c157870 2023-10-10T20:20:14.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.545+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c0b1c10 con 0x7f043c157870 2023-10-10T20:20:14.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.545+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f042c013820 con 0x7f043c157870 2023-10-10T20:20:14.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.546+0000 7f0432ffd700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 0x7f041c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.546+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f042c0848c0 con 0x7f043c157870 2023-10-10T20:20:14.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.546+0000 7f0431d9b700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 0x7f041c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.546+0000 7f0431d9b700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 0x7f041c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f043c0b14f0 tx=0x7f0424008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.550+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f042c051b30 con 0x7f043c157870 2023-10-10T20:20:14.665 INFO:tasks.ceph.mds.d.smithi161.stderr:2023-10-10T20:20:14.664+0000 7f9d68f45700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:20:14.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.730+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f043c04c3c0 con 0x7f041c066f30 2023-10-10T20:20:14.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.738+0000 7f0432ffd700 1 -- 172.21.15.148:0/3307605431 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+269164 (secure 0 0 0) 0x7f043c04c3c0 con 0x7f041c066f30 2023-10-10T20:20:14.743 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 msgr2=0x7f041c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 0x7f041c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f043c0b14f0 tx=0x7f0424008040 comp rx=0 tx=0).stop 2023-10-10T20:20:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 msgr2=0x7f043c0d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f042c00ec80 tx=0x7f042c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 shutdown_connections 2023-10-10T20:20:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f041c066f30 0x7f041c0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c15d580 0x7f043c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c157870 0x7f043c0d07e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 --2- 172.21.15.148:0/3307605431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f043c15a620 0x7f043c14b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 >> 172.21.15.148:0/3307605431 conn(0x7f043c068ce0 msgr2=0x7f043c14c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 shutdown_connections 2023-10-10T20:20:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.741+0000 7f0440cb2700 1 -- 172.21.15.148:0/3307605431 wait complete. 2023-10-10T20:20:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:20:14.755 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-10T20:20:13.333194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2174,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560325,"statfs":{"total":1600000000,"available":1597773394,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.664088"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483586+0000","last_change":"2023-10-10T20:20:10.483397+0000","last_active":"2023-10-10T20:20:10.483586+0000","last_peered":"2023-10-10T20:20:10.483586+0000","last_clean":"2023-10-10T20:20:10.483586+0000","last_became_active":"2023-10-10T20:20:10.483082+0000","last_became_peered":"2023-10-10T20:20:10.483082+0000","last_unstale":"2023-10-10T20:20:10.483586+0000","last_undegraded":"2023-10-10T20:20:10.483586+0000","last_fullsized":"2023-10-10T20:20:10.483586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487381+0000","last_change":"2023-10-10T20:20:10.487300+0000","last_active":"2023-10-10T20:20:10.487381+0000","last_peered":"2023-10-10T20:20:10.487381+0000","last_clean":"2023-10-10T20:20:10.487381+0000","last_became_active":"2023-10-10T20:20:10.487087+0000","last_became_peered":"2023-10-10T20:20:10.487087+0000","last_unstale":"2023-10-10T20:20:10.487381+0000","last_undegraded":"2023-10-10T20:20:10.487381+0000","last_fullsized":"2023-10-10T20:20:10.487381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486319+0000","last_change":"2023-10-10T20:20:10.486082+0000","last_active":"2023-10-10T20:20:10.486319+0000","last_peered":"2023-10-10T20:20:10.486319+0000","last_clean":"2023-10-10T20:20:10.486319+0000","last_became_active":"2023-10-10T20:20:10.485541+0000","last_became_peered":"2023-10-10T20:20:10.485541+0000","last_unstale":"2023-10-10T20:20:10.486319+0000","last_undegraded":"2023-10-10T20:20:10.486319+0000","last_fullsized":"2023-10-10T20:20:10.486319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:52.113199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487417+0000","last_change":"2023-10-10T20:20:10.487289+0000","last_active":"2023-10-10T20:20:10.487417+0000","last_peered":"2023-10-10T20:20:10.487417+0000","last_clean":"2023-10-10T20:20:10.487417+0000","last_became_active":"2023-10-10T20:20:10.486754+0000","last_became_peered":"2023-10-10T20:20:10.486754+0000","last_unstale":"2023-10-10T20:20:10.487417+0000","last_undegraded":"2023-10-10T20:20:10.487417+0000","last_fullsized":"2023-10-10T20:20:10.487417+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:27.703370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484818+0000","last_change":"2023-10-10T20:20:10.484711+0000","last_active":"2023-10-10T20:20:10.484818+0000","last_peered":"2023-10-10T20:20:10.484818+0000","last_clean":"2023-10-10T20:20:10.484818+0000","last_became_active":"2023-10-10T20:20:10.484485+0000","last_became_peered":"2023-10-10T20:20:10.484485+0000","last_unstale":"2023-10-10T20:20:10.484818+0000","last_undegraded":"2023-10-10T20:20:10.484818+0000","last_fullsized":"2023-10-10T20:20:10.484818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:49.483923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485688+0000","last_change":"2023-10-10T20:20:10.485611+0000","last_active":"2023-10-10T20:20:10.485688+0000","last_peered":"2023-10-10T20:20:10.485688+0000","last_clean":"2023-10-10T20:20:10.485688+0000","last_became_active":"2023-10-10T20:20:10.485202+0000","last_became_peered":"2023-10-10T20:20:10.485202+0000","last_unstale":"2023-10-10T20:20:10.485688+0000","last_undegraded":"2023-10-10T20:20:10.485688+0000","last_fullsized":"2023-10-10T20:20:10.485688+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486823+0000","last_change":"2023-10-10T20:20:10.486703+0000","last_active":"2023-10-10T20:20:10.486823+0000","last_peered":"2023-10-10T20:20:10.486823+0000","last_clean":"2023-10-10T20:20:10.486823+0000","last_became_active":"2023-10-10T20:20:10.483848+0000","last_became_peered":"2023-10-10T20:20:10.483848+0000","last_unstale":"2023-10-10T20:20:10.486823+0000","last_undegraded":"2023-10-10T20:20:10.486823+0000","last_fullsized":"2023-10-10T20:20:10.486823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489336+0000","last_change":"2023-10-10T20:20:10.489227+0000","last_active":"2023-10-10T20:20:10.489336+0000","last_peered":"2023-10-10T20:20:10.489336+0000","last_clean":"2023-10-10T20:20:10.489336+0000","last_became_active":"2023-10-10T20:20:10.488575+0000","last_became_peered":"2023-10-10T20:20:10.488575+0000","last_unstale":"2023-10-10T20:20:10.489336+0000","last_undegraded":"2023-10-10T20:20:10.489336+0000","last_fullsized":"2023-10-10T20:20:10.489336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486395+0000","last_change":"2023-10-10T20:20:10.485278+0000","last_active":"2023-10-10T20:20:10.486395+0000","last_peered":"2023-10-10T20:20:10.486395+0000","last_clean":"2023-10-10T20:20:10.486395+0000","last_became_active":"2023-10-10T20:20:10.484905+0000","last_became_peered":"2023-10-10T20:20:10.484905+0000","last_unstale":"2023-10-10T20:20:10.486395+0000","last_undegraded":"2023-10-10T20:20:10.486395+0000","last_fullsized":"2023-10-10T20:20:10.486395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:35.987508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485866+0000","last_change":"2023-10-10T20:20:10.485807+0000","last_active":"2023-10-10T20:20:10.485866+0000","last_peered":"2023-10-10T20:20:10.485866+0000","last_clean":"2023-10-10T20:20:10.485866+0000","last_became_active":"2023-10-10T20:20:10.485589+0000","last_became_peered":"2023-10-10T20:20:10.485589+0000","last_unstale":"2023-10-10T20:20:10.485866+0000","last_undegraded":"2023-10-10T20:20:10.485866+0000","last_fullsized":"2023-10-10T20:20:10.485866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490869+0000","last_change":"2023-10-10T20:20:10.490769+0000","last_active":"2023-10-10T20:20:10.490869+0000","last_peered":"2023-10-10T20:20:10.490869+0000","last_clean":"2023-10-10T20:20:10.490869+0000","last_became_active":"2023-10-10T20:20:10.490308+0000","last_became_peered":"2023-10-10T20:20:10.490308+0000","last_unstale":"2023-10-10T20:20:10.490869+0000","last_undegraded":"2023-10-10T20:20:10.490869+0000","last_fullsized":"2023-10-10T20:20:10.490869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:03.488531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485213+0000","last_change":"2023-10-10T20:20:10.485131+0000","last_active":"2023-10-10T20:20:10.485213+0000","last_peered":"2023-10-10T20:20:10.485213+0000","last_clean":"2023-10-10T20:20:10.485213+0000","last_became_active":"2023-10-10T20:20:10.484979+0000","last_became_peered":"2023-10-10T20:20:10.484979+0000","last_unstale":"2023-10-10T20:20:10.485213+0000","last_undegraded":"2023-10-10T20:20:10.485213+0000","last_fullsized":"2023-10-10T20:20:10.485213+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486894+0000","last_change":"2023-10-10T20:20:10.486798+0000","last_active":"2023-10-10T20:20:10.486894+0000","last_peered":"2023-10-10T20:20:10.486894+0000","last_clean":"2023-10-10T20:20:10.486894+0000","last_became_active":"2023-10-10T20:20:10.486580+0000","last_became_peered":"2023-10-10T20:20:10.486580+0000","last_unstale":"2023-10-10T20:20:10.486894+0000","last_undegraded":"2023-10-10T20:20:10.486894+0000","last_fullsized":"2023-10-10T20:20:10.486894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.543235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482832+0000","last_change":"2023-10-10T20:20:10.482582+0000","last_active":"2023-10-10T20:20:10.482832+0000","last_peered":"2023-10-10T20:20:10.482832+0000","last_clean":"2023-10-10T20:20:10.482832+0000","last_became_active":"2023-10-10T20:20:10.482370+0000","last_became_peered":"2023-10-10T20:20:10.482370+0000","last_unstale":"2023-10-10T20:20:10.482832+0000","last_undegraded":"2023-10-10T20:20:10.482832+0000","last_fullsized":"2023-10-10T20:20:10.482832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482095+0000","last_change":"2023-10-10T20:20:10.481846+0000","last_active":"2023-10-10T20:20:10.482095+0000","last_peered":"2023-10-10T20:20:10.482095+0000","last_clean":"2023-10-10T20:20:10.482095+0000","last_became_active":"2023-10-10T20:20:10.481609+0000","last_became_peered":"2023-10-10T20:20:10.481609+0000","last_unstale":"2023-10-10T20:20:10.482095+0000","last_undegraded":"2023-10-10T20:20:10.482095+0000","last_fullsized":"2023-10-10T20:20:10.482095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.491003+0000","last_change":"2023-10-10T20:20:10.490952+0000","last_active":"2023-10-10T20:20:10.491003+0000","last_peered":"2023-10-10T20:20:10.491003+0000","last_clean":"2023-10-10T20:20:10.491003+0000","last_became_active":"2023-10-10T20:20:10.490544+0000","last_became_peered":"2023-10-10T20:20:10.490544+0000","last_unstale":"2023-10-10T20:20:10.491003+0000","last_undegraded":"2023-10-10T20:20:10.491003+0000","last_fullsized":"2023-10-10T20:20:10.491003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:50.684042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486928+0000","last_change":"2023-10-10T20:20:10.486878+0000","last_active":"2023-10-10T20:20:10.486928+0000","last_peered":"2023-10-10T20:20:10.486928+0000","last_clean":"2023-10-10T20:20:10.486928+0000","last_became_active":"2023-10-10T20:20:10.484348+0000","last_became_peered":"2023-10-10T20:20:10.484348+0000","last_unstale":"2023-10-10T20:20:10.486928+0000","last_undegraded":"2023-10-10T20:20:10.486928+0000","last_fullsized":"2023-10-10T20:20:10.486928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:21.378539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486574+0000","last_change":"2023-10-10T20:20:10.486516+0000","last_active":"2023-10-10T20:20:10.486574+0000","last_peered":"2023-10-10T20:20:10.486574+0000","last_clean":"2023-10-10T20:20:10.486574+0000","last_became_active":"2023-10-10T20:20:10.486303+0000","last_became_peered":"2023-10-10T20:20:10.486303+0000","last_unstale":"2023-10-10T20:20:10.486574+0000","last_undegraded":"2023-10-10T20:20:10.486574+0000","last_fullsized":"2023-10-10T20:20:10.486574+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:52.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485043+0000","last_change":"2023-10-10T20:20:10.484974+0000","last_active":"2023-10-10T20:20:10.485043+0000","last_peered":"2023-10-10T20:20:10.485043+0000","last_clean":"2023-10-10T20:20:10.485043+0000","last_became_active":"2023-10-10T20:20:10.484485+0000","last_became_peered":"2023-10-10T20:20:10.484485+0000","last_unstale":"2023-10-10T20:20:10.485043+0000","last_undegraded":"2023-10-10T20:20:10.485043+0000","last_fullsized":"2023-10-10T20:20:10.485043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:26:46.604639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486057+0000","last_change":"2023-10-10T20:20:10.485987+0000","last_active":"2023-10-10T20:20:10.486057+0000","last_peered":"2023-10-10T20:20:10.486057+0000","last_clean":"2023-10-10T20:20:10.486057+0000","last_became_active":"2023-10-10T20:20:10.485782+0000","last_became_peered":"2023-10-10T20:20:10.485782+0000","last_unstale":"2023-10-10T20:20:10.486057+0000","last_undegraded":"2023-10-10T20:20:10.486057+0000","last_fullsized":"2023-10-10T20:20:10.486057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:57.013943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.477139+0000","last_change":"2023-10-10T20:20:10.476986+0000","last_active":"2023-10-10T20:20:10.477139+0000","last_peered":"2023-10-10T20:20:10.477139+0000","last_clean":"2023-10-10T20:20:10.477139+0000","last_became_active":"2023-10-10T20:20:10.476719+0000","last_became_peered":"2023-10-10T20:20:10.476719+0000","last_unstale":"2023-10-10T20:20:10.477139+0000","last_undegraded":"2023-10-10T20:20:10.477139+0000","last_fullsized":"2023-10-10T20:20:10.477139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:18.079608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488524+0000","last_change":"2023-10-10T20:20:10.488342+0000","last_active":"2023-10-10T20:20:10.488524+0000","last_peered":"2023-10-10T20:20:10.488524+0000","last_clean":"2023-10-10T20:20:10.488524+0000","last_became_active":"2023-10-10T20:20:10.488112+0000","last_became_peered":"2023-10-10T20:20:10.488112+0000","last_unstale":"2023-10-10T20:20:10.488524+0000","last_undegraded":"2023-10-10T20:20:10.488524+0000","last_fullsized":"2023-10-10T20:20:10.488524+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489216+0000","last_change":"2023-10-10T20:20:10.489176+0000","last_active":"2023-10-10T20:20:10.489216+0000","last_peered":"2023-10-10T20:20:10.489216+0000","last_clean":"2023-10-10T20:20:10.489216+0000","last_became_active":"2023-10-10T20:20:10.489039+0000","last_became_peered":"2023-10-10T20:20:10.489039+0000","last_unstale":"2023-10-10T20:20:10.489216+0000","last_undegraded":"2023-10-10T20:20:10.489216+0000","last_fullsized":"2023-10-10T20:20:10.489216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482246+0000","last_change":"2023-10-10T20:20:10.482124+0000","last_active":"2023-10-10T20:20:10.482246+0000","last_peered":"2023-10-10T20:20:10.482246+0000","last_clean":"2023-10-10T20:20:10.482246+0000","last_became_active":"2023-10-10T20:20:10.481869+0000","last_became_peered":"2023-10-10T20:20:10.481869+0000","last_unstale":"2023-10-10T20:20:10.482246+0000","last_undegraded":"2023-10-10T20:20:10.482246+0000","last_fullsized":"2023-10-10T20:20:10.482246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488491+0000","last_change":"2023-10-10T20:20:10.488415+0000","last_active":"2023-10-10T20:20:10.488491+0000","last_peered":"2023-10-10T20:20:10.488491+0000","last_clean":"2023-10-10T20:20:10.488491+0000","last_became_active":"2023-10-10T20:20:10.488171+0000","last_became_peered":"2023-10-10T20:20:10.488171+0000","last_unstale":"2023-10-10T20:20:10.488491+0000","last_undegraded":"2023-10-10T20:20:10.488491+0000","last_fullsized":"2023-10-10T20:20:10.488491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:48.655717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490619+0000","last_change":"2023-10-10T20:20:10.490562+0000","last_active":"2023-10-10T20:20:10.490619+0000","last_peered":"2023-10-10T20:20:10.490619+0000","last_clean":"2023-10-10T20:20:10.490619+0000","last_became_active":"2023-10-10T20:20:10.490303+0000","last_became_peered":"2023-10-10T20:20:10.490303+0000","last_unstale":"2023-10-10T20:20:10.490619+0000","last_undegraded":"2023-10-10T20:20:10.490619+0000","last_fullsized":"2023-10-10T20:20:10.490619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488296+0000","last_change":"2023-10-10T20:20:10.488102+0000","last_active":"2023-10-10T20:20:10.488296+0000","last_peered":"2023-10-10T20:20:10.488296+0000","last_clean":"2023-10-10T20:20:10.488296+0000","last_became_active":"2023-10-10T20:20:10.487628+0000","last_became_peered":"2023-10-10T20:20:10.487628+0000","last_unstale":"2023-10-10T20:20:10.488296+0000","last_undegraded":"2023-10-10T20:20:10.488296+0000","last_fullsized":"2023-10-10T20:20:10.488296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479728+0000","last_change":"2023-10-10T20:20:10.478212+0000","last_active":"2023-10-10T20:20:10.479728+0000","last_peered":"2023-10-10T20:20:10.479728+0000","last_clean":"2023-10-10T20:20:10.479728+0000","last_became_active":"2023-10-10T20:20:10.477981+0000","last_became_peered":"2023-10-10T20:20:10.477981+0000","last_unstale":"2023-10-10T20:20:10.479728+0000","last_undegraded":"2023-10-10T20:20:10.479728+0000","last_fullsized":"2023-10-10T20:20:10.479728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:50.684042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483084+0000","last_change":"2023-10-10T20:20:10.482284+0000","last_active":"2023-10-10T20:20:10.483084+0000","last_peered":"2023-10-10T20:20:10.483084+0000","last_clean":"2023-10-10T20:20:10.483084+0000","last_became_active":"2023-10-10T20:20:10.482042+0000","last_became_peered":"2023-10-10T20:20:10.482042+0000","last_unstale":"2023-10-10T20:20:10.483084+0000","last_undegraded":"2023-10-10T20:20:10.483084+0000","last_fullsized":"2023-10-10T20:20:10.483084+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:29.917088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488995+0000","last_change":"2023-10-10T20:20:10.488910+0000","last_active":"2023-10-10T20:20:10.488995+0000","last_peered":"2023-10-10T20:20:10.488995+0000","last_clean":"2023-10-10T20:20:10.488995+0000","last_became_active":"2023-10-10T20:20:10.488005+0000","last_became_peered":"2023-10-10T20:20:10.488005+0000","last_unstale":"2023-10-10T20:20:10.488995+0000","last_undegraded":"2023-10-10T20:20:10.488995+0000","last_fullsized":"2023-10-10T20:20:10.488995+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482544+0000","last_change":"2023-10-10T20:20:10.481673+0000","last_active":"2023-10-10T20:20:10.482544+0000","last_peered":"2023-10-10T20:20:10.482544+0000","last_clean":"2023-10-10T20:20:10.482544+0000","last_became_active":"2023-10-10T20:20:10.481424+0000","last_became_peered":"2023-10-10T20:20:10.481424+0000","last_unstale":"2023-10-10T20:20:10.482544+0000","last_undegraded":"2023-10-10T20:20:10.482544+0000","last_fullsized":"2023-10-10T20:20:10.482544+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489719+0000","last_change":"2023-10-10T20:20:10.489648+0000","last_active":"2023-10-10T20:20:10.489719+0000","last_peered":"2023-10-10T20:20:10.489719+0000","last_clean":"2023-10-10T20:20:10.489719+0000","last_became_active":"2023-10-10T20:20:10.489452+0000","last_became_peered":"2023-10-10T20:20:10.489452+0000","last_unstale":"2023-10-10T20:20:10.489719+0000","last_undegraded":"2023-10-10T20:20:10.489719+0000","last_fullsized":"2023-10-10T20:20:10.489719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488493+0000","last_change":"2023-10-10T20:20:10.488399+0000","last_active":"2023-10-10T20:20:10.488493+0000","last_peered":"2023-10-10T20:20:10.488493+0000","last_clean":"2023-10-10T20:20:10.488493+0000","last_became_active":"2023-10-10T20:20:10.488186+0000","last_became_peered":"2023-10-10T20:20:10.488186+0000","last_unstale":"2023-10-10T20:20:10.488493+0000","last_undegraded":"2023-10-10T20:20:10.488493+0000","last_fullsized":"2023-10-10T20:20:10.488493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701186+0000","last_change":"2023-10-10T20:20:09.681326+0000","last_active":"2023-10-10T20:20:10.701186+0000","last_peered":"2023-10-10T20:20:10.701186+0000","last_clean":"2023-10-10T20:20:10.701186+0000","last_became_active":"2023-10-10T20:20:09.681071+0000","last_became_peered":"2023-10-10T20:20:09.681071+0000","last_unstale":"2023-10-10T20:20:10.701186+0000","last_undegraded":"2023-10-10T20:20:10.701186+0000","last_fullsized":"2023-10-10T20:20:10.701186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.480687+0000","last_change":"2023-10-10T20:20:10.480494+0000","last_active":"2023-10-10T20:20:10.480687+0000","last_peered":"2023-10-10T20:20:10.480687+0000","last_clean":"2023-10-10T20:20:10.480687+0000","last_became_active":"2023-10-10T20:20:10.480248+0000","last_became_peered":"2023-10-10T20:20:10.480248+0000","last_unstale":"2023-10-10T20:20:10.480687+0000","last_undegraded":"2023-10-10T20:20:10.480687+0000","last_fullsized":"2023-10-10T20:20:10.480687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:18.079608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700142+0000","last_change":"2023-10-10T20:20:09.682229+0000","last_active":"2023-10-10T20:20:10.700142+0000","last_peered":"2023-10-10T20:20:10.700142+0000","last_clean":"2023-10-10T20:20:10.700142+0000","last_became_active":"2023-10-10T20:20:09.682000+0000","last_became_peered":"2023-10-10T20:20:09.682000+0000","last_unstale":"2023-10-10T20:20:10.700142+0000","last_undegraded":"2023-10-10T20:20:10.700142+0000","last_fullsized":"2023-10-10T20:20:10.700142+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483101+0000","last_change":"2023-10-10T20:20:10.483015+0000","last_active":"2023-10-10T20:20:10.483101+0000","last_peered":"2023-10-10T20:20:10.483101+0000","last_clean":"2023-10-10T20:20:10.483101+0000","last_became_active":"2023-10-10T20:20:10.482771+0000","last_became_peered":"2023-10-10T20:20:10.482771+0000","last_unstale":"2023-10-10T20:20:10.483101+0000","last_undegraded":"2023-10-10T20:20:10.483101+0000","last_fullsized":"2023-10-10T20:20:10.483101+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701592+0000","last_change":"2023-10-10T20:20:09.678445+0000","last_active":"2023-10-10T20:20:10.701592+0000","last_peered":"2023-10-10T20:20:10.701592+0000","last_clean":"2023-10-10T20:20:10.701592+0000","last_became_active":"2023-10-10T20:20:09.678228+0000","last_became_peered":"2023-10-10T20:20:09.678228+0000","last_unstale":"2023-10-10T20:20:10.701592+0000","last_undegraded":"2023-10-10T20:20:10.701592+0000","last_fullsized":"2023-10-10T20:20:10.701592+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:17.077946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484418+0000","last_change":"2023-10-10T20:20:10.483850+0000","last_active":"2023-10-10T20:20:10.484418+0000","last_peered":"2023-10-10T20:20:10.484418+0000","last_clean":"2023-10-10T20:20:10.484418+0000","last_became_active":"2023-10-10T20:20:10.483639+0000","last_became_peered":"2023-10-10T20:20:10.483639+0000","last_unstale":"2023-10-10T20:20:10.484418+0000","last_undegraded":"2023-10-10T20:20:10.484418+0000","last_fullsized":"2023-10-10T20:20:10.484418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:21.378539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700860+0000","last_change":"2023-10-10T20:20:09.679748+0000","last_active":"2023-10-10T20:20:10.700860+0000","last_peered":"2023-10-10T20:20:10.700860+0000","last_clean":"2023-10-10T20:20:10.700860+0000","last_became_active":"2023-10-10T20:20:09.677325+0000","last_became_peered":"2023-10-10T20:20:09.677325+0000","last_unstale":"2023-10-10T20:20:10.700860+0000","last_undegraded":"2023-10-10T20:20:10.700860+0000","last_fullsized":"2023-10-10T20:20:10.700860+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:24:27.405751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489628+0000","last_change":"2023-10-10T20:20:10.489557+0000","last_active":"2023-10-10T20:20:10.489628+0000","last_peered":"2023-10-10T20:20:10.489628+0000","last_clean":"2023-10-10T20:20:10.489628+0000","last_became_active":"2023-10-10T20:20:10.488862+0000","last_became_peered":"2023-10-10T20:20:10.488862+0000","last_unstale":"2023-10-10T20:20:10.489628+0000","last_undegraded":"2023-10-10T20:20:10.489628+0000","last_fullsized":"2023-10-10T20:20:10.489628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702065+0000","last_change":"2023-10-10T20:20:09.676133+0000","last_active":"2023-10-10T20:20:10.702065+0000","last_peered":"2023-10-10T20:20:10.702065+0000","last_clean":"2023-10-10T20:20:10.702065+0000","last_became_active":"2023-10-10T20:20:09.675860+0000","last_became_peered":"2023-10-10T20:20:09.675860+0000","last_unstale":"2023-10-10T20:20:10.702065+0000","last_undegraded":"2023-10-10T20:20:10.702065+0000","last_fullsized":"2023-10-10T20:20:10.702065+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:51.148353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.477785+0000","last_change":"2023-10-10T20:20:10.477710+0000","last_active":"2023-10-10T20:20:10.477785+0000","last_peered":"2023-10-10T20:20:10.477785+0000","last_clean":"2023-10-10T20:20:10.477785+0000","last_became_active":"2023-10-10T20:20:10.477464+0000","last_became_peered":"2023-10-10T20:20:10.477464+0000","last_unstale":"2023-10-10T20:20:10.477785+0000","last_undegraded":"2023-10-10T20:20:10.477785+0000","last_fullsized":"2023-10-10T20:20:10.477785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702039+0000","last_change":"2023-10-10T20:20:09.682940+0000","last_active":"2023-10-10T20:20:10.702039+0000","last_peered":"2023-10-10T20:20:10.702039+0000","last_clean":"2023-10-10T20:20:10.702039+0000","last_became_active":"2023-10-10T20:20:09.682394+0000","last_became_peered":"2023-10-10T20:20:09.682394+0000","last_unstale":"2023-10-10T20:20:10.702039+0000","last_undegraded":"2023-10-10T20:20:10.702039+0000","last_fullsized":"2023-10-10T20:20:10.702039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:23.970899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482161+0000","last_change":"2023-10-10T20:20:10.482085+0000","last_active":"2023-10-10T20:20:10.482161+0000","last_peered":"2023-10-10T20:20:10.482161+0000","last_clean":"2023-10-10T20:20:10.482161+0000","last_became_active":"2023-10-10T20:20:10.481392+0000","last_became_peered":"2023-10-10T20:20:10.481392+0000","last_unstale":"2023-10-10T20:20:10.482161+0000","last_undegraded":"2023-10-10T20:20:10.482161+0000","last_fullsized":"2023-10-10T20:20:10.482161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479303+0000","last_change":"2023-10-10T20:20:09.680023+0000","last_active":"2023-10-10T20:20:10.479303+0000","last_peered":"2023-10-10T20:20:10.479303+0000","last_clean":"2023-10-10T20:20:10.479303+0000","last_became_active":"2023-10-10T20:20:09.679002+0000","last_became_peered":"2023-10-10T20:20:09.679002+0000","last_unstale":"2023-10-10T20:20:10.479303+0000","last_undegraded":"2023-10-10T20:20:10.479303+0000","last_fullsized":"2023-10-10T20:20:10.479303+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:34.985846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487161+0000","last_change":"2023-10-10T20:20:10.487030+0000","last_active":"2023-10-10T20:20:10.487161+0000","last_peered":"2023-10-10T20:20:10.487161+0000","last_clean":"2023-10-10T20:20:10.487161+0000","last_became_active":"2023-10-10T20:20:10.485979+0000","last_became_peered":"2023-10-10T20:20:10.485979+0000","last_unstale":"2023-10-10T20:20:10.487161+0000","last_undegraded":"2023-10-10T20:20:10.487161+0000","last_fullsized":"2023-10-10T20:20:10.487161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476130+0000","last_change":"2023-10-10T20:20:09.681781+0000","last_active":"2023-10-10T20:20:10.476130+0000","last_peered":"2023-10-10T20:20:10.476130+0000","last_clean":"2023-10-10T20:20:10.476130+0000","last_became_active":"2023-10-10T20:20:09.680900+0000","last_became_peered":"2023-10-10T20:20:09.680900+0000","last_unstale":"2023-10-10T20:20:10.476130+0000","last_undegraded":"2023-10-10T20:20:10.476130+0000","last_fullsized":"2023-10-10T20:20:10.476130+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:09.524795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484895+0000","last_change":"2023-10-10T20:20:10.484780+0000","last_active":"2023-10-10T20:20:10.484895+0000","last_peered":"2023-10-10T20:20:10.484895+0000","last_clean":"2023-10-10T20:20:10.484895+0000","last_became_active":"2023-10-10T20:20:10.484543+0000","last_became_peered":"2023-10-10T20:20:10.484543+0000","last_unstale":"2023-10-10T20:20:10.484895+0000","last_undegraded":"2023-10-10T20:20:10.484895+0000","last_fullsized":"2023-10-10T20:20:10.484895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:22:03.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479218+0000","last_change":"2023-10-10T20:20:09.685511+0000","last_active":"2023-10-10T20:20:10.479218+0000","last_peered":"2023-10-10T20:20:10.479218+0000","last_clean":"2023-10-10T20:20:10.479218+0000","last_became_active":"2023-10-10T20:20:09.685266+0000","last_became_peered":"2023-10-10T20:20:09.685266+0000","last_unstale":"2023-10-10T20:20:10.479218+0000","last_undegraded":"2023-10-10T20:20:10.479218+0000","last_fullsized":"2023-10-10T20:20:10.479218+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:59:41.742268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488094+0000","last_change":"2023-10-10T20:20:10.487727+0000","last_active":"2023-10-10T20:20:10.488094+0000","last_peered":"2023-10-10T20:20:10.488094+0000","last_clean":"2023-10-10T20:20:10.488094+0000","last_became_active":"2023-10-10T20:20:10.487538+0000","last_became_peered":"2023-10-10T20:20:10.487538+0000","last_unstale":"2023-10-10T20:20:10.488094+0000","last_undegraded":"2023-10-10T20:20:10.488094+0000","last_fullsized":"2023-10-10T20:20:10.488094+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.475878+0000","last_change":"2023-10-10T20:20:09.682602+0000","last_active":"2023-10-10T20:20:10.475878+0000","last_peered":"2023-10-10T20:20:10.475878+0000","last_clean":"2023-10-10T20:20:10.475878+0000","last_became_active":"2023-10-10T20:20:09.682393+0000","last_became_peered":"2023-10-10T20:20:09.682393+0000","last_unstale":"2023-10-10T20:20:10.475878+0000","last_undegraded":"2023-10-10T20:20:10.475878+0000","last_fullsized":"2023-10-10T20:20:10.475878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:03.541574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488441+0000","last_change":"2023-10-10T20:20:10.488344+0000","last_active":"2023-10-10T20:20:10.488441+0000","last_peered":"2023-10-10T20:20:10.488441+0000","last_clean":"2023-10-10T20:20:10.488441+0000","last_became_active":"2023-10-10T20:20:10.488111+0000","last_became_peered":"2023-10-10T20:20:10.488111+0000","last_unstale":"2023-10-10T20:20:10.488441+0000","last_undegraded":"2023-10-10T20:20:10.488441+0000","last_fullsized":"2023-10-10T20:20:10.488441+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:10.322662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701163+0000","last_change":"2023-10-10T20:20:09.683932+0000","last_active":"2023-10-10T20:20:10.701163+0000","last_peered":"2023-10-10T20:20:10.701163+0000","last_clean":"2023-10-10T20:20:10.701163+0000","last_became_active":"2023-10-10T20:20:09.683006+0000","last_became_peered":"2023-10-10T20:20:09.683006+0000","last_unstale":"2023-10-10T20:20:10.701163+0000","last_undegraded":"2023-10-10T20:20:10.701163+0000","last_fullsized":"2023-10-10T20:20:10.701163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:48.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486338+0000","last_change":"2023-10-10T20:20:10.486227+0000","last_active":"2023-10-10T20:20:10.486338+0000","last_peered":"2023-10-10T20:20:10.486338+0000","last_clean":"2023-10-10T20:20:10.486338+0000","last_became_active":"2023-10-10T20:20:10.485862+0000","last_became_peered":"2023-10-10T20:20:10.485862+0000","last_unstale":"2023-10-10T20:20:10.486338+0000","last_undegraded":"2023-10-10T20:20:10.486338+0000","last_fullsized":"2023-10-10T20:20:10.486338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.703086+0000","last_change":"2023-10-10T20:20:09.682619+0000","last_active":"2023-10-10T20:20:10.703086+0000","last_peered":"2023-10-10T20:20:10.703086+0000","last_clean":"2023-10-10T20:20:10.703086+0000","last_became_active":"2023-10-10T20:20:09.682410+0000","last_became_peered":"2023-10-10T20:20:09.682410+0000","last_unstale":"2023-10-10T20:20:10.703086+0000","last_undegraded":"2023-10-10T20:20:10.703086+0000","last_fullsized":"2023-10-10T20:20:10.703086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:23.219897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482842+0000","last_change":"2023-10-10T20:20:10.482686+0000","last_active":"2023-10-10T20:20:10.482842+0000","last_peered":"2023-10-10T20:20:10.482842+0000","last_clean":"2023-10-10T20:20:10.482842+0000","last_became_active":"2023-10-10T20:20:10.482481+0000","last_became_peered":"2023-10-10T20:20:10.482481+0000","last_unstale":"2023-10-10T20:20:10.482842+0000","last_undegraded":"2023-10-10T20:20:10.482842+0000","last_fullsized":"2023-10-10T20:20:10.482842+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701349+0000","last_change":"2023-10-10T20:20:09.681619+0000","last_active":"2023-10-10T20:20:10.701349+0000","last_peered":"2023-10-10T20:20:10.701349+0000","last_clean":"2023-10-10T20:20:10.701349+0000","last_became_active":"2023-10-10T20:20:09.680593+0000","last_became_peered":"2023-10-10T20:20:09.680593+0000","last_unstale":"2023-10-10T20:20:10.701349+0000","last_undegraded":"2023-10-10T20:20:10.701349+0000","last_fullsized":"2023-10-10T20:20:10.701349+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:48.482262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479818+0000","last_change":"2023-10-10T20:20:10.479647+0000","last_active":"2023-10-10T20:20:10.479818+0000","last_peered":"2023-10-10T20:20:10.479818+0000","last_clean":"2023-10-10T20:20:10.479818+0000","last_became_active":"2023-10-10T20:20:10.479425+0000","last_became_peered":"2023-10-10T20:20:10.479425+0000","last_unstale":"2023-10-10T20:20:10.479818+0000","last_undegraded":"2023-10-10T20:20:10.479818+0000","last_fullsized":"2023-10-10T20:20:10.479818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479042+0000","last_change":"2023-10-10T20:20:09.679469+0000","last_active":"2023-10-10T20:20:10.479042+0000","last_peered":"2023-10-10T20:20:10.479042+0000","last_clean":"2023-10-10T20:20:10.479042+0000","last_became_active":"2023-10-10T20:20:09.679269+0000","last_became_peered":"2023-10-10T20:20:09.679269+0000","last_unstale":"2023-10-10T20:20:10.479042+0000","last_undegraded":"2023-10-10T20:20:10.479042+0000","last_fullsized":"2023-10-10T20:20:10.479042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489134+0000","last_change":"2023-10-10T20:20:10.489075+0000","last_active":"2023-10-10T20:20:10.489134+0000","last_peered":"2023-10-10T20:20:10.489134+0000","last_clean":"2023-10-10T20:20:10.489134+0000","last_became_active":"2023-10-10T20:20:10.488233+0000","last_became_peered":"2023-10-10T20:20:10.488233+0000","last_unstale":"2023-10-10T20:20:10.489134+0000","last_undegraded":"2023-10-10T20:20:10.489134+0000","last_fullsized":"2023-10-10T20:20:10.489134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483289+0000","last_change":"2023-10-10T20:20:09.685497+0000","last_active":"2023-10-10T20:20:10.483289+0000","last_peered":"2023-10-10T20:20:10.483289+0000","last_clean":"2023-10-10T20:20:10.483289+0000","last_became_active":"2023-10-10T20:20:09.685261+0000","last_became_peered":"2023-10-10T20:20:09.685261+0000","last_unstale":"2023-10-10T20:20:10.483289+0000","last_undegraded":"2023-10-10T20:20:10.483289+0000","last_fullsized":"2023-10-10T20:20:10.483289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:01:51.847685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486628+0000","last_change":"2023-10-10T20:20:10.486552+0000","last_active":"2023-10-10T20:20:10.486628+0000","last_peered":"2023-10-10T20:20:10.486628+0000","last_clean":"2023-10-10T20:20:10.486628+0000","last_became_active":"2023-10-10T20:20:10.486299+0000","last_became_peered":"2023-10-10T20:20:10.486299+0000","last_unstale":"2023-10-10T20:20:10.486628+0000","last_undegraded":"2023-10-10T20:20:10.486628+0000","last_fullsized":"2023-10-10T20:20:10.486628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:29.917088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702360+0000","last_change":"2023-10-10T20:20:09.675262+0000","last_active":"2023-10-10T20:20:10.702360+0000","last_peered":"2023-10-10T20:20:10.702360+0000","last_clean":"2023-10-10T20:20:10.702360+0000","last_became_active":"2023-10-10T20:20:09.675040+0000","last_became_peered":"2023-10-10T20:20:09.675040+0000","last_unstale":"2023-10-10T20:20:10.702360+0000","last_undegraded":"2023-10-10T20:20:10.702360+0000","last_fullsized":"2023-10-10T20:20:10.702360+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:48.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.478074+0000","last_change":"2023-10-10T20:20:10.477871+0000","last_active":"2023-10-10T20:20:10.478074+0000","last_peered":"2023-10-10T20:20:10.478074+0000","last_clean":"2023-10-10T20:20:10.478074+0000","last_became_active":"2023-10-10T20:20:10.477294+0000","last_became_peered":"2023-10-10T20:20:10.477294+0000","last_unstale":"2023-10-10T20:20:10.478074+0000","last_undegraded":"2023-10-10T20:20:10.478074+0000","last_fullsized":"2023-10-10T20:20:10.478074+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:24.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700891+0000","last_change":"2023-10-10T20:20:09.683845+0000","last_active":"2023-10-10T20:20:10.700891+0000","last_peered":"2023-10-10T20:20:10.700891+0000","last_clean":"2023-10-10T20:20:10.700891+0000","last_became_active":"2023-10-10T20:20:09.683392+0000","last_became_peered":"2023-10-10T20:20:09.683392+0000","last_unstale":"2023-10-10T20:20:10.700891+0000","last_undegraded":"2023-10-10T20:20:10.700891+0000","last_fullsized":"2023-10-10T20:20:10.700891+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:56.012282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479746+0000","last_change":"2023-10-10T20:20:10.478450+0000","last_active":"2023-10-10T20:20:10.479746+0000","last_peered":"2023-10-10T20:20:10.479746+0000","last_clean":"2023-10-10T20:20:10.479746+0000","last_became_active":"2023-10-10T20:20:10.478198+0000","last_became_peered":"2023-10-10T20:20:10.478198+0000","last_unstale":"2023-10-10T20:20:10.479746+0000","last_undegraded":"2023-10-10T20:20:10.479746+0000","last_fullsized":"2023-10-10T20:20:10.479746+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701112+0000","last_change":"2023-10-10T20:20:09.681376+0000","last_active":"2023-10-10T20:20:10.701112+0000","last_peered":"2023-10-10T20:20:10.701112+0000","last_clean":"2023-10-10T20:20:10.701112+0000","last_became_active":"2023-10-10T20:20:09.680270+0000","last_became_peered":"2023-10-10T20:20:09.680270+0000","last_unstale":"2023-10-10T20:20:10.701112+0000","last_undegraded":"2023-10-10T20:20:10.701112+0000","last_fullsized":"2023-10-10T20:20:10.701112+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:20.012792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488409+0000","last_change":"2023-10-10T20:20:10.488231+0000","last_active":"2023-10-10T20:20:10.488409+0000","last_peered":"2023-10-10T20:20:10.488409+0000","last_clean":"2023-10-10T20:20:10.488409+0000","last_became_active":"2023-10-10T20:20:10.487919+0000","last_became_peered":"2023-10-10T20:20:10.487919+0000","last_unstale":"2023-10-10T20:20:10.488409+0000","last_undegraded":"2023-10-10T20:20:10.488409+0000","last_fullsized":"2023-10-10T20:20:10.488409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:03.488531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700388+0000","last_change":"2023-10-10T20:20:09.676724+0000","last_active":"2023-10-10T20:20:10.700388+0000","last_peered":"2023-10-10T20:20:10.700388+0000","last_clean":"2023-10-10T20:20:10.700388+0000","last_became_active":"2023-10-10T20:20:09.676510+0000","last_became_peered":"2023-10-10T20:20:09.676510+0000","last_unstale":"2023-10-10T20:20:10.700388+0000","last_undegraded":"2023-10-10T20:20:10.700388+0000","last_fullsized":"2023-10-10T20:20:10.700388+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:46:43.101844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490822+0000","last_change":"2023-10-10T20:20:10.490689+0000","last_active":"2023-10-10T20:20:10.490822+0000","last_peered":"2023-10-10T20:20:10.490822+0000","last_clean":"2023-10-10T20:20:10.490822+0000","last_became_active":"2023-10-10T20:20:10.490461+0000","last_became_peered":"2023-10-10T20:20:10.490461+0000","last_unstale":"2023-10-10T20:20:10.490822+0000","last_undegraded":"2023-10-10T20:20:10.490822+0000","last_fullsized":"2023-10-10T20:20:10.490822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:51.721701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476840+0000","last_change":"2023-10-10T20:20:09.684269+0000","last_active":"2023-10-10T20:20:10.476840+0000","last_peered":"2023-10-10T20:20:10.476840+0000","last_clean":"2023-10-10T20:20:10.476840+0000","last_became_active":"2023-10-10T20:20:09.683954+0000","last_became_peered":"2023-10-10T20:20:09.683954+0000","last_unstale":"2023-10-10T20:20:10.476840+0000","last_undegraded":"2023-10-10T20:20:10.476840+0000","last_fullsized":"2023-10-10T20:20:10.476840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:36:23.499956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484788+0000","last_change":"2023-10-10T20:20:10.483948+0000","last_active":"2023-10-10T20:20:10.484788+0000","last_peered":"2023-10-10T20:20:10.484788+0000","last_clean":"2023-10-10T20:20:10.484788+0000","last_became_active":"2023-10-10T20:20:10.483551+0000","last_became_peered":"2023-10-10T20:20:10.483551+0000","last_unstale":"2023-10-10T20:20:10.484788+0000","last_undegraded":"2023-10-10T20:20:10.484788+0000","last_fullsized":"2023-10-10T20:20:10.484788+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:27.507265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702064+0000","last_change":"2023-10-10T20:20:09.679569+0000","last_active":"2023-10-10T20:20:10.702064+0000","last_peered":"2023-10-10T20:20:10.702064+0000","last_clean":"2023-10-10T20:20:10.702064+0000","last_became_active":"2023-10-10T20:20:09.679068+0000","last_became_peered":"2023-10-10T20:20:09.679068+0000","last_unstale":"2023-10-10T20:20:10.702064+0000","last_undegraded":"2023-10-10T20:20:10.702064+0000","last_fullsized":"2023-10-10T20:20:10.702064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:56.012282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483867+0000","last_change":"2023-10-10T20:20:10.482993+0000","last_active":"2023-10-10T20:20:10.483867+0000","last_peered":"2023-10-10T20:20:10.483867+0000","last_clean":"2023-10-10T20:20:10.483867+0000","last_became_active":"2023-10-10T20:20:10.482763+0000","last_became_peered":"2023-10-10T20:20:10.482763+0000","last_unstale":"2023-10-10T20:20:10.483867+0000","last_undegraded":"2023-10-10T20:20:10.483867+0000","last_fullsized":"2023-10-10T20:20:10.483867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:10.526457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479167+0000","last_change":"2023-10-10T20:20:09.679727+0000","last_active":"2023-10-10T20:20:10.479167+0000","last_peered":"2023-10-10T20:20:10.479167+0000","last_clean":"2023-10-10T20:20:10.479167+0000","last_became_active":"2023-10-10T20:20:09.679459+0000","last_became_peered":"2023-10-10T20:20:09.679459+0000","last_unstale":"2023-10-10T20:20:10.479167+0000","last_undegraded":"2023-10-10T20:20:10.479167+0000","last_fullsized":"2023-10-10T20:20:10.479167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483848+0000","last_change":"2023-10-10T20:20:10.482340+0000","last_active":"2023-10-10T20:20:10.483848+0000","last_peered":"2023-10-10T20:20:10.483848+0000","last_clean":"2023-10-10T20:20:10.483848+0000","last_became_active":"2023-10-10T20:20:10.482128+0000","last_became_peered":"2023-10-10T20:20:10.482128+0000","last_unstale":"2023-10-10T20:20:10.483848+0000","last_undegraded":"2023-10-10T20:20:10.483848+0000","last_fullsized":"2023-10-10T20:20:10.483848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:59.752331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476406+0000","last_change":"2023-10-10T20:20:09.678009+0000","last_active":"2023-10-10T20:20:10.476406+0000","last_peered":"2023-10-10T20:20:10.476406+0000","last_clean":"2023-10-10T20:20:10.476406+0000","last_became_active":"2023-10-10T20:20:09.677524+0000","last_became_peered":"2023-10-10T20:20:09.677524+0000","last_unstale":"2023-10-10T20:20:10.476406+0000","last_undegraded":"2023-10-10T20:20:10.476406+0000","last_fullsized":"2023-10-10T20:20:10.476406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:28.410103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489157+0000","last_change":"2023-10-10T20:20:10.489087+0000","last_active":"2023-10-10T20:20:10.489157+0000","last_peered":"2023-10-10T20:20:10.489157+0000","last_clean":"2023-10-10T20:20:10.489157+0000","last_became_active":"2023-10-10T20:20:10.488315+0000","last_became_peered":"2023-10-10T20:20:10.488315+0000","last_unstale":"2023-10-10T20:20:10.489157+0000","last_undegraded":"2023-10-10T20:20:10.489157+0000","last_fullsized":"2023-10-10T20:20:10.489157+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:22:03.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.474726+0000","last_change":"2023-10-10T20:20:09.681856+0000","last_active":"2023-10-10T20:20:10.474726+0000","last_peered":"2023-10-10T20:20:10.474726+0000","last_clean":"2023-10-10T20:20:10.474726+0000","last_became_active":"2023-10-10T20:20:09.681566+0000","last_became_peered":"2023-10-10T20:20:09.681566+0000","last_unstale":"2023-10-10T20:20:10.474726+0000","last_undegraded":"2023-10-10T20:20:10.474726+0000","last_fullsized":"2023-10-10T20:20:10.474726+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:34:16.967295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486039+0000","last_change":"2023-10-10T20:20:10.485829+0000","last_active":"2023-10-10T20:20:10.486039+0000","last_peered":"2023-10-10T20:20:10.486039+0000","last_clean":"2023-10-10T20:20:10.486039+0000","last_became_active":"2023-10-10T20:20:10.485583+0000","last_became_peered":"2023-10-10T20:20:10.485583+0000","last_unstale":"2023-10-10T20:20:10.486039+0000","last_undegraded":"2023-10-10T20:20:10.486039+0000","last_fullsized":"2023-10-10T20:20:10.486039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:15.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701177+0000","last_change":"2023-10-10T20:20:09.678682+0000","last_active":"2023-10-10T20:20:10.701177+0000","last_peered":"2023-10-10T20:20:10.701177+0000","last_clean":"2023-10-10T20:20:10.701177+0000","last_became_active":"2023-10-10T20:20:09.678460+0000","last_became_peered":"2023-10-10T20:20:09.678460+0000","last_unstale":"2023-10-10T20:20:10.701177+0000","last_undegraded":"2023-10-10T20:20:10.701177+0000","last_fullsized":"2023-10-10T20:20:10.701177+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483254+0000","last_change":"2023-10-10T20:20:10.482664+0000","last_active":"2023-10-10T20:20:10.483254+0000","last_peered":"2023-10-10T20:20:10.483254+0000","last_clean":"2023-10-10T20:20:10.483254+0000","last_became_active":"2023-10-10T20:20:10.482399+0000","last_became_peered":"2023-10-10T20:20:10.482399+0000","last_unstale":"2023-10-10T20:20:10.483254+0000","last_undegraded":"2023-10-10T20:20:10.483254+0000","last_fullsized":"2023-10-10T20:20:10.483254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:52.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476210+0000","last_change":"2023-10-10T20:20:09.679991+0000","last_active":"2023-10-10T20:20:10.476210+0000","last_peered":"2023-10-10T20:20:10.476210+0000","last_clean":"2023-10-10T20:20:10.476210+0000","last_became_active":"2023-10-10T20:20:09.679338+0000","last_became_peered":"2023-10-10T20:20:09.679338+0000","last_unstale":"2023-10-10T20:20:10.476210+0000","last_undegraded":"2023-10-10T20:20:10.476210+0000","last_fullsized":"2023-10-10T20:20:10.476210+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484588+0000","last_change":"2023-10-10T20:20:10.484537+0000","last_active":"2023-10-10T20:20:10.484588+0000","last_peered":"2023-10-10T20:20:10.484588+0000","last_clean":"2023-10-10T20:20:10.484588+0000","last_became_active":"2023-10-10T20:20:10.484331+0000","last_became_peered":"2023-10-10T20:20:10.484331+0000","last_unstale":"2023-10-10T20:20:10.484588+0000","last_undegraded":"2023-10-10T20:20:10.484588+0000","last_fullsized":"2023-10-10T20:20:10.484588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:35.987508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479127+0000","last_change":"2023-10-10T20:20:09.676902+0000","last_active":"2023-10-10T20:20:10.479127+0000","last_peered":"2023-10-10T20:20:10.479127+0000","last_clean":"2023-10-10T20:20:10.479127+0000","last_became_active":"2023-10-10T20:20:09.676622+0000","last_became_peered":"2023-10-10T20:20:09.676622+0000","last_unstale":"2023-10-10T20:20:10.479127+0000","last_undegraded":"2023-10-10T20:20:10.479127+0000","last_fullsized":"2023-10-10T20:20:10.479127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484058+0000","last_change":"2023-10-10T20:20:10.483806+0000","last_active":"2023-10-10T20:20:10.484058+0000","last_peered":"2023-10-10T20:20:10.484058+0000","last_clean":"2023-10-10T20:20:10.484058+0000","last_became_active":"2023-10-10T20:20:10.482602+0000","last_became_peered":"2023-10-10T20:20:10.482602+0000","last_unstale":"2023-10-10T20:20:10.484058+0000","last_undegraded":"2023-10-10T20:20:10.484058+0000","last_fullsized":"2023-10-10T20:20:10.484058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476150+0000","last_change":"2023-10-10T20:20:09.677480+0000","last_active":"2023-10-10T20:20:10.476150+0000","last_peered":"2023-10-10T20:20:10.476150+0000","last_clean":"2023-10-10T20:20:10.476150+0000","last_became_active":"2023-10-10T20:20:09.677261+0000","last_became_peered":"2023-10-10T20:20:09.677261+0000","last_unstale":"2023-10-10T20:20:10.476150+0000","last_undegraded":"2023-10-10T20:20:10.476150+0000","last_fullsized":"2023-10-10T20:20:10.476150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481754+0000","last_change":"2023-10-10T20:20:10.481545+0000","last_active":"2023-10-10T20:20:10.481754+0000","last_peered":"2023-10-10T20:20:10.481754+0000","last_clean":"2023-10-10T20:20:10.481754+0000","last_became_active":"2023-10-10T20:20:10.480932+0000","last_became_peered":"2023-10-10T20:20:10.480932+0000","last_unstale":"2023-10-10T20:20:10.481754+0000","last_undegraded":"2023-10-10T20:20:10.481754+0000","last_fullsized":"2023-10-10T20:20:10.481754+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:24.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.480338+0000","last_change":"2023-10-10T20:20:09.679397+0000","last_active":"2023-10-10T20:20:10.480338+0000","last_peered":"2023-10-10T20:20:10.480338+0000","last_clean":"2023-10-10T20:20:10.480338+0000","last_became_active":"2023-10-10T20:20:09.679174+0000","last_became_peered":"2023-10-10T20:20:09.679174+0000","last_unstale":"2023-10-10T20:20:10.480338+0000","last_undegraded":"2023-10-10T20:20:10.480338+0000","last_fullsized":"2023-10-10T20:20:10.480338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:03.541574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485153+0000","last_change":"2023-10-10T20:20:10.485111+0000","last_active":"2023-10-10T20:20:10.485153+0000","last_peered":"2023-10-10T20:20:10.485153+0000","last_clean":"2023-10-10T20:20:10.485153+0000","last_became_active":"2023-10-10T20:20:10.484740+0000","last_became_peered":"2023-10-10T20:20:10.484740+0000","last_unstale":"2023-10-10T20:20:10.485153+0000","last_undegraded":"2023-10-10T20:20:10.485153+0000","last_fullsized":"2023-10-10T20:20:10.485153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:52.113199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.475741+0000","last_change":"2023-10-10T20:20:09.682844+0000","last_active":"2023-10-10T20:20:10.475741+0000","last_peered":"2023-10-10T20:20:10.475741+0000","last_clean":"2023-10-10T20:20:10.475741+0000","last_became_active":"2023-10-10T20:20:09.682098+0000","last_became_peered":"2023-10-10T20:20:09.682098+0000","last_unstale":"2023-10-10T20:20:10.475741+0000","last_undegraded":"2023-10-10T20:20:10.475741+0000","last_fullsized":"2023-10-10T20:20:10.475741+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:57.213111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481756+0000","last_change":"2023-10-10T20:20:10.481028+0000","last_active":"2023-10-10T20:20:10.481756+0000","last_peered":"2023-10-10T20:20:10.481756+0000","last_clean":"2023-10-10T20:20:10.481756+0000","last_became_active":"2023-10-10T20:20:10.480547+0000","last_became_peered":"2023-10-10T20:20:10.480547+0000","last_unstale":"2023-10-10T20:20:10.481756+0000","last_undegraded":"2023-10-10T20:20:10.481756+0000","last_fullsized":"2023-10-10T20:20:10.481756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479748+0000","last_change":"2023-10-10T20:20:09.686431+0000","last_active":"2023-10-10T20:20:10.479748+0000","last_peered":"2023-10-10T20:20:10.479748+0000","last_clean":"2023-10-10T20:20:10.479748+0000","last_became_active":"2023-10-10T20:20:09.686217+0000","last_became_peered":"2023-10-10T20:20:09.686217+0000","last_unstale":"2023-10-10T20:20:10.479748+0000","last_undegraded":"2023-10-10T20:20:10.479748+0000","last_fullsized":"2023-10-10T20:20:10.479748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:34:16.967295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489315+0000","last_change":"2023-10-10T20:20:10.489227+0000","last_active":"2023-10-10T20:20:10.489315+0000","last_peered":"2023-10-10T20:20:10.489315+0000","last_clean":"2023-10-10T20:20:10.489315+0000","last_became_active":"2023-10-10T20:20:10.488689+0000","last_became_peered":"2023-10-10T20:20:10.488689+0000","last_unstale":"2023-10-10T20:20:10.489315+0000","last_undegraded":"2023-10-10T20:20:10.489315+0000","last_fullsized":"2023-10-10T20:20:10.489315+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479281+0000","last_change":"2023-10-10T20:20:09.684685+0000","last_active":"2023-10-10T20:20:10.479281+0000","last_peered":"2023-10-10T20:20:10.479281+0000","last_clean":"2023-10-10T20:20:10.479281+0000","last_became_active":"2023-10-10T20:20:09.684453+0000","last_became_peered":"2023-10-10T20:20:09.684453+0000","last_unstale":"2023-10-10T20:20:10.479281+0000","last_undegraded":"2023-10-10T20:20:10.479281+0000","last_fullsized":"2023-10-10T20:20:10.479281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481576+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:20:10.481576+0000","last_peered":"2023-10-10T20:20:10.481576+0000","last_clean":"2023-10-10T20:20:10.481576+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:20:10.481576+0000","last_undegraded":"2023-10-10T20:20:10.481576+0000","last_fullsized":"2023-10-10T20:20:10.481576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865858,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199277089,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.371},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":9.7260000000000009,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195189,"statfs":{"total":200000000,"available":199873675,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.372},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.41699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.68799999999999994}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.32700000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35599999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872630,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:20:14.761 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:20:14.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.907+0000 7f3c032c2700 1 Processor -- start 2023-10-10T20:20:14.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.907+0000 7f3c032c2700 1 -- start start 2023-10-10T20:20:14.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c032c2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bfc157ba0 0x7f3bfc06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c032c2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc15c360 con 0x7f3bfc1577d0 2023-10-10T20:20:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c032c2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc15b790 con 0x7f3bfc15b0d0 2023-10-10T20:20:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c032c2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bfc15ba30 con 0x7f3bfc157ba0 2023-10-10T20:20:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c0105e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3bfc1577d0 0x7f3bfc0cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60024/0 (socket says 172.21.15.148:60024) 2023-10-10T20:20:14.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c0105e700 1 -- 172.21.15.148:0/2899199042 learned_addr learned my addr 172.21.15.148:0/2899199042 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.908+0000 7f3c0085d700 1 --2- 172.21.15.148:0/2899199042 >> v2:172.21.15.148:3300/0 conn(0x7f3bfc157ba0 0x7f3bfc06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c0085d700 1 -- 172.21.15.148:0/2899199042 >> v1:172.21.15.161:6789/0 conn(0x7f3bfc15b0d0 legacy=0x7f3bfc15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c0085d700 1 -- 172.21.15.148:0/2899199042 >> v1:172.21.15.148:6789/0 conn(0x7f3bfc1577d0 legacy=0x7f3bfc0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c0085d700 1 -- 172.21.15.148:0/2899199042 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc15bd10 con 0x7f3bfc157ba0 2023-10-10T20:20:14.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3bf3fff700 1 -- 172.21.15.148:0/2899199042 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015193278 0 0) 0x7f3bfc15c360 con 0x7f3bfc1577d0 2023-10-10T20:20:14.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c0085d700 1 --2- 172.21.15.148:0/2899199042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc06a520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0098e0 tx=0x7f3bec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5551d4600955e1dd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3bf3fff700 1 -- 172.21.15.148:0/2899199042 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec009e80 con 0x7f3bfc157ba0 2023-10-10T20:20:14.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3bf3fff700 1 -- 172.21.15.148:0/2899199042 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec00b860 con 0x7f3bfc157ba0 2023-10-10T20:20:14.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3bf3fff700 1 -- 172.21.15.148:0/2899199042 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00bbe0 con 0x7f3bfc157ba0 2023-10-10T20:20:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c032c2700 1 -- 172.21.15.148:0/2899199042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 msgr2=0x7f3bfc06a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c032c2700 1 --2- 172.21.15.148:0/2899199042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc06a520 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0098e0 tx=0x7f3bec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c032c2700 1 -- 172.21.15.148:0/2899199042 shutdown_connections 2023-10-10T20:20:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c032c2700 1 --2- 172.21.15.148:0/2899199042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc06a520 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.909+0000 7f3c032c2700 1 -- 172.21.15.148:0/2899199042 >> 172.21.15.148:0/2899199042 conn(0x7f3bfc068ce0 msgr2=0x7f3bfc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:14.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.910+0000 7f3c032c2700 1 -- 172.21.15.148:0/2899199042 shutdown_connections 2023-10-10T20:20:14.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.910+0000 7f3c032c2700 1 -- 172.21.15.148:0/2899199042 wait complete. 2023-10-10T20:20:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.911+0000 7f3c032c2700 1 Processor -- start 2023-10-10T20:20:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.911+0000 7f3c032c2700 1 -- start start 2023-10-10T20:20:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bfc1577d0 0x7f3bfc1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc15b0d0 0x7f3bfc16e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc15db80 con 0x7f3bfc157ba0 2023-10-10T20:20:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0b2bd0 con 0x7f3bfc15b0d0 2023-10-10T20:20:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0b2e70 con 0x7f3bfc1577d0 2023-10-10T20:20:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47384/0 (socket says 172.21.15.148:47384) 2023-10-10T20:20:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 -- 172.21.15.148:0/3563678592 learned_addr learned my addr 172.21.15.148:0/3563678592 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0105e700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bfc1577d0 0x7f3bfc1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0185f700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc15b0d0 0x7f3bfc16e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 -- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bfc1577d0 msgr2=0x7f3bfc1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bfc1577d0 0x7f3bfc1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 -- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc15b0d0 msgr2=0x7f3bfc16e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc15b0d0 0x7f3bfc16e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 -- 172.21.15.148:0/3563678592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec009420 con 0x7f3bfc157ba0 2023-10-10T20:20:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c0085d700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc06b580 tx=0x7f3bec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec01f070 con 0x7f3bfc157ba0 2023-10-10T20:20:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec011420 con 0x7f3bfc157ba0 2023-10-10T20:20:14.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc0b1730 con 0x7f3bfc157ba0 2023-10-10T20:20:14.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00bbe0 con 0x7f3bfc157ba0 2023-10-10T20:20:14.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.912+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc0b1450 con 0x7f3bfc157ba0 2023-10-10T20:20:14.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.913+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc0b3030 con 0x7f3bfc157ba0 2023-10-10T20:20:14.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.914+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bec011a10 con 0x7f3bfc157ba0 2023-10-10T20:20:14.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.914+0000 7f3bf17fa700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 0x7f3be00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:14.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.914+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3bec085a70 con 0x7f3bfc157ba0 2023-10-10T20:20:14.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.914+0000 7f3c0105e700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 0x7f3be00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:14.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.914+0000 7f3c0105e700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 0x7f3be00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80011e0 tx=0x7f3bf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:14.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:14.918+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bec052ce0 con 0x7f3bfc157ba0 2023-10-10T20:20:15.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.094+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bfc04c3c0 con 0x7f3be0066f30 2023-10-10T20:20:15.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.102+0000 7f3bf17fa700 1 -- 172.21.15.148:0/3563678592 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+269164 (secure 0 0 0) 0x7f3bfc04c3c0 con 0x7f3be0066f30 2023-10-10T20:20:15.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:15.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.104+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 msgr2=0x7f3be00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.104+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 0x7f3be00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80011e0 tx=0x7f3bf8009040 comp rx=0 tx=0).stop 2023-10-10T20:20:15.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 msgr2=0x7f3bfc15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc06b580 tx=0x7f3bec017000 comp rx=0 tx=0).stop 2023-10-10T20:20:15.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 shutdown_connections 2023-10-10T20:20:15.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3be0066f30 0x7f3be00693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bfc15b0d0 0x7f3bfc16e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bfc1577d0 0x7f3bfc1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 --2- 172.21.15.148:0/3563678592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bfc157ba0 0x7f3bfc15d480 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 >> 172.21.15.148:0/3563678592 conn(0x7f3bfc068ce0 msgr2=0x7f3bfc14d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:15.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 shutdown_connections 2023-10-10T20:20:15.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.105+0000 7f3c032c2700 1 -- 172.21.15.148:0/3563678592 wait complete. 2023-10-10T20:20:15.114 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:20:15.118 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-10T20:20:13.333194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2174,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560325,"statfs":{"total":1600000000,"available":1597773394,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.664088"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483586+0000","last_change":"2023-10-10T20:20:10.483397+0000","last_active":"2023-10-10T20:20:10.483586+0000","last_peered":"2023-10-10T20:20:10.483586+0000","last_clean":"2023-10-10T20:20:10.483586+0000","last_became_active":"2023-10-10T20:20:10.483082+0000","last_became_peered":"2023-10-10T20:20:10.483082+0000","last_unstale":"2023-10-10T20:20:10.483586+0000","last_undegraded":"2023-10-10T20:20:10.483586+0000","last_fullsized":"2023-10-10T20:20:10.483586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487381+0000","last_change":"2023-10-10T20:20:10.487300+0000","last_active":"2023-10-10T20:20:10.487381+0000","last_peered":"2023-10-10T20:20:10.487381+0000","last_clean":"2023-10-10T20:20:10.487381+0000","last_became_active":"2023-10-10T20:20:10.487087+0000","last_became_peered":"2023-10-10T20:20:10.487087+0000","last_unstale":"2023-10-10T20:20:10.487381+0000","last_undegraded":"2023-10-10T20:20:10.487381+0000","last_fullsized":"2023-10-10T20:20:10.487381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486319+0000","last_change":"2023-10-10T20:20:10.486082+0000","last_active":"2023-10-10T20:20:10.486319+0000","last_peered":"2023-10-10T20:20:10.486319+0000","last_clean":"2023-10-10T20:20:10.486319+0000","last_became_active":"2023-10-10T20:20:10.485541+0000","last_became_peered":"2023-10-10T20:20:10.485541+0000","last_unstale":"2023-10-10T20:20:10.486319+0000","last_undegraded":"2023-10-10T20:20:10.486319+0000","last_fullsized":"2023-10-10T20:20:10.486319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:52.113199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487417+0000","last_change":"2023-10-10T20:20:10.487289+0000","last_active":"2023-10-10T20:20:10.487417+0000","last_peered":"2023-10-10T20:20:10.487417+0000","last_clean":"2023-10-10T20:20:10.487417+0000","last_became_active":"2023-10-10T20:20:10.486754+0000","last_became_peered":"2023-10-10T20:20:10.486754+0000","last_unstale":"2023-10-10T20:20:10.487417+0000","last_undegraded":"2023-10-10T20:20:10.487417+0000","last_fullsized":"2023-10-10T20:20:10.487417+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:27.703370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484818+0000","last_change":"2023-10-10T20:20:10.484711+0000","last_active":"2023-10-10T20:20:10.484818+0000","last_peered":"2023-10-10T20:20:10.484818+0000","last_clean":"2023-10-10T20:20:10.484818+0000","last_became_active":"2023-10-10T20:20:10.484485+0000","last_became_peered":"2023-10-10T20:20:10.484485+0000","last_unstale":"2023-10-10T20:20:10.484818+0000","last_undegraded":"2023-10-10T20:20:10.484818+0000","last_fullsized":"2023-10-10T20:20:10.484818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:49.483923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485688+0000","last_change":"2023-10-10T20:20:10.485611+0000","last_active":"2023-10-10T20:20:10.485688+0000","last_peered":"2023-10-10T20:20:10.485688+0000","last_clean":"2023-10-10T20:20:10.485688+0000","last_became_active":"2023-10-10T20:20:10.485202+0000","last_became_peered":"2023-10-10T20:20:10.485202+0000","last_unstale":"2023-10-10T20:20:10.485688+0000","last_undegraded":"2023-10-10T20:20:10.485688+0000","last_fullsized":"2023-10-10T20:20:10.485688+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486823+0000","last_change":"2023-10-10T20:20:10.486703+0000","last_active":"2023-10-10T20:20:10.486823+0000","last_peered":"2023-10-10T20:20:10.486823+0000","last_clean":"2023-10-10T20:20:10.486823+0000","last_became_active":"2023-10-10T20:20:10.483848+0000","last_became_peered":"2023-10-10T20:20:10.483848+0000","last_unstale":"2023-10-10T20:20:10.486823+0000","last_undegraded":"2023-10-10T20:20:10.486823+0000","last_fullsized":"2023-10-10T20:20:10.486823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489336+0000","last_change":"2023-10-10T20:20:10.489227+0000","last_active":"2023-10-10T20:20:10.489336+0000","last_peered":"2023-10-10T20:20:10.489336+0000","last_clean":"2023-10-10T20:20:10.489336+0000","last_became_active":"2023-10-10T20:20:10.488575+0000","last_became_peered":"2023-10-10T20:20:10.488575+0000","last_unstale":"2023-10-10T20:20:10.489336+0000","last_undegraded":"2023-10-10T20:20:10.489336+0000","last_fullsized":"2023-10-10T20:20:10.489336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486395+0000","last_change":"2023-10-10T20:20:10.485278+0000","last_active":"2023-10-10T20:20:10.486395+0000","last_peered":"2023-10-10T20:20:10.486395+0000","last_clean":"2023-10-10T20:20:10.486395+0000","last_became_active":"2023-10-10T20:20:10.484905+0000","last_became_peered":"2023-10-10T20:20:10.484905+0000","last_unstale":"2023-10-10T20:20:10.486395+0000","last_undegraded":"2023-10-10T20:20:10.486395+0000","last_fullsized":"2023-10-10T20:20:10.486395+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:35.987508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485866+0000","last_change":"2023-10-10T20:20:10.485807+0000","last_active":"2023-10-10T20:20:10.485866+0000","last_peered":"2023-10-10T20:20:10.485866+0000","last_clean":"2023-10-10T20:20:10.485866+0000","last_became_active":"2023-10-10T20:20:10.485589+0000","last_became_peered":"2023-10-10T20:20:10.485589+0000","last_unstale":"2023-10-10T20:20:10.485866+0000","last_undegraded":"2023-10-10T20:20:10.485866+0000","last_fullsized":"2023-10-10T20:20:10.485866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490869+0000","last_change":"2023-10-10T20:20:10.490769+0000","last_active":"2023-10-10T20:20:10.490869+0000","last_peered":"2023-10-10T20:20:10.490869+0000","last_clean":"2023-10-10T20:20:10.490869+0000","last_became_active":"2023-10-10T20:20:10.490308+0000","last_became_peered":"2023-10-10T20:20:10.490308+0000","last_unstale":"2023-10-10T20:20:10.490869+0000","last_undegraded":"2023-10-10T20:20:10.490869+0000","last_fullsized":"2023-10-10T20:20:10.490869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:03.488531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485213+0000","last_change":"2023-10-10T20:20:10.485131+0000","last_active":"2023-10-10T20:20:10.485213+0000","last_peered":"2023-10-10T20:20:10.485213+0000","last_clean":"2023-10-10T20:20:10.485213+0000","last_became_active":"2023-10-10T20:20:10.484979+0000","last_became_peered":"2023-10-10T20:20:10.484979+0000","last_unstale":"2023-10-10T20:20:10.485213+0000","last_undegraded":"2023-10-10T20:20:10.485213+0000","last_fullsized":"2023-10-10T20:20:10.485213+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486894+0000","last_change":"2023-10-10T20:20:10.486798+0000","last_active":"2023-10-10T20:20:10.486894+0000","last_peered":"2023-10-10T20:20:10.486894+0000","last_clean":"2023-10-10T20:20:10.486894+0000","last_became_active":"2023-10-10T20:20:10.486580+0000","last_became_peered":"2023-10-10T20:20:10.486580+0000","last_unstale":"2023-10-10T20:20:10.486894+0000","last_undegraded":"2023-10-10T20:20:10.486894+0000","last_fullsized":"2023-10-10T20:20:10.486894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.543235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482832+0000","last_change":"2023-10-10T20:20:10.482582+0000","last_active":"2023-10-10T20:20:10.482832+0000","last_peered":"2023-10-10T20:20:10.482832+0000","last_clean":"2023-10-10T20:20:10.482832+0000","last_became_active":"2023-10-10T20:20:10.482370+0000","last_became_peered":"2023-10-10T20:20:10.482370+0000","last_unstale":"2023-10-10T20:20:10.482832+0000","last_undegraded":"2023-10-10T20:20:10.482832+0000","last_fullsized":"2023-10-10T20:20:10.482832+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482095+0000","last_change":"2023-10-10T20:20:10.481846+0000","last_active":"2023-10-10T20:20:10.482095+0000","last_peered":"2023-10-10T20:20:10.482095+0000","last_clean":"2023-10-10T20:20:10.482095+0000","last_became_active":"2023-10-10T20:20:10.481609+0000","last_became_peered":"2023-10-10T20:20:10.481609+0000","last_unstale":"2023-10-10T20:20:10.482095+0000","last_undegraded":"2023-10-10T20:20:10.482095+0000","last_fullsized":"2023-10-10T20:20:10.482095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:51.626547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.491003+0000","last_change":"2023-10-10T20:20:10.490952+0000","last_active":"2023-10-10T20:20:10.491003+0000","last_peered":"2023-10-10T20:20:10.491003+0000","last_clean":"2023-10-10T20:20:10.491003+0000","last_became_active":"2023-10-10T20:20:10.490544+0000","last_became_peered":"2023-10-10T20:20:10.490544+0000","last_unstale":"2023-10-10T20:20:10.491003+0000","last_undegraded":"2023-10-10T20:20:10.491003+0000","last_fullsized":"2023-10-10T20:20:10.491003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:50.684042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486928+0000","last_change":"2023-10-10T20:20:10.486878+0000","last_active":"2023-10-10T20:20:10.486928+0000","last_peered":"2023-10-10T20:20:10.486928+0000","last_clean":"2023-10-10T20:20:10.486928+0000","last_became_active":"2023-10-10T20:20:10.484348+0000","last_became_peered":"2023-10-10T20:20:10.484348+0000","last_unstale":"2023-10-10T20:20:10.486928+0000","last_undegraded":"2023-10-10T20:20:10.486928+0000","last_fullsized":"2023-10-10T20:20:10.486928+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:21.378539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486574+0000","last_change":"2023-10-10T20:20:10.486516+0000","last_active":"2023-10-10T20:20:10.486574+0000","last_peered":"2023-10-10T20:20:10.486574+0000","last_clean":"2023-10-10T20:20:10.486574+0000","last_became_active":"2023-10-10T20:20:10.486303+0000","last_became_peered":"2023-10-10T20:20:10.486303+0000","last_unstale":"2023-10-10T20:20:10.486574+0000","last_undegraded":"2023-10-10T20:20:10.486574+0000","last_fullsized":"2023-10-10T20:20:10.486574+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:52.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485043+0000","last_change":"2023-10-10T20:20:10.484974+0000","last_active":"2023-10-10T20:20:10.485043+0000","last_peered":"2023-10-10T20:20:10.485043+0000","last_clean":"2023-10-10T20:20:10.485043+0000","last_became_active":"2023-10-10T20:20:10.484485+0000","last_became_peered":"2023-10-10T20:20:10.484485+0000","last_unstale":"2023-10-10T20:20:10.485043+0000","last_undegraded":"2023-10-10T20:20:10.485043+0000","last_fullsized":"2023-10-10T20:20:10.485043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:26:46.604639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486057+0000","last_change":"2023-10-10T20:20:10.485987+0000","last_active":"2023-10-10T20:20:10.486057+0000","last_peered":"2023-10-10T20:20:10.486057+0000","last_clean":"2023-10-10T20:20:10.486057+0000","last_became_active":"2023-10-10T20:20:10.485782+0000","last_became_peered":"2023-10-10T20:20:10.485782+0000","last_unstale":"2023-10-10T20:20:10.486057+0000","last_undegraded":"2023-10-10T20:20:10.486057+0000","last_fullsized":"2023-10-10T20:20:10.486057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:57.013943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.477139+0000","last_change":"2023-10-10T20:20:10.476986+0000","last_active":"2023-10-10T20:20:10.477139+0000","last_peered":"2023-10-10T20:20:10.477139+0000","last_clean":"2023-10-10T20:20:10.477139+0000","last_became_active":"2023-10-10T20:20:10.476719+0000","last_became_peered":"2023-10-10T20:20:10.476719+0000","last_unstale":"2023-10-10T20:20:10.477139+0000","last_undegraded":"2023-10-10T20:20:10.477139+0000","last_fullsized":"2023-10-10T20:20:10.477139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:18.079608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488524+0000","last_change":"2023-10-10T20:20:10.488342+0000","last_active":"2023-10-10T20:20:10.488524+0000","last_peered":"2023-10-10T20:20:10.488524+0000","last_clean":"2023-10-10T20:20:10.488524+0000","last_became_active":"2023-10-10T20:20:10.488112+0000","last_became_peered":"2023-10-10T20:20:10.488112+0000","last_unstale":"2023-10-10T20:20:10.488524+0000","last_undegraded":"2023-10-10T20:20:10.488524+0000","last_fullsized":"2023-10-10T20:20:10.488524+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489216+0000","last_change":"2023-10-10T20:20:10.489176+0000","last_active":"2023-10-10T20:20:10.489216+0000","last_peered":"2023-10-10T20:20:10.489216+0000","last_clean":"2023-10-10T20:20:10.489216+0000","last_became_active":"2023-10-10T20:20:10.489039+0000","last_became_peered":"2023-10-10T20:20:10.489039+0000","last_unstale":"2023-10-10T20:20:10.489216+0000","last_undegraded":"2023-10-10T20:20:10.489216+0000","last_fullsized":"2023-10-10T20:20:10.489216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482246+0000","last_change":"2023-10-10T20:20:10.482124+0000","last_active":"2023-10-10T20:20:10.482246+0000","last_peered":"2023-10-10T20:20:10.482246+0000","last_clean":"2023-10-10T20:20:10.482246+0000","last_became_active":"2023-10-10T20:20:10.481869+0000","last_became_peered":"2023-10-10T20:20:10.481869+0000","last_unstale":"2023-10-10T20:20:10.482246+0000","last_undegraded":"2023-10-10T20:20:10.482246+0000","last_fullsized":"2023-10-10T20:20:10.482246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488491+0000","last_change":"2023-10-10T20:20:10.488415+0000","last_active":"2023-10-10T20:20:10.488491+0000","last_peered":"2023-10-10T20:20:10.488491+0000","last_clean":"2023-10-10T20:20:10.488491+0000","last_became_active":"2023-10-10T20:20:10.488171+0000","last_became_peered":"2023-10-10T20:20:10.488171+0000","last_unstale":"2023-10-10T20:20:10.488491+0000","last_undegraded":"2023-10-10T20:20:10.488491+0000","last_fullsized":"2023-10-10T20:20:10.488491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:48.655717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490619+0000","last_change":"2023-10-10T20:20:10.490562+0000","last_active":"2023-10-10T20:20:10.490619+0000","last_peered":"2023-10-10T20:20:10.490619+0000","last_clean":"2023-10-10T20:20:10.490619+0000","last_became_active":"2023-10-10T20:20:10.490303+0000","last_became_peered":"2023-10-10T20:20:10.490303+0000","last_unstale":"2023-10-10T20:20:10.490619+0000","last_undegraded":"2023-10-10T20:20:10.490619+0000","last_fullsized":"2023-10-10T20:20:10.490619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488296+0000","last_change":"2023-10-10T20:20:10.488102+0000","last_active":"2023-10-10T20:20:10.488296+0000","last_peered":"2023-10-10T20:20:10.488296+0000","last_clean":"2023-10-10T20:20:10.488296+0000","last_became_active":"2023-10-10T20:20:10.487628+0000","last_became_peered":"2023-10-10T20:20:10.487628+0000","last_unstale":"2023-10-10T20:20:10.488296+0000","last_undegraded":"2023-10-10T20:20:10.488296+0000","last_fullsized":"2023-10-10T20:20:10.488296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479728+0000","last_change":"2023-10-10T20:20:10.478212+0000","last_active":"2023-10-10T20:20:10.479728+0000","last_peered":"2023-10-10T20:20:10.479728+0000","last_clean":"2023-10-10T20:20:10.479728+0000","last_became_active":"2023-10-10T20:20:10.477981+0000","last_became_peered":"2023-10-10T20:20:10.477981+0000","last_unstale":"2023-10-10T20:20:10.479728+0000","last_undegraded":"2023-10-10T20:20:10.479728+0000","last_fullsized":"2023-10-10T20:20:10.479728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:50.684042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483084+0000","last_change":"2023-10-10T20:20:10.482284+0000","last_active":"2023-10-10T20:20:10.483084+0000","last_peered":"2023-10-10T20:20:10.483084+0000","last_clean":"2023-10-10T20:20:10.483084+0000","last_became_active":"2023-10-10T20:20:10.482042+0000","last_became_peered":"2023-10-10T20:20:10.482042+0000","last_unstale":"2023-10-10T20:20:10.483084+0000","last_undegraded":"2023-10-10T20:20:10.483084+0000","last_fullsized":"2023-10-10T20:20:10.483084+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:29.917088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488995+0000","last_change":"2023-10-10T20:20:10.488910+0000","last_active":"2023-10-10T20:20:10.488995+0000","last_peered":"2023-10-10T20:20:10.488995+0000","last_clean":"2023-10-10T20:20:10.488995+0000","last_became_active":"2023-10-10T20:20:10.488005+0000","last_became_peered":"2023-10-10T20:20:10.488005+0000","last_unstale":"2023-10-10T20:20:10.488995+0000","last_undegraded":"2023-10-10T20:20:10.488995+0000","last_fullsized":"2023-10-10T20:20:10.488995+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:25:31.883607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482544+0000","last_change":"2023-10-10T20:20:10.481673+0000","last_active":"2023-10-10T20:20:10.482544+0000","last_peered":"2023-10-10T20:20:10.482544+0000","last_clean":"2023-10-10T20:20:10.482544+0000","last_became_active":"2023-10-10T20:20:10.481424+0000","last_became_peered":"2023-10-10T20:20:10.481424+0000","last_unstale":"2023-10-10T20:20:10.482544+0000","last_undegraded":"2023-10-10T20:20:10.482544+0000","last_fullsized":"2023-10-10T20:20:10.482544+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489719+0000","last_change":"2023-10-10T20:20:10.489648+0000","last_active":"2023-10-10T20:20:10.489719+0000","last_peered":"2023-10-10T20:20:10.489719+0000","last_clean":"2023-10-10T20:20:10.489719+0000","last_became_active":"2023-10-10T20:20:10.489452+0000","last_became_peered":"2023-10-10T20:20:10.489452+0000","last_unstale":"2023-10-10T20:20:10.489719+0000","last_undegraded":"2023-10-10T20:20:10.489719+0000","last_fullsized":"2023-10-10T20:20:10.489719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488493+0000","last_change":"2023-10-10T20:20:10.488399+0000","last_active":"2023-10-10T20:20:10.488493+0000","last_peered":"2023-10-10T20:20:10.488493+0000","last_clean":"2023-10-10T20:20:10.488493+0000","last_became_active":"2023-10-10T20:20:10.488186+0000","last_became_peered":"2023-10-10T20:20:10.488186+0000","last_unstale":"2023-10-10T20:20:10.488493+0000","last_undegraded":"2023-10-10T20:20:10.488493+0000","last_fullsized":"2023-10-10T20:20:10.488493+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701186+0000","last_change":"2023-10-10T20:20:09.681326+0000","last_active":"2023-10-10T20:20:10.701186+0000","last_peered":"2023-10-10T20:20:10.701186+0000","last_clean":"2023-10-10T20:20:10.701186+0000","last_became_active":"2023-10-10T20:20:09.681071+0000","last_became_peered":"2023-10-10T20:20:09.681071+0000","last_unstale":"2023-10-10T20:20:10.701186+0000","last_undegraded":"2023-10-10T20:20:10.701186+0000","last_fullsized":"2023-10-10T20:20:10.701186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.480687+0000","last_change":"2023-10-10T20:20:10.480494+0000","last_active":"2023-10-10T20:20:10.480687+0000","last_peered":"2023-10-10T20:20:10.480687+0000","last_clean":"2023-10-10T20:20:10.480687+0000","last_became_active":"2023-10-10T20:20:10.480248+0000","last_became_peered":"2023-10-10T20:20:10.480248+0000","last_unstale":"2023-10-10T20:20:10.480687+0000","last_undegraded":"2023-10-10T20:20:10.480687+0000","last_fullsized":"2023-10-10T20:20:10.480687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:18.079608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700142+0000","last_change":"2023-10-10T20:20:09.682229+0000","last_active":"2023-10-10T20:20:10.700142+0000","last_peered":"2023-10-10T20:20:10.700142+0000","last_clean":"2023-10-10T20:20:10.700142+0000","last_became_active":"2023-10-10T20:20:09.682000+0000","last_became_peered":"2023-10-10T20:20:09.682000+0000","last_unstale":"2023-10-10T20:20:10.700142+0000","last_undegraded":"2023-10-10T20:20:10.700142+0000","last_fullsized":"2023-10-10T20:20:10.700142+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483101+0000","last_change":"2023-10-10T20:20:10.483015+0000","last_active":"2023-10-10T20:20:10.483101+0000","last_peered":"2023-10-10T20:20:10.483101+0000","last_clean":"2023-10-10T20:20:10.483101+0000","last_became_active":"2023-10-10T20:20:10.482771+0000","last_became_peered":"2023-10-10T20:20:10.482771+0000","last_unstale":"2023-10-10T20:20:10.483101+0000","last_undegraded":"2023-10-10T20:20:10.483101+0000","last_fullsized":"2023-10-10T20:20:10.483101+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701592+0000","last_change":"2023-10-10T20:20:09.678445+0000","last_active":"2023-10-10T20:20:10.701592+0000","last_peered":"2023-10-10T20:20:10.701592+0000","last_clean":"2023-10-10T20:20:10.701592+0000","last_became_active":"2023-10-10T20:20:09.678228+0000","last_became_peered":"2023-10-10T20:20:09.678228+0000","last_unstale":"2023-10-10T20:20:10.701592+0000","last_undegraded":"2023-10-10T20:20:10.701592+0000","last_fullsized":"2023-10-10T20:20:10.701592+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:17:17.077946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484418+0000","last_change":"2023-10-10T20:20:10.483850+0000","last_active":"2023-10-10T20:20:10.484418+0000","last_peered":"2023-10-10T20:20:10.484418+0000","last_clean":"2023-10-10T20:20:10.484418+0000","last_became_active":"2023-10-10T20:20:10.483639+0000","last_became_peered":"2023-10-10T20:20:10.483639+0000","last_unstale":"2023-10-10T20:20:10.484418+0000","last_undegraded":"2023-10-10T20:20:10.484418+0000","last_fullsized":"2023-10-10T20:20:10.484418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:21.378539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700860+0000","last_change":"2023-10-10T20:20:09.679748+0000","last_active":"2023-10-10T20:20:10.700860+0000","last_peered":"2023-10-10T20:20:10.700860+0000","last_clean":"2023-10-10T20:20:10.700860+0000","last_became_active":"2023-10-10T20:20:09.677325+0000","last_became_peered":"2023-10-10T20:20:09.677325+0000","last_unstale":"2023-10-10T20:20:10.700860+0000","last_undegraded":"2023-10-10T20:20:10.700860+0000","last_fullsized":"2023-10-10T20:20:10.700860+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:24:27.405751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489628+0000","last_change":"2023-10-10T20:20:10.489557+0000","last_active":"2023-10-10T20:20:10.489628+0000","last_peered":"2023-10-10T20:20:10.489628+0000","last_clean":"2023-10-10T20:20:10.489628+0000","last_became_active":"2023-10-10T20:20:10.488862+0000","last_became_peered":"2023-10-10T20:20:10.488862+0000","last_unstale":"2023-10-10T20:20:10.489628+0000","last_undegraded":"2023-10-10T20:20:10.489628+0000","last_fullsized":"2023-10-10T20:20:10.489628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702065+0000","last_change":"2023-10-10T20:20:09.676133+0000","last_active":"2023-10-10T20:20:10.702065+0000","last_peered":"2023-10-10T20:20:10.702065+0000","last_clean":"2023-10-10T20:20:10.702065+0000","last_became_active":"2023-10-10T20:20:09.675860+0000","last_became_peered":"2023-10-10T20:20:09.675860+0000","last_unstale":"2023-10-10T20:20:10.702065+0000","last_undegraded":"2023-10-10T20:20:10.702065+0000","last_fullsized":"2023-10-10T20:20:10.702065+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:51.148353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.477785+0000","last_change":"2023-10-10T20:20:10.477710+0000","last_active":"2023-10-10T20:20:10.477785+0000","last_peered":"2023-10-10T20:20:10.477785+0000","last_clean":"2023-10-10T20:20:10.477785+0000","last_became_active":"2023-10-10T20:20:10.477464+0000","last_became_peered":"2023-10-10T20:20:10.477464+0000","last_unstale":"2023-10-10T20:20:10.477785+0000","last_undegraded":"2023-10-10T20:20:10.477785+0000","last_fullsized":"2023-10-10T20:20:10.477785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702039+0000","last_change":"2023-10-10T20:20:09.682940+0000","last_active":"2023-10-10T20:20:10.702039+0000","last_peered":"2023-10-10T20:20:10.702039+0000","last_clean":"2023-10-10T20:20:10.702039+0000","last_became_active":"2023-10-10T20:20:09.682394+0000","last_became_peered":"2023-10-10T20:20:09.682394+0000","last_unstale":"2023-10-10T20:20:10.702039+0000","last_undegraded":"2023-10-10T20:20:10.702039+0000","last_fullsized":"2023-10-10T20:20:10.702039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:23.970899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482161+0000","last_change":"2023-10-10T20:20:10.482085+0000","last_active":"2023-10-10T20:20:10.482161+0000","last_peered":"2023-10-10T20:20:10.482161+0000","last_clean":"2023-10-10T20:20:10.482161+0000","last_became_active":"2023-10-10T20:20:10.481392+0000","last_became_peered":"2023-10-10T20:20:10.481392+0000","last_unstale":"2023-10-10T20:20:10.482161+0000","last_undegraded":"2023-10-10T20:20:10.482161+0000","last_fullsized":"2023-10-10T20:20:10.482161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479303+0000","last_change":"2023-10-10T20:20:09.680023+0000","last_active":"2023-10-10T20:20:10.479303+0000","last_peered":"2023-10-10T20:20:10.479303+0000","last_clean":"2023-10-10T20:20:10.479303+0000","last_became_active":"2023-10-10T20:20:09.679002+0000","last_became_peered":"2023-10-10T20:20:09.679002+0000","last_unstale":"2023-10-10T20:20:10.479303+0000","last_undegraded":"2023-10-10T20:20:10.479303+0000","last_fullsized":"2023-10-10T20:20:10.479303+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:34.985846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.487161+0000","last_change":"2023-10-10T20:20:10.487030+0000","last_active":"2023-10-10T20:20:10.487161+0000","last_peered":"2023-10-10T20:20:10.487161+0000","last_clean":"2023-10-10T20:20:10.487161+0000","last_became_active":"2023-10-10T20:20:10.485979+0000","last_became_peered":"2023-10-10T20:20:10.485979+0000","last_unstale":"2023-10-10T20:20:10.487161+0000","last_undegraded":"2023-10-10T20:20:10.487161+0000","last_fullsized":"2023-10-10T20:20:10.487161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476130+0000","last_change":"2023-10-10T20:20:09.681781+0000","last_active":"2023-10-10T20:20:10.476130+0000","last_peered":"2023-10-10T20:20:10.476130+0000","last_clean":"2023-10-10T20:20:10.476130+0000","last_became_active":"2023-10-10T20:20:09.680900+0000","last_became_peered":"2023-10-10T20:20:09.680900+0000","last_unstale":"2023-10-10T20:20:10.476130+0000","last_undegraded":"2023-10-10T20:20:10.476130+0000","last_fullsized":"2023-10-10T20:20:10.476130+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:09.524795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484895+0000","last_change":"2023-10-10T20:20:10.484780+0000","last_active":"2023-10-10T20:20:10.484895+0000","last_peered":"2023-10-10T20:20:10.484895+0000","last_clean":"2023-10-10T20:20:10.484895+0000","last_became_active":"2023-10-10T20:20:10.484543+0000","last_became_peered":"2023-10-10T20:20:10.484543+0000","last_unstale":"2023-10-10T20:20:10.484895+0000","last_undegraded":"2023-10-10T20:20:10.484895+0000","last_fullsized":"2023-10-10T20:20:10.484895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:22:03.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479218+0000","last_change":"2023-10-10T20:20:09.685511+0000","last_active":"2023-10-10T20:20:10.479218+0000","last_peered":"2023-10-10T20:20:10.479218+0000","last_clean":"2023-10-10T20:20:10.479218+0000","last_became_active":"2023-10-10T20:20:09.685266+0000","last_became_peered":"2023-10-10T20:20:09.685266+0000","last_unstale":"2023-10-10T20:20:10.479218+0000","last_undegraded":"2023-10-10T20:20:10.479218+0000","last_fullsized":"2023-10-10T20:20:10.479218+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:59:41.742268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488094+0000","last_change":"2023-10-10T20:20:10.487727+0000","last_active":"2023-10-10T20:20:10.488094+0000","last_peered":"2023-10-10T20:20:10.488094+0000","last_clean":"2023-10-10T20:20:10.488094+0000","last_became_active":"2023-10-10T20:20:10.487538+0000","last_became_peered":"2023-10-10T20:20:10.487538+0000","last_unstale":"2023-10-10T20:20:10.488094+0000","last_undegraded":"2023-10-10T20:20:10.488094+0000","last_fullsized":"2023-10-10T20:20:10.488094+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:02:02.138072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.475878+0000","last_change":"2023-10-10T20:20:09.682602+0000","last_active":"2023-10-10T20:20:10.475878+0000","last_peered":"2023-10-10T20:20:10.475878+0000","last_clean":"2023-10-10T20:20:10.475878+0000","last_became_active":"2023-10-10T20:20:09.682393+0000","last_became_peered":"2023-10-10T20:20:09.682393+0000","last_unstale":"2023-10-10T20:20:10.475878+0000","last_undegraded":"2023-10-10T20:20:10.475878+0000","last_fullsized":"2023-10-10T20:20:10.475878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:03.541574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488441+0000","last_change":"2023-10-10T20:20:10.488344+0000","last_active":"2023-10-10T20:20:10.488441+0000","last_peered":"2023-10-10T20:20:10.488441+0000","last_clean":"2023-10-10T20:20:10.488441+0000","last_became_active":"2023-10-10T20:20:10.488111+0000","last_became_peered":"2023-10-10T20:20:10.488111+0000","last_unstale":"2023-10-10T20:20:10.488441+0000","last_undegraded":"2023-10-10T20:20:10.488441+0000","last_fullsized":"2023-10-10T20:20:10.488441+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:10.322662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701163+0000","last_change":"2023-10-10T20:20:09.683932+0000","last_active":"2023-10-10T20:20:10.701163+0000","last_peered":"2023-10-10T20:20:10.701163+0000","last_clean":"2023-10-10T20:20:10.701163+0000","last_became_active":"2023-10-10T20:20:09.683006+0000","last_became_peered":"2023-10-10T20:20:09.683006+0000","last_unstale":"2023-10-10T20:20:10.701163+0000","last_undegraded":"2023-10-10T20:20:10.701163+0000","last_fullsized":"2023-10-10T20:20:10.701163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:48.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486338+0000","last_change":"2023-10-10T20:20:10.486227+0000","last_active":"2023-10-10T20:20:10.486338+0000","last_peered":"2023-10-10T20:20:10.486338+0000","last_clean":"2023-10-10T20:20:10.486338+0000","last_became_active":"2023-10-10T20:20:10.485862+0000","last_became_peered":"2023-10-10T20:20:10.485862+0000","last_unstale":"2023-10-10T20:20:10.486338+0000","last_undegraded":"2023-10-10T20:20:10.486338+0000","last_fullsized":"2023-10-10T20:20:10.486338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.703086+0000","last_change":"2023-10-10T20:20:09.682619+0000","last_active":"2023-10-10T20:20:10.703086+0000","last_peered":"2023-10-10T20:20:10.703086+0000","last_clean":"2023-10-10T20:20:10.703086+0000","last_became_active":"2023-10-10T20:20:09.682410+0000","last_became_peered":"2023-10-10T20:20:09.682410+0000","last_unstale":"2023-10-10T20:20:10.703086+0000","last_undegraded":"2023-10-10T20:20:10.703086+0000","last_fullsized":"2023-10-10T20:20:10.703086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:23.219897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.482842+0000","last_change":"2023-10-10T20:20:10.482686+0000","last_active":"2023-10-10T20:20:10.482842+0000","last_peered":"2023-10-10T20:20:10.482842+0000","last_clean":"2023-10-10T20:20:10.482842+0000","last_became_active":"2023-10-10T20:20:10.482481+0000","last_became_peered":"2023-10-10T20:20:10.482481+0000","last_unstale":"2023-10-10T20:20:10.482842+0000","last_undegraded":"2023-10-10T20:20:10.482842+0000","last_fullsized":"2023-10-10T20:20:10.482842+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:53.714036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701349+0000","last_change":"2023-10-10T20:20:09.681619+0000","last_active":"2023-10-10T20:20:10.701349+0000","last_peered":"2023-10-10T20:20:10.701349+0000","last_clean":"2023-10-10T20:20:10.701349+0000","last_became_active":"2023-10-10T20:20:09.680593+0000","last_became_peered":"2023-10-10T20:20:09.680593+0000","last_unstale":"2023-10-10T20:20:10.701349+0000","last_undegraded":"2023-10-10T20:20:10.701349+0000","last_fullsized":"2023-10-10T20:20:10.701349+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:48.482262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479818+0000","last_change":"2023-10-10T20:20:10.479647+0000","last_active":"2023-10-10T20:20:10.479818+0000","last_peered":"2023-10-10T20:20:10.479818+0000","last_clean":"2023-10-10T20:20:10.479818+0000","last_became_active":"2023-10-10T20:20:10.479425+0000","last_became_peered":"2023-10-10T20:20:10.479425+0000","last_unstale":"2023-10-10T20:20:10.479818+0000","last_undegraded":"2023-10-10T20:20:10.479818+0000","last_fullsized":"2023-10-10T20:20:10.479818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:59:02.029692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479042+0000","last_change":"2023-10-10T20:20:09.679469+0000","last_active":"2023-10-10T20:20:10.479042+0000","last_peered":"2023-10-10T20:20:10.479042+0000","last_clean":"2023-10-10T20:20:10.479042+0000","last_became_active":"2023-10-10T20:20:09.679269+0000","last_became_peered":"2023-10-10T20:20:09.679269+0000","last_unstale":"2023-10-10T20:20:10.479042+0000","last_undegraded":"2023-10-10T20:20:10.479042+0000","last_fullsized":"2023-10-10T20:20:10.479042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489134+0000","last_change":"2023-10-10T20:20:10.489075+0000","last_active":"2023-10-10T20:20:10.489134+0000","last_peered":"2023-10-10T20:20:10.489134+0000","last_clean":"2023-10-10T20:20:10.489134+0000","last_became_active":"2023-10-10T20:20:10.488233+0000","last_became_peered":"2023-10-10T20:20:10.488233+0000","last_unstale":"2023-10-10T20:20:10.489134+0000","last_undegraded":"2023-10-10T20:20:10.489134+0000","last_fullsized":"2023-10-10T20:20:10.489134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:04:51.730228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483289+0000","last_change":"2023-10-10T20:20:09.685497+0000","last_active":"2023-10-10T20:20:10.483289+0000","last_peered":"2023-10-10T20:20:10.483289+0000","last_clean":"2023-10-10T20:20:10.483289+0000","last_became_active":"2023-10-10T20:20:09.685261+0000","last_became_peered":"2023-10-10T20:20:09.685261+0000","last_unstale":"2023-10-10T20:20:10.483289+0000","last_undegraded":"2023-10-10T20:20:10.483289+0000","last_fullsized":"2023-10-10T20:20:10.483289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:01:51.847685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486628+0000","last_change":"2023-10-10T20:20:10.486552+0000","last_active":"2023-10-10T20:20:10.486628+0000","last_peered":"2023-10-10T20:20:10.486628+0000","last_clean":"2023-10-10T20:20:10.486628+0000","last_became_active":"2023-10-10T20:20:10.486299+0000","last_became_peered":"2023-10-10T20:20:10.486299+0000","last_unstale":"2023-10-10T20:20:10.486628+0000","last_undegraded":"2023-10-10T20:20:10.486628+0000","last_fullsized":"2023-10-10T20:20:10.486628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:17:29.917088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702360+0000","last_change":"2023-10-10T20:20:09.675262+0000","last_active":"2023-10-10T20:20:10.702360+0000","last_peered":"2023-10-10T20:20:10.702360+0000","last_clean":"2023-10-10T20:20:10.702360+0000","last_became_active":"2023-10-10T20:20:09.675040+0000","last_became_peered":"2023-10-10T20:20:09.675040+0000","last_unstale":"2023-10-10T20:20:10.702360+0000","last_undegraded":"2023-10-10T20:20:10.702360+0000","last_fullsized":"2023-10-10T20:20:10.702360+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:48.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.478074+0000","last_change":"2023-10-10T20:20:10.477871+0000","last_active":"2023-10-10T20:20:10.478074+0000","last_peered":"2023-10-10T20:20:10.478074+0000","last_clean":"2023-10-10T20:20:10.478074+0000","last_became_active":"2023-10-10T20:20:10.477294+0000","last_became_peered":"2023-10-10T20:20:10.477294+0000","last_unstale":"2023-10-10T20:20:10.478074+0000","last_undegraded":"2023-10-10T20:20:10.478074+0000","last_fullsized":"2023-10-10T20:20:10.478074+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:24.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700891+0000","last_change":"2023-10-10T20:20:09.683845+0000","last_active":"2023-10-10T20:20:10.700891+0000","last_peered":"2023-10-10T20:20:10.700891+0000","last_clean":"2023-10-10T20:20:10.700891+0000","last_became_active":"2023-10-10T20:20:09.683392+0000","last_became_peered":"2023-10-10T20:20:09.683392+0000","last_unstale":"2023-10-10T20:20:10.700891+0000","last_undegraded":"2023-10-10T20:20:10.700891+0000","last_fullsized":"2023-10-10T20:20:10.700891+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:56.012282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479746+0000","last_change":"2023-10-10T20:20:10.478450+0000","last_active":"2023-10-10T20:20:10.479746+0000","last_peered":"2023-10-10T20:20:10.479746+0000","last_clean":"2023-10-10T20:20:10.479746+0000","last_became_active":"2023-10-10T20:20:10.478198+0000","last_became_peered":"2023-10-10T20:20:10.478198+0000","last_unstale":"2023-10-10T20:20:10.479746+0000","last_undegraded":"2023-10-10T20:20:10.479746+0000","last_fullsized":"2023-10-10T20:20:10.479746+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:11.915128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701112+0000","last_change":"2023-10-10T20:20:09.681376+0000","last_active":"2023-10-10T20:20:10.701112+0000","last_peered":"2023-10-10T20:20:10.701112+0000","last_clean":"2023-10-10T20:20:10.701112+0000","last_became_active":"2023-10-10T20:20:09.680270+0000","last_became_peered":"2023-10-10T20:20:09.680270+0000","last_unstale":"2023-10-10T20:20:10.701112+0000","last_undegraded":"2023-10-10T20:20:10.701112+0000","last_fullsized":"2023-10-10T20:20:10.701112+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:20.012792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.488409+0000","last_change":"2023-10-10T20:20:10.488231+0000","last_active":"2023-10-10T20:20:10.488409+0000","last_peered":"2023-10-10T20:20:10.488409+0000","last_clean":"2023-10-10T20:20:10.488409+0000","last_became_active":"2023-10-10T20:20:10.487919+0000","last_became_peered":"2023-10-10T20:20:10.487919+0000","last_unstale":"2023-10-10T20:20:10.488409+0000","last_undegraded":"2023-10-10T20:20:10.488409+0000","last_fullsized":"2023-10-10T20:20:10.488409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:38:03.488531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.700388+0000","last_change":"2023-10-10T20:20:09.676724+0000","last_active":"2023-10-10T20:20:10.700388+0000","last_peered":"2023-10-10T20:20:10.700388+0000","last_clean":"2023-10-10T20:20:10.700388+0000","last_became_active":"2023-10-10T20:20:09.676510+0000","last_became_peered":"2023-10-10T20:20:09.676510+0000","last_unstale":"2023-10-10T20:20:10.700388+0000","last_undegraded":"2023-10-10T20:20:10.700388+0000","last_fullsized":"2023-10-10T20:20:10.700388+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:46:43.101844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.490822+0000","last_change":"2023-10-10T20:20:10.490689+0000","last_active":"2023-10-10T20:20:10.490822+0000","last_peered":"2023-10-10T20:20:10.490822+0000","last_clean":"2023-10-10T20:20:10.490822+0000","last_became_active":"2023-10-10T20:20:10.490461+0000","last_became_peered":"2023-10-10T20:20:10.490461+0000","last_unstale":"2023-10-10T20:20:10.490822+0000","last_undegraded":"2023-10-10T20:20:10.490822+0000","last_fullsized":"2023-10-10T20:20:10.490822+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:51.721701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476840+0000","last_change":"2023-10-10T20:20:09.684269+0000","last_active":"2023-10-10T20:20:10.476840+0000","last_peered":"2023-10-10T20:20:10.476840+0000","last_clean":"2023-10-10T20:20:10.476840+0000","last_became_active":"2023-10-10T20:20:09.683954+0000","last_became_peered":"2023-10-10T20:20:09.683954+0000","last_unstale":"2023-10-10T20:20:10.476840+0000","last_undegraded":"2023-10-10T20:20:10.476840+0000","last_fullsized":"2023-10-10T20:20:10.476840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:36:23.499956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484788+0000","last_change":"2023-10-10T20:20:10.483948+0000","last_active":"2023-10-10T20:20:10.484788+0000","last_peered":"2023-10-10T20:20:10.484788+0000","last_clean":"2023-10-10T20:20:10.484788+0000","last_became_active":"2023-10-10T20:20:10.483551+0000","last_became_peered":"2023-10-10T20:20:10.483551+0000","last_unstale":"2023-10-10T20:20:10.484788+0000","last_undegraded":"2023-10-10T20:20:10.484788+0000","last_fullsized":"2023-10-10T20:20:10.484788+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:27.507265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.702064+0000","last_change":"2023-10-10T20:20:09.679569+0000","last_active":"2023-10-10T20:20:10.702064+0000","last_peered":"2023-10-10T20:20:10.702064+0000","last_clean":"2023-10-10T20:20:10.702064+0000","last_became_active":"2023-10-10T20:20:09.679068+0000","last_became_peered":"2023-10-10T20:20:09.679068+0000","last_unstale":"2023-10-10T20:20:10.702064+0000","last_undegraded":"2023-10-10T20:20:10.702064+0000","last_fullsized":"2023-10-10T20:20:10.702064+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:45:56.012282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483867+0000","last_change":"2023-10-10T20:20:10.482993+0000","last_active":"2023-10-10T20:20:10.483867+0000","last_peered":"2023-10-10T20:20:10.483867+0000","last_clean":"2023-10-10T20:20:10.483867+0000","last_became_active":"2023-10-10T20:20:10.482763+0000","last_became_peered":"2023-10-10T20:20:10.482763+0000","last_unstale":"2023-10-10T20:20:10.483867+0000","last_undegraded":"2023-10-10T20:20:10.483867+0000","last_fullsized":"2023-10-10T20:20:10.483867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:19:10.526457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479167+0000","last_change":"2023-10-10T20:20:09.679727+0000","last_active":"2023-10-10T20:20:10.479167+0000","last_peered":"2023-10-10T20:20:10.479167+0000","last_clean":"2023-10-10T20:20:10.479167+0000","last_became_active":"2023-10-10T20:20:09.679459+0000","last_became_peered":"2023-10-10T20:20:09.679459+0000","last_unstale":"2023-10-10T20:20:10.479167+0000","last_undegraded":"2023-10-10T20:20:10.479167+0000","last_fullsized":"2023-10-10T20:20:10.479167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483848+0000","last_change":"2023-10-10T20:20:10.482340+0000","last_active":"2023-10-10T20:20:10.483848+0000","last_peered":"2023-10-10T20:20:10.483848+0000","last_clean":"2023-10-10T20:20:10.483848+0000","last_became_active":"2023-10-10T20:20:10.482128+0000","last_became_peered":"2023-10-10T20:20:10.482128+0000","last_unstale":"2023-10-10T20:20:10.483848+0000","last_undegraded":"2023-10-10T20:20:10.483848+0000","last_fullsized":"2023-10-10T20:20:10.483848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:28:59.752331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476406+0000","last_change":"2023-10-10T20:20:09.678009+0000","last_active":"2023-10-10T20:20:10.476406+0000","last_peered":"2023-10-10T20:20:10.476406+0000","last_clean":"2023-10-10T20:20:10.476406+0000","last_became_active":"2023-10-10T20:20:09.677524+0000","last_became_peered":"2023-10-10T20:20:09.677524+0000","last_unstale":"2023-10-10T20:20:10.476406+0000","last_undegraded":"2023-10-10T20:20:10.476406+0000","last_fullsized":"2023-10-10T20:20:10.476406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:04:28.410103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489157+0000","last_change":"2023-10-10T20:20:10.489087+0000","last_active":"2023-10-10T20:20:10.489157+0000","last_peered":"2023-10-10T20:20:10.489157+0000","last_clean":"2023-10-10T20:20:10.489157+0000","last_became_active":"2023-10-10T20:20:10.488315+0000","last_became_peered":"2023-10-10T20:20:10.488315+0000","last_unstale":"2023-10-10T20:20:10.489157+0000","last_undegraded":"2023-10-10T20:20:10.489157+0000","last_fullsized":"2023-10-10T20:20:10.489157+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:22:03.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.474726+0000","last_change":"2023-10-10T20:20:09.681856+0000","last_active":"2023-10-10T20:20:10.474726+0000","last_peered":"2023-10-10T20:20:10.474726+0000","last_clean":"2023-10-10T20:20:10.474726+0000","last_became_active":"2023-10-10T20:20:09.681566+0000","last_became_peered":"2023-10-10T20:20:09.681566+0000","last_unstale":"2023-10-10T20:20:10.474726+0000","last_undegraded":"2023-10-10T20:20:10.474726+0000","last_fullsized":"2023-10-10T20:20:10.474726+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:34:16.967295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.486039+0000","last_change":"2023-10-10T20:20:10.485829+0000","last_active":"2023-10-10T20:20:10.486039+0000","last_peered":"2023-10-10T20:20:10.486039+0000","last_clean":"2023-10-10T20:20:10.486039+0000","last_became_active":"2023-10-10T20:20:10.485583+0000","last_became_peered":"2023-10-10T20:20:10.485583+0000","last_unstale":"2023-10-10T20:20:10.486039+0000","last_undegraded":"2023-10-10T20:20:10.486039+0000","last_fullsized":"2023-10-10T20:20:10.486039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:15.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.701177+0000","last_change":"2023-10-10T20:20:09.678682+0000","last_active":"2023-10-10T20:20:10.701177+0000","last_peered":"2023-10-10T20:20:10.701177+0000","last_clean":"2023-10-10T20:20:10.701177+0000","last_became_active":"2023-10-10T20:20:09.678460+0000","last_became_peered":"2023-10-10T20:20:09.678460+0000","last_unstale":"2023-10-10T20:20:10.701177+0000","last_undegraded":"2023-10-10T20:20:10.701177+0000","last_fullsized":"2023-10-10T20:20:10.701177+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.483254+0000","last_change":"2023-10-10T20:20:10.482664+0000","last_active":"2023-10-10T20:20:10.483254+0000","last_peered":"2023-10-10T20:20:10.483254+0000","last_clean":"2023-10-10T20:20:10.483254+0000","last_became_active":"2023-10-10T20:20:10.482399+0000","last_became_peered":"2023-10-10T20:20:10.482399+0000","last_unstale":"2023-10-10T20:20:10.483254+0000","last_undegraded":"2023-10-10T20:20:10.483254+0000","last_fullsized":"2023-10-10T20:20:10.483254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:52.484201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476210+0000","last_change":"2023-10-10T20:20:09.679991+0000","last_active":"2023-10-10T20:20:10.476210+0000","last_peered":"2023-10-10T20:20:10.476210+0000","last_clean":"2023-10-10T20:20:10.476210+0000","last_became_active":"2023-10-10T20:20:09.679338+0000","last_became_peered":"2023-10-10T20:20:09.679338+0000","last_unstale":"2023-10-10T20:20:10.476210+0000","last_undegraded":"2023-10-10T20:20:10.476210+0000","last_fullsized":"2023-10-10T20:20:10.476210+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:14.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484588+0000","last_change":"2023-10-10T20:20:10.484537+0000","last_active":"2023-10-10T20:20:10.484588+0000","last_peered":"2023-10-10T20:20:10.484588+0000","last_clean":"2023-10-10T20:20:10.484588+0000","last_became_active":"2023-10-10T20:20:10.484331+0000","last_became_peered":"2023-10-10T20:20:10.484331+0000","last_unstale":"2023-10-10T20:20:10.484588+0000","last_undegraded":"2023-10-10T20:20:10.484588+0000","last_fullsized":"2023-10-10T20:20:10.484588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:35.987508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479127+0000","last_change":"2023-10-10T20:20:09.676902+0000","last_active":"2023-10-10T20:20:10.479127+0000","last_peered":"2023-10-10T20:20:10.479127+0000","last_clean":"2023-10-10T20:20:10.479127+0000","last_became_active":"2023-10-10T20:20:09.676622+0000","last_became_peered":"2023-10-10T20:20:09.676622+0000","last_unstale":"2023-10-10T20:20:10.479127+0000","last_undegraded":"2023-10-10T20:20:10.479127+0000","last_fullsized":"2023-10-10T20:20:10.479127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:32:50.720040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.484058+0000","last_change":"2023-10-10T20:20:10.483806+0000","last_active":"2023-10-10T20:20:10.484058+0000","last_peered":"2023-10-10T20:20:10.484058+0000","last_clean":"2023-10-10T20:20:10.484058+0000","last_became_active":"2023-10-10T20:20:10.482602+0000","last_became_peered":"2023-10-10T20:20:10.482602+0000","last_unstale":"2023-10-10T20:20:10.484058+0000","last_undegraded":"2023-10-10T20:20:10.484058+0000","last_fullsized":"2023-10-10T20:20:10.484058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:46.352625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.476150+0000","last_change":"2023-10-10T20:20:09.677480+0000","last_active":"2023-10-10T20:20:10.476150+0000","last_peered":"2023-10-10T20:20:10.476150+0000","last_clean":"2023-10-10T20:20:10.476150+0000","last_became_active":"2023-10-10T20:20:09.677261+0000","last_became_peered":"2023-10-10T20:20:09.677261+0000","last_unstale":"2023-10-10T20:20:10.476150+0000","last_undegraded":"2023-10-10T20:20:10.476150+0000","last_fullsized":"2023-10-10T20:20:10.476150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481754+0000","last_change":"2023-10-10T20:20:10.481545+0000","last_active":"2023-10-10T20:20:10.481754+0000","last_peered":"2023-10-10T20:20:10.481754+0000","last_clean":"2023-10-10T20:20:10.481754+0000","last_became_active":"2023-10-10T20:20:10.480932+0000","last_became_peered":"2023-10-10T20:20:10.480932+0000","last_unstale":"2023-10-10T20:20:10.481754+0000","last_undegraded":"2023-10-10T20:20:10.481754+0000","last_fullsized":"2023-10-10T20:20:10.481754+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:50:24.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.480338+0000","last_change":"2023-10-10T20:20:09.679397+0000","last_active":"2023-10-10T20:20:10.480338+0000","last_peered":"2023-10-10T20:20:10.480338+0000","last_clean":"2023-10-10T20:20:10.480338+0000","last_became_active":"2023-10-10T20:20:09.679174+0000","last_became_peered":"2023-10-10T20:20:09.679174+0000","last_unstale":"2023-10-10T20:20:10.480338+0000","last_undegraded":"2023-10-10T20:20:10.480338+0000","last_fullsized":"2023-10-10T20:20:10.480338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:03.541574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.485153+0000","last_change":"2023-10-10T20:20:10.485111+0000","last_active":"2023-10-10T20:20:10.485153+0000","last_peered":"2023-10-10T20:20:10.485153+0000","last_clean":"2023-10-10T20:20:10.485153+0000","last_became_active":"2023-10-10T20:20:10.484740+0000","last_became_peered":"2023-10-10T20:20:10.484740+0000","last_unstale":"2023-10-10T20:20:10.485153+0000","last_undegraded":"2023-10-10T20:20:10.485153+0000","last_fullsized":"2023-10-10T20:20:10.485153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:52.113199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.475741+0000","last_change":"2023-10-10T20:20:09.682844+0000","last_active":"2023-10-10T20:20:10.475741+0000","last_peered":"2023-10-10T20:20:10.475741+0000","last_clean":"2023-10-10T20:20:10.475741+0000","last_became_active":"2023-10-10T20:20:09.682098+0000","last_became_peered":"2023-10-10T20:20:09.682098+0000","last_unstale":"2023-10-10T20:20:10.475741+0000","last_undegraded":"2023-10-10T20:20:10.475741+0000","last_fullsized":"2023-10-10T20:20:10.475741+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:07:57.213111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481756+0000","last_change":"2023-10-10T20:20:10.481028+0000","last_active":"2023-10-10T20:20:10.481756+0000","last_peered":"2023-10-10T20:20:10.481756+0000","last_clean":"2023-10-10T20:20:10.481756+0000","last_became_active":"2023-10-10T20:20:10.480547+0000","last_became_peered":"2023-10-10T20:20:10.480547+0000","last_unstale":"2023-10-10T20:20:10.481756+0000","last_undegraded":"2023-10-10T20:20:10.481756+0000","last_fullsized":"2023-10-10T20:20:10.481756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:35:51.514198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479748+0000","last_change":"2023-10-10T20:20:09.686431+0000","last_active":"2023-10-10T20:20:10.479748+0000","last_peered":"2023-10-10T20:20:10.479748+0000","last_clean":"2023-10-10T20:20:10.479748+0000","last_became_active":"2023-10-10T20:20:09.686217+0000","last_became_peered":"2023-10-10T20:20:09.686217+0000","last_unstale":"2023-10-10T20:20:10.479748+0000","last_undegraded":"2023-10-10T20:20:10.479748+0000","last_fullsized":"2023-10-10T20:20:10.479748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:34:16.967295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.489315+0000","last_change":"2023-10-10T20:20:10.489227+0000","last_active":"2023-10-10T20:20:10.489315+0000","last_peered":"2023-10-10T20:20:10.489315+0000","last_clean":"2023-10-10T20:20:10.489315+0000","last_became_active":"2023-10-10T20:20:10.488689+0000","last_became_peered":"2023-10-10T20:20:10.488689+0000","last_unstale":"2023-10-10T20:20:10.489315+0000","last_undegraded":"2023-10-10T20:20:10.489315+0000","last_fullsized":"2023-10-10T20:20:10.489315+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:09.668803+0000","last_clean_scrub_stamp":"2023-10-10T20:20:09.668803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:06.074761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.479281+0000","last_change":"2023-10-10T20:20:09.684685+0000","last_active":"2023-10-10T20:20:10.479281+0000","last_peered":"2023-10-10T20:20:10.479281+0000","last_clean":"2023-10-10T20:20:10.479281+0000","last_became_active":"2023-10-10T20:20:09.684453+0000","last_became_peered":"2023-10-10T20:20:09.684453+0000","last_unstale":"2023-10-10T20:20:10.479281+0000","last_undegraded":"2023-10-10T20:20:10.479281+0000","last_fullsized":"2023-10-10T20:20:10.479281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:08.667142+0000","last_clean_scrub_stamp":"2023-10-10T20:20:08.667142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:30:26.505603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-10T20:20:10.481576+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:20:10.481576+0000","last_peered":"2023-10-10T20:20:10.481576+0000","last_clean":"2023-10-10T20:20:10.481576+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:20:10.481576+0000","last_undegraded":"2023-10-10T20:20:10.481576+0000","last_fullsized":"2023-10-10T20:20:10.481576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865858,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199277089,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.371},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.33600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67900000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":9.7260000000000009,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195189,"statfs":{"total":200000000,"available":199873675,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.372},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.41699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.68799999999999994}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195186,"statfs":{"total":200000000,"available":199871176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.32700000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35599999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872630,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195181,"statfs":{"total":200000000,"available":199865802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:20:15.125 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:20:15.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:20:15.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.277+0000 7ff61ed3a700 1 Processor -- start 2023-10-10T20:20:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.278+0000 7ff61ed3a700 1 -- start start 2023-10-10T20:20:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61ed3a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6180cd630 0x7ff6180b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61ed3a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61806b0a0 con 0x7ff6181542a0 2023-10-10T20:20:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61ed3a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61815ad20 con 0x7ff61806aa00 2023-10-10T20:20:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61ed3a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61815b2e0 con 0x7ff6180cd630 2023-10-10T20:20:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61dd38700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff6181542a0 0x7ff61806a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60040/0 (socket says 172.21.15.148:60040) 2023-10-10T20:20:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61dd38700 1 -- 172.21.15.148:0/823612373 learned_addr learned my addr 172.21.15.148:0/823612373 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61e539700 1 --2- 172.21.15.148:0/823612373 >> v2:172.21.15.148:3300/0 conn(0x7ff6180cd630 0x7ff6180b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61cd36700 1 -- 172.21.15.148:0/823612373 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030601815 0 0) 0x7ff61806b0a0 con 0x7ff6181542a0 2023-10-10T20:20:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61cd36700 1 -- 172.21.15.148:0/823612373 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff600003570 con 0x7ff6181542a0 2023-10-10T20:20:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61e539700 1 -- 172.21.15.148:0/823612373 >> v1:172.21.15.161:6789/0 conn(0x7ff61806aa00 legacy=0x7ff61815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61e539700 1 -- 172.21.15.148:0/823612373 >> v1:172.21.15.148:6789/0 conn(0x7ff6181542a0 legacy=0x7ff61806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61e539700 1 -- 172.21.15.148:0/823612373 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61815b000 con 0x7ff6180cd630 2023-10-10T20:20:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61e539700 1 --2- 172.21.15.148:0/823612373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff6180b65a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff6100098e0 tx=0x7ff6100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b226706b2ac2393 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61cd36700 1 -- 172.21.15.148:0/823612373 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000b640 con 0x7ff6180cd630 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61cd36700 1 -- 172.21.15.148:0/823612373 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000b7a0 con 0x7ff6180cd630 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.279+0000 7ff61cd36700 1 -- 172.21.15.148:0/823612373 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000bb40 con 0x7ff6180cd630 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 -- 172.21.15.148:0/823612373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 msgr2=0x7ff6180b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/823612373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff6180b65a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff6100098e0 tx=0x7ff6100091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 -- 172.21.15.148:0/823612373 shutdown_connections 2023-10-10T20:20:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/823612373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff6180b65a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 -- 172.21.15.148:0/823612373 >> 172.21.15.148:0/823612373 conn(0x7ff618068cb0 msgr2=0x7ff6180690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 -- 172.21.15.148:0/823612373 shutdown_connections 2023-10-10T20:20:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.280+0000 7ff61ed3a700 1 -- 172.21.15.148:0/823612373 wait complete. 2023-10-10T20:20:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.281+0000 7ff61ed3a700 1 Processor -- start 2023-10-10T20:20:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.281+0000 7ff61ed3a700 1 -- start start 2023-10-10T20:20:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 0x7ff6181527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6181542a0 0x7ff618158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618158730 con 0x7ff6180cd630 2023-10-10T20:20:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618152d20 con 0x7ff61806aa00 2023-10-10T20:20:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61ed3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618157b20 con 0x7ff6181542a0 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47402/0 (socket says 172.21.15.148:47402) 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 -- 172.21.15.148:0/3573483631 learned_addr learned my addr 172.21.15.148:0/3573483631 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61dd38700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 0x7ff6181527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61e539700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6181542a0 0x7ff618158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 -- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6181542a0 msgr2=0x7ff618158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6181542a0 0x7ff618158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 -- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 msgr2=0x7ff6181527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 0x7ff6181527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 -- 172.21.15.148:0/3573483631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009420 con 0x7ff6180cd630 2023-10-10T20:20:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61dd38700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 0x7ff6181527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.282+0000 7ff61d537700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c560 tx=0x7ff60c007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.283+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011590 con 0x7ff6180cd630 2023-10-10T20:20:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.283+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c011710 con 0x7ff6180cd630 2023-10-10T20:20:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.283+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618157e00 con 0x7ff6180cd630 2023-10-10T20:20:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.283+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011ad0 con 0x7ff6180cd630 2023-10-10T20:20:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.283+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181583c0 con 0x7ff6180cd630 2023-10-10T20:20:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.284+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618089550 con 0x7ff6180cd630 2023-10-10T20:20:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.284+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff60c01b3b0 con 0x7ff6180cd630 2023-10-10T20:20:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.284+0000 7ff615ffb700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.285+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff60c083910 con 0x7ff6180cd630 2023-10-10T20:20:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.285+0000 7ff61dd38700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.285+0000 7ff61dd38700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 0x7ff6040693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6080016e0 tx=0x7ff608009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.289+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff60c050b80 con 0x7ff6180cd630 2023-10-10T20:20:15.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.546+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff618158870 con 0x7ff6180cd630 2023-10-10T20:20:15.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.550+0000 7ff615ffb700 1 -- 172.21.15.148:0/3573483631 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff60c017070 con 0x7ff6180cd630 2023-10-10T20:20:15.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 msgr2=0x7ff6040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 0x7ff6040693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6080016e0 tx=0x7ff608009040 comp rx=0 tx=0).stop 2023-10-10T20:20:15.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 msgr2=0x7ff61816b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c560 tx=0x7ff60c007f20 comp rx=0 tx=0).stop 2023-10-10T20:20:15.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 shutdown_connections 2023-10-10T20:20:15.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff604066f30 0x7ff6040693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6181542a0 0x7ff618158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff61806aa00 0x7ff6181527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 --2- 172.21.15.148:0/3573483631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6180cd630 0x7ff61816b0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 >> 172.21.15.148:0/3573483631 conn(0x7ff618068cb0 msgr2=0x7ff6180b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:15.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 shutdown_connections 2023-10-10T20:20:15.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.552+0000 7ff61ed3a700 1 -- 172.21.15.148:0/3573483631 wait complete. 2023-10-10T20:20:15.556 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:20:15.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:15.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.678+0000 7fef68f4f700 1 Processor -- start 2023-10-10T20:20:15.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.678+0000 7fef68f4f700 1 -- start start 2023-10-10T20:20:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef68f4f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cd6e0 0x7fef640cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef68f4f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef640cdab0 0x7fef6415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef68f4f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640cb7c0 con 0x7fef6415dda0 2023-10-10T20:20:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef68f4f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6415e850 con 0x7fef640cdab0 2023-10-10T20:20:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef68f4f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640cd030 con 0x7fef640cd6e0 2023-10-10T20:20:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef640cdab0 0x7fef6415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef63fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef6415dda0 0x7fef6415cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60050/0 (socket says 172.21.15.148:60050) 2023-10-10T20:20:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef63fff700 1 -- 172.21.15.148:0/2290948800 learned_addr learned my addr 172.21.15.148:0/2290948800 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef640cdab0 0x7fef6415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47414/0 (socket says 172.21.15.148:47414) 2023-10-10T20:20:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef637fe700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cd6e0 0x7fef640cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 -- 172.21.15.148:0/2290948800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cd6e0 msgr2=0x7fef640cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cd6e0 0x7fef640cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 -- 172.21.15.148:0/2290948800 >> v1:172.21.15.148:6789/0 conn(0x7fef6415dda0 legacy=0x7fef6415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef627fc700 1 -- 172.21.15.148:0/2290948800 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818397525 0 0) 0x7fef640cb7c0 con 0x7fef6415dda0 2023-10-10T20:20:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.679+0000 7fef62ffd700 1 -- 172.21.15.148:0/2290948800 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6415e290 con 0x7fef640cdab0 2023-10-10T20:20:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef62ffd700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef640cdab0 0x7fef6415d860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef580098e0 tx=0x7fef580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136ae66cd0d0b814 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef627fc700 1 -- 172.21.15.148:0/2290948800 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800b640 con 0x7fef640cdab0 2023-10-10T20:20:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef627fc700 1 -- 172.21.15.148:0/2290948800 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800b7a0 con 0x7fef640cdab0 2023-10-10T20:20:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef627fc700 1 -- 172.21.15.148:0/2290948800 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800bb40 con 0x7fef640cdab0 2023-10-10T20:20:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 -- 172.21.15.148:0/2290948800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef640cdab0 msgr2=0x7fef6415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef640cdab0 0x7fef6415d860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fef580098e0 tx=0x7fef580091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 -- 172.21.15.148:0/2290948800 shutdown_connections 2023-10-10T20:20:15.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cd6e0 0x7fef640cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 --2- 172.21.15.148:0/2290948800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef640cdab0 0x7fef6415d860 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 -- 172.21.15.148:0/2290948800 >> 172.21.15.148:0/2290948800 conn(0x7fef64067660 msgr2=0x7fef64067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:15.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 -- 172.21.15.148:0/2290948800 shutdown_connections 2023-10-10T20:20:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.680+0000 7fef68f4f700 1 -- 172.21.15.148:0/2290948800 wait complete. 2023-10-10T20:20:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.681+0000 7fef68f4f700 1 Processor -- start 2023-10-10T20:20:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 -- start start 2023-10-10T20:20:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 0x7fef6414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cdab0 0x7fef640ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640c8fa0 con 0x7fef6415dda0 2023-10-10T20:20:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6415c660 con 0x7fef640cd6e0 2023-10-10T20:20:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef68f4f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640c9f20 con 0x7fef640cdab0 2023-10-10T20:20:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef63fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef63fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47418/0 (socket says 172.21.15.148:47418) 2023-10-10T20:20:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.682+0000 7fef63fff700 1 -- 172.21.15.148:0/790885100 learned_addr learned my addr 172.21.15.148:0/790885100 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef637fe700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 0x7fef6414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef62ffd700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cdab0 0x7fef640ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 -- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cdab0 msgr2=0x7fef640ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cdab0 0x7fef640ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 -- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 msgr2=0x7fef6414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 0x7fef6414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 -- 172.21.15.148:0/790885100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009420 con 0x7fef6415dda0 2023-10-10T20:20:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef637fe700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 0x7fef6414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef63fff700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c7e0 tx=0x7fef5c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c005c00 con 0x7fef6415dda0 2023-10-10T20:20:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c005d60 con 0x7fef6415dda0 2023-10-10T20:20:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6414ae10 con 0x7fef6415dda0 2023-10-10T20:20:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004a80 con 0x7fef6415dda0 2023-10-10T20:20:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.683+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef640c98d0 con 0x7fef6415dda0 2023-10-10T20:20:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.684+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef640ca090 con 0x7fef6415dda0 2023-10-10T20:20:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.684+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef5c005030 con 0x7fef6415dda0 2023-10-10T20:20:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.685+0000 7fef4b7fe700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 0x7fef500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:15.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.685+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef5c0839b0 con 0x7fef6415dda0 2023-10-10T20:20:15.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.685+0000 7fef637fe700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 0x7fef500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:15.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.685+0000 7fef637fe700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 0x7fef500693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef54007640 tx=0x7fef5400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:15.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.689+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef5c04b5a0 con 0x7fef6415dda0 2023-10-10T20:20:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.922+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef6415e570 con 0x7fef6415dda0 2023-10-10T20:20:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.922+0000 7fef4b7fe700 1 -- 172.21.15.148:0/790885100 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fef5c050c20 con 0x7fef6415dda0 2023-10-10T20:20:15.925 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:15.925 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 msgr2=0x7fef500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 0x7fef500693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef54007640 tx=0x7fef5400b040 comp rx=0 tx=0).stop 2023-10-10T20:20:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 msgr2=0x7fef640b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c7e0 tx=0x7fef5c007d90 comp rx=0 tx=0).stop 2023-10-10T20:20:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 shutdown_connections 2023-10-10T20:20:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef50066f30 0x7fef500693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef6415dda0 0x7fef640b4a70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.924+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef640cd6e0 0x7fef6414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.925+0000 7fef68f4f700 1 --2- 172.21.15.148:0/790885100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef640cdab0 0x7fef640ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:15.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.925+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 >> 172.21.15.148:0/790885100 conn(0x7fef64067660 msgr2=0x7fef641567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:15.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.925+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 shutdown_connections 2023-10-10T20:20:15.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:15.925+0000 7fef68f4f700 1 -- 172.21.15.148:0/790885100 wait complete. 2023-10-10T20:20:15.929 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:16.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:20:10.053909+0000', 'modified': '2023-10-10T20:20:13.716539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4944}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4944': {'gid': 4944, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/812978511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 812978511}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 812978511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:20:16.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:20:16.342 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-10T20:20:16.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.457+0000 7f4e33175700 1 Processor -- start 2023-10-10T20:20:16.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.457+0000 7f4e33175700 1 -- start start 2023-10-10T20:20:16.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4e2c159da0 0x7f4e2c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c15a170 0x7f4e2c068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e2c069040 0x7f4e2c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c0694e0 con 0x7f4e2c069040 2023-10-10T20:20:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c15ea70 con 0x7f4e2c159da0 2023-10-10T20:20:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e33175700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0b3990 con 0x7f4e2c15a170 2023-10-10T20:20:16.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e2c069040 0x7f4e2c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e2c069040 0x7f4e2c0af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47430/0 (socket says 172.21.15.148:47430) 2023-10-10T20:20:16.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 -- 172.21.15.148:0/1307438053 learned_addr learned my addr 172.21.15.148:0/1307438053 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32173700 1 --2- 172.21.15.148:0/1307438053 >> v2:172.21.15.161:3300/0 conn(0x7f4e2c159da0 0x7f4e2c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e31972700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c15a170 0x7f4e2c068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 -- 172.21.15.148:0/1307438053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c15a170 msgr2=0x7f4e2c068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c15a170 0x7f4e2c068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 -- 172.21.15.148:0/1307438053 >> v2:172.21.15.161:3300/0 conn(0x7f4e2c159da0 msgr2=0x7f4e2c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 --2- 172.21.15.148:0/1307438053 >> v2:172.21.15.161:3300/0 conn(0x7f4e2c159da0 0x7f4e2c0b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.458+0000 7f4e32974700 1 -- 172.21.15.148:0/1307438053 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0b0b70 con 0x7f4e2c069040 2023-10-10T20:20:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e32974700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0af4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c06b290 tx=0x7f4e24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99393dc097b426 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e31171700 1 -- 172.21.15.148:0/1307438053 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400b860 con 0x7f4e2c069040 2023-10-10T20:20:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e31171700 1 -- 172.21.15.148:0/1307438053 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2400b9c0 con 0x7f4e2c069040 2023-10-10T20:20:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e31171700 1 -- 172.21.15.148:0/1307438053 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400bd80 con 0x7f4e2c069040 2023-10-10T20:20:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 -- 172.21.15.148:0/1307438053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 msgr2=0x7f4e2c0af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0af4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c06b290 tx=0x7f4e24009250 comp rx=0 tx=0).stop 2023-10-10T20:20:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 -- 172.21.15.148:0/1307438053 shutdown_connections 2023-10-10T20:20:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0af4a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 --2- 172.21.15.148:0/1307438053 >> v2:172.21.15.161:3300/0 conn(0x7f4e2c159da0 0x7f4e2c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 --2- 172.21.15.148:0/1307438053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c15a170 0x7f4e2c068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 -- 172.21.15.148:0/1307438053 >> 172.21.15.148:0/1307438053 conn(0x7f4e2c0d0bc0 msgr2=0x7f4e2c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 -- 172.21.15.148:0/1307438053 shutdown_connections 2023-10-10T20:20:16.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.459+0000 7f4e33175700 1 -- 172.21.15.148:0/1307438053 wait complete. 2023-10-10T20:20:16.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.460+0000 7f4e33175700 1 Processor -- start 2023-10-10T20:20:16.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.460+0000 7f4e33175700 1 -- start start 2023-10-10T20:20:16.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c159da0 0x7f4e2c14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e2c15a170 0x7f4e2c0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0c6380 con 0x7f4e2c069040 2023-10-10T20:20:16.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c14f680 con 0x7f4e2c15a170 2023-10-10T20:20:16.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e33175700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c16be80 con 0x7f4e2c159da0 2023-10-10T20:20:16.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e32173700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e32173700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47432/0 (socket says 172.21.15.148:47432) 2023-10-10T20:20:16.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e32173700 1 -- 172.21.15.148:0/3144437342 learned_addr learned my addr 172.21.15.148:0/3144437342 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e32974700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e2c15a170 0x7f4e2c0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.461+0000 7f4e31972700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c159da0 0x7f4e2c14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 -- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c159da0 msgr2=0x7f4e2c14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c159da0 0x7f4e2c14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 -- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e2c15a170 msgr2=0x7f4e2c0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e2c15a170 0x7f4e2c0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 -- 172.21.15.148:0/3144437342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240094f0 con 0x7f4e2c069040 2023-10-10T20:20:16.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e32173700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e190 tx=0x7f4e28009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e227fc700 1 -- 172.21.15.148:0/3144437342 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800ed80 con 0x7f4e2c069040 2023-10-10T20:20:16.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e227fc700 1 -- 172.21.15.148:0/3144437342 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28014420 con 0x7f4e2c069040 2023-10-10T20:20:16.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c0c60a0 con 0x7f4e2c069040 2023-10-10T20:20:16.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e227fc700 1 -- 172.21.15.148:0/3144437342 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800eee0 con 0x7f4e2c069040 2023-10-10T20:20:16.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.462+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c16bb10 con 0x7f4e2c069040 2023-10-10T20:20:16.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.463+0000 7f4e227fc700 1 -- 172.21.15.148:0/3144437342 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e28014a70 con 0x7f4e2c069040 2023-10-10T20:20:16.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.464+0000 7f4e227fc700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 0x7f4e100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.464+0000 7f4e227fc700 1 -- 172.21.15.148:0/3144437342 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e28084cb0 con 0x7f4e2c069040 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.464+0000 7f4e31972700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 0x7f4e100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.464+0000 7f4e31972700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 0x7f4e100693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0c7d10 tx=0x7f4e1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 Processor -- start 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 -- start start 2023-10-10T20:20:16.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 0x558f2ac137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac34850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ab40a70 con 0x558f2abe9b80 2023-10-10T20:20:16.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ab3e270 con 0x558f2abe75f0 2023-10-10T20:20:16.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f2ab3a060 con 0x558f2ac34480 2023-10-10T20:20:16.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47438/0 (socket says 172.21.15.148:47438) 2023-10-10T20:20:16.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.466+0000 7f4e32974700 1 -- 172.21.15.148:0/4237240604 learned_addr learned my addr 172.21.15.148:0/4237240604 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e31972700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac34850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32173700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 0x558f2ac137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 -- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 msgr2=0x558f2ac137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 0x558f2ac137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 -- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 msgr2=0x558f2ac34850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac34850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 -- 172.21.15.148:0/4237240604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2a9d4270 con 0x558f2abe9b80 2023-10-10T20:20:16.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32173700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 0x558f2ac137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:16.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e32974700 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b3c0 tx=0x7f4e24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a985c689f546bcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e0f7fe700 1 -- 172.21.15.148:0/4237240604 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400bb30 con 0x558f2abe9b80 2023-10-10T20:20:16.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e0f7fe700 1 -- 172.21.15.148:0/4237240604 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24020040 con 0x558f2abe9b80 2023-10-10T20:20:16.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.467+0000 7f4e0f7fe700 1 -- 172.21.15.148:0/4237240604 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400bee0 con 0x558f2abe9b80 2023-10-10T20:20:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/4237240604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 msgr2=0x558f2abe9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b3c0 tx=0x7f4e24017000 comp rx=0 tx=0).stop 2023-10-10T20:20:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/4237240604 shutdown_connections 2023-10-10T20:20:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe9ff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2ac34480 0x558f2ac137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/4237240604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac34850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/4237240604 >> 172.21.15.148:0/4237240604 conn(0x558f2abeb0c0 msgr2=0x558f2abeb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/4237240604 shutdown_connections 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/4237240604 wait complete. 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 -- start start 2023-10-10T20:20:16.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.468+0000 7f4e3d6d0b80 1 Processor -- start 2023-10-10T20:20:16.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2ac30150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 0x558f2abe0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2aae5890 con 0x558f2abe9b80 2023-10-10T20:20:16.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ab00ea0 con 0x558f2ac34480 2023-10-10T20:20:16.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f2a9bce30 con 0x558f2abe75f0 2023-10-10T20:20:16.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e3d6d0b80 10 client.0 _ll_register_callbacks cb 0x558f2abad2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:20:16.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e31972700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e31972700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47444/0 (socket says 172.21.15.148:47444) 2023-10-10T20:20:16.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e31972700 1 -- 172.21.15.148:0/2674582645 learned_addr learned my addr 172.21.15.148:0/2674582645 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.469+0000 7f4e32173700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2ac30150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e32974700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 0x558f2abe0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 -- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 msgr2=0x558f2ac30150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2ac30150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 -- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 msgr2=0x558f2abe0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 0x558f2abe0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 -- 172.21.15.148:0/2674582645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c0b0b70 con 0x558f2abe9b80 2023-10-10T20:20:16.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e32974700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 0x558f2abe0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:16.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e32173700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2ac30150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:16.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e31972700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c014810 tx=0x7f4e1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca6fee5fcb935c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:20:16.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c015720 con 0x558f2abe9b80 2023-10-10T20:20:16.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e1c015880 con 0x558f2abe9b80 2023-10-10T20:20:16.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f2abdecd0 con 0x558f2abe9b80 2023-10-10T20:20:16.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1c015c50 con 0x558f2abe9b80 2023-10-10T20:20:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e3d6d0b80 10 client.5142 fetch_fsmap learned FSMap version 17 2023-10-10T20:20:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558f2a976fe0 con 0x558f2abe9b80 2023-10-10T20:20:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f2abdecd0 con 0x558f2abe9b80 2023-10-10T20:20:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.470+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4e1c02a440 con 0x558f2abe9b80 2023-10-10T20:20:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 10 client.5142 fetch_fsmap finished waiting for FSMap version 17 2023-10-10T20:20:16.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 10 client.5142 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:20:16.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 20 client.5142 populate_metadata read hostname 'smithi148' 2023-10-10T20:20:16.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac14130 0x558f2ac4dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 4 client.5142 mds_command: new command op to 4944 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:20:16.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 --> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f2abbbaa0 con 0x558f2ac14130 2023-10-10T20:20:16.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e32173700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac14130 0x558f2ac4dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e0cff9700 10 client.5142 ms_handle_connect on v2:172.21.15.161:6833/812978511 2023-10-10T20:20:16.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.471+0000 7f4e32173700 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac14130 0x558f2ac4dc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70a1f56d6fa0137 server_cookie=488ab6c4f3a0f4f4 in_seq=0 out_seq=0 2023-10-10T20:20:16.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e0cff9700 1 -- 172.21.15.148:0/2674582645 <== mds.0 v2:172.21.15.161:6833/812978511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558f2abbbaa0 con 0x558f2ac14130 2023-10-10T20:20:16.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e0cff9700 10 client.5142 handle_command_reply: tid=0 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 client.5142 shutdown 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 msgr2=0x558f2abe0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c014810 tx=0x7f4e1c016000 comp rx=0 tx=0).stop 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 shutdown_connections 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac14130 0x558f2ac4dc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2ac34480 0x558f2abe0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2ac30150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e0cff9700 0 client.5142 ms_handle_reset on v2:172.21.15.161:6833/812978511 2023-10-10T20:20:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2674582645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2abe9b80 0x558f2abe0650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 >> 172.21.15.148:0/2674582645 conn(0x558f2abeb0c0 msgr2=0x558f2abe9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 shutdown_connections 2023-10-10T20:20:16.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2674582645 wait complete. 2023-10-10T20:20:16.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.472+0000 7f4e3d6d0b80 20 client.5142 trim_cache size 0 max 16384 2023-10-10T20:20:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.489+0000 7f4e3d6d0b80 1 Processor -- start 2023-10-10T20:20:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.489+0000 7f4e3d6d0b80 1 -- start start 2023-10-10T20:20:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2abed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe9b80 0x558f2abe6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2abecf30 con 0x558f2ac14130 2023-10-10T20:20:16.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ac17bc0 con 0x558f2abe9b80 2023-10-10T20:20:16.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f2ac19870 con 0x558f2abe75f0 2023-10-10T20:20:16.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47460/0 (socket says 172.21.15.148:47460) 2023-10-10T20:20:16.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 -- 172.21.15.148:0/2983325453 learned_addr learned my addr 172.21.15.148:0/2983325453 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e31972700 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe9b80 0x558f2abe6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32173700 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2abed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 -- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 msgr2=0x558f2abed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2abed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 -- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe9b80 msgr2=0x558f2abe6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe9b80 0x558f2abe6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 -- 172.21.15.148:0/2983325453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2ab08780 con 0x558f2ac14130 2023-10-10T20:20:16.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.490+0000 7f4e32974700 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000c00 tx=0x7f4e24021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300a6264335a517b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e0ffff700 1 -- 172.21.15.148:0/2983325453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24009300 con 0x558f2ac14130 2023-10-10T20:20:16.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e0ffff700 1 -- 172.21.15.148:0/2983325453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2400b650 con 0x558f2ac14130 2023-10-10T20:20:16.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e0ffff700 1 -- 172.21.15.148:0/2983325453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400bd80 con 0x558f2ac14130 2023-10-10T20:20:16.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2983325453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 msgr2=0x558f2abea070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000c00 tx=0x7f4e24021000 comp rx=0 tx=0).stop 2023-10-10T20:20:16.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2983325453 shutdown_connections 2023-10-10T20:20:16.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2abea070 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe75f0 0x558f2abed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/2983325453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe9b80 0x558f2abe6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2983325453 >> 172.21.15.148:0/2983325453 conn(0x558f2abeb0c0 msgr2=0x558f2ac34850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2983325453 shutdown_connections 2023-10-10T20:20:16.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.491+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/2983325453 wait complete. 2023-10-10T20:20:16.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.492+0000 7f4e3d6d0b80 1 -- start start 2023-10-10T20:20:16.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.492+0000 7f4e3d6d0b80 1 Processor -- start 2023-10-10T20:20:16.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe9b80 0x558f2ac8cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ac95e60 con 0x558f2ac14130 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2ac0e270 con 0x558f2abe75f0 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f2abea660 con 0x558f2abe9b80 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 10 client.0 _ll_register_callbacks cb 0x558f2abad2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47468/0 (socket says 172.21.15.148:47468) 2023-10-10T20:20:16.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 -- 172.21.15.148:0/3324807092 learned_addr learned my addr 172.21.15.148:0/3324807092 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32173700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e31972700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe9b80 0x558f2ac8cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 -- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe9b80 msgr2=0x558f2ac8cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe9b80 0x558f2ac8cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 -- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 msgr2=0x558f2ac89890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac89890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 -- 172.21.15.148:0/3324807092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e24029040 con 0x558f2ac14130 2023-10-10T20:20:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e32974700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e24033040 tx=0x7f4e2402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c9b9051bfe579a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:20:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400be10 con 0x558f2ac14130 2023-10-10T20:20:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e24011a50 con 0x558f2ac14130 2023-10-10T20:20:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e24020040 con 0x558f2ac14130 2023-10-10T20:20:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e24011e40 con 0x558f2ac14130 2023-10-10T20:20:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 10 client.5148 fetch_fsmap learned FSMap version 17 2023-10-10T20:20:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.493+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558f2aaef470 con 0x558f2ac14130 2023-10-10T20:20:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24020040 con 0x558f2ac14130 2023-10-10T20:20:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4e2402c070 con 0x558f2ac14130 2023-10-10T20:20:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 10 client.5148 fetch_fsmap finished waiting for FSMap version 17 2023-10-10T20:20:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 10 client.5148 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:20:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 20 client.5148 populate_metadata read hostname 'smithi148' 2023-10-10T20:20:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac34480 0x558f2ac0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 4 client.5148 mds_command: new command op to 4944 tid=0[{"prefix": "status"}] 2023-10-10T20:20:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 --> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f2aadce00 con 0x558f2ac34480 2023-10-10T20:20:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e32173700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac34480 0x558f2ac0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e0cff9700 10 client.5148 ms_handle_connect on v2:172.21.15.161:6833/812978511 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e32173700 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac34480 0x558f2ac0e980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beda5cc7689bb920 server_cookie=7dd5f03e59c936e3 in_seq=0 out_seq=0 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e0cff9700 1 -- 172.21.15.148:0/3324807092 <== mds.0 v2:172.21.15.161:6833/812978511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558f2aadce00 con 0x558f2ac34480 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e0cff9700 10 client.5148 handle_command_reply: tid=0 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.494+0000 7f4e3d6d0b80 1 client.5148 shutdown 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 msgr2=0x558f2ac8d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e24033040 tx=0x7f4e2402c000 comp rx=0 tx=0).stop 2023-10-10T20:20:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 shutdown_connections 2023-10-10T20:20:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f2ac14130 0x558f2ac8d2d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:6833/812978511,v1:172.21.15.161:6835/812978511] conn(0x558f2ac34480 0x558f2ac0e980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f2abe75f0 0x558f2ac89890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 --2- 172.21.15.148:0/3324807092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x558f2abe9b80 0x558f2ac8cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e0cff9700 0 client.5148 ms_handle_reset on v2:172.21.15.161:6833/812978511 2023-10-10T20:20:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 >> 172.21.15.148:0/3324807092 conn(0x558f2abeb0c0 msgr2=0x558f2abe6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 shutdown_connections 2023-10-10T20:20:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 1 -- 172.21.15.148:0/3324807092 wait complete. 2023-10-10T20:20:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e3d6d0b80 20 client.5148 trim_cache size 0 max 16384 2023-10-10T20:20:16.522 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:16.522 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:16.522 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:20:16.522 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4944, 2023-10-10T20:20:16.523 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:20:16.523 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:20:16.523 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:20:16.523 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.8013124280000001, 2023-10-10T20:20:16.523 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 17, 2023-10-10T20:20:16.524 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 31, 2023-10-10T20:20:16.524 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 31, 2023-10-10T20:20:16.524 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 13.69356123 2023-10-10T20:20:16.524 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:16.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 msgr2=0x7f4e100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 0x7f4e100693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0c7d10 tx=0x7f4e1c008040 comp rx=0 tx=0).stop 2023-10-10T20:20:16.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 msgr2=0x7f4e2c0c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e190 tx=0x7f4e28009920 comp rx=0 tx=0).stop 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 shutdown_connections 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e10066f30 0x7f4e100693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e2c15a170 0x7f4e2c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e2c069040 0x7f4e2c0c52e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 --2- 172.21.15.148:0/3144437342 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e2c159da0 0x7f4e2c14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 >> 172.21.15.148:0/3144437342 conn(0x7f4e2c0d0bc0 msgr2=0x7f4e2c0b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 shutdown_connections 2023-10-10T20:20:16.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.495+0000 7f4e33175700 1 -- 172.21.15.148:0/3144437342 wait complete. 2023-10-10T20:20:16.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:20:16.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 Processor -- start 2023-10-10T20:20:16.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 -- start start 2023-10-10T20:20:16.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157870 0x7f407815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4078157c40 0x7f4078158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407815cbb0 con 0x7f407815a620 2023-10-10T20:20:16.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407815bfa0 con 0x7f4078157c40 2023-10-10T20:20:16.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.642+0000 7f407d186700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407815c280 con 0x7f4078157870 2023-10-10T20:20:16.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f406e59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f407815a620 0x7f407815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60066/0 (socket says 172.21.15.148:60066) 2023-10-10T20:20:16.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4078157c40 0x7f4078158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f406e59c700 1 -- 172.21.15.148:0/509648966 learned_addr learned my addr 172.21.15.148:0/509648966 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f4076d9d700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157870 0x7f407815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 -- 172.21.15.148:0/509648966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157870 msgr2=0x7f407815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157870 0x7f407815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 -- 172.21.15.148:0/509648966 >> v1:172.21.15.148:6789/0 conn(0x7f407815a620 legacy=0x7f407815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407659c700 1 -- 172.21.15.148:0/509648966 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929891349 0 0) 0x7f407815cbb0 con 0x7f407815a620 2023-10-10T20:20:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 -- 172.21.15.148:0/509648966 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407815c560 con 0x7f4078157c40 2023-10-10T20:20:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407759e700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4078157c40 0x7f4078158110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4070009e60 tx=0x7f4070009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471704d7d422fc4f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407659c700 1 -- 172.21.15.148:0/509648966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000e070 con 0x7f4078157c40 2023-10-10T20:20:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407659c700 1 -- 172.21.15.148:0/509648966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000bc30 con 0x7f4078157c40 2023-10-10T20:20:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.643+0000 7f407659c700 1 -- 172.21.15.148:0/509648966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070012430 con 0x7f4078157c40 2023-10-10T20:20:16.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 -- 172.21.15.148:0/509648966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4078157c40 msgr2=0x7f4078158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4078157c40 0x7f4078158110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4070009e60 tx=0x7f4070009780 comp rx=0 tx=0).stop 2023-10-10T20:20:16.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 -- 172.21.15.148:0/509648966 shutdown_connections 2023-10-10T20:20:16.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4078157c40 0x7f4078158110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 --2- 172.21.15.148:0/509648966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157870 0x7f407815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 -- 172.21.15.148:0/509648966 >> 172.21.15.148:0/509648966 conn(0x7f4078068ce0 msgr2=0x7f40780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 -- 172.21.15.148:0/509648966 shutdown_connections 2023-10-10T20:20:16.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.644+0000 7f407d186700 1 -- 172.21.15.148:0/509648966 wait complete. 2023-10-10T20:20:16.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.645+0000 7f407d186700 1 Processor -- start 2023-10-10T20:20:16.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.645+0000 7f407d186700 1 -- start start 2023-10-10T20:20:16.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078157870 0x7f4078169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157c40 0x7f407816d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780b1740 con 0x7f407815a620 2023-10-10T20:20:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407816a530 con 0x7f4078157870 2023-10-10T20:20:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407815ec10 con 0x7f4078157c40 2023-10-10T20:20:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47480/0 (socket says 172.21.15.148:47480) 2023-10-10T20:20:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 -- 172.21.15.148:0/2969473666 learned_addr learned my addr 172.21.15.148:0/2969473666 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:16.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f4076d9d700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078157870 0x7f4078169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f406e59c700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157c40 0x7f407816d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 -- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157c40 msgr2=0x7f407816d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157c40 0x7f407816d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 -- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078157870 msgr2=0x7f4078169ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078157870 0x7f4078169ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 -- 172.21.15.148:0/2969473666 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009a20 con 0x7f407815a620 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407759e700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f407806b530 tx=0x7f4070019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070027470 con 0x7f407815a620 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40700275f0 con 0x7f407815a620 2023-10-10T20:20:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700279b0 con 0x7f407815a620 2023-10-10T20:20:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.646+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40780b0e10 con 0x7f407815a620 2023-10-10T20:20:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.647+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40780b13d0 con 0x7f407815a620 2023-10-10T20:20:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.647+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780b18f0 con 0x7f407815a620 2023-10-10T20:20:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.648+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4070024070 con 0x7f407815a620 2023-10-10T20:20:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.648+0000 7f406f7fe700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 0x7f40580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.648+0000 7f4076d9d700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 0x7f40580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.648+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4070086da0 con 0x7f407815a620 2023-10-10T20:20:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.649+0000 7f4076d9d700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 0x7f40580693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f406800acb0 tx=0x7f4068008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.653+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4070054010 con 0x7f407815a620 2023-10-10T20:20:16.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.882+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f407804c3c0 con 0x7f407815a620 2023-10-10T20:20:16.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.883+0000 7f406f7fe700 1 -- 172.21.15.148:0/2969473666 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f407002c060 con 0x7f407815a620 2023-10-10T20:20:16.885 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:20:16.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.884+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 msgr2=0x7f40580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.884+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 0x7f40580693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f406800acb0 tx=0x7f4068008040 comp rx=0 tx=0).stop 2023-10-10T20:20:16.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.884+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 msgr2=0x7f4078153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:16.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.884+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f407806b530 tx=0x7f4070019000 comp rx=0 tx=0).stop 2023-10-10T20:20:16.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 shutdown_connections 2023-10-10T20:20:16.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4058066f30 0x7f40580693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f407815a620 0x7f4078153210 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078157870 0x7f4078169ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 --2- 172.21.15.148:0/2969473666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4078157c40 0x7f407816d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:16.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 >> 172.21.15.148:0/2969473666 conn(0x7f4078068ce0 msgr2=0x7f407815ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:16.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 shutdown_connections 2023-10-10T20:20:16.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:16.885+0000 7f407d186700 1 -- 172.21.15.148:0/2969473666 wait complete. 2023-10-10T20:20:16.898 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:20:16.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:20:16.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:20:16.900 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:20:16.900 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:16.900 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:20:16.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:20:16.901 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:20:16.901 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:16.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:16.903 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:20:16.920 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:20:16.921 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:20:16.921 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:20:16.921 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.922 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:20:16.922 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.923 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:16.923 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:16.923 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:20:16.924 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:16.924 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:16.924 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:20:16.925 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:16.925 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:16.925 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:20:16.925 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:16.925 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:20:16.926 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42494sec preferred_lft 42494sec 2023-10-10T20:20:16.926 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:20:16.926 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.926 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:20:16.926 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:16.927 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:20:16.927 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.927 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:20:16.927 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:16.928 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:20:16.929 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:20:16.929 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:16.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:16.930 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:20:16.984 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:20:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:16.986 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:20:17.041 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:20:17.041 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:20:17.042 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:20:17.042 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:20:17.042 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:20:17.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.043 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:20:17.043 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:20:17.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:20:17.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:20:17.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:20:17.045 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:20:17.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.142 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:20:17.157 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:20:17.158 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:20:17.158 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:17.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:20:17.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:17.223 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:17.278 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:17.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:17.279 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:17.332 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:20:17.333 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:20:17.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:17.413 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:17.413 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:17.432 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:17.432 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:17.433 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:17.433 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:17.433 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:17.433 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:17.433 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:17.434 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:17.455 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:17.455 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:17.456 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:17.457 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:17.457 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:20:17.457 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:20:17.457 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:20:17.458 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:20:17.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:17.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:20:17.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.677+0000 7f94306bf700 1 Processor -- start 2023-10-10T20:20:17.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.677+0000 7f94306bf700 1 -- start start 2023-10-10T20:20:17.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f94306bf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f942815e2d0 0x7f942806e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f94306bf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9428063ac0 0x7f94280c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f94306bf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94280cadb0 con 0x7f94280c9540 2023-10-10T20:20:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f94306bf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942806f1c0 con 0x7f9428063ac0 2023-10-10T20:20:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f94306bf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94280c2390 con 0x7f942815e2d0 2023-10-10T20:20:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9428063ac0 0x7f94280c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9428063ac0 0x7f94280c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47486/0 (socket says 172.21.15.148:47486) 2023-10-10T20:20:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 -- 172.21.15.148:0/4208340986 learned_addr learned my addr 172.21.15.148:0/4208340986 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942e45b700 1 --2- 172.21.15.148:0/4208340986 >> v2:172.21.15.161:3300/0 conn(0x7f942815e2d0 0x7f942806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 -- 172.21.15.148:0/4208340986 >> v2:172.21.15.161:3300/0 conn(0x7f942815e2d0 msgr2=0x7f942806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 --2- 172.21.15.148:0/4208340986 >> v2:172.21.15.161:3300/0 conn(0x7f942815e2d0 0x7f942806e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 -- 172.21.15.148:0/4208340986 >> v1:172.21.15.148:6789/0 conn(0x7f94280c9540 legacy=0x7f9428161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 -- 172.21.15.148:0/4208340986 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942806ec00 con 0x7f9428063ac0 2023-10-10T20:20:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942d459700 1 -- 172.21.15.148:0/4208340986 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901949362 0 0) 0x7f94280cadb0 con 0x7f94280c9540 2023-10-10T20:20:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942dc5a700 1 --2- 172.21.15.148:0/4208340986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94280c9000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f94280ce2f0 tx=0x7f9418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad90552f8752ad74 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942d459700 1 -- 172.21.15.148:0/4208340986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418009e70 con 0x7f9428063ac0 2023-10-10T20:20:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942d459700 1 -- 172.21.15.148:0/4208340986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941800b860 con 0x7f9428063ac0 2023-10-10T20:20:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.679+0000 7f942d459700 1 -- 172.21.15.148:0/4208340986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941800bc00 con 0x7f9428063ac0 2023-10-10T20:20:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 -- 172.21.15.148:0/4208340986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 msgr2=0x7f94280c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 --2- 172.21.15.148:0/4208340986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94280c9000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f94280ce2f0 tx=0x7f9418009250 comp rx=0 tx=0).stop 2023-10-10T20:20:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 -- 172.21.15.148:0/4208340986 shutdown_connections 2023-10-10T20:20:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 --2- 172.21.15.148:0/4208340986 >> v2:172.21.15.161:3300/0 conn(0x7f942815e2d0 0x7f942806e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 --2- 172.21.15.148:0/4208340986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94280c9000 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 -- 172.21.15.148:0/4208340986 >> 172.21.15.148:0/4208340986 conn(0x7f94280cd1e0 msgr2=0x7f94280cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:17.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 -- 172.21.15.148:0/4208340986 shutdown_connections 2023-10-10T20:20:17.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.680+0000 7f94306bf700 1 -- 172.21.15.148:0/4208340986 wait complete. 2023-10-10T20:20:17.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.681+0000 7f94306bf700 1 Processor -- start 2023-10-10T20:20:17.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.681+0000 7f94306bf700 1 -- start start 2023-10-10T20:20:17.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 0x7f9428153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f942815e2d0 0x7f942815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94280c5910 con 0x7f9428063ac0 2023-10-10T20:20:17.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942809a8e0 con 0x7f94280c9540 2023-10-10T20:20:17.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f94306bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94280c70e0 con 0x7f942815e2d0 2023-10-10T20:20:17.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:17.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47492/0 (socket says 172.21.15.148:47492) 2023-10-10T20:20:17.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 -- 172.21.15.148:0/2789611300 learned_addr learned my addr 172.21.15.148:0/2789611300 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:17.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942dc5a700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 0x7f9428153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:17.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 -- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f942815e2d0 msgr2=0x7f942815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f942815e2d0 0x7f942815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 -- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 msgr2=0x7f9428153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 0x7f9428153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.682+0000 7f942e45b700 1 -- 172.21.15.148:0/2789611300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180094f0 con 0x7f9428063ac0 2023-10-10T20:20:17.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f942dc5a700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 0x7f9428153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:17.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f942e45b700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f942400ec80 tx=0x7f94240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:17.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400c9d0 con 0x7f9428063ac0 2023-10-10T20:20:17.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400cb30 con 0x7f9428063ac0 2023-10-10T20:20:17.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94280c5480 con 0x7f9428063ac0 2023-10-10T20:20:17.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424013470 con 0x7f9428063ac0 2023-10-10T20:20:17.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.683+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94280c4870 con 0x7f9428063ac0 2023-10-10T20:20:17.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.684+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942804c3c0 con 0x7f9428063ac0 2023-10-10T20:20:17.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.684+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9424013820 con 0x7f9428063ac0 2023-10-10T20:20:17.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.685+0000 7f941e7fc700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:17.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.685+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f94240848c0 con 0x7f9428063ac0 2023-10-10T20:20:17.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.685+0000 7f942dc5a700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:17.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.685+0000 7f942dc5a700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 0x7f940c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94280ce2f0 tx=0x7f9418017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:17.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.689+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9424051b30 con 0x7f9428063ac0 2023-10-10T20:20:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.921+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94280af370 con 0x7f9428063ac0 2023-10-10T20:20:17.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.921+0000 7f941e7fc700 1 -- 172.21.15.148:0/2789611300 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9424055150 con 0x7f9428063ac0 2023-10-10T20:20:17.923 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:20:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 msgr2=0x7f940c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 0x7f940c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94280ce2f0 tx=0x7f9418017000 comp rx=0 tx=0).stop 2023-10-10T20:20:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 msgr2=0x7f94281723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f942400ec80 tx=0x7f94240091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 shutdown_connections 2023-10-10T20:20:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f940c066f30 0x7f940c0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f942815e2d0 0x7f942815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9428063ac0 0x7f94281723b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 --2- 172.21.15.148:0/2789611300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94280c9540 0x7f9428153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 >> 172.21.15.148:0/2789611300 conn(0x7f94280cd1e0 msgr2=0x7f94280cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 shutdown_connections 2023-10-10T20:20:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:17.923+0000 7f94306bf700 1 -- 172.21.15.148:0/2789611300 wait complete. 2023-10-10T20:20:17.937 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:20:17.937 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:20:17.938 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:20:17.938 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:20:17.938 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:17.939 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:20:17.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:20:17.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:20:17.940 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:17.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.941 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:20:17.958 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:20:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:20:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:20:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:20:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.960 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:20:17.960 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:20:17.961 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:17.961 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:20:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:20:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42490sec preferred_lft 42490sec 2023-10-10T20:20:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:20:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.963 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:20:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:20:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:20:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.964 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:20:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:20:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.965 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:20:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:20:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:17.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:17.967 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:20:18.024 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:20:18.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:18.026 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:20:18.083 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:20:18.083 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:20:18.083 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:20:18.084 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:20:18.084 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:20:18.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:18.085 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:20:18.085 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:20:18.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:20:18.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:20:18.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:20:18.087 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:20:18.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:18.186 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:20:18.201 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:20:18.201 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:20:18.201 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:18.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:18.202 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:20:18.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:18.266 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:18.320 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:18.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:18.321 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:18.375 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:20:18.375 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:20:18.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:18.455 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:18.455 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:18.474 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:18.474 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:18.474 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:18.475 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:18.475 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:18.475 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:18.475 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:18.475 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:18.495 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:18.496 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:18.496 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:18.496 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:18.496 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:18.496 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:18.497 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:18.497 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:18.497 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:20:18.497 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:20:18.497 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:20:18.498 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:20:18.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:18.499 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:18.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:18.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.677+0000 7fa7f5333700 1 Processor -- start 2023-10-10T20:20:18.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.677+0000 7fa7f5333700 1 -- start start 2023-10-10T20:20:18.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7f5333700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7f0157870 0x7fa7f015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7f5333700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f0157c40 0x7fa7f0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7f5333700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f015cbb0 con 0x7fa7f015a620 2023-10-10T20:20:18.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7f5333700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f015bfa0 con 0x7fa7f0157870 2023-10-10T20:20:18.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7f5333700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015c280 con 0x7fa7f0157c40 2023-10-10T20:20:18.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7eeffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7f0157870 0x7fa7f015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7e67fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa7f015a620 0x7fa7f015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60086/0 (socket says 172.21.15.148:60086) 2023-10-10T20:20:18.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7e67fc700 1 -- 172.21.15.148:0/360874994 learned_addr learned my addr 172.21.15.148:0/360874994 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7ef7fe700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f0157c40 0x7fa7f0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7eeffd700 1 -- 172.21.15.148:0/360874994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f0157c40 msgr2=0x7fa7f0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f0157c40 0x7fa7f0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7eeffd700 1 -- 172.21.15.148:0/360874994 >> v1:172.21.15.148:6789/0 conn(0x7fa7f015a620 legacy=0x7fa7f015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7eeffd700 1 -- 172.21.15.148:0/360874994 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f015c560 con 0x7fa7f0157870 2023-10-10T20:20:18.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.678+0000 7fa7ee7fc700 1 -- 172.21.15.148:0/360874994 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212477802 0 0) 0x7fa7f015cbb0 con 0x7fa7f015a620 2023-10-10T20:20:18.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157870 0x7fa7f015b6a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098e0 tx=0x7fa7e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1f80dc6a0d6af2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:18.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7ee7fc700 1 -- 172.21.15.148:0/360874994 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000b640 con 0x7fa7f0157870 2023-10-10T20:20:18.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7ee7fc700 1 -- 172.21.15.148:0/360874994 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e000b7a0 con 0x7fa7f0157870 2023-10-10T20:20:18.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7ee7fc700 1 -- 172.21.15.148:0/360874994 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000bb40 con 0x7fa7f0157870 2023-10-10T20:20:18.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 -- 172.21.15.148:0/360874994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157870 msgr2=0x7fa7f015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157870 0x7fa7f015b6a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098e0 tx=0x7fa7e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 -- 172.21.15.148:0/360874994 shutdown_connections 2023-10-10T20:20:18.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f0157c40 0x7fa7f0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 --2- 172.21.15.148:0/360874994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157870 0x7fa7f015b6a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 -- 172.21.15.148:0/360874994 >> 172.21.15.148:0/360874994 conn(0x7fa7f0068ce0 msgr2=0x7fa7f00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:18.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 -- 172.21.15.148:0/360874994 shutdown_connections 2023-10-10T20:20:18.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.679+0000 7fa7f5333700 1 -- 172.21.15.148:0/360874994 wait complete. 2023-10-10T20:20:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.680+0000 7fa7f5333700 1 Processor -- start 2023-10-10T20:20:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 -- start start 2023-10-10T20:20:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 0x7fa7f016a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f015a620 0x7fa7f0148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c6ec0 con 0x7fa7f0157c40 2023-10-10T20:20:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015daa0 con 0x7fa7f0157870 2023-10-10T20:20:18.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7f5333700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015d170 con 0x7fa7f015a620 2023-10-10T20:20:18.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7e67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7e67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47502/0 (socket says 172.21.15.148:47502) 2023-10-10T20:20:18.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7e67fc700 1 -- 172.21.15.148:0/768574441 learned_addr learned my addr 172.21.15.148:0/768574441 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:18.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7ef7fe700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f015a620 0x7fa7f0148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.681+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 0x7fa7f016a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 -- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f015a620 msgr2=0x7fa7f0148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f015a620 0x7fa7f0148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 -- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 msgr2=0x7fa7f016a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 0x7fa7f016a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 -- 172.21.15.148:0/768574441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0009420 con 0x7fa7f0157c40 2023-10-10T20:20:18.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 0x7fa7f016a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:18.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e67fc700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c5e0 tx=0x7fa7d8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:18.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8005b30 con 0x7fa7f0157c40 2023-10-10T20:20:18.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8005c90 con 0x7fa7f0157c40 2023-10-10T20:20:18.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f015d450 con 0x7fa7f0157c40 2023-10-10T20:20:18.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80072e0 con 0x7fa7f0157c40 2023-10-10T20:20:18.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.682+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f016a930 con 0x7fa7f0157c40 2023-10-10T20:20:18.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.683+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f004c3c0 con 0x7fa7f0157c40 2023-10-10T20:20:18.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.683+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7d801b3b0 con 0x7fa7f0157c40 2023-10-10T20:20:18.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.684+0000 7fa7e77fe700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 0x7fa7d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:18.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.684+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 0x7fa7d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:18.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.684+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7d8083880 con 0x7fa7f0157c40 2023-10-10T20:20:18.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.684+0000 7fa7eeffd700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 0x7fa7d00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00093f0 tx=0x7fa7e0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:18.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.688+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7d804b5a0 con 0x7fa7f0157c40 2023-10-10T20:20:18.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.916+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f015dbe0 con 0x7fa7f0157c40 2023-10-10T20:20:18.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.916+0000 7fa7e77fe700 1 -- 172.21.15.148:0/768574441 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fa7d8017090 con 0x7fa7f0157c40 2023-10-10T20:20:18.919 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:18.919 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 msgr2=0x7fa7d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 0x7fa7d00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00093f0 tx=0x7fa7e0017000 comp rx=0 tx=0).stop 2023-10-10T20:20:18.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 msgr2=0x7fa7f00c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c5e0 tx=0x7fa7d8000fe0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 shutdown_connections 2023-10-10T20:20:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7d0066f30 0x7fa7d00693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7f015a620 0x7fa7f0148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f0157870 0x7fa7f016a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 --2- 172.21.15.148:0/768574441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7f0157c40 0x7fa7f00c72f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 >> 172.21.15.148:0/768574441 conn(0x7fa7f0068ce0 msgr2=0x7fa7f00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 shutdown_connections 2023-10-10T20:20:18.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:18.919+0000 7fa7f5333700 1 -- 172.21.15.148:0/768574441 wait complete. 2023-10-10T20:20:18.924 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:18.933 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-10T20:20:19.067 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":4944,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3620812489999992,"mdsmap_epoch":17,"osdmap_epoch":31,"osdmap_epoch_barrier":31,"uptime":16.254332306999999} 2023-10-10T20:20:19.067 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 4944, "mdsmap_epoch": 17, "osdmap_epoch": 31, "osdmap_epoch_barrier": 31, "rank_uptime": 8.362081249, "state": "up:active", "uptime": 16.254332307, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:20:19.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-10-10T20:20:19.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.180+0000 7fe5c8443700 1 Processor -- start 2023-10-10T20:20:19.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.180+0000 7fe5c8443700 1 -- start start 2023-10-10T20:20:19.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c8443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c8443700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 0x7fe5c0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c8443700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c015cbb0 con 0x7fe5c0157870 2023-10-10T20:20:19.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c8443700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c015bdb0 con 0x7fe5c015d580 2023-10-10T20:20:19.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c8443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c015bfa0 con 0x7fe5c015a620 2023-10-10T20:20:19.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c61df700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe5c0157870 0x7fe5c015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60098/0 (socket says 172.21.15.148:60098) 2023-10-10T20:20:19.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.181+0000 7fe5c61df700 1 -- 172.21.15.148:0/3926371675 learned_addr learned my addr 172.21.15.148:0/3926371675 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:19.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 --2- 172.21.15.148:0/3926371675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c69e0700 1 --2- 172.21.15.148:0/3926371675 >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 0x7fe5c0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c51dd700 1 -- 172.21.15.148:0/3926371675 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175755622 0 0) 0x7fe5c015cbb0 con 0x7fe5c0157870 2023-10-10T20:20:19.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c51dd700 1 -- 172.21.15.148:0/3926371675 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5ac003570 con 0x7fe5c0157870 2023-10-10T20:20:19.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 -- 172.21.15.148:0/3926371675 >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 msgr2=0x7fe5c0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:19.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 --2- 172.21.15.148:0/3926371675 >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 0x7fe5c0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 -- 172.21.15.148:0/3926371675 >> v1:172.21.15.148:6789/0 conn(0x7fe5c0157870 legacy=0x7fe5c015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 -- 172.21.15.148:0/3926371675 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c015c280 con 0x7fe5c015a620 2023-10-10T20:20:19.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c69e0700 1 --2- 172.21.15.148:0/3926371675 >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 0x7fe5c0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:19.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c59de700 1 --2- 172.21.15.148:0/3926371675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c015d040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc08ba57520e038 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:19.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c51dd700 1 -- 172.21.15.148:0/3926371675 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011630 con 0x7fe5c015a620 2023-10-10T20:20:19.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c51dd700 1 -- 172.21.15.148:0/3926371675 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0011790 con 0x7fe5c015a620 2023-10-10T20:20:19.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.182+0000 7fe5c51dd700 1 -- 172.21.15.148:0/3926371675 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011b10 con 0x7fe5c015a620 2023-10-10T20:20:19.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 -- 172.21.15.148:0/3926371675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 msgr2=0x7fe5c015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 --2- 172.21.15.148:0/3926371675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c015d040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00098e0 tx=0x7fe5b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 -- 172.21.15.148:0/3926371675 shutdown_connections 2023-10-10T20:20:19.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 --2- 172.21.15.148:0/3926371675 >> v2:172.21.15.161:3300/0 conn(0x7fe5c015d580 0x7fe5c0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 --2- 172.21.15.148:0/3926371675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c015d040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 -- 172.21.15.148:0/3926371675 >> 172.21.15.148:0/3926371675 conn(0x7fe5c0068ce0 msgr2=0x7fe5c00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:19.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 -- 172.21.15.148:0/3926371675 shutdown_connections 2023-10-10T20:20:19.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.183+0000 7fe5c8443700 1 -- 172.21.15.148:0/3926371675 wait complete. 2023-10-10T20:20:19.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.184+0000 7fe5c8443700 1 Processor -- start 2023-10-10T20:20:19.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.184+0000 7fe5c8443700 1 -- start start 2023-10-10T20:20:19.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5c0157870 0x7fe5c00d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c015ef90 con 0x7fe5c015d580 2023-10-10T20:20:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00b12f0 con 0x7fe5c0157870 2023-10-10T20:20:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c8443700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00b15d0 con 0x7fe5c015a620 2023-10-10T20:20:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47518/0 (socket says 172.21.15.148:47518) 2023-10-10T20:20:19.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 -- 172.21.15.148:0/33219761 learned_addr learned my addr 172.21.15.148:0/33219761 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:19.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c59de700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c61df700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5c0157870 0x7fe5c00d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 -- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 msgr2=0x7fe5c00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 -- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5c0157870 msgr2=0x7fe5c00d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5c0157870 0x7fe5c00d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 -- 172.21.15.148:0/33219761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b8009420 con 0x7fe5c015d580 2023-10-10T20:20:19.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c59de700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:19.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5c69e0700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800ec50 tx=0x7fe5b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:19.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.185+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800c850 con 0x7fe5c015d580 2023-10-10T20:20:19.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.186+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b800c9b0 con 0x7fe5c015d580 2023-10-10T20:20:19.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.186+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b0009420 con 0x7fe5c015d580 2023-10-10T20:20:19.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.186+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800cd70 con 0x7fe5c015d580 2023-10-10T20:20:19.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.186+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c00b1b90 con 0x7fe5c015d580 2023-10-10T20:20:19.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.186+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c004c3c0 con 0x7fe5c015d580 2023-10-10T20:20:19.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.187+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5b8013680 con 0x7fe5c015d580 2023-10-10T20:20:19.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.187+0000 7fe5b67fc700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 0x7fe5a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:19.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.187+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5b8084810 con 0x7fe5c015d580 2023-10-10T20:20:19.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.187+0000 7fe5c61df700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 0x7fe5a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:19.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.188+0000 7fe5c61df700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 0x7fe5a40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005c50 tx=0x7fe5bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:19.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.192+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5b8051a80 con 0x7fe5c015d580 2023-10-10T20:20:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.452+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fe5c015c560 con 0x7fe5c015d580 2023-10-10T20:20:19.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.452+0000 7fe5b67fc700 1 -- 172.21.15.148:0/33219761 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7fe5b8021670 con 0x7fe5c015d580 2023-10-10T20:20:19.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:19.455 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:19.455 INFO:teuthology.orchestra.run.smithi148.stdout: "stats": { 2023-10-10T20:20:19.456 INFO:teuthology.orchestra.run.smithi148.stdout: "total_bytes": 1600000000, 2023-10-10T20:20:19.456 INFO:teuthology.orchestra.run.smithi148.stdout: "total_avail_bytes": 1597748242, 2023-10-10T20:20:19.456 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_bytes": 2251758, 2023-10-10T20:20:19.456 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_raw_bytes": 2251758, 2023-10-10T20:20:19.456 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_raw_ratio": 0.0014073487836867571, 2023-10-10T20:20:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: "num_osds": 8, 2023-10-10T20:20:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: "num_per_pool_osds": 0, 2023-10-10T20:20:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: "num_per_pool_omap_osds": 0 2023-10-10T20:20:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:19.457 INFO:teuthology.orchestra.run.smithi148.stdout: "stats_by_class": { 2023-10-10T20:20:19.458 INFO:teuthology.orchestra.run.smithi148.stdout: "hdd": { 2023-10-10T20:20:19.458 INFO:teuthology.orchestra.run.smithi148.stdout: "total_bytes": 1600000000, 2023-10-10T20:20:19.458 INFO:teuthology.orchestra.run.smithi148.stdout: "total_avail_bytes": 1597748242, 2023-10-10T20:20:19.458 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_bytes": 2251758, 2023-10-10T20:20:19.458 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_raw_bytes": 2251758, 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: "total_used_raw_ratio": 0.0014073487836867571 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:19.459 INFO:teuthology.orchestra.run.smithi148.stdout: "name": ".mgr", 2023-10-10T20:20:19.460 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 1, 2023-10-10T20:20:19.460 INFO:teuthology.orchestra.run.smithi148.stdout: "stats": { 2023-10-10T20:20:19.460 INFO:teuthology.orchestra.run.smithi148.stdout: "stored": 590368, 2023-10-10T20:20:19.460 INFO:teuthology.orchestra.run.smithi148.stdout: "objects": 2, 2023-10-10T20:20:19.460 INFO:teuthology.orchestra.run.smithi148.stdout: "kb_used": 577, 2023-10-10T20:20:19.461 INFO:teuthology.orchestra.run.smithi148.stdout: "bytes_used": 590368, 2023-10-10T20:20:19.461 INFO:teuthology.orchestra.run.smithi148.stdout: "percent_used": 0.00052957510342821479, 2023-10-10T20:20:19.461 INFO:teuthology.orchestra.run.smithi148.stdout: "max_avail": 557102656 2023-10-10T20:20:19.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:19.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:19.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:19.462 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "cephfs_metadata", 2023-10-10T20:20:19.462 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4, 2023-10-10T20:20:19.462 INFO:teuthology.orchestra.run.smithi148.stdout: "stats": { 2023-10-10T20:20:19.462 INFO:teuthology.orchestra.run.smithi148.stdout: "stored": 2306, 2023-10-10T20:20:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: "objects": 22, 2023-10-10T20:20:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: "kb_used": 3, 2023-10-10T20:20:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: "bytes_used": 2306, 2023-10-10T20:20:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: "percent_used": 2.0696322735602735e-06, 2023-10-10T20:20:19.463 INFO:teuthology.orchestra.run.smithi148.stdout: "max_avail": 557102656 2023-10-10T20:20:19.464 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:19.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:19.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:19.464 INFO:teuthology.orchestra.run.smithi148.stdout: "name": "cephfs_data", 2023-10-10T20:20:19.464 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5, 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "stats": { 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "stored": 0, 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "objects": 0, 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "kb_used": 0, 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "bytes_used": 0, 2023-10-10T20:20:19.465 INFO:teuthology.orchestra.run.smithi148.stdout: "percent_used": 0, 2023-10-10T20:20:19.466 INFO:teuthology.orchestra.run.smithi148.stdout: "max_avail": 557102656 2023-10-10T20:20:19.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:19.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:19.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:19.466 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 msgr2=0x7fe5a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 0x7fe5a40693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005c50 tx=0x7fe5bc009040 comp rx=0 tx=0).stop 2023-10-10T20:20:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 msgr2=0x7fe5c00b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800ec50 tx=0x7fe5b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 shutdown_connections 2023-10-10T20:20:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe5a4066f30 0x7fe5a40693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c015d580 0x7fe5c00b0c00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5c0157870 0x7fe5c00d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 --2- 172.21.15.148:0/33219761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe5c015a620 0x7fe5c00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 >> 172.21.15.148:0/33219761 conn(0x7fe5c0068ce0 msgr2=0x7fe5c0155570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 shutdown_connections 2023-10-10T20:20:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:19.455+0000 7fe5c8443700 1 -- 172.21.15.148:0/33219761 wait complete. 2023-10-10T20:20:19.471 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:20:19.472 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:20:19.472 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:20:19.472 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:20:19.473 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:20:19.473 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:20:19.473 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:20:19.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:19.727 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:20:19.796 INFO:teuthology.orchestra.run.smithi148.stdout:client4736 2023-10-10T20:20:19.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:19.797 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client4736/osdmap') 2023-10-10T20:20:19.874 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 31 barrier 31 flags 0x588000 2023-10-10T20:20:19.874 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:19.874 INFO:teuthology.orchestra.run.smithi148.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:19.874 INFO:teuthology.orchestra.run.smithi148.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:20:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:20:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:20:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:20:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:20:19.876 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:20:19.876 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:20:19.876 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:20:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:20:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:20:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:799 bytes copied, 3.6242e-05 s, 22.0 MB/s 2023-10-10T20:20:19.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-10T20:20:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.031+0000 7fb938884700 1 Processor -- start 2023-10-10T20:20:20.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.031+0000 7fb938884700 1 -- start start 2023-10-10T20:20:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb938884700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb93406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb938884700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb93406fe50 0x7fb93406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb938884700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9340cab30 con 0x7fb9340b0d70 2023-10-10T20:20:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb938884700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9340ce120 con 0x7fb93406fe50 2023-10-10T20:20:20.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb938884700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340ce400 con 0x7fb9340b1140 2023-10-10T20:20:20.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb93406fe50 0x7fb93406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb93406fe50 0x7fb93406f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47524/0 (socket says 172.21.15.148:47524) 2023-10-10T20:20:20.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 -- 172.21.15.148:0/3724458018 learned_addr learned my addr 172.21.15.148:0/3724458018 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:20.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93259c700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb93406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 -- 172.21.15.148:0/3724458018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 msgr2=0x7fb93406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:20.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb93406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 -- 172.21.15.148:0/3724458018 >> v1:172.21.15.161:6789/0 conn(0x7fb9340b0d70 legacy=0x7fb9340ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 -- 172.21.15.148:0/3724458018 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9340ce6e0 con 0x7fb93406fe50 2023-10-10T20:20:20.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb93359e700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93406fe50 0x7fb93406f700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9240099b0 tx=0x7fb924009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45bf069ae9dcb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb931d9b700 1 -- 172.21.15.148:0/3724458018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400b860 con 0x7fb93406fe50 2023-10-10T20:20:20.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb931d9b700 1 -- 172.21.15.148:0/3724458018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92400b9c0 con 0x7fb93406fe50 2023-10-10T20:20:20.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.032+0000 7fb931d9b700 1 -- 172.21.15.148:0/3724458018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92400bd60 con 0x7fb93406fe50 2023-10-10T20:20:20.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 -- 172.21.15.148:0/3724458018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93406fe50 msgr2=0x7fb93406f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93406fe50 0x7fb93406f700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9240099b0 tx=0x7fb924009250 comp rx=0 tx=0).stop 2023-10-10T20:20:20.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 -- 172.21.15.148:0/3724458018 shutdown_connections 2023-10-10T20:20:20.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb93406fe50 0x7fb93406f700 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 --2- 172.21.15.148:0/3724458018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb93406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 -- 172.21.15.148:0/3724458018 >> 172.21.15.148:0/3724458018 conn(0x7fb93406c700 msgr2=0x7fb934069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:20.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 -- 172.21.15.148:0/3724458018 shutdown_connections 2023-10-10T20:20:20.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.033+0000 7fb938884700 1 -- 172.21.15.148:0/3724458018 wait complete. 2023-10-10T20:20:20.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.034+0000 7fb938884700 1 Processor -- start 2023-10-10T20:20:20.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.034+0000 7fb938884700 1 -- start start 2023-10-10T20:20:20.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 0x7fb9340afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb9340c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340b0710 con 0x7fb9340b0d70 2023-10-10T20:20:20.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340c7bf0 con 0x7fb93406fe50 2023-10-10T20:20:20.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb938884700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340c7e70 con 0x7fb9340b1140 2023-10-10T20:20:20.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47538/0 (socket says 172.21.15.148:47538) 2023-10-10T20:20:20.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 -- 172.21.15.148:0/866568948 learned_addr learned my addr 172.21.15.148:0/866568948 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:20.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93359e700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb9340c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb932d9d700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 0x7fb9340afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 -- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 msgr2=0x7fb9340c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb9340c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 -- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 msgr2=0x7fb9340afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 0x7fb9340afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 -- 172.21.15.148:0/866568948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240094f0 con 0x7fb9340b0d70 2023-10-10T20:20:20.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb932d9d700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 0x7fb9340afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:20.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.035+0000 7fb93259c700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d860 tx=0x7fb91c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.036+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0125f0 con 0x7fb9340b0d70 2023-10-10T20:20:20.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.036+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c012750 con 0x7fb9340b0d70 2023-10-10T20:20:20.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.036+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c012b10 con 0x7fb9340b0d70 2023-10-10T20:20:20.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.036+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9340c8150 con 0x7fb9340b0d70 2023-10-10T20:20:20.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.036+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9340c8710 con 0x7fb9340b0d70 2023-10-10T20:20:20.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.037+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93406de40 con 0x7fb9340b0d70 2023-10-10T20:20:20.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.037+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb91c01e460 con 0x7fb9340b0d70 2023-10-10T20:20:20.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.037+0000 7fb92affd700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.037+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb91c084ad0 con 0x7fb9340b0d70 2023-10-10T20:20:20.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.037+0000 7fb932d9d700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.038+0000 7fb932d9d700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 0x7fb9140693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb92c007fd0 tx=0x7fb92c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.042+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb91c051d40 con 0x7fb9340b0d70 2023-10-10T20:20:20.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.235+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7fb93415a330 con 0x7fb9340b0d70 2023-10-10T20:20:20.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.754+0000 7fb92affd700 1 -- 172.21.15.148:0/866568948 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v32) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb91c055360 con 0x7fb9340b0d70 2023-10-10T20:20:20.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 msgr2=0x7fb9140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 0x7fb9140693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb92c007fd0 tx=0x7fb92c009040 comp rx=0 tx=0).stop 2023-10-10T20:20:20.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 msgr2=0x7fb93414ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d860 tx=0x7fb91c008040 comp rx=0 tx=0).stop 2023-10-10T20:20:20.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 shutdown_connections 2023-10-10T20:20:20.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb914066f30 0x7fb9140693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9340b1140 0x7fb9340c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb93406fe50 0x7fb9340afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 --2- 172.21.15.148:0/866568948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9340b0d70 0x7fb93414ce20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 >> 172.21.15.148:0/866568948 conn(0x7fb93406c700 msgr2=0x7fb9340cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:20.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 shutdown_connections 2023-10-10T20:20:20.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.757+0000 7fb938884700 1 -- 172.21.15.148:0/866568948 wait complete. 2023-10-10T20:20:20.761 INFO:teuthology.orchestra.run.smithi148.stderr:pauserd,pausewr is set 2023-10-10T20:20:20.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-10T20:20:20.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.888+0000 7f7659a3a700 1 Processor -- start 2023-10-10T20:20:20.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.888+0000 7f7659a3a700 1 -- start start 2023-10-10T20:20:20.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7659a3a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7654159e10 0x7f76540b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7659a3a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 0x7f76540b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7659a3a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765415db80 con 0x7f76540b1610 2023-10-10T20:20:20.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7659a3a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765415e800 con 0x7f7654159e10 2023-10-10T20:20:20.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7659a3a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765415de60 con 0x7f765415a1e0 2023-10-10T20:20:20.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7654159e10 0x7f76540b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7654159e10 0x7f76540b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47546/0 (socket says 172.21.15.148:47546) 2023-10-10T20:20:20.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 -- 172.21.15.148:0/1655956193 learned_addr learned my addr 172.21.15.148:0/1655956193 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:20.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f76527fc700 1 --2- 172.21.15.148:0/1655956193 >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 0x7f76540b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 -- 172.21.15.148:0/1655956193 >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 msgr2=0x7f76540b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:20.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 --2- 172.21.15.148:0/1655956193 >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 0x7f76540b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 -- 172.21.15.148:0/1655956193 >> v1:172.21.15.161:6789/0 conn(0x7f76540b1610 legacy=0x7f76540cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 -- 172.21.15.148:0/1655956193 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765415e140 con 0x7f7654159e10 2023-10-10T20:20:20.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f76527fc700 1 --2- 172.21.15.148:0/1655956193 >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 0x7f76540b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:20.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7652ffd700 1 --2- 172.21.15.148:0/1655956193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540b2d20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7654068580 tx=0x7f764c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b59f4b3903a4cbf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7651ffb700 1 -- 172.21.15.148:0/1655956193 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00e070 con 0x7f7654159e10 2023-10-10T20:20:20.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7651ffb700 1 -- 172.21.15.148:0/1655956193 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00ba50 con 0x7f7654159e10 2023-10-10T20:20:20.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.889+0000 7f7651ffb700 1 -- 172.21.15.148:0/1655956193 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0092e0 con 0x7f7654159e10 2023-10-10T20:20:20.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 -- 172.21.15.148:0/1655956193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 msgr2=0x7f76540b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1655956193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540b2d20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7654068580 tx=0x7f764c009840 comp rx=0 tx=0).stop 2023-10-10T20:20:20.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 -- 172.21.15.148:0/1655956193 shutdown_connections 2023-10-10T20:20:20.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1655956193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540b2d20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1655956193 >> v2:172.21.15.161:3300/0 conn(0x7f765415a1e0 0x7f76540b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 -- 172.21.15.148:0/1655956193 >> 172.21.15.148:0/1655956193 conn(0x7f7654067660 msgr2=0x7f7654067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:20.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 -- 172.21.15.148:0/1655956193 shutdown_connections 2023-10-10T20:20:20.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.890+0000 7f7659a3a700 1 -- 172.21.15.148:0/1655956193 wait complete. 2023-10-10T20:20:20.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.891+0000 7f7659a3a700 1 Processor -- start 2023-10-10T20:20:20.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.891+0000 7f7659a3a700 1 -- start start 2023-10-10T20:20:20.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76540b1610 0x7f765416b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f765415a1e0 0x7f76540cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765416bc30 con 0x7f7654159e10 2023-10-10T20:20:20.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540cf130 con 0x7f765415a1e0 2023-10-10T20:20:20.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7659a3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540ce870 con 0x7f76540b1610 2023-10-10T20:20:20.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47556/0 (socket says 172.21.15.148:47556) 2023-10-10T20:20:20.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 -- 172.21.15.148:0/1194745505 learned_addr learned my addr 172.21.15.148:0/1194745505 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:20.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f7652ffd700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76540b1610 0x7f765416b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76537fe700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f765415a1e0 0x7f76540cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 -- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76540b1610 msgr2=0x7f765416b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76540b1610 0x7f765416b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 -- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f765415a1e0 msgr2=0x7f76540cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:20.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f765415a1e0 0x7f76540cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:20.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.892+0000 7f76527fc700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009ae0 con 0x7f7654159e10 2023-10-10T20:20:20.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f76527fc700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f764800d8c0 tx=0x7f7648008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480125f0 con 0x7f7654159e10 2023-10-10T20:20:20.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648012750 con 0x7f7654159e10 2023-10-10T20:20:20.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76540ca5c0 con 0x7f7654159e10 2023-10-10T20:20:20.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648012b10 con 0x7f7654159e10 2023-10-10T20:20:20.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.893+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76540cedc0 con 0x7f7654159e10 2023-10-10T20:20:20.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.894+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76540cf2c0 con 0x7f7654159e10 2023-10-10T20:20:20.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.894+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7648014430 con 0x7f7654159e10 2023-10-10T20:20:20.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.894+0000 7f763affd700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 0x7f7640069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:20.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.894+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f76480847c0 con 0x7f7654159e10 2023-10-10T20:20:20.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.895+0000 7f7652ffd700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 0x7f7640069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:20.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.895+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=33+}) v3 -- 0x7f764006ea80 con 0x7f7654159e10 2023-10-10T20:20:20.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.895+0000 7f7652ffd700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 0x7f7640069410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f764c009880 tx=0x7f764c018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:20.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:20.899+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76480554c0 con 0x7f7654159e10 2023-10-10T20:20:21.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.096+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f765404c3c0 con 0x7f7654159e10 2023-10-10T20:20:21.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.761+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(33..33 src has 1..33) v4 ==== 280+0+0 (secure 0 0 0) 0x7f7648055df0 con 0x7f7654159e10 2023-10-10T20:20:21.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.761+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=34}) v3 -- 0x7f764006f3d0 con 0x7f7654159e10 2023-10-10T20:20:21.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.763+0000 7f763affd700 1 -- 172.21.15.148:0/1194745505 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7648055050 con 0x7f7654159e10 2023-10-10T20:20:21.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 msgr2=0x7f7640069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 0x7f7640069410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f764c009880 tx=0x7f764c018000 comp rx=0 tx=0).stop 2023-10-10T20:20:21.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 msgr2=0x7f76540c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f764800d8c0 tx=0x7f7648008040 comp rx=0 tx=0).stop 2023-10-10T20:20:21.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 shutdown_connections 2023-10-10T20:20:21.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7640066f60 0x7f7640069410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f765415a1e0 0x7f76540cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76540b1610 0x7f765416b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 --2- 172.21.15.148:0/1194745505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7654159e10 0x7f76540c9ae0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 >> 172.21.15.148:0/1194745505 conn(0x7f7654067660 msgr2=0x7f7654159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:21.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 shutdown_connections 2023-10-10T20:20:21.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.766+0000 7f7659a3a700 1 -- 172.21.15.148:0/1194745505 wait complete. 2023-10-10T20:20:21.771 INFO:teuthology.orchestra.run.smithi148.stderr:pauserd,pausewr is unset 2023-10-10T20:20:21.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:21.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.894+0000 7f3ed2a93700 1 Processor -- start 2023-10-10T20:20:21.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.894+0000 7f3ed2a93700 1 -- start start 2023-10-10T20:20:21.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ed2a93700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ecc0cbc00 0x7f3ecc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:21.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ed2a93700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc15de60 con 0x7f3ecc0b1220 2023-10-10T20:20:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ed2a93700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc15e140 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ed2a93700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc15e420 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ed1030700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ecc0cbc00 0x7f3ecc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ec3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3ecc0b1220 0x7f3ecc0cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60102/0 (socket says 172.21.15.148:60102) 2023-10-10T20:20:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.895+0000 7f3ec3fff700 1 -- 172.21.15.148:0/2950666704 learned_addr learned my addr 172.21.15.148:0/2950666704 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed1030700 1 -- 172.21.15.148:0/2950666704 >> v1:172.21.15.161:6789/0 conn(0x7f3ecc0b0e50 legacy=0x7f3ecc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed1030700 1 -- 172.21.15.148:0/2950666704 >> v1:172.21.15.148:6789/0 conn(0x7f3ecc0b1220 legacy=0x7f3ecc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed1030700 1 -- 172.21.15.148:0/2950666704 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc15e700 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ec37fe700 1 -- 172.21.15.148:0/2950666704 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281042296 0 0) 0x7f3ecc15de60 con 0x7f3ecc0b1220 2023-10-10T20:20:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed1030700 1 --2- 172.21.15.148:0/2950666704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c9180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098e0 tx=0x7f3ec40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d2e98f947d4c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ec37fe700 1 -- 172.21.15.148:0/2950666704 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009e80 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ec37fe700 1 -- 172.21.15.148:0/2950666704 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec400b860 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ec37fe700 1 -- 172.21.15.148:0/2950666704 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400bbe0 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2950666704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0cbc00 msgr2=0x7f3ecc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.896+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2950666704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c9180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098e0 tx=0x7f3ec40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.897+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2950666704 shutdown_connections 2023-10-10T20:20:21.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.897+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2950666704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c9180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.897+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2950666704 >> 172.21.15.148:0/2950666704 conn(0x7f3ecc069870 msgr2=0x7f3ecc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:21.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.897+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2950666704 shutdown_connections 2023-10-10T20:20:21.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.897+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2950666704 wait complete. 2023-10-10T20:20:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 Processor -- start 2023-10-10T20:20:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 -- start start 2023-10-10T20:20:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 0x7f3ecc0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.898+0000 7f3ed2a93700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc169f30 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed2a93700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b3c10 con 0x7f3ecc0b1220 2023-10-10T20:20:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed2a93700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b3ef0 con 0x7f3ecc0cbc00 2023-10-10T20:20:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47576/0 (socket says 172.21.15.148:47576) 2023-10-10T20:20:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 -- 172.21.15.148:0/2896131600 learned_addr learned my addr 172.21.15.148:0/2896131600 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ec3fff700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 0x7f3ecc0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed1030700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 -- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ecc0cbc00 msgr2=0x7f3ecc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 -- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 msgr2=0x7f3ecc0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 0x7f3ecc0b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 -- 172.21.15.148:0/2896131600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec4009420 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ec3fff700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 0x7f3ecc0b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:21.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed082f700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800cb10 tx=0x7f3ec80017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:21.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800a760 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800a8e0 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc15b990 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800aca0 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.899+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc15bfd0 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.900+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc15c160 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.901+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ec8003030 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.901+0000 7f3ec0ff9700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 0x7f3eb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.901+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ec8084630 con 0x7f3ecc0b0e50 2023-10-10T20:20:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.901+0000 7f3ec3fff700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 0x7f3eb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:21.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.902+0000 7f3ec3fff700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 0x7f3eb80693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc16d450 tx=0x7f3ebc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:21.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:21.906+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ec80518a0 con 0x7f3ecc0b0e50 2023-10-10T20:20:22.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.090+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ecc04c3c0 con 0x7f3ecc0b0e50 2023-10-10T20:20:22.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.091+0000 7f3ec0ff9700 1 -- 172.21.15.148:0/2896131600 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f3ec8054ec0 con 0x7f3ecc0b0e50 2023-10-10T20:20:22.093 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:22.093 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:21.752332+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:22.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 msgr2=0x7f3eb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 0x7f3eb80693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc16d450 tx=0x7f3ebc009040 comp rx=0 tx=0).stop 2023-10-10T20:20:22.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 msgr2=0x7f3ecc159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800cb10 tx=0x7f3ec80017b0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 shutdown_connections 2023-10-10T20:20:22.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3eb8066f30 0x7f3eb80693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ecc0cbc00 0x7f3ecc0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ecc0b0e50 0x7f3ecc159690 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 --2- 172.21.15.148:0/2896131600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ecc0b1220 0x7f3ecc0b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 >> 172.21.15.148:0/2896131600 conn(0x7f3ecc069870 msgr2=0x7f3ecc0c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:22.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 shutdown_connections 2023-10-10T20:20:22.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.093+0000 7f3ed2a93700 1 -- 172.21.15.148:0/2896131600 wait complete. 2023-10-10T20:20:22.107 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:20:22.108 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:20:22.108 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:20:22.108 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:20:22.109 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:20:22.109 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:20:22.109 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:20:22.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:22.362 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:20:22.377 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:20:22.405 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:20:22.430 INFO:teuthology.orchestra.run.smithi148.stdout:client4736 2023-10-10T20:20:22.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:22.431 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client4736/osdmap') 2023-10-10T20:20:22.506 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 31 barrier 31 flags 0x588000 2023-10-10T20:20:22.506 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:20:22.507 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:20:22.508 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:20:22.508 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:20:22.508 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:20:22.508 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:20:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:20:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:20:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:799 bytes copied, 3.6387e-05 s, 22.0 MB/s 2023-10-10T20:20:22.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:22.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.662+0000 7f31f7253700 1 Processor -- start 2023-10-10T20:20:22.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.662+0000 7f31f7253700 1 -- start start 2023-10-10T20:20:22.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f7253700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31f00cd6e0 0x7f31f00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f7253700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f7253700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00cb7c0 con 0x7f31f00cdab0 2023-10-10T20:20:22.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f7253700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f015e850 con 0x7f31f00cd6e0 2023-10-10T20:20:22.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f7253700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cd030 con 0x7f31f015cbd0 2023-10-10T20:20:22.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6251700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31f00cd6e0 0x7f31f00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f5a50700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f31f00cdab0 0x7f31f015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52654/0 (socket says 172.21.15.148:52654) 2023-10-10T20:20:22.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f5a50700 1 -- 172.21.15.148:0/3513580435 learned_addr learned my addr 172.21.15.148:0/3513580435 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:22.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6a52700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6251700 1 -- 172.21.15.148:0/3513580435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 msgr2=0x7f31f015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:22.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6251700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6251700 1 -- 172.21.15.148:0/3513580435 >> v1:172.21.15.148:6789/0 conn(0x7f31f00cdab0 legacy=0x7f31f015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6251700 1 -- 172.21.15.148:0/3513580435 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f015e290 con 0x7f31f00cd6e0 2023-10-10T20:20:22.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f524f700 1 -- 172.21.15.148:0/3513580435 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895870007 0 0) 0x7f31f00cb7c0 con 0x7f31f00cdab0 2023-10-10T20:20:22.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.663+0000 7f31f6a52700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:22.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f6251700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00cbc50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0098e0 tx=0x7f31ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2067adb66e6dfd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:22.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f524f700 1 -- 172.21.15.148:0/3513580435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec009e80 con 0x7f31f00cd6e0 2023-10-10T20:20:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f524f700 1 -- 172.21.15.148:0/3513580435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00b860 con 0x7f31f00cd6e0 2023-10-10T20:20:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f524f700 1 -- 172.21.15.148:0/3513580435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00bbe0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 -- 172.21.15.148:0/3513580435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 msgr2=0x7f31f00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00cbc50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0098e0 tx=0x7f31ec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 -- 172.21.15.148:0/3513580435 shutdown_connections 2023-10-10T20:20:22.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 --2- 172.21.15.148:0/3513580435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00cbc50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 -- 172.21.15.148:0/3513580435 >> 172.21.15.148:0/3513580435 conn(0x7f31f0067660 msgr2=0x7f31f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:22.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 -- 172.21.15.148:0/3513580435 shutdown_connections 2023-10-10T20:20:22.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.664+0000 7f31f7253700 1 -- 172.21.15.148:0/3513580435 wait complete. 2023-10-10T20:20:22.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.665+0000 7f31f7253700 1 Processor -- start 2023-10-10T20:20:22.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.665+0000 7f31f7253700 1 -- start start 2023-10-10T20:20:22.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdab0 0x7f31f015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00c9df0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00c91e0 con 0x7f31f00cdab0 2023-10-10T20:20:22.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f7253700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00c94c0 con 0x7f31f015cbd0 2023-10-10T20:20:22.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47610/0 (socket says 172.21.15.148:47610) 2023-10-10T20:20:22.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 -- 172.21.15.148:0/3487338895 learned_addr learned my addr 172.21.15.148:0/3487338895 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:22.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6a52700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f5a50700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdab0 0x7f31f015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 -- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 msgr2=0x7f31f015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 -- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdab0 msgr2=0x7f31f015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.666+0000 7f31f6251700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdab0 0x7f31f015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31f6251700 1 -- 172.21.15.148:0/3487338895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec009420 con 0x7f31f00cd6e0 2023-10-10T20:20:22.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31f6a52700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:22.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31f6251700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31f00c7b10 tx=0x7f31ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:22.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00bbe0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec011420 con 0x7f31f00cd6e0 2023-10-10T20:20:22.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00c97a0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0117c0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.667+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00c8e00 con 0x7f31f00cd6e0 2023-10-10T20:20:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.668+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0089280 con 0x7f31f00cd6e0 2023-10-10T20:20:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.668+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31ec011bf0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.669+0000 7f31e67fc700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.669+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31ec085f30 con 0x7f31f00cd6e0 2023-10-10T20:20:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.669+0000 7f31f5a50700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.669+0000 7f31f5a50700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f31f00cf690 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.673+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31ec0531a0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.900+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f00ca0d0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.901+0000 7f31e67fc700 1 -- 172.21.15.148:0/3487338895 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f31ec0567c0 con 0x7f31f00cd6e0 2023-10-10T20:20:22.903 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:22.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 msgr2=0x7f31d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 0x7f31d40693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f31f00cf690 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2023-10-10T20:20:22.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 msgr2=0x7f31f00ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f31f00c7b10 tx=0x7f31ec017000 comp rx=0 tx=0).stop 2023-10-10T20:20:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 shutdown_connections 2023-10-10T20:20:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31d4066f30 0x7f31d40693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31f015cbd0 0x7f31f015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31f00cd6e0 0x7f31f00ca280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 --2- 172.21.15.148:0/3487338895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdab0 0x7f31f015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 >> 172.21.15.148:0/3487338895 conn(0x7f31f0067660 msgr2=0x7f31f00c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 shutdown_connections 2023-10-10T20:20:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:22.903+0000 7f31f7253700 1 -- 172.21.15.148:0/3487338895 wait complete. 2023-10-10T20:20:22.908 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:22.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json osdmap barrier 33 2023-10-10T20:20:23.062 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-10T20:20:23.063 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:20:23.063 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:20:23.063 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:20:23.064 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:20:23.064 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:20:23.064 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:20:23.065 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:20:23.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:23.318 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:20:23.387 INFO:teuthology.orchestra.run.smithi148.stdout:client4736 2023-10-10T20:20:23.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:23.388 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client4736/osdmap') 2023-10-10T20:20:23.463 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-10T20:20:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:20:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:20:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:20:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:20:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:20:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:20:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:20:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:20:23.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:20:23.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:20:23.467 INFO:teuthology.orchestra.run.smithi148.stderr:799 bytes copied, 3.4529e-05 s, 23.1 MB/s 2023-10-10T20:20:23.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:23.468 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:20:23.535 INFO:teuthology.orchestra.run.smithi148.stdout:client4736 2023-10-10T20:20:23.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:23.536 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client4736/osdmap') 2023-10-10T20:20:23.611 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-10T20:20:23.612 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.612 INFO:teuthology.orchestra.run.smithi148.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.612 INFO:teuthology.orchestra.run.smithi148.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:20:23.612 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:20:23.613 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:20:23.614 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:20:23.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:20:23.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:20:23.615 INFO:teuthology.orchestra.run.smithi148.stderr:799 bytes copied, 3.3975e-05 s, 23.5 MB/s 2023-10-10T20:20:23.615 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:20:23.616 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:20:23.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:20:23.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:20:23.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:20:23.734 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:20:23.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:23.735 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:23.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:20:23.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:23.793 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:20:23.811 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:20:23.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:23.812 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:23.848 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:20:23.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:23.849 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:20:23.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:24.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.016+0000 7f1d271ed700 1 Processor -- start 2023-10-10T20:20:24.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.016+0000 7f1d271ed700 1 -- start start 2023-10-10T20:20:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.017+0000 7f1d271ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 0x7f1d2015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.017+0000 7f1d271ed700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 0x7f1d200cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.017+0000 7f1d271ed700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2006c010 con 0x7f1d201523e0 2023-10-10T20:20:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.017+0000 7f1d271ed700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2006bd10 con 0x7f1d20158e50 2023-10-10T20:20:24.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.017+0000 7f1d271ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2015c060 con 0x7f1d20152010 2023-10-10T20:20:24.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d2578a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 0x7f1d200cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 0x7f1d2015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d1ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1d201523e0 0x7f1d2006c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43660/0 (socket says 172.21.15.148:43660) 2023-10-10T20:20:24.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d1ffff700 1 -- 172.21.15.148:0/1743269646 learned_addr learned my addr 172.21.15.148:0/1743269646 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 -- 172.21.15.148:0/1743269646 >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 msgr2=0x7f1d200cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 --2- 172.21.15.148:0/1743269646 >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 0x7f1d200cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 -- 172.21.15.148:0/1743269646 >> v1:172.21.15.161:6789/0 conn(0x7f1d201523e0 legacy=0x7f1d2006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 -- 172.21.15.148:0/1743269646 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2015c340 con 0x7f1d20152010 2023-10-10T20:20:24.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d2578a700 1 --2- 172.21.15.148:0/1743269646 >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 0x7f1d200cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d24f89700 1 --2- 172.21.15.148:0/1743269646 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 0x7f1d2015b810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d140098e0 tx=0x7f1d140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b11d305d22423b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d1f7fe700 1 -- 172.21.15.148:0/1743269646 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400b640 con 0x7f1d20152010 2023-10-10T20:20:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d1f7fe700 1 -- 172.21.15.148:0/1743269646 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1400b7a0 con 0x7f1d20152010 2023-10-10T20:20:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.018+0000 7f1d1f7fe700 1 -- 172.21.15.148:0/1743269646 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400bb40 con 0x7f1d20152010 2023-10-10T20:20:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 -- 172.21.15.148:0/1743269646 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 msgr2=0x7f1d2015b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 --2- 172.21.15.148:0/1743269646 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 0x7f1d2015b810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d140098e0 tx=0x7f1d140091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 -- 172.21.15.148:0/1743269646 shutdown_connections 2023-10-10T20:20:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 --2- 172.21.15.148:0/1743269646 >> v2:172.21.15.161:3300/0 conn(0x7f1d20158e50 0x7f1d200cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 --2- 172.21.15.148:0/1743269646 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20152010 0x7f1d2015b810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 -- 172.21.15.148:0/1743269646 >> 172.21.15.148:0/1743269646 conn(0x7f1d20069870 msgr2=0x7f1d20069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 -- 172.21.15.148:0/1743269646 shutdown_connections 2023-10-10T20:20:24.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.019+0000 7f1d271ed700 1 -- 172.21.15.148:0/1743269646 wait complete. 2023-10-10T20:20:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.020+0000 7f1d271ed700 1 Processor -- start 2023-10-10T20:20:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.020+0000 7f1d271ed700 1 -- start start 2023-10-10T20:20:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201523e0 0x7f1d200c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 0x7f1d200c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200b1c10 con 0x7f1d20152010 2023-10-10T20:20:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20168b30 con 0x7f1d201523e0 2023-10-10T20:20:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d271ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d201536e0 con 0x7f1d20158e50 2023-10-10T20:20:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47614/0 (socket says 172.21.15.148:47614) 2023-10-10T20:20:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 -- 172.21.15.148:0/58928826 learned_addr learned my addr 172.21.15.148:0/58928826 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d1ffff700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201523e0 0x7f1d200c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d2578a700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 0x7f1d200c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 -- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 msgr2=0x7f1d200c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 0x7f1d200c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 -- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201523e0 msgr2=0x7f1d200c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201523e0 0x7f1d200c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d24f89700 1 -- 172.21.15.148:0/58928826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009a20 con 0x7f1d20152010 2023-10-10T20:20:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.021+0000 7f1d2578a700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 0x7f1d200c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d24f89700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d2006edb0 tx=0x7f1d14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1401bda0 con 0x7f1d20152010 2023-10-10T20:20:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d14011420 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d14009420 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d140117e0 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.022+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d201687c0 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.023+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2004c3c0 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.023+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d14011b90 con 0x7f1d20152010 2023-10-10T20:20:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.023+0000 7f1d1cff9700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.024+0000 7f1d1ffff700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.024+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d14085ba0 con 0x7f1d20152010 2023-10-10T20:20:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.024+0000 7f1d1ffff700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 0x7f1d040693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d2016b780 tx=0x7f1d0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.028+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d14052e10 con 0x7f1d20152010 2023-10-10T20:20:24.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.251+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d200c8f00 con 0x7f1d20152010 2023-10-10T20:20:24.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.252+0000 7f1d1cff9700 1 -- 172.21.15.148:0/58928826 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f1d14056430 con 0x7f1d20152010 2023-10-10T20:20:24.255 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:24.255 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 msgr2=0x7f1d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 0x7f1d040693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d2016b780 tx=0x7f1d0c009040 comp rx=0 tx=0).stop 2023-10-10T20:20:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 msgr2=0x7f1d200b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d2006edb0 tx=0x7f1d14017000 comp rx=0 tx=0).stop 2023-10-10T20:20:24.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 shutdown_connections 2023-10-10T20:20:24.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d20158e50 0x7f1d200c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d20152010 0x7f1d200b2280 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 --2- 172.21.15.148:0/58928826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201523e0 0x7f1d200c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 >> 172.21.15.148:0/58928826 conn(0x7f1d20069870 msgr2=0x7f1d200caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 shutdown_connections 2023-10-10T20:20:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.254+0000 7f1d271ed700 1 -- 172.21.15.148:0/58928826 wait complete. 2023-10-10T20:20:24.262 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:24.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.381+0000 7ff88887e700 1 Processor -- start 2023-10-10T20:20:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.381+0000 7ff88887e700 1 -- start start 2023-10-10T20:20:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff88887e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff8840cd6e0 0x7ff8840cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff88887e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff88415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff88887e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840cb7c0 con 0x7ff8840cdab0 2023-10-10T20:20:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff88887e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88415e850 con 0x7ff8840cd6e0 2023-10-10T20:20:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff88887e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840cd030 con 0x7ff88415cbd0 2023-10-10T20:20:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff882ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff8840cdab0 0x7ff88415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52662/0 (socket says 172.21.15.148:52662) 2023-10-10T20:20:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff882ffd700 1 -- 172.21.15.148:0/1641799259 learned_addr learned my addr 172.21.15.148:0/1641799259 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1641799259 >> v2:172.21.15.161:3300/0 conn(0x7ff8840cd6e0 0x7ff8840cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff883fff700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff88415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153244288 0 0) 0x7ff8840cb7c0 con 0x7ff8840cdab0 2023-10-10T20:20:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c003570 con 0x7ff8840cdab0 2023-10-10T20:20:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849548349 0 0) 0x7ff86c003570 con 0x7ff8840cdab0 2023-10-10T20:20:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8837fe700 1 -- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 msgr2=0x7ff88415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff88415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8837fe700 1 -- 172.21.15.148:0/1641799259 >> v1:172.21.15.148:6789/0 conn(0x7ff8840cdab0 legacy=0x7ff88415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.382+0000 7ff8837fe700 1 -- 172.21.15.148:0/1641799259 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88415e290 con 0x7ff8840cd6e0 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cd6e0 0x7ff8840cbc50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098e0 tx=0x7ff8740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e75ff802fe53f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878001470 con 0x7ff8840cd6e0 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874011420 con 0x7ff8840cd6e0 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff8827fc700 1 -- 172.21.15.148:0/1641799259 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8740117c0 con 0x7ff8840cd6e0 2023-10-10T20:20:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 -- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cd6e0 msgr2=0x7ff8840cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cd6e0 0x7ff8840cbc50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098e0 tx=0x7ff8740091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 -- 172.21.15.148:0/1641799259 shutdown_connections 2023-10-10T20:20:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff88415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 --2- 172.21.15.148:0/1641799259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cd6e0 0x7ff8840cbc50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 -- 172.21.15.148:0/1641799259 >> 172.21.15.148:0/1641799259 conn(0x7ff884067660 msgr2=0x7ff884067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 -- 172.21.15.148:0/1641799259 shutdown_connections 2023-10-10T20:20:24.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.383+0000 7ff88887e700 1 -- 172.21.15.148:0/1641799259 wait complete. 2023-10-10T20:20:24.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 Processor -- start 2023-10-10T20:20:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 -- start start 2023-10-10T20:20:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cdab0 0x7ff884150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff8840b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884143f90 con 0x7ff8840cd6e0 2023-10-10T20:20:24.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88414f1f0 con 0x7ff8840cdab0 2023-10-10T20:20:24.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff88887e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88414f4d0 con 0x7ff88415cbd0 2023-10-10T20:20:24.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47620/0 (socket says 172.21.15.148:47620) 2023-10-10T20:20:24.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.385+0000 7ff8837fe700 1 -- 172.21.15.148:0/1213438142 learned_addr learned my addr 172.21.15.148:0/1213438142 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:24.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff882ffd700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cdab0 0x7ff884150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff883fff700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff8840b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 -- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 msgr2=0x7ff8840b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff8840b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 -- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cdab0 msgr2=0x7ff884150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cdab0 0x7ff884150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 -- 172.21.15.148:0/1213438142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874009420 con 0x7ff8840cd6e0 2023-10-10T20:20:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff8837fe700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8740093f0 tx=0x7ff874016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87401cb30 con 0x7ff8840cd6e0 2023-10-10T20:20:24.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87401cc90 con 0x7ff8840cd6e0 2023-10-10T20:20:24.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88414f7b0 con 0x7ff8840cd6e0 2023-10-10T20:20:24.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87401a660 con 0x7ff8840cd6e0 2023-10-10T20:20:24.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.386+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884150d80 con 0x7ff8840cd6e0 2023-10-10T20:20:24.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.387+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884089280 con 0x7ff8840cd6e0 2023-10-10T20:20:24.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.387+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff87401aa50 con 0x7ff8840cd6e0 2023-10-10T20:20:24.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.388+0000 7ff86b7fe700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 0x7ff8700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.388+0000 7ff882ffd700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 0x7ff8700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.388+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff874085d20 con 0x7ff8840cd6e0 2023-10-10T20:20:24.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.388+0000 7ff882ffd700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 0x7ff8700693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b24d0 tx=0x7ff878000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.392+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff874053040 con 0x7ff8840cd6e0 2023-10-10T20:20:24.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.574+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff88415e570 con 0x7ff8840cd6e0 2023-10-10T20:20:24.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.575+0000 7ff86b7fe700 1 -- 172.21.15.148:0/1213438142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7ff874056660 con 0x7ff8840cd6e0 2023-10-10T20:20:24.577 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:24.577 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:21.752332+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:24.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 msgr2=0x7ff8700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 0x7ff8700693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b24d0 tx=0x7ff878000bc0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 msgr2=0x7ff88414ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8740093f0 tx=0x7ff874016000 comp rx=0 tx=0).stop 2023-10-10T20:20:24.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 shutdown_connections 2023-10-10T20:20:24.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff870066f30 0x7ff8700693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff88415cbd0 0x7ff8840b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8840cd6e0 0x7ff88414ffb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 --2- 172.21.15.148:0/1213438142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8840cdab0 0x7ff884150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 >> 172.21.15.148:0/1213438142 conn(0x7ff884067660 msgr2=0x7ff8840cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 shutdown_connections 2023-10-10T20:20:24.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.577+0000 7ff88887e700 1 -- 172.21.15.148:0/1213438142 wait complete. 2023-10-10T20:20:24.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:20:07.830881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:20:09.018572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:20:24.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:20:24.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.708+0000 7f0cff59e700 1 Processor -- start 2023-10-10T20:20:24.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.708+0000 7f0cff59e700 1 -- start start 2023-10-10T20:20:24.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d0006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cff59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0d0006a830 0x7f0d000b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cff59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000b4d70 con 0x7f0d000cce30 2023-10-10T20:20:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cff59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0006ac70 con 0x7f0d0006a830 2023-10-10T20:20:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cff59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0015b3b0 con 0x7f0d000b6580 2023-10-10T20:20:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfe59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0d000cce30 0x7f0d0015aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52670/0 (socket says 172.21.15.148:52670) 2023-10-10T20:20:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfe59c700 1 -- 172.21.15.148:0/2049479218 learned_addr learned my addr 172.21.15.148:0/2049479218 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/2049479218 >> v2:172.21.15.161:3300/0 conn(0x7f0d0006a830 0x7f0d000b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfdd9b700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d0006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfd59a700 1 -- 172.21.15.148:0/2049479218 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349629176 0 0) 0x7f0d000b4d70 con 0x7f0d000cce30 2023-10-10T20:20:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfd59a700 1 -- 172.21.15.148:0/2049479218 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce8003570 con 0x7f0d000cce30 2023-10-10T20:20:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 -- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 msgr2=0x7f0d0006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:24.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d0006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 -- 172.21.15.148:0/2049479218 >> v1:172.21.15.148:6789/0 conn(0x7f0d000cce30 legacy=0x7f0d0015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 -- 172.21.15.148:0/2049479218 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000cd8d0 con 0x7f0d0006a830 2023-10-10T20:20:24.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.709+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d000b5200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009e60 tx=0x7f0cf8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450e182f2c745804 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cfd59a700 1 -- 172.21.15.148:0/2049479218 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800e070 con 0x7f0d0006a830 2023-10-10T20:20:24.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cfd59a700 1 -- 172.21.15.148:0/2049479218 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800bab0 con 0x7f0d0006a830 2023-10-10T20:20:24.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cfd59a700 1 -- 172.21.15.148:0/2049479218 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800be30 con 0x7f0d0006a830 2023-10-10T20:20:24.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 -- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 msgr2=0x7f0d000b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d000b5200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009e60 tx=0x7f0cf8009780 comp rx=0 tx=0).stop 2023-10-10T20:20:24.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 -- 172.21.15.148:0/2049479218 shutdown_connections 2023-10-10T20:20:24.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d000b5200 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 --2- 172.21.15.148:0/2049479218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d0006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 -- 172.21.15.148:0/2049479218 >> 172.21.15.148:0/2049479218 conn(0x7f0d00068cb0 msgr2=0x7f0d000690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 -- 172.21.15.148:0/2049479218 shutdown_connections 2023-10-10T20:20:24.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.710+0000 7f0cff59e700 1 -- 172.21.15.148:0/2049479218 wait complete. 2023-10-10T20:20:24.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.711+0000 7f0cff59e700 1 Processor -- start 2023-10-10T20:20:24.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.711+0000 7f0cff59e700 1 -- start start 2023-10-10T20:20:24.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d00151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d00153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00151e20 con 0x7f0d000cce30 2023-10-10T20:20:24.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00152100 con 0x7f0d0006a830 2023-10-10T20:20:24.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cff59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001523e0 con 0x7f0d000b6580 2023-10-10T20:20:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47632/0 (socket says 172.21.15.148:47632) 2023-10-10T20:20:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 -- 172.21.15.148:0/1956163252 learned_addr learned my addr 172.21.15.148:0/1956163252 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfe59c700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d00151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfdd9b700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d00153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 -- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 msgr2=0x7f0d00153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d00153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 -- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 msgr2=0x7f0d00151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d00151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfed9d700 1 -- 172.21.15.148:0/1956163252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009a20 con 0x7f0d000cce30 2023-10-10T20:20:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.712+0000 7f0cfdd9b700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d00153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cfed9d700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d0006b300 tx=0x7f0cf8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8027470 con 0x7f0d000cce30 2023-10-10T20:20:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf80275f0 con 0x7f0d000cce30 2023-10-10T20:20:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80279b0 con 0x7f0d000cce30 2023-10-10T20:20:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d001526c0 con 0x7f0d000cce30 2023-10-10T20:20:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.713+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d00151a50 con 0x7f0d000cce30 2023-10-10T20:20:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.714+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00152be0 con 0x7f0d000cce30 2023-10-10T20:20:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.714+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cf8024070 con 0x7f0d000cce30 2023-10-10T20:20:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.714+0000 7f0cee7fc700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 0x7f0ce0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.715+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0cf8086fb0 con 0x7f0d000cce30 2023-10-10T20:20:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.715+0000 7f0cfe59c700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 0x7f0ce0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.715+0000 7f0cfe59c700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 0x7f0ce0069410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058d0 tx=0x7f0cf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.719+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cf80b85a0 con 0x7f0d000cce30 2023-10-10T20:20:24.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.903+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d000b4440 con 0x7f0d000cce30 2023-10-10T20:20:24.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.904+0000 7f0cee7fc700 1 -- 172.21.15.148:0/1956163252 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d00152be0 con 0x7f0d000cce30 2023-10-10T20:20:24.906 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:20:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 msgr2=0x7f0ce0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 0x7f0ce0069410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058d0 tx=0x7f0cf0009040 comp rx=0 tx=0).stop 2023-10-10T20:20:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 msgr2=0x7f0d00152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d0006b300 tx=0x7f0cf8019000 comp rx=0 tx=0).stop 2023-10-10T20:20:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 shutdown_connections 2023-10-10T20:20:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ce0066f60 0x7f0ce0069410 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d000cce30 0x7f0d00152ec0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d0006a830 0x7f0d00151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 --2- 172.21.15.148:0/1956163252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d000b6580 0x7f0d00153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 >> 172.21.15.148:0/1956163252 conn(0x7f0d00068cb0 msgr2=0x7f0d0015a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 shutdown_connections 2023-10-10T20:20:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:24.906+0000 7f0cff59e700 1 -- 172.21.15.148:0/1956163252 wait complete. 2023-10-10T20:20:24.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:20:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.035+0000 7fc3a1ae5700 1 Processor -- start 2023-10-10T20:20:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.035+0000 7fc3a1ae5700 1 -- start start 2023-10-10T20:20:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc3a1ae5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc39c0cbf10 0x7fc39c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc3a1ae5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc39c0c9390 0x7fc39c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc3a1ae5700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39c0cae70 con 0x7fc39c0678b0 2023-10-10T20:20:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc3a1ae5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39c0cb150 con 0x7fc39c0c9390 2023-10-10T20:20:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc3a1ae5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39c0cb430 con 0x7fc39c0cbf10 2023-10-10T20:20:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc39c0c9390 0x7fc39c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc39c0c9390 0x7fc39c15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47648/0 (socket says 172.21.15.148:47648) 2023-10-10T20:20:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 -- 172.21.15.148:0/3531802629 learned_addr learned my addr 172.21.15.148:0/3531802629 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc393fff700 1 --2- 172.21.15.148:0/3531802629 >> v2:172.21.15.161:3300/0 conn(0x7fc39c0cbf10 0x7fc39c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 -- 172.21.15.148:0/3531802629 >> v2:172.21.15.161:3300/0 conn(0x7fc39c0cbf10 msgr2=0x7fc39c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 --2- 172.21.15.148:0/3531802629 >> v2:172.21.15.161:3300/0 conn(0x7fc39c0cbf10 0x7fc39c15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 -- 172.21.15.148:0/3531802629 >> v1:172.21.15.161:6789/0 conn(0x7fc39c0678b0 legacy=0x7fc39c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 -- 172.21.15.148:0/3531802629 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c0cb710 con 0x7fc39c0c9390 2023-10-10T20:20:25.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.036+0000 7fc39bfff700 1 --2- 172.21.15.148:0/3531802629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c15a370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0ca0a0 tx=0x7fc394009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d232eac0d0f39e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc39affd700 1 -- 172.21.15.148:0/3531802629 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400e070 con 0x7fc39c0c9390 2023-10-10T20:20:25.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc39affd700 1 -- 172.21.15.148:0/3531802629 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39400ba10 con 0x7fc39c0c9390 2023-10-10T20:20:25.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc39affd700 1 -- 172.21.15.148:0/3531802629 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940092e0 con 0x7fc39c0c9390 2023-10-10T20:20:25.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/3531802629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 msgr2=0x7fc39c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/3531802629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c15a370 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0ca0a0 tx=0x7fc394009840 comp rx=0 tx=0).stop 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/3531802629 shutdown_connections 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/3531802629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c15a370 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/3531802629 >> v2:172.21.15.161:3300/0 conn(0x7fc39c0cbf10 0x7fc39c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/3531802629 >> 172.21.15.148:0/3531802629 conn(0x7fc39c069870 msgr2=0x7fc39c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/3531802629 shutdown_connections 2023-10-10T20:20:25.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.037+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/3531802629 wait complete. 2023-10-10T20:20:25.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.038+0000 7fc3a1ae5700 1 Processor -- start 2023-10-10T20:20:25.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.038+0000 7fc3a1ae5700 1 -- start start 2023-10-10T20:20:25.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc39c0678b0 0x7fc39c0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0cbf10 0x7fc39c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0b35b0 con 0x7fc39c0c9390 2023-10-10T20:20:25.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0c8090 con 0x7fc39c0cbf10 2023-10-10T20:20:25.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc3a1ae5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0b2510 con 0x7fc39c0678b0 2023-10-10T20:20:25.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc393fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc393fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47650/0 (socket says 172.21.15.148:47650) 2023-10-10T20:20:25.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.039+0000 7fc393fff700 1 -- 172.21.15.148:0/1883185973 learned_addr learned my addr 172.21.15.148:0/1883185973 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc39bfff700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0cbf10 0x7fc39c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc39b7fe700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc39c0678b0 0x7fc39c0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 -- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc39c0678b0 msgr2=0x7fc39c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc39c0678b0 0x7fc39c0b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 -- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0cbf10 msgr2=0x7fc39c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0cbf10 0x7fc39c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 -- 172.21.15.148:0/1883185973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394009ae0 con 0x7fc39c0c9390 2023-10-10T20:20:25.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc393fff700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c7d0 tx=0x7fc384008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384011600 con 0x7fc39c0c9390 2023-10-10T20:20:25.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384011760 con 0x7fc39c0c9390 2023-10-10T20:20:25.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c0b27f0 con 0x7fc39c0c9390 2023-10-10T20:20:25.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384011b20 con 0x7fc39c0c9390 2023-10-10T20:20:25.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.040+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c0b2db0 con 0x7fc39c0c9390 2023-10-10T20:20:25.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.041+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39c04c3c0 con 0x7fc39c0c9390 2023-10-10T20:20:25.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.041+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3840193f0 con 0x7fc39c0c9390 2023-10-10T20:20:25.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.042+0000 7fc3937fe700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 0x7fc37c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.042+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc384084910 con 0x7fc39c0c9390 2023-10-10T20:20:25.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.042+0000 7fc39b7fe700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 0x7fc37c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.042+0000 7fc39b7fe700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 0x7fc37c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00a8a0 tx=0x7fc38c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.046+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc384051b80 con 0x7fc39c0c9390 2023-10-10T20:20:25.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.226+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc39c1500a0 con 0x7fc39c0c9390 2023-10-10T20:20:25.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.226+0000 7fc3937fe700 1 -- 172.21.15.148:0/1883185973 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3840551a0 con 0x7fc39c0c9390 2023-10-10T20:20:25.229 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:20:25.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 msgr2=0x7fc37c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 0x7fc37c0693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00a8a0 tx=0x7fc38c008040 comp rx=0 tx=0).stop 2023-10-10T20:20:25.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 msgr2=0x7fc39c0b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c7d0 tx=0x7fc384008040 comp rx=0 tx=0).stop 2023-10-10T20:20:25.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 shutdown_connections 2023-10-10T20:20:25.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc37c066f30 0x7fc37c0693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc39c0cbf10 0x7fc39c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc39c0678b0 0x7fc39c0b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 --2- 172.21.15.148:0/1883185973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc39c0c9390 0x7fc39c0b1ee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.228+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 >> 172.21.15.148:0/1883185973 conn(0x7fc39c069870 msgr2=0x7fc39c1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.229+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 shutdown_connections 2023-10-10T20:20:25.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.229+0000 7fc3a1ae5700 1 -- 172.21.15.148:0/1883185973 wait complete. 2023-10-10T20:20:25.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:20:25.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.360+0000 7f3b7ad9a700 1 Processor -- start 2023-10-10T20:20:25.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.360+0000 7f3b7ad9a700 1 -- start start 2023-10-10T20:20:25.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b7415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b7415a230 0x7f3b740690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b740695f0 0x7f3b740af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b740d1cf0 con 0x7f3b740695f0 2023-10-10T20:20:25.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b74069a90 con 0x7f3b7415a230 2023-10-10T20:20:25.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b7ad9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7415ea90 con 0x7f3b74159e60 2023-10-10T20:20:25.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b79337700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b740695f0 0x7f3b740af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b79337700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b740695f0 0x7f3b740af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47666/0 (socket says 172.21.15.148:47666) 2023-10-10T20:20:25.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b79337700 1 -- 172.21.15.148:0/3655786600 learned_addr learned my addr 172.21.15.148:0/3655786600 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b6bfff700 1 --2- 172.21.15.148:0/3655786600 >> v2:172.21.15.161:3300/0 conn(0x7f3b7415a230 0x7f3b740690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.361+0000 7f3b78b36700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b7415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 -- 172.21.15.148:0/3655786600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 msgr2=0x7f3b7415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:25.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b7415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 -- 172.21.15.148:0/3655786600 >> v2:172.21.15.161:3300/0 conn(0x7f3b7415a230 msgr2=0x7f3b740690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:25.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 --2- 172.21.15.148:0/3655786600 >> v2:172.21.15.161:3300/0 conn(0x7f3b7415a230 0x7f3b740690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 -- 172.21.15.148:0/3655786600 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740b3100 con 0x7f3b740695f0 2023-10-10T20:20:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b78b36700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b7415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b79337700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b740695f0 0x7f3b740af730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b740d0ca0 tx=0x7f3b6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f6e6b3a76e4a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b6b7fe700 1 -- 172.21.15.148:0/3655786600 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c009e70 con 0x7f3b740695f0 2023-10-10T20:20:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b6b7fe700 1 -- 172.21.15.148:0/3655786600 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c00b860 con 0x7f3b740695f0 2023-10-10T20:20:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b6b7fe700 1 -- 172.21.15.148:0/3655786600 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00bc00 con 0x7f3b740695f0 2023-10-10T20:20:25.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/3655786600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b740695f0 msgr2=0x7f3b740af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b740695f0 0x7f3b740af730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b740d0ca0 tx=0x7f3b6c009250 comp rx=0 tx=0).stop 2023-10-10T20:20:25.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/3655786600 shutdown_connections 2023-10-10T20:20:25.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b740695f0 0x7f3b740af730 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/3655786600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b7415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/3655786600 >> v2:172.21.15.161:3300/0 conn(0x7f3b7415a230 0x7f3b740690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/3655786600 >> 172.21.15.148:0/3655786600 conn(0x7f3b74070700 msgr2=0x7f3b74070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/3655786600 shutdown_connections 2023-10-10T20:20:25.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.362+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/3655786600 wait complete. 2023-10-10T20:20:25.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.363+0000 7f3b7ad9a700 1 Processor -- start 2023-10-10T20:20:25.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.364+0000 7f3b7ad9a700 1 -- start start 2023-10-10T20:20:25.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.364+0000 7f3b7ad9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 0x7f3b7414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.364+0000 7f3b7ad9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b740cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.364+0000 7f3b7ad9a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.364+0000 7f3b7ad9a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7416a450 con 0x7f3b7415a230 2023-10-10T20:20:25.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b7ad9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7416a730 con 0x7f3b740695f0 2023-10-10T20:20:25.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b7ad9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7416aa10 con 0x7f3b74159e60 2023-10-10T20:20:25.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47668/0 (socket says 172.21.15.148:47668) 2023-10-10T20:20:25.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 -- 172.21.15.148:0/409762538 learned_addr learned my addr 172.21.15.148:0/409762538 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b78b36700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 0x7f3b7414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b6bfff700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b740cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 -- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 msgr2=0x7f3b740cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b740cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 -- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 msgr2=0x7f3b7414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 0x7f3b7414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 -- 172.21.15.148:0/409762538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c0094f0 con 0x7f3b7415a230 2023-10-10T20:20:25.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b78b36700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 0x7f3b7414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:25.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b79337700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0090b0 tx=0x7f3b6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c026480 con 0x7f3b7415a230 2023-10-10T20:20:25.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c01f070 con 0x7f3b7415a230 2023-10-10T20:20:25.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c026830 con 0x7f3b7415a230 2023-10-10T20:20:25.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7416acf0 con 0x7f3b7415a230 2023-10-10T20:20:25.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.365+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7416a090 con 0x7f3b7415a230 2023-10-10T20:20:25.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.366+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b740c5ec0 con 0x7f3b7415a230 2023-10-10T20:20:25.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.367+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b6c026ba0 con 0x7f3b7415a230 2023-10-10T20:20:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.367+0000 7f3b68ff9700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 0x7f3b600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.367+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b6c021070 con 0x7f3b7415a230 2023-10-10T20:20:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.367+0000 7f3b78b36700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 0x7f3b600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.367+0000 7f3b78b36700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 0x7f3b600693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b06e0 tx=0x7f3b70009910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.371+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b740c5ec0 con 0x7f3b7415a230 2023-10-10T20:20:25.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.555+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b740c5ec0 con 0x7f3b7415a230 2023-10-10T20:20:25.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.556+0000 7f3b68ff9700 1 -- 172.21.15.148:0/409762538 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b740c5ec0 con 0x7f3b7415a230 2023-10-10T20:20:25.558 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:20:25.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 msgr2=0x7f3b600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 0x7f3b600693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b06e0 tx=0x7f3b70009910 comp rx=0 tx=0).stop 2023-10-10T20:20:25.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 msgr2=0x7f3b740af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0090b0 tx=0x7f3b6c017000 comp rx=0 tx=0).stop 2023-10-10T20:20:25.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 shutdown_connections 2023-10-10T20:20:25.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3b60066f30 0x7f3b600693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7415a230 0x7f3b740af9f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b740695f0 0x7f3b7414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 --2- 172.21.15.148:0/409762538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b74159e60 0x7f3b740cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 >> 172.21.15.148:0/409762538 conn(0x7f3b74070700 msgr2=0x7f3b740cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 shutdown_connections 2023-10-10T20:20:25.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.558+0000 7f3b7ad9a700 1 -- 172.21.15.148:0/409762538 wait complete. 2023-10-10T20:20:25.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:25.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.685+0000 7f0703598700 1 Processor -- start 2023-10-10T20:20:25.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.685+0000 7f0703598700 1 -- start start 2023-10-10T20:20:25.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0703598700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06fc06ab90 0x7f06fc0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0703598700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 0x7f06fc0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0703598700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0cac20 con 0x7f06fc0cc710 2023-10-10T20:20:25.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0703598700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0640c0 con 0x7f06fc06ab90 2023-10-10T20:20:25.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0703598700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0cb4c0 con 0x7f06fc15abb0 2023-10-10T20:20:25.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06fc06ab90 0x7f06fc0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06fc06ab90 0x7f06fc0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47676/0 (socket says 172.21.15.148:47676) 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 -- 172.21.15.148:0/1748640493 learned_addr learned my addr 172.21.15.148:0/1748640493 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702d97700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 0x7f06fc0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 -- 172.21.15.148:0/1748640493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 msgr2=0x7f06fc0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 0x7f06fc0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 -- 172.21.15.148:0/1748640493 >> v1:172.21.15.161:6789/0 conn(0x7f06fc0cc710 legacy=0x7f06fc06af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 -- 172.21.15.148:0/1748640493 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0ccc10 con 0x7f06fc06ab90 2023-10-10T20:20:25.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702d97700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 0x7f06fc0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:25.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0702596700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc06ab90 0x7f06fc0cb950 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099b0 tx=0x7f06f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a641e4ab47125228 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0701594700 1 -- 172.21.15.148:0/1748640493 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800b860 con 0x7f06fc06ab90 2023-10-10T20:20:25.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.686+0000 7f0701594700 1 -- 172.21.15.148:0/1748640493 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800b9c0 con 0x7f06fc06ab90 2023-10-10T20:20:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0701594700 1 -- 172.21.15.148:0/1748640493 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800bd60 con 0x7f06fc06ab90 2023-10-10T20:20:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 -- 172.21.15.148:0/1748640493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc06ab90 msgr2=0x7f06fc0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc06ab90 0x7f06fc0cb950 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099b0 tx=0x7f06f8009250 comp rx=0 tx=0).stop 2023-10-10T20:20:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 -- 172.21.15.148:0/1748640493 shutdown_connections 2023-10-10T20:20:25.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc06ab90 0x7f06fc0cb950 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 --2- 172.21.15.148:0/1748640493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc15abb0 0x7f06fc0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 -- 172.21.15.148:0/1748640493 >> 172.21.15.148:0/1748640493 conn(0x7f06fc0cd050 msgr2=0x7f06fc0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 -- 172.21.15.148:0/1748640493 shutdown_connections 2023-10-10T20:20:25.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.687+0000 7f0703598700 1 -- 172.21.15.148:0/1748640493 wait complete. 2023-10-10T20:20:25.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.688+0000 7f0703598700 1 Processor -- start 2023-10-10T20:20:25.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.688+0000 7f0703598700 1 -- start start 2023-10-10T20:20:25.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06fc06ab90 0x7f06fc1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc0cc710 0x7f06fc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc097c70 con 0x7f06fc15abb0 2023-10-10T20:20:25.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0c74a0 con 0x7f06fc06ab90 2023-10-10T20:20:25.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0703598700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1486e0 con 0x7f06fc0cc710 2023-10-10T20:20:25.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47690/0 (socket says 172.21.15.148:47690) 2023-10-10T20:20:25.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 -- 172.21.15.148:0/3972266601 learned_addr learned my addr 172.21.15.148:0/3972266601 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:25.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702596700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06fc06ab90 0x7f06fc1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0701d95700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc0cc710 0x7f06fc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 -- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc0cc710 msgr2=0x7f06fc0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc0cc710 0x7f06fc0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 -- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06fc06ab90 msgr2=0x7f06fc1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06fc06ab90 0x7f06fc1481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 -- 172.21.15.148:0/3972266601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80094f0 con 0x7f06fc15abb0 2023-10-10T20:20:25.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.689+0000 7f0702d97700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d100 tx=0x7f06f40099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400dda0 con 0x7f06fc15abb0 2023-10-10T20:20:25.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f400f040 con 0x7f06fc15abb0 2023-10-10T20:20:25.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0c6c00 con 0x7f06fc15abb0 2023-10-10T20:20:25.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4015690 con 0x7f06fc15abb0 2023-10-10T20:20:25.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc0c71c0 con 0x7f06fc15abb0 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.690+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc097a40 con 0x7f06fc15abb0 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.691+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06f401e3b0 con 0x7f06fc15abb0 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.691+0000 7f06f27fc700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.691+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06f4084ac0 con 0x7f06fc15abb0 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.691+0000 7f0702596700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.692+0000 7f0702596700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099b0 tx=0x7f06f8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:25.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.696+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06fc097a40 con 0x7f06fc15abb0 2023-10-10T20:20:25.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.922+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06fc097a40 con 0x7f06fc15abb0 2023-10-10T20:20:25.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.923+0000 7f06f27fc700 1 -- 172.21.15.148:0/3972266601 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f06fc097a40 con 0x7f06fc15abb0 2023-10-10T20:20:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:25.926 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:25.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 msgr2=0x7f06e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06f80099b0 tx=0x7f06f8017000 comp rx=0 tx=0).stop 2023-10-10T20:20:25.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 msgr2=0x7f06fc0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:25.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d100 tx=0x7f06f40099b0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 shutdown_connections 2023-10-10T20:20:25.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06fc15abb0 0x7f06fc0c7e40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06fc06ab90 0x7f06fc1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 --2- 172.21.15.148:0/3972266601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06fc0cc710 0x7f06fc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:25.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 >> 172.21.15.148:0/3972266601 conn(0x7f06fc0cd050 msgr2=0x7f06fc06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:25.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 shutdown_connections 2023-10-10T20:20:25.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:25.925+0000 7f0703598700 1 -- 172.21.15.148:0/3972266601 wait complete. 2023-10-10T20:20:25.930 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:25.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.052+0000 7fa18c9c0700 1 Processor -- start 2023-10-10T20:20:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.052+0000 7fa18c9c0700 1 -- start start 2023-10-10T20:20:26.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa18c9c0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1880cd6e0 0x7fa1880cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa18c9c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa18c9c0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1880cb7c0 con 0x7fa1880cdab0 2023-10-10T20:20:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa18c9c0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18815e850 con 0x7fa1880cd6e0 2023-10-10T20:20:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa18c9c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880cd030 con 0x7fa18815cbd0 2023-10-10T20:20:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1880cd6e0 0x7fa1880cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1880cd6e0 0x7fa1880cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47700/0 (socket says 172.21.15.148:47700) 2023-10-10T20:20:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 -- 172.21.15.148:0/573900825 learned_addr learned my addr 172.21.15.148:0/573900825 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa186ffd700 1 --1- 172.21.15.148:0/573900825 >> v1:172.21.15.148:6789/0 conn(0x7fa1880cdab0 0x7fa18815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52686/0 (socket says 172.21.15.148:52686) 2023-10-10T20:20:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa187fff700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 -- 172.21.15.148:0/573900825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 msgr2=0x7fa18815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 -- 172.21.15.148:0/573900825 >> v1:172.21.15.148:6789/0 conn(0x7fa1880cdab0 legacy=0x7fa18815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 -- 172.21.15.148:0/573900825 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18815e290 con 0x7fa1880cd6e0 2023-10-10T20:20:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1867fc700 1 -- 172.21.15.148:0/573900825 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750430481 0 0) 0x7fa1880cb7c0 con 0x7fa1880cdab0 2023-10-10T20:20:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.053+0000 7fa1877fe700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa1880cbc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa178009f20 tx=0x7fa178009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6875f4c00e58b2e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa1867fc700 1 -- 172.21.15.148:0/573900825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800e070 con 0x7fa1880cd6e0 2023-10-10T20:20:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa1867fc700 1 -- 172.21.15.148:0/573900825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800ba80 con 0x7fa1880cd6e0 2023-10-10T20:20:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa1867fc700 1 -- 172.21.15.148:0/573900825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780092e0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 -- 172.21.15.148:0/573900825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 msgr2=0x7fa1880cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa1880cbc50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa178009f20 tx=0x7fa178009840 comp rx=0 tx=0).stop 2023-10-10T20:20:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 -- 172.21.15.148:0/573900825 shutdown_connections 2023-10-10T20:20:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/573900825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa1880cbc50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 -- 172.21.15.148:0/573900825 >> 172.21.15.148:0/573900825 conn(0x7fa188067660 msgr2=0x7fa188067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 -- 172.21.15.148:0/573900825 shutdown_connections 2023-10-10T20:20:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.054+0000 7fa18c9c0700 1 -- 172.21.15.148:0/573900825 wait complete. 2023-10-10T20:20:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.055+0000 7fa18c9c0700 1 Processor -- start 2023-10-10T20:20:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.055+0000 7fa18c9c0700 1 -- start start 2023-10-10T20:20:26.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 0x7fa188147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18816ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa188155f40 con 0x7fa1880cd6e0 2023-10-10T20:20:26.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa188147cb0 con 0x7fa1880cdab0 2023-10-10T20:20:26.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa18c9c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18815c1d0 con 0x7fa18815cbd0 2023-10-10T20:20:26.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47712/0 (socket says 172.21.15.148:47712) 2023-10-10T20:20:26.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 -- 172.21.15.148:0/2204424314 learned_addr learned my addr 172.21.15.148:0/2204424314 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:26.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa186ffd700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 0x7fa188147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa187fff700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18816ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 -- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 msgr2=0x7fa18816ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18816ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 -- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 msgr2=0x7fa188147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 0x7fa188147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.056+0000 7fa1877fe700 1 -- 172.21.15.148:0/2204424314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178009ae0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa187fff700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18816ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:26.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa186ffd700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 0x7fa188147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:26.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa1877fe700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa1880c7b10 tx=0x7fa178018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178008bb0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa178008d50 con 0x7fa1880cd6e0 2023-10-10T20:20:26.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18816b2a0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780266b0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.057+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18815bb80 con 0x7fa1880cd6e0 2023-10-10T20:20:26.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.058+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa188089280 con 0x7fa1880cd6e0 2023-10-10T20:20:26.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.058+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa178003040 con 0x7fa1880cd6e0 2023-10-10T20:20:26.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.059+0000 7fa16f7fe700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 0x7fa1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.059+0000 7fa186ffd700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 0x7fa1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.059+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1780864a0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.059+0000 7fa186ffd700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 0x7fa1740693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa18815c5f0 tx=0x7fa17c007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.063+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1780537c0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.244+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa18815e570 con 0x7fa1880cd6e0 2023-10-10T20:20:26.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.244+0000 7fa16f7fe700 1 -- 172.21.15.148:0/2204424314 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa178056de0 con 0x7fa1880cd6e0 2023-10-10T20:20:26.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:26.247 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:21.752332+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 msgr2=0x7fa1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 0x7fa1740693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa18815c5f0 tx=0x7fa17c007c90 comp rx=0 tx=0).stop 2023-10-10T20:20:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 msgr2=0x7fa18815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa1880c7b10 tx=0x7fa178018000 comp rx=0 tx=0).stop 2023-10-10T20:20:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 shutdown_connections 2023-10-10T20:20:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa174066f30 0x7fa1740693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa18815cbd0 0x7fa18816ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1880cd6e0 0x7fa18815afc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 --2- 172.21.15.148:0/2204424314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1880cdab0 0x7fa188147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.246+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 >> 172.21.15.148:0/2204424314 conn(0x7fa188067660 msgr2=0x7fa1880b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.247+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 shutdown_connections 2023-10-10T20:20:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.247+0000 7fa18c9c0700 1 -- 172.21.15.148:0/2204424314 wait complete. 2023-10-10T20:20:26.261 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:20:26.261 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.374+0000 7f0c156d1700 1 Processor -- start 2023-10-10T20:20:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.374+0000 7f0c156d1700 1 -- start start 2023-10-10T20:20:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c156d1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c10157870 0x7f0c1015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c156d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c10158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c156d1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1015cbb0 con 0x7f0c1015a620 2023-10-10T20:20:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c156d1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1015bfa0 con 0x7f0c10157870 2023-10-10T20:20:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c156d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1015c280 con 0x7f0c10157c40 2023-10-10T20:20:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c10158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c067fc700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0c1015a620 0x7f0c1015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43684/0 (socket says 172.21.15.148:43684) 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c0effd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c10157870 0x7f0c1015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c067fc700 1 -- 172.21.15.148:0/830581690 learned_addr learned my addr 172.21.15.148:0/830581690 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c0effd700 1 -- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 msgr2=0x7f0c10158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.375+0000 7f0c0effd700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c10158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0effd700 1 -- 172.21.15.148:0/830581690 >> v1:172.21.15.161:6789/0 conn(0x7f0c1015a620 legacy=0x7f0c1015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0effd700 1 -- 172.21.15.148:0/830581690 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1015c560 con 0x7f0c10157870 2023-10-10T20:20:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0f7fe700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c10158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0e7fc700 1 -- 172.21.15.148:0/830581690 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157729228 0 0) 0x7f0c1015cbb0 con 0x7f0c1015a620 2023-10-10T20:20:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0effd700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10157870 0x7f0c1015b6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c000098e0 tx=0x7f0c000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7480026d2e407be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0e7fc700 1 -- 172.21.15.148:0/830581690 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000b640 con 0x7f0c10157870 2023-10-10T20:20:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0e7fc700 1 -- 172.21.15.148:0/830581690 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0000b7a0 con 0x7f0c10157870 2023-10-10T20:20:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c0e7fc700 1 -- 172.21.15.148:0/830581690 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000bb40 con 0x7f0c10157870 2023-10-10T20:20:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 -- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10157870 msgr2=0x7f0c1015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10157870 0x7f0c1015b6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c000098e0 tx=0x7f0c000091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 -- 172.21.15.148:0/830581690 shutdown_connections 2023-10-10T20:20:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10157870 0x7f0c1015b6a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 --2- 172.21.15.148:0/830581690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c10158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 -- 172.21.15.148:0/830581690 >> 172.21.15.148:0/830581690 conn(0x7f0c10068ce0 msgr2=0x7f0c100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.376+0000 7f0c156d1700 1 -- 172.21.15.148:0/830581690 shutdown_connections 2023-10-10T20:20:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.377+0000 7f0c156d1700 1 -- 172.21.15.148:0/830581690 wait complete. 2023-10-10T20:20:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 Processor -- start 2023-10-10T20:20:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 -- start start 2023-10-10T20:20:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c1016ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1015a620 0x7f0c10151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101393d0 con 0x7f0c10157870 2023-10-10T20:20:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10154200 con 0x7f0c1015a620 2023-10-10T20:20:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c156d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100b1760 con 0x7f0c10157c40 2023-10-10T20:20:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.378+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47714/0 (socket says 172.21.15.148:47714) 2023-10-10T20:20:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 -- 172.21.15.148:0/4087242708 learned_addr learned my addr 172.21.15.148:0/4087242708 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0f7fe700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1015a620 0x7f0c10151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c067fc700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c1016ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 -- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 msgr2=0x7f0c1016ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c1016ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 -- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1015a620 msgr2=0x7f0c10151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1015a620 0x7f0c10151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 -- 172.21.15.148:0/4087242708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009a20 con 0x7f0c10157870 2023-10-10T20:20:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c0effd700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0c000093f0 tx=0x7f0c00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000bb20 con 0x7f0c10157870 2023-10-10T20:20:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00011420 con 0x7f0c10157870 2023-10-10T20:20:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c00009420 con 0x7f0c10157870 2023-10-10T20:20:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00011800 con 0x7f0c10157870 2023-10-10T20:20:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.379+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c100b1d20 con 0x7f0c10157870 2023-10-10T20:20:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.380+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1004c3c0 con 0x7f0c10157870 2023-10-10T20:20:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.381+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c00011c30 con 0x7f0c10157870 2023-10-10T20:20:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.381+0000 7f0c077fe700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.381+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c00085fc0 con 0x7f0c10157870 2023-10-10T20:20:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.381+0000 7f0c067fc700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.381+0000 7f0c067fc700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 0x7f0bf00693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c1014d770 tx=0x7f0bf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.385+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c000b75a0 con 0x7f0c10157870 2023-10-10T20:20:26.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.613+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c1015c840 con 0x7f0c10157870 2023-10-10T20:20:26.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.614+0000 7f0c077fe700 1 -- 172.21.15.148:0/4087242708 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f0c00053230 con 0x7f0c10157870 2023-10-10T20:20:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:26.617 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 msgr2=0x7f0bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 0x7f0bf00693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0c1014d770 tx=0x7f0bf8009040 comp rx=0 tx=0).stop 2023-10-10T20:20:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 msgr2=0x7f0c1016b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0c000093f0 tx=0x7f0c00017000 comp rx=0 tx=0).stop 2023-10-10T20:20:26.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 shutdown_connections 2023-10-10T20:20:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bf0066f30 0x7f0bf00693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1015a620 0x7f0c10151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c10157870 0x7f0c1016b470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 --2- 172.21.15.148:0/4087242708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c10157c40 0x7f0c1016ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 >> 172.21.15.148:0/4087242708 conn(0x7f0c10068ce0 msgr2=0x7f0c100b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 shutdown_connections 2023-10-10T20:20:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.616+0000 7f0c156d1700 1 -- 172.21.15.148:0/4087242708 wait complete. 2023-10-10T20:20:26.621 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:26.630 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:26.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.743+0000 7f164759e700 1 Processor -- start 2023-10-10T20:20:26.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.743+0000 7f164759e700 1 -- start start 2023-10-10T20:20:26.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16480cdab0 0x7f164815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16480cb7c0 con 0x7f16480cd6e0 2023-10-10T20:20:26.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164815e290 con 0x7f164815dda0 2023-10-10T20:20:26.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164759e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164815e850 con 0x7f16480cdab0 2023-10-10T20:20:26.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16480cdab0 0x7f164815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16480cdab0 0x7f164815d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47716/0 (socket says 172.21.15.148:47716) 2023-10-10T20:20:26.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 -- 172.21.15.148:0/1014076320 learned_addr learned my addr 172.21.15.148:0/1014076320 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f16480cd6e0 0x7f16480cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52696/0 (socket says 172.21.15.148:52696) 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 -- 172.21.15.148:0/1014076320 >> v1:172.21.15.161:6789/0 conn(0x7f164815dda0 legacy=0x7f164815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 -- 172.21.15.148:0/1014076320 >> v1:172.21.15.148:6789/0 conn(0x7f16480cd6e0 legacy=0x7f16480cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f164559a700 1 -- 172.21.15.148:0/1014076320 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657944767 0 0) 0x7f16480cb7c0 con 0x7f16480cd6e0 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 -- 172.21.15.148:0/1014076320 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164815e570 con 0x7f16480cdab0 2023-10-10T20:20:26.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.744+0000 7f1645d9b700 1 --2- 172.21.15.148:0/1014076320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16480cdab0 0x7f164815d860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f163c0099b0 tx=0x7f163c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490ec066aab962aa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164559a700 1 -- 172.21.15.148:0/1014076320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00b820 con 0x7f16480cdab0 2023-10-10T20:20:26.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164559a700 1 -- 172.21.15.148:0/1014076320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00b980 con 0x7f16480cdab0 2023-10-10T20:20:26.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164559a700 1 -- 172.21.15.148:0/1014076320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00bd20 con 0x7f16480cdab0 2023-10-10T20:20:26.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 -- 172.21.15.148:0/1014076320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16480cdab0 msgr2=0x7f164815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 --2- 172.21.15.148:0/1014076320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16480cdab0 0x7f164815d860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f163c0099b0 tx=0x7f163c009250 comp rx=0 tx=0).stop 2023-10-10T20:20:26.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 -- 172.21.15.148:0/1014076320 shutdown_connections 2023-10-10T20:20:26.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 --2- 172.21.15.148:0/1014076320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16480cdab0 0x7f164815d860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 -- 172.21.15.148:0/1014076320 >> 172.21.15.148:0/1014076320 conn(0x7f1648067660 msgr2=0x7f1648067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 -- 172.21.15.148:0/1014076320 shutdown_connections 2023-10-10T20:20:26.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.745+0000 7f164759e700 1 -- 172.21.15.148:0/1014076320 wait complete. 2023-10-10T20:20:26.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.746+0000 7f164759e700 1 Processor -- start 2023-10-10T20:20:26.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.746+0000 7f164759e700 1 -- start start 2023-10-10T20:20:26.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16480cd6e0 0x7f1648153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16480cdab0 0x7f16480c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480cc360 con 0x7f164815dda0 2023-10-10T20:20:26.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480b3bb0 con 0x7f16480cd6e0 2023-10-10T20:20:26.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480b3e90 con 0x7f16480cdab0 2023-10-10T20:20:26.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47720/0 (socket says 172.21.15.148:47720) 2023-10-10T20:20:26.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 -- 172.21.15.148:0/509249229 learned_addr learned my addr 172.21.15.148:0/509249229 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:26.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1645d9b700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16480cdab0 0x7f16480c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f164659c700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16480cd6e0 0x7f1648153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 -- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16480cdab0 msgr2=0x7f16480c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16480cdab0 0x7f16480c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 -- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16480cd6e0 msgr2=0x7f1648153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16480cd6e0 0x7f1648153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.747+0000 7f1646d9d700 1 -- 172.21.15.148:0/509249229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0094f0 con 0x7f164815dda0 2023-10-10T20:20:26.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f1646d9d700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f163800b5e0 tx=0x7f1638009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800f510 con 0x7f164815dda0 2023-10-10T20:20:26.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163800f670 con 0x7f164815dda0 2023-10-10T20:20:26.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16480b4170 con 0x7f164815dda0 2023-10-10T20:20:26.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800fa30 con 0x7f164815dda0 2023-10-10T20:20:26.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.748+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16480b4730 con 0x7f164815dda0 2023-10-10T20:20:26.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.749+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648089280 con 0x7f164815dda0 2023-10-10T20:20:26.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.749+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f163801a400 con 0x7f164815dda0 2023-10-10T20:20:26.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.749+0000 7f16367fc700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 0x7f16280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:26.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.749+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1638083ed0 con 0x7f164815dda0 2023-10-10T20:20:26.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.749+0000 7f164659c700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 0x7f16280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:26.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.750+0000 7f164659c700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 0x7f16280693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1640001560 tx=0x7f1640001b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:26.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.754+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16380511c0 con 0x7f164815dda0 2023-10-10T20:20:26.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.976+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f164804bb80 con 0x7f164815dda0 2023-10-10T20:20:26.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.976+0000 7f16367fc700 1 -- 172.21.15.148:0/509249229 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f16380547e0 con 0x7f164815dda0 2023-10-10T20:20:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:10.053909+0000","modified":"2023-10-10T20:20:13.716539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/812978511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":812978511},{"type":"v1","addr":"172.21.15.161:6835","nonce":812978511}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-10T20:20:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 msgr2=0x7f16280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 0x7f16280693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1640001560 tx=0x7f1640001b00 comp rx=0 tx=0).stop 2023-10-10T20:20:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 msgr2=0x7f1648159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f163800b5e0 tx=0x7f1638009060 comp rx=0 tx=0).stop 2023-10-10T20:20:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 shutdown_connections 2023-10-10T20:20:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1628066f30 0x7f16280693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f164815dda0 0x7f1648159810 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16480cd6e0 0x7f1648153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 --2- 172.21.15.148:0/509249229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16480cdab0 0x7f16480c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 >> 172.21.15.148:0/509249229 conn(0x7f1648067660 msgr2=0x7f16480c7020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 shutdown_connections 2023-10-10T20:20:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:26.978+0000 7f164759e700 1 -- 172.21.15.148:0/509249229 wait complete. 2023-10-10T20:20:26.984 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2023-10-10T20:20:26.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:27.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.104+0000 7f9b49b9f700 1 Processor -- start 2023-10-10T20:20:27.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.104+0000 7f9b49b9f700 1 -- start start 2023-10-10T20:20:27.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b49b9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b49b9f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440cb7c0 con 0x7f9b440cdab0 2023-10-10T20:20:27.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b49b9f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4415e290 con 0x7f9b4415cbd0 2023-10-10T20:20:27.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b49b9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4415e850 con 0x7f9b440cd6e0 2023-10-10T20:20:27.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b43fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9b440cdab0 0x7f9b4415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52700/0 (socket says 172.21.15.148:52700) 2023-10-10T20:20:27.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b43fff700 1 -- 172.21.15.148:0/2909867073 learned_addr learned my addr 172.21.15.148:0/2909867073 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:27.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.105+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/2909867073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938179163 0 0) 0x7f9b440cb7c0 con 0x7f9b440cdab0 2023-10-10T20:20:27.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b2c003570 con 0x7f9b440cdab0 2023-10-10T20:20:27.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963205429 0 0) 0x7f9b4415e290 con 0x7f9b4415cbd0 2023-10-10T20:20:27.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b440cb7c0 con 0x7f9b4415cbd0 2023-10-10T20:20:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b48b9d700 1 -- 172.21.15.148:0/2909867073 >> v1:172.21.15.161:6789/0 conn(0x7f9b4415cbd0 legacy=0x7f9b4415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b48b9d700 1 -- 172.21.15.148:0/2909867073 >> v1:172.21.15.148:6789/0 conn(0x7f9b440cdab0 legacy=0x7f9b4415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b48b9d700 1 -- 172.21.15.148:0/2909867073 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4415e570 con 0x7f9b440cd6e0 2023-10-10T20:20:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/2909867073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440cbc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009e90 tx=0x7f9b34009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd9c041ee54af88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400e070 con 0x7f9b440cd6e0 2023-10-10T20:20:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3400bb80 con 0x7f9b440cd6e0 2023-10-10T20:20:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b437fe700 1 -- 172.21.15.148:0/2909867073 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34012430 con 0x7f9b440cd6e0 2023-10-10T20:20:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b49b9f700 1 -- 172.21.15.148:0/2909867073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 msgr2=0x7f9b440cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/2909867073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440cbc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009e90 tx=0x7f9b34009780 comp rx=0 tx=0).stop 2023-10-10T20:20:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b49b9f700 1 -- 172.21.15.148:0/2909867073 shutdown_connections 2023-10-10T20:20:27.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/2909867073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440cbc50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.106+0000 7f9b49b9f700 1 -- 172.21.15.148:0/2909867073 >> 172.21.15.148:0/2909867073 conn(0x7f9b44067660 msgr2=0x7f9b44067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:27.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.107+0000 7f9b49b9f700 1 -- 172.21.15.148:0/2909867073 shutdown_connections 2023-10-10T20:20:27.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.107+0000 7f9b49b9f700 1 -- 172.21.15.148:0/2909867073 wait complete. 2023-10-10T20:20:27.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 Processor -- start 2023-10-10T20:20:27.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 -- start start 2023-10-10T20:20:27.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b440cdab0 0x7f9b441537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44151380 con 0x7f9b4415cbd0 2023-10-10T20:20:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4414d0b0 con 0x7f9b440cdab0 2023-10-10T20:20:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.108+0000 7f9b49b9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44151510 con 0x7f9b440cd6e0 2023-10-10T20:20:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47736/0 (socket says 172.21.15.148:47736) 2023-10-10T20:20:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 -- 172.21.15.148:0/3710148718 learned_addr learned my addr 172.21.15.148:0/3710148718 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b43fff700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b440cdab0 0x7f9b441537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 -- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 msgr2=0x7f9b440b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 -- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b440cdab0 msgr2=0x7f9b441537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b440cdab0 0x7f9b441537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 -- 172.21.15.148:0/3710148718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b34009a20 con 0x7f9b4415cbd0 2023-10-10T20:20:27.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:27.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b4939e700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c4e0 tx=0x7f9b3c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c011660 con 0x7f9b4415cbd0 2023-10-10T20:20:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c0117e0 con 0x7f9b4415cbd0 2023-10-10T20:20:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440b6550 con 0x7f9b4415cbd0 2023-10-10T20:20:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c011ba0 con 0x7f9b4415cbd0 2023-10-10T20:20:27.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.109+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440b4ce0 con 0x7f9b4415cbd0 2023-10-10T20:20:27.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.110+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b440c7870 con 0x7f9b4415cbd0 2023-10-10T20:20:27.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.111+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b3c012420 con 0x7f9b4415cbd0 2023-10-10T20:20:27.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.111+0000 7f9b40ff9700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.111+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b3c002030 con 0x7f9b4415cbd0 2023-10-10T20:20:27.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.111+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.111+0000 7f9b48b9d700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 0x7f9b300693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009de0 tx=0x7f9b34019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.115+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b3c0514d0 con 0x7f9b4415cbd0 2023-10-10T20:20:27.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.300+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b44089320 con 0x7f9b4415cbd0 2023-10-10T20:20:27.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.301+0000 7f9b40ff9700 1 -- 172.21.15.148:0/3710148718 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f9b44089320 con 0x7f9b4415cbd0 2023-10-10T20:20:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:27.305 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:21.752332+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:07.830881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:09.018572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:27.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 msgr2=0x7f9b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 0x7f9b300693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009de0 tx=0x7f9b34019000 comp rx=0 tx=0).stop 2023-10-10T20:20:27.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 msgr2=0x7f9b440b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c4e0 tx=0x7f9b3c007d80 comp rx=0 tx=0).stop 2023-10-10T20:20:27.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 shutdown_connections 2023-10-10T20:20:27.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9b30066f30 0x7f9b300693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b4415cbd0 0x7f9b440b6010 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b440cd6e0 0x7f9b440b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 --2- 172.21.15.148:0/3710148718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b440cdab0 0x7f9b441537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 >> 172.21.15.148:0/3710148718 conn(0x7f9b44067660 msgr2=0x7f9b440cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:27.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 shutdown_connections 2023-10-10T20:20:27.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.303+0000 7f9b49b9f700 1 -- 172.21.15.148:0/3710148718 wait complete. 2023-10-10T20:20:27.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:20:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.432+0000 7f9436918700 1 Processor -- start 2023-10-10T20:20:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.432+0000 7f9436918700 1 -- start start 2023-10-10T20:20:27.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f9436918700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f943015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f9436918700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943015cbb0 con 0x7f9430157870 2023-10-10T20:20:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f9436918700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943015bfa0 con 0x7f943015d580 2023-10-10T20:20:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f9436918700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943015bdb0 con 0x7f943015a620 2023-10-10T20:20:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f942ffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9430157870 0x7f943015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52706/0 (socket says 172.21.15.148:52706) 2023-10-10T20:20:27.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.433+0000 7f942ffff700 1 -- 172.21.15.148:0/600555431 learned_addr learned my addr 172.21.15.148:0/600555431 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:27.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f943015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513175117 0 0) 0x7f943015cbb0 con 0x7f9430157870 2023-10-10T20:20:27.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9418003570 con 0x7f9430157870 2023-10-10T20:20:27.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113117272 0 0) 0x7f943015bfa0 con 0x7f943015d580 2023-10-10T20:20:27.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f943015cbb0 con 0x7f943015d580 2023-10-10T20:20:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1661079839 0 0) 0x7f9418003570 con 0x7f9430157870 2023-10-10T20:20:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 msgr2=0x7f943015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f943015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 >> v1:172.21.15.161:6789/0 conn(0x7f943015d580 legacy=0x7f9430158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943015c280 con 0x7f9430157870 2023-10-10T20:20:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9420001470 con 0x7f9430157870 2023-10-10T20:20:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f943015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 >> v1:172.21.15.148:6789/0 conn(0x7f9430157870 legacy=0x7f943015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9418003c20 0x7f9418004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418004810 0x7f9418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f94180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941800bb10 con 0x7f9418004810 2023-10-10T20:20:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941800bc80 con 0x7f9418003c20 2023-10-10T20:20:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941800be40 con 0x7f943015a620 2023-10-10T20:20:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418004810 0x7f9418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942ffff700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9418003c20 0x7f9418004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f9434eb5700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f94180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 -- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 msgr2=0x7f94180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:27.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f94180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 -- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9418003c20 msgr2=0x7f9418004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9418003c20 0x7f9418004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f942f7fe700 1 -- 172.21.15.148:0/600555431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009420 con 0x7f9418004810 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.434+0000 7f9434eb5700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f943015a620 0x7f94180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f942f7fe700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418004810 0x7f9418004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f943015aa20 tx=0x7f94240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a27618e564f0ef server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424009740 con 0x7f9418004810 2023-10-10T20:20:27.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400e920 con 0x7f9418004810 2023-10-10T20:20:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f942effd700 1 -- 172.21.15.148:0/600555431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400ece0 con 0x7f9418004810 2023-10-10T20:20:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 -- 172.21.15.148:0/600555431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418004810 msgr2=0x7f9418004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 --2- 172.21.15.148:0/600555431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418004810 0x7f9418004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f943015aa20 tx=0x7f94240091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9434eb5700 1 -- 172.21.15.148:0/600555431 reap_dead start 2023-10-10T20:20:27.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 -- 172.21.15.148:0/600555431 shutdown_connections 2023-10-10T20:20:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 -- 172.21.15.148:0/600555431 >> 172.21.15.148:0/600555431 conn(0x7f9430068ce0 msgr2=0x7f94300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 -- 172.21.15.148:0/600555431 shutdown_connections 2023-10-10T20:20:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.435+0000 7f9436918700 1 -- 172.21.15.148:0/600555431 wait complete. 2023-10-10T20:20:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.436+0000 7f9436918700 1 Processor -- start 2023-10-10T20:20:27.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.436+0000 7f9436918700 1 -- start start 2023-10-10T20:20:27.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 0x7f943014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94301553b0 0x7f9430155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300d0420 con 0x7f94300cfa70 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943016b380 con 0x7f94301553b0 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9436918700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9430153d40 con 0x7f9418004810 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9434eb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9434eb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47754/0 (socket says 172.21.15.148:47754) 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f9434eb5700 1 -- 172.21.15.148:0/2344707422 learned_addr learned my addr 172.21.15.148:0/2344707422 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:27.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f942f7fe700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94301553b0 0x7f9430155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.437+0000 7f942ffff700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 0x7f943014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 -- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 msgr2=0x7f943014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 0x7f943014a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 -- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94301553b0 msgr2=0x7f9430155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94301553b0 0x7f9430155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 -- 172.21.15.148:0/2344707422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943014ace0 con 0x7f94300cfa70 2023-10-10T20:20:27.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f942ffff700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 0x7f943014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:27.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9434eb5700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f943006b4b0 tx=0x7f9428009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94280032e0 con 0x7f94300cfa70 2023-10-10T20:20:27.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9428003440 con 0x7f94300cfa70 2023-10-10T20:20:27.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428001930 con 0x7f94300cfa70 2023-10-10T20:20:27.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9430155eb0 con 0x7f94300cfa70 2023-10-10T20:20:27.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.438+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943016ad30 con 0x7f94300cfa70 2023-10-10T20:20:27.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.439+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943004c3c0 con 0x7f94300cfa70 2023-10-10T20:20:27.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.439+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9428001d20 con 0x7f94300cfa70 2023-10-10T20:20:27.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.440+0000 7f9417fff700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 0x7f941c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.440+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9428083f80 con 0x7f94300cfa70 2023-10-10T20:20:27.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.440+0000 7f942ffff700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 0x7f941c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.440+0000 7f942ffff700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 0x7f941c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9420001ca0 tx=0x7f9420009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.444+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94280511f0 con 0x7f94300cfa70 2023-10-10T20:20:27.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.668+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f943016b4c0 con 0x7f94300cfa70 2023-10-10T20:20:27.670 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:20:27.668+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:20:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.673+0000 7f9417fff700 1 -- 172.21.15.148:0/2344707422 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9428019090 con 0x7f94300cfa70 2023-10-10T20:20:27.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 msgr2=0x7f941c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 0x7f941c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9420001ca0 tx=0x7f9420009040 comp rx=0 tx=0).stop 2023-10-10T20:20:27.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 msgr2=0x7f94300cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f943006b4b0 tx=0x7f9428009040 comp rx=0 tx=0).stop 2023-10-10T20:20:27.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 shutdown_connections 2023-10-10T20:20:27.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f941c066f30 0x7f941c0693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94300cfa70 0x7f94300cfee0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9418004810 0x7f943014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 --2- 172.21.15.148:0/2344707422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94301553b0 0x7f9430155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 >> 172.21.15.148:0/2344707422 conn(0x7f9430068ce0 msgr2=0x7f94300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 shutdown_connections 2023-10-10T20:20:27.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.675+0000 7f9436918700 1 -- 172.21.15.148:0/2344707422 wait complete. 2023-10-10T20:20:27.680 INFO:tasks.ceph.mds.d.smithi161.stderr: -3322> 2023-10-10T20:20:14.664+0000 7f9d68f45700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:20:27.681 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:20:27.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:20:27.702 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:20:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.806+0000 7fbd5ad63700 1 Processor -- start 2023-10-10T20:20:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.806+0000 7fbd5ad63700 1 -- start start 2023-10-10T20:20:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd5ad63700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbd5415a290 0x7fbd5406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd5ad63700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd540b0790 con 0x7fbd5406aae0 2023-10-10T20:20:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd5ad63700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5406af20 con 0x7fbd54159ec0 2023-10-10T20:20:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd5ad63700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd540ce1f0 con 0x7fbd5415a290 2023-10-10T20:20:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd59300700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbd5406aae0 0x7fbd540cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52712/0 (socket says 172.21.15.148:52712) 2023-10-10T20:20:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd59300700 1 -- 172.21.15.148:0/2824787486 learned_addr learned my addr 172.21.15.148:0/2824787486 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:27.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.807+0000 7fbd4bfff700 1 --2- 172.21.15.148:0/2824787486 >> v2:172.21.15.161:3300/0 conn(0x7fbd5415a290 0x7fbd5406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47837019 0 0) 0x7fbd540b0790 con 0x7fbd5406aae0 2023-10-10T20:20:27.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd3c003570 con 0x7fbd5406aae0 2023-10-10T20:20:27.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991108036 0 0) 0x7fbd5406af20 con 0x7fbd54159ec0 2023-10-10T20:20:27.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd540b0790 con 0x7fbd54159ec0 2023-10-10T20:20:27.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4bfff700 1 -- 172.21.15.148:0/2824787486 >> v1:172.21.15.161:6789/0 conn(0x7fbd54159ec0 legacy=0x7fbd540af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197305377 0 0) 0x7fbd3c003570 con 0x7fbd5406aae0 2023-10-10T20:20:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4bfff700 1 -- 172.21.15.148:0/2824787486 >> v1:172.21.15.148:6789/0 conn(0x7fbd5406aae0 legacy=0x7fbd540cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4bfff700 1 -- 172.21.15.148:0/2824787486 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540b0b20 con 0x7fbd5415a290 2023-10-10T20:20:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4bfff700 1 --2- 172.21.15.148:0/2824787486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd5406a5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd440098e0 tx=0x7fbd440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e4809bf5ddad50 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c001470 con 0x7fbd5415a290 2023-10-10T20:20:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44011420 con 0x7fbd5415a290 2023-10-10T20:20:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.808+0000 7fbd4b7fe700 1 -- 172.21.15.148:0/2824787486 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440117c0 con 0x7fbd5415a290 2023-10-10T20:20:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 -- 172.21.15.148:0/2824787486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 msgr2=0x7fbd5406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/2824787486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd5406a5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd440098e0 tx=0x7fbd440091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 -- 172.21.15.148:0/2824787486 shutdown_connections 2023-10-10T20:20:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/2824787486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd5406a5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 -- 172.21.15.148:0/2824787486 >> 172.21.15.148:0/2824787486 conn(0x7fbd540d1880 msgr2=0x7fbd540cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 -- 172.21.15.148:0/2824787486 shutdown_connections 2023-10-10T20:20:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.809+0000 7fbd5ad63700 1 -- 172.21.15.148:0/2824787486 wait complete. 2023-10-10T20:20:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.810+0000 7fbd5ad63700 1 Processor -- start 2023-10-10T20:20:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.810+0000 7fbd5ad63700 1 -- start start 2023-10-10T20:20:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd54159ec0 0x7fbd54155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd540c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5416a390 con 0x7fbd5406aae0 2023-10-10T20:20:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540ca3f0 con 0x7fbd5415a290 2023-10-10T20:20:27.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5416a600 con 0x7fbd54159ec0 2023-10-10T20:20:27.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47770/0 (socket says 172.21.15.148:47770) 2023-10-10T20:20:27.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 -- 172.21.15.148:0/1960767262 learned_addr learned my addr 172.21.15.148:0/1960767262 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:27.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd59300700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd540c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd4bfff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd54159ec0 0x7fbd54155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 -- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd54159ec0 msgr2=0x7fbd54155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd54159ec0 0x7fbd54155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 -- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 msgr2=0x7fbd540c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:27.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd540c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:27.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 -- 172.21.15.148:0/1960767262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009420 con 0x7fbd5406aae0 2023-10-10T20:20:27.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd58aff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000cb40 tx=0x7fbd50002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000ce50 con 0x7fbd5406aae0 2023-10-10T20:20:27.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5000a7a0 con 0x7fbd5406aae0 2023-10-10T20:20:27.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.811+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54156100 con 0x7fbd5406aae0 2023-10-10T20:20:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.812+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000ab40 con 0x7fbd5406aae0 2023-10-10T20:20:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.812+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540c9b50 con 0x7fbd5406aae0 2023-10-10T20:20:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.812+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5404c3c0 con 0x7fbd5406aae0 2023-10-10T20:20:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.813+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd50012420 con 0x7fbd5406aae0 2023-10-10T20:20:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.813+0000 7fbd48ff9700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:27.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.814+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbd500848f0 con 0x7fbd5406aae0 2023-10-10T20:20:27.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.814+0000 7fbd4bfff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:27.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.814+0000 7fbd4bfff700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 0x7fbd400693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd440093f0 tx=0x7fbd44016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:27.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:27.818+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd50051b10 con 0x7fbd5406aae0 2023-10-10T20:20:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.042+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbd540b04b0 con 0x7fbd5406aae0 2023-10-10T20:20:28.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.673+0000 7fbd48ff9700 1 -- 172.21.15.148:0/1960767262 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbd50055130 con 0x7fbd5406aae0 2023-10-10T20:20:28.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 msgr2=0x7fbd400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 0x7fbd400693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd440093f0 tx=0x7fbd44016000 comp rx=0 tx=0).stop 2023-10-10T20:20:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 msgr2=0x7fbd540b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000cb40 tx=0x7fbd50002820 comp rx=0 tx=0).stop 2023-10-10T20:20:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 shutdown_connections 2023-10-10T20:20:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd40066f30 0x7fbd400693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5415a290 0x7fbd540c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5406aae0 0x7fbd540b60a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 --2- 172.21.15.148:0/1960767262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd54159ec0 0x7fbd54155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 >> 172.21.15.148:0/1960767262 conn(0x7fbd540d1880 msgr2=0x7fbd540c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:28.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 shutdown_connections 2023-10-10T20:20:28.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.675+0000 7fbd5ad63700 1 -- 172.21.15.148:0/1960767262 wait complete. 2023-10-10T20:20:28.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:20:28.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.809+0000 7f7a6cf9f700 1 Processor -- start 2023-10-10T20:20:28.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.809+0000 7f7a6cf9f700 1 -- start start 2023-10-10T20:20:28.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6cf9f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7a680b4e00 0x7f7a6815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6cf9f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b17b0 con 0x7f7a68159ef0 2023-10-10T20:20:28.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6cf9f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b1a60 con 0x7f7a680b6a10 2023-10-10T20:20:28.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6cf9f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680b1d40 con 0x7f7a680b4e00 2023-10-10T20:20:28.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7a68159ef0 0x7f7a680b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52714/0 (socket says 172.21.15.148:52714) 2023-10-10T20:20:28.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6659c700 1 -- 172.21.15.148:0/1119799215 learned_addr learned my addr 172.21.15.148:0/1119799215 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:28.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/1119799215 >> v2:172.21.15.161:3300/0 conn(0x7f7a680b4e00 0x7f7a6815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485512731 0 0) 0x7f7a680b17b0 con 0x7f7a68159ef0 2023-10-10T20:20:28.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a50003570 con 0x7f7a68159ef0 2023-10-10T20:20:28.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347762259 0 0) 0x7f7a680b1a60 con 0x7f7a680b6a10 2023-10-10T20:20:28.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a680b17b0 con 0x7f7a680b6a10 2023-10-10T20:20:28.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865032998 0 0) 0x7f7a50003570 con 0x7f7a68159ef0 2023-10-10T20:20:28.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 >> v2:172.21.15.161:3300/0 conn(0x7f7a680b4e00 msgr2=0x7f7a6815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:28.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 --2- 172.21.15.148:0/1119799215 >> v2:172.21.15.161:3300/0 conn(0x7f7a680b4e00 0x7f7a6815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 >> v1:172.21.15.161:6789/0 conn(0x7f7a680b6a10 legacy=0x7f7a680b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680b2020 con 0x7f7a68159ef0 2023-10-10T20:20:28.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.810+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7a580015f0 con 0x7f7a68159ef0 2023-10-10T20:20:28.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 >> v1:172.21.15.148:6789/0 conn(0x7f7a68159ef0 legacy=0x7f7a680b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 0x7f7a50003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b1d40 con 0x7f7a500051c0 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5000c0b0 con 0x7f7a50003bf0 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5000c270 con 0x7f7a50004810 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/1119799215 >> v2:172.21.15.161:3300/0 conn(0x7f7a680b4e00 0x7f7a6815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 0x7f7a50003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a65d9b700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 -- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50004810 msgr2=0x7f7a50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:28.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 -- 172.21.15.148:0/1119799215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a500051c0 msgr2=0x7f7a5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:28.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 -- 172.21.15.148:0/1119799215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50003570 con 0x7f7a50003bf0 2023-10-10T20:20:28.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:28.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a65d9b700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:28.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a66d9d700 1 -- 172.21.15.148:0/1119799215 reap_dead start 2023-10-10T20:20:28.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6659c700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 0x7f7a50003fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c2d0 tx=0x7f7a580078a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2a8fbf35ddf205 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:28.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800cc00 con 0x7f7a50003bf0 2023-10-10T20:20:28.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5800cd60 con 0x7f7a50003bf0 2023-10-10T20:20:28.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.811+0000 7f7a6559a700 1 -- 172.21.15.148:0/1119799215 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007320 con 0x7f7a50003bf0 2023-10-10T20:20:28.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 msgr2=0x7f7a50003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 0x7f7a50003fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c2d0 tx=0x7f7a580078a0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/1119799215 shutdown_connections 2023-10-10T20:20:28.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/1119799215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50003bf0 0x7f7a50003fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/1119799215 >> 172.21.15.148:0/1119799215 conn(0x7f7a6809a0a0 msgr2=0x7f7a68067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:28.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/1119799215 shutdown_connections 2023-10-10T20:20:28.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.812+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/1119799215 wait complete. 2023-10-10T20:20:28.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.813+0000 7f7a6cf9f700 1 Processor -- start 2023-10-10T20:20:28.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.813+0000 7f7a6cf9f700 1 -- start start 2023-10-10T20:20:28.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003bf0 0x7f7a6816def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a6809ce50 0x7f7a6809d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68156f40 con 0x7f7a6815e2e0 2023-10-10T20:20:28.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6815f360 con 0x7f7a6809ce50 2023-10-10T20:20:28.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6cf9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6815f080 con 0x7f7a50003bf0 2023-10-10T20:20:28.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47786/0 (socket says 172.21.15.148:47786) 2023-10-10T20:20:28.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 -- 172.21.15.148:0/2957094261 learned_addr learned my addr 172.21.15.148:0/2957094261 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:28.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a65d9b700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a6809ce50 0x7f7a6809d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a6659c700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003bf0 0x7f7a6816def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 -- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003bf0 msgr2=0x7f7a6816def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003bf0 0x7f7a6816def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 -- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a6809ce50 msgr2=0x7f7a6809d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:28.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a6809ce50 0x7f7a6809d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:28.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 -- 172.21.15.148:0/2957094261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a681568f0 con 0x7f7a6815e2e0 2023-10-10T20:20:28.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a66d9d700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b5200 tx=0x7f7a6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:28.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.814+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000e040 con 0x7f7a6815e2e0 2023-10-10T20:20:28.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.815+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6000f040 con 0x7f7a6815e2e0 2023-10-10T20:20:28.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.815+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68156bd0 con 0x7f7a6815e2e0 2023-10-10T20:20:28.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.815+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60015430 con 0x7f7a6815e2e0 2023-10-10T20:20:28.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.815+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6809d800 con 0x7f7a6815e2e0 2023-10-10T20:20:28.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.815+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6804c3c0 con 0x7f7a6815e2e0 2023-10-10T20:20:28.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.816+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a60015820 con 0x7f7a6815e2e0 2023-10-10T20:20:28.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.816+0000 7f7a567fc700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:28.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.816+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7a60085110 con 0x7f7a6815e2e0 2023-10-10T20:20:28.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.816+0000 7f7a6659c700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:28.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.817+0000 7f7a6659c700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 0x7f7a480693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000ef0 tx=0x7f7a58015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:28.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:28.821+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a60055dc0 con 0x7f7a6815e2e0 2023-10-10T20:20:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.005+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a6809dd70 con 0x7f7a6815e2e0 2023-10-10T20:20:29.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.777+0000 7f7a567fc700 1 -- 172.21.15.148:0/2957094261 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7a60055950 con 0x7f7a6815e2e0 2023-10-10T20:20:29.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 msgr2=0x7f7a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 0x7f7a480693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000ef0 tx=0x7f7a58015000 comp rx=0 tx=0).stop 2023-10-10T20:20:29.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 msgr2=0x7f7a6815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b5200 tx=0x7f7a6000b040 comp rx=0 tx=0).stop 2023-10-10T20:20:29.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 shutdown_connections 2023-10-10T20:20:29.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a48066f30 0x7f7a480693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a6815e2e0 0x7f7a6815e750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003bf0 0x7f7a6816def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 --2- 172.21.15.148:0/2957094261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a6809ce50 0x7f7a6809d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 >> 172.21.15.148:0/2957094261 conn(0x7f7a6809a0a0 msgr2=0x7f7a68155940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:29.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 shutdown_connections 2023-10-10T20:20:29.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.780+0000 7f7a6cf9f700 1 -- 172.21.15.148:0/2957094261 wait complete. 2023-10-10T20:20:29.785 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:20:29.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:20:29.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.916+0000 7fba98c5d700 1 Processor -- start 2023-10-10T20:20:29.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.916+0000 7fba98c5d700 1 -- start start 2023-10-10T20:20:29.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba98c5d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba940cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba98c5d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba9406dad0 0x7fba940cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba98c5d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba940c6f20 con 0x7fba94156210 2023-10-10T20:20:29.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba98c5d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940c7200 con 0x7fba9406dad0 2023-10-10T20:20:29.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba98c5d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940c74e0 con 0x7fba940af8f0 2023-10-10T20:20:29.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba9406dad0 0x7fba940cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9259c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fba94156210 0x7fba9406d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52728/0 (socket says 172.21.15.148:52728) 2023-10-10T20:20:29.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9259c700 1 -- 172.21.15.148:0/2691935908 learned_addr learned my addr 172.21.15.148:0/2691935908 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:29.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba91d9b700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba940cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 -- 172.21.15.148:0/2691935908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 msgr2=0x7fba940cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:29.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba940cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 -- 172.21.15.148:0/2691935908 >> v1:172.21.15.148:6789/0 conn(0x7fba94156210 legacy=0x7fba9406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 -- 172.21.15.148:0/2691935908 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940c77c0 con 0x7fba9406dad0 2023-10-10T20:20:29.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9159a700 1 -- 172.21.15.148:0/2691935908 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806092294 0 0) 0x7fba940c6f20 con 0x7fba94156210 2023-10-10T20:20:29.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba92d9d700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba9406dad0 0x7fba940cbcb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009e60 tx=0x7fba8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba99f98cbb4d1d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:29.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9159a700 1 -- 172.21.15.148:0/2691935908 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00e070 con 0x7fba9406dad0 2023-10-10T20:20:29.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9159a700 1 -- 172.21.15.148:0/2691935908 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00bc30 con 0x7fba9406dad0 2023-10-10T20:20:29.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.917+0000 7fba9159a700 1 -- 172.21.15.148:0/2691935908 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c012430 con 0x7fba9406dad0 2023-10-10T20:20:29.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 -- 172.21.15.148:0/2691935908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba9406dad0 msgr2=0x7fba940cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba9406dad0 0x7fba940cbcb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009e60 tx=0x7fba8c009780 comp rx=0 tx=0).stop 2023-10-10T20:20:29.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 -- 172.21.15.148:0/2691935908 shutdown_connections 2023-10-10T20:20:29.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba9406dad0 0x7fba940cbcb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2691935908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba940cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 -- 172.21.15.148:0/2691935908 >> 172.21.15.148:0/2691935908 conn(0x7fba940cd450 msgr2=0x7fba940c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:29.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 -- 172.21.15.148:0/2691935908 shutdown_connections 2023-10-10T20:20:29.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.918+0000 7fba98c5d700 1 -- 172.21.15.148:0/2691935908 wait complete. 2023-10-10T20:20:29.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.919+0000 7fba98c5d700 1 Processor -- start 2023-10-10T20:20:29.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.919+0000 7fba98c5d700 1 -- start start 2023-10-10T20:20:29.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9406dad0 0x7fba940b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba9413ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba940c5490 con 0x7fba94156210 2023-10-10T20:20:29.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba940b1090 con 0x7fba9406dad0 2023-10-10T20:20:29.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940c4880 con 0x7fba940af8f0 2023-10-10T20:20:29.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47790/0 (socket says 172.21.15.148:47790) 2023-10-10T20:20:29.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 -- 172.21.15.148:0/2719591901 learned_addr learned my addr 172.21.15.148:0/2719591901 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:29.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba91d9b700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba9413ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba9259c700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9406dad0 0x7fba940b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 -- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 msgr2=0x7fba9413ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba9413ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 -- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9406dad0 msgr2=0x7fba940b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:29.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9406dad0 0x7fba940b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:29.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 -- 172.21.15.148:0/2719591901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009a20 con 0x7fba94156210 2023-10-10T20:20:29.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba92d9d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba8c013000 tx=0x7fba8c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:29.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c027470 con 0x7fba94156210 2023-10-10T20:20:29.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c027610 con 0x7fba94156210 2023-10-10T20:20:29.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0279d0 con 0x7fba94156210 2023-10-10T20:20:29.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba940c4b60 con 0x7fba94156210 2023-10-10T20:20:29.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.920+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba940c5120 con 0x7fba94156210 2023-10-10T20:20:29.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.921+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9404c3c0 con 0x7fba94156210 2023-10-10T20:20:29.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.922+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba8c024070 con 0x7fba94156210 2023-10-10T20:20:29.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.922+0000 7fba827fc700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 0x7fba740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:29.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.922+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fba8c086a60 con 0x7fba94156210 2023-10-10T20:20:29.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.922+0000 7fba9259c700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 0x7fba740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:29.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.923+0000 7fba9259c700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 0x7fba740693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba84007fd0 tx=0x7fba84007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:29.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:29.927+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba8c053e30 con 0x7fba94156210 2023-10-10T20:20:30.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.111+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba9406b220 con 0x7fba94156210 2023-10-10T20:20:30.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.785+0000 7fba827fc700 1 -- 172.21.15.148:0/2719591901 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7fba8c057450 con 0x7fba94156210 2023-10-10T20:20:30.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 msgr2=0x7fba740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 0x7fba740693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba84007fd0 tx=0x7fba84007d30 comp rx=0 tx=0).stop 2023-10-10T20:20:30.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 msgr2=0x7fba940c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba8c013000 tx=0x7fba8c019000 comp rx=0 tx=0).stop 2023-10-10T20:20:30.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 shutdown_connections 2023-10-10T20:20:30.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba74066f30 0x7fba740693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba94156210 0x7fba940c1870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba9406dad0 0x7fba940b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 --2- 172.21.15.148:0/2719591901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba940af8f0 0x7fba9413ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 >> 172.21.15.148:0/2719591901 conn(0x7fba940cd450 msgr2=0x7fba940b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:30.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 shutdown_connections 2023-10-10T20:20:30.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.788+0000 7fba98c5d700 1 -- 172.21.15.148:0/2719591901 wait complete. 2023-10-10T20:20:30.793 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:20:30.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:20:30.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.918+0000 7f3ae0a3e700 1 Processor -- start 2023-10-10T20:20:30.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.918+0000 7f3ae0a3e700 1 -- start start 2023-10-10T20:20:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ae0a3e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3adc0b6580 0x7f3adc06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ae0a3e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3adc06a830 0x7f3adc0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ae0a3e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc06ac90 con 0x7f3adc0cce30 2023-10-10T20:20:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ae0a3e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0b4d70 con 0x7f3adc0b6580 2023-10-10T20:20:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ae0a3e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc15b3b0 con 0x7f3adc06a830 2023-10-10T20:20:30.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3adc0b6580 0x7f3adc06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3adb7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3adc0cce30 0x7f3adc15aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52742/0 (socket says 172.21.15.148:52742) 2023-10-10T20:20:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3adb7fe700 1 -- 172.21.15.148:0/3933851442 learned_addr learned my addr 172.21.15.148:0/3933851442 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3adbfff700 1 --2- 172.21.15.148:0/3933851442 >> v2:172.21.15.161:3300/0 conn(0x7f3adc06a830 0x7f3adc0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/3933851442 >> v2:172.21.15.161:3300/0 conn(0x7f3adc06a830 msgr2=0x7f3adc0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 --2- 172.21.15.148:0/3933851442 >> v2:172.21.15.161:3300/0 conn(0x7f3adc06a830 0x7f3adc0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/3933851442 >> v1:172.21.15.148:6789/0 conn(0x7f3adc0cce30 legacy=0x7f3adc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/3933851442 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0cd8d0 con 0x7f3adc0b6580 2023-10-10T20:20:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3adaffd700 1 -- 172.21.15.148:0/3933851442 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163852510 0 0) 0x7f3acc001110 con 0x7f3adc0cce30 2023-10-10T20:20:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.919+0000 7f3ad2ffd700 1 --2- 172.21.15.148:0/3933851442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc06a2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0b4ce0 tx=0x7f3ac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e532796a94bc5585 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3adaffd700 1 -- 172.21.15.148:0/3933851442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400b6b0 con 0x7f3adc0b6580 2023-10-10T20:20:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3adaffd700 1 -- 172.21.15.148:0/3933851442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac400b810 con 0x7f3adc0b6580 2023-10-10T20:20:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3adaffd700 1 -- 172.21.15.148:0/3933851442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400bbb0 con 0x7f3adc0b6580 2023-10-10T20:20:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/3933851442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 msgr2=0x7f3adc06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/3933851442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc06a2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0b4ce0 tx=0x7f3ac40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/3933851442 shutdown_connections 2023-10-10T20:20:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/3933851442 >> v2:172.21.15.161:3300/0 conn(0x7f3adc06a830 0x7f3adc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/3933851442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc06a2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/3933851442 >> 172.21.15.148:0/3933851442 conn(0x7f3adc068cb0 msgr2=0x7f3adc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:30.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/3933851442 shutdown_connections 2023-10-10T20:20:30.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.920+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/3933851442 wait complete. 2023-10-10T20:20:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.921+0000 7f3ae0a3e700 1 Processor -- start 2023-10-10T20:20:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.921+0000 7f3ae0a3e700 1 -- start start 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3adc06a830 0x7f3adc153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3adc0cce30 0x7f3adc16a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14f3f0 con 0x7f3adc0b6580 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc0b1b40 con 0x7f3adc0cce30 2023-10-10T20:20:30.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ae0a3e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc16adc0 con 0x7f3adc06a830 2023-10-10T20:20:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ad2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ad2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47808/0 (socket says 172.21.15.148:47808) 2023-10-10T20:20:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/2598714870 learned_addr learned my addr 172.21.15.148:0/2598714870 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3adb7fe700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3adc06a830 0x7f3adc153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.922+0000 7f3adbfff700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3adc0cce30 0x7f3adc16a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3adc06a830 msgr2=0x7f3adc153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3adc06a830 0x7f3adc153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3adc0cce30 msgr2=0x7f3adc16a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3adc0cce30 0x7f3adc16a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:30.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 -- 172.21.15.148:0/2598714870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009420 con 0x7f3adc0b6580 2023-10-10T20:20:30.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad2ffd700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06b300 tx=0x7f3ac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:30.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400bb90 con 0x7f3adc0b6580 2023-10-10T20:20:30.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4011420 con 0x7f3adc0b6580 2023-10-10T20:20:30.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc16b0a0 con 0x7f3adc0b6580 2023-10-10T20:20:30.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40117c0 con 0x7f3adc0b6580 2023-10-10T20:20:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.923+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc0b17d0 con 0x7f3adc0b6580 2023-10-10T20:20:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.924+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc0b1cf0 con 0x7f3adc0b6580 2023-10-10T20:20:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.924+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ac4011b70 con 0x7f3adc0b6580 2023-10-10T20:20:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.925+0000 7f3ad3fff700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 0x7f3abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:30.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.925+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3ac4085530 con 0x7f3adc0b6580 2023-10-10T20:20:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.925+0000 7f3adb7fe700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 0x7f3abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.925+0000 7f3adb7fe700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 0x7f3abc0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3acc007160 tx=0x7f3acc007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:30.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:30.929+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3adc0b1cf0 con 0x7f3adc0b6580 2023-10-10T20:20:31.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.113+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3adc0b1cb0 con 0x7f3adc0b6580 2023-10-10T20:20:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.118+0000 7f3ad3fff700 1 -- 172.21.15.148:0/2598714870 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ac4085020 con 0x7f3adc0b6580 2023-10-10T20:20:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 msgr2=0x7f3abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 0x7f3abc0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3acc007160 tx=0x7f3acc007f50 comp rx=0 tx=0).stop 2023-10-10T20:20:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 msgr2=0x7f3adc14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3adc06b300 tx=0x7f3ac4017000 comp rx=0 tx=0).stop 2023-10-10T20:20:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 shutdown_connections 2023-10-10T20:20:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3abc066f30 0x7f3abc0693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3adc0cce30 0x7f3adc16a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3adc06a830 0x7f3adc153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 --2- 172.21.15.148:0/2598714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3adc0b6580 0x7f3adc14eeb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 >> 172.21.15.148:0/2598714870 conn(0x7f3adc068cb0 msgr2=0x7f3adc0d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:31.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 shutdown_connections 2023-10-10T20:20:31.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.121+0000 7f3ae0a3e700 1 -- 172.21.15.148:0/2598714870 wait complete. 2023-10-10T20:20:31.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-10T20:20:31.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.249+0000 7f437c252700 1 Processor -- start 2023-10-10T20:20:31.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.249+0000 7f437c252700 1 -- start start 2023-10-10T20:20:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437c252700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4374157870 0x7f437415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437c252700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4374157c40 0x7f4374158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437c252700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437415cbb0 con 0x7f437415a620 2023-10-10T20:20:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437c252700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437415bfa0 con 0x7f4374157c40 2023-10-10T20:20:31.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437c252700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437415c280 con 0x7f4374157870 2023-10-10T20:20:31.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437a7ef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4374157c40 0x7f4374158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437a7ef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4374157c40 0x7f4374158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47818/0 (socket says 172.21.15.148:47818) 2023-10-10T20:20:31.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.250+0000 7f437a7ef700 1 -- 172.21.15.148:0/1655253845 learned_addr learned my addr 172.21.15.148:0/1655253845 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:31.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437a7ef700 1 -- 172.21.15.148:0/1655253845 >> v2:172.21.15.161:3300/0 conn(0x7f4374157870 msgr2=0x7f437415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437a7ef700 1 --2- 172.21.15.148:0/1655253845 >> v2:172.21.15.161:3300/0 conn(0x7f4374157870 0x7f437415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437a7ef700 1 -- 172.21.15.148:0/1655253845 >> v1:172.21.15.161:6789/0 conn(0x7f437415a620 legacy=0x7f437415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437a7ef700 1 -- 172.21.15.148:0/1655253845 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437415c560 con 0x7f4374157c40 2023-10-10T20:20:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437a7ef700 1 --2- 172.21.15.148:0/1655253845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4374157c40 0x7f4374158110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4374069d80 tx=0x7f436c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fa9037d03b67ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f4378fec700 1 -- 172.21.15.148:0/1655253845 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00b820 con 0x7f4374157c40 2023-10-10T20:20:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f4378fec700 1 -- 172.21.15.148:0/1655253845 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c00b980 con 0x7f4374157c40 2023-10-10T20:20:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f4378fec700 1 -- 172.21.15.148:0/1655253845 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00bd20 con 0x7f4374157c40 2023-10-10T20:20:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 -- 172.21.15.148:0/1655253845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4374157c40 msgr2=0x7f4374158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 --2- 172.21.15.148:0/1655253845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4374157c40 0x7f4374158110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4374069d80 tx=0x7f436c009250 comp rx=0 tx=0).stop 2023-10-10T20:20:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 -- 172.21.15.148:0/1655253845 shutdown_connections 2023-10-10T20:20:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 --2- 172.21.15.148:0/1655253845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4374157c40 0x7f4374158110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 --2- 172.21.15.148:0/1655253845 >> v2:172.21.15.161:3300/0 conn(0x7f4374157870 0x7f437415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.251+0000 7f437c252700 1 -- 172.21.15.148:0/1655253845 >> 172.21.15.148:0/1655253845 conn(0x7f4374068ce0 msgr2=0x7f43740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:31.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.252+0000 7f437c252700 1 -- 172.21.15.148:0/1655253845 shutdown_connections 2023-10-10T20:20:31.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.252+0000 7f437c252700 1 -- 172.21.15.148:0/1655253845 wait complete. 2023-10-10T20:20:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 Processor -- start 2023-10-10T20:20:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 -- start start 2023-10-10T20:20:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4374157870 0x7f4374155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4374157c40 0x7f43740c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c9660 con 0x7f437415a620 2023-10-10T20:20:31.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c97f0 con 0x7f4374157c40 2023-10-10T20:20:31.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.253+0000 7f437c252700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43740cd540 con 0x7f4374157870 2023-10-10T20:20:31.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47824/0 (socket says 172.21.15.148:47824) 2023-10-10T20:20:31.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 -- 172.21.15.148:0/713309587 learned_addr learned my addr 172.21.15.148:0/713309587 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:31.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f4379fee700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4374157870 0x7f4374155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f43797ed700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4374157c40 0x7f43740c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 -- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4374157870 msgr2=0x7f4374155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4374157870 0x7f4374155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 -- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4374157c40 msgr2=0x7f43740c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4374157c40 0x7f43740c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 -- 172.21.15.148:0/713309587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c0094f0 con 0x7f437415a620 2023-10-10T20:20:31.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437a7ef700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f437406b530 tx=0x7f436c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c01a670 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c01a7f0 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740c6710 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c01abb0 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.254+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43740c6f50 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.255+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43740c68a0 con 0x7f437415a620 2023-10-10T20:20:31.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.256+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f436c01b440 con 0x7f437415a620 2023-10-10T20:20:31.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.256+0000 7f4369ffb700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 0x7f43580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.256+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f436c085e00 con 0x7f437415a620 2023-10-10T20:20:31.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.256+0000 7f4379fee700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 0x7f43580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.256+0000 7f4379fee700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 0x7f43580693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4370001250 tx=0x7f4370009620 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.260+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f436c053370 con 0x7f437415a620 2023-10-10T20:20:31.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.518+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f437404c3c0 con 0x7f437415a620 2023-10-10T20:20:31.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.786+0000 7f4369ffb700 1 -- 172.21.15.148:0/713309587 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v82) v1 ==== 130+0+0 (secure 0 0 0) 0x7f436c0200c0 con 0x7f437415a620 2023-10-10T20:20:31.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 msgr2=0x7f43580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 0x7f43580693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4370001250 tx=0x7f4370009620 comp rx=0 tx=0).stop 2023-10-10T20:20:31.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 msgr2=0x7f43740c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f437406b530 tx=0x7f436c016000 comp rx=0 tx=0).stop 2023-10-10T20:20:31.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 shutdown_connections 2023-10-10T20:20:31.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4358066f30 0x7f43580693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f437415a620 0x7f43740c6050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4374157870 0x7f4374155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 --2- 172.21.15.148:0/713309587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4374157c40 0x7f43740c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 >> 172.21.15.148:0/713309587 conn(0x7f4374068ce0 msgr2=0x7f43740690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:31.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 shutdown_connections 2023-10-10T20:20:31.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.789+0000 7f437c252700 1 -- 172.21.15.148:0/713309587 wait complete. 2023-10-10T20:20:31.805 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-10T20:20:31.805 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-10T20:20:31.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-10T20:20:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.918+0000 7fec5029a700 1 Processor -- start 2023-10-10T20:20:31.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.918+0000 7fec5029a700 1 -- start start 2023-10-10T20:20:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec48157870 0x7fec4815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec4815a620 0x7fec4815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fec4815d580 0x7fec48158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4815cbb0 con 0x7fec48157870 2023-10-10T20:20:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4815bfa0 con 0x7fec4815d580 2023-10-10T20:20:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec5029a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4815c280 con 0x7fec4815a620 2023-10-10T20:20:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec4e036700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec48157870 0x7fec4815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec4e036700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fec48157870 0x7fec4815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47828/0 (socket says 172.21.15.148:47828) 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.919+0000 7fec4e036700 1 -- 172.21.15.148:0/2777180612 learned_addr learned my addr 172.21.15.148:0/2777180612 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4d835700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec4815a620 0x7fec4815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e837700 1 --2- 172.21.15.148:0/2777180612 >> v2:172.21.15.161:3300/0 conn(0x7fec4815d580 0x7fec48158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 -- 172.21.15.148:0/2777180612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec4815a620 msgr2=0x7fec4815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec4815a620 0x7fec4815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 -- 172.21.15.148:0/2777180612 >> v2:172.21.15.161:3300/0 conn(0x7fec4815d580 msgr2=0x7fec48158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 --2- 172.21.15.148:0/2777180612 >> v2:172.21.15.161:3300/0 conn(0x7fec4815d580 0x7fec48158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 -- 172.21.15.148:0/2777180612 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4815c560 con 0x7fec48157870 2023-10-10T20:20:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4e036700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec48157870 0x7fec4815b6a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fec480b03b0 tx=0x7fec44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7994c6f6470e66e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4d034700 1 -- 172.21.15.148:0/2777180612 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400e070 con 0x7fec48157870 2023-10-10T20:20:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4d034700 1 -- 172.21.15.148:0/2777180612 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4400ba20 con 0x7fec48157870 2023-10-10T20:20:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.920+0000 7fec4d034700 1 -- 172.21.15.148:0/2777180612 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440092e0 con 0x7fec48157870 2023-10-10T20:20:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 -- 172.21.15.148:0/2777180612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec48157870 msgr2=0x7fec4815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec48157870 0x7fec4815b6a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fec480b03b0 tx=0x7fec44009840 comp rx=0 tx=0).stop 2023-10-10T20:20:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 -- 172.21.15.148:0/2777180612 shutdown_connections 2023-10-10T20:20:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 --2- 172.21.15.148:0/2777180612 >> v2:172.21.15.161:3300/0 conn(0x7fec4815d580 0x7fec48158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec48157870 0x7fec4815b6a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 --2- 172.21.15.148:0/2777180612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec4815a620 0x7fec4815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 -- 172.21.15.148:0/2777180612 >> 172.21.15.148:0/2777180612 conn(0x7fec48068ce0 msgr2=0x7fec480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 -- 172.21.15.148:0/2777180612 shutdown_connections 2023-10-10T20:20:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.921+0000 7fec5029a700 1 -- 172.21.15.148:0/2777180612 wait complete. 2023-10-10T20:20:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.922+0000 7fec5029a700 1 Processor -- start 2023-10-10T20:20:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.922+0000 7fec5029a700 1 -- start start 2023-10-10T20:20:31.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.922+0000 7fec5029a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec48157870 0x7fec4814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4815d580 0x7fec480d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4814c020 con 0x7fec4815a620 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec480b0610 con 0x7fec4815d580 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec480b0bd0 con 0x7fec48157870 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47832/0 (socket says 172.21.15.148:47832) 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 -- 172.21.15.148:0/1267933685 learned_addr learned my addr 172.21.15.148:0/1267933685 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:31.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4e837700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4815d580 0x7fec480d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4e036700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec48157870 0x7fec4814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 -- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec48157870 msgr2=0x7fec4814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec48157870 0x7fec4814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 -- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4815d580 msgr2=0x7fec480d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:31.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4815d580 0x7fec480d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 -- 172.21.15.148:0/1267933685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec44009ae0 con 0x7fec4815a620 2023-10-10T20:20:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec4d835700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fec3800eaa0 tx=0x7fec380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800eee0 con 0x7fec4815a620 2023-10-10T20:20:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3800c850 con 0x7fec4815a620 2023-10-10T20:20:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec48155290 con 0x7fec4815a620 2023-10-10T20:20:31.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800cbf0 con 0x7fec4815a620 2023-10-10T20:20:31.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.923+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec480b0860 con 0x7fec4815a620 2023-10-10T20:20:31.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.924+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4804c3c0 con 0x7fec4815a620 2023-10-10T20:20:31.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.925+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec38013470 con 0x7fec4815a620 2023-10-10T20:20:31.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.925+0000 7fec3e7fc700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 0x7fec2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:31.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.925+0000 7fec4e036700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 0x7fec2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:31.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.925+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fec38084430 con 0x7fec4815a620 2023-10-10T20:20:31.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.926+0000 7fec4e036700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 0x7fec2c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec44009880 tx=0x7fec44018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:31.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:31.930+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec380519a0 con 0x7fec4815a620 2023-10-10T20:20:32.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.186+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7fec480b0d10 con 0x7fec4815a620 2023-10-10T20:20:32.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.788+0000 7fec3e7fc700 1 -- 172.21.15.148:0/1267933685 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v83) v1 ==== 145+0+0 (secure 0 0 0) 0x7fec38054fc0 con 0x7fec4815a620 2023-10-10T20:20:32.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 msgr2=0x7fec2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:32.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 0x7fec2c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec44009880 tx=0x7fec44018000 comp rx=0 tx=0).stop 2023-10-10T20:20:32.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 msgr2=0x7fec480d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:32.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fec3800eaa0 tx=0x7fec380091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:32.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 shutdown_connections 2023-10-10T20:20:32.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fec2c066f30 0x7fec2c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4815d580 0x7fec480d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec48157870 0x7fec4814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 --2- 172.21.15.148:0/1267933685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec4815a620 0x7fec480d0270 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 >> 172.21.15.148:0/1267933685 conn(0x7fec48068ce0 msgr2=0x7fec48159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:32.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 shutdown_connections 2023-10-10T20:20:32.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:32.790+0000 7fec5029a700 1 -- 172.21.15.148:0/1267933685 wait complete. 2023-10-10T20:20:32.805 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-10T20:20:32.934 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_objectstore":"memstore"} 2023-10-10T20:20:32.934 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-10T20:20:32.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:20:33.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.050+0000 7ff9f2c17700 1 Processor -- start 2023-10-10T20:20:33.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.050+0000 7ff9f2c17700 1 -- start start 2023-10-10T20:20:33.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f2c17700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9ec0678b0 0x7ff9ec15db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f2c17700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0cae70 con 0x7ff9ec0c9390 2023-10-10T20:20:33.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f2c17700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0cb150 con 0x7ff9ec0cbf10 2023-10-10T20:20:33.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f2c17700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0cb430 con 0x7ff9ec0678b0 2023-10-10T20:20:33.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f09b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9ec0678b0 0x7ff9ec15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f11b4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff9ec0c9390 0x7ff9ec15a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33236/0 (socket says 172.21.15.148:33236) 2023-10-10T20:20:33.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f11b4700 1 -- 172.21.15.148:0/864608543 learned_addr learned my addr 172.21.15.148:0/864608543 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:33.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f09b3700 1 -- 172.21.15.148:0/864608543 >> v1:172.21.15.161:6789/0 conn(0x7ff9ec0cbf10 legacy=0x7ff9ec15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f09b3700 1 -- 172.21.15.148:0/864608543 >> v1:172.21.15.148:6789/0 conn(0x7ff9ec0c9390 legacy=0x7ff9ec15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9eb7fe700 1 -- 172.21.15.148:0/864608543 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858766247 0 0) 0x7ff9ec0cae70 con 0x7ff9ec0c9390 2023-10-10T20:20:33.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.051+0000 7ff9f09b3700 1 -- 172.21.15.148:0/864608543 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0cb710 con 0x7ff9ec0678b0 2023-10-10T20:20:33.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f09b3700 1 --2- 172.21.15.148:0/864608543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec15db80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00098e0 tx=0x7ff9e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7423b1bb56461213 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9eb7fe700 1 -- 172.21.15.148:0/864608543 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0009e80 con 0x7ff9ec0678b0 2023-10-10T20:20:33.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9eb7fe700 1 -- 172.21.15.148:0/864608543 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000b860 con 0x7ff9ec0678b0 2023-10-10T20:20:33.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9eb7fe700 1 -- 172.21.15.148:0/864608543 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000bbe0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 -- 172.21.15.148:0/864608543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 msgr2=0x7ff9ec15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/864608543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec15db80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00098e0 tx=0x7ff9e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 -- 172.21.15.148:0/864608543 shutdown_connections 2023-10-10T20:20:33.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/864608543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec15db80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 -- 172.21.15.148:0/864608543 >> 172.21.15.148:0/864608543 conn(0x7ff9ec069870 msgr2=0x7ff9ec069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:33.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 -- 172.21.15.148:0/864608543 shutdown_connections 2023-10-10T20:20:33.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.052+0000 7ff9f2c17700 1 -- 172.21.15.148:0/864608543 wait complete. 2023-10-10T20:20:33.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.053+0000 7ff9f2c17700 1 Processor -- start 2023-10-10T20:20:33.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 -- start start 2023-10-10T20:20:33.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9ec0c9390 0x7ff9ec0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 0x7ff9ec0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0cd0f0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0cdef0 con 0x7ff9ec0cbf10 2023-10-10T20:20:33.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f2c17700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec150cf0 con 0x7ff9ec0c9390 2023-10-10T20:20:33.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f09b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f09b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50354/0 (socket says 172.21.15.148:50354) 2023-10-10T20:20:33.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.054+0000 7ff9f09b3700 1 -- 172.21.15.148:0/3933194152 learned_addr learned my addr 172.21.15.148:0/3933194152 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:33.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f11b4700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 0x7ff9ec0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9ebfff700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9ec0c9390 0x7ff9ec0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 -- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9ec0c9390 msgr2=0x7ff9ec0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9ec0c9390 0x7ff9ec0b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 -- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 msgr2=0x7ff9ec0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 0x7ff9ec0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 -- 172.21.15.148:0/3933194152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009420 con 0x7ff9ec0678b0 2023-10-10T20:20:33.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f11b4700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 0x7ff9ec0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:33.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f09b3700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec06a380 tx=0x7ff9e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e001f070 con 0x7ff9ec0678b0 2023-10-10T20:20:33.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000b9d0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec0cd2e0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0025670 con 0x7ff9ec0678b0 2023-10-10T20:20:33.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.055+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0cd670 con 0x7ff9ec0678b0 2023-10-10T20:20:33.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.056+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec0ce1d0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.056+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9e0023070 con 0x7ff9ec0678b0 2023-10-10T20:20:33.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.057+0000 7ff9e8ff9700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 0x7ff9d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.057+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff9e00858c0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.057+0000 7ff9ebfff700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 0x7ff9d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.057+0000 7ff9ebfff700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 0x7ff9d00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0c89d0 tx=0x7ff9d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.061+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9e0052e30 con 0x7ff9ec0678b0 2023-10-10T20:20:33.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.245+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff9ec04c3c0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.251+0000 7ff9e8ff9700 1 -- 172.21.15.148:0/3933194152 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9e002a0c0 con 0x7ff9ec0678b0 2023-10-10T20:20:33.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 msgr2=0x7ff9d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 0x7ff9d00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0c89d0 tx=0x7ff9d8009040 comp rx=0 tx=0).stop 2023-10-10T20:20:33.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 msgr2=0x7ff9ec0aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec06a380 tx=0x7ff9e0017000 comp rx=0 tx=0).stop 2023-10-10T20:20:33.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 shutdown_connections 2023-10-10T20:20:33.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff9d0066f30 0x7ff9d00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9ec0cbf10 0x7ff9ec0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9ec0678b0 0x7ff9ec0aebc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 --2- 172.21.15.148:0/3933194152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9ec0c9390 0x7ff9ec0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 >> 172.21.15.148:0/3933194152 conn(0x7ff9ec069870 msgr2=0x7ff9ec0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:33.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 shutdown_connections 2023-10-10T20:20:33.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.253+0000 7ff9f2c17700 1 -- 172.21.15.148:0/3933194152 wait complete. 2023-10-10T20:20:33.267 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:20:33.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:20:33.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:20:33.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:20:33.350 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:20:33.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:33.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.462+0000 7fca7e049700 1 Processor -- start 2023-10-10T20:20:33.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.462+0000 7fca7e049700 1 -- start start 2023-10-10T20:20:33.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.463+0000 7fca7e049700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.463+0000 7fca7e049700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 0x7fca7806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.463+0000 7fca7e049700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7806c010 con 0x7fca78158e50 2023-10-10T20:20:33.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.463+0000 7fca7e049700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7806bd10 con 0x7fca781523e0 2023-10-10T20:20:33.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.463+0000 7fca7e049700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7815c060 con 0x7fca78152010 2023-10-10T20:20:33.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca76ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 0x7fca7806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:39570/0 (socket says 172.21.15.148:39570) 2023-10-10T20:20:33.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca77fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fca78158e50 0x7fca780cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:35990/0 (socket says 172.21.15.148:35990) 2023-10-10T20:20:33.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 -- 172.21.15.148:0/3245730720 learned_addr learned my addr 172.21.15.148:0/3245730720 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:33.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 -- 172.21.15.148:0/3245730720 >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 msgr2=0x7fca7806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:33.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 --2- 172.21.15.148:0/3245730720 >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 0x7fca7806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 -- 172.21.15.148:0/3245730720 >> v1:172.21.15.161:6789/0 conn(0x7fca78158e50 legacy=0x7fca780cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 -- 172.21.15.148:0/3245730720 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7815c340 con 0x7fca78152010 2023-10-10T20:20:33.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca76ffd700 1 --2- 172.21.15.148:0/3245730720 >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 0x7fca7806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:33.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca777fe700 1 --2- 172.21.15.148:0/3245730720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f20 tx=0x7fca68009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca19ea9dae929c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.464+0000 7fca767fc700 1 -- 172.21.15.148:0/3245730720 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6800e070 con 0x7fca78152010 2023-10-10T20:20:33.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca767fc700 1 -- 172.21.15.148:0/3245730720 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6800b9d0 con 0x7fca78152010 2023-10-10T20:20:33.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca767fc700 1 -- 172.21.15.148:0/3245730720 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680092e0 con 0x7fca78152010 2023-10-10T20:20:33.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 -- 172.21.15.148:0/3245730720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 msgr2=0x7fca7815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 --2- 172.21.15.148:0/3245730720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f20 tx=0x7fca68009840 comp rx=0 tx=0).stop 2023-10-10T20:20:33.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 -- 172.21.15.148:0/3245730720 shutdown_connections 2023-10-10T20:20:33.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 --2- 172.21.15.148:0/3245730720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca78152010 0x7fca7815b810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 --2- 172.21.15.148:0/3245730720 >> v2:172.21.15.161:3300/0 conn(0x7fca781523e0 0x7fca7806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 -- 172.21.15.148:0/3245730720 >> 172.21.15.148:0/3245730720 conn(0x7fca78069870 msgr2=0x7fca78069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 -- 172.21.15.148:0/3245730720 shutdown_connections 2023-10-10T20:20:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.465+0000 7fca7e049700 1 -- 172.21.15.148:0/3245730720 wait complete. 2023-10-10T20:20:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.466+0000 7fca7e049700 1 Processor -- start 2023-10-10T20:20:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.466+0000 7fca7e049700 1 -- start start 2023-10-10T20:20:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca781523e0 0x7fca780c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78158e50 0x7fca780c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780c6670 con 0x7fca78152010 2023-10-10T20:20:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780b2e00 con 0x7fca78158e50 2023-10-10T20:20:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca7e049700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca780b21f0 con 0x7fca781523e0 2023-10-10T20:20:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50362/0 (socket says 172.21.15.148:50362) 2023-10-10T20:20:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.467+0000 7fca777fe700 1 -- 172.21.15.148:0/284023265 learned_addr learned my addr 172.21.15.148:0/284023265 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca76ffd700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca781523e0 0x7fca780c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 -- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca781523e0 msgr2=0x7fca780c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca781523e0 0x7fca780c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 -- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78158e50 msgr2=0x7fca780c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78158e50 0x7fca780c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 -- 172.21.15.148:0/284023265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009420 con 0x7fca78152010 2023-10-10T20:20:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca777fe700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca68003440 tx=0x7fca68018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68008a80 con 0x7fca78152010 2023-10-10T20:20:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68008be0 con 0x7fca78152010 2023-10-10T20:20:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68025460 con 0x7fca78152010 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca68009ae0 con 0x7fca78152010 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.468+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca780b2830 con 0x7fca78152010 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.469+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7804c3c0 con 0x7fca78152010 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.469+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca68003040 con 0x7fca78152010 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.470+0000 7fca5f7fe700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 0x7fca640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.470+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fca68085760 con 0x7fca78152010 2023-10-10T20:20:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.470+0000 7fca76ffd700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 0x7fca640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.470+0000 7fca76ffd700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 0x7fca640693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca780b4a90 tx=0x7fca6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.474+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca68052d50 con 0x7fca78152010 2023-10-10T20:20:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.699+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca7806d1b0 con 0x7fca78152010 2023-10-10T20:20:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.699+0000 7fca5f7fe700 1 -- 172.21.15.148:0/284023265 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fca68020df0 con 0x7fca78152010 2023-10-10T20:20:33.702 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:33.702 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2400317094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2400317094},{"type":"v1","addr":"172.21.15.161:6835","nonce":2400317094}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2213329872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213329872},{"type":"v1","addr":"172.21.15.148:6835","nonce":2213329872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4932,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3642257653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3642257653},{"type":"v1","addr":"172.21.15.148:6837","nonce":3642257653}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4941,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2262001181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2262001181},{"type":"v1","addr":"172.21.15.161:6834","nonce":2262001181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-10-10T20:20:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 msgr2=0x7fca640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 0x7fca640693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca780b4a90 tx=0x7fca6c008040 comp rx=0 tx=0).stop 2023-10-10T20:20:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 msgr2=0x7fca780c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca68003440 tx=0x7fca68018000 comp rx=0 tx=0).stop 2023-10-10T20:20:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 shutdown_connections 2023-10-10T20:20:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca64066f30 0x7fca640693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca78158e50 0x7fca780c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca78152010 0x7fca780c5bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 --2- 172.21.15.148:0/284023265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca781523e0 0x7fca780c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.701+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 >> 172.21.15.148:0/284023265 conn(0x7fca78069870 msgr2=0x7fca7806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.702+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 shutdown_connections 2023-10-10T20:20:33.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.702+0000 7fca7e049700 1 -- 172.21.15.148:0/284023265 wait complete. 2023-10-10T20:20:33.706 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2023-10-10T20:20:33.715 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:20:33.716 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:20:33.716 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:20:33.716 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:33.717 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:20:33.717 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:20:33.717 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:20:33.717 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:33.718 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:20:33.718 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:20:33.718 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:20:33.719 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:33.719 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:20:33.719 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:20:33.719 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:20:33.720 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:20:33.725 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:20:33.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:20:33.726 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:20:33.726 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:20:33.729 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:20:33.729 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:20:33.730 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:20:33.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:20:33.767 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:20:33.768 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:33.768 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:20:33.769 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:33.771 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:20:33.772 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:33.772 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:20:33.773 DEBUG:teuthology.parallel:result is None 2023-10-10T20:20:33.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:20:33.841 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:20:33.842 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:20:33.842 INFO:tasks.ceph.mds.c.smithi148.stderr:2023-10-10T20:20:33.840+0000 7f0d135ab700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-10T20:20:33.851 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:20:33.852 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:20:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.894+0000 7f3c14d46700 1 Processor -- start 2023-10-10T20:20:33.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.894+0000 7f3c14d46700 1 -- start start 2023-10-10T20:20:33.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c10157870 0x7f3c1015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c1015a620 0x7f3c1015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3c1015d580 0x7f3c10158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1015cbb0 con 0x7f3c1015a620 2023-10-10T20:20:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1015bfa0 con 0x7f3c1015d580 2023-10-10T20:20:33.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c14d46700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1015c280 con 0x7f3c10157870 2023-10-10T20:20:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c07fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c1015a620 0x7f3c1015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c07fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c1015a620 0x7f3c1015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50382/0 (socket says 172.21.15.148:50382) 2023-10-10T20:20:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c07fff700 1 -- 172.21.15.148:0/996935673 learned_addr learned my addr 172.21.15.148:0/996935673 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:33.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.895+0000 7f3c0e59c700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c10157870 0x7f3c1015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c0ed9d700 1 --2- 172.21.15.148:0/996935673 >> v2:172.21.15.161:3300/0 conn(0x7f3c1015d580 0x7f3c10158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 -- 172.21.15.148:0/996935673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c10157870 msgr2=0x7f3c1015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c10157870 0x7f3c1015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 -- 172.21.15.148:0/996935673 >> v2:172.21.15.161:3300/0 conn(0x7f3c1015d580 msgr2=0x7f3c10158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:33.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 --2- 172.21.15.148:0/996935673 >> v2:172.21.15.161:3300/0 conn(0x7f3c1015d580 0x7f3c10158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 -- 172.21.15.148:0/996935673 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1015c560 con 0x7f3c1015a620 2023-10-10T20:20:33.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c07fff700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c1015d040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3c100b03b0 tx=0x7f3bf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b656e94d30aaf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c0dd9b700 1 -- 172.21.15.148:0/996935673 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800b860 con 0x7f3c1015a620 2023-10-10T20:20:33.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c0dd9b700 1 -- 172.21.15.148:0/996935673 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf800b9c0 con 0x7f3c1015a620 2023-10-10T20:20:33.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c0dd9b700 1 -- 172.21.15.148:0/996935673 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800bd60 con 0x7f3c1015a620 2023-10-10T20:20:33.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 -- 172.21.15.148:0/996935673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 msgr2=0x7f3c1015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c1015d040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3c100b03b0 tx=0x7f3bf8009250 comp rx=0 tx=0).stop 2023-10-10T20:20:33.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 -- 172.21.15.148:0/996935673 shutdown_connections 2023-10-10T20:20:33.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 --2- 172.21.15.148:0/996935673 >> v2:172.21.15.161:3300/0 conn(0x7f3c1015d580 0x7f3c10158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c10157870 0x7f3c1015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 --2- 172.21.15.148:0/996935673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c1015d040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 -- 172.21.15.148:0/996935673 >> 172.21.15.148:0/996935673 conn(0x7f3c10068ce0 msgr2=0x7f3c100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:33.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 -- 172.21.15.148:0/996935673 shutdown_connections 2023-10-10T20:20:33.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.896+0000 7f3c14d46700 1 -- 172.21.15.148:0/996935673 wait complete. 2023-10-10T20:20:33.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.897+0000 7f3c14d46700 1 Processor -- start 2023-10-10T20:20:33.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 -- start start 2023-10-10T20:20:33.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c10157870 0x7f3c101519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c1015d580 0x7f3c100b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100b2600 con 0x7f3c1015a620 2023-10-10T20:20:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100b1380 con 0x7f3c10157870 2023-10-10T20:20:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c14d46700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100b1660 con 0x7f3c1015d580 2023-10-10T20:20:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50392/0 (socket says 172.21.15.148:50392) 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c07fff700 1 -- 172.21.15.148:0/2443989558 learned_addr learned my addr 172.21.15.148:0/2443989558 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c0e59c700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c10157870 0x7f3c101519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.898+0000 7f3c0ed9d700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c1015d580 0x7f3c100b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 -- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c1015d580 msgr2=0x7f3c100b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c1015d580 0x7f3c100b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 -- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c10157870 msgr2=0x7f3c101519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:33.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c10157870 0x7f3c101519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:33.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 -- 172.21.15.148:0/2443989558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80094f0 con 0x7f3c1015a620 2023-10-10T20:20:33.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c07fff700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006b530 tx=0x7f3bf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8011dc0 con 0x7f3c1015a620 2023-10-10T20:20:33.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf8024440 con 0x7f3c1015a620 2023-10-10T20:20:33.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c100b1940 con 0x7f3c1015a620 2023-10-10T20:20:33.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8024800 con 0x7f3c1015a620 2023-10-10T20:20:33.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.899+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10151f00 con 0x7f3c1015a620 2023-10-10T20:20:33.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.900+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c100b2770 con 0x7f3c1015a620 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.900+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bf801a420 con 0x7f3c1015a620 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.900+0000 7f3c067fc700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 0x7f3bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.901+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3bf8086140 con 0x7f3c1015a620 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.901+0000 7f3c0e59c700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 0x7f3bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.901+0000 7f3c0e59c700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 0x7f3bf00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006fd0 tx=0x7f3c00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:33.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:33.905+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bf80536b0 con 0x7f3c1015a620 2023-10-10T20:20:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:34.102+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c1004c3c0 con 0x7f3c1015a620 2023-10-10T20:20:35.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.261+0000 7f3c067fc700 1 -- 172.21.15.148:0/2443989558 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3bf8056cd0 con 0x7f3c1015a620 2023-10-10T20:20:35.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 msgr2=0x7f3bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 0x7f3bf00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c00006fd0 tx=0x7f3c00008040 comp rx=0 tx=0).stop 2023-10-10T20:20:35.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 msgr2=0x7f3c101553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006b530 tx=0x7f3bf8017000 comp rx=0 tx=0).stop 2023-10-10T20:20:35.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 shutdown_connections 2023-10-10T20:20:35.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bf0066f30 0x7f3bf00693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c1015d580 0x7f3c100b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c10157870 0x7f3c101519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 --2- 172.21.15.148:0/2443989558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c1015a620 0x7f3c101553d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 >> 172.21.15.148:0/2443989558 conn(0x7f3c10068ce0 msgr2=0x7f3c1014c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:35.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 shutdown_connections 2023-10-10T20:20:35.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.264+0000 7f3c14d46700 1 -- 172.21.15.148:0/2443989558 wait complete. 2023-10-10T20:20:35.269 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:20:35.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:20:35.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.395+0000 7f25d6231700 1 Processor -- start 2023-10-10T20:20:35.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.395+0000 7f25d6231700 1 -- start start 2023-10-10T20:20:35.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25d6231700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25d6231700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25d015a620 0x7f25d015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25d6231700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d015cbb0 con 0x7f25d015d580 2023-10-10T20:20:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25d6231700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d015bfa0 con 0x7f25d015a620 2023-10-10T20:20:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25d6231700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d015c280 con 0x7f25d0157870 2023-10-10T20:20:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25ceffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25d015a620 0x7f25d015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25cffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f25d015d580 0x7f25d0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33274/0 (socket says 172.21.15.148:33274) 2023-10-10T20:20:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25cffff700 1 -- 172.21.15.148:0/1372938128 learned_addr learned my addr 172.21.15.148:0/1372938128 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.396+0000 7f25cf7fe700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ceffd700 1 -- 172.21.15.148:0/1372938128 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 msgr2=0x7f25d015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ceffd700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ceffd700 1 -- 172.21.15.148:0/1372938128 >> v1:172.21.15.148:6789/0 conn(0x7f25d015d580 legacy=0x7f25d0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ceffd700 1 -- 172.21.15.148:0/1372938128 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d015c560 con 0x7f25d015a620 2023-10-10T20:20:35.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ce7fc700 1 -- 172.21.15.148:0/1372938128 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915842756 0 0) 0x7f25d015cbb0 con 0x7f25d015d580 2023-10-10T20:20:35.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ceffd700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015a620 0x7f25d015d040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f25c40098e0 tx=0x7f25c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d988f8eaf8fba48e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ce7fc700 1 -- 172.21.15.148:0/1372938128 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4009e80 con 0x7f25d015a620 2023-10-10T20:20:35.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ce7fc700 1 -- 172.21.15.148:0/1372938128 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400b860 con 0x7f25d015a620 2023-10-10T20:20:35.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25ce7fc700 1 -- 172.21.15.148:0/1372938128 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400bbe0 con 0x7f25d015a620 2023-10-10T20:20:35.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 -- 172.21.15.148:0/1372938128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015a620 msgr2=0x7f25d015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015a620 0x7f25d015d040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f25c40098e0 tx=0x7f25c40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 -- 172.21.15.148:0/1372938128 shutdown_connections 2023-10-10T20:20:35.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 --2- 172.21.15.148:0/1372938128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015a620 0x7f25d015d040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 -- 172.21.15.148:0/1372938128 >> 172.21.15.148:0/1372938128 conn(0x7f25d0068ce0 msgr2=0x7f25d00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 -- 172.21.15.148:0/1372938128 shutdown_connections 2023-10-10T20:20:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.397+0000 7f25d6231700 1 -- 172.21.15.148:0/1372938128 wait complete. 2023-10-10T20:20:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 Processor -- start 2023-10-10T20:20:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 -- start start 2023-10-10T20:20:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25d015a620 0x7f25d00d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00d1370 con 0x7f25d015d580 2023-10-10T20:20:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00b07f0 con 0x7f25d015a620 2023-10-10T20:20:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.399+0000 7f25d6231700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00d0ad0 con 0x7f25d0157870 2023-10-10T20:20:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50412/0 (socket says 172.21.15.148:50412) 2023-10-10T20:20:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 -- 172.21.15.148:0/2635404687 learned_addr learned my addr 172.21.15.148:0/2635404687 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cf7fe700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25ceffd700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25d015a620 0x7f25d00d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 -- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 msgr2=0x7f25d00b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d00b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 -- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25d015a620 msgr2=0x7f25d00d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25d015a620 0x7f25d00d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 -- 172.21.15.148:0/2635404687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c4009420 con 0x7f25d015d580 2023-10-10T20:20:35.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cf7fe700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d00b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:35.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25cffff700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25c800c500 tx=0x7f25c8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8005b30 con 0x7f25d015d580 2023-10-10T20:20:35.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8005c90 con 0x7f25d015d580 2023-10-10T20:20:35.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d00d0db0 con 0x7f25d015d580 2023-10-10T20:20:35.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8007320 con 0x7f25d015d580 2023-10-10T20:20:35.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.400+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d00d05e0 con 0x7f25d015d580 2023-10-10T20:20:35.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.401+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d004c3c0 con 0x7f25d015d580 2023-10-10T20:20:35.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.402+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25c801b3b0 con 0x7f25d015d580 2023-10-10T20:20:35.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.402+0000 7f25b77fe700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.402+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f25c8083630 con 0x7f25d015d580 2023-10-10T20:20:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.402+0000 7f25cf7fe700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.402+0000 7f25cf7fe700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 0x7f25bc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f25c00075e0 tx=0x7f25c00097e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.406+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25c8050bf0 con 0x7f25d015d580 2023-10-10T20:20:35.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.662+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f25d015c840 con 0x7f25d015d580 2023-10-10T20:20:35.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.662+0000 7f25b77fe700 1 -- 172.21.15.148:0/2635404687 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f25c8054210 con 0x7f25d015d580 2023-10-10T20:20:35.665 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:35.666 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:35.666 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:20:35.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.666 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:20:35.666 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:20:35.667 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.667 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.667 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:35.667 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:35.667 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.669 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:20:35.669 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:20:35.669 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.669 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.670 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:35.670 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:35.670 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.670 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:20:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:20:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.671 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.673 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:20:35.673 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:20:35.673 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.673 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.673 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:35.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:35.675 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.676 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:20:35.676 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:20:35.677 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.677 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.677 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.677 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:20:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:20:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.679 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.679 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.679 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.679 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:20:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:20:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.682 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:20:35.682 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:20:35.682 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.683 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.686 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.686 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.686 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.687 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:20:35.687 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:20:35.687 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.687 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.688 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.688 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.688 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.689 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:20:35.689 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:20:35.689 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.689 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.690 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.690 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.691 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:20:35.691 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:20:35.692 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.692 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.692 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.692 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.693 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:20:35.694 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:20:35.694 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.694 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:20:35.694 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:20:35.694 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.695 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:20:35.695 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:20:35.696 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.696 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.696 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:20:35.696 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:20:35.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.697 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:20:35.697 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:20:35.697 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.698 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.698 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:35.698 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:35.698 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:35.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.699 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:20:35.699 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:20:35.699 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.700 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.700 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:35.700 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:35.700 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:35.701 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.701 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:20:35.702 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:20:35.702 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.703 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:20:35.703 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:20:35.704 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:20:35.704 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:20:35.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.705 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:20:35.705 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:20:35.705 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.705 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:35.706 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:20:35.706 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:20:35.706 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.706 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.707 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:20:35.707 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:20:35.707 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.707 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:35.708 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:20:35.708 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:35.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:35.709 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:20:35.710 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:20:35.710 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:20:35.710 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:20:35.710 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:20:35.710 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:20:35.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:35.711 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:35.711 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:35.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 msgr2=0x7f25bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 0x7f25bc0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f25c00075e0 tx=0x7f25c00097e0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 msgr2=0x7f25d016aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f25c800c500 tx=0x7f25c8007f60 comp rx=0 tx=0).stop 2023-10-10T20:20:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 shutdown_connections 2023-10-10T20:20:35.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25bc066f30 0x7f25bc0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d015d580 0x7f25d016aa60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25d0157870 0x7f25d00b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 --2- 172.21.15.148:0/2635404687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25d015a620 0x7f25d00d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 >> 172.21.15.148:0/2635404687 conn(0x7f25d0068ce0 msgr2=0x7f25d014c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:35.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 shutdown_connections 2023-10-10T20:20:35.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.664+0000 7f25d6231700 1 -- 172.21.15.148:0/2635404687 wait complete. 2023-10-10T20:20:35.716 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:35.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.831+0000 7fdccd93f700 1 Processor -- start 2023-10-10T20:20:35.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.831+0000 7fdccd93f700 1 -- start start 2023-10-10T20:20:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 0x7fdcc806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc806f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcc806f670 0x7fdcc806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc806e6b0 con 0x7fdcc806f670 2023-10-10T20:20:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc806daa0 con 0x7fdcc81579a0 2023-10-10T20:20:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdccd93f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc806dd80 con 0x7fdcc815b620 2023-10-10T20:20:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdcc77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcc806f670 0x7fdcc806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdcc77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcc806f670 0x7fdcc806a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50424/0 (socket says 172.21.15.148:50424) 2023-10-10T20:20:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdcc77fe700 1 -- 172.21.15.148:0/574701615 learned_addr learned my addr 172.21.15.148:0/574701615 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdcc6ffd700 1 --2- 172.21.15.148:0/574701615 >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 0x7fdcc806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.832+0000 7fdcbe7fc700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 -- 172.21.15.148:0/574701615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 msgr2=0x7fdcc806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:35.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc806f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 -- 172.21.15.148:0/574701615 >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 msgr2=0x7fdcc806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 --2- 172.21.15.148:0/574701615 >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 0x7fdcc806ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 -- 172.21.15.148:0/574701615 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc806e060 con 0x7fdcc806f670 2023-10-10T20:20:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc6ffd700 1 --2- 172.21.15.148:0/574701615 >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 0x7fdcc806ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:35.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcbe7fc700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc77fe700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc806f670 0x7fdcc806a7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdcc806fce0 tx=0x7fdcc0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744ee6027e8383c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc67fc700 1 -- 172.21.15.148:0/574701615 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000e070 con 0x7fdcc806f670 2023-10-10T20:20:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc67fc700 1 -- 172.21.15.148:0/574701615 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc000ba50 con 0x7fdcc806f670 2023-10-10T20:20:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdcc67fc700 1 -- 172.21.15.148:0/574701615 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00092e0 con 0x7fdcc806f670 2023-10-10T20:20:35.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 -- 172.21.15.148:0/574701615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc806f670 msgr2=0x7fdcc806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc806f670 0x7fdcc806a7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdcc806fce0 tx=0x7fdcc0009840 comp rx=0 tx=0).stop 2023-10-10T20:20:35.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 -- 172.21.15.148:0/574701615 shutdown_connections 2023-10-10T20:20:35.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc806f670 0x7fdcc806a7d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 --2- 172.21.15.148:0/574701615 >> v2:172.21.15.161:3300/0 conn(0x7fdcc81579a0 0x7fdcc806ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 --2- 172.21.15.148:0/574701615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc806f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.833+0000 7fdccd93f700 1 -- 172.21.15.148:0/574701615 >> 172.21.15.148:0/574701615 conn(0x7fdcc80ccfa0 msgr2=0x7fdcc80cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.834+0000 7fdccd93f700 1 -- 172.21.15.148:0/574701615 shutdown_connections 2023-10-10T20:20:35.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.834+0000 7fdccd93f700 1 -- 172.21.15.148:0/574701615 wait complete. 2023-10-10T20:20:35.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 Processor -- start 2023-10-10T20:20:35.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 -- start start 2023-10-10T20:20:35.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc806f670 0x7fdcc80b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc80afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80c84d0 con 0x7fdcc81579a0 2023-10-10T20:20:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80c75e0 con 0x7fdcc806f670 2023-10-10T20:20:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.835+0000 7fdccd93f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80c78c0 con 0x7fdcc815b620 2023-10-10T20:20:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50438/0 (socket says 172.21.15.148:50438) 2023-10-10T20:20:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 -- 172.21.15.148:0/932972142 learned_addr learned my addr 172.21.15.148:0/932972142 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 -- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 msgr2=0x7fdcc80afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc80afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 -- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc806f670 msgr2=0x7fdcc80b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcc6ffd700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc806f670 0x7fdcc80b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc806f670 0x7fdcc80b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 -- 172.21.15.148:0/932972142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009ae0 con 0x7fdcc81579a0 2023-10-10T20:20:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbe7fc700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000e850 tx=0x7fdcb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0013410 con 0x7fdcc81579a0 2023-10-10T20:20:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb0013570 con 0x7fdcc81579a0 2023-10-10T20:20:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc80c7ba0 con 0x7fdcc81579a0 2023-10-10T20:20:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0013910 con 0x7fdcc81579a0 2023-10-10T20:20:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.836+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc80c8160 con 0x7fdcc81579a0 2023-10-10T20:20:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.837+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc806c4b0 con 0x7fdcc81579a0 2023-10-10T20:20:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.837+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcb0013d00 con 0x7fdcc81579a0 2023-10-10T20:20:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.838+0000 7fdcbf7fe700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 0x7fdca80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.838+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdcb0083f30 con 0x7fdcc81579a0 2023-10-10T20:20:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.838+0000 7fdcc6ffd700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 0x7fdca80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.838+0000 7fdcc6ffd700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 0x7fdca80693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006f00 tx=0x7fdcb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:35.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:35.842+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcb00514f0 con 0x7fdcc81579a0 2023-10-10T20:20:36.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.026+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcc804bb80 con 0x7fdcc81579a0 2023-10-10T20:20:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.026+0000 7fdcbf7fe700 1 -- 172.21.15.148:0/932972142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12182 (secure 0 0 0) 0x7fdcb0054b10 con 0x7fdcc81579a0 2023-10-10T20:20:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:36.029 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":41,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:35.259611+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 msgr2=0x7fdca80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 0x7fdca80693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006f00 tx=0x7fdcb8008040 comp rx=0 tx=0).stop 2023-10-10T20:20:36.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 msgr2=0x7fdcc80c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000e850 tx=0x7fdcb0008040 comp rx=0 tx=0).stop 2023-10-10T20:20:36.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 shutdown_connections 2023-10-10T20:20:36.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdca8066f30 0x7fdca80693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdcc815b620 0x7fdcc80afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcc806f670 0x7fdcc80b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 --2- 172.21.15.148:0/932972142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcc81579a0 0x7fdcc80c88e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 >> 172.21.15.148:0/932972142 conn(0x7fdcc80ccfa0 msgr2=0x7fdcc80c1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 shutdown_connections 2023-10-10T20:20:36.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.028+0000 7fdccd93f700 1 -- 172.21.15.148:0/932972142 wait complete. 2023-10-10T20:20:36.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:20:36.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:20:36.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.158+0000 7f432611c700 1 Processor -- start 2023-10-10T20:20:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.158+0000 7f432611c700 1 -- start start 2023-10-10T20:20:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f432611c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4320159ec0 0x7f43200af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f432611c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200b0790 con 0x7f432015a290 2023-10-10T20:20:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f432611c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200ce1f0 con 0x7f432006acb0 2023-10-10T20:20:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f432611c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43200b0b20 con 0x7f4320159ec0 2023-10-10T20:20:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f431effd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f432015a290 0x7f432006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33280/0 (socket says 172.21.15.148:33280) 2023-10-10T20:20:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.159+0000 7f431effd700 1 -- 172.21.15.148:0/2300411816 learned_addr learned my addr 172.21.15.148:0/2300411816 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431f7fe700 1 --2- 172.21.15.148:0/2300411816 >> v2:172.21.15.161:3300/0 conn(0x7f4320159ec0 0x7f43200af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370343396 0 0) 0x7f43200b0790 con 0x7f432015a290 2023-10-10T20:20:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4308003570 con 0x7f432015a290 2023-10-10T20:20:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467370764 0 0) 0x7f4308003570 con 0x7f432015a290 2023-10-10T20:20:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 >> v2:172.21.15.161:3300/0 conn(0x7f4320159ec0 msgr2=0x7f43200af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 --2- 172.21.15.148:0/2300411816 >> v2:172.21.15.161:3300/0 conn(0x7f4320159ec0 0x7f43200af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 >> v1:172.21.15.161:6789/0 conn(0x7f432006acb0 legacy=0x7f43200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200b04b0 con 0x7f432015a290 2023-10-10T20:20:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f4314001470 con 0x7f432015a290 2023-10-10T20:20:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 >> v1:172.21.15.148:6789/0 conn(0x7f432015a290 legacy=0x7f432006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308003a60 0x7f4308003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 0x7f4308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43080051c0 0x7f430800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b0b20 con 0x7f4308004810 2023-10-10T20:20:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430800c160 con 0x7f43080051c0 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430800c320 con 0x7f4308003a60 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 0x7f4308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431f7fe700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308003a60 0x7f4308003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431ffff700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43080051c0 0x7f430800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 -- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308003a60 msgr2=0x7f4308003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308003a60 0x7f4308003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 -- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43080051c0 msgr2=0x7f430800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43080051c0 0x7f430800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431ffff700 1 -- 172.21.15.148:0/2300411816 reap_dead start 2023-10-10T20:20:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.160+0000 7f431effd700 1 -- 172.21.15.148:0/2300411816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308003570 con 0x7f4308004810 2023-10-10T20:20:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f431effd700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 0x7f4308004c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f431400c3c0 tx=0x7f4314001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3e8e7440fc3272 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400cc90 con 0x7f4308004810 2023-10-10T20:20:36.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400cdf0 con 0x7f4308004810 2023-10-10T20:20:36.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f431e7fc700 1 -- 172.21.15.148:0/2300411816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314005b50 con 0x7f4308004810 2023-10-10T20:20:36.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 -- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 msgr2=0x7f4308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 0x7f4308004c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f431400c3c0 tx=0x7f4314001e50 comp rx=0 tx=0).stop 2023-10-10T20:20:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 -- 172.21.15.148:0/2300411816 shutdown_connections 2023-10-10T20:20:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 --2- 172.21.15.148:0/2300411816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4308004810 0x7f4308004c80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 -- 172.21.15.148:0/2300411816 >> 172.21.15.148:0/2300411816 conn(0x7f43200d1880 msgr2=0x7f43200cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 -- 172.21.15.148:0/2300411816 shutdown_connections 2023-10-10T20:20:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.161+0000 7f432611c700 1 -- 172.21.15.148:0/2300411816 wait complete. 2023-10-10T20:20:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.162+0000 7f432611c700 1 Processor -- start 2023-10-10T20:20:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.162+0000 7f432611c700 1 -- start start 2023-10-10T20:20:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308004810 0x7f4320157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f432016e290 0x7f4320152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b4350 con 0x7f4320154850 2023-10-10T20:20:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432016e7a0 con 0x7f432016e290 2023-10-10T20:20:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f432611c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432016ea80 con 0x7f4308004810 2023-10-10T20:20:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f431effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f431effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50444/0 (socket says 172.21.15.148:50444) 2023-10-10T20:20:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f431effd700 1 -- 172.21.15.148:0/2454843377 learned_addr learned my addr 172.21.15.148:0/2454843377 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f431f7fe700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308004810 0x7f4320157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.163+0000 7f431ffff700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f432016e290 0x7f4320152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 -- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308004810 msgr2=0x7f4320157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308004810 0x7f4320157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 -- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f432016e290 msgr2=0x7f4320152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f432016e290 0x7f4320152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 -- 172.21.15.148:0/2454843377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200b3fe0 con 0x7f4320154850 2023-10-10T20:20:36.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f431effd700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f430800c130 tx=0x7f4314015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314000fa0 con 0x7f4320154850 2023-10-10T20:20:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314001430 con 0x7f4320154850 2023-10-10T20:20:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314007320 con 0x7f4320154850 2023-10-10T20:20:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4320156860 con 0x7f4320154850 2023-10-10T20:20:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.164+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320154e80 con 0x7f4320154850 2023-10-10T20:20:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.165+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200b3e40 con 0x7f4320154850 2023-10-10T20:20:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.165+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4314007710 con 0x7f4320154850 2023-10-10T20:20:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.166+0000 7f43077fe700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 0x7f430c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.166+0000 7f431f7fe700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 0x7f430c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.166+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f43140857b0 con 0x7f4320154850 2023-10-10T20:20:36.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.166+0000 7f431f7fe700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 0x7f430c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f43100094d0 tx=0x7f431000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.170+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43200b3e40 con 0x7f4320154850 2023-10-10T20:20:36.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.353+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43200b3e40 con 0x7f4320154850 2023-10-10T20:20:36.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.354+0000 7f43077fe700 1 -- 172.21.15.148:0/2454843377 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43200b3e40 con 0x7f4320154850 2023-10-10T20:20:36.356 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:20:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 msgr2=0x7f430c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 0x7f430c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f43100094d0 tx=0x7f431000b040 comp rx=0 tx=0).stop 2023-10-10T20:20:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 msgr2=0x7f432016dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f430800c130 tx=0x7f4314015000 comp rx=0 tx=0).stop 2023-10-10T20:20:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 shutdown_connections 2023-10-10T20:20:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f430c066f30 0x7f430c0693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f432016e290 0x7f4320152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4308004810 0x7f4320157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 --2- 172.21.15.148:0/2454843377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4320154850 0x7f432016dd50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 >> 172.21.15.148:0/2454843377 conn(0x7f43200d1880 msgr2=0x7f4320155840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 shutdown_connections 2023-10-10T20:20:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.356+0000 7f432611c700 1 -- 172.21.15.148:0/2454843377 wait complete. 2023-10-10T20:20:36.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:20:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.482+0000 7f0d87aba700 1 Processor -- start 2023-10-10T20:20:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.482+0000 7f0d87aba700 1 -- start start 2023-10-10T20:20:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d87aba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d800c9af0 0x7f0d8006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d87aba700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800ca0b0 con 0x7f0d801510f0 2023-10-10T20:20:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d87aba700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d80068a10 con 0x7f0d80070700 2023-10-10T20:20:36.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d87aba700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d80068cf0 con 0x7f0d800c9af0 2023-10-10T20:20:36.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85055700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d800c9af0 0x7f0d8006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85856700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0d801510f0 0x7f0d8015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33288/0 (socket says 172.21.15.148:33288) 2023-10-10T20:20:36.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85856700 1 -- 172.21.15.148:0/748734545 learned_addr learned my addr 172.21.15.148:0/748734545 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:36.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85055700 1 -- 172.21.15.148:0/748734545 >> v1:172.21.15.161:6789/0 conn(0x7f0d80070700 legacy=0x7f0d80070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85055700 1 -- 172.21.15.148:0/748734545 >> v1:172.21.15.148:6789/0 conn(0x7f0d801510f0 legacy=0x7f0d8015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85055700 1 -- 172.21.15.148:0/748734545 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800cb230 con 0x7f0d800c9af0 2023-10-10T20:20:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d84854700 1 -- 172.21.15.148:0/748734545 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953432584 0 0) 0x7f0d800ca0b0 con 0x7f0d801510f0 2023-10-10T20:20:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d85055700 1 --2- 172.21.15.148:0/748734545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d800c9af0 0x7f0d8006ecc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098e0 tx=0x7f0d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d30a0f4b26e008b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d84854700 1 -- 172.21.15.148:0/748734545 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74009e80 con 0x7f0d800c9af0 2023-10-10T20:20:36.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d84854700 1 -- 172.21.15.148:0/748734545 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400b820 con 0x7f0d800c9af0 2023-10-10T20:20:36.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.483+0000 7f0d84854700 1 -- 172.21.15.148:0/748734545 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400bba0 con 0x7f0d800c9af0 2023-10-10T20:20:36.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 -- 172.21.15.148:0/748734545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d800c9af0 msgr2=0x7f0d8006ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 --2- 172.21.15.148:0/748734545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d800c9af0 0x7f0d8006ecc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098e0 tx=0x7f0d740091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 -- 172.21.15.148:0/748734545 shutdown_connections 2023-10-10T20:20:36.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 --2- 172.21.15.148:0/748734545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d800c9af0 0x7f0d8006ecc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 -- 172.21.15.148:0/748734545 >> 172.21.15.148:0/748734545 conn(0x7f0d800ca4f0 msgr2=0x7f0d800ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 -- 172.21.15.148:0/748734545 shutdown_connections 2023-10-10T20:20:36.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.484+0000 7f0d87aba700 1 -- 172.21.15.148:0/748734545 wait complete. 2023-10-10T20:20:36.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.485+0000 7f0d87aba700 1 Processor -- start 2023-10-10T20:20:36.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.485+0000 7f0d87aba700 1 -- start start 2023-10-10T20:20:36.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d800c9af0 0x7f0d800c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d801510f0 0x7f0d801577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b1940 con 0x7f0d80070700 2023-10-10T20:20:36.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b1c20 con 0x7f0d800c9af0 2023-10-10T20:20:36.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d87aba700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800c7620 con 0x7f0d801510f0 2023-10-10T20:20:36.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50472/0 (socket says 172.21.15.148:50472) 2023-10-10T20:20:36.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 -- 172.21.15.148:0/1673312334 learned_addr learned my addr 172.21.15.148:0/1673312334 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:36.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85055700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d800c9af0 0x7f0d800c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d86057700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d801510f0 0x7f0d801577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 -- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d801510f0 msgr2=0x7f0d801577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d801510f0 0x7f0d801577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 -- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d800c9af0 msgr2=0x7f0d800c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d800c9af0 0x7f0d800c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.486+0000 7f0d85856700 1 -- 172.21.15.148:0/1673312334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009420 con 0x7f0d80070700 2023-10-10T20:20:36.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d85856700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800cca0 tx=0x7f0d78001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800ab30 con 0x7f0d80070700 2023-10-10T20:20:36.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800ac90 con 0x7f0d80070700 2023-10-10T20:20:36.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800c7900 con 0x7f0d80070700 2023-10-10T20:20:36.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78007300 con 0x7f0d80070700 2023-10-10T20:20:36.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.487+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80150aa0 con 0x7f0d80070700 2023-10-10T20:20:36.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.488+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8004c3c0 con 0x7f0d80070700 2023-10-10T20:20:36.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.488+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d780076f0 con 0x7f0d80070700 2023-10-10T20:20:36.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.488+0000 7f0d7dffb700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 0x7f0d6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.489+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0d78084660 con 0x7f0d80070700 2023-10-10T20:20:36.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.489+0000 7f0d85055700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 0x7f0d6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.489+0000 7f0d85055700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 0x7f0d6c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d740030c0 tx=0x7f0d74016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.493+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d78051ca0 con 0x7f0d80070700 2023-10-10T20:20:36.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.676+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0d8006b620 con 0x7f0d80070700 2023-10-10T20:20:36.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.676+0000 7f0d7dffb700 1 -- 172.21.15.148:0/1673312334 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0d780552c0 con 0x7f0d80070700 2023-10-10T20:20:36.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:36.679 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:20:36.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:36.680 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:20:36.680 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:20:36.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:36.680 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:20:36.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 msgr2=0x7f0d6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 0x7f0d6c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d740030c0 tx=0x7f0d74016000 comp rx=0 tx=0).stop 2023-10-10T20:20:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 msgr2=0x7f0d80150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800cca0 tx=0x7f0d78001a30 comp rx=0 tx=0).stop 2023-10-10T20:20:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 shutdown_connections 2023-10-10T20:20:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d6c066f30 0x7f0d6c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d801510f0 0x7f0d801577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d80070700 0x7f0d80150560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 --2- 172.21.15.148:0/1673312334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d800c9af0 0x7f0d800c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 >> 172.21.15.148:0/1673312334 conn(0x7f0d800ca4f0 msgr2=0x7f0d8015d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 shutdown_connections 2023-10-10T20:20:36.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.678+0000 7f0d87aba700 1 -- 172.21.15.148:0/1673312334 wait complete. 2023-10-10T20:20:36.692 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:20:36.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:20:36.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.810+0000 7fad4626c700 1 Processor -- start 2023-10-10T20:20:36.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.810+0000 7fad4626c700 1 -- start start 2023-10-10T20:20:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4626c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad4015dfd0 0x7fad40159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4626c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4015e5b0 con 0x7fad400cd7f0 2023-10-10T20:20:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4626c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad400ceea0 con 0x7fad400cdbc0 2023-10-10T20:20:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4626c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4015e890 con 0x7fad4015dfd0 2023-10-10T20:20:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4526a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fad400cd7f0 0x7fad4015d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33304/0 (socket says 172.21.15.148:33304) 2023-10-10T20:20:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad4526a700 1 -- 172.21.15.148:0/2287264770 learned_addr learned my addr 172.21.15.148:0/2287264770 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262156727 0 0) 0x7fad4015e5b0 con 0x7fad400cd7f0 2023-10-10T20:20:36.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad45a6b700 1 --2- 172.21.15.148:0/2287264770 >> v2:172.21.15.161:3300/0 conn(0x7fad4015dfd0 0x7fad40159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad2c003570 con 0x7fad400cd7f0 2023-10-10T20:20:36.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3492481071 0 0) 0x7fad2c003570 con 0x7fad400cd7f0 2023-10-10T20:20:36.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 >> v2:172.21.15.161:3300/0 conn(0x7fad4015dfd0 msgr2=0x7fad40159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 --2- 172.21.15.148:0/2287264770 >> v2:172.21.15.161:3300/0 conn(0x7fad4015dfd0 0x7fad40159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 >> v1:172.21.15.161:6789/0 conn(0x7fad400cdbc0 legacy=0x7fad401595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400cf180 con 0x7fad400cd7f0 2023-10-10T20:20:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fad3c001470 con 0x7fad400cd7f0 2023-10-10T20:20:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 >> v1:172.21.15.148:6789/0 conn(0x7fad400cd7f0 legacy=0x7fad4015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 0x7fad2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c004810 0x7fad2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c0051c0 0x7fad2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4015e890 con 0x7fad2c003a60 2023-10-10T20:20:36.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c00c0d0 con 0x7fad2c004810 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.811+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c00c290 con 0x7fad2c0051c0 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 0x7fad2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad44a69700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c004810 0x7fad2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad45a6b700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c0051c0 0x7fad2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 -- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c0051c0 msgr2=0x7fad2c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c0051c0 0x7fad2c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 -- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c004810 msgr2=0x7fad2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c004810 0x7fad2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 -- 172.21.15.148:0/2287264770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c003570 con 0x7fad2c003a60 2023-10-10T20:20:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad45a6b700 1 -- 172.21.15.148:0/2287264770 reap_dead start 2023-10-10T20:20:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad4526a700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 0x7fad2c003e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c3c0 tx=0x7fad3c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68552e8a3f71c527 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00cc10 con 0x7fad2c003a60 2023-10-10T20:20:36.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c00cd70 con 0x7fad2c003a60 2023-10-10T20:20:36.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.812+0000 7fad37fff700 1 -- 172.21.15.148:0/2287264770 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c005b50 con 0x7fad2c003a60 2023-10-10T20:20:36.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 -- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 msgr2=0x7fad2c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 0x7fad2c003e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c3c0 tx=0x7fad3c001e50 comp rx=0 tx=0).stop 2023-10-10T20:20:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 -- 172.21.15.148:0/2287264770 shutdown_connections 2023-10-10T20:20:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 --2- 172.21.15.148:0/2287264770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c003a60 0x7fad2c003e70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 -- 172.21.15.148:0/2287264770 >> 172.21.15.148:0/2287264770 conn(0x7fad40068bf0 msgr2=0x7fad40068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 -- 172.21.15.148:0/2287264770 shutdown_connections 2023-10-10T20:20:36.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.813+0000 7fad4626c700 1 -- 172.21.15.148:0/2287264770 wait complete. 2023-10-10T20:20:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.814+0000 7fad4626c700 1 Processor -- start 2023-10-10T20:20:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.814+0000 7fad4626c700 1 -- start start 2023-10-10T20:20:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c003a60 0x7fad40142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad400b4bb0 0x7fad400b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad40142850 con 0x7fad400b3b60 2023-10-10T20:20:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad400b48d0 con 0x7fad2c003a60 2023-10-10T20:20:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4626c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad40142d00 con 0x7fad400b4bb0 2023-10-10T20:20:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad4526a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c003a60 0x7fad40142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50492/0 (socket says 172.21.15.148:50492) 2023-10-10T20:20:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 -- 172.21.15.148:0/1986734515 learned_addr learned my addr 172.21.15.148:0/1986734515 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad44a69700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad400b4bb0 0x7fad400b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 -- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad400b4bb0 msgr2=0x7fad400b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad400b4bb0 0x7fad400b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 -- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c003a60 msgr2=0x7fad40142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c003a60 0x7fad40142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 -- 172.21.15.148:0/1986734515 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40142fe0 con 0x7fad400b3b60 2023-10-10T20:20:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad45a6b700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad400b0470 tx=0x7fad3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.815+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800e040 con 0x7fad400b3b60 2023-10-10T20:20:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.816+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3800f040 con 0x7fad400b3b60 2023-10-10T20:20:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.816+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad401432c0 con 0x7fad400b3b60 2023-10-10T20:20:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.816+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800fd70 con 0x7fad400b3b60 2023-10-10T20:20:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.816+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad400b4520 con 0x7fad400b3b60 2023-10-10T20:20:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.816+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad401434b0 con 0x7fad400b3b60 2023-10-10T20:20:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.817+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad38015640 con 0x7fad400b3b60 2023-10-10T20:20:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.817+0000 7fad357fa700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 0x7fad240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.817+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fad38084870 con 0x7fad400b3b60 2023-10-10T20:20:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.817+0000 7fad4526a700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 0x7fad240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.818+0000 7fad4526a700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 0x7fad240693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001ed0 tx=0x7fad3c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:36.822+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad38051e30 con 0x7fad400b3b60 2023-10-10T20:20:37.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.004+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fad40089520 con 0x7fad400b3b60 2023-10-10T20:20:37.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.800+0000 7fad357fa700 1 -- 172.21.15.148:0/1986734515 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7fad38055450 con 0x7fad400b3b60 2023-10-10T20:20:37.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 msgr2=0x7fad240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 0x7fad240693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001ed0 tx=0x7fad3c015000 comp rx=0 tx=0).stop 2023-10-10T20:20:37.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 msgr2=0x7fad400b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fad400b0470 tx=0x7fad3800b040 comp rx=0 tx=0).stop 2023-10-10T20:20:37.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 shutdown_connections 2023-10-10T20:20:37.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad24066f30 0x7fad240693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad400b3b60 0x7fad400b3fe0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c003a60 0x7fad40142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 --2- 172.21.15.148:0/1986734515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad400b4bb0 0x7fad400b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 >> 172.21.15.148:0/1986734515 conn(0x7fad40068bf0 msgr2=0x7fad401587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:37.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 shutdown_connections 2023-10-10T20:20:37.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.803+0000 7fad4626c700 1 -- 172.21.15.148:0/1986734515 wait complete. 2023-10-10T20:20:37.809 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:20:37.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:20:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.939+0000 7feb084a9700 1 Processor -- start 2023-10-10T20:20:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.939+0000 7feb084a9700 1 -- start start 2023-10-10T20:20:37.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb084a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feb000caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb084a9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb0006d370 0x7feb0006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb084a9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0015bdf0 con 0x7feb00152300 2023-10-10T20:20:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb084a9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000b1400 con 0x7feb0006d370 2023-10-10T20:20:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb084a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0015c8d0 con 0x7feb0015b9a0 2023-10-10T20:20:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb06245700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feb00152300 0x7feb00158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33318/0 (socket says 172.21.15.148:33318) 2023-10-10T20:20:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.940+0000 7feb06245700 1 -- 172.21.15.148:0/1257821652 learned_addr learned my addr 172.21.15.148:0/1257821652 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220059124 0 0) 0x7feafc001110 con 0x7feb00152300 2023-10-10T20:20:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec003570 con 0x7feb00152300 2023-10-10T20:20:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb06a46700 1 --2- 172.21.15.148:0/1257821652 >> v2:172.21.15.161:3300/0 conn(0x7feb0006d370 0x7feb0006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05a44700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feb000caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955694225 0 0) 0x7feaec003570 con 0x7feb00152300 2023-10-10T20:20:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 msgr2=0x7feb000caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feb000caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 >> v2:172.21.15.161:3300/0 conn(0x7feb0006d370 msgr2=0x7feb0006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 --2- 172.21.15.148:0/1257821652 >> v2:172.21.15.161:3300/0 conn(0x7feb0006d370 0x7feb0006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb000c9370 con 0x7feb00152300 2023-10-10T20:20:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7feafc001600 con 0x7feb00152300 2023-10-10T20:20:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 >> v1:172.21.15.148:6789/0 conn(0x7feb00152300 legacy=0x7feb00158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feaec003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feaec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaec004d50 0x7feaec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0015c8d0 con 0x7feaec003fe0 2023-10-10T20:20:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00ba60 con 0x7feaec004d50 2023-10-10T20:20:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00bbd0 con 0x7feb0015b9a0 2023-10-10T20:20:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb05a44700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feaec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb06245700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feaec003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.941+0000 7feb06a46700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaec004d50 0x7feaec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 -- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 msgr2=0x7feaec003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb0015b9a0 0x7feaec003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 -- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaec004d50 msgr2=0x7feaec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaec004d50 0x7feaec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 -- 172.21.15.148:0/1257821652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec003570 con 0x7feaec003fe0 2023-10-10T20:20:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05a44700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feaec004810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009c00 tx=0x7feaf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87791fa8246f15 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:37.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000e6b0 con 0x7feaec003fe0 2023-10-10T20:20:37.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000e810 con 0x7feaec003fe0 2023-10-10T20:20:37.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb05243700 1 -- 172.21.15.148:0/1257821652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000ebd0 con 0x7feaec003fe0 2023-10-10T20:20:37.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb084a9700 1 -- 172.21.15.148:0/1257821652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 msgr2=0x7feaec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb084a9700 1 --2- 172.21.15.148:0/1257821652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feaec004810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009c00 tx=0x7feaf00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb06a46700 1 -- 172.21.15.148:0/1257821652 reap_dead start 2023-10-10T20:20:37.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb084a9700 1 -- 172.21.15.148:0/1257821652 shutdown_connections 2023-10-10T20:20:37.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.942+0000 7feb084a9700 1 -- 172.21.15.148:0/1257821652 >> 172.21.15.148:0/1257821652 conn(0x7feb00069870 msgr2=0x7feb00069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:37.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.943+0000 7feb084a9700 1 -- 172.21.15.148:0/1257821652 shutdown_connections 2023-10-10T20:20:37.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.943+0000 7feb084a9700 1 -- 172.21.15.148:0/1257821652 wait complete. 2023-10-10T20:20:37.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 Processor -- start 2023-10-10T20:20:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 -- start start 2023-10-10T20:20:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb000c5cb0 0x7feb00150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb000c8090 0x7feb000c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000c6220 con 0x7feaec003fe0 2023-10-10T20:20:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000c6bc0 con 0x7feb000c8090 2023-10-10T20:20:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb084a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0015aa50 con 0x7feb000c5cb0 2023-10-10T20:20:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb06245700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb06245700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50522/0 (socket says 172.21.15.148:50522) 2023-10-10T20:20:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.944+0000 7feb06245700 1 -- 172.21.15.148:0/2492119677 learned_addr learned my addr 172.21.15.148:0/2492119677 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06a46700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb000c8090 0x7feb000c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb05a44700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb000c5cb0 0x7feb00150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 -- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb000c5cb0 msgr2=0x7feb00150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb000c5cb0 0x7feb00150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 -- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb000c8090 msgr2=0x7feb000c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:37.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb000c8090 0x7feb000c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:37.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 -- 172.21.15.148:0/2492119677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb000b30a0 con 0x7feaec003fe0 2023-10-10T20:20:37.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb06245700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feafc0073c0 tx=0x7feb000c92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:37.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc018030 con 0x7feaec003fe0 2023-10-10T20:20:37.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00f640 con 0x7feaec003fe0 2023-10-10T20:20:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00fa00 con 0x7feaec003fe0 2023-10-10T20:20:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb000b26d0 con 0x7feaec003fe0 2023-10-10T20:20:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.945+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb000b2d30 con 0x7feaec003fe0 2023-10-10T20:20:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.946+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0004c3c0 con 0x7feaec003fe0 2023-10-10T20:20:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.946+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feafc01b070 con 0x7feaec003fe0 2023-10-10T20:20:37.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.947+0000 7feaf67fc700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 0x7feae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:37.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.947+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5223+0+0 (secure 0 0 0) 0x7feafc019070 con 0x7feaec003fe0 2023-10-10T20:20:37.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.947+0000 7feb05a44700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 0x7feae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:37.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.947+0000 7feb05a44700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 0x7feae40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feaf00091e0 tx=0x7feaf001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:37.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:37.951+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feafc0513e0 con 0x7feaec003fe0 2023-10-10T20:20:38.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.133+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feb0015bdf0 con 0x7feaec003fe0 2023-10-10T20:20:38.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.805+0000 7feaf67fc700 1 -- 172.21.15.148:0/2492119677 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7feb0015bdf0 con 0x7feaec003fe0 2023-10-10T20:20:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 msgr2=0x7feae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 0x7feae40693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feaf00091e0 tx=0x7feaf001a000 comp rx=0 tx=0).stop 2023-10-10T20:20:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 msgr2=0x7feb000c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feafc0073c0 tx=0x7feb000c92e0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 shutdown_connections 2023-10-10T20:20:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feae4066f30 0x7feae40693e0 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feaf00091e0 tx=0x7feaf001a000 comp rx=0 tx=0).stop 2023-10-10T20:20:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb000c8090 0x7feb000c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaec003fe0 0x7feb000c6680 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feafc0073c0 tx=0x7feb000c92e0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 --2- 172.21.15.148:0/2492119677 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb000c5cb0 0x7feb00150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 >> 172.21.15.148:0/2492119677 conn(0x7feb00069870 msgr2=0x7feb0015ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:38.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 shutdown_connections 2023-10-10T20:20:38.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.816+0000 7feb084a9700 1 -- 172.21.15.148:0/2492119677 wait complete. 2023-10-10T20:20:38.821 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:20:38.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:20:38.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.948+0000 7f0c2bfff700 1 Processor -- start 2023-10-10T20:20:38.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.948+0000 7f0c2bfff700 1 -- start start 2023-10-10T20:20:38.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 0x7f0c2c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c2c0b0460 0x7f0c2c0b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0ce9e0 0x7f0c2c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b0e40 con 0x7f0c2c0b0460 2023-10-10T20:20:38.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b1850 con 0x7f0c2c0adc80 2023-10-10T20:20:38.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0b20f0 con 0x7f0c2c0ce9e0 2023-10-10T20:20:38.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2a7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c2c0b0460 0x7f0c2c0b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2a7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c2c0b0460 0x7f0c2c0b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50528/0 (socket says 172.21.15.148:50528) 2023-10-10T20:20:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/3895062307 learned_addr learned my addr 172.21.15.148:0/3895062307 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2b7fe700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0ce9e0 0x7f0c2c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.949+0000 7f0c2affd700 1 --2- 172.21.15.148:0/3895062307 >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 0x7f0c2c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/3895062307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0ce9e0 msgr2=0x7f0c2c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0ce9e0 0x7f0c2c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/3895062307 >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 msgr2=0x7f0c2c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/3895062307 >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 0x7f0c2c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/3895062307 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0cf750 con 0x7f0c2c0b0460 2023-10-10T20:20:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2affd700 1 --2- 172.21.15.148:0/3895062307 >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 0x7f0c2c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0b0900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c070300 tx=0x7f0c20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be33f6f51267d3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c29ffb700 1 -- 172.21.15.148:0/3895062307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000b860 con 0x7f0c2c0b0460 2023-10-10T20:20:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c29ffb700 1 -- 172.21.15.148:0/3895062307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2000b9c0 con 0x7f0c2c0b0460 2023-10-10T20:20:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c29ffb700 1 -- 172.21.15.148:0/3895062307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000bd60 con 0x7f0c2c0b0460 2023-10-10T20:20:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 -- 172.21.15.148:0/3895062307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 msgr2=0x7f0c2c0b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0b0900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c070300 tx=0x7f0c20009250 comp rx=0 tx=0).stop 2023-10-10T20:20:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 -- 172.21.15.148:0/3895062307 shutdown_connections 2023-10-10T20:20:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0ce9e0 0x7f0c2c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/3895062307 >> v2:172.21.15.161:3300/0 conn(0x7f0c2c0adc80 0x7f0c2c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/3895062307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0b0900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 -- 172.21.15.148:0/3895062307 >> 172.21.15.148:0/3895062307 conn(0x7f0c2c06c700 msgr2=0x7f0c2c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 -- 172.21.15.148:0/3895062307 shutdown_connections 2023-10-10T20:20:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.950+0000 7f0c2bfff700 1 -- 172.21.15.148:0/3895062307 wait complete. 2023-10-10T20:20:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 Processor -- start 2023-10-10T20:20:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 -- start start 2023-10-10T20:20:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0adc80 0x7f0c2c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c2c0ce9e0 0x7f0c2c147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c148390 con 0x7f0c2c0b0460 2023-10-10T20:20:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0c7cd0 con 0x7f0c2c0ce9e0 2023-10-10T20:20:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.952+0000 7f0c2bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0ca660 con 0x7f0c2c0adc80 2023-10-10T20:20:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50540/0 (socket says 172.21.15.148:50540) 2023-10-10T20:20:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/2445567018 learned_addr learned my addr 172.21.15.148:0/2445567018 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c2c0ce9e0 0x7f0c2c147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2affd700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0adc80 0x7f0c2c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0adc80 msgr2=0x7f0c2c0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0adc80 0x7f0c2c0c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c2c0ce9e0 msgr2=0x7f0c2c147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c2c0ce9e0 0x7f0c2c147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 -- 172.21.15.148:0/2445567018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200094f0 con 0x7f0c2c0b0460 2023-10-10T20:20:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2a7fc700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0c200090b0 tx=0x7f0c20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:38.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011e00 con 0x7f0c2c0b0460 2023-10-10T20:20:38.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2001f070 con 0x7f0c2c0b0460 2023-10-10T20:20:38.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200266d0 con 0x7f0c2c0b0460 2023-10-10T20:20:38.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c0c7710 con 0x7f0c2c0b0460 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.953+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c148020 con 0x7f0c2c0b0460 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.954+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c147e80 con 0x7f0c2c0b0460 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.955+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c20026a40 con 0x7f0c2c0b0460 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.955+0000 7f0c12ffd700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 0x7f0c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.955+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0c20021070 con 0x7f0c2c0b0460 2023-10-10T20:20:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.955+0000 7f0c2affd700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 0x7f0c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.955+0000 7f0c2affd700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 0x7f0c180693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c007740 tx=0x7f0c1c0099d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:38.960+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c2c147e80 con 0x7f0c2c0b0460 2023-10-10T20:20:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.182+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0c2c147e80 con 0x7f0c2c0b0460 2023-10-10T20:20:39.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.821+0000 7f0c12ffd700 1 -- 172.21.15.148:0/2445567018 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c2c147e80 con 0x7f0c2c0b0460 2023-10-10T20:20:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.826+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 msgr2=0x7f0c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.826+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 0x7f0c180693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c007740 tx=0x7f0c1c0099d0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.826+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 msgr2=0x7f0c2c0ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.826+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0c200090b0 tx=0x7f0c20017000 comp rx=0 tx=0).stop 2023-10-10T20:20:39.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 shutdown_connections 2023-10-10T20:20:39.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c18066f30 0x7f0c180693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c2c0ce9e0 0x7f0c2c147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c2c0adc80 0x7f0c2c0c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 --2- 172.21.15.148:0/2445567018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c2c0b0460 0x7f0c2c0ca120 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0c200090b0 tx=0x7f0c20017000 comp rx=0 tx=0).stop 2023-10-10T20:20:39.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 >> 172.21.15.148:0/2445567018 conn(0x7f0c2c06c700 msgr2=0x7f0c2c0cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:39.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 shutdown_connections 2023-10-10T20:20:39.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.828+0000 7f0c2bfff700 1 -- 172.21.15.148:0/2445567018 wait complete. 2023-10-10T20:20:39.832 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 6 and data pool 7 2023-10-10T20:20:39.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:39.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 Processor -- start 2023-10-10T20:20:39.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 -- start start 2023-10-10T20:20:39.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fea08165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea080b26d0 con 0x7fea080a79f0 2023-10-10T20:20:39.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea080aeab0 con 0x7fea0806f1c0 2023-10-10T20:20:39.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0f0dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea080b2b60 con 0x7fea080b15e0 2023-10-10T20:20:39.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0ce79700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fea080a79f0 0x7fea0806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33326/0 (socket says 172.21.15.148:33326) 2023-10-10T20:20:39.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0ce79700 1 -- 172.21.15.148:0/4290163469 learned_addr learned my addr 172.21.15.148:0/4290163469 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:39.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285457579 0 0) 0x7fea080b26d0 con 0x7fea080a79f0 2023-10-10T20:20:39.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea0d67a700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fea08165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f8003570 con 0x7fea080a79f0 2023-10-10T20:20:39.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373918323 0 0) 0x7fe9f8003570 con 0x7fea080a79f0 2023-10-10T20:20:39.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 msgr2=0x7fea08165540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:39.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fea08165540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 >> v1:172.21.15.161:6789/0 conn(0x7fea0806f1c0 legacy=0x7fea08161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080b1ac0 con 0x7fea080a79f0 2023-10-10T20:20:39.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fe9fc001470 con 0x7fea080a79f0 2023-10-10T20:20:39.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.959+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 >> v1:172.21.15.148:6789/0 conn(0x7fea080a79f0 legacy=0x7fea0806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fe9f8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fe9f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f8004d50 0x7fe9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080b2b60 con 0x7fe9f8004010 2023-10-10T20:20:39.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f800bbf0 con 0x7fe9f8004d50 2023-10-10T20:20:39.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f800bdb0 con 0x7fea080b15e0 2023-10-10T20:20:39.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fe9f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea0ce79700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fe9f8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea0d67a700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f8004d50 0x7fe9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 -- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 msgr2=0x7fe9f8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:39.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea080b15e0 0x7fe9f8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 -- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f8004d50 msgr2=0x7fe9f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:39.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f8004d50 0x7fe9f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 -- 172.21.15.148:0/4290163469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8003570 con 0x7fe9f8004010 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea0d67a700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9f8004d50 0x7fe9f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea07fff700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fe9f8004810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea0806f5c0 tx=0x7fe9f4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2068d41e82d990dc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4012420 con 0x7fe9f8004010 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4012580 con 0x7fe9f8004010 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.960+0000 7fea077fe700 1 -- 172.21.15.148:0/4290163469 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4012920 con 0x7fe9f8004010 2023-10-10T20:20:39.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 -- 172.21.15.148:0/4290163469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 msgr2=0x7fe9f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/4290163469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fe9f8004810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea0806f5c0 tx=0x7fe9f4007d30 comp rx=0 tx=0).stop 2023-10-10T20:20:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0d67a700 1 -- 172.21.15.148:0/4290163469 reap_dead start 2023-10-10T20:20:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 -- 172.21.15.148:0/4290163469 shutdown_connections 2023-10-10T20:20:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 -- 172.21.15.148:0/4290163469 >> 172.21.15.148:0/4290163469 conn(0x7fea080cd1e0 msgr2=0x7fea080cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 -- 172.21.15.148:0/4290163469 shutdown_connections 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.961+0000 7fea0f0dd700 1 -- 172.21.15.148:0/4290163469 wait complete. 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.963+0000 7fea0f0dd700 1 Processor -- start 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.963+0000 7fea0f0dd700 1 -- start start 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0809a7a0 0x7fea0815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0809ac10 0x7fea0815d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea08099700 con 0x7fe9f8004010 2023-10-10T20:20:39.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080999e0 con 0x7fea0809a7a0 2023-10-10T20:20:39.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea08099cc0 con 0x7fea0809ac10 2023-10-10T20:20:39.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50548/0 (socket says 172.21.15.148:50548) 2023-10-10T20:20:39.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 -- 172.21.15.148:0/351329599 learned_addr learned my addr 172.21.15.148:0/351329599 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:39.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea07fff700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0809a7a0 0x7fea0815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0d67a700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0809ac10 0x7fea0815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 -- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0809ac10 msgr2=0x7fea0815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0809ac10 0x7fea0815d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 -- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0809a7a0 msgr2=0x7fea0815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:39.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0809a7a0 0x7fea0815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:39.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 -- 172.21.15.148:0/351329599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08099fa0 con 0x7fe9f8004010 2023-10-10T20:20:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0ce79700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000f60 tx=0x7fe9fc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc018030 con 0x7fe9f8004010 2023-10-10T20:20:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00f850 con 0x7fe9f8004010 2023-10-10T20:20:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0809a280 con 0x7fe9f8004010 2023-10-10T20:20:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00fc10 con 0x7fe9f8004010 2023-10-10T20:20:39.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea081473c0 con 0x7fe9f8004010 2023-10-10T20:20:39.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.964+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0809a4c0 con 0x7fe9f8004010 2023-10-10T20:20:39.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.965+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9fc01b070 con 0x7fe9f8004010 2023-10-10T20:20:39.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.965+0000 7fea04ff9700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 0x7fe9ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:39.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.965+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe9fc019070 con 0x7fe9f8004010 2023-10-10T20:20:39.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.965+0000 7fea07fff700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 0x7fe9ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:39.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.966+0000 7fea07fff700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 0x7fe9ec0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea080aea60 tx=0x7fe9f4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:39.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:39.970+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea0809a4c0 con 0x7fe9f8004010 2023-10-10T20:20:40.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.150+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0809a4c0 con 0x7fe9f8004010 2023-10-10T20:20:40.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.151+0000 7fea04ff9700 1 -- 172.21.15.148:0/351329599 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fe9fc051680 con 0x7fe9f8004010 2023-10-10T20:20:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:40.153 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":44,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:39.184484+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:40.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 msgr2=0x7fe9ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 0x7fe9ec0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea080aea60 tx=0x7fe9f4015000 comp rx=0 tx=0).stop 2023-10-10T20:20:40.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 msgr2=0x7fea080c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000f60 tx=0x7fe9fc001e50 comp rx=0 tx=0).stop 2023-10-10T20:20:40.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 shutdown_connections 2023-10-10T20:20:40.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe9ec066f30 0x7fe9ec0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0809ac10 0x7fea0815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f8004010 0x7fea080c37b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 --2- 172.21.15.148:0/351329599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea0809a7a0 0x7fea0815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 >> 172.21.15.148:0/351329599 conn(0x7fea080cd1e0 msgr2=0x7fea080ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:40.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 shutdown_connections 2023-10-10T20:20:40.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.154+0000 7fea0f0dd700 1 -- 172.21.15.148:0/351329599 wait complete. 2023-10-10T20:20:40.167 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:40.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 Processor -- start 2023-10-10T20:20:40.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 -- start start 2023-10-10T20:20:40.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 0x7f3d6415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 0x7f3d6415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640cb7c0 con 0x7f3d640cd6e0 2023-10-10T20:20:40.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6415e850 con 0x7f3d640cdab0 2023-10-10T20:20:40.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6b384700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cc360 con 0x7f3d6415dda0 2023-10-10T20:20:40.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6ab83700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 0x7f3d6415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d69b81700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 0x7f3d6415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6a382700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3d640cd6e0 0x7f3d640cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:36052/0 (socket says 172.21.15.148:36052) 2023-10-10T20:20:40.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.281+0000 7f3d6a382700 1 -- 172.21.15.148:0/1244665618 learned_addr learned my addr 172.21.15.148:0/1244665618 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:40.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d69380700 1 -- 172.21.15.148:0/1244665618 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201595929 0 0) 0x7f3d640cb7c0 con 0x7f3d640cd6e0 2023-10-10T20:20:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1244665618 >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 msgr2=0x7f3d6415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6ab83700 1 --2- 172.21.15.148:0/1244665618 >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 0x7f3d6415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1244665618 >> v1:172.21.15.161:6789/0 conn(0x7f3d640cd6e0 legacy=0x7f3d640cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1244665618 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6415e290 con 0x7f3d6415dda0 2023-10-10T20:20:40.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d69b81700 1 --2- 172.21.15.148:0/1244665618 >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 0x7f3d6415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:40.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6ab83700 1 --2- 172.21.15.148:0/1244665618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 0x7f3d6415cbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009e60 tx=0x7f3d5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1547ce176537f0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d69380700 1 -- 172.21.15.148:0/1244665618 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00e070 con 0x7f3d6415dda0 2023-10-10T20:20:40.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d69380700 1 -- 172.21.15.148:0/1244665618 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00bb60 con 0x7f3d6415dda0 2023-10-10T20:20:40.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d69380700 1 -- 172.21.15.148:0/1244665618 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c012430 con 0x7f3d6415dda0 2023-10-10T20:20:40.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6b384700 1 -- 172.21.15.148:0/1244665618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 msgr2=0x7f3d6415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.282+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1244665618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 0x7f3d6415cbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009e60 tx=0x7f3d5c009780 comp rx=0 tx=0).stop 2023-10-10T20:20:40.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 -- 172.21.15.148:0/1244665618 shutdown_connections 2023-10-10T20:20:40.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1244665618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d6415dda0 0x7f3d6415cbd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1244665618 >> v2:172.21.15.161:3300/0 conn(0x7f3d640cdab0 0x7f3d6415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 -- 172.21.15.148:0/1244665618 >> 172.21.15.148:0/1244665618 conn(0x7f3d64067660 msgr2=0x7f3d64067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:40.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 -- 172.21.15.148:0/1244665618 shutdown_connections 2023-10-10T20:20:40.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.283+0000 7f3d6b384700 1 -- 172.21.15.148:0/1244665618 wait complete. 2023-10-10T20:20:40.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 Processor -- start 2023-10-10T20:20:40.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 -- start start 2023-10-10T20:20:40.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d640cd6e0 0x7f3d640b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 0x7f3d6414c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6414cf10 con 0x7f3d6415dda0 2023-10-10T20:20:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c8ec0 con 0x7f3d640cd6e0 2023-10-10T20:20:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.284+0000 7f3d6b384700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c91a0 con 0x7f3d640cdab0 2023-10-10T20:20:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50564/0 (socket says 172.21.15.148:50564) 2023-10-10T20:20:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d69b81700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 0x7f3d6414c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1102780633 learned_addr learned my addr 172.21.15.148:0/1102780633 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:40.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6a382700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d640cd6e0 0x7f3d640b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 msgr2=0x7f3d6414c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 0x7f3d6414c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d640cd6e0 msgr2=0x7f3d640b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d640cd6e0 0x7f3d640b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 -- 172.21.15.148:0/1102780633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54009420 con 0x7f3d6415dda0 2023-10-10T20:20:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d69b81700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 0x7f3d6414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6ab83700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ccdd0 tx=0x7f3d5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c012ea0 con 0x7f3d6415dda0 2023-10-10T20:20:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c027470 con 0x7f3d6415dda0 2023-10-10T20:20:40.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c009a20 con 0x7f3d6415dda0 2023-10-10T20:20:40.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c027830 con 0x7f3d6415dda0 2023-10-10T20:20:40.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.285+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640b4e10 con 0x7f3d6415dda0 2023-10-10T20:20:40.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.286+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64089280 con 0x7f3d6415dda0 2023-10-10T20:20:40.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.287+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d5c024070 con 0x7f3d6415dda0 2023-10-10T20:20:40.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.287+0000 7f3d5a7fc700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 0x7f3d480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.287+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3d5c086c40 con 0x7f3d6415dda0 2023-10-10T20:20:40.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.287+0000 7f3d6a382700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 0x7f3d480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.287+0000 7f3d6a382700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 0x7f3d480693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007160 tx=0x7f3d60007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.291+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d5c053eb0 con 0x7f3d6415dda0 2023-10-10T20:20:40.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.476+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d640cbf80 con 0x7f3d6415dda0 2023-10-10T20:20:40.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.478+0000 7f3d5a7fc700 1 -- 172.21.15.148:0/1102780633 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f3d5c02c020 con 0x7f3d6415dda0 2023-10-10T20:20:40.480 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:40.481 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":44,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:39.184484+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:40.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 msgr2=0x7f3d480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 0x7f3d480693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d60007160 tx=0x7f3d60007f50 comp rx=0 tx=0).stop 2023-10-10T20:20:40.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 msgr2=0x7f3d6414c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ccdd0 tx=0x7f3d5c019000 comp rx=0 tx=0).stop 2023-10-10T20:20:40.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 shutdown_connections 2023-10-10T20:20:40.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d48066f30 0x7f3d480693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415dda0 0x7f3d6414c9d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d640cd6e0 0x7f3d640b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 --2- 172.21.15.148:0/1102780633 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3d640cdab0 0x7f3d6414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 >> 172.21.15.148:0/1102780633 conn(0x7f3d64067660 msgr2=0x7f3d6415e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:40.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 shutdown_connections 2023-10-10T20:20:40.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.481+0000 7f3d6b384700 1 -- 172.21.15.148:0/1102780633 wait complete. 2023-10-10T20:20:40.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:20:40.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.613+0000 7f607a500700 1 Processor -- start 2023-10-10T20:20:40.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.613+0000 7f607a500700 1 -- start start 2023-10-10T20:20:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f607a500700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f607a500700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f607415ac10 0x7f60740cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f607a500700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60740cc760 con 0x7f60740cb970 2023-10-10T20:20:40.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f607a500700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607415b460 con 0x7f607415ac10 2023-10-10T20:20:40.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f607a500700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607415ba20 con 0x7f607415a300 2023-10-10T20:20:40.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f607415ac10 0x7f60740cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f607415ac10 0x7f60740cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50576/0 (socket says 172.21.15.148:50576) 2023-10-10T20:20:40.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 -- 172.21.15.148:0/3937723483 learned_addr learned my addr 172.21.15.148:0/3937723483 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:40.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f60794fe700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 -- 172.21.15.148:0/3937723483 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 msgr2=0x7f607415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:40.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607415a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 -- 172.21.15.148:0/3937723483 >> v1:172.21.15.161:6789/0 conn(0x7f60740cb970 legacy=0x7f607415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 -- 172.21.15.148:0/3937723483 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740cc1a0 con 0x7f607415ac10 2023-10-10T20:20:40.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.614+0000 7f6078cfd700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f60740cb430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f60640099b0 tx=0x7f6064009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8147645be7002519 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f606bfff700 1 -- 172.21.15.148:0/3937723483 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400b860 con 0x7f607415ac10 2023-10-10T20:20:40.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f606bfff700 1 -- 172.21.15.148:0/3937723483 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400b9c0 con 0x7f607415ac10 2023-10-10T20:20:40.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f606bfff700 1 -- 172.21.15.148:0/3937723483 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400bd60 con 0x7f607415ac10 2023-10-10T20:20:40.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 -- 172.21.15.148:0/3937723483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 msgr2=0x7f60740cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f60740cb430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f60640099b0 tx=0x7f6064009250 comp rx=0 tx=0).stop 2023-10-10T20:20:40.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 -- 172.21.15.148:0/3937723483 shutdown_connections 2023-10-10T20:20:40.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 --2- 172.21.15.148:0/3937723483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f60740cb430 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 -- 172.21.15.148:0/3937723483 >> 172.21.15.148:0/3937723483 conn(0x7f6074099f50 msgr2=0x7f6074067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:40.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 -- 172.21.15.148:0/3937723483 shutdown_connections 2023-10-10T20:20:40.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.615+0000 7f607a500700 1 -- 172.21.15.148:0/3937723483 wait complete. 2023-10-10T20:20:40.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.616+0000 7f607a500700 1 Processor -- start 2023-10-10T20:20:40.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.616+0000 7f607a500700 1 -- start start 2023-10-10T20:20:40.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60740cb970 0x7f607416b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607416d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740b4a80 con 0x7f607415ac10 2023-10-10T20:20:40.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740b4d20 con 0x7f60740cb970 2023-10-10T20:20:40.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f607a500700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740b5000 con 0x7f607415a300 2023-10-10T20:20:40.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f6079cff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f6079cff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50592/0 (socket says 172.21.15.148:50592) 2023-10-10T20:20:40.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f6079cff700 1 -- 172.21.15.148:0/2432311233 learned_addr learned my addr 172.21.15.148:0/2432311233 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:40.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f6078cfd700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607416d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.617+0000 7f60794fe700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60740cb970 0x7f607416b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 -- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 msgr2=0x7f607416d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607416d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 -- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60740cb970 msgr2=0x7f607416b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:40.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60740cb970 0x7f607416b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:40.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 -- 172.21.15.148:0/2432311233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640094f0 con 0x7f607415ac10 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f6079cff700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c840 tx=0x7f606c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c012630 con 0x7f607415ac10 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0127b0 con 0x7f607415ac10 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60740b52e0 con 0x7f607415ac10 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c012b70 con 0x7f607415ac10 2023-10-10T20:20:40.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.618+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607416b990 con 0x7f607415ac10 2023-10-10T20:20:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.619+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60740b54d0 con 0x7f607415ac10 2023-10-10T20:20:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.619+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f606c01a3a0 con 0x7f607415ac10 2023-10-10T20:20:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.620+0000 7f60697fa700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.620+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f606c084660 con 0x7f607415ac10 2023-10-10T20:20:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.620+0000 7f60794fe700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:40.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.620+0000 7f60794fe700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6070006aa0 tx=0x7f6070008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:40.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.624+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f606c0518d0 con 0x7f607415ac10 2023-10-10T20:20:40.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:40.850+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f607406ada0 con 0x7f607415ac10 2023-10-10T20:20:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.830+0000 7f60697fa700 1 -- 172.21.15.148:0/2432311233 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f606c054ef0 con 0x7f607415ac10 2023-10-10T20:20:41.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 msgr2=0x7f60580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:41.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6070006aa0 tx=0x7f6070008040 comp rx=0 tx=0).stop 2023-10-10T20:20:41.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 msgr2=0x7f607416da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:41.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c840 tx=0x7f606c001670 comp rx=0 tx=0).stop 2023-10-10T20:20:41.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 shutdown_connections 2023-10-10T20:20:41.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607415ac10 0x7f607416da90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60740cb970 0x7f607416b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 --2- 172.21.15.148:0/2432311233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f607415a300 0x7f607416d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.832+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 >> 172.21.15.148:0/2432311233 conn(0x7f6074099f50 msgr2=0x7f607415bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:41.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.833+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 shutdown_connections 2023-10-10T20:20:41.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.833+0000 7f607a500700 1 -- 172.21.15.148:0/2432311233 wait complete. 2023-10-10T20:20:41.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:20:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.971+0000 7fa86eb23700 1 Processor -- start 2023-10-10T20:20:41.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 -- start start 2023-10-10T20:20:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680678b0 0x7fa86815db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8680cbf10 0x7fa86815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8680c9390 0x7fa86815e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680cae70 con 0x7fa8680c9390 2023-10-10T20:20:41.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680cb150 con 0x7fa8680cbf10 2023-10-10T20:20:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86eb23700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680cb430 con 0x7fa8680678b0 2023-10-10T20:20:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.972+0000 7fa86d0c0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8680c9390 0x7fa86815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8680c9390 0x7fa86815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50598/0 (socket says 172.21.15.148:50598) 2023-10-10T20:20:41.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 -- 172.21.15.148:0/342794333 learned_addr learned my addr 172.21.15.148:0/342794333 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa85ffff700 1 --2- 172.21.15.148:0/342794333 >> v2:172.21.15.161:3300/0 conn(0x7fa8680cbf10 0x7fa86815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86c8bf700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680678b0 0x7fa86815db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 -- 172.21.15.148:0/342794333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680678b0 msgr2=0x7fa86815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680678b0 0x7fa86815db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 -- 172.21.15.148:0/342794333 >> v2:172.21.15.161:3300/0 conn(0x7fa8680cbf10 msgr2=0x7fa86815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/342794333 >> v2:172.21.15.161:3300/0 conn(0x7fa8680cbf10 0x7fa86815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 -- 172.21.15.148:0/342794333 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680cb710 con 0x7fa8680c9390 2023-10-10T20:20:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680c9390 0x7fa86815e600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8600099b0 tx=0x7fa860009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89cb326ecfc7f03 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa85f7fe700 1 -- 172.21.15.148:0/342794333 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000b640 con 0x7fa8680c9390 2023-10-10T20:20:41.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa85f7fe700 1 -- 172.21.15.148:0/342794333 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000b7a0 con 0x7fa8680c9390 2023-10-10T20:20:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.973+0000 7fa85f7fe700 1 -- 172.21.15.148:0/342794333 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000bb40 con 0x7fa8680c9390 2023-10-10T20:20:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 -- 172.21.15.148:0/342794333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680c9390 msgr2=0x7fa86815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680c9390 0x7fa86815e600 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8600099b0 tx=0x7fa860009250 comp rx=0 tx=0).stop 2023-10-10T20:20:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 -- 172.21.15.148:0/342794333 shutdown_connections 2023-10-10T20:20:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680c9390 0x7fa86815e600 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 --2- 172.21.15.148:0/342794333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680678b0 0x7fa86815db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 --2- 172.21.15.148:0/342794333 >> v2:172.21.15.161:3300/0 conn(0x7fa8680cbf10 0x7fa86815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 -- 172.21.15.148:0/342794333 >> 172.21.15.148:0/342794333 conn(0x7fa868069870 msgr2=0x7fa868069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 -- 172.21.15.148:0/342794333 shutdown_connections 2023-10-10T20:20:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.974+0000 7fa86eb23700 1 -- 172.21.15.148:0/342794333 wait complete. 2023-10-10T20:20:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.975+0000 7fa86eb23700 1 Processor -- start 2023-10-10T20:20:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.975+0000 7fa86eb23700 1 -- start start 2023-10-10T20:20:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8680678b0 0x7fa8680ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680c9390 0x7fa8681575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680cf090 con 0x7fa8680cbf10 2023-10-10T20:20:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86815c280 con 0x7fa8680678b0 2023-10-10T20:20:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86815c540 con 0x7fa8680c9390 2023-10-10T20:20:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50608/0 (socket says 172.21.15.148:50608) 2023-10-10T20:20:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 -- 172.21.15.148:0/3428143631 learned_addr learned my addr 172.21.15.148:0/3428143631 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86c8bf700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8680678b0 0x7fa8680ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa85ffff700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680c9390 0x7fa8681575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 -- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680c9390 msgr2=0x7fa8681575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680c9390 0x7fa8681575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 -- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8680678b0 msgr2=0x7fa8680ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8680678b0 0x7fa8680ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 -- 172.21.15.148:0/3428143631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600094f0 con 0x7fa8680cbf10 2023-10-10T20:20:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86d0c0700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa860009290 tx=0x7fa860017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000bb20 con 0x7fa8680cbf10 2023-10-10T20:20:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860011420 con 0x7fa8680cbf10 2023-10-10T20:20:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8681567f0 con 0x7fa8680cbf10 2023-10-10T20:20:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600117e0 con 0x7fa8680cbf10 2023-10-10T20:20:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.976+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8680cead0 con 0x7fa8680cbf10 2023-10-10T20:20:41.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.977+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86815c6f0 con 0x7fa8680cbf10 2023-10-10T20:20:41.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.978+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa860011bd0 con 0x7fa8680cbf10 2023-10-10T20:20:41.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.978+0000 7fa85cff9700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 0x7fa8540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:41.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.978+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa860085e30 con 0x7fa8680cbf10 2023-10-10T20:20:41.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.978+0000 7fa86c8bf700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 0x7fa8540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.979+0000 7fa86c8bf700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 0x7fa8540693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fd0 tx=0x7fa86400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:41.983+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa86815c6f0 con 0x7fa8680cbf10 2023-10-10T20:20:42.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.208+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa86815c6b0 con 0x7fa8680cbf10 2023-10-10T20:20:42.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.832+0000 7fa85cff9700 1 -- 172.21.15.148:0/3428143631 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8600530a0 con 0x7fa8680cbf10 2023-10-10T20:20:42.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 msgr2=0x7fa8540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:42.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 0x7fa8540693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa864009fd0 tx=0x7fa86400b040 comp rx=0 tx=0).stop 2023-10-10T20:20:42.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 msgr2=0x7fa8681562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:42.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa860009290 tx=0x7fa860017000 comp rx=0 tx=0).stop 2023-10-10T20:20:42.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 shutdown_connections 2023-10-10T20:20:42.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa854066f30 0x7fa8540693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8680cbf10 0x7fa8681562b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8680678b0 0x7fa8680ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 --2- 172.21.15.148:0/3428143631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8680c9390 0x7fa8681575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 >> 172.21.15.148:0/3428143631 conn(0x7fa868069870 msgr2=0x7fa8680c7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:42.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 shutdown_connections 2023-10-10T20:20:42.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.835+0000 7fa86eb23700 1 -- 172.21.15.148:0/3428143631 wait complete. 2023-10-10T20:20:42.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:42.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.962+0000 7fda049a0700 1 Processor -- start 2023-10-10T20:20:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.962+0000 7fda049a0700 1 -- start start 2023-10-10T20:20:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fda049a0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda00159be0 0x7fda00068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fda049a0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 0x7fda000b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fda049a0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda00067800 con 0x7fda00159fb0 2023-10-10T20:20:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fda049a0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda000701f0 con 0x7fda00159be0 2023-10-10T20:20:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fda049a0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda00067ae0 con 0x7fda000cf060 2023-10-10T20:20:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fd9ff7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda00159be0 0x7fda00068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fd9ff7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda00159be0 0x7fda00068db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53376/0 (socket says 172.21.15.148:53376) 2023-10-10T20:20:42.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.963+0000 7fd9ff7fe700 1 -- 172.21.15.148:0/794608839 learned_addr learned my addr 172.21.15.148:0/794608839 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:42.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9fffff700 1 --2- 172.21.15.148:0/794608839 >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 0x7fda000b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9ff7fe700 1 -- 172.21.15.148:0/794608839 >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 msgr2=0x7fda000b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:42.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9ff7fe700 1 --2- 172.21.15.148:0/794608839 >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 0x7fda000b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9ff7fe700 1 -- 172.21.15.148:0/794608839 >> v1:172.21.15.161:6789/0 conn(0x7fda00159fb0 legacy=0x7fda000ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:42.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9ff7fe700 1 -- 172.21.15.148:0/794608839 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00067dc0 con 0x7fda00159be0 2023-10-10T20:20:42.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9fffff700 1 --2- 172.21.15.148:0/794608839 >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 0x7fda000b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:42.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9ff7fe700 1 --2- 172.21.15.148:0/794608839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159be0 0x7fda00068db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda0006f9b0 tx=0x7fd9f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18330f171fbc5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:42.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9fe7fc700 1 -- 172.21.15.148:0/794608839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000e070 con 0x7fda00159be0 2023-10-10T20:20:42.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9fe7fc700 1 -- 172.21.15.148:0/794608839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000ba10 con 0x7fda00159be0 2023-10-10T20:20:42.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.964+0000 7fd9fe7fc700 1 -- 172.21.15.148:0/794608839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00092e0 con 0x7fda00159be0 2023-10-10T20:20:42.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 -- 172.21.15.148:0/794608839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159be0 msgr2=0x7fda00068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:42.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 --2- 172.21.15.148:0/794608839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159be0 0x7fda00068db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda0006f9b0 tx=0x7fd9f0009840 comp rx=0 tx=0).stop 2023-10-10T20:20:42.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 -- 172.21.15.148:0/794608839 shutdown_connections 2023-10-10T20:20:42.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 --2- 172.21.15.148:0/794608839 >> v2:172.21.15.161:3300/0 conn(0x7fda000cf060 0x7fda000b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 --2- 172.21.15.148:0/794608839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159be0 0x7fda00068db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 -- 172.21.15.148:0/794608839 >> 172.21.15.148:0/794608839 conn(0x7fda0006bc70 msgr2=0x7fda0006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:42.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 -- 172.21.15.148:0/794608839 shutdown_connections 2023-10-10T20:20:42.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.965+0000 7fda049a0700 1 -- 172.21.15.148:0/794608839 wait complete. 2023-10-10T20:20:42.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.966+0000 7fda049a0700 1 Processor -- start 2023-10-10T20:20:42.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.966+0000 7fda049a0700 1 -- start start 2023-10-10T20:20:42.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda000cf060 0x7fda00158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda00159be0 0x7fda00159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000c6d10 con 0x7fda00159fb0 2023-10-10T20:20:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000b2670 con 0x7fda00159be0 2023-10-10T20:20:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000b2950 con 0x7fda000cf060 2023-10-10T20:20:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53390/0 (socket says 172.21.15.148:53390) 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 -- 172.21.15.148:0/645116209 learned_addr learned my addr 172.21.15.148:0/645116209 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9feffd700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda00159be0 0x7fda00159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9ff7fe700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda000cf060 0x7fda00158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 -- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda000cf060 msgr2=0x7fda00158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda000cf060 0x7fda00158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 -- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda00159be0 msgr2=0x7fda00159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda00159be0 0x7fda00159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 -- 172.21.15.148:0/645116209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0009ae0 con 0x7fda00159fb0 2023-10-10T20:20:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9fffff700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c850 tx=0x7fd9f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011600 con 0x7fda00159fb0 2023-10-10T20:20:42.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8011760 con 0x7fda00159fb0 2023-10-10T20:20:42.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda000b2c30 con 0x7fda00159fb0 2023-10-10T20:20:42.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011b20 con 0x7fda00159fb0 2023-10-10T20:20:42.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.967+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001582f0 con 0x7fda00159fb0 2023-10-10T20:20:42.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.968+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda000c6ec0 con 0x7fda00159fb0 2023-10-10T20:20:42.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.969+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9f8018350 con 0x7fda00159fb0 2023-10-10T20:20:42.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.969+0000 7fd9e77fe700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 0x7fd9ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:42.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.969+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9f80842d0 con 0x7fda00159fb0 2023-10-10T20:20:42.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.969+0000 7fd9ff7fe700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 0x7fd9ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:42.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.970+0000 7fd9ff7fe700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 0x7fd9ec0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009880 tx=0x7fd9f0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:42.974+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda000c6ec0 con 0x7fda00159fb0 2023-10-10T20:20:43.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.196+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda000c6e50 con 0x7fda00159fb0 2023-10-10T20:20:43.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.197+0000 7fd9e77fe700 1 -- 172.21.15.148:0/645116209 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fd9f8054fd0 con 0x7fda00159fb0 2023-10-10T20:20:43.198 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:43.198 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:20:43.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 msgr2=0x7fd9ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 0x7fd9ec0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009880 tx=0x7fd9f0017000 comp rx=0 tx=0).stop 2023-10-10T20:20:43.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 msgr2=0x7fda000b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c850 tx=0x7fd9f8008040 comp rx=0 tx=0).stop 2023-10-10T20:20:43.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 shutdown_connections 2023-10-10T20:20:43.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd9ec066f30 0x7fd9ec0693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda00159fb0 0x7fda000b1900 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda000cf060 0x7fda00158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 --2- 172.21.15.148:0/645116209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda00159be0 0x7fda00159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 >> 172.21.15.148:0/645116209 conn(0x7fda0006bc70 msgr2=0x7fda0015c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:43.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 shutdown_connections 2023-10-10T20:20:43.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.199+0000 7fda049a0700 1 -- 172.21.15.148:0/645116209 wait complete. 2023-10-10T20:20:43.202 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:20:43.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:20:43.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.326+0000 7f777d2b3700 1 Processor -- start 2023-10-10T20:20:43.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.326+0000 7f777d2b3700 1 -- start start 2023-10-10T20:20:43.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f777d2b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f777815a620 0x7f777815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f777d2b3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f777815d580 0x7f7778158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f777d2b3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777815cbb0 con 0x7f7778157870 2023-10-10T20:20:43.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f777d2b3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777815bdb0 con 0x7f777815a620 2023-10-10T20:20:43.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f777d2b3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777815bfa0 con 0x7f777815d580 2023-10-10T20:20:43.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f777815a620 0x7f777815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f7776ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7778157870 0x7f777815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60794/0 (socket says 172.21.15.148:60794) 2023-10-10T20:20:43.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f7776ffd700 1 -- 172.21.15.148:0/2474695372 learned_addr learned my addr 172.21.15.148:0/2474695372 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:43.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f77777fe700 1 --2- 172.21.15.148:0/2474695372 >> v2:172.21.15.161:3300/0 conn(0x7f777815d580 0x7f7778158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 -- 172.21.15.148:0/2474695372 >> v2:172.21.15.161:3300/0 conn(0x7f777815d580 msgr2=0x7f7778158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:43.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 --2- 172.21.15.148:0/2474695372 >> v2:172.21.15.161:3300/0 conn(0x7f777815d580 0x7f7778158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 -- 172.21.15.148:0/2474695372 >> v1:172.21.15.148:6789/0 conn(0x7f7778157870 legacy=0x7f777815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f77767fc700 1 -- 172.21.15.148:0/2474695372 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408070298 0 0) 0x7f777815cbb0 con 0x7f7778157870 2023-10-10T20:20:43.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 -- 172.21.15.148:0/2474695372 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777815c280 con 0x7f777815a620 2023-10-10T20:20:43.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.327+0000 7f776e7fc700 1 --2- 172.21.15.148:0/2474695372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815a620 0x7f777815d040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f777806b4e0 tx=0x7f77600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb84029ec1ac779 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f77767fc700 1 -- 172.21.15.148:0/2474695372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000b6b0 con 0x7f777815a620 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f77767fc700 1 -- 172.21.15.148:0/2474695372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776000b810 con 0x7f777815a620 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f77767fc700 1 -- 172.21.15.148:0/2474695372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000bbb0 con 0x7f777815a620 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 -- 172.21.15.148:0/2474695372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815a620 msgr2=0x7f777815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 --2- 172.21.15.148:0/2474695372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815a620 0x7f777815d040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f777806b4e0 tx=0x7f77600091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 -- 172.21.15.148:0/2474695372 shutdown_connections 2023-10-10T20:20:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 --2- 172.21.15.148:0/2474695372 >> v2:172.21.15.161:3300/0 conn(0x7f777815d580 0x7f7778158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 --2- 172.21.15.148:0/2474695372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815a620 0x7f777815d040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 -- 172.21.15.148:0/2474695372 >> 172.21.15.148:0/2474695372 conn(0x7f7778068ce0 msgr2=0x7f77780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 -- 172.21.15.148:0/2474695372 shutdown_connections 2023-10-10T20:20:43.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.328+0000 7f777d2b3700 1 -- 172.21.15.148:0/2474695372 wait complete. 2023-10-10T20:20:43.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.329+0000 7f777d2b3700 1 Processor -- start 2023-10-10T20:20:43.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.329+0000 7f777d2b3700 1 -- start start 2023-10-10T20:20:43.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7778157870 0x7f77780cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777815a620 0x7f777814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778151030 con 0x7f777815d580 2023-10-10T20:20:43.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778151310 con 0x7f777815a620 2023-10-10T20:20:43.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f777d2b3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77781515f0 con 0x7f7778157870 2023-10-10T20:20:43.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f77777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f77777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53414/0 (socket says 172.21.15.148:53414) 2023-10-10T20:20:43.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f77777fe700 1 -- 172.21.15.148:0/1300044415 learned_addr learned my addr 172.21.15.148:0/1300044415 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:43.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f7776ffd700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7778157870 0x7f77780cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.330+0000 7f776e7fc700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777815a620 0x7f777814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 -- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7778157870 msgr2=0x7f77780cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7778157870 0x7f77780cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 -- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777815a620 msgr2=0x7f777814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777815a620 0x7f777814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 -- 172.21.15.148:0/1300044415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7760009420 con 0x7f777815d580 2023-10-10T20:20:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f77777fe700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7770007ef0 tx=0x7f7770009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777000eb30 con 0x7f777815d580 2023-10-10T20:20:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777000ec90 con 0x7f777815d580 2023-10-10T20:20:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77781518d0 con 0x7f777815d580 2023-10-10T20:20:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770014670 con 0x7f777815d580 2023-10-10T20:20:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.331+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77780d01d0 con 0x7f777815d580 2023-10-10T20:20:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.332+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7778151df0 con 0x7f777815d580 2023-10-10T20:20:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.332+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7770014a60 con 0x7f777815d580 2023-10-10T20:20:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.333+0000 7f776f7fe700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 0x7f77580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.333+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7770084ad0 con 0x7f777815d580 2023-10-10T20:20:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.333+0000 7f7776ffd700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 0x7f77580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.333+0000 7f7776ffd700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 0x7f77580693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7768007fd0 tx=0x7f7768007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.337+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7778151df0 con 0x7f777815d580 2023-10-10T20:20:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.518+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7778151df0 con 0x7f777815d580 2023-10-10T20:20:43.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.519+0000 7f776f7fe700 1 -- 172.21.15.148:0/1300044415 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7778151df0 con 0x7f777815d580 2023-10-10T20:20:43.521 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:43.521 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":45,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:20:40.806624+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:20:43.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.521+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 msgr2=0x7f77580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.521+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 0x7f77580693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7768007fd0 tx=0x7f7768007d30 comp rx=0 tx=0).stop 2023-10-10T20:20:43.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.521+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 msgr2=0x7f77780b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.521+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7770007ef0 tx=0x7f7770009840 comp rx=0 tx=0).stop 2023-10-10T20:20:43.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 shutdown_connections 2023-10-10T20:20:43.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7758066f30 0x7f77580693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f777815d580 0x7f77780b5400 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7778157870 0x7f77780cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 --2- 172.21.15.148:0/1300044415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777815a620 0x7f777814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 >> 172.21.15.148:0/1300044415 conn(0x7f7778068ce0 msgr2=0x7f777815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:43.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 shutdown_connections 2023-10-10T20:20:43.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.522+0000 7f777d2b3700 1 -- 172.21.15.148:0/1300044415 wait complete. 2023-10-10T20:20:43.535 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:20:43.535 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:20:43.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.649+0000 7fc5598b8700 1 Processor -- start 2023-10-10T20:20:43.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.649+0000 7fc5598b8700 1 -- start start 2023-10-10T20:20:43.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5598b8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc55415cbd0 0x7fc55415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5598b8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540cb7c0 con 0x7fc5540cd6e0 2023-10-10T20:20:43.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5598b8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55415e290 con 0x7fc5540cdab0 2023-10-10T20:20:43.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5598b8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55415e850 con 0x7fc55415cbd0 2023-10-10T20:20:43.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5588b6700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc5540cd6e0 0x7fc5540cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60802/0 (socket says 172.21.15.148:60802) 2023-10-10T20:20:43.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5588b6700 1 -- 172.21.15.148:0/1747814038 learned_addr learned my addr 172.21.15.148:0/1747814038 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:43.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc5590b7700 1 --2- 172.21.15.148:0/1747814038 >> v2:172.21.15.161:3300/0 conn(0x7fc55415cbd0 0x7fc55415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc54b7fe700 1 -- 172.21.15.148:0/1747814038 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392482132 0 0) 0x7fc5540cb7c0 con 0x7fc5540cd6e0 2023-10-10T20:20:43.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.650+0000 7fc54b7fe700 1 -- 172.21.15.148:0/1747814038 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc53c003570 con 0x7fc5540cd6e0 2023-10-10T20:20:43.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5590b7700 1 -- 172.21.15.148:0/1747814038 >> v1:172.21.15.161:6789/0 conn(0x7fc5540cdab0 legacy=0x7fc55415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5590b7700 1 -- 172.21.15.148:0/1747814038 >> v1:172.21.15.148:6789/0 conn(0x7fc5540cd6e0 legacy=0x7fc5540cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5590b7700 1 -- 172.21.15.148:0/1747814038 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55415e570 con 0x7fc55415cbd0 2023-10-10T20:20:43.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5590b7700 1 --2- 172.21.15.148:0/1747814038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc55415cbd0 0x7fc55415d050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009e60 tx=0x7fc54c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da437a059ef5c81c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc54b7fe700 1 -- 172.21.15.148:0/1747814038 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00e070 con 0x7fc55415cbd0 2023-10-10T20:20:43.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc54b7fe700 1 -- 172.21.15.148:0/1747814038 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c00bb60 con 0x7fc55415cbd0 2023-10-10T20:20:43.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc54b7fe700 1 -- 172.21.15.148:0/1747814038 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c012430 con 0x7fc55415cbd0 2023-10-10T20:20:43.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5598b8700 1 -- 172.21.15.148:0/1747814038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc55415cbd0 msgr2=0x7fc55415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5598b8700 1 --2- 172.21.15.148:0/1747814038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc55415cbd0 0x7fc55415d050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009e60 tx=0x7fc54c009780 comp rx=0 tx=0).stop 2023-10-10T20:20:43.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5598b8700 1 -- 172.21.15.148:0/1747814038 shutdown_connections 2023-10-10T20:20:43.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5598b8700 1 --2- 172.21.15.148:0/1747814038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc55415cbd0 0x7fc55415d050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.651+0000 7fc5598b8700 1 -- 172.21.15.148:0/1747814038 >> 172.21.15.148:0/1747814038 conn(0x7fc554067660 msgr2=0x7fc554067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:43.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.652+0000 7fc5598b8700 1 -- 172.21.15.148:0/1747814038 shutdown_connections 2023-10-10T20:20:43.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.652+0000 7fc5598b8700 1 -- 172.21.15.148:0/1747814038 wait complete. 2023-10-10T20:20:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 Processor -- start 2023-10-10T20:20:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 -- start start 2023-10-10T20:20:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5540cd6e0 0x7fc55416a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5540cdab0 0x7fc5540c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540cc360 con 0x7fc55415cbd0 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b46c0 con 0x7fc5540cd6e0 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.653+0000 7fc5598b8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540ca170 con 0x7fc5540cdab0 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53430/0 (socket says 172.21.15.148:53430) 2023-10-10T20:20:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 -- 172.21.15.148:0/2165379726 learned_addr learned my addr 172.21.15.148:0/2165379726 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5588b6700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5540cd6e0 0x7fc55416a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc54bfff700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5540cdab0 0x7fc5540c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 -- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5540cdab0 msgr2=0x7fc5540c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5540cdab0 0x7fc5540c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 -- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5540cd6e0 msgr2=0x7fc55416a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5540cd6e0 0x7fc55416a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 -- 172.21.15.148:0/2165379726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009a20 con 0x7fc55415cbd0 2023-10-10T20:20:43.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5590b7700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc5540c7b10 tx=0x7fc54c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c027470 con 0x7fc55415cbd0 2023-10-10T20:20:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c0275f0 con 0x7fc55415cbd0 2023-10-10T20:20:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540b3ab0 con 0x7fc55415cbd0 2023-10-10T20:20:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0279b0 con 0x7fc55415cbd0 2023-10-10T20:20:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.654+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540b4070 con 0x7fc55415cbd0 2023-10-10T20:20:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.655+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5540b4870 con 0x7fc55415cbd0 2023-10-10T20:20:43.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.656+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc54c024070 con 0x7fc55415cbd0 2023-10-10T20:20:43.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.656+0000 7fc548ff9700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 0x7fc5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:43.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.656+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc54c086e80 con 0x7fc55415cbd0 2023-10-10T20:20:43.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.656+0000 7fc5588b6700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 0x7fc5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:43.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.656+0000 7fc5588b6700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 0x7fc5400693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc550007fd0 tx=0x7fc550009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:43.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.660+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5540b4870 con 0x7fc55415cbd0 2023-10-10T20:20:43.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.840+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5540b4830 con 0x7fc540066f30 2023-10-10T20:20:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.849+0000 7fc548ff9700 1 -- 172.21.15.148:0/2165379726 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284804 (secure 0 0 0) 0x7fc5540b4830 con 0x7fc540066f30 2023-10-10T20:20:43.853 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 msgr2=0x7fc5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 0x7fc5400693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc550007fd0 tx=0x7fc550009040 comp rx=0 tx=0).stop 2023-10-10T20:20:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 msgr2=0x7fc5540c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc5540c7b10 tx=0x7fc54c019000 comp rx=0 tx=0).stop 2023-10-10T20:20:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 shutdown_connections 2023-10-10T20:20:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc540066f30 0x7fc5400693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc55415cbd0 0x7fc5540c9c30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5540cd6e0 0x7fc55416a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 --2- 172.21.15.148:0/2165379726 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5540cdab0 0x7fc5540c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 >> 172.21.15.148:0/2165379726 conn(0x7fc554067660 msgr2=0x7fc55415d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 shutdown_connections 2023-10-10T20:20:43.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:43.852+0000 7fc5598b8700 1 -- 172.21.15.148:0/2165379726 wait complete. 2023-10-10T20:20:43.858 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:20:43.866 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-10-10T20:20:43.339320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2812,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559687,"statfs":{"total":1600000000,"available":1597119562,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.538726"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812493+0000","last_change":"2023-10-10T20:20:39.812414+0000","last_active":"2023-10-10T20:20:39.812493+0000","last_peered":"2023-10-10T20:20:39.812493+0000","last_clean":"2023-10-10T20:20:39.812493+0000","last_became_active":"2023-10-10T20:20:39.812172+0000","last_became_peered":"2023-10-10T20:20:39.812172+0000","last_unstale":"2023-10-10T20:20:39.812493+0000","last_undegraded":"2023-10-10T20:20:39.812493+0000","last_fullsized":"2023-10-10T20:20:39.812493+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823712+0000","last_change":"2023-10-10T20:20:39.821705+0000","last_active":"2023-10-10T20:20:39.823712+0000","last_peered":"2023-10-10T20:20:39.823712+0000","last_clean":"2023-10-10T20:20:39.823712+0000","last_became_active":"2023-10-10T20:20:39.821453+0000","last_became_peered":"2023-10-10T20:20:39.821453+0000","last_unstale":"2023-10-10T20:20:39.823712+0000","last_undegraded":"2023-10-10T20:20:39.823712+0000","last_fullsized":"2023-10-10T20:20:39.823712+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814512+0000","last_change":"2023-10-10T20:20:39.813745+0000","last_active":"2023-10-10T20:20:39.814512+0000","last_peered":"2023-10-10T20:20:39.814512+0000","last_clean":"2023-10-10T20:20:39.814512+0000","last_became_active":"2023-10-10T20:20:39.813176+0000","last_became_peered":"2023-10-10T20:20:39.813176+0000","last_unstale":"2023-10-10T20:20:39.814512+0000","last_undegraded":"2023-10-10T20:20:39.814512+0000","last_fullsized":"2023-10-10T20:20:39.814512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:14.381937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824627+0000","last_change":"2023-10-10T20:20:39.824567+0000","last_active":"2023-10-10T20:20:39.824627+0000","last_peered":"2023-10-10T20:20:39.824627+0000","last_clean":"2023-10-10T20:20:39.824627+0000","last_became_active":"2023-10-10T20:20:39.824358+0000","last_became_peered":"2023-10-10T20:20:39.824358+0000","last_unstale":"2023-10-10T20:20:39.824627+0000","last_undegraded":"2023-10-10T20:20:39.824627+0000","last_fullsized":"2023-10-10T20:20:39.824627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:07.616379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816217+0000","last_change":"2023-10-10T20:20:39.816141+0000","last_active":"2023-10-10T20:20:39.816217+0000","last_peered":"2023-10-10T20:20:39.816217+0000","last_clean":"2023-10-10T20:20:39.816217+0000","last_became_active":"2023-10-10T20:20:39.815940+0000","last_became_peered":"2023-10-10T20:20:39.815940+0000","last_unstale":"2023-10-10T20:20:39.816217+0000","last_undegraded":"2023-10-10T20:20:39.816217+0000","last_fullsized":"2023-10-10T20:20:39.816217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820765+0000","last_change":"2023-10-10T20:20:39.820666+0000","last_active":"2023-10-10T20:20:39.820765+0000","last_peered":"2023-10-10T20:20:39.820765+0000","last_clean":"2023-10-10T20:20:39.820765+0000","last_became_active":"2023-10-10T20:20:39.820299+0000","last_became_peered":"2023-10-10T20:20:39.820299+0000","last_unstale":"2023-10-10T20:20:39.820765+0000","last_undegraded":"2023-10-10T20:20:39.820765+0000","last_fullsized":"2023-10-10T20:20:39.820765+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:09.184793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816079+0000","last_change":"2023-10-10T20:20:39.816009+0000","last_active":"2023-10-10T20:20:39.816079+0000","last_peered":"2023-10-10T20:20:39.816079+0000","last_clean":"2023-10-10T20:20:39.816079+0000","last_became_active":"2023-10-10T20:20:39.815777+0000","last_became_peered":"2023-10-10T20:20:39.815777+0000","last_unstale":"2023-10-10T20:20:39.816079+0000","last_undegraded":"2023-10-10T20:20:39.816079+0000","last_fullsized":"2023-10-10T20:20:39.816079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822566+0000","last_change":"2023-10-10T20:20:39.822513+0000","last_active":"2023-10-10T20:20:39.822566+0000","last_peered":"2023-10-10T20:20:39.822566+0000","last_clean":"2023-10-10T20:20:39.822566+0000","last_became_active":"2023-10-10T20:20:39.822356+0000","last_became_peered":"2023-10-10T20:20:39.822356+0000","last_unstale":"2023-10-10T20:20:39.822566+0000","last_undegraded":"2023-10-10T20:20:39.822566+0000","last_fullsized":"2023-10-10T20:20:39.822566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.815586+0000","last_change":"2023-10-10T20:20:39.815515+0000","last_active":"2023-10-10T20:20:39.815586+0000","last_peered":"2023-10-10T20:20:39.815586+0000","last_clean":"2023-10-10T20:20:39.815586+0000","last_became_active":"2023-10-10T20:20:39.815284+0000","last_became_peered":"2023-10-10T20:20:39.815284+0000","last_unstale":"2023-10-10T20:20:39.815586+0000","last_undegraded":"2023-10-10T20:20:39.815586+0000","last_fullsized":"2023-10-10T20:20:39.815586+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822058+0000","last_change":"2023-10-10T20:20:39.821174+0000","last_active":"2023-10-10T20:20:39.822058+0000","last_peered":"2023-10-10T20:20:39.822058+0000","last_clean":"2023-10-10T20:20:39.822058+0000","last_became_active":"2023-10-10T20:20:39.819531+0000","last_became_peered":"2023-10-10T20:20:39.819531+0000","last_unstale":"2023-10-10T20:20:39.822058+0000","last_undegraded":"2023-10-10T20:20:39.822058+0000","last_fullsized":"2023-10-10T20:20:39.822058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818336+0000","last_change":"2023-10-10T20:20:39.818264+0000","last_active":"2023-10-10T20:20:39.818336+0000","last_peered":"2023-10-10T20:20:39.818336+0000","last_clean":"2023-10-10T20:20:39.818336+0000","last_became_active":"2023-10-10T20:20:39.818043+0000","last_became_peered":"2023-10-10T20:20:39.818043+0000","last_unstale":"2023-10-10T20:20:39.818336+0000","last_undegraded":"2023-10-10T20:20:39.818336+0000","last_fullsized":"2023-10-10T20:20:39.818336+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:09.232095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818733+0000","last_change":"2023-10-10T20:20:39.818198+0000","last_active":"2023-10-10T20:20:39.818733+0000","last_peered":"2023-10-10T20:20:39.818733+0000","last_clean":"2023-10-10T20:20:39.818733+0000","last_became_active":"2023-10-10T20:20:39.817952+0000","last_became_peered":"2023-10-10T20:20:39.817952+0000","last_unstale":"2023-10-10T20:20:39.818733+0000","last_undegraded":"2023-10-10T20:20:39.818733+0000","last_fullsized":"2023-10-10T20:20:39.818733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816554+0000","last_change":"2023-10-10T20:20:39.816134+0000","last_active":"2023-10-10T20:20:39.816554+0000","last_peered":"2023-10-10T20:20:39.816554+0000","last_clean":"2023-10-10T20:20:39.816554+0000","last_became_active":"2023-10-10T20:20:39.815348+0000","last_became_peered":"2023-10-10T20:20:39.815348+0000","last_unstale":"2023-10-10T20:20:39.816554+0000","last_undegraded":"2023-10-10T20:20:39.816554+0000","last_fullsized":"2023-10-10T20:20:39.816554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:05.124451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818165+0000","last_change":"2023-10-10T20:20:39.818109+0000","last_active":"2023-10-10T20:20:39.818165+0000","last_peered":"2023-10-10T20:20:39.818165+0000","last_clean":"2023-10-10T20:20:39.818165+0000","last_became_active":"2023-10-10T20:20:39.817885+0000","last_became_peered":"2023-10-10T20:20:39.817885+0000","last_unstale":"2023-10-10T20:20:39.818165+0000","last_undegraded":"2023-10-10T20:20:39.818165+0000","last_fullsized":"2023-10-10T20:20:39.818165+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817530+0000","last_change":"2023-10-10T20:20:39.817457+0000","last_active":"2023-10-10T20:20:39.817530+0000","last_peered":"2023-10-10T20:20:39.817530+0000","last_clean":"2023-10-10T20:20:39.817530+0000","last_became_active":"2023-10-10T20:20:39.817200+0000","last_became_peered":"2023-10-10T20:20:39.817200+0000","last_unstale":"2023-10-10T20:20:39.817530+0000","last_undegraded":"2023-10-10T20:20:39.817530+0000","last_fullsized":"2023-10-10T20:20:39.817530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:48.978431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821250+0000","last_change":"2023-10-10T20:20:39.820022+0000","last_active":"2023-10-10T20:20:39.821250+0000","last_peered":"2023-10-10T20:20:39.821250+0000","last_clean":"2023-10-10T20:20:39.821250+0000","last_became_active":"2023-10-10T20:20:39.819048+0000","last_became_peered":"2023-10-10T20:20:39.819048+0000","last_unstale":"2023-10-10T20:20:39.821250+0000","last_undegraded":"2023-10-10T20:20:39.821250+0000","last_fullsized":"2023-10-10T20:20:39.821250+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:30.861156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816121+0000","last_change":"2023-10-10T20:20:39.816047+0000","last_active":"2023-10-10T20:20:39.816121+0000","last_peered":"2023-10-10T20:20:39.816121+0000","last_clean":"2023-10-10T20:20:39.816121+0000","last_became_active":"2023-10-10T20:20:39.815443+0000","last_became_peered":"2023-10-10T20:20:39.815443+0000","last_unstale":"2023-10-10T20:20:39.816121+0000","last_undegraded":"2023-10-10T20:20:39.816121+0000","last_fullsized":"2023-10-10T20:20:39.816121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814533+0000","last_change":"2023-10-10T20:20:39.813889+0000","last_active":"2023-10-10T20:20:39.814533+0000","last_peered":"2023-10-10T20:20:39.814533+0000","last_clean":"2023-10-10T20:20:39.814533+0000","last_became_active":"2023-10-10T20:20:39.813502+0000","last_became_peered":"2023-10-10T20:20:39.813502+0000","last_unstale":"2023-10-10T20:20:39.814533+0000","last_undegraded":"2023-10-10T20:20:39.814533+0000","last_fullsized":"2023-10-10T20:20:39.814533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824932+0000","last_change":"2023-10-10T20:20:39.824820+0000","last_active":"2023-10-10T20:20:39.824932+0000","last_peered":"2023-10-10T20:20:39.824932+0000","last_clean":"2023-10-10T20:20:39.824932+0000","last_became_active":"2023-10-10T20:20:39.824380+0000","last_became_peered":"2023-10-10T20:20:39.824380+0000","last_unstale":"2023-10-10T20:20:39.824932+0000","last_undegraded":"2023-10-10T20:20:39.824932+0000","last_fullsized":"2023-10-10T20:20:39.824932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:08.662779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814772+0000","last_change":"2023-10-10T20:20:39.814713+0000","last_active":"2023-10-10T20:20:39.814772+0000","last_peered":"2023-10-10T20:20:39.814772+0000","last_clean":"2023-10-10T20:20:39.814772+0000","last_became_active":"2023-10-10T20:20:39.814489+0000","last_became_peered":"2023-10-10T20:20:39.814489+0000","last_unstale":"2023-10-10T20:20:39.814772+0000","last_undegraded":"2023-10-10T20:20:39.814772+0000","last_fullsized":"2023-10-10T20:20:39.814772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:20.579237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821881+0000","last_change":"2023-10-10T20:20:39.821830+0000","last_active":"2023-10-10T20:20:39.821881+0000","last_peered":"2023-10-10T20:20:39.821881+0000","last_clean":"2023-10-10T20:20:39.821881+0000","last_became_active":"2023-10-10T20:20:39.821646+0000","last_became_peered":"2023-10-10T20:20:39.821646+0000","last_unstale":"2023-10-10T20:20:39.821881+0000","last_undegraded":"2023-10-10T20:20:39.821881+0000","last_fullsized":"2023-10-10T20:20:39.821881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:53.098641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822095+0000","last_change":"2023-10-10T20:20:39.821360+0000","last_active":"2023-10-10T20:20:39.822095+0000","last_peered":"2023-10-10T20:20:39.822095+0000","last_clean":"2023-10-10T20:20:39.822095+0000","last_became_active":"2023-10-10T20:20:39.819892+0000","last_became_peered":"2023-10-10T20:20:39.819892+0000","last_unstale":"2023-10-10T20:20:39.822095+0000","last_undegraded":"2023-10-10T20:20:39.822095+0000","last_fullsized":"2023-10-10T20:20:39.822095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822567+0000","last_change":"2023-10-10T20:20:39.821941+0000","last_active":"2023-10-10T20:20:39.822567+0000","last_peered":"2023-10-10T20:20:39.822567+0000","last_clean":"2023-10-10T20:20:39.822567+0000","last_became_active":"2023-10-10T20:20:39.821693+0000","last_became_peered":"2023-10-10T20:20:39.821693+0000","last_unstale":"2023-10-10T20:20:39.822567+0000","last_undegraded":"2023-10-10T20:20:39.822567+0000","last_fullsized":"2023-10-10T20:20:39.822567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824910+0000","last_change":"2023-10-10T20:20:39.824638+0000","last_active":"2023-10-10T20:20:39.824910+0000","last_peered":"2023-10-10T20:20:39.824910+0000","last_clean":"2023-10-10T20:20:39.824910+0000","last_became_active":"2023-10-10T20:20:39.823415+0000","last_became_peered":"2023-10-10T20:20:39.823415+0000","last_unstale":"2023-10-10T20:20:39.824910+0000","last_undegraded":"2023-10-10T20:20:39.824910+0000","last_fullsized":"2023-10-10T20:20:39.824910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821162+0000","last_change":"2023-10-10T20:20:39.818814+0000","last_active":"2023-10-10T20:20:39.821162+0000","last_peered":"2023-10-10T20:20:39.821162+0000","last_clean":"2023-10-10T20:20:39.821162+0000","last_became_active":"2023-10-10T20:20:39.818336+0000","last_became_peered":"2023-10-10T20:20:39.818336+0000","last_unstale":"2023-10-10T20:20:39.821162+0000","last_undegraded":"2023-10-10T20:20:39.821162+0000","last_fullsized":"2023-10-10T20:20:39.821162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:06.927011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.813932+0000","last_change":"2023-10-10T20:20:39.813876+0000","last_active":"2023-10-10T20:20:39.813932+0000","last_peered":"2023-10-10T20:20:39.813932+0000","last_clean":"2023-10-10T20:20:39.813932+0000","last_became_active":"2023-10-10T20:20:39.813618+0000","last_became_peered":"2023-10-10T20:20:39.813618+0000","last_unstale":"2023-10-10T20:20:39.813932+0000","last_undegraded":"2023-10-10T20:20:39.813932+0000","last_fullsized":"2023-10-10T20:20:39.813932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:32.078076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816271+0000","last_change":"2023-10-10T20:20:39.816205+0000","last_active":"2023-10-10T20:20:39.816271+0000","last_peered":"2023-10-10T20:20:39.816271+0000","last_clean":"2023-10-10T20:20:39.816271+0000","last_became_active":"2023-10-10T20:20:39.815789+0000","last_became_peered":"2023-10-10T20:20:39.815789+0000","last_unstale":"2023-10-10T20:20:39.816271+0000","last_undegraded":"2023-10-10T20:20:39.816271+0000","last_fullsized":"2023-10-10T20:20:39.816271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:31.459651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817685+0000","last_change":"2023-10-10T20:20:39.817582+0000","last_active":"2023-10-10T20:20:39.817685+0000","last_peered":"2023-10-10T20:20:39.817685+0000","last_clean":"2023-10-10T20:20:39.817685+0000","last_became_active":"2023-10-10T20:20:39.817303+0000","last_became_peered":"2023-10-10T20:20:39.817303+0000","last_unstale":"2023-10-10T20:20:39.817685+0000","last_undegraded":"2023-10-10T20:20:39.817685+0000","last_fullsized":"2023-10-10T20:20:39.817685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:07.329623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823800+0000","last_change":"2023-10-10T20:20:39.823646+0000","last_active":"2023-10-10T20:20:39.823800+0000","last_peered":"2023-10-10T20:20:39.823800+0000","last_clean":"2023-10-10T20:20:39.823800+0000","last_became_active":"2023-10-10T20:20:39.822151+0000","last_became_peered":"2023-10-10T20:20:39.822151+0000","last_unstale":"2023-10-10T20:20:39.823800+0000","last_undegraded":"2023-10-10T20:20:39.823800+0000","last_fullsized":"2023-10-10T20:20:39.823800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:31.459651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811905+0000","last_change":"2023-10-10T20:20:39.811320+0000","last_active":"2023-10-10T20:20:39.811905+0000","last_peered":"2023-10-10T20:20:39.811905+0000","last_clean":"2023-10-10T20:20:39.811905+0000","last_became_active":"2023-10-10T20:20:39.811081+0000","last_became_peered":"2023-10-10T20:20:39.811081+0000","last_unstale":"2023-10-10T20:20:39.811905+0000","last_undegraded":"2023-10-10T20:20:39.811905+0000","last_fullsized":"2023-10-10T20:20:39.811905+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817836+0000","last_change":"2023-10-10T20:20:39.817552+0000","last_active":"2023-10-10T20:20:39.817836+0000","last_peered":"2023-10-10T20:20:39.817836+0000","last_clean":"2023-10-10T20:20:39.817836+0000","last_became_active":"2023-10-10T20:20:39.817177+0000","last_became_peered":"2023-10-10T20:20:39.817177+0000","last_unstale":"2023-10-10T20:20:39.817836+0000","last_undegraded":"2023-10-10T20:20:39.817836+0000","last_fullsized":"2023-10-10T20:20:39.817836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:58.719801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817807+0000","last_change":"2023-10-10T20:20:39.817714+0000","last_active":"2023-10-10T20:20:39.817807+0000","last_peered":"2023-10-10T20:20:39.817807+0000","last_clean":"2023-10-10T20:20:39.817807+0000","last_became_active":"2023-10-10T20:20:39.817518+0000","last_became_peered":"2023-10-10T20:20:39.817518+0000","last_unstale":"2023-10-10T20:20:39.817807+0000","last_undegraded":"2023-10-10T20:20:39.817807+0000","last_fullsized":"2023-10-10T20:20:39.817807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:31.748344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819451+0000","last_change":"2023-10-10T20:20:39.819145+0000","last_active":"2023-10-10T20:20:39.819451+0000","last_peered":"2023-10-10T20:20:39.819451+0000","last_clean":"2023-10-10T20:20:39.819451+0000","last_became_active":"2023-10-10T20:20:39.818932+0000","last_became_peered":"2023-10-10T20:20:39.818932+0000","last_unstale":"2023-10-10T20:20:39.819451+0000","last_undegraded":"2023-10-10T20:20:39.819451+0000","last_fullsized":"2023-10-10T20:20:39.819451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:30.861156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830546+0000","last_change":"2023-10-10T20:20:38.816726+0000","last_active":"2023-10-10T20:20:39.830546+0000","last_peered":"2023-10-10T20:20:39.830546+0000","last_clean":"2023-10-10T20:20:39.830546+0000","last_became_active":"2023-10-10T20:20:38.816569+0000","last_became_peered":"2023-10-10T20:20:38.816569+0000","last_unstale":"2023-10-10T20:20:39.830546+0000","last_undegraded":"2023-10-10T20:20:39.830546+0000","last_fullsized":"2023-10-10T20:20:39.830546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:36.874602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821265+0000","last_change":"2023-10-10T20:20:39.820750+0000","last_active":"2023-10-10T20:20:39.821265+0000","last_peered":"2023-10-10T20:20:39.821265+0000","last_clean":"2023-10-10T20:20:39.821265+0000","last_became_active":"2023-10-10T20:20:39.820516+0000","last_became_peered":"2023-10-10T20:20:39.820516+0000","last_unstale":"2023-10-10T20:20:39.821265+0000","last_undegraded":"2023-10-10T20:20:39.821265+0000","last_fullsized":"2023-10-10T20:20:39.821265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:34.812174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830719+0000","last_change":"2023-10-10T20:20:38.817703+0000","last_active":"2023-10-10T20:20:39.830719+0000","last_peered":"2023-10-10T20:20:39.830719+0000","last_clean":"2023-10-10T20:20:39.830719+0000","last_became_active":"2023-10-10T20:20:38.817457+0000","last_became_peered":"2023-10-10T20:20:38.817457+0000","last_unstale":"2023-10-10T20:20:39.830719+0000","last_undegraded":"2023-10-10T20:20:39.830719+0000","last_fullsized":"2023-10-10T20:20:39.830719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:31.071477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818074+0000","last_change":"2023-10-10T20:20:39.817995+0000","last_active":"2023-10-10T20:20:39.818074+0000","last_peered":"2023-10-10T20:20:39.818074+0000","last_clean":"2023-10-10T20:20:39.818074+0000","last_became_active":"2023-10-10T20:20:39.817711+0000","last_became_peered":"2023-10-10T20:20:39.817711+0000","last_unstale":"2023-10-10T20:20:39.818074+0000","last_undegraded":"2023-10-10T20:20:39.818074+0000","last_fullsized":"2023-10-10T20:20:39.818074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829125+0000","last_change":"2023-10-10T20:20:38.813109+0000","last_active":"2023-10-10T20:20:39.829125+0000","last_peered":"2023-10-10T20:20:39.829125+0000","last_clean":"2023-10-10T20:20:39.829125+0000","last_became_active":"2023-10-10T20:20:38.812564+0000","last_became_peered":"2023-10-10T20:20:38.812564+0000","last_unstale":"2023-10-10T20:20:39.829125+0000","last_undegraded":"2023-10-10T20:20:39.829125+0000","last_fullsized":"2023-10-10T20:20:39.829125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:11.059562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820624+0000","last_change":"2023-10-10T20:20:39.820570+0000","last_active":"2023-10-10T20:20:39.820624+0000","last_peered":"2023-10-10T20:20:39.820624+0000","last_clean":"2023-10-10T20:20:39.820624+0000","last_became_active":"2023-10-10T20:20:39.820120+0000","last_became_peered":"2023-10-10T20:20:39.820120+0000","last_unstale":"2023-10-10T20:20:39.820624+0000","last_undegraded":"2023-10-10T20:20:39.820624+0000","last_fullsized":"2023-10-10T20:20:39.820624+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:28.149085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829992+0000","last_change":"2023-10-10T20:20:38.815786+0000","last_active":"2023-10-10T20:20:39.829992+0000","last_peered":"2023-10-10T20:20:39.829992+0000","last_clean":"2023-10-10T20:20:39.829992+0000","last_became_active":"2023-10-10T20:20:38.815555+0000","last_became_peered":"2023-10-10T20:20:38.815555+0000","last_unstale":"2023-10-10T20:20:39.829992+0000","last_undegraded":"2023-10-10T20:20:39.829992+0000","last_fullsized":"2023-10-10T20:20:39.829992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:07.656181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822244+0000","last_change":"2023-10-10T20:20:39.822170+0000","last_active":"2023-10-10T20:20:39.822244+0000","last_peered":"2023-10-10T20:20:39.822244+0000","last_clean":"2023-10-10T20:20:39.822244+0000","last_became_active":"2023-10-10T20:20:39.821757+0000","last_became_peered":"2023-10-10T20:20:39.821757+0000","last_unstale":"2023-10-10T20:20:39.822244+0000","last_undegraded":"2023-10-10T20:20:39.822244+0000","last_fullsized":"2023-10-10T20:20:39.822244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:52.659983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.828737+0000","last_change":"2023-10-10T20:20:38.816117+0000","last_active":"2023-10-10T20:20:39.828737+0000","last_peered":"2023-10-10T20:20:39.828737+0000","last_clean":"2023-10-10T20:20:39.828737+0000","last_became_active":"2023-10-10T20:20:38.815574+0000","last_became_peered":"2023-10-10T20:20:38.815574+0000","last_unstale":"2023-10-10T20:20:39.828737+0000","last_undegraded":"2023-10-10T20:20:39.828737+0000","last_fullsized":"2023-10-10T20:20:39.828737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:30.611160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814981+0000","last_change":"2023-10-10T20:20:39.814758+0000","last_active":"2023-10-10T20:20:39.814981+0000","last_peered":"2023-10-10T20:20:39.814981+0000","last_clean":"2023-10-10T20:20:39.814981+0000","last_became_active":"2023-10-10T20:20:39.814070+0000","last_became_peered":"2023-10-10T20:20:39.814070+0000","last_unstale":"2023-10-10T20:20:39.814981+0000","last_undegraded":"2023-10-10T20:20:39.814981+0000","last_fullsized":"2023-10-10T20:20:39.814981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:14.381937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829644+0000","last_change":"2023-10-10T20:20:38.812003+0000","last_active":"2023-10-10T20:20:39.829644+0000","last_peered":"2023-10-10T20:20:39.829644+0000","last_clean":"2023-10-10T20:20:39.829644+0000","last_became_active":"2023-10-10T20:20:38.811701+0000","last_became_peered":"2023-10-10T20:20:38.811701+0000","last_unstale":"2023-10-10T20:20:39.829644+0000","last_undegraded":"2023-10-10T20:20:39.829644+0000","last_fullsized":"2023-10-10T20:20:39.829644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:46.562248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822080+0000","last_change":"2023-10-10T20:20:39.821308+0000","last_active":"2023-10-10T20:20:39.822080+0000","last_peered":"2023-10-10T20:20:39.822080+0000","last_clean":"2023-10-10T20:20:39.822080+0000","last_became_active":"2023-10-10T20:20:39.819845+0000","last_became_peered":"2023-10-10T20:20:39.819845+0000","last_unstale":"2023-10-10T20:20:39.822080+0000","last_undegraded":"2023-10-10T20:20:39.822080+0000","last_fullsized":"2023-10-10T20:20:39.822080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814432+0000","last_change":"2023-10-10T20:20:38.812248+0000","last_active":"2023-10-10T20:20:39.814432+0000","last_peered":"2023-10-10T20:20:39.814432+0000","last_clean":"2023-10-10T20:20:39.814432+0000","last_became_active":"2023-10-10T20:20:38.811996+0000","last_became_peered":"2023-10-10T20:20:38.811996+0000","last_unstale":"2023-10-10T20:20:39.814432+0000","last_undegraded":"2023-10-10T20:20:39.814432+0000","last_fullsized":"2023-10-10T20:20:39.814432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:27.142486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824125+0000","last_change":"2023-10-10T20:20:39.824051+0000","last_active":"2023-10-10T20:20:39.824125+0000","last_peered":"2023-10-10T20:20:39.824125+0000","last_clean":"2023-10-10T20:20:39.824125+0000","last_became_active":"2023-10-10T20:20:39.822431+0000","last_became_peered":"2023-10-10T20:20:39.822431+0000","last_unstale":"2023-10-10T20:20:39.824125+0000","last_undegraded":"2023-10-10T20:20:39.824125+0000","last_fullsized":"2023-10-10T20:20:39.824125+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810642+0000","last_change":"2023-10-10T20:20:38.817447+0000","last_active":"2023-10-10T20:20:39.810642+0000","last_peered":"2023-10-10T20:20:39.810642+0000","last_clean":"2023-10-10T20:20:39.810642+0000","last_became_active":"2023-10-10T20:20:38.815962+0000","last_became_peered":"2023-10-10T20:20:38.815962+0000","last_unstale":"2023-10-10T20:20:39.810642+0000","last_undegraded":"2023-10-10T20:20:39.810642+0000","last_fullsized":"2023-10-10T20:20:39.810642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:19.572639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814760+0000","last_change":"2023-10-10T20:20:39.814564+0000","last_active":"2023-10-10T20:20:39.814760+0000","last_peered":"2023-10-10T20:20:39.814760+0000","last_clean":"2023-10-10T20:20:39.814760+0000","last_became_active":"2023-10-10T20:20:39.814300+0000","last_became_peered":"2023-10-10T20:20:39.814300+0000","last_unstale":"2023-10-10T20:20:39.814760+0000","last_undegraded":"2023-10-10T20:20:39.814760+0000","last_fullsized":"2023-10-10T20:20:39.814760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:34.812174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.808231+0000","last_change":"2023-10-10T20:20:38.812705+0000","last_active":"2023-10-10T20:20:39.808231+0000","last_peered":"2023-10-10T20:20:39.808231+0000","last_clean":"2023-10-10T20:20:39.808231+0000","last_became_active":"2023-10-10T20:20:38.812480+0000","last_became_peered":"2023-10-10T20:20:38.812480+0000","last_unstale":"2023-10-10T20:20:39.808231+0000","last_undegraded":"2023-10-10T20:20:39.808231+0000","last_fullsized":"2023-10-10T20:20:39.808231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:20:20.226518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816839+0000","last_change":"2023-10-10T20:20:39.816768+0000","last_active":"2023-10-10T20:20:39.816839+0000","last_peered":"2023-10-10T20:20:39.816839+0000","last_clean":"2023-10-10T20:20:39.816839+0000","last_became_active":"2023-10-10T20:20:39.816489+0000","last_became_peered":"2023-10-10T20:20:39.816489+0000","last_unstale":"2023-10-10T20:20:39.816839+0000","last_undegraded":"2023-10-10T20:20:39.816839+0000","last_fullsized":"2023-10-10T20:20:39.816839+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810087+0000","last_change":"2023-10-10T20:20:38.815263+0000","last_active":"2023-10-10T20:20:39.810087+0000","last_peered":"2023-10-10T20:20:39.810087+0000","last_clean":"2023-10-10T20:20:39.810087+0000","last_became_active":"2023-10-10T20:20:38.815045+0000","last_became_peered":"2023-10-10T20:20:38.815045+0000","last_unstale":"2023-10-10T20:20:39.810087+0000","last_undegraded":"2023-10-10T20:20:39.810087+0000","last_fullsized":"2023-10-10T20:20:39.810087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:04.117852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820966+0000","last_change":"2023-10-10T20:20:39.820914+0000","last_active":"2023-10-10T20:20:39.820966+0000","last_peered":"2023-10-10T20:20:39.820966+0000","last_clean":"2023-10-10T20:20:39.820966+0000","last_became_active":"2023-10-10T20:20:39.819866+0000","last_became_peered":"2023-10-10T20:20:39.819866+0000","last_unstale":"2023-10-10T20:20:39.820966+0000","last_undegraded":"2023-10-10T20:20:39.820966+0000","last_fullsized":"2023-10-10T20:20:39.820966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:30.593449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830297+0000","last_change":"2023-10-10T20:20:38.818792+0000","last_active":"2023-10-10T20:20:39.830297+0000","last_peered":"2023-10-10T20:20:39.830297+0000","last_clean":"2023-10-10T20:20:39.830297+0000","last_became_active":"2023-10-10T20:20:38.818582+0000","last_became_peered":"2023-10-10T20:20:38.818582+0000","last_unstale":"2023-10-10T20:20:39.830297+0000","last_undegraded":"2023-10-10T20:20:39.830297+0000","last_fullsized":"2023-10-10T20:20:39.830297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:05.920412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811572+0000","last_change":"2023-10-10T20:20:39.811500+0000","last_active":"2023-10-10T20:20:39.811572+0000","last_peered":"2023-10-10T20:20:39.811572+0000","last_clean":"2023-10-10T20:20:39.811572+0000","last_became_active":"2023-10-10T20:20:39.811258+0000","last_became_peered":"2023-10-10T20:20:39.811258+0000","last_unstale":"2023-10-10T20:20:39.811572+0000","last_undegraded":"2023-10-10T20:20:39.811572+0000","last_fullsized":"2023-10-10T20:20:39.811572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.831791+0000","last_change":"2023-10-10T20:20:38.810394+0000","last_active":"2023-10-10T20:20:39.831791+0000","last_peered":"2023-10-10T20:20:39.831791+0000","last_clean":"2023-10-10T20:20:39.831791+0000","last_became_active":"2023-10-10T20:20:38.810028+0000","last_became_peered":"2023-10-10T20:20:38.810028+0000","last_unstale":"2023-10-10T20:20:39.831791+0000","last_undegraded":"2023-10-10T20:20:39.831791+0000","last_fullsized":"2023-10-10T20:20:39.831791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:20.237910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821234+0000","last_change":"2023-10-10T20:20:39.819864+0000","last_active":"2023-10-10T20:20:39.821234+0000","last_peered":"2023-10-10T20:20:39.821234+0000","last_clean":"2023-10-10T20:20:39.821234+0000","last_became_active":"2023-10-10T20:20:39.818712+0000","last_became_peered":"2023-10-10T20:20:39.818712+0000","last_unstale":"2023-10-10T20:20:39.821234+0000","last_undegraded":"2023-10-10T20:20:39.821234+0000","last_fullsized":"2023-10-10T20:20:39.821234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:06.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830253+0000","last_change":"2023-10-10T20:20:38.818732+0000","last_active":"2023-10-10T20:20:39.830253+0000","last_peered":"2023-10-10T20:20:39.830253+0000","last_clean":"2023-10-10T20:20:39.830253+0000","last_became_active":"2023-10-10T20:20:38.818533+0000","last_became_peered":"2023-10-10T20:20:38.818533+0000","last_unstale":"2023-10-10T20:20:39.830253+0000","last_undegraded":"2023-10-10T20:20:39.830253+0000","last_fullsized":"2023-10-10T20:20:39.830253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:13.375338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816269+0000","last_change":"2023-10-10T20:20:39.816122+0000","last_active":"2023-10-10T20:20:39.816269+0000","last_peered":"2023-10-10T20:20:39.816269+0000","last_clean":"2023-10-10T20:20:39.816269+0000","last_became_active":"2023-10-10T20:20:39.815778+0000","last_became_peered":"2023-10-10T20:20:39.815778+0000","last_unstale":"2023-10-10T20:20:39.816269+0000","last_undegraded":"2023-10-10T20:20:39.816269+0000","last_fullsized":"2023-10-10T20:20:39.816269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:42.377091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.809717+0000","last_change":"2023-10-10T20:20:38.816241+0000","last_active":"2023-10-10T20:20:39.809717+0000","last_peered":"2023-10-10T20:20:39.809717+0000","last_clean":"2023-10-10T20:20:39.809717+0000","last_became_active":"2023-10-10T20:20:38.816001+0000","last_became_peered":"2023-10-10T20:20:38.816001+0000","last_unstale":"2023-10-10T20:20:39.809717+0000","last_undegraded":"2023-10-10T20:20:39.809717+0000","last_fullsized":"2023-10-10T20:20:39.809717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:33.805575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820029+0000","last_change":"2023-10-10T20:20:39.818951+0000","last_active":"2023-10-10T20:20:39.820029+0000","last_peered":"2023-10-10T20:20:39.820029+0000","last_clean":"2023-10-10T20:20:39.820029+0000","last_became_active":"2023-10-10T20:20:39.818748+0000","last_became_peered":"2023-10-10T20:20:39.818748+0000","last_unstale":"2023-10-10T20:20:39.820029+0000","last_undegraded":"2023-10-10T20:20:39.820029+0000","last_fullsized":"2023-10-10T20:20:39.820029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:17.124275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812586+0000","last_change":"2023-10-10T20:20:38.815937+0000","last_active":"2023-10-10T20:20:39.812586+0000","last_peered":"2023-10-10T20:20:39.812586+0000","last_clean":"2023-10-10T20:20:39.812586+0000","last_became_active":"2023-10-10T20:20:38.815241+0000","last_became_peered":"2023-10-10T20:20:38.815241+0000","last_unstale":"2023-10-10T20:20:39.812586+0000","last_undegraded":"2023-10-10T20:20:39.812586+0000","last_fullsized":"2023-10-10T20:20:39.812586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:06.323024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823119+0000","last_change":"2023-10-10T20:20:39.823043+0000","last_active":"2023-10-10T20:20:39.823119+0000","last_peered":"2023-10-10T20:20:39.823119+0000","last_clean":"2023-10-10T20:20:39.823119+0000","last_became_active":"2023-10-10T20:20:39.822809+0000","last_became_peered":"2023-10-10T20:20:39.822809+0000","last_unstale":"2023-10-10T20:20:39.823119+0000","last_undegraded":"2023-10-10T20:20:39.823119+0000","last_fullsized":"2023-10-10T20:20:39.823119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:42.377091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830852+0000","last_change":"2023-10-10T20:20:38.812192+0000","last_active":"2023-10-10T20:20:39.830852+0000","last_peered":"2023-10-10T20:20:39.830852+0000","last_clean":"2023-10-10T20:20:39.830852+0000","last_became_active":"2023-10-10T20:20:38.812031+0000","last_became_peered":"2023-10-10T20:20:38.812031+0000","last_unstale":"2023-10-10T20:20:39.830852+0000","last_undegraded":"2023-10-10T20:20:39.830852+0000","last_fullsized":"2023-10-10T20:20:39.830852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:32:36.757262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816533+0000","last_change":"2023-10-10T20:20:39.815991+0000","last_active":"2023-10-10T20:20:39.816533+0000","last_peered":"2023-10-10T20:20:39.816533+0000","last_clean":"2023-10-10T20:20:39.816533+0000","last_became_active":"2023-10-10T20:20:39.815776+0000","last_became_peered":"2023-10-10T20:20:39.815776+0000","last_unstale":"2023-10-10T20:20:39.816533+0000","last_undegraded":"2023-10-10T20:20:39.816533+0000","last_fullsized":"2023-10-10T20:20:39.816533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:32:37.763860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829599+0000","last_change":"2023-10-10T20:20:38.813549+0000","last_active":"2023-10-10T20:20:39.829599+0000","last_peered":"2023-10-10T20:20:39.829599+0000","last_clean":"2023-10-10T20:20:39.829599+0000","last_became_active":"2023-10-10T20:20:38.813331+0000","last_became_peered":"2023-10-10T20:20:38.813331+0000","last_unstale":"2023-10-10T20:20:39.829599+0000","last_undegraded":"2023-10-10T20:20:39.829599+0000","last_fullsized":"2023-10-10T20:20:39.829599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:59.778093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818962+0000","last_change":"2023-10-10T20:20:39.818782+0000","last_active":"2023-10-10T20:20:39.818962+0000","last_peered":"2023-10-10T20:20:39.818962+0000","last_clean":"2023-10-10T20:20:39.818962+0000","last_became_active":"2023-10-10T20:20:39.818542+0000","last_became_peered":"2023-10-10T20:20:39.818542+0000","last_unstale":"2023-10-10T20:20:39.818962+0000","last_undegraded":"2023-10-10T20:20:39.818962+0000","last_fullsized":"2023-10-10T20:20:39.818962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:07.329623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829998+0000","last_change":"2023-10-10T20:20:38.810569+0000","last_active":"2023-10-10T20:20:39.829998+0000","last_peered":"2023-10-10T20:20:39.829998+0000","last_clean":"2023-10-10T20:20:39.829998+0000","last_became_active":"2023-10-10T20:20:38.810329+0000","last_became_peered":"2023-10-10T20:20:38.810329+0000","last_unstale":"2023-10-10T20:20:39.829998+0000","last_undegraded":"2023-10-10T20:20:39.829998+0000","last_fullsized":"2023-10-10T20:20:39.829998+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:08.007313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822585+0000","last_change":"2023-10-10T20:20:39.822138+0000","last_active":"2023-10-10T20:20:39.822585+0000","last_peered":"2023-10-10T20:20:39.822585+0000","last_clean":"2023-10-10T20:20:39.822585+0000","last_became_active":"2023-10-10T20:20:39.821920+0000","last_became_peered":"2023-10-10T20:20:39.821920+0000","last_unstale":"2023-10-10T20:20:39.822585+0000","last_undegraded":"2023-10-10T20:20:39.822585+0000","last_fullsized":"2023-10-10T20:20:39.822585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:13.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.828746+0000","last_change":"2023-10-10T20:20:38.809908+0000","last_active":"2023-10-10T20:20:39.828746+0000","last_peered":"2023-10-10T20:20:39.828746+0000","last_clean":"2023-10-10T20:20:39.828746+0000","last_became_active":"2023-10-10T20:20:38.809662+0000","last_became_peered":"2023-10-10T20:20:38.809662+0000","last_unstale":"2023-10-10T20:20:39.828746+0000","last_undegraded":"2023-10-10T20:20:39.828746+0000","last_fullsized":"2023-10-10T20:20:39.828746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:30.453052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821216+0000","last_change":"2023-10-10T20:20:39.819713+0000","last_active":"2023-10-10T20:20:39.821216+0000","last_peered":"2023-10-10T20:20:39.821216+0000","last_clean":"2023-10-10T20:20:39.821216+0000","last_became_active":"2023-10-10T20:20:39.818433+0000","last_became_peered":"2023-10-10T20:20:39.818433+0000","last_unstale":"2023-10-10T20:20:39.821216+0000","last_undegraded":"2023-10-10T20:20:39.821216+0000","last_fullsized":"2023-10-10T20:20:39.821216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:28.149085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812674+0000","last_change":"2023-10-10T20:20:38.816704+0000","last_active":"2023-10-10T20:20:39.812674+0000","last_peered":"2023-10-10T20:20:39.812674+0000","last_clean":"2023-10-10T20:20:39.812674+0000","last_became_active":"2023-10-10T20:20:38.816484+0000","last_became_peered":"2023-10-10T20:20:38.816484+0000","last_unstale":"2023-10-10T20:20:39.812674+0000","last_undegraded":"2023-10-10T20:20:39.812674+0000","last_fullsized":"2023-10-10T20:20:39.812674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:12.896902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821032+0000","last_change":"2023-10-10T20:20:39.821000+0000","last_active":"2023-10-10T20:20:39.821032+0000","last_peered":"2023-10-10T20:20:39.821032+0000","last_clean":"2023-10-10T20:20:39.821032+0000","last_became_active":"2023-10-10T20:20:39.820340+0000","last_became_peered":"2023-10-10T20:20:39.820340+0000","last_unstale":"2023-10-10T20:20:39.821032+0000","last_undegraded":"2023-10-10T20:20:39.821032+0000","last_fullsized":"2023-10-10T20:20:39.821032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:10:37.019625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829787+0000","last_change":"2023-10-10T20:20:38.815415+0000","last_active":"2023-10-10T20:20:39.829787+0000","last_peered":"2023-10-10T20:20:39.829787+0000","last_clean":"2023-10-10T20:20:39.829787+0000","last_became_active":"2023-10-10T20:20:38.815165+0000","last_became_peered":"2023-10-10T20:20:38.815165+0000","last_unstale":"2023-10-10T20:20:39.829787+0000","last_undegraded":"2023-10-10T20:20:39.829787+0000","last_fullsized":"2023-10-10T20:20:39.829787+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:25.401834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819431+0000","last_change":"2023-10-10T20:20:39.817747+0000","last_active":"2023-10-10T20:20:39.819431+0000","last_peered":"2023-10-10T20:20:39.819431+0000","last_clean":"2023-10-10T20:20:39.819431+0000","last_became_active":"2023-10-10T20:20:39.817526+0000","last_became_peered":"2023-10-10T20:20:39.817526+0000","last_unstale":"2023-10-10T20:20:39.819431+0000","last_undegraded":"2023-10-10T20:20:39.819431+0000","last_fullsized":"2023-10-10T20:20:39.819431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:06.927011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814531+0000","last_change":"2023-10-10T20:20:38.816842+0000","last_active":"2023-10-10T20:20:39.814531+0000","last_peered":"2023-10-10T20:20:39.814531+0000","last_clean":"2023-10-10T20:20:39.814531+0000","last_became_active":"2023-10-10T20:20:38.816590+0000","last_became_peered":"2023-10-10T20:20:38.816590+0000","last_unstale":"2023-10-10T20:20:39.814531+0000","last_undegraded":"2023-10-10T20:20:39.814531+0000","last_fullsized":"2023-10-10T20:20:39.814531+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:29.854558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811552+0000","last_change":"2023-10-10T20:20:39.811480+0000","last_active":"2023-10-10T20:20:39.811552+0000","last_peered":"2023-10-10T20:20:39.811552+0000","last_clean":"2023-10-10T20:20:39.811552+0000","last_became_active":"2023-10-10T20:20:39.811249+0000","last_became_peered":"2023-10-10T20:20:39.811249+0000","last_unstale":"2023-10-10T20:20:39.811552+0000","last_undegraded":"2023-10-10T20:20:39.811552+0000","last_fullsized":"2023-10-10T20:20:39.811552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:50.016672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810539+0000","last_change":"2023-10-10T20:20:38.817607+0000","last_active":"2023-10-10T20:20:39.810539+0000","last_peered":"2023-10-10T20:20:39.810539+0000","last_clean":"2023-10-10T20:20:39.810539+0000","last_became_active":"2023-10-10T20:20:38.817217+0000","last_became_peered":"2023-10-10T20:20:38.817217+0000","last_unstale":"2023-10-10T20:20:39.810539+0000","last_undegraded":"2023-10-10T20:20:39.810539+0000","last_fullsized":"2023-10-10T20:20:39.810539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:31.071477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822024+0000","last_change":"2023-10-10T20:20:39.819298+0000","last_active":"2023-10-10T20:20:39.822024+0000","last_peered":"2023-10-10T20:20:39.822024+0000","last_clean":"2023-10-10T20:20:39.822024+0000","last_became_active":"2023-10-10T20:20:39.819047+0000","last_became_peered":"2023-10-10T20:20:39.819047+0000","last_unstale":"2023-10-10T20:20:39.822024+0000","last_undegraded":"2023-10-10T20:20:39.822024+0000","last_fullsized":"2023-10-10T20:20:39.822024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.809803+0000","last_change":"2023-10-10T20:20:38.817415+0000","last_active":"2023-10-10T20:20:39.809803+0000","last_peered":"2023-10-10T20:20:39.809803+0000","last_clean":"2023-10-10T20:20:39.809803+0000","last_became_active":"2023-10-10T20:20:38.817184+0000","last_became_peered":"2023-10-10T20:20:38.817184+0000","last_unstale":"2023-10-10T20:20:39.809803+0000","last_undegraded":"2023-10-10T20:20:39.809803+0000","last_fullsized":"2023-10-10T20:20:39.809803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:30.453052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821182+0000","last_change":"2023-10-10T20:20:39.819290+0000","last_active":"2023-10-10T20:20:39.821182+0000","last_peered":"2023-10-10T20:20:39.821182+0000","last_clean":"2023-10-10T20:20:39.821182+0000","last_became_active":"2023-10-10T20:20:39.818853+0000","last_became_peered":"2023-10-10T20:20:39.818853+0000","last_unstale":"2023-10-10T20:20:39.821182+0000","last_undegraded":"2023-10-10T20:20:39.821182+0000","last_fullsized":"2023-10-10T20:20:39.821182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:05.531228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.813632+0000","last_change":"2023-10-10T20:20:38.814741+0000","last_active":"2023-10-10T20:20:39.813632+0000","last_peered":"2023-10-10T20:20:39.813632+0000","last_clean":"2023-10-10T20:20:39.813632+0000","last_became_active":"2023-10-10T20:20:38.814492+0000","last_became_peered":"2023-10-10T20:20:38.814492+0000","last_unstale":"2023-10-10T20:20:39.813632+0000","last_undegraded":"2023-10-10T20:20:39.813632+0000","last_fullsized":"2023-10-10T20:20:39.813632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:51.653384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816251+0000","last_change":"2023-10-10T20:20:39.815991+0000","last_active":"2023-10-10T20:20:39.816251+0000","last_peered":"2023-10-10T20:20:39.816251+0000","last_clean":"2023-10-10T20:20:39.816251+0000","last_became_active":"2023-10-10T20:20:39.815490+0000","last_became_peered":"2023-10-10T20:20:39.815490+0000","last_unstale":"2023-10-10T20:20:39.816251+0000","last_undegraded":"2023-10-10T20:20:39.816251+0000","last_fullsized":"2023-10-10T20:20:39.816251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810248+0000","last_change":"2023-10-10T20:20:38.818559+0000","last_active":"2023-10-10T20:20:39.810248+0000","last_peered":"2023-10-10T20:20:39.810248+0000","last_clean":"2023-10-10T20:20:39.810248+0000","last_became_active":"2023-10-10T20:20:38.818304+0000","last_became_peered":"2023-10-10T20:20:38.818304+0000","last_unstale":"2023-10-10T20:20:39.810248+0000","last_undegraded":"2023-10-10T20:20:39.810248+0000","last_fullsized":"2023-10-10T20:20:39.810248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:18.737399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817803+0000","last_change":"2023-10-10T20:20:39.817734+0000","last_active":"2023-10-10T20:20:39.817803+0000","last_peered":"2023-10-10T20:20:39.817803+0000","last_clean":"2023-10-10T20:20:39.817803+0000","last_became_active":"2023-10-10T20:20:39.817510+0000","last_became_peered":"2023-10-10T20:20:39.817510+0000","last_unstale":"2023-10-10T20:20:39.817803+0000","last_undegraded":"2023-10-10T20:20:39.817803+0000","last_fullsized":"2023-10-10T20:20:39.817803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810433+0000","last_change":"2023-10-10T20:20:38.813231+0000","last_active":"2023-10-10T20:20:39.810433+0000","last_peered":"2023-10-10T20:20:39.810433+0000","last_clean":"2023-10-10T20:20:39.810433+0000","last_became_active":"2023-10-10T20:20:38.812830+0000","last_became_peered":"2023-10-10T20:20:38.812830+0000","last_unstale":"2023-10-10T20:20:39.810433+0000","last_undegraded":"2023-10-10T20:20:39.810433+0000","last_fullsized":"2023-10-10T20:20:39.810433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:20:20.226518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820063+0000","last_change":"2023-10-10T20:20:39.819111+0000","last_active":"2023-10-10T20:20:39.820063+0000","last_peered":"2023-10-10T20:20:39.820063+0000","last_clean":"2023-10-10T20:20:39.820063+0000","last_became_active":"2023-10-10T20:20:39.818514+0000","last_became_peered":"2023-10-10T20:20:39.818514+0000","last_unstale":"2023-10-10T20:20:39.820063+0000","last_undegraded":"2023-10-10T20:20:39.820063+0000","last_fullsized":"2023-10-10T20:20:39.820063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:16.384587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810713+0000","last_change":"2023-10-10T20:20:38.818604+0000","last_active":"2023-10-10T20:20:39.810713+0000","last_peered":"2023-10-10T20:20:39.810713+0000","last_clean":"2023-10-10T20:20:39.810713+0000","last_became_active":"2023-10-10T20:20:38.818328+0000","last_became_peered":"2023-10-10T20:20:38.818328+0000","last_unstale":"2023-10-10T20:20:39.810713+0000","last_undegraded":"2023-10-10T20:20:39.810713+0000","last_fullsized":"2023-10-10T20:20:39.810713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:27.142486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817513+0000","last_change":"2023-10-10T20:20:39.817433+0000","last_active":"2023-10-10T20:20:39.817513+0000","last_peered":"2023-10-10T20:20:39.817513+0000","last_clean":"2023-10-10T20:20:39.817513+0000","last_became_active":"2023-10-10T20:20:39.816707+0000","last_became_peered":"2023-10-10T20:20:39.816707+0000","last_unstale":"2023-10-10T20:20:39.817513+0000","last_undegraded":"2023-10-10T20:20:39.817513+0000","last_fullsized":"2023-10-10T20:20:39.817513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:03.750215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811780+0000","last_change":"2023-10-10T20:20:38.814577+0000","last_active":"2023-10-10T20:20:39.811780+0000","last_peered":"2023-10-10T20:20:39.811780+0000","last_clean":"2023-10-10T20:20:39.811780+0000","last_became_active":"2023-10-10T20:20:38.814344+0000","last_became_peered":"2023-10-10T20:20:38.814344+0000","last_unstale":"2023-10-10T20:20:39.811780+0000","last_undegraded":"2023-10-10T20:20:39.811780+0000","last_fullsized":"2023-10-10T20:20:39.811780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:19.572639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819100+0000","last_change":"2023-10-10T20:20:39.819046+0000","last_active":"2023-10-10T20:20:39.819100+0000","last_peered":"2023-10-10T20:20:39.819100+0000","last_clean":"2023-10-10T20:20:39.819100+0000","last_became_active":"2023-10-10T20:20:39.818908+0000","last_became_peered":"2023-10-10T20:20:39.818908+0000","last_unstale":"2023-10-10T20:20:39.819100+0000","last_undegraded":"2023-10-10T20:20:39.819100+0000","last_fullsized":"2023-10-10T20:20:39.819100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810383+0000","last_change":"2023-10-10T20:20:38.817907+0000","last_active":"2023-10-10T20:20:39.810383+0000","last_peered":"2023-10-10T20:20:39.810383+0000","last_clean":"2023-10-10T20:20:39.810383+0000","last_became_active":"2023-10-10T20:20:38.817683+0000","last_became_peered":"2023-10-10T20:20:38.817683+0000","last_unstale":"2023-10-10T20:20:39.810383+0000","last_undegraded":"2023-10-10T20:20:39.810383+0000","last_fullsized":"2023-10-10T20:20:39.810383+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:08.007313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816746+0000","last_change":"2023-10-10T20:20:39.816655+0000","last_active":"2023-10-10T20:20:39.816746+0000","last_peered":"2023-10-10T20:20:39.816746+0000","last_clean":"2023-10-10T20:20:39.816746+0000","last_became_active":"2023-10-10T20:20:39.816371+0000","last_became_peered":"2023-10-10T20:20:39.816371+0000","last_unstale":"2023-10-10T20:20:39.816746+0000","last_undegraded":"2023-10-10T20:20:39.816746+0000","last_fullsized":"2023-10-10T20:20:39.816746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:13.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810160+0000","last_change":"2023-10-10T20:20:38.814651+0000","last_active":"2023-10-10T20:20:39.810160+0000","last_peered":"2023-10-10T20:20:39.810160+0000","last_clean":"2023-10-10T20:20:39.810160+0000","last_became_active":"2023-10-10T20:20:38.814072+0000","last_became_peered":"2023-10-10T20:20:38.814072+0000","last_unstale":"2023-10-10T20:20:39.810160+0000","last_undegraded":"2023-10-10T20:20:39.810160+0000","last_fullsized":"2023-10-10T20:20:39.810160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:11.756984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818149+0000","last_change":"2023-10-10T20:20:39.818091+0000","last_active":"2023-10-10T20:20:39.818149+0000","last_peered":"2023-10-10T20:20:39.818149+0000","last_clean":"2023-10-10T20:20:39.818149+0000","last_became_active":"2023-10-10T20:20:39.817850+0000","last_became_peered":"2023-10-10T20:20:39.817850+0000","last_unstale":"2023-10-10T20:20:39.818149+0000","last_undegraded":"2023-10-10T20:20:39.818149+0000","last_fullsized":"2023-10-10T20:20:39.818149+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:52.659983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829062+0000","last_change":"2023-10-10T20:20:38.812366+0000","last_active":"2023-10-10T20:20:39.829062+0000","last_peered":"2023-10-10T20:20:39.829062+0000","last_clean":"2023-10-10T20:20:39.829062+0000","last_became_active":"2023-10-10T20:20:38.812137+0000","last_became_peered":"2023-10-10T20:20:38.812137+0000","last_unstale":"2023-10-10T20:20:39.829062+0000","last_undegraded":"2023-10-10T20:20:39.829062+0000","last_fullsized":"2023-10-10T20:20:39.829062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:07.656181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810338+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:20:39.810338+0000","last_peered":"2023-10-10T20:20:39.810338+0000","last_clean":"2023-10-10T20:20:39.810338+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:20:39.810338+0000","last_undegraded":"2023-10-10T20:20:39.810338+0000","last_fullsized":"2023-10-10T20:20:39.810338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194533,"statfs":{"total":200000000,"available":199202119,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.371},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.27500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194528,"statfs":{"total":200000000,"available":199196898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":9.7260000000000009,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.747},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785742,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.23400000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.249},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.38},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.309},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.504},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784753,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.63200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.753},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786219,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.63}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995}]}]}],"pool_statfs":[]}} 2023-10-10T20:20:43.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:20:44.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.020+0000 7f4311f0f700 1 Processor -- start 2023-10-10T20:20:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.020+0000 7f4311f0f700 1 -- start start 2023-10-10T20:20:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f4311f0f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c0a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f4311f0f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f4311f0f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430c0b0790 con 0x7f430c0c9320 2023-10-10T20:20:44.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f4311f0f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430c0a7df0 con 0x7f430c0c96f0 2023-10-10T20:20:44.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f4311f0f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0c9b70 con 0x7f430c0b0c20 2023-10-10T20:20:44.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f430bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f430b7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f430c0c9320 0x7f430c15c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60806/0 (socket says 172.21.15.148:60806) 2023-10-10T20:20:44.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f430b7fe700 1 -- 172.21.15.148:0/2546873238 learned_addr learned my addr 172.21.15.148:0/2546873238 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:44.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.021+0000 7f430affd700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c0a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430bfff700 1 -- 172.21.15.148:0/2546873238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 msgr2=0x7f430c0a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:44.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430bfff700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c0a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430bfff700 1 -- 172.21.15.148:0/2546873238 >> v1:172.21.15.148:6789/0 conn(0x7f430c0c9320 legacy=0x7f430c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430a7fc700 1 -- 172.21.15.148:0/2546873238 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866765384 0 0) 0x7f430c0b0790 con 0x7f430c0c9320 2023-10-10T20:20:44.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430bfff700 1 -- 172.21.15.148:0/2546873238 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0a7b10 con 0x7f430c0c96f0 2023-10-10T20:20:44.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430affd700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c0a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:44.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430bfff700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4304009910 tx=0x7f43040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c1527b8f175e10 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430a7fc700 1 -- 172.21.15.148:0/2546873238 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400b820 con 0x7f430c0c96f0 2023-10-10T20:20:44.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430a7fc700 1 -- 172.21.15.148:0/2546873238 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430400b980 con 0x7f430c0c96f0 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f430a7fc700 1 -- 172.21.15.148:0/2546873238 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400bd20 con 0x7f430c0c96f0 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 -- 172.21.15.148:0/2546873238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c96f0 msgr2=0x7f430c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4304009910 tx=0x7f43040091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 -- 172.21.15.148:0/2546873238 shutdown_connections 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 --2- 172.21.15.148:0/2546873238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c0a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 -- 172.21.15.148:0/2546873238 >> 172.21.15.148:0/2546873238 conn(0x7f430c0cd1e0 msgr2=0x7f430c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:44.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 -- 172.21.15.148:0/2546873238 shutdown_connections 2023-10-10T20:20:44.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.022+0000 7f4311f0f700 1 -- 172.21.15.148:0/2546873238 wait complete. 2023-10-10T20:20:44.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 Processor -- start 2023-10-10T20:20:44.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 -- start start 2023-10-10T20:20:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c099d70 con 0x7f430c0c9320 2023-10-10T20:20:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0a9b70 con 0x7f430c0c96f0 2023-10-10T20:20:44.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.024+0000 7f4311f0f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0a9890 con 0x7f430c0b0c20 2023-10-10T20:20:44.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53454/0 (socket says 172.21.15.148:53454) 2023-10-10T20:20:44.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 -- 172.21.15.148:0/547216879 learned_addr learned my addr 172.21.15.148:0/547216879 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:44.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430b7fe700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430bfff700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 -- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 msgr2=0x7f430c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 -- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f430c0c96f0 msgr2=0x7f430c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 -- 172.21.15.148:0/547216879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009420 con 0x7f430c0c9320 2023-10-10T20:20:44.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f430affd700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f430000c850 tx=0x7f4300008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300011600 con 0x7f430c0c9320 2023-10-10T20:20:44.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300011760 con 0x7f430c0c9320 2023-10-10T20:20:44.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c099720 con 0x7f430c0c9320 2023-10-10T20:20:44.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300011b00 con 0x7f430c0c9320 2023-10-10T20:20:44.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.025+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c099ab0 con 0x7f430c0c9320 2023-10-10T20:20:44.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.026+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c04c3c0 con 0x7f430c0c9320 2023-10-10T20:20:44.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.026+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4300018350 con 0x7f430c0c9320 2023-10-10T20:20:44.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.027+0000 7f42f37fe700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 0x7f42f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.027+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4300084380 con 0x7f430c0c9320 2023-10-10T20:20:44.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.027+0000 7f430b7fe700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 0x7f42f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.027+0000 7f430b7fe700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 0x7f42f80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f42fc002250 tx=0x7f42fc001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.031+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4300055060 con 0x7f430c0c9320 2023-10-10T20:20:44.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.209+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f430c0b0140 con 0x7f42f8066f30 2023-10-10T20:20:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.216+0000 7f42f37fe700 1 -- 172.21.15.148:0/547216879 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284804 (secure 0 0 0) 0x7f430c0b0140 con 0x7f42f8066f30 2023-10-10T20:20:44.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 msgr2=0x7f42f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 0x7f42f80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f42fc002250 tx=0x7f42fc001a20 comp rx=0 tx=0).stop 2023-10-10T20:20:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 msgr2=0x7f430c167c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f430000c850 tx=0x7f4300008040 comp rx=0 tx=0).stop 2023-10-10T20:20:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 shutdown_connections 2023-10-10T20:20:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f42f8066f30 0x7f42f80693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f430c0c96f0 0x7f430c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f430c0b0c20 0x7f430c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 --2- 172.21.15.148:0/547216879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f430c0c9320 0x7f430c167c90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 >> 172.21.15.148:0/547216879 conn(0x7f430c0cd1e0 msgr2=0x7f430c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 shutdown_connections 2023-10-10T20:20:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.220+0000 7f4311f0f700 1 -- 172.21.15.148:0/547216879 wait complete. 2023-10-10T20:20:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:20:44.233 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-10-10T20:20:43.339320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2812,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559687,"statfs":{"total":1600000000,"available":1597119562,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.538726"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812493+0000","last_change":"2023-10-10T20:20:39.812414+0000","last_active":"2023-10-10T20:20:39.812493+0000","last_peered":"2023-10-10T20:20:39.812493+0000","last_clean":"2023-10-10T20:20:39.812493+0000","last_became_active":"2023-10-10T20:20:39.812172+0000","last_became_peered":"2023-10-10T20:20:39.812172+0000","last_unstale":"2023-10-10T20:20:39.812493+0000","last_undegraded":"2023-10-10T20:20:39.812493+0000","last_fullsized":"2023-10-10T20:20:39.812493+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823712+0000","last_change":"2023-10-10T20:20:39.821705+0000","last_active":"2023-10-10T20:20:39.823712+0000","last_peered":"2023-10-10T20:20:39.823712+0000","last_clean":"2023-10-10T20:20:39.823712+0000","last_became_active":"2023-10-10T20:20:39.821453+0000","last_became_peered":"2023-10-10T20:20:39.821453+0000","last_unstale":"2023-10-10T20:20:39.823712+0000","last_undegraded":"2023-10-10T20:20:39.823712+0000","last_fullsized":"2023-10-10T20:20:39.823712+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814512+0000","last_change":"2023-10-10T20:20:39.813745+0000","last_active":"2023-10-10T20:20:39.814512+0000","last_peered":"2023-10-10T20:20:39.814512+0000","last_clean":"2023-10-10T20:20:39.814512+0000","last_became_active":"2023-10-10T20:20:39.813176+0000","last_became_peered":"2023-10-10T20:20:39.813176+0000","last_unstale":"2023-10-10T20:20:39.814512+0000","last_undegraded":"2023-10-10T20:20:39.814512+0000","last_fullsized":"2023-10-10T20:20:39.814512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:14.381937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824627+0000","last_change":"2023-10-10T20:20:39.824567+0000","last_active":"2023-10-10T20:20:39.824627+0000","last_peered":"2023-10-10T20:20:39.824627+0000","last_clean":"2023-10-10T20:20:39.824627+0000","last_became_active":"2023-10-10T20:20:39.824358+0000","last_became_peered":"2023-10-10T20:20:39.824358+0000","last_unstale":"2023-10-10T20:20:39.824627+0000","last_undegraded":"2023-10-10T20:20:39.824627+0000","last_fullsized":"2023-10-10T20:20:39.824627+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:07.616379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816217+0000","last_change":"2023-10-10T20:20:39.816141+0000","last_active":"2023-10-10T20:20:39.816217+0000","last_peered":"2023-10-10T20:20:39.816217+0000","last_clean":"2023-10-10T20:20:39.816217+0000","last_became_active":"2023-10-10T20:20:39.815940+0000","last_became_peered":"2023-10-10T20:20:39.815940+0000","last_unstale":"2023-10-10T20:20:39.816217+0000","last_undegraded":"2023-10-10T20:20:39.816217+0000","last_fullsized":"2023-10-10T20:20:39.816217+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820765+0000","last_change":"2023-10-10T20:20:39.820666+0000","last_active":"2023-10-10T20:20:39.820765+0000","last_peered":"2023-10-10T20:20:39.820765+0000","last_clean":"2023-10-10T20:20:39.820765+0000","last_became_active":"2023-10-10T20:20:39.820299+0000","last_became_peered":"2023-10-10T20:20:39.820299+0000","last_unstale":"2023-10-10T20:20:39.820765+0000","last_undegraded":"2023-10-10T20:20:39.820765+0000","last_fullsized":"2023-10-10T20:20:39.820765+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:09.184793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816079+0000","last_change":"2023-10-10T20:20:39.816009+0000","last_active":"2023-10-10T20:20:39.816079+0000","last_peered":"2023-10-10T20:20:39.816079+0000","last_clean":"2023-10-10T20:20:39.816079+0000","last_became_active":"2023-10-10T20:20:39.815777+0000","last_became_peered":"2023-10-10T20:20:39.815777+0000","last_unstale":"2023-10-10T20:20:39.816079+0000","last_undegraded":"2023-10-10T20:20:39.816079+0000","last_fullsized":"2023-10-10T20:20:39.816079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822566+0000","last_change":"2023-10-10T20:20:39.822513+0000","last_active":"2023-10-10T20:20:39.822566+0000","last_peered":"2023-10-10T20:20:39.822566+0000","last_clean":"2023-10-10T20:20:39.822566+0000","last_became_active":"2023-10-10T20:20:39.822356+0000","last_became_peered":"2023-10-10T20:20:39.822356+0000","last_unstale":"2023-10-10T20:20:39.822566+0000","last_undegraded":"2023-10-10T20:20:39.822566+0000","last_fullsized":"2023-10-10T20:20:39.822566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.815586+0000","last_change":"2023-10-10T20:20:39.815515+0000","last_active":"2023-10-10T20:20:39.815586+0000","last_peered":"2023-10-10T20:20:39.815586+0000","last_clean":"2023-10-10T20:20:39.815586+0000","last_became_active":"2023-10-10T20:20:39.815284+0000","last_became_peered":"2023-10-10T20:20:39.815284+0000","last_unstale":"2023-10-10T20:20:39.815586+0000","last_undegraded":"2023-10-10T20:20:39.815586+0000","last_fullsized":"2023-10-10T20:20:39.815586+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822058+0000","last_change":"2023-10-10T20:20:39.821174+0000","last_active":"2023-10-10T20:20:39.822058+0000","last_peered":"2023-10-10T20:20:39.822058+0000","last_clean":"2023-10-10T20:20:39.822058+0000","last_became_active":"2023-10-10T20:20:39.819531+0000","last_became_peered":"2023-10-10T20:20:39.819531+0000","last_unstale":"2023-10-10T20:20:39.822058+0000","last_undegraded":"2023-10-10T20:20:39.822058+0000","last_fullsized":"2023-10-10T20:20:39.822058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818336+0000","last_change":"2023-10-10T20:20:39.818264+0000","last_active":"2023-10-10T20:20:39.818336+0000","last_peered":"2023-10-10T20:20:39.818336+0000","last_clean":"2023-10-10T20:20:39.818336+0000","last_became_active":"2023-10-10T20:20:39.818043+0000","last_became_peered":"2023-10-10T20:20:39.818043+0000","last_unstale":"2023-10-10T20:20:39.818336+0000","last_undegraded":"2023-10-10T20:20:39.818336+0000","last_fullsized":"2023-10-10T20:20:39.818336+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:09.232095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818733+0000","last_change":"2023-10-10T20:20:39.818198+0000","last_active":"2023-10-10T20:20:39.818733+0000","last_peered":"2023-10-10T20:20:39.818733+0000","last_clean":"2023-10-10T20:20:39.818733+0000","last_became_active":"2023-10-10T20:20:39.817952+0000","last_became_peered":"2023-10-10T20:20:39.817952+0000","last_unstale":"2023-10-10T20:20:39.818733+0000","last_undegraded":"2023-10-10T20:20:39.818733+0000","last_fullsized":"2023-10-10T20:20:39.818733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816554+0000","last_change":"2023-10-10T20:20:39.816134+0000","last_active":"2023-10-10T20:20:39.816554+0000","last_peered":"2023-10-10T20:20:39.816554+0000","last_clean":"2023-10-10T20:20:39.816554+0000","last_became_active":"2023-10-10T20:20:39.815348+0000","last_became_peered":"2023-10-10T20:20:39.815348+0000","last_unstale":"2023-10-10T20:20:39.816554+0000","last_undegraded":"2023-10-10T20:20:39.816554+0000","last_fullsized":"2023-10-10T20:20:39.816554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:05.124451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818165+0000","last_change":"2023-10-10T20:20:39.818109+0000","last_active":"2023-10-10T20:20:39.818165+0000","last_peered":"2023-10-10T20:20:39.818165+0000","last_clean":"2023-10-10T20:20:39.818165+0000","last_became_active":"2023-10-10T20:20:39.817885+0000","last_became_peered":"2023-10-10T20:20:39.817885+0000","last_unstale":"2023-10-10T20:20:39.818165+0000","last_undegraded":"2023-10-10T20:20:39.818165+0000","last_fullsized":"2023-10-10T20:20:39.818165+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817530+0000","last_change":"2023-10-10T20:20:39.817457+0000","last_active":"2023-10-10T20:20:39.817530+0000","last_peered":"2023-10-10T20:20:39.817530+0000","last_clean":"2023-10-10T20:20:39.817530+0000","last_became_active":"2023-10-10T20:20:39.817200+0000","last_became_peered":"2023-10-10T20:20:39.817200+0000","last_unstale":"2023-10-10T20:20:39.817530+0000","last_undegraded":"2023-10-10T20:20:39.817530+0000","last_fullsized":"2023-10-10T20:20:39.817530+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:39:48.978431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821250+0000","last_change":"2023-10-10T20:20:39.820022+0000","last_active":"2023-10-10T20:20:39.821250+0000","last_peered":"2023-10-10T20:20:39.821250+0000","last_clean":"2023-10-10T20:20:39.821250+0000","last_became_active":"2023-10-10T20:20:39.819048+0000","last_became_peered":"2023-10-10T20:20:39.819048+0000","last_unstale":"2023-10-10T20:20:39.821250+0000","last_undegraded":"2023-10-10T20:20:39.821250+0000","last_fullsized":"2023-10-10T20:20:39.821250+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:30.861156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816121+0000","last_change":"2023-10-10T20:20:39.816047+0000","last_active":"2023-10-10T20:20:39.816121+0000","last_peered":"2023-10-10T20:20:39.816121+0000","last_clean":"2023-10-10T20:20:39.816121+0000","last_became_active":"2023-10-10T20:20:39.815443+0000","last_became_peered":"2023-10-10T20:20:39.815443+0000","last_unstale":"2023-10-10T20:20:39.816121+0000","last_undegraded":"2023-10-10T20:20:39.816121+0000","last_fullsized":"2023-10-10T20:20:39.816121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814533+0000","last_change":"2023-10-10T20:20:39.813889+0000","last_active":"2023-10-10T20:20:39.814533+0000","last_peered":"2023-10-10T20:20:39.814533+0000","last_clean":"2023-10-10T20:20:39.814533+0000","last_became_active":"2023-10-10T20:20:39.813502+0000","last_became_peered":"2023-10-10T20:20:39.813502+0000","last_unstale":"2023-10-10T20:20:39.814533+0000","last_undegraded":"2023-10-10T20:20:39.814533+0000","last_fullsized":"2023-10-10T20:20:39.814533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824932+0000","last_change":"2023-10-10T20:20:39.824820+0000","last_active":"2023-10-10T20:20:39.824932+0000","last_peered":"2023-10-10T20:20:39.824932+0000","last_clean":"2023-10-10T20:20:39.824932+0000","last_became_active":"2023-10-10T20:20:39.824380+0000","last_became_peered":"2023-10-10T20:20:39.824380+0000","last_unstale":"2023-10-10T20:20:39.824932+0000","last_undegraded":"2023-10-10T20:20:39.824932+0000","last_fullsized":"2023-10-10T20:20:39.824932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:08.662779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814772+0000","last_change":"2023-10-10T20:20:39.814713+0000","last_active":"2023-10-10T20:20:39.814772+0000","last_peered":"2023-10-10T20:20:39.814772+0000","last_clean":"2023-10-10T20:20:39.814772+0000","last_became_active":"2023-10-10T20:20:39.814489+0000","last_became_peered":"2023-10-10T20:20:39.814489+0000","last_unstale":"2023-10-10T20:20:39.814772+0000","last_undegraded":"2023-10-10T20:20:39.814772+0000","last_fullsized":"2023-10-10T20:20:39.814772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:20.579237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821881+0000","last_change":"2023-10-10T20:20:39.821830+0000","last_active":"2023-10-10T20:20:39.821881+0000","last_peered":"2023-10-10T20:20:39.821881+0000","last_clean":"2023-10-10T20:20:39.821881+0000","last_became_active":"2023-10-10T20:20:39.821646+0000","last_became_peered":"2023-10-10T20:20:39.821646+0000","last_unstale":"2023-10-10T20:20:39.821881+0000","last_undegraded":"2023-10-10T20:20:39.821881+0000","last_fullsized":"2023-10-10T20:20:39.821881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:53.098641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822095+0000","last_change":"2023-10-10T20:20:39.821360+0000","last_active":"2023-10-10T20:20:39.822095+0000","last_peered":"2023-10-10T20:20:39.822095+0000","last_clean":"2023-10-10T20:20:39.822095+0000","last_became_active":"2023-10-10T20:20:39.819892+0000","last_became_peered":"2023-10-10T20:20:39.819892+0000","last_unstale":"2023-10-10T20:20:39.822095+0000","last_undegraded":"2023-10-10T20:20:39.822095+0000","last_fullsized":"2023-10-10T20:20:39.822095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822567+0000","last_change":"2023-10-10T20:20:39.821941+0000","last_active":"2023-10-10T20:20:39.822567+0000","last_peered":"2023-10-10T20:20:39.822567+0000","last_clean":"2023-10-10T20:20:39.822567+0000","last_became_active":"2023-10-10T20:20:39.821693+0000","last_became_peered":"2023-10-10T20:20:39.821693+0000","last_unstale":"2023-10-10T20:20:39.822567+0000","last_undegraded":"2023-10-10T20:20:39.822567+0000","last_fullsized":"2023-10-10T20:20:39.822567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:30.646812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824910+0000","last_change":"2023-10-10T20:20:39.824638+0000","last_active":"2023-10-10T20:20:39.824910+0000","last_peered":"2023-10-10T20:20:39.824910+0000","last_clean":"2023-10-10T20:20:39.824910+0000","last_became_active":"2023-10-10T20:20:39.823415+0000","last_became_peered":"2023-10-10T20:20:39.823415+0000","last_unstale":"2023-10-10T20:20:39.824910+0000","last_undegraded":"2023-10-10T20:20:39.824910+0000","last_fullsized":"2023-10-10T20:20:39.824910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821162+0000","last_change":"2023-10-10T20:20:39.818814+0000","last_active":"2023-10-10T20:20:39.821162+0000","last_peered":"2023-10-10T20:20:39.821162+0000","last_clean":"2023-10-10T20:20:39.821162+0000","last_became_active":"2023-10-10T20:20:39.818336+0000","last_became_peered":"2023-10-10T20:20:39.818336+0000","last_unstale":"2023-10-10T20:20:39.821162+0000","last_undegraded":"2023-10-10T20:20:39.821162+0000","last_fullsized":"2023-10-10T20:20:39.821162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:06.927011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.813932+0000","last_change":"2023-10-10T20:20:39.813876+0000","last_active":"2023-10-10T20:20:39.813932+0000","last_peered":"2023-10-10T20:20:39.813932+0000","last_clean":"2023-10-10T20:20:39.813932+0000","last_became_active":"2023-10-10T20:20:39.813618+0000","last_became_peered":"2023-10-10T20:20:39.813618+0000","last_unstale":"2023-10-10T20:20:39.813932+0000","last_undegraded":"2023-10-10T20:20:39.813932+0000","last_fullsized":"2023-10-10T20:20:39.813932+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:32.078076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816271+0000","last_change":"2023-10-10T20:20:39.816205+0000","last_active":"2023-10-10T20:20:39.816271+0000","last_peered":"2023-10-10T20:20:39.816271+0000","last_clean":"2023-10-10T20:20:39.816271+0000","last_became_active":"2023-10-10T20:20:39.815789+0000","last_became_peered":"2023-10-10T20:20:39.815789+0000","last_unstale":"2023-10-10T20:20:39.816271+0000","last_undegraded":"2023-10-10T20:20:39.816271+0000","last_fullsized":"2023-10-10T20:20:39.816271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:31.459651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817685+0000","last_change":"2023-10-10T20:20:39.817582+0000","last_active":"2023-10-10T20:20:39.817685+0000","last_peered":"2023-10-10T20:20:39.817685+0000","last_clean":"2023-10-10T20:20:39.817685+0000","last_became_active":"2023-10-10T20:20:39.817303+0000","last_became_peered":"2023-10-10T20:20:39.817303+0000","last_unstale":"2023-10-10T20:20:39.817685+0000","last_undegraded":"2023-10-10T20:20:39.817685+0000","last_fullsized":"2023-10-10T20:20:39.817685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:07.329623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823800+0000","last_change":"2023-10-10T20:20:39.823646+0000","last_active":"2023-10-10T20:20:39.823800+0000","last_peered":"2023-10-10T20:20:39.823800+0000","last_clean":"2023-10-10T20:20:39.823800+0000","last_became_active":"2023-10-10T20:20:39.822151+0000","last_became_peered":"2023-10-10T20:20:39.822151+0000","last_unstale":"2023-10-10T20:20:39.823800+0000","last_undegraded":"2023-10-10T20:20:39.823800+0000","last_fullsized":"2023-10-10T20:20:39.823800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:31.459651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811905+0000","last_change":"2023-10-10T20:20:39.811320+0000","last_active":"2023-10-10T20:20:39.811905+0000","last_peered":"2023-10-10T20:20:39.811905+0000","last_clean":"2023-10-10T20:20:39.811905+0000","last_became_active":"2023-10-10T20:20:39.811081+0000","last_became_peered":"2023-10-10T20:20:39.811081+0000","last_unstale":"2023-10-10T20:20:39.811905+0000","last_undegraded":"2023-10-10T20:20:39.811905+0000","last_fullsized":"2023-10-10T20:20:39.811905+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:31.617758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817836+0000","last_change":"2023-10-10T20:20:39.817552+0000","last_active":"2023-10-10T20:20:39.817836+0000","last_peered":"2023-10-10T20:20:39.817836+0000","last_clean":"2023-10-10T20:20:39.817836+0000","last_became_active":"2023-10-10T20:20:39.817177+0000","last_became_peered":"2023-10-10T20:20:39.817177+0000","last_unstale":"2023-10-10T20:20:39.817836+0000","last_undegraded":"2023-10-10T20:20:39.817836+0000","last_fullsized":"2023-10-10T20:20:39.817836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:58.719801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817807+0000","last_change":"2023-10-10T20:20:39.817714+0000","last_active":"2023-10-10T20:20:39.817807+0000","last_peered":"2023-10-10T20:20:39.817807+0000","last_clean":"2023-10-10T20:20:39.817807+0000","last_became_active":"2023-10-10T20:20:39.817518+0000","last_became_peered":"2023-10-10T20:20:39.817518+0000","last_unstale":"2023-10-10T20:20:39.817807+0000","last_undegraded":"2023-10-10T20:20:39.817807+0000","last_fullsized":"2023-10-10T20:20:39.817807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:31.748344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819451+0000","last_change":"2023-10-10T20:20:39.819145+0000","last_active":"2023-10-10T20:20:39.819451+0000","last_peered":"2023-10-10T20:20:39.819451+0000","last_clean":"2023-10-10T20:20:39.819451+0000","last_became_active":"2023-10-10T20:20:39.818932+0000","last_became_peered":"2023-10-10T20:20:39.818932+0000","last_unstale":"2023-10-10T20:20:39.819451+0000","last_undegraded":"2023-10-10T20:20:39.819451+0000","last_fullsized":"2023-10-10T20:20:39.819451+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:30.861156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830546+0000","last_change":"2023-10-10T20:20:38.816726+0000","last_active":"2023-10-10T20:20:39.830546+0000","last_peered":"2023-10-10T20:20:39.830546+0000","last_clean":"2023-10-10T20:20:39.830546+0000","last_became_active":"2023-10-10T20:20:38.816569+0000","last_became_peered":"2023-10-10T20:20:38.816569+0000","last_unstale":"2023-10-10T20:20:39.830546+0000","last_undegraded":"2023-10-10T20:20:39.830546+0000","last_fullsized":"2023-10-10T20:20:39.830546+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:36.874602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821265+0000","last_change":"2023-10-10T20:20:39.820750+0000","last_active":"2023-10-10T20:20:39.821265+0000","last_peered":"2023-10-10T20:20:39.821265+0000","last_clean":"2023-10-10T20:20:39.821265+0000","last_became_active":"2023-10-10T20:20:39.820516+0000","last_became_peered":"2023-10-10T20:20:39.820516+0000","last_unstale":"2023-10-10T20:20:39.821265+0000","last_undegraded":"2023-10-10T20:20:39.821265+0000","last_fullsized":"2023-10-10T20:20:39.821265+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:34.812174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830719+0000","last_change":"2023-10-10T20:20:38.817703+0000","last_active":"2023-10-10T20:20:39.830719+0000","last_peered":"2023-10-10T20:20:39.830719+0000","last_clean":"2023-10-10T20:20:39.830719+0000","last_became_active":"2023-10-10T20:20:38.817457+0000","last_became_peered":"2023-10-10T20:20:38.817457+0000","last_unstale":"2023-10-10T20:20:39.830719+0000","last_undegraded":"2023-10-10T20:20:39.830719+0000","last_fullsized":"2023-10-10T20:20:39.830719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:31.071477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818074+0000","last_change":"2023-10-10T20:20:39.817995+0000","last_active":"2023-10-10T20:20:39.818074+0000","last_peered":"2023-10-10T20:20:39.818074+0000","last_clean":"2023-10-10T20:20:39.818074+0000","last_became_active":"2023-10-10T20:20:39.817711+0000","last_became_peered":"2023-10-10T20:20:39.817711+0000","last_unstale":"2023-10-10T20:20:39.818074+0000","last_undegraded":"2023-10-10T20:20:39.818074+0000","last_fullsized":"2023-10-10T20:20:39.818074+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829125+0000","last_change":"2023-10-10T20:20:38.813109+0000","last_active":"2023-10-10T20:20:39.829125+0000","last_peered":"2023-10-10T20:20:39.829125+0000","last_clean":"2023-10-10T20:20:39.829125+0000","last_became_active":"2023-10-10T20:20:38.812564+0000","last_became_peered":"2023-10-10T20:20:38.812564+0000","last_unstale":"2023-10-10T20:20:39.829125+0000","last_undegraded":"2023-10-10T20:20:39.829125+0000","last_fullsized":"2023-10-10T20:20:39.829125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:11.059562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820624+0000","last_change":"2023-10-10T20:20:39.820570+0000","last_active":"2023-10-10T20:20:39.820624+0000","last_peered":"2023-10-10T20:20:39.820624+0000","last_clean":"2023-10-10T20:20:39.820624+0000","last_became_active":"2023-10-10T20:20:39.820120+0000","last_became_peered":"2023-10-10T20:20:39.820120+0000","last_unstale":"2023-10-10T20:20:39.820624+0000","last_undegraded":"2023-10-10T20:20:39.820624+0000","last_fullsized":"2023-10-10T20:20:39.820624+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:28.149085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829992+0000","last_change":"2023-10-10T20:20:38.815786+0000","last_active":"2023-10-10T20:20:39.829992+0000","last_peered":"2023-10-10T20:20:39.829992+0000","last_clean":"2023-10-10T20:20:39.829992+0000","last_became_active":"2023-10-10T20:20:38.815555+0000","last_became_peered":"2023-10-10T20:20:38.815555+0000","last_unstale":"2023-10-10T20:20:39.829992+0000","last_undegraded":"2023-10-10T20:20:39.829992+0000","last_fullsized":"2023-10-10T20:20:39.829992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:07.656181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822244+0000","last_change":"2023-10-10T20:20:39.822170+0000","last_active":"2023-10-10T20:20:39.822244+0000","last_peered":"2023-10-10T20:20:39.822244+0000","last_clean":"2023-10-10T20:20:39.822244+0000","last_became_active":"2023-10-10T20:20:39.821757+0000","last_became_peered":"2023-10-10T20:20:39.821757+0000","last_unstale":"2023-10-10T20:20:39.822244+0000","last_undegraded":"2023-10-10T20:20:39.822244+0000","last_fullsized":"2023-10-10T20:20:39.822244+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:52.659983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.828737+0000","last_change":"2023-10-10T20:20:38.816117+0000","last_active":"2023-10-10T20:20:39.828737+0000","last_peered":"2023-10-10T20:20:39.828737+0000","last_clean":"2023-10-10T20:20:39.828737+0000","last_became_active":"2023-10-10T20:20:38.815574+0000","last_became_peered":"2023-10-10T20:20:38.815574+0000","last_unstale":"2023-10-10T20:20:39.828737+0000","last_undegraded":"2023-10-10T20:20:39.828737+0000","last_fullsized":"2023-10-10T20:20:39.828737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:30.611160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814981+0000","last_change":"2023-10-10T20:20:39.814758+0000","last_active":"2023-10-10T20:20:39.814981+0000","last_peered":"2023-10-10T20:20:39.814981+0000","last_clean":"2023-10-10T20:20:39.814981+0000","last_became_active":"2023-10-10T20:20:39.814070+0000","last_became_peered":"2023-10-10T20:20:39.814070+0000","last_unstale":"2023-10-10T20:20:39.814981+0000","last_undegraded":"2023-10-10T20:20:39.814981+0000","last_fullsized":"2023-10-10T20:20:39.814981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:14.381937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829644+0000","last_change":"2023-10-10T20:20:38.812003+0000","last_active":"2023-10-10T20:20:39.829644+0000","last_peered":"2023-10-10T20:20:39.829644+0000","last_clean":"2023-10-10T20:20:39.829644+0000","last_became_active":"2023-10-10T20:20:38.811701+0000","last_became_peered":"2023-10-10T20:20:38.811701+0000","last_unstale":"2023-10-10T20:20:39.829644+0000","last_undegraded":"2023-10-10T20:20:39.829644+0000","last_fullsized":"2023-10-10T20:20:39.829644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:46.562248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822080+0000","last_change":"2023-10-10T20:20:39.821308+0000","last_active":"2023-10-10T20:20:39.822080+0000","last_peered":"2023-10-10T20:20:39.822080+0000","last_clean":"2023-10-10T20:20:39.822080+0000","last_became_active":"2023-10-10T20:20:39.819845+0000","last_became_peered":"2023-10-10T20:20:39.819845+0000","last_unstale":"2023-10-10T20:20:39.822080+0000","last_undegraded":"2023-10-10T20:20:39.822080+0000","last_fullsized":"2023-10-10T20:20:39.822080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814432+0000","last_change":"2023-10-10T20:20:38.812248+0000","last_active":"2023-10-10T20:20:39.814432+0000","last_peered":"2023-10-10T20:20:39.814432+0000","last_clean":"2023-10-10T20:20:39.814432+0000","last_became_active":"2023-10-10T20:20:38.811996+0000","last_became_peered":"2023-10-10T20:20:38.811996+0000","last_unstale":"2023-10-10T20:20:39.814432+0000","last_undegraded":"2023-10-10T20:20:39.814432+0000","last_fullsized":"2023-10-10T20:20:39.814432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:27.142486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.824125+0000","last_change":"2023-10-10T20:20:39.824051+0000","last_active":"2023-10-10T20:20:39.824125+0000","last_peered":"2023-10-10T20:20:39.824125+0000","last_clean":"2023-10-10T20:20:39.824125+0000","last_became_active":"2023-10-10T20:20:39.822431+0000","last_became_peered":"2023-10-10T20:20:39.822431+0000","last_unstale":"2023-10-10T20:20:39.824125+0000","last_undegraded":"2023-10-10T20:20:39.824125+0000","last_fullsized":"2023-10-10T20:20:39.824125+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:29:37.881201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810642+0000","last_change":"2023-10-10T20:20:38.817447+0000","last_active":"2023-10-10T20:20:39.810642+0000","last_peered":"2023-10-10T20:20:39.810642+0000","last_clean":"2023-10-10T20:20:39.810642+0000","last_became_active":"2023-10-10T20:20:38.815962+0000","last_became_peered":"2023-10-10T20:20:38.815962+0000","last_unstale":"2023-10-10T20:20:39.810642+0000","last_undegraded":"2023-10-10T20:20:39.810642+0000","last_fullsized":"2023-10-10T20:20:39.810642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:19.572639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814760+0000","last_change":"2023-10-10T20:20:39.814564+0000","last_active":"2023-10-10T20:20:39.814760+0000","last_peered":"2023-10-10T20:20:39.814760+0000","last_clean":"2023-10-10T20:20:39.814760+0000","last_became_active":"2023-10-10T20:20:39.814300+0000","last_became_peered":"2023-10-10T20:20:39.814300+0000","last_unstale":"2023-10-10T20:20:39.814760+0000","last_undegraded":"2023-10-10T20:20:39.814760+0000","last_fullsized":"2023-10-10T20:20:39.814760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:34.812174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.808231+0000","last_change":"2023-10-10T20:20:38.812705+0000","last_active":"2023-10-10T20:20:39.808231+0000","last_peered":"2023-10-10T20:20:39.808231+0000","last_clean":"2023-10-10T20:20:39.808231+0000","last_became_active":"2023-10-10T20:20:38.812480+0000","last_became_peered":"2023-10-10T20:20:38.812480+0000","last_unstale":"2023-10-10T20:20:39.808231+0000","last_undegraded":"2023-10-10T20:20:39.808231+0000","last_fullsized":"2023-10-10T20:20:39.808231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:20:20.226518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816839+0000","last_change":"2023-10-10T20:20:39.816768+0000","last_active":"2023-10-10T20:20:39.816839+0000","last_peered":"2023-10-10T20:20:39.816839+0000","last_clean":"2023-10-10T20:20:39.816839+0000","last_became_active":"2023-10-10T20:20:39.816489+0000","last_became_peered":"2023-10-10T20:20:39.816489+0000","last_unstale":"2023-10-10T20:20:39.816839+0000","last_undegraded":"2023-10-10T20:20:39.816839+0000","last_fullsized":"2023-10-10T20:20:39.816839+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:12.845705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810087+0000","last_change":"2023-10-10T20:20:38.815263+0000","last_active":"2023-10-10T20:20:39.810087+0000","last_peered":"2023-10-10T20:20:39.810087+0000","last_clean":"2023-10-10T20:20:39.810087+0000","last_became_active":"2023-10-10T20:20:38.815045+0000","last_became_peered":"2023-10-10T20:20:38.815045+0000","last_unstale":"2023-10-10T20:20:39.810087+0000","last_undegraded":"2023-10-10T20:20:39.810087+0000","last_fullsized":"2023-10-10T20:20:39.810087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:04.117852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820966+0000","last_change":"2023-10-10T20:20:39.820914+0000","last_active":"2023-10-10T20:20:39.820966+0000","last_peered":"2023-10-10T20:20:39.820966+0000","last_clean":"2023-10-10T20:20:39.820966+0000","last_became_active":"2023-10-10T20:20:39.819866+0000","last_became_peered":"2023-10-10T20:20:39.819866+0000","last_unstale":"2023-10-10T20:20:39.820966+0000","last_undegraded":"2023-10-10T20:20:39.820966+0000","last_fullsized":"2023-10-10T20:20:39.820966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:30.593449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830297+0000","last_change":"2023-10-10T20:20:38.818792+0000","last_active":"2023-10-10T20:20:39.830297+0000","last_peered":"2023-10-10T20:20:39.830297+0000","last_clean":"2023-10-10T20:20:39.830297+0000","last_became_active":"2023-10-10T20:20:38.818582+0000","last_became_peered":"2023-10-10T20:20:38.818582+0000","last_unstale":"2023-10-10T20:20:39.830297+0000","last_undegraded":"2023-10-10T20:20:39.830297+0000","last_fullsized":"2023-10-10T20:20:39.830297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:05.920412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811572+0000","last_change":"2023-10-10T20:20:39.811500+0000","last_active":"2023-10-10T20:20:39.811572+0000","last_peered":"2023-10-10T20:20:39.811572+0000","last_clean":"2023-10-10T20:20:39.811572+0000","last_became_active":"2023-10-10T20:20:39.811258+0000","last_became_peered":"2023-10-10T20:20:39.811258+0000","last_unstale":"2023-10-10T20:20:39.811572+0000","last_undegraded":"2023-10-10T20:20:39.811572+0000","last_fullsized":"2023-10-10T20:20:39.811572+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.831791+0000","last_change":"2023-10-10T20:20:38.810394+0000","last_active":"2023-10-10T20:20:39.831791+0000","last_peered":"2023-10-10T20:20:39.831791+0000","last_clean":"2023-10-10T20:20:39.831791+0000","last_became_active":"2023-10-10T20:20:38.810028+0000","last_became_peered":"2023-10-10T20:20:38.810028+0000","last_unstale":"2023-10-10T20:20:39.831791+0000","last_undegraded":"2023-10-10T20:20:39.831791+0000","last_fullsized":"2023-10-10T20:20:39.831791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:20.237910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821234+0000","last_change":"2023-10-10T20:20:39.819864+0000","last_active":"2023-10-10T20:20:39.821234+0000","last_peered":"2023-10-10T20:20:39.821234+0000","last_clean":"2023-10-10T20:20:39.821234+0000","last_became_active":"2023-10-10T20:20:39.818712+0000","last_became_peered":"2023-10-10T20:20:39.818712+0000","last_unstale":"2023-10-10T20:20:39.821234+0000","last_undegraded":"2023-10-10T20:20:39.821234+0000","last_fullsized":"2023-10-10T20:20:39.821234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:06.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830253+0000","last_change":"2023-10-10T20:20:38.818732+0000","last_active":"2023-10-10T20:20:39.830253+0000","last_peered":"2023-10-10T20:20:39.830253+0000","last_clean":"2023-10-10T20:20:39.830253+0000","last_became_active":"2023-10-10T20:20:38.818533+0000","last_became_peered":"2023-10-10T20:20:38.818533+0000","last_unstale":"2023-10-10T20:20:39.830253+0000","last_undegraded":"2023-10-10T20:20:39.830253+0000","last_fullsized":"2023-10-10T20:20:39.830253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:36:13.375338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816269+0000","last_change":"2023-10-10T20:20:39.816122+0000","last_active":"2023-10-10T20:20:39.816269+0000","last_peered":"2023-10-10T20:20:39.816269+0000","last_clean":"2023-10-10T20:20:39.816269+0000","last_became_active":"2023-10-10T20:20:39.815778+0000","last_became_peered":"2023-10-10T20:20:39.815778+0000","last_unstale":"2023-10-10T20:20:39.816269+0000","last_undegraded":"2023-10-10T20:20:39.816269+0000","last_fullsized":"2023-10-10T20:20:39.816269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:42.377091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.809717+0000","last_change":"2023-10-10T20:20:38.816241+0000","last_active":"2023-10-10T20:20:39.809717+0000","last_peered":"2023-10-10T20:20:39.809717+0000","last_clean":"2023-10-10T20:20:39.809717+0000","last_became_active":"2023-10-10T20:20:38.816001+0000","last_became_peered":"2023-10-10T20:20:38.816001+0000","last_unstale":"2023-10-10T20:20:39.809717+0000","last_undegraded":"2023-10-10T20:20:39.809717+0000","last_fullsized":"2023-10-10T20:20:39.809717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:45:33.805575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820029+0000","last_change":"2023-10-10T20:20:39.818951+0000","last_active":"2023-10-10T20:20:39.820029+0000","last_peered":"2023-10-10T20:20:39.820029+0000","last_clean":"2023-10-10T20:20:39.820029+0000","last_became_active":"2023-10-10T20:20:39.818748+0000","last_became_peered":"2023-10-10T20:20:39.818748+0000","last_unstale":"2023-10-10T20:20:39.820029+0000","last_undegraded":"2023-10-10T20:20:39.820029+0000","last_fullsized":"2023-10-10T20:20:39.820029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:17.124275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812586+0000","last_change":"2023-10-10T20:20:38.815937+0000","last_active":"2023-10-10T20:20:39.812586+0000","last_peered":"2023-10-10T20:20:39.812586+0000","last_clean":"2023-10-10T20:20:39.812586+0000","last_became_active":"2023-10-10T20:20:38.815241+0000","last_became_peered":"2023-10-10T20:20:38.815241+0000","last_unstale":"2023-10-10T20:20:39.812586+0000","last_undegraded":"2023-10-10T20:20:39.812586+0000","last_fullsized":"2023-10-10T20:20:39.812586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:06.323024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.823119+0000","last_change":"2023-10-10T20:20:39.823043+0000","last_active":"2023-10-10T20:20:39.823119+0000","last_peered":"2023-10-10T20:20:39.823119+0000","last_clean":"2023-10-10T20:20:39.823119+0000","last_became_active":"2023-10-10T20:20:39.822809+0000","last_became_peered":"2023-10-10T20:20:39.822809+0000","last_unstale":"2023-10-10T20:20:39.823119+0000","last_undegraded":"2023-10-10T20:20:39.823119+0000","last_fullsized":"2023-10-10T20:20:39.823119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:45:42.377091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.830852+0000","last_change":"2023-10-10T20:20:38.812192+0000","last_active":"2023-10-10T20:20:39.830852+0000","last_peered":"2023-10-10T20:20:39.830852+0000","last_clean":"2023-10-10T20:20:39.830852+0000","last_became_active":"2023-10-10T20:20:38.812031+0000","last_became_peered":"2023-10-10T20:20:38.812031+0000","last_unstale":"2023-10-10T20:20:39.830852+0000","last_undegraded":"2023-10-10T20:20:39.830852+0000","last_fullsized":"2023-10-10T20:20:39.830852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:32:36.757262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816533+0000","last_change":"2023-10-10T20:20:39.815991+0000","last_active":"2023-10-10T20:20:39.816533+0000","last_peered":"2023-10-10T20:20:39.816533+0000","last_clean":"2023-10-10T20:20:39.816533+0000","last_became_active":"2023-10-10T20:20:39.815776+0000","last_became_peered":"2023-10-10T20:20:39.815776+0000","last_unstale":"2023-10-10T20:20:39.816533+0000","last_undegraded":"2023-10-10T20:20:39.816533+0000","last_fullsized":"2023-10-10T20:20:39.816533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:32:37.763860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829599+0000","last_change":"2023-10-10T20:20:38.813549+0000","last_active":"2023-10-10T20:20:39.829599+0000","last_peered":"2023-10-10T20:20:39.829599+0000","last_clean":"2023-10-10T20:20:39.829599+0000","last_became_active":"2023-10-10T20:20:38.813331+0000","last_became_peered":"2023-10-10T20:20:38.813331+0000","last_unstale":"2023-10-10T20:20:39.829599+0000","last_undegraded":"2023-10-10T20:20:39.829599+0000","last_fullsized":"2023-10-10T20:20:39.829599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:17:59.778093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818962+0000","last_change":"2023-10-10T20:20:39.818782+0000","last_active":"2023-10-10T20:20:39.818962+0000","last_peered":"2023-10-10T20:20:39.818962+0000","last_clean":"2023-10-10T20:20:39.818962+0000","last_became_active":"2023-10-10T20:20:39.818542+0000","last_became_peered":"2023-10-10T20:20:39.818542+0000","last_unstale":"2023-10-10T20:20:39.818962+0000","last_undegraded":"2023-10-10T20:20:39.818962+0000","last_fullsized":"2023-10-10T20:20:39.818962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:05:07.329623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829998+0000","last_change":"2023-10-10T20:20:38.810569+0000","last_active":"2023-10-10T20:20:39.829998+0000","last_peered":"2023-10-10T20:20:39.829998+0000","last_clean":"2023-10-10T20:20:39.829998+0000","last_became_active":"2023-10-10T20:20:38.810329+0000","last_became_peered":"2023-10-10T20:20:38.810329+0000","last_unstale":"2023-10-10T20:20:39.829998+0000","last_undegraded":"2023-10-10T20:20:39.829998+0000","last_fullsized":"2023-10-10T20:20:39.829998+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:08.007313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822585+0000","last_change":"2023-10-10T20:20:39.822138+0000","last_active":"2023-10-10T20:20:39.822585+0000","last_peered":"2023-10-10T20:20:39.822585+0000","last_clean":"2023-10-10T20:20:39.822585+0000","last_became_active":"2023-10-10T20:20:39.821920+0000","last_became_peered":"2023-10-10T20:20:39.821920+0000","last_unstale":"2023-10-10T20:20:39.822585+0000","last_undegraded":"2023-10-10T20:20:39.822585+0000","last_fullsized":"2023-10-10T20:20:39.822585+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:13.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.828746+0000","last_change":"2023-10-10T20:20:38.809908+0000","last_active":"2023-10-10T20:20:39.828746+0000","last_peered":"2023-10-10T20:20:39.828746+0000","last_clean":"2023-10-10T20:20:39.828746+0000","last_became_active":"2023-10-10T20:20:38.809662+0000","last_became_peered":"2023-10-10T20:20:38.809662+0000","last_unstale":"2023-10-10T20:20:39.828746+0000","last_undegraded":"2023-10-10T20:20:39.828746+0000","last_fullsized":"2023-10-10T20:20:39.828746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:30.453052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821216+0000","last_change":"2023-10-10T20:20:39.819713+0000","last_active":"2023-10-10T20:20:39.821216+0000","last_peered":"2023-10-10T20:20:39.821216+0000","last_clean":"2023-10-10T20:20:39.821216+0000","last_became_active":"2023-10-10T20:20:39.818433+0000","last_became_peered":"2023-10-10T20:20:39.818433+0000","last_unstale":"2023-10-10T20:20:39.821216+0000","last_undegraded":"2023-10-10T20:20:39.821216+0000","last_fullsized":"2023-10-10T20:20:39.821216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:28.149085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.812674+0000","last_change":"2023-10-10T20:20:38.816704+0000","last_active":"2023-10-10T20:20:39.812674+0000","last_peered":"2023-10-10T20:20:39.812674+0000","last_clean":"2023-10-10T20:20:39.812674+0000","last_became_active":"2023-10-10T20:20:38.816484+0000","last_became_peered":"2023-10-10T20:20:38.816484+0000","last_unstale":"2023-10-10T20:20:39.812674+0000","last_undegraded":"2023-10-10T20:20:39.812674+0000","last_fullsized":"2023-10-10T20:20:39.812674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:12.896902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821032+0000","last_change":"2023-10-10T20:20:39.821000+0000","last_active":"2023-10-10T20:20:39.821032+0000","last_peered":"2023-10-10T20:20:39.821032+0000","last_clean":"2023-10-10T20:20:39.821032+0000","last_became_active":"2023-10-10T20:20:39.820340+0000","last_became_peered":"2023-10-10T20:20:39.820340+0000","last_unstale":"2023-10-10T20:20:39.821032+0000","last_undegraded":"2023-10-10T20:20:39.821032+0000","last_fullsized":"2023-10-10T20:20:39.821032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:10:37.019625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829787+0000","last_change":"2023-10-10T20:20:38.815415+0000","last_active":"2023-10-10T20:20:39.829787+0000","last_peered":"2023-10-10T20:20:39.829787+0000","last_clean":"2023-10-10T20:20:39.829787+0000","last_became_active":"2023-10-10T20:20:38.815165+0000","last_became_peered":"2023-10-10T20:20:38.815165+0000","last_unstale":"2023-10-10T20:20:39.829787+0000","last_undegraded":"2023-10-10T20:20:39.829787+0000","last_fullsized":"2023-10-10T20:20:39.829787+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:25.401834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819431+0000","last_change":"2023-10-10T20:20:39.817747+0000","last_active":"2023-10-10T20:20:39.819431+0000","last_peered":"2023-10-10T20:20:39.819431+0000","last_clean":"2023-10-10T20:20:39.819431+0000","last_became_active":"2023-10-10T20:20:39.817526+0000","last_became_peered":"2023-10-10T20:20:39.817526+0000","last_unstale":"2023-10-10T20:20:39.819431+0000","last_undegraded":"2023-10-10T20:20:39.819431+0000","last_fullsized":"2023-10-10T20:20:39.819431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:06.927011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.814531+0000","last_change":"2023-10-10T20:20:38.816842+0000","last_active":"2023-10-10T20:20:39.814531+0000","last_peered":"2023-10-10T20:20:39.814531+0000","last_clean":"2023-10-10T20:20:39.814531+0000","last_became_active":"2023-10-10T20:20:38.816590+0000","last_became_peered":"2023-10-10T20:20:38.816590+0000","last_unstale":"2023-10-10T20:20:39.814531+0000","last_undegraded":"2023-10-10T20:20:39.814531+0000","last_fullsized":"2023-10-10T20:20:39.814531+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:01:29.854558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811552+0000","last_change":"2023-10-10T20:20:39.811480+0000","last_active":"2023-10-10T20:20:39.811552+0000","last_peered":"2023-10-10T20:20:39.811552+0000","last_clean":"2023-10-10T20:20:39.811552+0000","last_became_active":"2023-10-10T20:20:39.811249+0000","last_became_peered":"2023-10-10T20:20:39.811249+0000","last_unstale":"2023-10-10T20:20:39.811552+0000","last_undegraded":"2023-10-10T20:20:39.811552+0000","last_fullsized":"2023-10-10T20:20:39.811552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:50.016672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810539+0000","last_change":"2023-10-10T20:20:38.817607+0000","last_active":"2023-10-10T20:20:39.810539+0000","last_peered":"2023-10-10T20:20:39.810539+0000","last_clean":"2023-10-10T20:20:39.810539+0000","last_became_active":"2023-10-10T20:20:38.817217+0000","last_became_peered":"2023-10-10T20:20:38.817217+0000","last_unstale":"2023-10-10T20:20:39.810539+0000","last_undegraded":"2023-10-10T20:20:39.810539+0000","last_fullsized":"2023-10-10T20:20:39.810539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:27:31.071477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.822024+0000","last_change":"2023-10-10T20:20:39.819298+0000","last_active":"2023-10-10T20:20:39.822024+0000","last_peered":"2023-10-10T20:20:39.822024+0000","last_clean":"2023-10-10T20:20:39.822024+0000","last_became_active":"2023-10-10T20:20:39.819047+0000","last_became_peered":"2023-10-10T20:20:39.819047+0000","last_unstale":"2023-10-10T20:20:39.822024+0000","last_undegraded":"2023-10-10T20:20:39.822024+0000","last_fullsized":"2023-10-10T20:20:39.822024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:43.413890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.809803+0000","last_change":"2023-10-10T20:20:38.817415+0000","last_active":"2023-10-10T20:20:39.809803+0000","last_peered":"2023-10-10T20:20:39.809803+0000","last_clean":"2023-10-10T20:20:39.809803+0000","last_became_active":"2023-10-10T20:20:38.817184+0000","last_became_peered":"2023-10-10T20:20:38.817184+0000","last_unstale":"2023-10-10T20:20:39.809803+0000","last_undegraded":"2023-10-10T20:20:39.809803+0000","last_fullsized":"2023-10-10T20:20:39.809803+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:30.453052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.821182+0000","last_change":"2023-10-10T20:20:39.819290+0000","last_active":"2023-10-10T20:20:39.821182+0000","last_peered":"2023-10-10T20:20:39.821182+0000","last_clean":"2023-10-10T20:20:39.821182+0000","last_became_active":"2023-10-10T20:20:39.818853+0000","last_became_peered":"2023-10-10T20:20:39.818853+0000","last_unstale":"2023-10-10T20:20:39.821182+0000","last_undegraded":"2023-10-10T20:20:39.821182+0000","last_fullsized":"2023-10-10T20:20:39.821182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:05.531228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.813632+0000","last_change":"2023-10-10T20:20:38.814741+0000","last_active":"2023-10-10T20:20:39.813632+0000","last_peered":"2023-10-10T20:20:39.813632+0000","last_clean":"2023-10-10T20:20:39.813632+0000","last_became_active":"2023-10-10T20:20:38.814492+0000","last_became_peered":"2023-10-10T20:20:38.814492+0000","last_unstale":"2023-10-10T20:20:39.813632+0000","last_undegraded":"2023-10-10T20:20:39.813632+0000","last_fullsized":"2023-10-10T20:20:39.813632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:51.653384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816251+0000","last_change":"2023-10-10T20:20:39.815991+0000","last_active":"2023-10-10T20:20:39.816251+0000","last_peered":"2023-10-10T20:20:39.816251+0000","last_clean":"2023-10-10T20:20:39.816251+0000","last_became_active":"2023-10-10T20:20:39.815490+0000","last_became_peered":"2023-10-10T20:20:39.815490+0000","last_unstale":"2023-10-10T20:20:39.816251+0000","last_undegraded":"2023-10-10T20:20:39.816251+0000","last_fullsized":"2023-10-10T20:20:39.816251+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:01.629692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810248+0000","last_change":"2023-10-10T20:20:38.818559+0000","last_active":"2023-10-10T20:20:39.810248+0000","last_peered":"2023-10-10T20:20:39.810248+0000","last_clean":"2023-10-10T20:20:39.810248+0000","last_became_active":"2023-10-10T20:20:38.818304+0000","last_became_peered":"2023-10-10T20:20:38.818304+0000","last_unstale":"2023-10-10T20:20:39.810248+0000","last_undegraded":"2023-10-10T20:20:39.810248+0000","last_fullsized":"2023-10-10T20:20:39.810248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:28:18.737399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817803+0000","last_change":"2023-10-10T20:20:39.817734+0000","last_active":"2023-10-10T20:20:39.817803+0000","last_peered":"2023-10-10T20:20:39.817803+0000","last_clean":"2023-10-10T20:20:39.817803+0000","last_became_active":"2023-10-10T20:20:39.817510+0000","last_became_peered":"2023-10-10T20:20:39.817510+0000","last_unstale":"2023-10-10T20:20:39.817803+0000","last_undegraded":"2023-10-10T20:20:39.817803+0000","last_fullsized":"2023-10-10T20:20:39.817803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:11:26.408432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810433+0000","last_change":"2023-10-10T20:20:38.813231+0000","last_active":"2023-10-10T20:20:39.810433+0000","last_peered":"2023-10-10T20:20:39.810433+0000","last_clean":"2023-10-10T20:20:39.810433+0000","last_became_active":"2023-10-10T20:20:38.812830+0000","last_became_peered":"2023-10-10T20:20:38.812830+0000","last_unstale":"2023-10-10T20:20:39.810433+0000","last_undegraded":"2023-10-10T20:20:39.810433+0000","last_fullsized":"2023-10-10T20:20:39.810433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:20:20.226518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.820063+0000","last_change":"2023-10-10T20:20:39.819111+0000","last_active":"2023-10-10T20:20:39.820063+0000","last_peered":"2023-10-10T20:20:39.820063+0000","last_clean":"2023-10-10T20:20:39.820063+0000","last_became_active":"2023-10-10T20:20:39.818514+0000","last_became_peered":"2023-10-10T20:20:39.818514+0000","last_unstale":"2023-10-10T20:20:39.820063+0000","last_undegraded":"2023-10-10T20:20:39.820063+0000","last_fullsized":"2023-10-10T20:20:39.820063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:16.384587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810713+0000","last_change":"2023-10-10T20:20:38.818604+0000","last_active":"2023-10-10T20:20:39.810713+0000","last_peered":"2023-10-10T20:20:39.810713+0000","last_clean":"2023-10-10T20:20:39.810713+0000","last_became_active":"2023-10-10T20:20:38.818328+0000","last_became_peered":"2023-10-10T20:20:38.818328+0000","last_unstale":"2023-10-10T20:20:39.810713+0000","last_undegraded":"2023-10-10T20:20:39.810713+0000","last_fullsized":"2023-10-10T20:20:39.810713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:27.142486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.817513+0000","last_change":"2023-10-10T20:20:39.817433+0000","last_active":"2023-10-10T20:20:39.817513+0000","last_peered":"2023-10-10T20:20:39.817513+0000","last_clean":"2023-10-10T20:20:39.817513+0000","last_became_active":"2023-10-10T20:20:39.816707+0000","last_became_peered":"2023-10-10T20:20:39.816707+0000","last_unstale":"2023-10-10T20:20:39.817513+0000","last_undegraded":"2023-10-10T20:20:39.817513+0000","last_fullsized":"2023-10-10T20:20:39.817513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:03.750215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.811780+0000","last_change":"2023-10-10T20:20:38.814577+0000","last_active":"2023-10-10T20:20:39.811780+0000","last_peered":"2023-10-10T20:20:39.811780+0000","last_clean":"2023-10-10T20:20:39.811780+0000","last_became_active":"2023-10-10T20:20:38.814344+0000","last_became_peered":"2023-10-10T20:20:38.814344+0000","last_unstale":"2023-10-10T20:20:39.811780+0000","last_undegraded":"2023-10-10T20:20:39.811780+0000","last_fullsized":"2023-10-10T20:20:39.811780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:19.572639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.819100+0000","last_change":"2023-10-10T20:20:39.819046+0000","last_active":"2023-10-10T20:20:39.819100+0000","last_peered":"2023-10-10T20:20:39.819100+0000","last_clean":"2023-10-10T20:20:39.819100+0000","last_became_active":"2023-10-10T20:20:39.818908+0000","last_became_peered":"2023-10-10T20:20:39.818908+0000","last_unstale":"2023-10-10T20:20:39.819100+0000","last_undegraded":"2023-10-10T20:20:39.819100+0000","last_fullsized":"2023-10-10T20:20:39.819100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:41.292546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810383+0000","last_change":"2023-10-10T20:20:38.817907+0000","last_active":"2023-10-10T20:20:39.810383+0000","last_peered":"2023-10-10T20:20:39.810383+0000","last_clean":"2023-10-10T20:20:39.810383+0000","last_became_active":"2023-10-10T20:20:38.817683+0000","last_became_peered":"2023-10-10T20:20:38.817683+0000","last_unstale":"2023-10-10T20:20:39.810383+0000","last_undegraded":"2023-10-10T20:20:39.810383+0000","last_fullsized":"2023-10-10T20:20:39.810383+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:08.007313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.816746+0000","last_change":"2023-10-10T20:20:39.816655+0000","last_active":"2023-10-10T20:20:39.816746+0000","last_peered":"2023-10-10T20:20:39.816746+0000","last_clean":"2023-10-10T20:20:39.816746+0000","last_became_active":"2023-10-10T20:20:39.816371+0000","last_became_peered":"2023-10-10T20:20:39.816371+0000","last_unstale":"2023-10-10T20:20:39.816746+0000","last_undegraded":"2023-10-10T20:20:39.816746+0000","last_fullsized":"2023-10-10T20:20:39.816746+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:49:13.903500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810160+0000","last_change":"2023-10-10T20:20:38.814651+0000","last_active":"2023-10-10T20:20:39.810160+0000","last_peered":"2023-10-10T20:20:39.810160+0000","last_clean":"2023-10-10T20:20:39.810160+0000","last_became_active":"2023-10-10T20:20:38.814072+0000","last_became_peered":"2023-10-10T20:20:38.814072+0000","last_unstale":"2023-10-10T20:20:39.810160+0000","last_undegraded":"2023-10-10T20:20:39.810160+0000","last_fullsized":"2023-10-10T20:20:39.810160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:11.756984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.818149+0000","last_change":"2023-10-10T20:20:39.818091+0000","last_active":"2023-10-10T20:20:39.818149+0000","last_peered":"2023-10-10T20:20:39.818149+0000","last_clean":"2023-10-10T20:20:39.818149+0000","last_became_active":"2023-10-10T20:20:39.817850+0000","last_became_peered":"2023-10-10T20:20:39.817850+0000","last_unstale":"2023-10-10T20:20:39.818149+0000","last_undegraded":"2023-10-10T20:20:39.818149+0000","last_fullsized":"2023-10-10T20:20:39.818149+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:38.800638+0000","last_clean_scrub_stamp":"2023-10-10T20:20:38.800638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:03:52.659983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.829062+0000","last_change":"2023-10-10T20:20:38.812366+0000","last_active":"2023-10-10T20:20:39.829062+0000","last_peered":"2023-10-10T20:20:39.829062+0000","last_clean":"2023-10-10T20:20:39.829062+0000","last_became_active":"2023-10-10T20:20:38.812137+0000","last_became_peered":"2023-10-10T20:20:38.812137+0000","last_unstale":"2023-10-10T20:20:39.829062+0000","last_undegraded":"2023-10-10T20:20:39.829062+0000","last_fullsized":"2023-10-10T20:20:39.829062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:20:37.794040+0000","last_clean_scrub_stamp":"2023-10-10T20:20:37.794040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:58:07.656181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":183,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-10T20:20:39.810338+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:20:39.810338+0000","last_peered":"2023-10-10T20:20:39.810338+0000","last_clean":"2023-10-10T20:20:39.810338+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:20:39.810338+0000","last_undegraded":"2023-10-10T20:20:39.810338+0000","last_fullsized":"2023-10-10T20:20:39.810338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194533,"statfs":{"total":200000000,"available":199202119,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.371},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44400000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785645,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.27500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194528,"statfs":{"total":200000000,"available":199196898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.75600000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":9.7260000000000009,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.747},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785742,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.23400000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.249},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.38},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.307}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.309},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.504},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.625},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199784753,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58099999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.63200000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.753},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199786219,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.63}]},{"osd":6,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995}]}]}],"pool_statfs":[]}} 2023-10-10T20:20:44.239 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:20:44.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:20:44.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.393+0000 7f66709d0700 1 Processor -- start 2023-10-10T20:20:44.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.393+0000 7f66709d0700 1 -- start start 2023-10-10T20:20:44.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f66709d0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 0x7f666c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f66709d0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 0x7f666c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f66709d0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c0b0790 con 0x7f666c15a290 2023-10-10T20:20:44.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f66709d0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c0ce1f0 con 0x7f666c06acb0 2023-10-10T20:20:44.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f66709d0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0b0b20 con 0x7f666c159ec0 2023-10-10T20:20:44.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f666a59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 0x7f666c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f666ad9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 0x7f666c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f6663fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f666c15a290 0x7f666c06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:40786/0 (socket says 172.21.15.148:40786) 2023-10-10T20:20:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.394+0000 7f6663fff700 1 -- 172.21.15.148:0/2672874524 learned_addr learned my addr 172.21.15.148:0/2672874524 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f6669d9b700 1 -- 172.21.15.148:0/2672874524 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147981796 0 0) 0x7f666c0b0790 con 0x7f666c15a290 2023-10-10T20:20:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666a59c700 1 -- 172.21.15.148:0/2672874524 >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 msgr2=0x7f666c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666a59c700 1 --2- 172.21.15.148:0/2672874524 >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 0x7f666c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666a59c700 1 -- 172.21.15.148:0/2672874524 >> v1:172.21.15.161:6789/0 conn(0x7f666c15a290 legacy=0x7f666c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666a59c700 1 -- 172.21.15.148:0/2672874524 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0b04b0 con 0x7f666c159ec0 2023-10-10T20:20:44.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666ad9d700 1 --2- 172.21.15.148:0/2672874524 >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 0x7f666c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:44.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f666a59c700 1 --2- 172.21.15.148:0/2672874524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 0x7f666c0af940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f665c0098e0 tx=0x7f665c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841d41d82928e37f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f6669d9b700 1 -- 172.21.15.148:0/2672874524 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00b640 con 0x7f666c159ec0 2023-10-10T20:20:44.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f6669d9b700 1 -- 172.21.15.148:0/2672874524 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00b7a0 con 0x7f666c159ec0 2023-10-10T20:20:44.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.395+0000 7f6669d9b700 1 -- 172.21.15.148:0/2672874524 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00bb40 con 0x7f666c159ec0 2023-10-10T20:20:44.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 -- 172.21.15.148:0/2672874524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 msgr2=0x7f666c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 --2- 172.21.15.148:0/2672874524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 0x7f666c0af940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f665c0098e0 tx=0x7f665c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 -- 172.21.15.148:0/2672874524 shutdown_connections 2023-10-10T20:20:44.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 --2- 172.21.15.148:0/2672874524 >> v2:172.21.15.161:3300/0 conn(0x7f666c06acb0 0x7f666c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 --2- 172.21.15.148:0/2672874524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c159ec0 0x7f666c0af940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 -- 172.21.15.148:0/2672874524 >> 172.21.15.148:0/2672874524 conn(0x7f666c0d1880 msgr2=0x7f666c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:44.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 -- 172.21.15.148:0/2672874524 shutdown_connections 2023-10-10T20:20:44.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.396+0000 7f66709d0700 1 -- 172.21.15.148:0/2672874524 wait complete. 2023-10-10T20:20:44.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.397+0000 7f66709d0700 1 Processor -- start 2023-10-10T20:20:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.397+0000 7f66709d0700 1 -- start start 2023-10-10T20:20:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f666c159ec0 0x7f666c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c15a290 0x7f666c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0c9ff0 con 0x7f666c06acb0 2023-10-10T20:20:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0c93e0 con 0x7f666c159ec0 2023-10-10T20:20:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0c96c0 con 0x7f666c15a290 2023-10-10T20:20:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53466/0 (socket says 172.21.15.148:53466) 2023-10-10T20:20:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 -- 172.21.15.148:0/2969337299 learned_addr learned my addr 172.21.15.148:0/2969337299 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:44.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f6663fff700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f666c159ec0 0x7f666c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666ad9d700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c15a290 0x7f666c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 -- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c15a290 msgr2=0x7f666c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c15a290 0x7f666c0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 -- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f666c159ec0 msgr2=0x7f666c15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f666c159ec0 0x7f666c15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 -- 172.21.15.148:0/2969337299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664009a20 con 0x7f666c06acb0 2023-10-10T20:20:44.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f666a59c700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f665c0093f0 tx=0x7f665c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c01f070 con 0x7f666c06acb0 2023-10-10T20:20:44.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00b910 con 0x7f666c06acb0 2023-10-10T20:20:44.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c011420 con 0x7f666c06acb0 2023-10-10T20:20:44.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c009420 con 0x7f666c06acb0 2023-10-10T20:20:44.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.398+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c0c9c80 con 0x7f666c06acb0 2023-10-10T20:20:44.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.399+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c04c3c0 con 0x7f666c06acb0 2023-10-10T20:20:44.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.400+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f665c011810 con 0x7f666c06acb0 2023-10-10T20:20:44.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.400+0000 7f66627fc700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 0x7f664c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.400+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f665c085870 con 0x7f666c06acb0 2023-10-10T20:20:44.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.400+0000 7f6663fff700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 0x7f664c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.401+0000 7f6663fff700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 0x7f664c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f666c155fb0 tx=0x7f6654007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.404+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f665c052ae0 con 0x7f666c06acb0 2023-10-10T20:20:44.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.658+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f666c0ceab0 con 0x7f666c06acb0 2023-10-10T20:20:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.662+0000 7f66627fc700 1 -- 172.21.15.148:0/2969337299 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f665c028020 con 0x7f666c06acb0 2023-10-10T20:20:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.664+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 msgr2=0x7f664c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.664+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 0x7f664c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f666c155fb0 tx=0x7f6654007f50 comp rx=0 tx=0).stop 2023-10-10T20:20:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.664+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 msgr2=0x7f666c15f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.664+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f665c0093f0 tx=0x7f665c017000 comp rx=0 tx=0).stop 2023-10-10T20:20:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 shutdown_connections 2023-10-10T20:20:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f664c066f30 0x7f664c0693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f666c15a290 0x7f666c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666c06acb0 0x7f666c15f060 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 --2- 172.21.15.148:0/2969337299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f666c159ec0 0x7f666c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 >> 172.21.15.148:0/2969337299 conn(0x7f666c0d1880 msgr2=0x7f666c0b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:44.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 shutdown_connections 2023-10-10T20:20:44.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.665+0000 7f66709d0700 1 -- 172.21.15.148:0/2969337299 wait complete. 2023-10-10T20:20:44.668 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:20:44.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:44.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.792+0000 7f517064c700 1 Processor -- start 2023-10-10T20:20:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.792+0000 7f517064c700 1 -- start start 2023-10-10T20:20:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f517064c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f51680c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f517064c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51680cb4f0 0x7f51680cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f517064c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516815de60 con 0x7f51680b1220 2023-10-10T20:20:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f517064c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51680cbed0 con 0x7f51680cb4f0 2023-10-10T20:20:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f517064c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516815e140 con 0x7f51680b0e50 2023-10-10T20:20:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.793+0000 7f516ebe9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51680cb4f0 0x7f51680cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51680cb4f0 0x7f51680cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53470/0 (socket says 172.21.15.148:53470) 2023-10-10T20:20:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 -- 172.21.15.148:0/2569023200 learned_addr learned my addr 172.21.15.148:0/2569023200 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516e3e8700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f51680c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 -- 172.21.15.148:0/2569023200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 msgr2=0x7f51680c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f51680c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 -- 172.21.15.148:0/2569023200 >> v1:172.21.15.161:6789/0 conn(0x7f51680b1220 legacy=0x7f516806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 -- 172.21.15.148:0/2569023200 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516815e420 con 0x7f51680cb4f0 2023-10-10T20:20:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516ebe9700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680cb4f0 0x7f51680cb990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5160009f20 tx=0x7f5160009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570f3287f1419336 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516d3e6700 1 -- 172.21.15.148:0/2569023200 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000e070 con 0x7f51680cb4f0 2023-10-10T20:20:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516d3e6700 1 -- 172.21.15.148:0/2569023200 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516000b9d0 con 0x7f51680cb4f0 2023-10-10T20:20:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.794+0000 7f516d3e6700 1 -- 172.21.15.148:0/2569023200 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600092e0 con 0x7f51680cb4f0 2023-10-10T20:20:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 -- 172.21.15.148:0/2569023200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680cb4f0 msgr2=0x7f51680cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680cb4f0 0x7f51680cb990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5160009f20 tx=0x7f5160009840 comp rx=0 tx=0).stop 2023-10-10T20:20:44.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 -- 172.21.15.148:0/2569023200 shutdown_connections 2023-10-10T20:20:44.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680cb4f0 0x7f51680cb990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 --2- 172.21.15.148:0/2569023200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f51680c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 -- 172.21.15.148:0/2569023200 >> 172.21.15.148:0/2569023200 conn(0x7f5168069870 msgr2=0x7f5168069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:44.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 -- 172.21.15.148:0/2569023200 shutdown_connections 2023-10-10T20:20:44.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.795+0000 7f517064c700 1 -- 172.21.15.148:0/2569023200 wait complete. 2023-10-10T20:20:44.802 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-10-10T20:20:44.796+0000 7fe592c76700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:20:44.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.796+0000 7f517064c700 1 Processor -- start 2023-10-10T20:20:44.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.796+0000 7f517064c700 1 -- start start 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f5168067a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51680cb4f0 0x7f5168153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516815b500 con 0x7f51680b1220 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516815a980 con 0x7f51680cb4f0 2023-10-10T20:20:44.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516815ac60 con 0x7f51680b0e50 2023-10-10T20:20:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53484/0 (socket says 172.21.15.148:53484) 2023-10-10T20:20:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516e3e8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f5168067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516ebe9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51680cb4f0 0x7f5168153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 -- 172.21.15.148:0/3211989830 learned_addr learned my addr 172.21.15.148:0/3211989830 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 -- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 msgr2=0x7f5168067a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f5168067a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 -- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51680cb4f0 msgr2=0x7f5168153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51680cb4f0 0x7f5168153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:44.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 -- 172.21.15.148:0/3211989830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009ae0 con 0x7f51680b1220 2023-10-10T20:20:44.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f516dbe7700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f515800c590 tx=0x7f515800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011590 con 0x7f51680b1220 2023-10-10T20:20:44.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5158011710 con 0x7f51680b1220 2023-10-10T20:20:44.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516815af40 con 0x7f51680b1220 2023-10-10T20:20:44.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.797+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011ad0 con 0x7f51680b1220 2023-10-10T20:20:44.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.798+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5168067fb0 con 0x7f51680b1220 2023-10-10T20:20:44.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.799+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516815b980 con 0x7f51680b1220 2023-10-10T20:20:44.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.799+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f515801b3b0 con 0x7f51680b1220 2023-10-10T20:20:44.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.799+0000 7f515e7fc700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 0x7f514c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:44.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.799+0000 7f516e3e8700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 0x7f514c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:44.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.799+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5158083a60 con 0x7f51680b1220 2023-10-10T20:20:44.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.800+0000 7f516e3e8700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 0x7f514c0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5164009db0 tx=0x7f51640091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:44.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:44.804+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f516815b980 con 0x7f51680b1220 2023-10-10T20:20:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.025+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f516815b980 con 0x7f51680b1220 2023-10-10T20:20:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.025+0000 7f515e7fc700 1 -- 172.21.15.148:0/3211989830 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f516815b980 con 0x7f51680b1220 2023-10-10T20:20:45.027 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:45.027 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:20:45.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 msgr2=0x7f514c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 0x7f514c0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5164009db0 tx=0x7f51640091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 msgr2=0x7f5168153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f515800c590 tx=0x7f515800c8a0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 shutdown_connections 2023-10-10T20:20:45.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f514c066f30 0x7f514c0693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51680cb4f0 0x7f5168153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f51680b0e50 0x7f5168067a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 --2- 172.21.15.148:0/3211989830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51680b1220 0x7f5168153950 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 >> 172.21.15.148:0/3211989830 conn(0x7f5168069870 msgr2=0x7f5168159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 shutdown_connections 2023-10-10T20:20:45.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.028+0000 7f517064c700 1 -- 172.21.15.148:0/3211989830 wait complete. 2023-10-10T20:20:45.032 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:20:45.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:20:39.184854+0000', 'modified': '2023-10-10T20:20:42.828507+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5367}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5367': {'gid': 5367, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/3613275465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 3613275465}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3613275465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:20:45.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:20:45.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:20:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.559+0000 7f5cf359e700 1 Processor -- start 2023-10-10T20:20:45.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.559+0000 7f5cf359e700 1 -- start start 2023-10-10T20:20:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5cf415a170 0x7f5cf4068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5cf4069040 0x7f5cf40af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf40694e0 con 0x7f5cf4069040 2023-10-10T20:20:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf415ea70 con 0x7f5cf415a170 2023-10-10T20:20:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40b3990 con 0x7f5cf4159da0 2023-10-10T20:20:45.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf2d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5cf4069040 0x7f5cf40af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf2d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5cf4069040 0x7f5cf40af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53490/0 (socket says 172.21.15.148:53490) 2023-10-10T20:20:45.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/936731550 learned_addr learned my addr 172.21.15.148:0/936731550 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cebfff700 1 --2- 172.21.15.148:0/936731550 >> v2:172.21.15.161:3300/0 conn(0x7f5cf415a170 0x7f5cf4068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.560+0000 7f5cf259c700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/936731550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 msgr2=0x7f5cf40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/936731550 >> v2:172.21.15.161:3300/0 conn(0x7f5cf415a170 msgr2=0x7f5cf4068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/936731550 >> v2:172.21.15.161:3300/0 conn(0x7f5cf415a170 0x7f5cf4068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/936731550 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40b0b70 con 0x7f5cf4069040 2023-10-10T20:20:45.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf259c700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:45.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf4069040 0x7f5cf40af4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5cf406b2c0 tx=0x7f5cec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628e1fdb3763f3a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf1d9b700 1 -- 172.21.15.148:0/936731550 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00e070 con 0x7f5cf4069040 2023-10-10T20:20:45.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf1d9b700 1 -- 172.21.15.148:0/936731550 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00ba20 con 0x7f5cf4069040 2023-10-10T20:20:45.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf1d9b700 1 -- 172.21.15.148:0/936731550 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0092e0 con 0x7f5cf4069040 2023-10-10T20:20:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 -- 172.21.15.148:0/936731550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf4069040 msgr2=0x7f5cf40af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf4069040 0x7f5cf40af4a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5cf406b2c0 tx=0x7f5cec009840 comp rx=0 tx=0).stop 2023-10-10T20:20:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 -- 172.21.15.148:0/936731550 shutdown_connections 2023-10-10T20:20:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf4069040 0x7f5cf40af4a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 --2- 172.21.15.148:0/936731550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 --2- 172.21.15.148:0/936731550 >> v2:172.21.15.161:3300/0 conn(0x7f5cf415a170 0x7f5cf4068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 -- 172.21.15.148:0/936731550 >> 172.21.15.148:0/936731550 conn(0x7f5cf40d0bc0 msgr2=0x7f5cf40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.561+0000 7f5cf359e700 1 -- 172.21.15.148:0/936731550 shutdown_connections 2023-10-10T20:20:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.562+0000 7f5cf359e700 1 -- 172.21.15.148:0/936731550 wait complete. 2023-10-10T20:20:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 Processor -- start 2023-10-10T20:20:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 -- start start 2023-10-10T20:20:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cf4069040 0x7f5cf4147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c5560 con 0x7f5cf415a170 2023-10-10T20:20:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c50d0 con 0x7f5cf4069040 2023-10-10T20:20:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c44c0 con 0x7f5cf4159da0 2023-10-10T20:20:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53496/0 (socket says 172.21.15.148:53496) 2023-10-10T20:20:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/197460383 learned_addr learned my addr 172.21.15.148:0/197460383 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cf259c700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cf4069040 0x7f5cf4147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.563+0000 7f5cebfff700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 msgr2=0x7f5cf40cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cf4069040 msgr2=0x7f5cf4147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cf4069040 0x7f5cf4147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/197460383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec009ae0 con 0x7f5cf415a170 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b6c0 tx=0x7f5cec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cea7fc700 1 -- 172.21.15.148:0/197460383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0092e0 con 0x7f5cf415a170 2023-10-10T20:20:45.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cea7fc700 1 -- 172.21.15.148:0/197460383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec007000 con 0x7f5cf415a170 2023-10-10T20:20:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cea7fc700 1 -- 172.21.15.148:0/197460383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec01d420 con 0x7f5cf415a170 2023-10-10T20:20:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40c47a0 con 0x7f5cf415a170 2023-10-10T20:20:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.564+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40c4d60 con 0x7f5cf415a170 2023-10-10T20:20:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.565+0000 7f5cea7fc700 1 -- 172.21.15.148:0/197460383 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5cec003050 con 0x7f5cf415a170 2023-10-10T20:20:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.566+0000 7f5cea7fc700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 0x7f5cd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.566+0000 7f5cea7fc700 1 -- 172.21.15.148:0/197460383 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cec085fc0 con 0x7f5cf415a170 2023-10-10T20:20:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.566+0000 7f5cf259c700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 0x7f5cd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.566+0000 7f5cf259c700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 0x7f5cd40693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4006f00 tx=0x7f5ce4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 Processor -- start 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 -- start start 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad4293f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4286ca70 con 0x55ad42915b80 2023-10-10T20:20:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4286a270 con 0x55ad429135f0 2023-10-10T20:20:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.568+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad42866060 con 0x55ad42960480 2023-10-10T20:20:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53502/0 (socket says 172.21.15.148:53502) 2023-10-10T20:20:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/3893217034 learned_addr learned my addr 172.21.15.148:0/3893217034 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 msgr2=0x55ad4293f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad4293f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 msgr2=0x55ad42960850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/3893217034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad42700270 con 0x55ad42915b80 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0844e0 tx=0x7f5cec084440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa33d9ccdb9e7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cd37fe700 1 -- 172.21.15.148:0/3893217034 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00e040 con 0x55ad42915b80 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cd37fe700 1 -- 172.21.15.148:0/3893217034 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec0575f0 con 0x55ad42915b80 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.569+0000 7f5cd37fe700 1 -- 172.21.15.148:0/3893217034 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec056ef0 con 0x55ad42915b80 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- 172.21.15.148:0/3893217034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 msgr2=0x55ad42915ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0844e0 tx=0x7f5cec084440 comp rx=0 tx=0).stop 2023-10-10T20:20:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- 172.21.15.148:0/3893217034 shutdown_connections 2023-10-10T20:20:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42915ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad4293f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/3893217034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- 172.21.15.148:0/3893217034 >> 172.21.15.148:0/3893217034 conn(0x55ad429170c0 msgr2=0x55ad429178f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- 172.21.15.148:0/3893217034 shutdown_connections 2023-10-10T20:20:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- 172.21.15.148:0/3893217034 wait complete. 2023-10-10T20:20:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.570+0000 7f5d01bafb80 1 -- start start 2023-10-10T20:20:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 Processor -- start 2023-10-10T20:20:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42940630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42915b80 0x55ad4295f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad42811890 con 0x55ad42960480 2023-10-10T20:20:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4282cea0 con 0x55ad429135f0 2023-10-10T20:20:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad426e8e30 con 0x55ad42915b80 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.571+0000 7f5d01bafb80 10 client.0 _ll_register_callbacks cb 0x55ad428d92c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53512/0 (socket says 172.21.15.148:53512) 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/83973243 learned_addr learned my addr 172.21.15.148:0/83973243 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf259c700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42940630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cebfff700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42915b80 0x55ad4295f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42915b80 msgr2=0x55ad4295f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42915b80 0x55ad4295f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 msgr2=0x55ad42940630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42940630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/83973243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40c4d60 con 0x55ad42960480 2023-10-10T20:20:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0533f0 tx=0x7f5cec06c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3e5d1b47adb5c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:20:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00e070 con 0x55ad42960480 2023-10-10T20:20:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cec05de90 con 0x55ad42960480 2023-10-10T20:20:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad4290acd0 con 0x55ad42960480 2023-10-10T20:20:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cec060050 con 0x55ad42960480 2023-10-10T20:20:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 10 client.5493 fetch_fsmap learned FSMap version 24 2023-10-10T20:20:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ad426a2fe0 con 0x55ad42960480 2023-10-10T20:20:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad4290acd0 con 0x55ad42960480 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5cec063020 con 0x55ad42960480 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 10 client.5493 fetch_fsmap finished waiting for FSMap version 24 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 10 client.5493 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 20 client.5493 populate_metadata read hostname 'smithi148' 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4295c430 0x55ad4295cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 4 client.5493 mds_command: new command op to 5367 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:20:45.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.572+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 --> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad428e7aa0 con 0x55ad4295c430 2023-10-10T20:20:45.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.573+0000 7f5cf259c700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4295c430 0x55ad4295cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.573+0000 7f5cd0ff9700 10 client.5493 ms_handle_connect on v2:172.21.15.161:6833/3613275465 2023-10-10T20:20:45.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.573+0000 7f5cf259c700 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4295c430 0x55ad4295cbc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e989cd8586047ac7 server_cookie=32a119e512d2626d in_seq=0 out_seq=0 2023-10-10T20:20:45.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/83973243 <== mds.0 v2:172.21.15.161:6833/3613275465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ad428e7aa0 con 0x55ad4295c430 2023-10-10T20:20:45.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5cd0ff9700 10 client.5493 handle_command_reply: tid=0 2023-10-10T20:20:45.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 client.5493 shutdown 2023-10-10T20:20:45.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 msgr2=0x55ad42945540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0533f0 tx=0x7f5cec06c000 comp rx=0 tx=0).stop 2023-10-10T20:20:45.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 shutdown_connections 2023-10-10T20:20:45.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4295c430 0x55ad4295cbc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42945540 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad42940630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/83973243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42915b80 0x55ad4295f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5cd0ff9700 0 client.5493 ms_handle_reset on v2:172.21.15.161:6833/3613275465 2023-10-10T20:20:45.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 >> 172.21.15.148:0/83973243 conn(0x55ad429170c0 msgr2=0x55ad4293b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 shutdown_connections 2023-10-10T20:20:45.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 1 -- 172.21.15.148:0/83973243 wait complete. 2023-10-10T20:20:45.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.574+0000 7f5d01bafb80 20 client.5493 trim_cache size 0 max 16384 2023-10-10T20:20:45.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 Processor -- start 2023-10-10T20:20:45.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 -- start start 2023-10-10T20:20:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 0x55ad42960d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad42913450 con 0x55ad42960480 2023-10-10T20:20:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4295edb0 con 0x55ad42915b80 2023-10-10T20:20:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad428f5510 con 0x55ad429135f0 2023-10-10T20:20:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.591+0000 7f5cf2d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53522/0 (socket says 172.21.15.148:53522) 2023-10-10T20:20:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/2202920071 learned_addr learned my addr 172.21.15.148:0/2202920071 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cebfff700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 0x55ad42960d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf259c700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad429135f0 msgr2=0x55ad42960850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 msgr2=0x55ad42960d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 0x55ad42960d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 -- 172.21.15.148:0/2202920071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad42834780 con 0x55ad42960480 2023-10-10T20:20:45.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cebfff700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 0x55ad42960d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:45.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cec084170 tx=0x7f5cec06b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adbd72722a679a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cd3fff700 1 -- 172.21.15.148:0/2202920071 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec060070 con 0x55ad42960480 2023-10-10T20:20:45.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cd3fff700 1 -- 172.21.15.148:0/2202920071 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec05de90 con 0x55ad42960480 2023-10-10T20:20:45.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5cd3fff700 1 -- 172.21.15.148:0/2202920071 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00e020 con 0x55ad42960480 2023-10-10T20:20:45.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5d01bafb80 1 -- 172.21.15.148:0/2202920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 msgr2=0x55ad42911e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5cec084170 tx=0x7f5cec06b040 comp rx=0 tx=0).stop 2023-10-10T20:20:45.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5d01bafb80 1 -- 172.21.15.148:0/2202920071 shutdown_connections 2023-10-10T20:20:45.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.592+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42960480 0x55ad42911e60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad429135f0 0x55ad42960850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/2202920071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad42915b80 0x55ad42960d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 -- 172.21.15.148:0/2202920071 >> 172.21.15.148:0/2202920071 conn(0x55ad429170c0 msgr2=0x55ad4295e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 -- 172.21.15.148:0/2202920071 shutdown_connections 2023-10-10T20:20:45.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 -- 172.21.15.148:0/2202920071 wait complete. 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 -- start start 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.593+0000 7f5d01bafb80 1 Processor -- start 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad4295db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad429123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4293cad0 con 0x55ad42915b80 2023-10-10T20:20:45.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4295d360 con 0x55ad429135f0 2023-10-10T20:20:45.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad4293aa20 con 0x55ad42960480 2023-10-10T20:20:45.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5d01bafb80 10 client.0 _ll_register_callbacks cb 0x55ad428d92c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:20:45.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53532/0 (socket says 172.21.15.148:53532) 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 -- 172.21.15.148:0/16639464 learned_addr learned my addr 172.21.15.148:0/16639464 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cf259c700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad4295db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cf2d9d700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad429123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 -- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 msgr2=0x55ad429123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad429123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 -- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 msgr2=0x55ad4295db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad4295db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.594+0000 7f5cebfff700 1 -- 172.21.15.148:0/16639464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad426a2fe0 con 0x55ad42915b80 2023-10-10T20:20:45.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cebfff700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55ad42914d50 tx=0x7f5cdc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255431425f8bacaa server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:20:45.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc014720 con 0x55ad42915b80 2023-10-10T20:20:45.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cdc014880 con 0x55ad42915b80 2023-10-10T20:20:45.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cec0575f0 con 0x55ad42915b80 2023-10-10T20:20:45.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cdc014c50 con 0x55ad42915b80 2023-10-10T20:20:45.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 10 client.5499 fetch_fsmap learned FSMap version 24 2023-10-10T20:20:45.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ad4281b470 con 0x55ad42915b80 2023-10-10T20:20:45.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec0575f0 con 0x55ad42915b80 2023-10-10T20:20:45.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5cdc020470 con 0x55ad42915b80 2023-10-10T20:20:45.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 10 client.5499 fetch_fsmap finished waiting for FSMap version 24 2023-10-10T20:20:45.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 10 client.5499 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:20:45.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 20 client.5499 populate_metadata read hostname 'smithi148' 2023-10-10T20:20:45.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4293be00 0x55ad4293c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 4 client.5499 mds_command: new command op to 5367 tid=0[{"prefix": "status"}] 2023-10-10T20:20:45.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 --> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ad42808e00 con 0x55ad4293be00 2023-10-10T20:20:45.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.595+0000 7f5cf259c700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4293be00 0x55ad4293c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5cd0ff9700 10 client.5499 ms_handle_connect on v2:172.21.15.161:6833/3613275465 2023-10-10T20:20:45.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5cf259c700 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4293be00 0x55ad4293c1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd54bc5bb700187 server_cookie=8751d0cb7b80d3ac in_seq=0 out_seq=0 2023-10-10T20:20:45.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5cd0ff9700 1 -- 172.21.15.148:0/16639464 <== mds.0 v2:172.21.15.161:6833/3613275465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ad42808e00 con 0x55ad4293be00 2023-10-10T20:20:45.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5cd0ff9700 10 client.5499 handle_command_reply: tid=0 2023-10-10T20:20:45.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 client.5499 shutdown 2023-10-10T20:20:45.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 msgr2=0x55ad42911e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55ad42914d50 tx=0x7f5cdc008040 comp rx=0 tx=0).stop 2023-10-10T20:20:45.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 shutdown_connections 2023-10-10T20:20:45.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ad42960480 0x55ad429123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad429135f0 0x55ad4295db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.161:6833/3613275465,v1:172.21.15.161:6835/3613275465] conn(0x55ad4293be00 0x55ad4293c1d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 --2- 172.21.15.148:0/16639464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad42915b80 0x55ad42911e60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5cd0ff9700 0 client.5499 ms_handle_reset on v2:172.21.15.161:6833/3613275465 2023-10-10T20:20:45.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 >> 172.21.15.148:0/16639464 conn(0x55ad429170c0 msgr2=0x55ad4295e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 shutdown_connections 2023-10-10T20:20:45.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 1 -- 172.21.15.148:0/16639464 wait complete. 2023-10-10T20:20:45.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.596+0000 7f5d01bafb80 20 client.5499 trim_cache size 0 max 16384 2023-10-10T20:20:45.622 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:45.622 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:45.622 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:20:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5367, 2023-10-10T20:20:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:20:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:20:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:20:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.776527915, 2023-10-10T20:20:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 24, 2023-10-10T20:20:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 45, 2023-10-10T20:20:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 45, 2023-10-10T20:20:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 11.743769761999999 2023-10-10T20:20:45.624 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:45.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 msgr2=0x7f5cd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 0x7f5cd40693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4006f00 tx=0x7f5ce4008040 comp rx=0 tx=0).stop 2023-10-10T20:20:45.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 msgr2=0x7f5cf40cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b6c0 tx=0x7f5cec018000 comp rx=0 tx=0).stop 2023-10-10T20:20:45.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 shutdown_connections 2023-10-10T20:20:45.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cd4066f30 0x7f5cd40693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf415a170 0x7f5cf40cfc00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cf4069040 0x7f5cf4147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 --2- 172.21.15.148:0/197460383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5cf4159da0 0x7f5cf40cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 >> 172.21.15.148:0/197460383 conn(0x7f5cf40d0bc0 msgr2=0x7f5cf40cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 shutdown_connections 2023-10-10T20:20:45.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.597+0000 7f5cf359e700 1 -- 172.21.15.148:0/197460383 wait complete. 2023-10-10T20:20:45.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:20:45.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.742+0000 7f4fe43f6700 1 Processor -- start 2023-10-10T20:20:45.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.742+0000 7f4fe43f6700 1 -- start start 2023-10-10T20:20:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe43f6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fdc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe43f6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc067ba0 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe43f6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc067870 con 0x7f4fdc0ad9b0 2023-10-10T20:20:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe43f6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc15aa90 con 0x7f4fdc1568d0 2023-10-10T20:20:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe2993700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4fdc0c87a0 0x7f4fdc068030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60814/0 (socket says 172.21.15.148:60814) 2023-10-10T20:20:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.743+0000 7f4fe2993700 1 -- 172.21.15.148:0/3878731908 learned_addr learned my addr 172.21.15.148:0/3878731908 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe2192700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fdc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055949429 0 0) 0x7f4fdc067ba0 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003570 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128740075 0 0) 0x7f4fc8003570 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 msgr2=0x7f4fdc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fdc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 >> v1:172.21.15.161:6789/0 conn(0x7f4fdc0ad9b0 legacy=0x7f4fdc0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0c8ca0 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f4fd40017f0 con 0x7f4fdc0c87a0 2023-10-10T20:20:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 >> v1:172.21.15.148:6789/0 conn(0x7f4fdc0c87a0 legacy=0x7f4fdc068030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fc8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc8004d50 0x7f4fc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc15aa90 con 0x7f4fc8004010 2023-10-10T20:20:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800bbf0 con 0x7f4fc8004d50 2023-10-10T20:20:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800dfc0 con 0x7f4fdc1568d0 2023-10-10T20:20:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe2192700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fc8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe2993700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc8004d50 0x7f4fc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 -- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 msgr2=0x7f4fc8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1568d0 0x7f4fc8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 -- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc8004d50 msgr2=0x7f4fc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:45.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fc8004d50 0x7f4fc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.744+0000 7f4fe1991700 1 -- 172.21.15.148:0/3878731908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8003570 con 0x7f4fc8004010 2023-10-10T20:20:45.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe1991700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fc8004810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ade00 tx=0x7f4fcc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71510b2e3a2781c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc012420 con 0x7f4fc8004010 2023-10-10T20:20:45.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc012580 con 0x7f4fc8004010 2023-10-10T20:20:45.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe1190700 1 -- 172.21.15.148:0/3878731908 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc012920 con 0x7f4fc8004010 2023-10-10T20:20:45.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 -- 172.21.15.148:0/3878731908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 msgr2=0x7f4fc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/3878731908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fc8004810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ade00 tx=0x7f4fcc007d30 comp rx=0 tx=0).stop 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe2993700 1 -- 172.21.15.148:0/3878731908 reap_dead start 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 -- 172.21.15.148:0/3878731908 shutdown_connections 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 -- 172.21.15.148:0/3878731908 >> 172.21.15.148:0/3878731908 conn(0x7f4fdc0cda50 msgr2=0x7f4fdc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 -- 172.21.15.148:0/3878731908 shutdown_connections 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.745+0000 7f4fe43f6700 1 -- 172.21.15.148:0/3878731908 wait complete. 2023-10-10T20:20:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.746+0000 7f4fe43f6700 1 Processor -- start 2023-10-10T20:20:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 -- start start 2023-10-10T20:20:45.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fdc0af430 0x7f4fdc0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1583f0 0x7f4fdc158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0afde0 con 0x7f4fc8004010 2023-10-10T20:20:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b2750 con 0x7f4fdc0af430 2023-10-10T20:20:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe43f6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b32d0 con 0x7f4fdc1583f0 2023-10-10T20:20:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe2192700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe2192700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53540/0 (socket says 172.21.15.148:53540) 2023-10-10T20:20:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe2192700 1 -- 172.21.15.148:0/1463358054 learned_addr learned my addr 172.21.15.148:0/1463358054 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.747+0000 7f4fe1991700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fdc0af430 0x7f4fdc0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2993700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1583f0 0x7f4fdc158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 -- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1583f0 msgr2=0x7f4fdc158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1583f0 0x7f4fdc158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 -- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fdc0af430 msgr2=0x7f4fdc0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fdc0af430 0x7f4fdc0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 -- 172.21.15.148:0/1463358054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0b3760 con 0x7f4fc8004010 2023-10-10T20:20:45.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe2192700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8011d40 tx=0x7f4fd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800fac0 con 0x7f4fc8004010 2023-10-10T20:20:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800fc20 con 0x7f4fc8004010 2023-10-10T20:20:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc0b29a0 con 0x7f4fc8004010 2023-10-10T20:20:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8014440 con 0x7f4fc8004010 2023-10-10T20:20:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.748+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc0b2f60 con 0x7f4fc8004010 2023-10-10T20:20:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.749+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc04c3c0 con 0x7f4fc8004010 2023-10-10T20:20:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.749+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fd8014870 con 0x7f4fc8004010 2023-10-10T20:20:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.750+0000 7f4fd27fc700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 0x7f4fc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.750+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fd8018030 con 0x7f4fc8004010 2023-10-10T20:20:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.750+0000 7f4fe1991700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 0x7f4fc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.750+0000 7f4fe1991700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 0x7f4fc00693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc000fb0 tx=0x7f4fcc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:45.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.754+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fd8052260 con 0x7f4fc8004010 2023-10-10T20:20:45.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.982+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fdc0c8f80 con 0x7f4fc8004010 2023-10-10T20:20:45.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.983+0000 7f4fd27fc700 1 -- 172.21.15.148:0/1463358054 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4fd8022070 con 0x7f4fc8004010 2023-10-10T20:20:45.984 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:20:45.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.984+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 msgr2=0x7f4fc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.984+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 0x7f4fc00693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc000fb0 tx=0x7f4fcc015000 comp rx=0 tx=0).stop 2023-10-10T20:20:45.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.984+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 msgr2=0x7f4fdc152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:45.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.984+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8011d40 tx=0x7f4fd80091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 shutdown_connections 2023-10-10T20:20:45.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4fc0066f30 0x7f4fc00693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4fdc1583f0 0x7f4fdc158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc8004010 0x7f4fdc152360 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 --2- 172.21.15.148:0/1463358054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fdc0af430 0x7f4fdc0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:45.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 >> 172.21.15.148:0/1463358054 conn(0x7f4fdc0cda50 msgr2=0x7f4fdc06aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:45.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 shutdown_connections 2023-10-10T20:20:45.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:45.985+0000 7f4fe43f6700 1 -- 172.21.15.148:0/1463358054 wait complete. 2023-10-10T20:20:45.997 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:20:45.998 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:20:45.998 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:20:45.998 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:20:45.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:45.999 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:20:46.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:20:46.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:20:46.000 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:46.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.001 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:20:46.017 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:20:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:20:46.017 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:20:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:20:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.018 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:46.018 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:20:46.019 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:46.019 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:20:46.019 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:46.020 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:20:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:20:46.020 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42465sec preferred_lft 42465sec 2023-10-10T20:20:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:20:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.021 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:20:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:46.021 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:20:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:20:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.022 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:46.022 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:20:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:20:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.023 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:20:46.023 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:20:46.024 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:46.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.024 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:20:46.081 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:20:46.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.083 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:20:46.139 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:20:46.139 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:20:46.139 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:20:46.139 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:20:46.140 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:20:46.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.140 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:20:46.141 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:20:46.141 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:20:46.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:20:46.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:20:46.143 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:20:46.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.239 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:20:46.254 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:20:46.255 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:20:46.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:46.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:20:46.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:46.319 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:46.373 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:20:46.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:46.374 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:46.428 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:20:46.429 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:20:46.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:46.508 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:46.508 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:46.527 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:46.527 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:46.527 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:46.528 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:46.528 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:46.528 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:46.528 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:46.529 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:46.549 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:46.550 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:46.550 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:46.550 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:46.553 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:46.553 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:46.553 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:46.554 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:46.554 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:20:46.554 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:20:46.554 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:20:46.555 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:20:46.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:46.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:20:46.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:20:46.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.775+0000 7fd979975700 1 Processor -- start 2023-10-10T20:20:46.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.775+0000 7fd979975700 1 -- start start 2023-10-10T20:20:46.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979975700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd97415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:46.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979975700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9740cb7c0 con 0x7fd9740cdab0 2023-10-10T20:20:46.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979975700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97415e290 con 0x7fd9740cd6e0 2023-10-10T20:20:46.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979975700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97415e850 con 0x7fd97415cbd0 2023-10-10T20:20:46.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd973fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9740cdab0 0x7fd97415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60824/0 (socket says 172.21.15.148:60824) 2023-10-10T20:20:46.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd973fff700 1 -- 172.21.15.148:0/1813016661 learned_addr learned my addr 172.21.15.148:0/1813016661 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:46.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979174700 1 --2- 172.21.15.148:0/1813016661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd97415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:46.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333130816 0 0) 0x7fd9740cb7c0 con 0x7fd9740cdab0 2023-10-10T20:20:46.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd960003570 con 0x7fd9740cdab0 2023-10-10T20:20:46.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111356698 0 0) 0x7fd97415e290 con 0x7fd9740cd6e0 2023-10-10T20:20:46.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9740cb7c0 con 0x7fd9740cd6e0 2023-10-10T20:20:46.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1573041265 0 0) 0x7fd960003570 con 0x7fd9740cdab0 2023-10-10T20:20:46.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979174700 1 -- 172.21.15.148:0/1813016661 >> v1:172.21.15.161:6789/0 conn(0x7fd9740cd6e0 legacy=0x7fd9740cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:46.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979174700 1 -- 172.21.15.148:0/1813016661 >> v1:172.21.15.148:6789/0 conn(0x7fd9740cdab0 legacy=0x7fd97415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:46.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.776+0000 7fd979174700 1 -- 172.21.15.148:0/1813016661 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97415e570 con 0x7fd97415cbd0 2023-10-10T20:20:46.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979174700 1 --2- 172.21.15.148:0/1813016661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd97415d020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009910 tx=0x7fd95c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c7184758fa5394 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:46.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640017f0 con 0x7fd97415cbd0 2023-10-10T20:20:46.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c011420 con 0x7fd97415cbd0 2023-10-10T20:20:46.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd9737fe700 1 -- 172.21.15.148:0/1813016661 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0117c0 con 0x7fd97415cbd0 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 -- 172.21.15.148:0/1813016661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 msgr2=0x7fd97415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 --2- 172.21.15.148:0/1813016661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd97415d020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009910 tx=0x7fd95c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 -- 172.21.15.148:0/1813016661 shutdown_connections 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 --2- 172.21.15.148:0/1813016661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd97415d020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 -- 172.21.15.148:0/1813016661 >> 172.21.15.148:0/1813016661 conn(0x7fd974067660 msgr2=0x7fd974067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:46.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 -- 172.21.15.148:0/1813016661 shutdown_connections 2023-10-10T20:20:46.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.777+0000 7fd979975700 1 -- 172.21.15.148:0/1813016661 wait complete. 2023-10-10T20:20:46.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.778+0000 7fd979975700 1 Processor -- start 2023-10-10T20:20:46.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 -- start start 2023-10-10T20:20:46.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9740cd6e0 0x7fd97414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:46.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:46.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd9740b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:46.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740cc360 con 0x7fd9740cdab0 2023-10-10T20:20:46.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b44c0 con 0x7fd9740cd6e0 2023-10-10T20:20:46.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd979975700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b5b30 con 0x7fd97415cbd0 2023-10-10T20:20:46.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:46.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53542/0 (socket says 172.21.15.148:53542) 2023-10-10T20:20:46.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.779+0000 7fd973fff700 1 -- 172.21.15.148:0/868579825 learned_addr learned my addr 172.21.15.148:0/868579825 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:46.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd979174700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd9740b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:46.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd978973700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9740cd6e0 0x7fd97414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:46.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 -- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 msgr2=0x7fd9740b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:46.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd9740b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:46.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 -- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9740cd6e0 msgr2=0x7fd97414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:46.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9740cd6e0 0x7fd97414caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 -- 172.21.15.148:0/868579825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c009420 con 0x7fd9740cdab0 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd973fff700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd96400ccf0 tx=0x7fd964007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964004dc0 con 0x7fd9740cdab0 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96400a8e0 con 0x7fd9740cdab0 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9740b3b90 con 0x7fd9740cdab0 2023-10-10T20:20:46.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400aca0 con 0x7fd9740cdab0 2023-10-10T20:20:46.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.780+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9740b4150 con 0x7fd9740cdab0 2023-10-10T20:20:46.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.781+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974089280 con 0x7fd9740cdab0 2023-10-10T20:20:46.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.781+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd964007360 con 0x7fd9740cdab0 2023-10-10T20:20:46.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.782+0000 7fd970ff9700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 0x7fd9540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:46.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.782+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9640840b0 con 0x7fd9740cdab0 2023-10-10T20:20:46.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.782+0000 7fd978973700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 0x7fd9540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:46.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.782+0000 7fd978973700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 0x7fd9540693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0027b0 tx=0x7fd96c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:46.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:46.786+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9640513a0 con 0x7fd9740cdab0 2023-10-10T20:20:47.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.012+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9740b4850 con 0x7fd9740cdab0 2023-10-10T20:20:47.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.012+0000 7fd970ff9700 1 -- 172.21.15.148:0/868579825 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd9740b4850 con 0x7fd9740cdab0 2023-10-10T20:20:47.013 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:20:47.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 msgr2=0x7fd9540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 0x7fd9540693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0027b0 tx=0x7fd96c009040 comp rx=0 tx=0).stop 2023-10-10T20:20:47.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 msgr2=0x7fd9740b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd96400ccf0 tx=0x7fd964007da0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 shutdown_connections 2023-10-10T20:20:47.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd954066f30 0x7fd9540693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd97415cbd0 0x7fd9740b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9740cd6e0 0x7fd97414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 --2- 172.21.15.148:0/868579825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9740cdab0 0x7fd9740b6220 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 >> 172.21.15.148:0/868579825 conn(0x7fd974067660 msgr2=0x7fd9740c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:47.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 shutdown_connections 2023-10-10T20:20:47.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.014+0000 7fd979975700 1 -- 172.21.15.148:0/868579825 wait complete. 2023-10-10T20:20:47.027 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:20:47.028 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:20:47.028 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:20:47.028 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:20:47.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:47.029 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:20:47.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:20:47.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:20:47.030 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:47.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.032 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:20:47.051 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:20:47.051 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:20:47.052 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:20:47.052 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.052 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:20:47.053 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.053 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:47.053 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:47.053 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:20:47.054 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:47.054 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:47.054 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:20:47.055 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:20:47.055 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:47.055 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:20:47.056 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:47.056 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:20:47.056 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42460sec preferred_lft 42460sec 2023-10-10T20:20:47.056 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:20:47.056 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.057 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:20:47.057 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:20:47.057 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:20:47.057 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.057 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:20:47.058 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.058 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:47.058 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:20:47.058 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:20:47.058 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.059 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:20:47.059 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:20:47.059 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:20:47.059 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:20:47.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.060 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:20:47.119 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:20:47.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.121 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:20:47.178 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:20:47.179 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:20:47.179 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:20:47.179 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:20:47.180 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:20:47.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.180 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:20:47.181 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:20:47.181 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:20:47.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:20:47.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:20:47.182 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:20:47.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.284 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:20:47.299 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:20:47.299 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:20:47.299 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:47.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.300 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:20:47.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:47.365 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:47.419 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:20:47.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:20:47.420 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:47.474 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:20:47.475 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:20:47.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:47.554 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:47.555 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:47.573 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:47.574 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:47.574 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:47.574 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:47.574 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:47.575 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:47.575 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:47.575 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:47.595 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:20:47.595 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:20:47.595 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:20:47.595 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:20:47.596 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:20:47.596 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:20:47.596 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:20:47.596 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:20:47.597 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:20:47.597 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:20:47.597 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:20:47.598 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:20:47.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:20:47.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:20:47.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:47.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.780+0000 7fb63c0b2700 1 Processor -- start 2023-10-10T20:20:47.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 -- start start 2023-10-10T20:20:47.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6340cf1c0 0x7fb63415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb63415a0c0 0x7fb634068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340cca20 con 0x7fb634159cf0 2023-10-10T20:20:47.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63415ea40 con 0x7fb6340cf1c0 2023-10-10T20:20:47.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63c0b2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb634067660 con 0x7fb63415a0c0 2023-10-10T20:20:47.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb63964d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6340cf1c0 0x7fb63415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.781+0000 7fb639e4e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb634159cf0 0x7fb6340ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60832/0 (socket says 172.21.15.148:60832) 2023-10-10T20:20:47.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb639e4e700 1 -- 172.21.15.148:0/774374685 learned_addr learned my addr 172.21.15.148:0/774374685 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63a64f700 1 --2- 172.21.15.148:0/774374685 >> v2:172.21.15.161:3300/0 conn(0x7fb63415a0c0 0x7fb634068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63964d700 1 -- 172.21.15.148:0/774374685 >> v2:172.21.15.161:3300/0 conn(0x7fb63415a0c0 msgr2=0x7fb634068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63964d700 1 --2- 172.21.15.148:0/774374685 >> v2:172.21.15.161:3300/0 conn(0x7fb63415a0c0 0x7fb634068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63964d700 1 -- 172.21.15.148:0/774374685 >> v1:172.21.15.148:6789/0 conn(0x7fb634159cf0 legacy=0x7fb6340ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb638e4c700 1 -- 172.21.15.148:0/774374685 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101928646 0 0) 0x7fb6340cca20 con 0x7fb634159cf0 2023-10-10T20:20:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63964d700 1 -- 172.21.15.148:0/774374685 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634067f00 con 0x7fb6340cf1c0 2023-10-10T20:20:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb63964d700 1 --2- 172.21.15.148:0/774374685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6340cf1c0 0x7fb63415e500 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb63406f260 tx=0x7fb6240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363446bd6b50262d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb638e4c700 1 -- 172.21.15.148:0/774374685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011630 con 0x7fb6340cf1c0 2023-10-10T20:20:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb638e4c700 1 -- 172.21.15.148:0/774374685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624011790 con 0x7fb6340cf1c0 2023-10-10T20:20:47.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.782+0000 7fb638e4c700 1 -- 172.21.15.148:0/774374685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011b30 con 0x7fb6340cf1c0 2023-10-10T20:20:47.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 -- 172.21.15.148:0/774374685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6340cf1c0 msgr2=0x7fb63415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/774374685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6340cf1c0 0x7fb63415e500 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb63406f260 tx=0x7fb6240091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 -- 172.21.15.148:0/774374685 shutdown_connections 2023-10-10T20:20:47.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/774374685 >> v2:172.21.15.161:3300/0 conn(0x7fb63415a0c0 0x7fb634068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/774374685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6340cf1c0 0x7fb63415e500 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 -- 172.21.15.148:0/774374685 >> 172.21.15.148:0/774374685 conn(0x7fb63406c6e0 msgr2=0x7fb634069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:47.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 -- 172.21.15.148:0/774374685 shutdown_connections 2023-10-10T20:20:47.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.783+0000 7fb63c0b2700 1 -- 172.21.15.148:0/774374685 wait complete. 2023-10-10T20:20:47.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.784+0000 7fb63c0b2700 1 Processor -- start 2023-10-10T20:20:47.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.784+0000 7fb63c0b2700 1 -- start start 2023-10-10T20:20:47.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6340cf1c0 0x7fb6341555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb634159cf0 0x7fb6340acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340adb90 con 0x7fb63415a0c0 2023-10-10T20:20:47.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340c8c30 con 0x7fb6340cf1c0 2023-10-10T20:20:47.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340c8020 con 0x7fb634159cf0 2023-10-10T20:20:47.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53556/0 (socket says 172.21.15.148:53556) 2023-10-10T20:20:47.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 -- 172.21.15.148:0/1820833608 learned_addr learned my addr 172.21.15.148:0/1820833608 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:47.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb639e4e700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6340cf1c0 0x7fb6341555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63964d700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb634159cf0 0x7fb6340acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 -- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb634159cf0 msgr2=0x7fb6340acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb634159cf0 0x7fb6340acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 -- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6340cf1c0 msgr2=0x7fb6341555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6340cf1c0 0x7fb6341555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 -- 172.21.15.148:0/1820833608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624009420 con 0x7fb63415a0c0 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63a64f700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e900 tx=0x7fb62c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c013680 con 0x7fb63415a0c0 2023-10-10T20:20:47.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0137e0 con 0x7fb63415a0c0 2023-10-10T20:20:47.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6340c8300 con 0x7fb63415a0c0 2023-10-10T20:20:47.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.785+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c013ba0 con 0x7fb63415a0c0 2023-10-10T20:20:47.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.786+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6340c88c0 con 0x7fb63415a0c0 2023-10-10T20:20:47.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.786+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6340692e0 con 0x7fb63415a0c0 2023-10-10T20:20:47.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.787+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb62c0123d0 con 0x7fb63415a0c0 2023-10-10T20:20:47.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.787+0000 7fb629ffb700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 0x7fb618069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:47.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.787+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb62c084650 con 0x7fb63415a0c0 2023-10-10T20:20:47.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.787+0000 7fb639e4e700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 0x7fb618069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:47.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.788+0000 7fb639e4e700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 0x7fb618069460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb630002250 tx=0x7fb630001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:47.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:47.792+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb62c051820 con 0x7fb63415a0c0 2023-10-10T20:20:48.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.021+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb63404c3c0 con 0x7fb63415a0c0 2023-10-10T20:20:48.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.022+0000 7fb629ffb700 1 -- 172.21.15.148:0/1820833608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fb62c054e40 con 0x7fb63415a0c0 2023-10-10T20:20:48.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:48.024 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:20:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 msgr2=0x7fb618069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 0x7fb618069460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb630002250 tx=0x7fb630001a20 comp rx=0 tx=0).stop 2023-10-10T20:20:48.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 msgr2=0x7fb6340c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:48.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e900 tx=0x7fb62c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:48.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 shutdown_connections 2023-10-10T20:20:48.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb618066fb0 0x7fb618069460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:48.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb63415a0c0 0x7fb6340c4710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:48.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6340cf1c0 0x7fb6341555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:48.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 --2- 172.21.15.148:0/1820833608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb634159cf0 0x7fb6340acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:48.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 >> 172.21.15.148:0/1820833608 conn(0x7fb63406c6e0 msgr2=0x7fb634068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:48.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 shutdown_connections 2023-10-10T20:20:48.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:48.024+0000 7fb63c0b2700 1 -- 172.21.15.148:0/1820833608 wait complete. 2023-10-10T20:20:48.028 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:20:48.037 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:20:48.171 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":5367,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3406483070000004,"mdsmap_epoch":24,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":14.307892363000001} 2023-10-10T20:20:48.171 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 5367, "mdsmap_epoch": 24, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 8.340648307, "state": "up:active", "uptime": 14.307892363, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:20:48.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:20:48.301 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:20:48.302 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:20:48.302 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-10T20:20:48.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:48.303 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-10T20:20:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:265+0 records in 2023-10-10T20:20:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:265+0 records out 2023-10-10T20:20:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.31284 s, 212 MB/s 2023-10-10T20:20:49.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:49.637 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-10T20:20:51.118 INFO:teuthology.orchestra.run.smithi148.stderr:292+0 records in 2023-10-10T20:20:51.119 INFO:teuthology.orchestra.run.smithi148.stderr:292+0 records out 2023-10-10T20:20:51.119 INFO:teuthology.orchestra.run.smithi148.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.46539 s, 209 MB/s 2023-10-10T20:20:51.120 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-10T20:20:51.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:20:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.234+0000 7f1dcd140700 1 Processor -- start 2023-10-10T20:20:51.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.234+0000 7f1dcd140700 1 -- start start 2023-10-10T20:20:51.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.234+0000 7f1dcd140700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dc8157870 0x7f1dc815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dcd140700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 0x7f1dc8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dcd140700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc815cbb0 con 0x7f1dc815a620 2023-10-10T20:20:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dcd140700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc815bfa0 con 0x7f1dc8157870 2023-10-10T20:20:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dcd140700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc815c280 con 0x7f1dc8157c40 2023-10-10T20:20:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dc8157870 0x7f1dc815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1dc815a620 0x7f1dc815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60846/0 (socket says 172.21.15.148:60846) 2023-10-10T20:20:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc659c700 1 -- 172.21.15.148:0/3543667207 learned_addr learned my addr 172.21.15.148:0/3543667207 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc759e700 1 --2- 172.21.15.148:0/3543667207 >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 0x7f1dc8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 -- 172.21.15.148:0/3543667207 >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 msgr2=0x7f1dc8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/3543667207 >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 0x7f1dc8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 -- 172.21.15.148:0/3543667207 >> v1:172.21.15.148:6789/0 conn(0x7f1dc815a620 legacy=0x7f1dc815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 -- 172.21.15.148:0/3543667207 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc815c560 con 0x7f1dc8157870 2023-10-10T20:20:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc5d9b700 1 -- 172.21.15.148:0/3543667207 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585355663 0 0) 0x7f1dc815cbb0 con 0x7f1dc815a620 2023-10-10T20:20:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc759e700 1 --2- 172.21.15.148:0/3543667207 >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 0x7f1dc8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/3543667207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc8157870 0x7f1dc815b6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1dc806b4e0 tx=0x7f1db80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45972d907a3979f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:51.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc5d9b700 1 -- 172.21.15.148:0/3543667207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db800b6b0 con 0x7f1dc8157870 2023-10-10T20:20:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc5d9b700 1 -- 172.21.15.148:0/3543667207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db800b810 con 0x7f1dc8157870 2023-10-10T20:20:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.235+0000 7f1dc5d9b700 1 -- 172.21.15.148:0/3543667207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db800bbb0 con 0x7f1dc8157870 2023-10-10T20:20:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 -- 172.21.15.148:0/3543667207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc8157870 msgr2=0x7f1dc815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 --2- 172.21.15.148:0/3543667207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc8157870 0x7f1dc815b6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1dc806b4e0 tx=0x7f1db80091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 -- 172.21.15.148:0/3543667207 shutdown_connections 2023-10-10T20:20:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 --2- 172.21.15.148:0/3543667207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc8157870 0x7f1dc815b6a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 --2- 172.21.15.148:0/3543667207 >> v2:172.21.15.161:3300/0 conn(0x7f1dc8157c40 0x7f1dc8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 -- 172.21.15.148:0/3543667207 >> 172.21.15.148:0/3543667207 conn(0x7f1dc8068ce0 msgr2=0x7f1dc80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 -- 172.21.15.148:0/3543667207 shutdown_connections 2023-10-10T20:20:51.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.236+0000 7f1dcd140700 1 -- 172.21.15.148:0/3543667207 wait complete. 2023-10-10T20:20:51.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.237+0000 7f1dcd140700 1 Processor -- start 2023-10-10T20:20:51.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.237+0000 7f1dcd140700 1 -- start start 2023-10-10T20:20:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 0x7f1dc8156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dc8157c40 0x7f1dc80c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c6050 con 0x7f1dc815a620 2023-10-10T20:20:51.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c62f0 con 0x7f1dc8157870 2023-10-10T20:20:51.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80c65d0 con 0x7f1dc8157c40 2023-10-10T20:20:51.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53582/0 (socket says 172.21.15.148:53582) 2023-10-10T20:20:51.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 -- 172.21.15.148:0/924260941 learned_addr learned my addr 172.21.15.148:0/924260941 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:51.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 0x7f1dc8156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc659c700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dc8157c40 0x7f1dc80c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 -- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dc8157c40 msgr2=0x7f1dc80c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dc8157c40 0x7f1dc80c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 -- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 msgr2=0x7f1dc8156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 0x7f1dc8156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 -- 172.21.15.148:0/924260941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db8009420 con 0x7f1dc815a620 2023-10-10T20:20:51.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 0x7f1dc8156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:51.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dc759e700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000d180 tx=0x7f1dc0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:51.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000ddc0 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc000f040 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc80c68b0 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.238+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0015650 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.239+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81604e0 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.239+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc815d680 con 0x7f1dc815a620 2023-10-10T20:20:51.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.240+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dc0015a40 con 0x7f1dc815a620 2023-10-10T20:20:51.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.240+0000 7f1daeffd700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:51.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.240+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1dc0084e40 con 0x7f1dc815a620 2023-10-10T20:20:51.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.240+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:51.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.241+0000 7f1dc6d9d700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1db8003080 tx=0x7f1db8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:51.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.245+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dc815d680 con 0x7f1dc815a620 2023-10-10T20:20:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.430+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1dc815d680 con 0x7f1dc815a620 2023-10-10T20:20:51.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.431+0000 7f1daeffd700 1 -- 172.21.15.148:0/924260941 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+32308 (secure 0 0 0) 0x7f1dc815d680 con 0x7f1dc815a620 2023-10-10T20:20:51.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:51.434 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:51.434 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 45, 2023-10-10T20:20:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:20:51.435 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:20:40.806624+0000", 2023-10-10T20:20:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:20:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:20:51.436 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:20:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:20:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:20:51.437 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:20:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:20:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:20:51.438 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:20:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:20:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:20:51.439 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:20:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:20:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:20:51.440 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:20:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:20:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:20:51.441 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:20:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:20:51.442 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:20:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:20:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:20:51.443 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:20:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:20:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:20:51.444 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:51.445 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:51.446 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:20:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:20:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:20:51.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:20:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:20:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:51.448 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.449 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "16", 2023-10-10T20:20:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:51.450 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:51.451 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:51.452 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:51.453 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:51.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:51.455 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:51.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:51.457 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:51.458 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:51.459 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:20:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:20:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.461 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:20:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:20:51.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:20:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:20:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:20:51.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:20:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:51.465 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:51.466 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:51.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:20:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:20:51.468 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:20:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:20:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:20:51.469 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:51.470 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.471 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "44", 2023-10-10T20:20:51.472 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:51.474 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:51.475 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:51.476 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:51.477 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:51.478 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:51.479 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:51.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:51.481 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:51.482 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:51.483 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:51.484 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:20:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:20:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:20:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.485 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:20:51.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:20:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:20:51.488 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:20:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:20:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:20:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:51.489 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:51.490 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:20:51.491 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:20:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:20:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:20:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:20:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:51.492 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:51.493 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "45", 2023-10-10T20:20:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:51.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:51.495 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:51.496 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:51.497 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:51.498 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:51.500 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:51.501 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:51.502 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:20:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:20:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:51.503 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:20:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.504 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:20:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.505 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.506 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.507 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.508 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:20:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.509 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:20:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.510 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:20:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.512 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:20:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:20:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.515 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:20:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:20:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.518 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:20:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:20:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:20:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:20:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.519 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.520 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.521 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.522 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:20:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.524 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.525 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:20:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:20:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.527 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:20:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:20:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:20:51.533 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:20:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:20:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:20:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.534 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:51.535 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.536 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.537 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:20:51.538 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:20:51.539 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:20:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:20:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.542 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.543 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:20:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.545 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:20:51.546 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:20:51.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:20:51.548 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:20:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:20:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.549 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:51.550 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.551 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.552 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:20:51.553 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:20:51.554 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:20:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:20:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.557 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:20:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.560 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:20:51.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:20:51.562 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:20:51.563 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:20:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:20:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.565 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.566 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.567 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:20:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:20:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.569 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.571 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:20:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:20:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:20:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:20:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:51.577 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:20:51.578 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:20:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.580 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.581 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:20:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:20:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.584 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.585 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:20:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:20:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.587 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:20:51.588 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:20:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:20:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.592 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:20:51.593 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.595 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.596 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.597 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:20:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.598 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:20:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:20:51.601 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:20:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:20:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.604 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:20:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:20:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.608 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:51.610 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:51.611 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:20:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:20:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.614 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:20:51.615 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.616 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:20:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:51.620 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:20:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:51.621 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:20:51.622 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:20:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:20:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:20:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:51.623 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:20:51.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.625 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:20:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.627 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:20:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.628 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.629 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:51.630 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.631 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:20:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:51.632 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:20:51.633 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:20:51.635 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.636 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:20:51.637 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:51.638 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:51.639 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:20:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:20:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:20:51.640 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:20:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:20:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:20:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:20:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:20:51.641 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:20:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:20:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:20:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:20:51.643 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:20:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:20:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:20:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:20:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:20:51.644 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:20:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:20:51.645 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:51.645 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 msgr2=0x7f1db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1db8003080 tx=0x7f1db8017000 comp rx=0 tx=0).stop 2023-10-10T20:20:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 msgr2=0x7f1dc80cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:51.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000d180 tx=0x7f1dc0009a50 comp rx=0 tx=0).stop 2023-10-10T20:20:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 shutdown_connections 2023-10-10T20:20:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dc815a620 0x7f1dc80cd050 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dc8157870 0x7f1dc8156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 --2- 172.21.15.148:0/924260941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1dc8157c40 0x7f1dc80c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 >> 172.21.15.148:0/924260941 conn(0x7f1dc8068ce0 msgr2=0x7f1dc80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 shutdown_connections 2023-10-10T20:20:51.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:51.433+0000 7f1dcd140700 1 -- 172.21.15.148:0/924260941 wait complete. 2023-10-10T20:20:51.650 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:20:52.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:20:52.480 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:20:53.491 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:20:53.489+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-10-10T20:20:56.652 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:20:56.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.778+0000 7fb300fdf700 1 Processor -- start 2023-10-10T20:20:56.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.778+0000 7fb300fdf700 1 -- start start 2023-10-10T20:20:56.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb300fdf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 0x7fb2fc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb300fdf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb300fdf700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0cb7c0 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb300fdf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc15e850 con 0x7fb2fc0cdab0 2023-10-10T20:20:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb300fdf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cc360 con 0x7fb2fc15dda0 2023-10-10T20:20:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb2faffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 0x7fb2fc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb2fbfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb2fb7fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb2fc0cd6e0 0x7fb2fc0cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:50932/0 (socket says 172.21.15.148:50932) 2023-10-10T20:20:56.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.779+0000 7fb2fb7fe700 1 -- 172.21.15.148:0/3705995005 learned_addr learned my addr 172.21.15.148:0/3705995005 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:56.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fbfff700 1 -- 172.21.15.148:0/3705995005 >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 msgr2=0x7fb2fc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:56.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fbfff700 1 --2- 172.21.15.148:0/3705995005 >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 0x7fb2fc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fbfff700 1 -- 172.21.15.148:0/3705995005 >> v1:172.21.15.161:6789/0 conn(0x7fb2fc0cd6e0 legacy=0x7fb2fc0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:56.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fbfff700 1 -- 172.21.15.148:0/3705995005 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc15e290 con 0x7fb2fc15dda0 2023-10-10T20:20:56.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2faffd700 1 --2- 172.21.15.148:0/3705995005 >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 0x7fb2fc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:20:56.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fbfff700 1 --2- 172.21.15.148:0/3705995005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc15cbd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40098e0 tx=0x7fb2f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4766f3f25298a9ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:56.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fa7fc700 1 -- 172.21.15.148:0/3705995005 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400b640 con 0x7fb2fc15dda0 2023-10-10T20:20:56.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fa7fc700 1 -- 172.21.15.148:0/3705995005 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f400b7a0 con 0x7fb2fc15dda0 2023-10-10T20:20:56.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.780+0000 7fb2fa7fc700 1 -- 172.21.15.148:0/3705995005 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400bb40 con 0x7fb2fc15dda0 2023-10-10T20:20:56.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 -- 172.21.15.148:0/3705995005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 msgr2=0x7fb2fc15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:56.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 --2- 172.21.15.148:0/3705995005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc15cbd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40098e0 tx=0x7fb2f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 -- 172.21.15.148:0/3705995005 shutdown_connections 2023-10-10T20:20:56.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 --2- 172.21.15.148:0/3705995005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc15cbd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 --2- 172.21.15.148:0/3705995005 >> v2:172.21.15.161:3300/0 conn(0x7fb2fc0cdab0 0x7fb2fc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 -- 172.21.15.148:0/3705995005 >> 172.21.15.148:0/3705995005 conn(0x7fb2fc067660 msgr2=0x7fb2fc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:56.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 -- 172.21.15.148:0/3705995005 shutdown_connections 2023-10-10T20:20:56.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.781+0000 7fb300fdf700 1 -- 172.21.15.148:0/3705995005 wait complete. 2023-10-10T20:20:56.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.782+0000 7fb300fdf700 1 Processor -- start 2023-10-10T20:20:56.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.782+0000 7fb300fdf700 1 -- start start 2023-10-10T20:20:56.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0cdab0 0x7fb2fc0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b5360 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b6180 con 0x7fb2fc0cdab0 2023-10-10T20:20:56.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb300fdf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b5570 con 0x7fb2fc15dda0 2023-10-10T20:20:56.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44112/0 (socket says 172.21.15.148:44112) 2023-10-10T20:20:56.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 -- 172.21.15.148:0/1013889193 learned_addr learned my addr 172.21.15.148:0/1013889193 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:56.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2faffd700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0cdab0 0x7fb2fc0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fbfff700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 -- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 msgr2=0x7fb2fc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:56.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 -- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0cdab0 msgr2=0x7fb2fc0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:56.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0cdab0 0x7fb2fc0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:56.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009420 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.783+0000 7fb2fb7fe700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00cc70 tx=0x7fb2ec001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:56.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.784+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00a890 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.784+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec00aa10 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.784+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f4009420 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.784+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc0b5b30 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.784+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00add0 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.785+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc0052b0 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.785+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2ec007300 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.786+0000 7fb2e37fe700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:56.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.786+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2ec084840 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.786+0000 7fb2faffd700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:56.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.786+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7fb2e806ea50 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.786+0000 7fb2faffd700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 0x7fb2e80693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0d0b10 tx=0x7fb2f00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:56.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.790+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2ec051b30 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.973+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb2dc005830 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.973+0000 7fb2e37fe700 1 -- 172.21.15.148:0/1013889193 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32451 (secure 0 0 0) 0x7fb2ec055150 con 0x7fb2fc0cd6e0 2023-10-10T20:20:56.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:56.976 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:56.976 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 49, 2023-10-10T20:20:56.977 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:56.977 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:20:56.977 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:20:55.503638+0000", 2023-10-10T20:20:56.977 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:20:56.977 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:20:56.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:20:56.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:20:56.978 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:20:56.978 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:20:56.978 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:20:56.979 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:20:56.979 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:20:56.979 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:56.979 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:20:56.979 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:20:56.980 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:20:56.980 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:20:56.980 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:20:56.980 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:20:56.980 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:20:56.981 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:20:56.981 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:20:56.981 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:20:56.981 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:20:56.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:56.982 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:20:56.982 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:20:56.982 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:20:56.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:56.982 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:56.983 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:56.984 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:56.984 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:56.984 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:56.984 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:56.984 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:20:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:20:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:20:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:20:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:20:56.985 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:56.986 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:56.986 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:56.986 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:56.986 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:56.986 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:56.987 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:56.988 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:56.988 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:56.988 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:56.988 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:56.988 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:56.989 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:56.990 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:56.990 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:56.990 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:56.990 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:56.990 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:56.992 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:56.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:56.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:56.992 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:56.992 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:56.993 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:56.993 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:56.993 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:56.993 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:56.993 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:56.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:20:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:20:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:20:56.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:20:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:20:56.996 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:20:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:56.997 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:56.998 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:56.998 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:56.998 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:56.998 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:57.001 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:57.001 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:57.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:57.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:20:57.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:20:57.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:20:57.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:20:57.003 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:20:57.003 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:57.003 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:57.003 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:57.003 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:57.004 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:57.004 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:57.004 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:57.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.004 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:57.005 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:57.006 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:57.006 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:57.006 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:57.006 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:57.006 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:57.007 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:57.007 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:57.007 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:57.007 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:57.007 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:57.008 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:57.008 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:57.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:57.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:57.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:57.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.010 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:57.010 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:57.010 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:57.010 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:57.010 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:57.011 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:57.011 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:57.011 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:57.011 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:57.011 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:57.012 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:57.012 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:57.012 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:20:57.012 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:20:57.012 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.013 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.014 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:20:57.014 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:20:57.014 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:20:57.015 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:57.015 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:57.015 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:57.015 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:57.015 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:57.016 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:20:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:20:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:20:57.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:20:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:20:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:57.019 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:57.019 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:57.019 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:57.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.019 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:57.020 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:57.021 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:57.021 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:57.021 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:57.021 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:57.021 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:57.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:57.022 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:57.022 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:57.022 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:57.022 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:57.023 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:57.024 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:57.024 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:57.024 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:57.024 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:57.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.025 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:57.025 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:57.025 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:57.025 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:57.025 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:57.026 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:57.026 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:57.026 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:57.026 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:57.026 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:57.027 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:57.027 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:57.027 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:20:57.027 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:20:57.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.029 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.029 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:20:57.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.029 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:57.029 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.030 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.030 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.030 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.030 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.031 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.032 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.032 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:20:57.032 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.033 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.033 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:20:57.033 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.035 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.035 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:20:57.035 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.036 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.037 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.037 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:20:57.038 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.039 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.040 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:20:57.040 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.041 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.041 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:20:57.041 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:20:57.042 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.043 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.046 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.046 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.046 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.046 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.046 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:20:57.047 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.048 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:20:57.048 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:20:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:20:57.051 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.051 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.052 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:20:57.054 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.054 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.055 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:20:57.055 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.056 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:20:57.056 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:20:57.057 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:20:57.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:20:57.058 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.058 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.058 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.059 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:57.059 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.059 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.060 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.061 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.061 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.061 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.061 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.062 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.062 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:20:57.062 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:20:57.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:20:57.065 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:20:57.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.066 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.066 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:20:57.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:20:57.068 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.069 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.069 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.070 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.070 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:20:57.070 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.071 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.071 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:20:57.071 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:20:57.072 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:20:57.073 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.073 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.073 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.073 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.073 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.074 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:57.074 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.074 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.075 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.075 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.075 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.075 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.075 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.076 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.076 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.076 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.076 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.076 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.077 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:20:57.077 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.078 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.079 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.079 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:20:57.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:20:57.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.082 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:20:57.083 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.083 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.084 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:20:57.085 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:20:57.086 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.087 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.087 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:20:57.087 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:20:57.088 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:20:57.088 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:20:57.088 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.088 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:20:57.088 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.089 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.089 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.089 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.090 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.091 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.091 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.091 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.091 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.091 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:20:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:20:57.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.096 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:20:57.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:20:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:20:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:20:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:20:57.102 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:20:57.103 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:20:57.103 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:20:57.103 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.103 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.103 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.104 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.105 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.106 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.106 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.106 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.106 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.107 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.107 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.107 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.107 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:20:57.108 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.108 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.110 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.110 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:20:57.110 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:20:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.113 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:20:57.113 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.113 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:20:57.114 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.114 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.114 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:20:57.115 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.116 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.116 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:20:57.116 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.117 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.117 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.117 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:20:57.117 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:20:57.118 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:20:57.118 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:20:57.118 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.118 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.118 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.119 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.119 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:57.119 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.122 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.122 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:20:57.122 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:20:57.123 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.125 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:20:57.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:20:57.126 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:20:57.128 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.128 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.128 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.129 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:20:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.131 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.131 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:20:57.131 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.131 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:20:57.132 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:20:57.133 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.133 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.133 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.133 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.134 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:57.134 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.134 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.135 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.135 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.135 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.135 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.135 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.136 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:20:57.137 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.138 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.139 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.139 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.140 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:20:57.140 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.141 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.142 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.142 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.142 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:20:57.142 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.144 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.144 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.144 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.145 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.145 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:20:57.145 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.147 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:20:57.147 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:20:57.147 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:20:57.147 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:20:57.147 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.148 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.149 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:20:57.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.149 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:57.149 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.149 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.150 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.150 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.150 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.150 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:20:57.150 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.151 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:57.151 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.151 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.153 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.154 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.154 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.154 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:20:57.154 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:57.155 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.155 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.155 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:57.157 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.157 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.157 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.157 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:20:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:57.159 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.159 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.159 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.159 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:57.161 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.161 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.161 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.161 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.161 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.162 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:20:57.162 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.162 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.163 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:20:57.164 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.164 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.164 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.164 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:20:57.164 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:20:57.165 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:20:57.165 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:20:57.165 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:20:57.165 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:20:57.165 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:20:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:20:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:20:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:20:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:20:57.166 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:20:57.167 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:20:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:20:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:20:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:20:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:20:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:20:57.169 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:20:57.169 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:20:57.169 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:20:57.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.169 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 msgr2=0x7fb2e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 0x7fb2e80693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0d0b10 tx=0x7fb2f00091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 msgr2=0x7fb2fc14cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00cc70 tx=0x7fb2ec001a30 comp rx=0 tx=0).stop 2023-10-10T20:20:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 shutdown_connections 2023-10-10T20:20:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb2e8066f30 0x7fb2e80693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2fc15dda0 0x7fb2fc0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0cd6e0 0x7fb2fc14cb00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 --2- 172.21.15.148:0/1013889193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2fc0cdab0 0x7fb2fc0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 >> 172.21.15.148:0/1013889193 conn(0x7fb2fc067660 msgr2=0x7fb2fc14d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:57.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.975+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 shutdown_connections 2023-10-10T20:20:57.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:56.976+0000 7fb300fdf700 1 -- 172.21.15.148:0/1013889193 wait complete. 2023-10-10T20:20:57.174 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-10T20:20:57.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:57.175 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-10T20:20:57.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:20:57.425 INFO:teuthology.orchestra.run.smithi148.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-10T20:20:57.425 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:20:57.425 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:20:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.2273 s, 231 MB/s 2023-10-10T20:20:57.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:20:57.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.540+0000 7f08879ce700 1 Processor -- start 2023-10-10T20:20:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.540+0000 7f08879ce700 1 -- start start 2023-10-10T20:20:57.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f08879ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f088015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f08879ce700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088015cbb0 con 0x7f088015d580 2023-10-10T20:20:57.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f08879ce700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088015bfa0 con 0x7f0880157870 2023-10-10T20:20:57.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f08879ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088015bdb0 con 0x7f088015a620 2023-10-10T20:20:57.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f0885f6b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f088015d580 0x7f0880158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39056/0 (socket says 172.21.15.148:39056) 2023-10-10T20:20:57.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.541+0000 7f0885f6b700 1 -- 172.21.15.148:0/1262638749 learned_addr learned my addr 172.21.15.148:0/1262638749 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:57.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875107845 0 0) 0x7f088015cbb0 con 0x7f088015d580 2023-10-10T20:20:57.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f088015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f086c003570 con 0x7f088015d580 2023-10-10T20:20:57.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3644466741 0 0) 0x7f086c003570 con 0x7f088015d580 2023-10-10T20:20:57.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 msgr2=0x7f088015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:57.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f088015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 >> v1:172.21.15.161:6789/0 conn(0x7f0880157870 legacy=0x7f088015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088015c280 con 0x7f088015d580 2023-10-10T20:20:57.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f0878001470 con 0x7f088015d580 2023-10-10T20:20:57.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 >> v1:172.21.15.148:6789/0 conn(0x7f088015d580 legacy=0x7f0880158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c003a90 0x7f086c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f086c004810 0x7f086c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f086c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00bb60 con 0x7f086c004810 2023-10-10T20:20:57.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00bcd0 con 0x7f086c003a90 2023-10-10T20:20:57.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00be90 con 0x7f088015a620 2023-10-10T20:20:57.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f086c004810 0x7f086c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f088576a700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c003a90 0x7f086c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0885f6b700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f086c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 -- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 msgr2=0x7f086c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f088015a620 0x7f086c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 -- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c003a90 msgr2=0x7f086c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c003a90 0x7f086c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 -- 172.21.15.148:0/1262638749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c003570 con 0x7f086c004810 2023-10-10T20:20:57.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f088576a700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c003a90 0x7f086c003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:20:57.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.542+0000 7f0884f69700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f086c004810 0x7f086c004c80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f088015aa20 tx=0x7f08700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e584fe6366bd5f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:57.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000e710 con 0x7f086c004810 2023-10-10T20:20:57.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000e870 con 0x7f086c004810 2023-10-10T20:20:57.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f0877fff700 1 -- 172.21.15.148:0/1262638749 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ec30 con 0x7f086c004810 2023-10-10T20:20:57.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 -- 172.21.15.148:0/1262638749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f086c004810 msgr2=0x7f086c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 --2- 172.21.15.148:0/1262638749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f086c004810 0x7f086c004c80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f088015aa20 tx=0x7f08700091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f0885f6b700 1 -- 172.21.15.148:0/1262638749 reap_dead start 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 -- 172.21.15.148:0/1262638749 shutdown_connections 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 -- 172.21.15.148:0/1262638749 >> 172.21.15.148:0/1262638749 conn(0x7f0880068ce0 msgr2=0x7f08800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 -- 172.21.15.148:0/1262638749 shutdown_connections 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.543+0000 7f08879ce700 1 -- 172.21.15.148:0/1262638749 wait complete. 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.544+0000 7f08879ce700 1 Processor -- start 2023-10-10T20:20:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.544+0000 7f08879ce700 1 -- start start 2023-10-10T20:20:57.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 0x7f0880159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0880158ce0 0x7f0880149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880158850 con 0x7f088014b6c0 2023-10-10T20:20:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088016b180 con 0x7f086c004810 2023-10-10T20:20:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f08879ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0880157f20 con 0x7f0880158ce0 2023-10-10T20:20:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44120/0 (socket says 172.21.15.148:44120) 2023-10-10T20:20:57.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 -- 172.21.15.148:0/3145723025 learned_addr learned my addr 172.21.15.148:0/3145723025 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:57.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0885f6b700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0880158ce0 0x7f0880149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f088576a700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 0x7f0880159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 -- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0880158ce0 msgr2=0x7f0880149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0880158ce0 0x7f0880149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 -- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 msgr2=0x7f0880159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 0x7f0880159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f0884f69700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880158200 con 0x7f088014b6c0 2023-10-10T20:20:57.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.545+0000 7f088576a700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 0x7f0880159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:57.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f0884f69700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f086c00bca0 tx=0x7f087001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:57.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870022070 con 0x7f088014b6c0 2023-10-10T20:20:57.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000e9e0 con 0x7f088014b6c0 2023-10-10T20:20:57.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087001c450 con 0x7f088014b6c0 2023-10-10T20:20:57.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08801584e0 con 0x7f088014b6c0 2023-10-10T20:20:57.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.546+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08801499a0 con 0x7f088014b6c0 2023-10-10T20:20:57.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.547+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08801589c0 con 0x7f088014b6c0 2023-10-10T20:20:57.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.547+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f087001c800 con 0x7f088014b6c0 2023-10-10T20:20:57.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.547+0000 7f08757fa700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:57.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.548+0000 7f088576a700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:57.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.548+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0870085da0 con 0x7f088014b6c0 2023-10-10T20:20:57.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.548+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f086406ea50 con 0x7f088014b6c0 2023-10-10T20:20:57.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.548+0000 7f088576a700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 0x7f08640693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f087c0012e0 tx=0x7f087c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:57.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.552+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0870017060 con 0x7f088014b6c0 2023-10-10T20:20:57.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.735+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f088004c3c0 con 0x7f088014b6c0 2023-10-10T20:20:57.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.736+0000 7f08757fa700 1 -- 172.21.15.148:0/3145723025 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32451 (secure 0 0 0) 0x7f087002b030 con 0x7f088014b6c0 2023-10-10T20:20:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:57.739 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 49, 2023-10-10T20:20:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:57.739 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:20:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:20:55.503638+0000", 2023-10-10T20:20:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:20:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:20:57.740 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:20:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:20:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:20:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:20:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:20:57.741 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:20:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:20:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:20:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:20:57.742 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:20:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:20:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:20:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:20:57.743 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:20:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:20:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:20:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:20:57.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:20:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:20:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:20:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:20:57.745 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:57.746 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:57.747 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:20:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:20:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:20:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:20:57.748 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:20:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:57.749 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:57.750 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:57.750 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:57.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.750 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:57.751 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:57.751 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:57.751 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:57.751 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:57.751 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:57.752 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:57.752 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:57.752 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:57.752 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:57.752 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:57.753 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:57.753 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:57.753 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:57.753 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:57.753 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:57.754 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:57.754 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:57.754 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:57.754 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:57.754 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:57.755 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:57.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:57.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.756 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:57.756 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:57.756 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:57.757 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:57.757 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:57.757 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:57.757 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:57.757 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:57.758 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:57.758 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:57.758 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:57.758 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:57.759 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:20:57.759 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:20:57.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.759 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:20:57.760 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:20:57.761 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:20:57.761 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:57.761 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:57.761 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:57.761 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:57.762 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:57.762 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:57.762 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:57.762 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:57.762 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:57.763 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:57.763 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:57.763 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:57.763 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:20:57.763 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:20:57.764 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:20:57.764 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:20:57.764 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:20:57.764 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:57.765 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:57.766 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:57.767 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:57.767 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:57.767 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:57.768 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:57.768 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:57.768 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:57.768 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:57.768 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:57.769 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:57.769 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:57.769 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:57.769 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:57.770 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:57.770 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:57.770 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:57.770 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:57.770 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:57.771 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:57.771 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:57.771 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:57.771 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:57.772 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:57.772 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:57.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.772 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:57.772 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:57.773 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:57.773 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:57.773 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:57.773 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:57.773 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:57.774 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:57.774 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:57.774 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:57.774 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:57.774 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:57.775 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:57.776 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:20:57.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.777 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:20:57.777 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:20:57.777 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:20:57.777 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:57.777 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:57.778 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:20:57.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:20:57.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:20:57.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:20:57.780 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:20:57.780 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:57.783 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:57.784 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:57.785 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:57.785 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:57.785 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:57.785 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:57.785 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:57.786 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:57.786 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:57.786 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:57.786 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:57.786 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:57.788 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:57.788 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:57.788 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:57.788 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:57.788 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:57.789 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:57.789 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:57.789 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:57.789 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:57.789 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:20:57.790 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:20:57.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.790 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:57.790 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:57.790 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:20:57.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.791 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.792 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:20:57.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.792 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:57.792 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.792 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.793 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.793 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.793 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.793 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.793 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.794 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.794 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.794 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.794 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.794 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:20:57.795 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.797 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.797 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.797 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.798 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.798 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:20:57.798 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.800 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:20:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:20:57.802 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.802 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.802 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.802 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.803 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.803 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.803 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:20:57.804 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.804 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.804 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:20:57.805 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:57.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.805 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.805 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:20:57.806 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:20:57.806 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:20:57.806 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:20:57.806 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.806 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:57.808 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.808 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.808 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.808 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.809 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.810 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.810 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.810 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.812 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:20:57.812 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.812 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:20:57.813 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.814 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.814 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:20:57.814 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.815 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.815 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.816 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.816 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:20:57.816 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:20:57.817 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.818 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.818 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.818 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.819 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.819 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.819 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:20:57.819 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.820 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.820 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:20:57.820 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:57.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.821 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.821 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:20:57.822 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:20:57.822 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:20:57.822 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:20:57.822 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.823 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.823 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.823 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.824 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.825 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.825 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.825 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.825 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.825 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.827 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:20:57.827 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.828 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.828 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:20:57.828 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.828 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.828 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.829 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.829 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.830 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.830 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:20:57.830 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.831 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:20:57.831 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.832 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.832 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.832 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.833 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.833 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:20:57.833 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.834 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:20:57.834 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.835 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.835 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.836 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.836 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.837 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.837 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:20:57.837 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.838 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.838 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:20:57.838 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:57.839 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.839 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:20:57.839 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:20:57.840 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:20:57.840 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:20:57.840 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.840 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.841 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.841 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.842 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:57.842 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.842 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.843 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.843 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.843 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.843 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.843 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.844 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.845 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.845 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.845 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:20:57.846 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.846 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.846 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:20:57.847 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.847 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.848 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.848 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.848 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.848 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:20:57.849 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.850 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:20:57.850 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.850 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.851 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.851 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.852 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:20:57.852 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.853 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.853 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:20:57.853 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.854 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.854 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.854 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.855 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:20:57.855 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.856 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.856 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:20:57.856 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:57.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.857 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:20:57.858 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:20:57.858 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:20:57.858 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:20:57.858 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.859 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:20:57.859 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.859 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.859 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.859 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.860 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:57.860 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.860 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.860 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.861 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.861 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.861 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.861 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.862 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.863 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.863 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:20:57.864 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.864 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.865 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:20:57.865 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.865 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.866 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.866 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.867 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:20:57.867 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.868 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.868 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:20:57.868 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.869 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.869 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.869 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.869 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.870 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.870 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:20:57.870 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.871 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.871 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:20:57.872 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.872 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.872 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.872 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.873 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.873 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.873 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:20:57.874 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.874 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.874 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:20:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.875 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.876 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:20:57.876 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:20:57.876 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:20:57.876 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:20:57.877 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.877 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.877 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.877 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.878 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:57.878 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.879 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.879 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.879 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.879 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.879 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.880 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.880 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.880 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.880 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.880 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.881 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.881 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.881 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.881 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.882 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:20:57.882 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.883 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:20:57.883 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.884 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.884 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.885 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.885 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.885 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.885 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:20:57.885 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.886 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.887 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:20:57.887 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.887 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.888 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.888 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.888 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.889 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:20:57.890 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.890 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.890 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.891 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.891 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.891 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.891 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:20:57.891 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.892 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.892 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:20:57.892 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:20:57.893 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:20:57.894 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:20:57.894 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.894 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.894 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.894 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.895 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.895 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:57.895 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.895 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.896 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.897 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.897 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.897 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.897 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.897 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.898 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.898 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.898 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:20:57.898 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.899 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.899 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:20:57.899 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.899 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.900 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.901 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:20:57.901 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.901 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.901 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.902 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:20:57.902 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.902 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.902 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:20:57.903 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.904 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.904 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:20:57.904 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.905 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.906 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.906 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:20:57.906 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:20:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:20:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:20:57.909 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:20:57.909 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:20:57.909 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.909 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.909 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:57.911 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:57.911 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:57.911 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:57.911 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:57.912 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:57.912 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:57.912 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:57.912 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:57.912 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:57.913 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:57.913 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:57.913 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:57.913 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.914 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:20:57.914 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.915 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.915 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:20:57.916 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.916 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.916 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.916 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:57.917 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.917 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.917 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.917 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:20:57.917 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.918 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.918 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.918 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:20:57.919 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.919 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.920 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:57.920 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.920 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.921 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:20:57.921 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.921 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:20:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.923 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.923 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.923 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:57.923 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:57.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.924 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:57.924 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:20:57.924 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.925 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.925 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:57.925 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:20:57.926 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:57.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.926 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:20:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:20:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:20:57.927 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:20:57.928 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:57.928 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:57.928 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:57.929 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:57.929 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:57.929 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.929 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.929 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:20:57.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.930 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:57.930 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.930 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.930 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.931 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.931 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.931 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:20:57.931 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.932 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.932 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:57.932 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.932 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.932 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.933 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.933 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.933 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:20:57.933 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.933 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.934 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.934 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:57.934 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.934 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.934 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.936 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:57.936 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.936 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.936 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.936 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.937 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.937 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:20:57.937 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.938 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.938 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.938 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:57.938 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.938 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.939 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.939 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.939 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.939 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:20:57.939 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.940 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.940 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:57.940 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.940 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.942 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:57.942 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.942 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.942 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.943 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.943 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.943 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:20:57.943 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.944 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.944 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:57.944 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:57.944 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:57.944 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:57.945 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:57.945 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:57.945 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:57.945 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:20:57.946 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:57.946 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.946 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:57.946 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:20:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:20:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:20:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:20:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:20:57.947 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:20:57.948 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:20:57.949 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:20:57.949 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:57.949 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:20:57.949 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:20:57.950 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:20:57.950 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:20:57.950 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:20:57.951 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:20:57.951 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:20:57.951 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:20:57.951 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:20:57.952 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:20:57.952 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:20:57.952 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:57.952 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:57.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 msgr2=0x7f08640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 0x7f08640693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f087c0012e0 tx=0x7f087c009040 comp rx=0 tx=0).stop 2023-10-10T20:20:57.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 msgr2=0x7f08801599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:57.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f086c00bca0 tx=0x7f087001a000 comp rx=0 tx=0).stop 2023-10-10T20:20:57.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 shutdown_connections 2023-10-10T20:20:57.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0864066f30 0x7f08640693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0880158ce0 0x7f0880149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f086c004810 0x7f0880159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 --2- 172.21.15.148:0/3145723025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f088014b6c0 0x7f08801599c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:57.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.738+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 >> 172.21.15.148:0/3145723025 conn(0x7f0880068ce0 msgr2=0x7f088014c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:57.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.739+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 shutdown_connections 2023-10-10T20:20:57.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:57.739+0000 7f08879ce700 1 -- 172.21.15.148:0/3145723025 wait complete. 2023-10-10T20:20:57.958 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:20:58.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.070+0000 7f89c8e33700 1 Processor -- start 2023-10-10T20:20:58.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.070+0000 7f89c8e33700 1 -- start start 2023-10-10T20:20:58.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c8e33700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 0x7f89c406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c8e33700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f89c406a5a0 0x7f89c4156350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c8e33700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c4068340 con 0x7f89c40cbfa0 2023-10-10T20:20:58.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c8e33700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c406abd0 con 0x7f89c406a5a0 2023-10-10T20:20:58.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c8e33700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4068ee0 con 0x7f89c40cc370 2023-10-10T20:20:58.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f89c406a5a0 0x7f89c4156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f89c40cbfa0 0x7f89c40687d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39060/0 (socket says 172.21.15.148:39060) 2023-10-10T20:20:58.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c37fe700 1 -- 172.21.15.148:0/507774838 learned_addr learned my addr 172.21.15.148:0/507774838 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:58.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.071+0000 7f89c2ffd700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 0x7f89c406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c3fff700 1 -- 172.21.15.148:0/507774838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 msgr2=0x7f89c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:20:58.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c3fff700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 0x7f89c406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c3fff700 1 -- 172.21.15.148:0/507774838 >> v1:172.21.15.148:6789/0 conn(0x7f89c40cbfa0 legacy=0x7f89c40687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c3fff700 1 -- 172.21.15.148:0/507774838 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c406f040 con 0x7f89c406a5a0 2023-10-10T20:20:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c27fc700 1 -- 172.21.15.148:0/507774838 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893064010 0 0) 0x7f89c4068340 con 0x7f89c40cbfa0 2023-10-10T20:20:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c2ffd700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 0x7f89c406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:20:58.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c3fff700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c406a5a0 0x7f89c4156350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0098e0 tx=0x7f89bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcae5d17c40e0643 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c27fc700 1 -- 172.21.15.148:0/507774838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc009e80 con 0x7f89c406a5a0 2023-10-10T20:20:58.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c27fc700 1 -- 172.21.15.148:0/507774838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00b860 con 0x7f89c406a5a0 2023-10-10T20:20:58.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c27fc700 1 -- 172.21.15.148:0/507774838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00bbe0 con 0x7f89c406a5a0 2023-10-10T20:20:58.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 -- 172.21.15.148:0/507774838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c406a5a0 msgr2=0x7f89c4156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c406a5a0 0x7f89c4156350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0098e0 tx=0x7f89bc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 -- 172.21.15.148:0/507774838 shutdown_connections 2023-10-10T20:20:58.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c406a5a0 0x7f89c4156350 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 --2- 172.21.15.148:0/507774838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c40cc370 0x7f89c406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 -- 172.21.15.148:0/507774838 >> 172.21.15.148:0/507774838 conn(0x7f89c40cd8f0 msgr2=0x7f89c40cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:58.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 -- 172.21.15.148:0/507774838 shutdown_connections 2023-10-10T20:20:58.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.072+0000 7f89c8e33700 1 -- 172.21.15.148:0/507774838 wait complete. 2023-10-10T20:20:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 Processor -- start 2023-10-10T20:20:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 -- start start 2023-10-10T20:20:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c406a5a0 0x7f89c4141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89c40cbfa0 0x7f89c413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4141a60 con 0x7f89c40cc370 2023-10-10T20:20:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4141090 con 0x7f89c40cbfa0 2023-10-10T20:20:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.074+0000 7f89c8e33700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c413f4b0 con 0x7f89c406a5a0 2023-10-10T20:20:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44134/0 (socket says 172.21.15.148:44134) 2023-10-10T20:20:58.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 -- 172.21.15.148:0/1198961943 learned_addr learned my addr 172.21.15.148:0/1198961943 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:58.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c37fe700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c406a5a0 0x7f89c4141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c2ffd700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89c40cbfa0 0x7f89c413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 -- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c406a5a0 msgr2=0x7f89c4141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c406a5a0 0x7f89c4141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 -- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89c40cbfa0 msgr2=0x7f89c413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89c40cbfa0 0x7f89c413ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009420 con 0x7f89c40cc370 2023-10-10T20:20:58.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c3fff700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89bc01bf90 tx=0x7f89bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc011630 con 0x7f89c40cc370 2023-10-10T20:20:58.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc011790 con 0x7f89c40cc370 2023-10-10T20:20:58.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c4140480 con 0x7f89c40cc370 2023-10-10T20:20:58.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc011b50 con 0x7f89c40cc370 2023-10-10T20:20:58.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.075+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c4140a40 con 0x7f89c40cc370 2023-10-10T20:20:58.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.076+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c406d5d0 con 0x7f89c40cc370 2023-10-10T20:20:58.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.077+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89bc0253f0 con 0x7f89c40cc370 2023-10-10T20:20:58.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.077+0000 7f89ab7fe700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.077+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89bc0861e0 con 0x7f89c40cc370 2023-10-10T20:20:58.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.077+0000 7f89c37fe700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.077+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f89b006ea50 con 0x7f89c40cc370 2023-10-10T20:20:58.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.078+0000 7f89c37fe700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b2dc0 tx=0x7f89b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.082+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89bc053450 con 0x7f89c40cc370 2023-10-10T20:20:58.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.306+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c404bb80 con 0x7f89c40cc370 2023-10-10T20:20:58.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.306+0000 7f89ab7fe700 1 -- 172.21.15.148:0/1198961943 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f89bc015070 con 0x7f89c40cc370 2023-10-10T20:20:58.309 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:58.310 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:20:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 msgr2=0x7f89b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b2dc0 tx=0x7f89b4009040 comp rx=0 tx=0).stop 2023-10-10T20:20:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 msgr2=0x7f89c40c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89bc01bf90 tx=0x7f89bc017000 comp rx=0 tx=0).stop 2023-10-10T20:20:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 shutdown_connections 2023-10-10T20:20:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89c40cc370 0x7f89c40c69b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89c406a5a0 0x7f89c4141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 --2- 172.21.15.148:0/1198961943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89c40cbfa0 0x7f89c413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 >> 172.21.15.148:0/1198961943 conn(0x7f89c40cd8f0 msgr2=0x7f89c40c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 shutdown_connections 2023-10-10T20:20:58.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.309+0000 7f89c8e33700 1 -- 172.21.15.148:0/1198961943 wait complete. 2023-10-10T20:20:58.314 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:20:58.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:20:58.467 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":5367,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.636612346,"mdsmap_epoch":24,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":24.603857789999999} 2023-10-10T20:20:58.467 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 5367, "mdsmap_epoch": 24, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 18.636612346, "state": "up:active", "uptime": 24.60385779, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:20:58.468 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:20:58.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:58.469 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:20:58.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:20:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-10T20:20:58.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:58.485 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-10T20:20:58.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:20:58.541 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-10T20:20:58.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:20:58.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.753+0000 7f204e164700 1 Processor -- start 2023-10-10T20:20:58.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.753+0000 7f204e164700 1 -- start start 2023-10-10T20:20:58.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f204e164700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f204815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f204e164700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204815cbb0 con 0x7f204815a620 2023-10-10T20:20:58.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f204e164700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204815bfa0 con 0x7f2048157c40 2023-10-10T20:20:58.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f204e164700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204815c280 con 0x7f2048157870 2023-10-10T20:20:58.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f2046ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f204815a620 0x7f204815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39068/0 (socket says 172.21.15.148:39068) 2023-10-10T20:20:58.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f2046ffd700 1 -- 172.21.15.148:0/758315123 learned_addr learned my addr 172.21.15.148:0/758315123 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:20:58.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20477fe700 1 --2- 172.21.15.148:0/758315123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f204815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120930456 0 0) 0x7f204815cbb0 con 0x7f204815a620 2023-10-10T20:20:58.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2030003570 con 0x7f204815a620 2023-10-10T20:20:58.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388167103 0 0) 0x7f204815bfa0 con 0x7f2048157c40 2023-10-10T20:20:58.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204815cbb0 con 0x7f2048157c40 2023-10-10T20:20:58.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20477fe700 1 -- 172.21.15.148:0/758315123 >> v1:172.21.15.161:6789/0 conn(0x7f2048157c40 legacy=0x7f2048158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20477fe700 1 -- 172.21.15.148:0/758315123 >> v1:172.21.15.148:6789/0 conn(0x7f204815a620 legacy=0x7f204815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.754+0000 7f20477fe700 1 -- 172.21.15.148:0/758315123 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204815c560 con 0x7f2048157870 2023-10-10T20:20:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f20477fe700 1 --2- 172.21.15.148:0/758315123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f204815b6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2038009910 tx=0x7f20380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac31bacd38f9a413 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038011630 con 0x7f2048157870 2023-10-10T20:20:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038011790 con 0x7f2048157870 2023-10-10T20:20:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f20467fc700 1 -- 172.21.15.148:0/758315123 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038011b30 con 0x7f2048157870 2023-10-10T20:20:58.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 -- 172.21.15.148:0/758315123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 msgr2=0x7f204815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 --2- 172.21.15.148:0/758315123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f204815b6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2038009910 tx=0x7f20380091a0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 -- 172.21.15.148:0/758315123 shutdown_connections 2023-10-10T20:20:58.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 --2- 172.21.15.148:0/758315123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f204815b6a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 -- 172.21.15.148:0/758315123 >> 172.21.15.148:0/758315123 conn(0x7f2048068ce0 msgr2=0x7f20480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:58.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 -- 172.21.15.148:0/758315123 shutdown_connections 2023-10-10T20:20:58.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.755+0000 7f204e164700 1 -- 172.21.15.148:0/758315123 wait complete. 2023-10-10T20:20:58.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 Processor -- start 2023-10-10T20:20:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 -- start start 2023-10-10T20:20:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f2048152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204815a620 0x7f20480b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204814ffc0 con 0x7f2048157c40 2023-10-10T20:20:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480b2fb0 con 0x7f204815a620 2023-10-10T20:20:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f204e164700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204814f3b0 con 0x7f2048157870 2023-10-10T20:20:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f2046ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f2046ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44148/0 (socket says 172.21.15.148:44148) 2023-10-10T20:20:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.757+0000 7f2046ffd700 1 -- 172.21.15.148:0/943193383 learned_addr learned my addr 172.21.15.148:0/943193383 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:20:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f20477fe700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f2048152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2047fff700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204815a620 0x7f20480b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 -- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 msgr2=0x7f2048152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f2048152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 -- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204815a620 msgr2=0x7f20480b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:58.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204815a620 0x7f20480b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:58.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038009420 con 0x7f2048157c40 2023-10-10T20:20:58.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f2046ffd700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c7c0 tx=0x7f203c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c005b30 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c005c90 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204814f690 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c007300 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.758+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204814fc50 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.759+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048150170 con 0x7f2048157c40 2023-10-10T20:20:58.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.759+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f203c01b3b0 con 0x7f2048157c40 2023-10-10T20:20:58.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.760+0000 7f202f7fe700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:20:58.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.760+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f203c083930 con 0x7f2048157c40 2023-10-10T20:20:58.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.760+0000 7f20477fe700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:20:58.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.760+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f203406ea50 con 0x7f2048157c40 2023-10-10T20:20:58.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.760+0000 7f20477fe700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 0x7f20340693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2038009860 tx=0x7f2038017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:20:58.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.764+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f203c017020 con 0x7f2048157c40 2023-10-10T20:20:58.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.964+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f20480d1cb0 con 0x7f2048157c40 2023-10-10T20:20:58.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.965+0000 7f202f7fe700 1 -- 172.21.15.148:0/943193383 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32451 (secure 0 0 0) 0x7f2048150170 con 0x7f2048157c40 2023-10-10T20:20:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:20:58.968 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:20:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 49, 2023-10-10T20:20:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:20:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:20:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:20:55.503638+0000", 2023-10-10T20:20:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:20:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:20:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:20:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:20:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:20:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:20:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:20:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:20:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:20:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:20:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:20:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:20:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:20:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:20:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:20:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:20:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:20:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:20:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:20:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:20:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:20:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:20:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:20:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:20:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:20:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:20:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:20:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:20:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:20:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:20:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:20:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:20:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:20:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:20:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:20:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:20:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:20:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:20:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:20:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:20:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:20:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:20:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:20:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:20:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:20:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:20:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:20:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:20:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:20:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:20:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:20:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:20:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:20:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:20:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:20:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:20:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:20:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:20:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:20:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:20:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:20:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:20:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:20:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:20:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:20:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:20:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:20:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:20:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:20:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:20:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:20:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:20:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:20:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:20:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:20:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:20:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:20:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:20:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:20:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:20:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:20:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:20:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:20:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:20:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:20:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:20:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:20:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:20:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:20:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:20:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:20:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:20:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:20:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:20:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:20:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:20:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:20:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:20:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:20:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:20:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:20:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:20:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:20:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:20:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:20:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:20:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:20:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:20:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:20:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:20:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:20:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:20:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:20:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:20:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:20:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:20:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:20:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:20:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:20:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:20:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:20:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:20:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:20:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:20:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:20:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:20:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:20:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:20:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:20:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:20:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:20:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:20:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:20:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:20:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:20:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:20:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:20:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:20:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:20:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:20:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:20:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:20:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:20:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:20:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:20:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:20:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:20:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:20:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:20:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:20:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:20:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:20:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:20:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:20:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:20:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:20:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:20:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:20:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:20:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:20:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:20:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:20:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:20:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:20:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:20:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:20:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:20:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:20:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:20:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:20:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:20:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:20:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:20:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:20:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:20:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:20:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:20:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:20:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:20:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:20:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:20:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:20:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:20:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:20:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:20:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:20:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:20:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:20:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:20:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:20:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:20:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:20:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:20:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:20:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:20:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:20:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:20:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:20:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:20:59.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.158 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:20:59.158 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:20:59.158 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.159 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:20:59.159 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:20:59.159 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:20:59.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.159 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.160 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:20:59.160 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:20:59.160 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:20:59.161 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:20:59.161 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:20:59.161 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:20:59.161 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:20:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:20:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:20:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:59.162 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.163 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:20:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:20:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.165 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.166 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.166 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:20:59.166 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:20:59.167 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.167 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.167 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:20:59.169 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.169 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.169 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.169 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.170 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.170 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:20:59.170 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.171 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:20:59.171 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.171 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.171 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:20:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.173 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.174 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.174 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:20:59.174 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.174 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.175 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:20:59.175 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.175 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.175 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.175 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:20:59.176 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:20:59.177 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:20:59.177 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:20:59.177 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:20:59.177 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:20:59.177 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:20:59.178 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:20:59.178 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:20:59.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.178 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:20:59.178 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:20:59.179 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:20:59.179 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:20:59.179 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:20:59.179 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:20:59.179 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:20:59.180 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:20:59.180 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:20:59.180 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:20:59.180 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:20:59.180 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:20:59.181 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:20:59.182 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:20:59.182 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:20:59.182 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:20:59.182 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:20:59.182 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:20:59.183 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:20:59.183 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:20:59.183 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:20:59.183 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:20:59.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:20:59.184 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:20:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 msgr2=0x7f20340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 0x7f20340693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2038009860 tx=0x7f2038017000 comp rx=0 tx=0).stop 2023-10-10T20:20:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 msgr2=0x7f2048150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:20:59.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c7c0 tx=0x7f203c007ea0 comp rx=0 tx=0).stop 2023-10-10T20:20:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 shutdown_connections 2023-10-10T20:20:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2034066f30 0x7f20340693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:59.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f204815a620 0x7f20480b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2048157870 0x7f2048152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 --2- 172.21.15.148:0/943193383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2048157c40 0x7f2048150450 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:20:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 >> 172.21.15.148:0/943193383 conn(0x7f2048068ce0 msgr2=0x7f20480b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:20:59.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 shutdown_connections 2023-10-10T20:20:59.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:20:58.967+0000 7f204e164700 1 -- 172.21.15.148:0/943193383 wait complete. 2023-10-10T20:20:59.189 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:21:00.481 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:21:00.478+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-10T20:21:04.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:21:04.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.316+0000 7f597e3df700 1 Processor -- start 2023-10-10T20:21:04.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.316+0000 7f597e3df700 1 -- start start 2023-10-10T20:21:04.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f597e3df700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5978157870 0x7f597815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:04.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f597e3df700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597815cbb0 con 0x7f5978157c40 2023-10-10T20:21:04.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f597e3df700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597815bfa0 con 0x7f597815a620 2023-10-10T20:21:04.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f597e3df700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597815c280 con 0x7f5978157870 2023-10-10T20:21:04.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5978157870 0x7f597815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5978157870 0x7f597815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58828/0 (socket says 172.21.15.148:58828) 2023-10-10T20:21:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 -- 172.21.15.148:0/4204526384 learned_addr learned my addr 172.21.15.148:0/4204526384 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 -- 172.21.15.148:0/4204526384 >> v1:172.21.15.161:6789/0 conn(0x7f597815a620 legacy=0x7f597815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 -- 172.21.15.148:0/4204526384 >> v1:172.21.15.148:6789/0 conn(0x7f5978157c40 legacy=0x7f5978158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5976ffd700 1 -- 172.21.15.148:0/4204526384 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506291301 0 0) 0x7f597815cbb0 con 0x7f5978157c40 2023-10-10T20:21:04.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 -- 172.21.15.148:0/4204526384 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597815c560 con 0x7f5978157870 2023-10-10T20:21:04.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5977fff700 1 --2- 172.21.15.148:0/4204526384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5978157870 0x7f597815b6a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f59680099b0 tx=0x7f5968009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398a80ad01e27754 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:04.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5976ffd700 1 -- 172.21.15.148:0/4204526384 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800b860 con 0x7f5978157870 2023-10-10T20:21:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5976ffd700 1 -- 172.21.15.148:0/4204526384 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800b9c0 con 0x7f5978157870 2023-10-10T20:21:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.317+0000 7f5976ffd700 1 -- 172.21.15.148:0/4204526384 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800bd60 con 0x7f5978157870 2023-10-10T20:21:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 -- 172.21.15.148:0/4204526384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5978157870 msgr2=0x7f597815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 --2- 172.21.15.148:0/4204526384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5978157870 0x7f597815b6a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f59680099b0 tx=0x7f5968009250 comp rx=0 tx=0).stop 2023-10-10T20:21:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 -- 172.21.15.148:0/4204526384 shutdown_connections 2023-10-10T20:21:04.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 --2- 172.21.15.148:0/4204526384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5978157870 0x7f597815b6a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 -- 172.21.15.148:0/4204526384 >> 172.21.15.148:0/4204526384 conn(0x7f5978068ce0 msgr2=0x7f59780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:04.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 -- 172.21.15.148:0/4204526384 shutdown_connections 2023-10-10T20:21:04.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.318+0000 7f597e3df700 1 -- 172.21.15.148:0/4204526384 wait complete. 2023-10-10T20:21:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.319+0000 7f597e3df700 1 Processor -- start 2023-10-10T20:21:04.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.319+0000 7f597e3df700 1 -- start start 2023-10-10T20:21:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5978157870 0x7f597813d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5978157c40 0x7f5978153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:04.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978152b60 con 0x7f597815a620 2023-10-10T20:21:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597813d610 con 0x7f5978157c40 2023-10-10T20:21:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597e3df700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5978154500 con 0x7f5978157870 2023-10-10T20:21:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597c97c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:04.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597c97c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58834/0 (socket says 172.21.15.148:58834) 2023-10-10T20:21:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f597c97c700 1 -- 172.21.15.148:0/4279020365 learned_addr learned my addr 172.21.15.148:0/4279020365 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f5977fff700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5978157870 0x7f597813d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.320+0000 7f59777fe700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5978157c40 0x7f5978153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 -- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5978157870 msgr2=0x7f597813d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5978157870 0x7f597813d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 -- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5978157c40 msgr2=0x7f5978153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5978157c40 0x7f5978153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680094f0 con 0x7f597815a620 2023-10-10T20:21:04.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597c97c700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f597000c930 tx=0x7f59700016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:04.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970012630 con 0x7f597815a620 2023-10-10T20:21:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970012790 con 0x7f597815a620 2023-10-10T20:21:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970012b50 con 0x7f597815a620 2023-10-10T20:21:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597816b7d0 con 0x7f597815a620 2023-10-10T20:21:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.321+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781534e0 con 0x7f597815a620 2023-10-10T20:21:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.322+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597804c3c0 con 0x7f597815a620 2023-10-10T20:21:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.322+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f597001a3a0 con 0x7f597815a620 2023-10-10T20:21:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.323+0000 7f595ffff700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 0x7f59640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:04.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.323+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59700846b0 con 0x7f597815a620 2023-10-10T20:21:04.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.323+0000 7f5977fff700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 0x7f59640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:04.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.323+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f596406ea50 con 0x7f597815a620 2023-10-10T20:21:04.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.323+0000 7f5977fff700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 0x7f59640693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59680099b0 tx=0x7f5968017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:04.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.327+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5970051920 con 0x7f597815a620 2023-10-10T20:21:04.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.510+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f597815c890 con 0x7f597815a620 2023-10-10T20:21:04.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.511+0000 7f595ffff700 1 -- 172.21.15.148:0/4279020365 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32451 (secure 0 0 0) 0x7f5970054f40 con 0x7f597815a620 2023-10-10T20:21:04.515 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:04.515 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:21:04.515 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 49, 2023-10-10T20:21:04.515 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:21:04.515 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:21:04.516 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:20:55.503638+0000", 2023-10-10T20:21:04.516 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:21:04.516 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:21:04.516 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:21:04.516 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:21:04.517 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:21:04.517 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:21:04.517 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:21:04.517 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:21:04.517 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:21:04.518 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:21:04.519 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:21:04.519 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:21:04.519 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:21:04.519 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:21:04.519 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:21:04.520 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:21:04.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.520 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:21:04.520 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:21:04.520 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:04.521 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:04.522 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:04.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:04.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:21:04.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:21:04.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:21:04.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:04.524 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:04.525 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:04.525 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:04.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:21:04.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:04.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:04.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:04.526 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:04.526 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:04.526 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:04.527 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:04.527 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:04.527 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:04.527 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:04.527 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:04.528 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:04.529 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:04.529 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:04.529 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:04.529 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:04.529 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:04.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:04.530 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:04.530 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:04.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:04.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.531 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:04.531 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:04.531 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:04.531 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:04.531 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:04.532 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:04.533 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:04.533 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:21:04.533 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:21:04.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.533 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:21:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.534 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:21:04.535 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:21:04.535 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:21:04.535 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:21:04.535 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:21:04.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:04.536 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:04.536 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:04.536 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:04.536 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:04.536 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:21:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:21:04.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:21:04.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:21:04.538 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:21:04.538 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:04.538 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:04.539 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:04.539 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:04.539 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:04.539 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:04.539 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:04.540 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:04.541 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:04.541 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:04.541 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:04.541 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:04.541 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:04.542 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:04.542 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:04.542 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:04.542 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:04.542 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:04.543 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:04.543 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:04.543 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:04.543 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:04.543 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:04.544 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:04.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:04.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.545 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:04.545 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:04.545 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:04.546 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:04.546 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:04.546 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:04.546 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:04.546 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:04.547 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:04.547 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:04.547 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:04.547 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:04.547 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:04.548 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:21:04.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:04.550 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:04.551 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:04.551 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:04.551 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:04.551 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:04.551 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:04.552 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:04.552 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:04.552 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:04.552 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:21:04.552 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:04.553 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:04.554 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:04.554 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:04.554 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:04.554 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:04.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "47", 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:04.555 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:04.556 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:04.557 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:04.558 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:04.559 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:04.559 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:04.559 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:04.559 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:04.559 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:04.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.560 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:04.560 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:04.560 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:04.560 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:04.561 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:04.562 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:04.562 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:04.562 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:21:04.562 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:21:04.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.563 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.564 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:04.564 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:21:04.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.564 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:04.564 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.565 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.566 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.567 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.567 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.567 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:21:04.567 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:21:04.568 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:21:04.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.571 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.571 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:21:04.571 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:21:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:21:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:21:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:04.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:21:04.577 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:21:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:21:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.578 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.579 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.580 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.581 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.583 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.583 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:21:04.583 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:21:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.585 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:21:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.586 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:21:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:21:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.592 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.593 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.594 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.595 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.596 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:21:04.597 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.598 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:21:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.600 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.601 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:21:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.603 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:21:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:04.605 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:21:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:21:04.606 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.607 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.608 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.609 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.610 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:21:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.612 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:21:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:21:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.618 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.619 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:21:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.620 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:21:04.621 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.622 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.623 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.624 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.625 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:21:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.626 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.627 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:21:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.629 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.630 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.631 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:21:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.632 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:21:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:04.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:21:04.635 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:21:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.637 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.638 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.639 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.640 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:21:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.641 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:21:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.642 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:21:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.644 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:21:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.646 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:21:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:21:04.648 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:21:04.649 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:21:04.650 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.651 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.652 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.653 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.654 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:21:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.656 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:21:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.658 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:21:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.659 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.661 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:21:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.664 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:21:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:21:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:21:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:21:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.665 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:21:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:04.667 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:04.668 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.669 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:21:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.670 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:04.671 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:21:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.675 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:21:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.676 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:21:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:04.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:21:04.679 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:21:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:21:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:04.680 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:21:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.682 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:21:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:04.683 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:21:04.684 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.685 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.687 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:21:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.691 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:04.692 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:21:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:04.694 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:21:04.695 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:21:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:21:04.696 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:21:04.697 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:21:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:21:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:21:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:21:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:21:04.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:21:04.699 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:21:04.700 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:21:04.700 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:21:04.700 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:21:04.700 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:21:04.700 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:21:04.701 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:21:04.701 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:04.701 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:21:04.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 msgr2=0x7f59640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 0x7f59640693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59680099b0 tx=0x7f5968017000 comp rx=0 tx=0).stop 2023-10-10T20:21:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 msgr2=0x7f59780b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f597000c930 tx=0x7f59700016f0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 shutdown_connections 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5964066f30 0x7f59640693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f597815a620 0x7f59780b5f10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5978157870 0x7f597813d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 --2- 172.21.15.148:0/4279020365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5978157c40 0x7f5978153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 >> 172.21.15.148:0/4279020365 conn(0x7f5978068ce0 msgr2=0x7f597814d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:04.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 shutdown_connections 2023-10-10T20:21:04.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:04.513+0000 7f597e3df700 1 -- 172.21.15.148:0/4279020365 wait complete. 2023-10-10T20:21:04.705 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:21:05.535 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:21:05.533+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-10T20:21:09.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:21:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.831+0000 7f7f0866f700 1 Processor -- start 2023-10-10T20:21:09.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.832+0000 7f7f0866f700 1 -- start start 2023-10-10T20:21:09.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.832+0000 7f7f0866f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7f000b1220 0x7f7f000c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0866f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0015de60 con 0x7f7f000b0e50 2023-10-10T20:21:09.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0866f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0015e140 con 0x7f7f000cb4f0 2023-10-10T20:21:09.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0866f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0015e420 con 0x7f7f000b1220 2023-10-10T20:21:09.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0640b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7f000b0e50 0x7f7f0006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44530/0 (socket says 172.21.15.148:44530) 2023-10-10T20:21:09.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0640b700 1 -- 172.21.15.148:0/3604791416 learned_addr learned my addr 172.21.15.148:0/3604791416 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:09.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05c0a700 1 --2- 172.21.15.148:0/3604791416 >> v2:172.21.15.161:3300/0 conn(0x7f7f000b1220 0x7f7f000c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958626770 0 0) 0x7f7f0015de60 con 0x7f7f000b0e50 2023-10-10T20:21:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eec003570 con 0x7f7f000b0e50 2023-10-10T20:21:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123972666 0 0) 0x7f7f0015e140 con 0x7f7f000cb4f0 2023-10-10T20:21:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f0015de60 con 0x7f7f000cb4f0 2023-10-10T20:21:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96862697 0 0) 0x7f7eec003570 con 0x7f7f000b0e50 2023-10-10T20:21:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 >> v2:172.21.15.161:3300/0 conn(0x7f7f000b1220 msgr2=0x7f7f000c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 --2- 172.21.15.148:0/3604791416 >> v2:172.21.15.161:3300/0 conn(0x7f7f000b1220 0x7f7f000c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 >> v1:172.21.15.161:6789/0 conn(0x7f7f000cb4f0 legacy=0x7f7f000679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0015e700 con 0x7f7f000b0e50 2023-10-10T20:21:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7efc001470 con 0x7f7f000b0e50 2023-10-10T20:21:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 >> v1:172.21.15.148:6789/0 conn(0x7f7f000b0e50 legacy=0x7f7f0006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:09.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 0x7f7eec003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec004810 0x7f7eec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7eec005230 0x7f7eec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0015e420 con 0x7f7eec003bf0 2023-10-10T20:21:09.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec00c200 con 0x7f7eec005230 2023-10-10T20:21:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec00c3c0 con 0x7f7eec004810 2023-10-10T20:21:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.833+0000 7f7f0640b700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 0x7f7eec003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f06c0c700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7eec005230 0x7f7eec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f05c0a700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec004810 0x7f7eec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 -- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec004810 msgr2=0x7f7eec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:09.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec004810 0x7f7eec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 -- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7eec005230 msgr2=0x7f7eec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:09.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7eec005230 0x7f7eec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 -- 172.21.15.148:0/3604791416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec003570 con 0x7f7eec003bf0 2023-10-10T20:21:09.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f06c0c700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7eec005230 0x7f7eec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:09.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f05c0a700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec004810 0x7f7eec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:09.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f06c0c700 1 -- 172.21.15.148:0/3604791416 reap_dead start 2023-10-10T20:21:09.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0640b700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 0x7f7eec003fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00c2d0 tx=0x7f7efc001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c63280af24072e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:09.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00cba0 con 0x7f7eec003bf0 2023-10-10T20:21:09.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00cd00 con 0x7f7eec003bf0 2023-10-10T20:21:09.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f05409700 1 -- 172.21.15.148:0/3604791416 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc005b70 con 0x7f7eec003bf0 2023-10-10T20:21:09.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0866f700 1 -- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 msgr2=0x7f7eec003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:09.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.834+0000 7f7f0866f700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 0x7f7eec003fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00c2d0 tx=0x7f7efc001e10 comp rx=0 tx=0).stop 2023-10-10T20:21:09.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.835+0000 7f7f0866f700 1 -- 172.21.15.148:0/3604791416 shutdown_connections 2023-10-10T20:21:09.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.835+0000 7f7f0866f700 1 --2- 172.21.15.148:0/3604791416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eec003bf0 0x7f7eec003fe0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.835+0000 7f7f0866f700 1 -- 172.21.15.148:0/3604791416 >> 172.21.15.148:0/3604791416 conn(0x7f7f00069870 msgr2=0x7f7f00069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.835+0000 7f7f0866f700 1 -- 172.21.15.148:0/3604791416 shutdown_connections 2023-10-10T20:21:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.835+0000 7f7f0866f700 1 -- 172.21.15.148:0/3604791416 wait complete. 2023-10-10T20:21:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 Processor -- start 2023-10-10T20:21:09.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 -- start start 2023-10-10T20:21:09.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 0x7f7f0014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0014e480 0x7f7f00156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00156fc0 con 0x7f7f00156b60 2023-10-10T20:21:09.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f001509e0 con 0x7f7f0014e480 2023-10-10T20:21:09.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.836+0000 7f7f0866f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00150550 con 0x7f7eec003bf0 2023-10-10T20:21:09.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58838/0 (socket says 172.21.15.148:58838) 2023-10-10T20:21:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 -- 172.21.15.148:0/2144241435 learned_addr learned my addr 172.21.15.148:0/2144241435 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f05c0a700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0014e480 0x7f7f00156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f0640b700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 0x7f7f0014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 -- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 msgr2=0x7f7f0014df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:09.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 0x7f7f0014df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 -- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0014e480 msgr2=0x7f7f00156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:09.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0014e480 0x7f7f00156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:09.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 -- 172.21.15.148:0/2144241435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0014f940 con 0x7f7f00156b60 2023-10-10T20:21:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f0640b700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 0x7f7f0014df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f06c0c700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7f000cb8c0 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80032e0 con 0x7f7f00156b60 2023-10-10T20:21:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8003440 con 0x7f7f00156b60 2023-10-10T20:21:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0014fc20 con 0x7f7f00156b60 2023-10-10T20:21:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8006df0 con 0x7f7f00156b60 2023-10-10T20:21:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.837+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f00150260 con 0x7f7f00156b60 2023-10-10T20:21:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.838+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0004c3c0 con 0x7f7f00156b60 2023-10-10T20:21:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.839+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ef80018f0 con 0x7f7f00156b60 2023-10-10T20:21:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.839+0000 7f7ef67fc700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 0x7f7ee4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.839+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ef8083ba0 con 0x7f7f00156b60 2023-10-10T20:21:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.839+0000 7f7f0640b700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 0x7f7ee4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.839+0000 7f7f0640b700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 0x7f7ee4069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00c2d0 tx=0x7f7efc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:09.843+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ef8050d70 con 0x7f7f00156b60 2023-10-10T20:21:10.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.025+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7f00150690 con 0x7f7f00156b60 2023-10-10T20:21:10.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.026+0000 7f7ef67fc700 1 -- 172.21.15.148:0/2144241435 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+32308 (secure 0 0 0) 0x7f7ef8019020 con 0x7f7f00156b60 2023-10-10T20:21:10.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:10.029 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:21:10.030 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 52, 2023-10-10T20:21:10.030 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:21:10.030 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:21:10.030 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:21:08.536665+0000", 2023-10-10T20:21:10.031 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:21:10.031 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:21:10.031 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:21:10.031 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:21:10.031 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:21:10.032 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:21:10.032 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:21:10.032 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:21:10.032 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:21:10.032 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.033 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:21:10.033 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:21:10.033 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:21:10.033 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:21:10.033 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:21:10.034 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:21:10.034 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:21:10.034 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:21:10.034 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:21:10.035 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:21:10.035 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:21:10.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.035 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:21:10.035 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:21:10.036 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:21:10.036 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.036 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.037 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.037 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.037 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.037 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.037 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.038 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.038 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.038 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.038 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.038 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:21:10.039 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:21:10.039 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:21:10.039 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:21:10.039 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:21:10.040 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.040 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.040 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.040 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.040 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.041 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.041 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.041 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.042 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.042 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.042 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.042 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.042 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.043 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.043 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.043 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.044 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.044 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.044 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.044 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.044 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.045 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.045 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.045 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.045 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.047 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.047 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.048 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.048 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.048 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.048 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.048 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.049 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.049 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.049 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.049 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.050 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.050 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.050 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.050 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:21:10.050 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:21:10.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.051 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.051 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:21:10.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.052 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:21:10.052 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:21:10.052 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:21:10.053 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.053 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.053 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.053 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.054 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.054 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.054 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.054 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.055 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.055 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.055 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.055 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:21:10.055 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:21:10.056 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:21:10.056 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:21:10.056 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:21:10.056 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.057 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.057 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.057 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.057 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.057 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.058 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.058 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.058 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.058 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.059 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.059 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.059 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.059 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.060 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.060 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.060 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.060 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.060 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.061 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.061 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.061 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.061 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.061 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.062 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.062 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.062 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.062 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.064 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.064 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.064 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.065 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.065 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.065 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.065 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.065 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.066 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.066 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.066 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.066 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.066 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:21:10.067 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:21:10.067 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:21:10.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.067 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.068 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:10.068 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:21:10.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.068 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.069 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:21:10.069 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:21:10.069 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:21:10.069 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.070 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.070 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.070 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.070 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.071 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.071 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.071 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.071 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.071 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.072 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.072 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.072 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:21:10.072 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:21:10.073 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:21:10.073 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:21:10.073 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:21:10.073 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.073 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.074 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.074 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.074 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.074 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.074 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.075 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.075 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.075 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.076 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.076 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.076 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.076 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.076 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.077 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.077 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.077 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.077 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.078 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.078 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.078 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.078 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.078 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.079 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.079 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.079 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.079 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.079 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.080 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.080 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.080 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.080 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.081 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.081 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.081 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.082 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.082 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.082 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.082 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.082 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.083 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.083 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.083 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.083 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.083 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:21:10.084 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:21:10.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.084 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:10.085 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:21:10.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.085 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.086 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:21:10.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.086 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:10.086 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.086 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.087 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.087 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.087 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.088 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.088 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.088 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.088 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.089 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:21:10.090 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.090 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.090 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:21:10.091 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.091 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.091 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.092 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:21:10.093 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:21:10.094 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.095 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.095 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.096 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:21:10.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:21:10.097 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.097 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.098 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.101 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:21:10.101 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:21:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:21:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.104 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:10.105 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.105 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.106 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.106 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.106 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.106 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.107 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.107 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.107 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.107 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.107 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.108 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.108 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:21:10.109 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.109 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.110 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:21:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.113 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.113 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.113 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.114 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:21:10.114 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.115 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:21:10.115 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.115 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.115 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.116 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.116 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.117 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.118 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:21:10.118 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.118 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.119 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:21:10.119 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:21:10.119 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:21:10.119 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:21:10.119 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.120 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.120 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.120 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.121 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.122 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.122 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.122 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.122 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.122 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.123 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.123 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:21:10.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.125 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.125 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:21:10.126 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:21:10.127 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.128 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.128 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.128 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.129 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.129 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:21:10.129 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.130 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.130 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.130 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:21:10.130 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.131 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.131 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.131 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.132 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.132 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:21:10.132 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.133 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.133 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:21:10.133 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:21:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:21:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:21:10.134 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:21:10.135 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.135 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.135 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.135 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.136 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:10.136 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.136 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.136 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.137 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.137 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.137 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.137 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.137 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.138 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.138 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.138 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.138 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.138 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.139 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:21:10.139 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.140 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.141 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.142 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.142 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:21:10.142 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.143 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.143 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:21:10.143 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.143 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.144 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.144 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.144 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.145 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:21:10.145 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.145 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:21:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:21:10.147 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.148 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.148 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:21:10.148 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.149 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.149 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.149 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:21:10.149 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:21:10.150 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:21:10.150 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:21:10.150 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.150 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.150 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.151 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.151 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:10.151 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.154 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.154 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.154 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.154 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:21:10.155 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.156 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.156 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.156 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.157 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.157 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:21:10.157 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.159 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.159 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.159 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.160 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.160 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:21:10.160 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.162 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.162 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.162 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.162 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:21:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.163 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:21:10.164 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.164 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.164 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.164 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.165 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.166 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:10.166 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.167 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.167 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.167 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.167 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.168 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.168 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.168 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.168 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.168 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.169 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.169 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.169 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.169 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.170 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:21:10.171 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.171 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.171 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.171 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.172 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.172 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.172 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.172 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:21:10.172 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.173 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.173 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:21:10.173 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.174 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.174 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.174 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.175 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.175 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:21:10.175 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.176 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.176 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.176 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:21:10.176 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.176 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.177 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.177 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.177 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.177 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.178 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.179 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:21:10.179 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.179 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.179 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.180 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:21:10.180 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:21:10.180 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:21:10.180 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:21:10.180 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.181 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.181 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.181 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.181 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.181 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.182 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.183 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.183 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.183 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.183 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.183 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.184 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.184 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.184 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.184 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.185 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:21:10.186 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.186 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.186 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.186 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.187 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.187 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.187 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:21:10.187 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.188 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.188 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:21:10.188 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.190 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.190 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:21:10.190 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.190 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.191 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.191 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.191 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:21:10.191 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.191 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.192 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.192 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.192 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.192 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.193 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.193 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:21:10.193 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.194 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.194 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:21:10.194 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.194 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.195 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:21:10.195 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:21:10.195 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:21:10.195 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:21:10.196 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.196 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.196 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.196 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.196 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.197 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:10.197 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.197 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.197 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.198 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.198 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.198 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.198 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.199 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.199 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.199 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.199 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.199 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.200 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.200 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.200 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:21:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.201 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.202 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.203 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.203 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:21:10.203 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.204 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.205 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.205 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.205 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.206 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:21:10.206 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.206 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.207 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.208 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.208 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.208 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:21:10.208 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.209 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.209 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:21:10.209 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:21:10.210 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:21:10.211 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:21:10.211 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.211 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.211 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.211 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.212 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.212 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:21:10.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.212 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:10.213 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.213 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.213 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.213 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.213 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:10.214 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.215 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.215 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.215 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.215 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.215 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:21:10.216 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.216 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.216 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:10.216 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.217 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.217 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.217 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.217 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.217 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:21:10.218 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.218 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:10.218 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.219 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.219 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.219 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.219 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:21:10.220 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.220 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:10.220 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.221 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.221 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.221 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.221 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.221 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:21:10.222 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.222 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.222 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.222 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:10.222 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.223 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.223 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.223 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.223 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.223 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.224 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:21:10.225 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.226 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.226 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:10.226 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.227 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.227 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.227 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.227 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.227 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:21:10.228 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.228 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.228 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.228 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:21:10.228 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:21:10.229 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:21:10.229 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:21:10.229 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:21:10.229 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:21:10.229 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:21:10.230 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:21:10.230 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:21:10.230 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:21:10.230 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:21:10.230 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:21:10.231 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:21:10.232 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:21:10.232 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:21:10.232 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:21:10.232 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:21:10.232 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:21:10.233 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:21:10.233 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:21:10.233 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:21:10.233 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.233 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:21:10.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.028+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 msgr2=0x7f7ee4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.028+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 0x7f7ee4069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00c2d0 tx=0x7f7efc015000 comp rx=0 tx=0).stop 2023-10-10T20:21:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.028+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 msgr2=0x7f7f0014ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.028+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7f000cb8c0 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2023-10-10T20:21:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 shutdown_connections 2023-10-10T20:21:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ee4066f60 0x7f7ee4069410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f00156b60 0x7f7f0014ebc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7eec003bf0 0x7f7f0014df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 --2- 172.21.15.148:0/2144241435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0014e480 0x7f7f00156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 >> 172.21.15.148:0/2144241435 conn(0x7f7f00069870 msgr2=0x7f7f001580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 shutdown_connections 2023-10-10T20:21:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.029+0000 7f7f0866f700 1 -- 172.21.15.148:0/2144241435 wait complete. 2023-10-10T20:21:10.238 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:21:10.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:21:10.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.351+0000 7f8ca4701700 1 Processor -- start 2023-10-10T20:21:10.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.351+0000 7f8ca4701700 1 -- start start 2023-10-10T20:21:10.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca4701700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c9c152010 0x7f8c9c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca4701700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c9c158e50 0x7f8c9c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca4701700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c06c010 con 0x7f8c9c1523e0 2023-10-10T20:21:10.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca4701700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c06bd10 con 0x7f8c9c158e50 2023-10-10T20:21:10.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca4701700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c15c060 con 0x7f8c9c152010 2023-10-10T20:21:10.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca2c9e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c9c158e50 0x7f8c9c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca1c9c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8c9c1523e0 0x7f8c9c06c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44534/0 (socket says 172.21.15.148:44534) 2023-10-10T20:21:10.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.352+0000 7f8ca1c9c700 1 -- 172.21.15.148:0/1891623672 learned_addr learned my addr 172.21.15.148:0/1891623672 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:10.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca249d700 1 --2- 172.21.15.148:0/1891623672 >> v2:172.21.15.161:3300/0 conn(0x7f8c9c152010 0x7f8c9c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1891623672 >> v2:172.21.15.161:3300/0 conn(0x7f8c9c152010 msgr2=0x7f8c9c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:10.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca2c9e700 1 --2- 172.21.15.148:0/1891623672 >> v2:172.21.15.161:3300/0 conn(0x7f8c9c152010 0x7f8c9c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1891623672 >> v1:172.21.15.148:6789/0 conn(0x7f8c9c1523e0 legacy=0x7f8c9c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca149b700 1 -- 172.21.15.148:0/1891623672 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165601242 0 0) 0x7f8c9c06c010 con 0x7f8c9c1523e0 2023-10-10T20:21:10.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1891623672 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c15c340 con 0x7f8c9c158e50 2023-10-10T20:21:10.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca2c9e700 1 --2- 172.21.15.148:0/1891623672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c0cdf30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0cc680 tx=0x7f8c94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae64d25800ed75c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca149b700 1 -- 172.21.15.148:0/1891623672 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400e070 con 0x7f8c9c158e50 2023-10-10T20:21:10.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca149b700 1 -- 172.21.15.148:0/1891623672 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9400bc30 con 0x7f8c9c158e50 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.353+0000 7f8ca149b700 1 -- 172.21.15.148:0/1891623672 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94012430 con 0x7f8c9c158e50 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 -- 172.21.15.148:0/1891623672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 msgr2=0x7f8c9c0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1891623672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c0cdf30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0cc680 tx=0x7f8c94009780 comp rx=0 tx=0).stop 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 -- 172.21.15.148:0/1891623672 shutdown_connections 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1891623672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c0cdf30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1891623672 >> v2:172.21.15.161:3300/0 conn(0x7f8c9c152010 0x7f8c9c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 -- 172.21.15.148:0/1891623672 >> 172.21.15.148:0/1891623672 conn(0x7f8c9c069870 msgr2=0x7f8c9c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:10.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 -- 172.21.15.148:0/1891623672 shutdown_connections 2023-10-10T20:21:10.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.354+0000 7f8ca4701700 1 -- 172.21.15.148:0/1891623672 wait complete. 2023-10-10T20:21:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.355+0000 7f8ca4701700 1 Processor -- start 2023-10-10T20:21:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.355+0000 7f8ca4701700 1 -- start start 2023-10-10T20:21:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c152010 0x7f8c9c0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c9c1523e0 0x7f8c9c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c6d90 con 0x7f8c9c158e50 2023-10-10T20:21:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c6440 con 0x7f8c9c152010 2023-10-10T20:21:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c6720 con 0x7f8c9c1523e0 2023-10-10T20:21:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58856/0 (socket says 172.21.15.148:58856) 2023-10-10T20:21:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1903735435 learned_addr learned my addr 172.21.15.148:0/1903735435 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca249d700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c152010 0x7f8c9c0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca1c9c700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c9c1523e0 0x7f8c9c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c9c1523e0 msgr2=0x7f8c9c0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c9c1523e0 0x7f8c9c0c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c152010 msgr2=0x7f8c9c0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c152010 0x7f8c9c0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 -- 172.21.15.148:0/1903735435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94009a20 con 0x7f8c9c158e50 2023-10-10T20:21:10.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca2c9e700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097c0 tx=0x7f8c94019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94027470 con 0x7f8c9c158e50 2023-10-10T20:21:10.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c940275f0 con 0x7f8c9c158e50 2023-10-10T20:21:10.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c0c6a00 con 0x7f8c9c158e50 2023-10-10T20:21:10.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940279b0 con 0x7f8c9c158e50 2023-10-10T20:21:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.356+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c0c6130 con 0x7f8c9c158e50 2023-10-10T20:21:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.357+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c0c7220 con 0x7f8c9c158e50 2023-10-10T20:21:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.358+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c94024070 con 0x7f8c9c158e50 2023-10-10T20:21:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.358+0000 7f8c927fc700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.358+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c94086cf0 con 0x7f8c9c158e50 2023-10-10T20:21:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.358+0000 7f8ca249d700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.358+0000 7f8ca249d700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 0x7f8c800693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006fd0 tx=0x7f8c98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.362+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c9408b070 con 0x7f8c9c158e50 2023-10-10T20:21:10.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.544+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8c9c04c3c0 con 0x7f8c9c158e50 2023-10-10T20:21:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.545+0000 7f8c927fc700 1 -- 172.21.15.148:0/1903735435 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+32308 (secure 0 0 0) 0x7f8c94053f60 con 0x7f8c9c158e50 2023-10-10T20:21:10.548 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:10.548 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:21:10.548 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 52, 2023-10-10T20:21:10.548 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:21:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:21:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:21:08.536665+0000", 2023-10-10T20:21:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:21:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:21:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:21:10.550 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:21:10.550 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:21:10.550 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:21:10.550 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:21:10.550 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:21:10.551 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:21:10.551 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.551 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:21:10.551 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:21:10.551 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:21:10.552 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:21:10.552 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:21:10.552 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 7, 2023-10-10T20:21:10.552 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:21:10.554 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:21:10.554 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:21:10.554 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.554 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.555 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.555 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.555 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.555 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.555 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.556 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.556 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.556 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.556 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.556 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.557 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:21:10.557 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:21:10.557 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:21:10.557 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:21:10.558 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:21:10.558 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.558 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.558 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.559 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.559 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.559 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.559 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.560 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.561 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.561 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.561 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.561 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.562 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.564 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.564 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.564 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.564 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.564 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.565 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.565 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.565 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.565 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.566 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.566 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.566 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.566 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.566 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.567 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.567 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.567 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.567 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.568 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:21:10.568 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:21:10.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.568 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.569 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:21:10.569 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.569 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 6, 2023-10-10T20:21:10.570 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:21:10.570 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:37.022631+0000", 2023-10-10T20:21:10.570 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.570 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.571 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.572 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.572 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:21:10.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:21:10.573 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:21:10.573 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:21:10.573 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:21:10.573 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.574 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.574 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.574 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.574 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.575 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.575 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.575 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.575 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.576 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.576 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.576 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.576 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.576 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.577 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.577 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.577 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.577 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.577 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.578 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.578 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.578 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.578 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.579 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.579 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.580 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.580 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.580 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.580 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.580 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.581 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.581 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.581 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.581 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.582 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.582 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.582 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.582 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.582 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.583 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.583 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.583 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.583 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.583 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:21:10.584 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:21:10.584 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:21:10.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.584 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.584 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:10.585 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:21:10.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 7, 2023-10-10T20:21:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:21:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:20:38.151667+0000", 2023-10-10T20:21:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:21:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:21:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:21:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:21:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:21:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:21:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:21:10.588 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:21:10.588 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:21:10.588 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:21:10.588 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:21:10.588 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:21:10.589 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:21:10.589 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:21:10.589 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:21:10.589 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:21:10.589 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:21:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:21:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:21:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:21:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:21:10.590 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:21:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:21:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:21:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "52", 2023-10-10T20:21:10.591 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:21:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:21:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:21:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:21:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:21:10.592 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:21:10.593 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:21:10.593 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:21:10.593 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:21:10.593 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:21:10.593 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:21:10.594 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:21:10.594 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:21:10.594 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:21:10.594 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:21:10.595 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:21:10.595 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:21:10.595 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:21:10.595 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:21:10.595 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:21:10.596 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:21:10.596 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:21:10.596 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:21:10.596 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:21:10.596 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:21:10.597 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:21:10.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.597 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:21:10.597 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:21:10.597 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:21:10.598 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:21:10.598 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:21:10.598 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:21:10.598 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:21:10.599 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:21:10.599 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:21:10.599 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:21:10.599 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:21:10.599 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:21:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:21:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:21:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:21:10.600 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:21:10.601 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:21:10.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.601 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.602 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:21:10.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.602 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:10.602 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.602 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.603 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.603 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.603 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.603 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.604 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:21:10.606 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:21:10.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.607 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.607 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.608 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:21:10.609 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.609 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:21:10.610 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.610 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.610 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.610 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:21:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:21:10.612 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.613 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.613 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.614 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:21:10.615 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:21:10.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:21:10.616 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.617 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:21:10.617 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:21:10.617 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:21:10.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:21:10.618 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.618 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.618 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.618 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.619 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:10.619 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.619 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.620 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.620 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.620 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.620 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.620 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.621 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.621 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.621 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.621 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.622 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.622 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:21:10.623 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.623 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.623 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:21:10.624 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.624 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.624 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.625 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.625 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:21:10.626 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.626 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.626 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:21:10.627 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.627 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.627 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.629 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:21:10.629 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.629 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.631 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:21:10.631 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.632 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.632 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:21:10.632 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:21:10.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.633 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:21:10.633 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:21:10.633 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:21:10.633 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:21:10.634 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.634 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.634 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.635 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:10.635 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.636 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.636 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.636 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.636 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.637 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.637 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.637 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.637 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.638 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.638 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.638 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.638 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.639 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.639 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:21:10.639 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.640 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.640 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:21:10.640 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.641 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.641 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.641 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.641 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.642 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.642 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:21:10.642 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.643 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.644 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.645 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:21:10.645 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:21:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.647 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.647 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:21:10.648 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.648 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.648 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:21:10.649 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:21:10.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.649 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.650 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:21:10.650 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:21:10.650 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:21:10.650 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:21:10.650 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.651 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.651 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.651 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.652 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.653 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.653 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.653 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.653 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.653 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.654 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.654 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.654 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.654 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.655 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.655 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:21:10.655 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:21:10.656 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.657 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.657 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.658 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.658 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:21:10.658 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.659 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.659 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:21:10.659 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.659 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.660 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.660 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.661 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:21:10.661 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.662 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:21:10.662 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.662 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.662 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.663 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.663 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.663 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:21:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.665 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:21:10.666 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:21:10.666 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:21:10.666 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:21:10.666 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.666 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.667 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:10.668 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.668 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.668 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.668 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.668 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.669 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.669 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.669 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.669 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.669 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.670 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.670 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.670 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.670 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.671 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:21:10.672 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.672 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.673 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.673 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.673 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.673 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:21:10.674 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.674 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:21:10.675 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.675 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.675 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:21:10.676 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:21:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.678 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.678 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.678 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.679 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:21:10.679 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.680 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.680 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:21:10.680 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:21:10.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.681 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.681 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:21:10.681 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:21:10.682 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:21:10.682 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:21:10.682 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.682 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.682 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.683 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.683 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:10.683 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.684 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.684 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.684 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.684 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.684 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.685 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.686 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.686 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.686 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.686 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:21:10.686 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.687 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.687 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:21:10.687 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.688 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.688 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.688 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.688 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.689 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.689 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.689 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:21:10.689 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.690 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.690 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:21:10.690 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.690 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.692 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:21:10.692 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.692 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.692 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.693 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.694 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.694 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:21:10.694 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.695 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.695 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:21:10.695 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:21:10.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.696 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.696 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:21:10.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:21:10.697 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:21:10.697 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:21:10.697 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.697 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.697 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.698 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.698 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:10.698 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.699 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.700 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.701 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.701 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.701 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:21:10.701 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.702 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.702 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:21:10.702 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.703 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.703 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.703 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.703 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.704 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.704 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.704 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:21:10.704 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.706 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:21:10.707 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.707 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.707 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.707 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.707 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:21:10.708 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.708 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.708 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.708 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:21:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:21:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:21:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:21:10.712 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.712 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.712 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.712 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:21:10.713 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:21:10.714 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:21:10.714 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:21:10.714 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:21:10.714 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:21:10.714 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 43, 2023-10-10T20:21:10.715 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:21:10.715 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:21:10.715 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:21:10.715 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.716 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.716 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:21:10.717 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.718 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.718 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:21:10.718 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.719 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.719 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.719 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:21:10.719 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.722 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:21:10.722 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.722 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.722 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:21:10.723 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:21:10.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.725 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:21:10.726 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:21:10.726 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:21:10.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.727 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:21:10.727 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:21:10.727 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:21:10.727 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:21:10.727 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.728 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:21:10.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.729 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:21:10.729 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.729 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.729 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.730 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.730 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.730 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:21:10.730 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.731 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:21:10.731 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.731 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.731 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.733 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:21:10.733 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.733 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.733 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.733 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.734 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.734 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:21:10.734 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.735 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.736 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:21:10.736 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.736 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:21:10.737 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.737 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.737 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.737 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.737 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.738 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:21:10.738 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.738 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:21:10.739 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.739 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.739 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.739 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.739 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:21:10.740 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.741 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.741 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.741 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.741 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.741 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:21:10.742 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:10.742 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:21:10.742 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:21:10.743 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:21:10.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.744 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:21:10.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:21:10.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:21:10.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:21:10.745 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:21:10.746 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:21:10.746 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:21:10.746 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:21:10.746 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:21:10.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:10.747 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:21:10.747 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:21:10.747 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:21:10.747 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:21:10.748 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:21:10.749 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:21:10.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:10.749 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:21:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 msgr2=0x7f8c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 0x7f8c800693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006fd0 tx=0x7f8c98008040 comp rx=0 tx=0).stop 2023-10-10T20:21:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 msgr2=0x7f8c9c143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097c0 tx=0x7f8c94019000 comp rx=0 tx=0).stop 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 shutdown_connections 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c80066f30 0x7f8c800693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c9c158e50 0x7f8c9c143380 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c152010 0x7f8c9c0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 --2- 172.21.15.148:0/1903735435 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c9c1523e0 0x7f8c9c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 >> 172.21.15.148:0/1903735435 conn(0x7f8c9c069870 msgr2=0x7f8c9c151060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 shutdown_connections 2023-10-10T20:21:10.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.547+0000 7f8ca4701700 1 -- 172.21.15.148:0/1903735435 wait complete. 2023-10-10T20:21:10.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:10.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.868+0000 7f4e64978700 1 Processor -- start 2023-10-10T20:21:10.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.868+0000 7f4e64978700 1 -- start start 2023-10-10T20:21:10.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e64978700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e6015a620 0x7f4e6015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e64978700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6015cbb0 con 0x7f4e6015d580 2023-10-10T20:21:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e64978700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6015bfa0 con 0x7f4e60157870 2023-10-10T20:21:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e64978700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6015bdb0 con 0x7f4e6015a620 2023-10-10T20:21:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e55d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e6015a620 0x7f4e6015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e5ed9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4e6015d580 0x7f4e60158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44544/0 (socket says 172.21.15.148:44544) 2023-10-10T20:21:10.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e5ed9d700 1 -- 172.21.15.148:0/2603707616 learned_addr learned my addr 172.21.15.148:0/2603707616 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:10.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.869+0000 7f4e55d9b700 1 -- 172.21.15.148:0/2603707616 >> v1:172.21.15.161:6789/0 conn(0x7f4e60157870 legacy=0x7f4e6015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e55d9b700 1 -- 172.21.15.148:0/2603707616 >> v1:172.21.15.148:6789/0 conn(0x7f4e6015d580 legacy=0x7f4e60158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e55d9b700 1 -- 172.21.15.148:0/2603707616 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6015c280 con 0x7f4e6015a620 2023-10-10T20:21:10.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e5dd9b700 1 -- 172.21.15.148:0/2603707616 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355582193 0 0) 0x7f4e6015cbb0 con 0x7f4e6015d580 2023-10-10T20:21:10.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e55d9b700 1 --2- 172.21.15.148:0/2603707616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015a620 0x7f4e6015d040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098e0 tx=0x7f4e480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968d564458026346 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e5dd9b700 1 -- 172.21.15.148:0/2603707616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48009e80 con 0x7f4e6015a620 2023-10-10T20:21:10.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e5dd9b700 1 -- 172.21.15.148:0/2603707616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4800b860 con 0x7f4e6015a620 2023-10-10T20:21:10.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e5dd9b700 1 -- 172.21.15.148:0/2603707616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800bbe0 con 0x7f4e6015a620 2023-10-10T20:21:10.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 -- 172.21.15.148:0/2603707616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015a620 msgr2=0x7f4e6015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 --2- 172.21.15.148:0/2603707616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015a620 0x7f4e6015d040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098e0 tx=0x7f4e480091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 -- 172.21.15.148:0/2603707616 shutdown_connections 2023-10-10T20:21:10.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 --2- 172.21.15.148:0/2603707616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015a620 0x7f4e6015d040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 -- 172.21.15.148:0/2603707616 >> 172.21.15.148:0/2603707616 conn(0x7f4e60068ce0 msgr2=0x7f4e600690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:10.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 -- 172.21.15.148:0/2603707616 shutdown_connections 2023-10-10T20:21:10.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.870+0000 7f4e64978700 1 -- 172.21.15.148:0/2603707616 wait complete. 2023-10-10T20:21:10.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 Processor -- start 2023-10-10T20:21:10.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 -- start start 2023-10-10T20:21:10.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e60157870 0x7f4e6014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e6015a620 0x7f4e600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600d09c0 con 0x7f4e6015d580 2023-10-10T20:21:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cfdf0 con 0x7f4e60157870 2023-10-10T20:21:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e64978700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600d0090 con 0x7f4e6015a620 2023-10-10T20:21:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e5ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.872+0000 7f4e5ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58872/0 (socket says 172.21.15.148:58872) 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 -- 172.21.15.148:0/3817545839 learned_addr learned my addr 172.21.15.148:0/3817545839 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 -- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e6015a620 msgr2=0x7f4e600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e55d9b700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e6015a620 0x7f4e600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5e59c700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e60157870 0x7f4e6014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e6015a620 0x7f4e600d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 -- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e60157870 msgr2=0x7f4e6014ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e60157870 0x7f4e6014ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 -- 172.21.15.148:0/3817545839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48009420 con 0x7f4e6015d580 2023-10-10T20:21:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e5ed9d700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800cbc0 tx=0x7f4e58001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800a9d0 con 0x7f4e6015d580 2023-10-10T20:21:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5800ab30 con 0x7f4e6015d580 2023-10-10T20:21:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e580070f0 con 0x7f4e6015d580 2023-10-10T20:21:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e600d0370 con 0x7f4e6015d580 2023-10-10T20:21:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.873+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e600b2f10 con 0x7f4e6015d580 2023-10-10T20:21:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.874+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e600d0b30 con 0x7f4e6015d580 2023-10-10T20:21:10.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.874+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e58007520 con 0x7f4e6015d580 2023-10-10T20:21:10.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.875+0000 7f4e56ffd700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 0x7f4e400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:10.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.875+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e58084970 con 0x7f4e6015d580 2023-10-10T20:21:10.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.875+0000 7f4e5e59c700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 0x7f4e400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:10.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.875+0000 7f4e5e59c700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 0x7f4e400693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001760 tx=0x7f4e50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:10.879+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e58051c60 con 0x7f4e6015d580 2023-10-10T20:21:11.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.103+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e6004c3c0 con 0x7f4e6015d580 2023-10-10T20:21:11.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.104+0000 7f4e56ffd700 1 -- 172.21.15.148:0/3817545839 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f4e58055280 con 0x7f4e6015d580 2023-10-10T20:21:11.106 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:11.106 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:21:11.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.105+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 msgr2=0x7f4e400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.105+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 0x7f4e400693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e50001760 tx=0x7f4e50009040 comp rx=0 tx=0).stop 2023-10-10T20:21:11.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 msgr2=0x7f4e6014b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800cbc0 tx=0x7f4e58001980 comp rx=0 tx=0).stop 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 shutdown_connections 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4e40066f30 0x7f4e400693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e6015d580 0x7f4e6014b670 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e60157870 0x7f4e6014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 --2- 172.21.15.148:0/3817545839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4e6015a620 0x7f4e600d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 >> 172.21.15.148:0/3817545839 conn(0x7f4e60068ce0 msgr2=0x7f4e6014c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:11.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 shutdown_connections 2023-10-10T20:21:11.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.106+0000 7f4e64978700 1 -- 172.21.15.148:0/3817545839 wait complete. 2023-10-10T20:21:11.110 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:21:11.120 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:21:11.264 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":5367,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.433940150000002,"mdsmap_epoch":24,"osdmap_epoch":52,"osdmap_epoch_barrier":49,"uptime":37.401188883000003} 2023-10-10T20:21:11.265 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 5367, "mdsmap_epoch": 24, "osdmap_epoch": 52, "osdmap_epoch_barrier": 49, "rank_uptime": 31.43394015, "state": "up:active", "uptime": 37.401188883, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:21:11.265 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:21:11.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:21:11.266 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-10T20:21:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:21:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:21:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.263432 s, 199 MB/s 2023-10-10T20:21:11.548 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:21:11.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:21:11.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:21:11.633 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:21:11.634 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:21:11.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:21:11.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:21:11.703 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:21:11.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:11.704 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:21:11.719 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:21:11.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:21:11.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:21:11.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:21:11.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:11.758 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:21:11.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:11.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.925+0000 7f4fa123a700 1 Processor -- start 2023-10-10T20:21:11.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.925+0000 7f4fa123a700 1 -- start start 2023-10-10T20:21:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4fa123a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4fa123a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4fa123a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c15db80 con 0x7f4f9c159e10 2023-10-10T20:21:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4fa123a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c15e800 con 0x7f4f9c0b3430 2023-10-10T20:21:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4fa123a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c15de60 con 0x7f4f9c15a1e0 2023-10-10T20:21:11.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58878/0 (socket says 172.21.15.148:58878) 2023-10-10T20:21:11.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 -- 172.21.15.148:0/1887187574 learned_addr learned my addr 172.21.15.148:0/1887187574 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:11.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9ad9d700 1 --1- 172.21.15.148:0/1887187574 >> v1:172.21.15.161:6789/0 conn(0x7f4f9c159e10 0x7f4f9c0b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:56526/0 (socket says 172.21.15.148:56526) 2023-10-10T20:21:11.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9259c700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 -- 172.21.15.148:0/1887187574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 msgr2=0x7f4f9c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:11.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9a59c700 1 -- 172.21.15.148:0/1887187574 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901079725 0 0) 0x7f4f9c15db80 con 0x7f4f9c159e10 2023-10-10T20:21:11.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 -- 172.21.15.148:0/1887187574 >> v1:172.21.15.161:6789/0 conn(0x7f4f9c159e10 legacy=0x7f4f9c0b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.926+0000 7f4f9b59e700 1 -- 172.21.15.148:0/1887187574 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c15e140 con 0x7f4f9c0b3430 2023-10-10T20:21:11.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4f9b59e700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009f20 tx=0x7f4f94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88eafdb1d3c181cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:11.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4f9a59c700 1 -- 172.21.15.148:0/1887187574 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400e070 con 0x7f4f9c0b3430 2023-10-10T20:21:11.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4f9a59c700 1 -- 172.21.15.148:0/1887187574 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9400b9d0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4f9a59c700 1 -- 172.21.15.148:0/1887187574 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940092e0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 -- 172.21.15.148:0/1887187574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 msgr2=0x7f4f9c0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009f20 tx=0x7f4f94009840 comp rx=0 tx=0).stop 2023-10-10T20:21:11.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 -- 172.21.15.148:0/1887187574 shutdown_connections 2023-10-10T20:21:11.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c0cdc80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 --2- 172.21.15.148:0/1887187574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 -- 172.21.15.148:0/1887187574 >> 172.21.15.148:0/1887187574 conn(0x7f4f9c067660 msgr2=0x7f4f9c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:11.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 -- 172.21.15.148:0/1887187574 shutdown_connections 2023-10-10T20:21:11.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.927+0000 7f4fa123a700 1 -- 172.21.15.148:0/1887187574 wait complete. 2023-10-10T20:21:11.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.928+0000 7f4fa123a700 1 Processor -- start 2023-10-10T20:21:11.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 -- start start 2023-10-10T20:21:11.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f9c159e10 0x7f4f9c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c16e350 con 0x7f4f9c0b3430 2023-10-10T20:21:11.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0ceff0 con 0x7f4f9c159e10 2023-10-10T20:21:11.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4fa123a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0c7360 con 0x7f4f9c15a1e0 2023-10-10T20:21:11.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58894/0 (socket says 172.21.15.148:58894) 2023-10-10T20:21:11.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4f9ad9d700 1 -- 172.21.15.148:0/897893103 learned_addr learned my addr 172.21.15.148:0/897893103 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:11.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4f9b59e700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.929+0000 7f4f9259c700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f9c159e10 0x7f4f9c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 -- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 msgr2=0x7f4f9c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 -- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f9c159e10 msgr2=0x7f4f9c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:11.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f9c159e10 0x7f4f9c0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:11.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 -- 172.21.15.148:0/897893103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009ae0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f9ad9d700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c7e0 tx=0x7f4f8c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:11.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c005a70 con 0x7f4f9c0b3430 2023-10-10T20:21:11.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c005bd0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c16e070 con 0x7f4f9c0b3430 2023-10-10T20:21:11.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0070d0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.930+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c0cec80 con 0x7f4f9c0b3430 2023-10-10T20:21:11.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.931+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c04c3c0 con 0x7f4f9c0b3430 2023-10-10T20:21:11.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.931+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f8c007440 con 0x7f4f9c0b3430 2023-10-10T20:21:11.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.932+0000 7f4f937fe700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:11.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.932+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4f8c083a30 con 0x7f4f9c0b3430 2023-10-10T20:21:11.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.932+0000 7f4f9259c700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:11.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.932+0000 7f4f9259c700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 0x7f4f7c0693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0c7760 tx=0x7f4f840091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:11.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:11.936+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f8c050c60 con 0x7f4f9c0b3430 2023-10-10T20:21:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.159+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f9c15e420 con 0x7f4f9c0b3430 2023-10-10T20:21:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.159+0000 7f4f937fe700 1 -- 172.21.15.148:0/897893103 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f4f8c017070 con 0x7f4f9c0b3430 2023-10-10T20:21:12.162 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:12.162 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:21:12.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 msgr2=0x7f4f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 0x7f4f7c0693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0c7760 tx=0x7f4f840091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 msgr2=0x7f4f9c156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c7e0 tx=0x7f4f8c007e10 comp rx=0 tx=0).stop 2023-10-10T20:21:12.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 shutdown_connections 2023-10-10T20:21:12.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f9c15a1e0 0x7f4f9c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f9c0b3430 0x7f4f9c156100 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 --2- 172.21.15.148:0/897893103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f9c159e10 0x7f4f9c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 >> 172.21.15.148:0/897893103 conn(0x7f4f9c067660 msgr2=0x7f4f9c0cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 shutdown_connections 2023-10-10T20:21:12.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.161+0000 7f4fa123a700 1 -- 172.21.15.148:0/897893103 wait complete. 2023-10-10T20:21:12.170 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:21:12.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:12.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.287+0000 7f7fbabd2700 1 Processor -- start 2023-10-10T20:21:12.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.287+0000 7f7fbabd2700 1 -- start start 2023-10-10T20:21:12.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fbabd2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7fb40af990 0x7f7fb415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fbabd2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb4159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fbabd2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb415ddf0 con 0x7f7fb40afd60 2023-10-10T20:21:12.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fbabd2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40cf060 con 0x7f7fb40af990 2023-10-10T20:21:12.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fbabd2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b07f0 con 0x7f7fb40cba20 2023-10-10T20:21:12.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fb9bd0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7fb40af990 0x7f7fb415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fba3d1700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7fb40afd60 0x7f7fb406a320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44548/0 (socket says 172.21.15.148:44548) 2023-10-10T20:21:12.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.288+0000 7f7fba3d1700 1 -- 172.21.15.148:0/251468220 learned_addr learned my addr 172.21.15.148:0/251468220 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:12.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb93cf700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb4159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/251468220 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 msgr2=0x7f7fb4159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:12.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb9bd0700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb4159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/251468220 >> v1:172.21.15.148:6789/0 conn(0x7f7fb40afd60 legacy=0x7f7fb406a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb8bce700 1 -- 172.21.15.148:0/251468220 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986585608 0 0) 0x7f7fb415ddf0 con 0x7f7fb40afd60 2023-10-10T20:21:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/251468220 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40cf2d0 con 0x7f7fb40af990 2023-10-10T20:21:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb9bd0700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb415d8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00098e0 tx=0x7f7fb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dd0161df9845da server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb8bce700 1 -- 172.21.15.148:0/251468220 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000b640 con 0x7f7fb40af990 2023-10-10T20:21:12.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb8bce700 1 -- 172.21.15.148:0/251468220 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000b7a0 con 0x7f7fb40af990 2023-10-10T20:21:12.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.289+0000 7f7fb8bce700 1 -- 172.21.15.148:0/251468220 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000bb40 con 0x7f7fb40af990 2023-10-10T20:21:12.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 -- 172.21.15.148:0/251468220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 msgr2=0x7f7fb415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb415d8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00098e0 tx=0x7f7fb00091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 -- 172.21.15.148:0/251468220 shutdown_connections 2023-10-10T20:21:12.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb415d8b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/251468220 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb4159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 -- 172.21.15.148:0/251468220 >> 172.21.15.148:0/251468220 conn(0x7f7fb4068bf0 msgr2=0x7f7fb4068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 -- 172.21.15.148:0/251468220 shutdown_connections 2023-10-10T20:21:12.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.290+0000 7f7fbabd2700 1 -- 172.21.15.148:0/251468220 wait complete. 2023-10-10T20:21:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.291+0000 7f7fbabd2700 1 Processor -- start 2023-10-10T20:21:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.291+0000 7f7fbabd2700 1 -- start start 2023-10-10T20:21:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb40afd60 0x7f7fb4159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb416a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b6590 con 0x7f7fb40af990 2023-10-10T20:21:12.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb4159540 con 0x7f7fb40afd60 2023-10-10T20:21:12.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb4159b00 con 0x7f7fb40cba20 2023-10-10T20:21:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58912/0 (socket says 172.21.15.148:58912) 2023-10-10T20:21:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/3424084507 learned_addr learned my addr 172.21.15.148:0/3424084507 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb93cf700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb40afd60 0x7f7fb4159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fba3d1700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb416a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 msgr2=0x7f7fb416a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb416a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb40afd60 msgr2=0x7f7fb4159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb40afd60 0x7f7fb4159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 -- 172.21.15.148:0/3424084507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb0009420 con 0x7f7fb40af990 2023-10-10T20:21:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fb9bd0700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009f80 tx=0x7f7fb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000bb20 con 0x7f7fb40af990 2023-10-10T20:21:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb0000c90 con 0x7f7fb40af990 2023-10-10T20:21:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0011630 con 0x7f7fb40af990 2023-10-10T20:21:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.292+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb415a4d0 con 0x7f7fb40af990 2023-10-10T20:21:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.293+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb416b070 con 0x7f7fb40af990 2023-10-10T20:21:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.293+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb4089500 con 0x7f7fb40af990 2023-10-10T20:21:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.294+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fb00119a0 con 0x7f7fb40af990 2023-10-10T20:21:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.294+0000 7f7fa9ffb700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.294+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f7fb00858e0 con 0x7f7fb40af990 2023-10-10T20:21:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.294+0000 7f7fb93cf700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.295+0000 7f7fb93cf700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 0x7f7f980693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40cb600 tx=0x7f7fa40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.299+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fb0052b10 con 0x7f7fb40af990 2023-10-10T20:21:12.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.481+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fb415e020 con 0x7f7fb40af990 2023-10-10T20:21:12.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.481+0000 7f7fa9ffb700 1 -- 172.21.15.148:0/3424084507 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15494 (secure 0 0 0) 0x7f7fb0056130 con 0x7f7fb40af990 2023-10-10T20:21:12.484 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:12.484 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:11.555553+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:12.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 msgr2=0x7f7f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 0x7f7f980693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40cb600 tx=0x7f7fa40091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 msgr2=0x7f7fb40b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009f80 tx=0x7f7fb0017000 comp rx=0 tx=0).stop 2023-10-10T20:21:12.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 shutdown_connections 2023-10-10T20:21:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.483+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fb40cba20 0x7f7fb416a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fb40af990 0x7f7fb40b6050 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 --2- 172.21.15.148:0/3424084507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb40afd60 0x7f7fb4159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 >> 172.21.15.148:0/3424084507 conn(0x7f7fb4068bf0 msgr2=0x7f7fb40b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 shutdown_connections 2023-10-10T20:21:12.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.484+0000 7f7fbabd2700 1 -- 172.21.15.148:0/3424084507 wait complete. 2023-10-10T20:21:12.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:20:37.022631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:20:38.151667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:21:12.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:21:12.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.616+0000 7f6c2ebc4700 1 Processor -- start 2023-10-10T20:21:12.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.616+0000 7f6c2ebc4700 1 -- start start 2023-10-10T20:21:12.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c28159de0 0x7f6c280cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c280cc680 0x7f6c2815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 0x7f6c280b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280ce160 con 0x7f6c280cc680 2023-10-10T20:21:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c280cb440 con 0x7f6c2815df70 2023-10-10T20:21:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c2ebc4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2815e3f0 con 0x7f6c28159de0 2023-10-10T20:21:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c1ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c280cc680 0x7f6c2815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c1ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c280cc680 0x7f6c2815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34470/0 (socket says 172.21.15.148:34470) 2023-10-10T20:21:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.617+0000 7f6c1ffff700 1 -- 172.21.15.148:0/3474881635 learned_addr learned my addr 172.21.15.148:0/3474881635 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2c960700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c28159de0 0x7f6c280cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2d161700 1 --2- 172.21.15.148:0/3474881635 >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 0x7f6c280b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 -- 172.21.15.148:0/3474881635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c28159de0 msgr2=0x7f6c280cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c28159de0 0x7f6c280cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 -- 172.21.15.148:0/3474881635 >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 msgr2=0x7f6c280b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 --2- 172.21.15.148:0/3474881635 >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 0x7f6c280b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 -- 172.21.15.148:0/3474881635 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280cd790 con 0x7f6c280cc680 2023-10-10T20:21:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2d161700 1 --2- 172.21.15.148:0/3474881635 >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 0x7f6c280b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1ffff700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c280cc680 0x7f6c2815da30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6c280b0810 tx=0x7f6c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c1cd891e3d22a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1f7fe700 1 -- 172.21.15.148:0/3474881635 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800b820 con 0x7f6c280cc680 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1f7fe700 1 -- 172.21.15.148:0/3474881635 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1800b980 con 0x7f6c280cc680 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c1f7fe700 1 -- 172.21.15.148:0/3474881635 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800bd20 con 0x7f6c280cc680 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/3474881635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c280cc680 msgr2=0x7f6c2815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c280cc680 0x7f6c2815da30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6c280b0810 tx=0x7f6c18009250 comp rx=0 tx=0).stop 2023-10-10T20:21:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/3474881635 shutdown_connections 2023-10-10T20:21:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/3474881635 >> v2:172.21.15.161:3300/0 conn(0x7f6c2815df70 0x7f6c280b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c28159de0 0x7f6c280cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/3474881635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c280cc680 0x7f6c2815da30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/3474881635 >> 172.21.15.148:0/3474881635 conn(0x7f6c280cb5c0 msgr2=0x7f6c280cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/3474881635 shutdown_connections 2023-10-10T20:21:12.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.618+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/3474881635 wait complete. 2023-10-10T20:21:12.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.619+0000 7f6c2ebc4700 1 Processor -- start 2023-10-10T20:21:12.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 -- start start 2023-10-10T20:21:12.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c280cc680 0x7f6c2815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28159de0 0x7f6c28069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280c2870 con 0x7f6c2815df70 2023-10-10T20:21:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28068410 con 0x7f6c28159de0 2023-10-10T20:21:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.620+0000 7f6c2ebc4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280686f0 con 0x7f6c280cc680 2023-10-10T20:21:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34476/0 (socket says 172.21.15.148:34476) 2023-10-10T20:21:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 -- 172.21.15.148:0/294125564 learned_addr learned my addr 172.21.15.148:0/294125564 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2c960700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c280cc680 0x7f6c2815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c1ffff700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28159de0 0x7f6c28069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 -- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c280cc680 msgr2=0x7f6c2815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c280cc680 0x7f6c2815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 -- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28159de0 msgr2=0x7f6c28069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28159de0 0x7f6c28069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 -- 172.21.15.148:0/294125564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180094f0 con 0x7f6c2815df70 2023-10-10T20:21:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2d161700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c850 tx=0x7f6c20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20011600 con 0x7f6c2815df70 2023-10-10T20:21:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20011760 con 0x7f6c2815df70 2023-10-10T20:21:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280c2220 con 0x7f6c2815df70 2023-10-10T20:21:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20011b20 con 0x7f6c2815df70 2023-10-10T20:21:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.621+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2815caf0 con 0x7f6c2815df70 2023-10-10T20:21:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.622+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c280c29e0 con 0x7f6c2815df70 2023-10-10T20:21:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.623+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c20018350 con 0x7f6c2815df70 2023-10-10T20:21:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.623+0000 7f6c1cff9700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.623+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6c20084310 con 0x7f6c2815df70 2023-10-10T20:21:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.623+0000 7f6c2c960700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.623+0000 7f6c2c960700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 0x7f6c140693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009fd0 tx=0x7f6c24009880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.627+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c20054fd0 con 0x7f6c2815df70 2023-10-10T20:21:12.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.812+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c2804c3c0 con 0x7f6c2815df70 2023-10-10T20:21:12.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.812+0000 7f6c1cff9700 1 -- 172.21.15.148:0/294125564 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c20054b60 con 0x7f6c2815df70 2023-10-10T20:21:12.814 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:21:12.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 msgr2=0x7f6c140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 0x7f6c140693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009fd0 tx=0x7f6c24009880 comp rx=0 tx=0).stop 2023-10-10T20:21:12.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 msgr2=0x7f6c281570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c850 tx=0x7f6c20008040 comp rx=0 tx=0).stop 2023-10-10T20:21:12.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 shutdown_connections 2023-10-10T20:21:12.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6c14066f30 0x7f6c140693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c2815df70 0x7f6c281570a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c280cc680 0x7f6c2815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 --2- 172.21.15.148:0/294125564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28159de0 0x7f6c28069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 >> 172.21.15.148:0/294125564 conn(0x7f6c280cb5c0 msgr2=0x7f6c280b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 shutdown_connections 2023-10-10T20:21:12.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.814+0000 7f6c2ebc4700 1 -- 172.21.15.148:0/294125564 wait complete. 2023-10-10T20:21:12.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:21:12.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.944+0000 7ff9a07ed700 1 Processor -- start 2023-10-10T20:21:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.944+0000 7ff9a07ed700 1 -- start start 2023-10-10T20:21:12.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff9a07ed700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 0x7ff99806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff9a07ed700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff99806aae0 0x7ff9980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff9a07ed700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9980b0790 con 0x7ff998159ec0 2023-10-10T20:21:12.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff9a07ed700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99806af20 con 0x7ff99806aae0 2023-10-10T20:21:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff9a07ed700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9980ce1f0 con 0x7ff99815a290 2023-10-10T20:21:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff99ed8a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff99806aae0 0x7ff9980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff99e589700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff998159ec0 0x7ff9980af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37276/0 (socket says 172.21.15.148:37276) 2023-10-10T20:21:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff99e589700 1 -- 172.21.15.148:0/3788503736 learned_addr learned my addr 172.21.15.148:0/3788503736 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.945+0000 7ff99dd88700 1 --2- 172.21.15.148:0/3788503736 >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 0x7ff99806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3788503736 >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 msgr2=0x7ff99806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99ed8a700 1 --2- 172.21.15.148:0/3788503736 >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 0x7ff99806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3788503736 >> v1:172.21.15.148:6789/0 conn(0x7ff998159ec0 legacy=0x7ff9980af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3788503736 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980b0b20 con 0x7ff99806aae0 2023-10-10T20:21:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99dd88700 1 --2- 172.21.15.148:0/3788503736 >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 0x7ff99806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99d587700 1 -- 172.21.15.148:0/3788503736 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675648026 0 0) 0x7ff9980b0790 con 0x7ff998159ec0 2023-10-10T20:21:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99ed8a700 1 --2- 172.21.15.148:0/3788503736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99806aae0 0x7ff9980cf590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff998089910 tx=0x7ff990009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737da148abda772a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99d587700 1 -- 172.21.15.148:0/3788503736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000e070 con 0x7ff99806aae0 2023-10-10T20:21:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99d587700 1 -- 172.21.15.148:0/3788503736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99000bc30 con 0x7ff99806aae0 2023-10-10T20:21:12.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff99d587700 1 -- 172.21.15.148:0/3788503736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990012430 con 0x7ff99806aae0 2023-10-10T20:21:12.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3788503736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99806aae0 msgr2=0x7ff9980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3788503736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99806aae0 0x7ff9980cf590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff998089910 tx=0x7ff990009780 comp rx=0 tx=0).stop 2023-10-10T20:21:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3788503736 shutdown_connections 2023-10-10T20:21:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3788503736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99806aae0 0x7ff9980cf590 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3788503736 >> v2:172.21.15.161:3300/0 conn(0x7ff99815a290 0x7ff99806a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.946+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3788503736 >> 172.21.15.148:0/3788503736 conn(0x7ff9980d1880 msgr2=0x7ff9980cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.947+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3788503736 shutdown_connections 2023-10-10T20:21:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.947+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3788503736 wait complete. 2023-10-10T20:21:12.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 Processor -- start 2023-10-10T20:21:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 -- start start 2023-10-10T20:21:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff99806aae0 0x7ff99816a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff998159ec0 0x7ff998155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99816a6e0 con 0x7ff99815a290 2023-10-10T20:21:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980b4ee0 con 0x7ff998159ec0 2023-10-10T20:21:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff9a07ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9980b5780 con 0x7ff99806aae0 2023-10-10T20:21:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff99ed8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff99ed8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34486/0 (socket says 172.21.15.148:34486) 2023-10-10T20:21:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3839979606 learned_addr learned my addr 172.21.15.148:0/3839979606 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:12.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff99dd88700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff998159ec0 0x7ff998155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.948+0000 7ff99e589700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff99806aae0 0x7ff99816a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff99806aae0 msgr2=0x7ff99816a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff99806aae0 0x7ff99816a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff998159ec0 msgr2=0x7ff998155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:12.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff998159ec0 0x7ff998155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:12.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 -- 172.21.15.148:0/3839979606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff990009a20 con 0x7ff99815a290 2023-10-10T20:21:12.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff99ed8a700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff998071eb0 tx=0x7ff990019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900032f0 con 0x7ff99815a290 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff990027470 con 0x7ff99815a290 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff998155750 con 0x7ff99815a290 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990027810 con 0x7ff99815a290 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.949+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9980b51b0 con 0x7ff99815a290 2023-10-10T20:21:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.950+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9980b58f0 con 0x7ff99815a290 2023-10-10T20:21:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.950+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff990024070 con 0x7ff99815a290 2023-10-10T20:21:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.951+0000 7ff98e7fc700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 0x7ff97c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.951+0000 7ff99e589700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 0x7ff97c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.951+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff990086bd0 con 0x7ff99815a290 2023-10-10T20:21:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.951+0000 7ff99e589700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 0x7ff97c0693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff994007fd0 tx=0x7ff994007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:12.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:12.955+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff990053d60 con 0x7ff99815a290 2023-10-10T20:21:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.137+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff99804c3f0 con 0x7ff99815a290 2023-10-10T20:21:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.137+0000 7ff98e7fc700 1 -- 172.21.15.148:0/3839979606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff99002c0a0 con 0x7ff99815a290 2023-10-10T20:21:13.140 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:21:13.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 msgr2=0x7ff97c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 0x7ff97c0693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff994007fd0 tx=0x7ff994007d30 comp rx=0 tx=0).stop 2023-10-10T20:21:13.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 msgr2=0x7ff9980cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff998071eb0 tx=0x7ff990019000 comp rx=0 tx=0).stop 2023-10-10T20:21:13.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 shutdown_connections 2023-10-10T20:21:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff97c066f30 0x7ff97c0693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99815a290 0x7ff9980cab00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff99806aae0 0x7ff99816a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 --2- 172.21.15.148:0/3839979606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff998159ec0 0x7ff998155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.139+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 >> 172.21.15.148:0/3839979606 conn(0x7ff9980d1880 msgr2=0x7ff9980c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.140+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 shutdown_connections 2023-10-10T20:21:13.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.140+0000 7ff9a07ed700 1 -- 172.21.15.148:0/3839979606 wait complete. 2023-10-10T20:21:13.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:21:13.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.269+0000 7f8249cf8700 1 Processor -- start 2023-10-10T20:21:13.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.269+0000 7f8249cf8700 1 -- start start 2023-10-10T20:21:13.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8249cf8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f824415a290 0x7f824406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8249cf8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440b0790 con 0x7f824406aae0 2023-10-10T20:21:13.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8249cf8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824406af20 con 0x7f8244159ec0 2023-10-10T20:21:13.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8249cf8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440ce1f0 con 0x7f824415a290 2023-10-10T20:21:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8243fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f824406aae0 0x7f82440cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37278/0 (socket says 172.21.15.148:37278) 2023-10-10T20:21:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f823bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f824415a290 0x7f824406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8243fff700 1 -- 172.21.15.148:0/2575075475 learned_addr learned my addr 172.21.15.148:0/2575075475 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959241910 0 0) 0x7f82440b0790 con 0x7f824406aae0 2023-10-10T20:21:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8230003570 con 0x7f824406aae0 2023-10-10T20:21:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f823bfff700 1 -- 172.21.15.148:0/2575075475 >> v1:172.21.15.161:6789/0 conn(0x7f8244159ec0 legacy=0x7f82440af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037892254 0 0) 0x7f824406af20 con 0x7f8244159ec0 2023-10-10T20:21:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f823bfff700 1 -- 172.21.15.148:0/2575075475 >> v1:172.21.15.148:6789/0 conn(0x7f824406aae0 legacy=0x7f82440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.270+0000 7f823bfff700 1 -- 172.21.15.148:0/2575075475 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440b0b20 con 0x7f824415a290 2023-10-10T20:21:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f823bfff700 1 --2- 172.21.15.148:0/2575075475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f824415a290 0x7f824406a5a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f822c0098e0 tx=0x7f822c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134049d4765d6a53 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00b640 con 0x7f824415a290 2023-10-10T20:21:13.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00b7a0 con 0x7f824415a290 2023-10-10T20:21:13.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8242ffd700 1 -- 172.21.15.148:0/2575075475 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00bb40 con 0x7f824415a290 2023-10-10T20:21:13.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 -- 172.21.15.148:0/2575075475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f824415a290 msgr2=0x7f824406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 --2- 172.21.15.148:0/2575075475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f824415a290 0x7f824406a5a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f822c0098e0 tx=0x7f822c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 -- 172.21.15.148:0/2575075475 shutdown_connections 2023-10-10T20:21:13.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 --2- 172.21.15.148:0/2575075475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f824415a290 0x7f824406a5a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 -- 172.21.15.148:0/2575075475 >> 172.21.15.148:0/2575075475 conn(0x7f82440d1880 msgr2=0x7f82440cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 -- 172.21.15.148:0/2575075475 shutdown_connections 2023-10-10T20:21:13.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.271+0000 7f8249cf8700 1 -- 172.21.15.148:0/2575075475 wait complete. 2023-10-10T20:21:13.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.272+0000 7f8249cf8700 1 Processor -- start 2023-10-10T20:21:13.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 -- start start 2023-10-10T20:21:13.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f824406aae0 0x7f82440c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 0x7f824415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440b4ae0 con 0x7f8244159ec0 2023-10-10T20:21:13.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440b4dc0 con 0x7f824415a290 2023-10-10T20:21:13.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f8249cf8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82440ca070 con 0x7f824406aae0 2023-10-10T20:21:13.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f823bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f823bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34498/0 (socket says 172.21.15.148:34498) 2023-10-10T20:21:13.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.273+0000 7f823bfff700 1 -- 172.21.15.148:0/4073002033 learned_addr learned my addr 172.21.15.148:0/4073002033 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f8243fff700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 0x7f824415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f82437fe700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f824406aae0 0x7f82440c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 -- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f824406aae0 msgr2=0x7f82440c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f824406aae0 0x7f82440c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 -- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 msgr2=0x7f824415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 0x7f824415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 -- 172.21.15.148:0/4073002033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009420 con 0x7f8244159ec0 2023-10-10T20:21:13.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f8243fff700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 0x7f824415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:13.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823bfff700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f822c0093f0 tx=0x7f822c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c01f070 con 0x7f8244159ec0 2023-10-10T20:21:13.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c000c90 con 0x7f8244159ec0 2023-10-10T20:21:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82440ca350 con 0x7f8244159ec0 2023-10-10T20:21:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c025420 con 0x7f8244159ec0 2023-10-10T20:21:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.274+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82440c9d80 con 0x7f8244159ec0 2023-10-10T20:21:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.275+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82440b4f30 con 0x7f8244159ec0 2023-10-10T20:21:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.275+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f822c023070 con 0x7f8244159ec0 2023-10-10T20:21:13.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.276+0000 7f823b7fe700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 0x7f82240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.276+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f822c085920 con 0x7f8244159ec0 2023-10-10T20:21:13.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.276+0000 7f82437fe700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 0x7f82240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.276+0000 7f82437fe700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 0x7f82240693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82340011e0 tx=0x7f8234009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.280+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f822c052b50 con 0x7f8244159ec0 2023-10-10T20:21:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.463+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f824404c3c0 con 0x7f8244159ec0 2023-10-10T20:21:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.463+0000 7f823b7fe700 1 -- 172.21.15.148:0/4073002033 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f822c02a0a0 con 0x7f8244159ec0 2023-10-10T20:21:13.466 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:21:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 msgr2=0x7f82240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 0x7f82240693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82340011e0 tx=0x7f8234009040 comp rx=0 tx=0).stop 2023-10-10T20:21:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 msgr2=0x7f82441552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f822c0093f0 tx=0x7f822c017000 comp rx=0 tx=0).stop 2023-10-10T20:21:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 shutdown_connections 2023-10-10T20:21:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8224066f30 0x7f82240693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f824415a290 0x7f824415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f824406aae0 0x7f82440c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 --2- 172.21.15.148:0/4073002033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8244159ec0 0x7f82441552d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 >> 172.21.15.148:0/4073002033 conn(0x7f82440d1880 msgr2=0x7f82440b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.465+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 shutdown_connections 2023-10-10T20:21:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.466+0000 7f8249cf8700 1 -- 172.21.15.148:0/4073002033 wait complete. 2023-10-10T20:21:13.479 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:13.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 Processor -- start 2023-10-10T20:21:13.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 -- start start 2023-10-10T20:21:13.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c156ea0 0x7fbf5c0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5c06e2b0 0x7fbf5c06e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c0c42b0 con 0x7fbf5c157270 2023-10-10T20:21:13.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c06fac0 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.593+0000 7fbf62094700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06ec70 con 0x7fbf5c156ea0 2023-10-10T20:21:13.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5c06e2b0 0x7fbf5c06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5c06e2b0 0x7fbf5c06e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34510/0 (socket says 172.21.15.148:34510) 2023-10-10T20:21:13.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 -- 172.21.15.148:0/1317035028 learned_addr learned my addr 172.21.15.148:0/1317035028 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:13.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5b7fe700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c156ea0 0x7fbf5c0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 -- 172.21.15.148:0/1317035028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c156ea0 msgr2=0x7fbf5c0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:13.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c156ea0 0x7fbf5c0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 -- 172.21.15.148:0/1317035028 >> v1:172.21.15.161:6789/0 conn(0x7fbf5c157270 legacy=0x7fbf5c06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 -- 172.21.15.148:0/1317035028 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0c36a0 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5bfff700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c06e2b0 0x7fbf5c06e730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf540099b0 tx=0x7fbf54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ae51e04253e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5a7fc700 1 -- 172.21.15.148:0/1317035028 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54009ea0 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5a7fc700 1 -- 172.21.15.148:0/1317035028 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400b860 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.594+0000 7fbf5a7fc700 1 -- 172.21.15.148:0/1317035028 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400bc00 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 -- 172.21.15.148:0/1317035028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c06e2b0 msgr2=0x7fbf5c06e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c06e2b0 0x7fbf5c06e730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf540099b0 tx=0x7fbf54009250 comp rx=0 tx=0).stop 2023-10-10T20:21:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 -- 172.21.15.148:0/1317035028 shutdown_connections 2023-10-10T20:21:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c06e2b0 0x7fbf5c06e730 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 --2- 172.21.15.148:0/1317035028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c156ea0 0x7fbf5c0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 -- 172.21.15.148:0/1317035028 >> 172.21.15.148:0/1317035028 conn(0x7fbf5c0ca4f0 msgr2=0x7fbf5c0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 -- 172.21.15.148:0/1317035028 shutdown_connections 2023-10-10T20:21:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.595+0000 7fbf62094700 1 -- 172.21.15.148:0/1317035028 wait complete. 2023-10-10T20:21:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.596+0000 7fbf62094700 1 Processor -- start 2023-10-10T20:21:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.596+0000 7fbf62094700 1 -- start start 2023-10-10T20:21:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 0x7fbf5c16bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 0x7fbf5c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c14c7a0 con 0x7fbf5c157270 2023-10-10T20:21:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c14b430 con 0x7fbf5c156ea0 2023-10-10T20:21:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf62094700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c16c310 con 0x7fbf5c06e2b0 2023-10-10T20:21:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34522/0 (socket says 172.21.15.148:34522) 2023-10-10T20:21:13.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 -- 172.21.15.148:0/826453411 learned_addr learned my addr 172.21.15.148:0/826453411 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:13.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5affd700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 0x7fbf5c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5b7fe700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 0x7fbf5c16bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 -- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 msgr2=0x7fbf5c16bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 0x7fbf5c16bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 -- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 msgr2=0x7fbf5c14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 0x7fbf5c14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 -- 172.21.15.148:0/826453411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540094f0 con 0x7fbf5c157270 2023-10-10T20:21:13.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5affd700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 0x7fbf5c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:13.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5b7fe700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 0x7fbf5c16bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:13.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf5bfff700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c06d110 tx=0x7fbf54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.597+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54011630 con 0x7fbf5c157270 2023-10-10T20:21:13.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.598+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54011790 con 0x7fbf5c157270 2023-10-10T20:21:13.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.598+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54011b50 con 0x7fbf5c157270 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.598+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c16c5f0 con 0x7fbf5c157270 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.598+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c16cbb0 con 0x7fbf5c157270 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.599+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c06b110 con 0x7fbf5c157270 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.599+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf540253f0 con 0x7fbf5c157270 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.599+0000 7fbf437fe700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 0x7fbf480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.599+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbf54086100 con 0x7fbf5c157270 2023-10-10T20:21:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.599+0000 7fbf5b7fe700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 0x7fbf480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.600+0000 7fbf5b7fe700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 0x7fbf480693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0076c0 tx=0x7fbf4c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.604+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf54053330 con 0x7fbf5c157270 2023-10-10T20:21:13.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.828+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf5c04c3c0 con 0x7fbf5c157270 2023-10-10T20:21:13.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.829+0000 7fbf437fe700 1 -- 172.21.15.148:0/826453411 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fbf54056950 con 0x7fbf5c157270 2023-10-10T20:21:13.832 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:13.832 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:21:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 msgr2=0x7fbf480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 0x7fbf480693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0076c0 tx=0x7fbf4c00b040 comp rx=0 tx=0).stop 2023-10-10T20:21:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 msgr2=0x7fbf5c14aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c06d110 tx=0x7fbf54017000 comp rx=0 tx=0).stop 2023-10-10T20:21:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 shutdown_connections 2023-10-10T20:21:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf48066f30 0x7fbf480693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5c157270 0x7fbf5c14aef0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf5c06e2b0 0x7fbf5c16bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 --2- 172.21.15.148:0/826453411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf5c156ea0 0x7fbf5c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 >> 172.21.15.148:0/826453411 conn(0x7fbf5c0ca4f0 msgr2=0x7fbf5c14cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 shutdown_connections 2023-10-10T20:21:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.831+0000 7fbf62094700 1 -- 172.21.15.148:0/826453411 wait complete. 2023-10-10T20:21:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:21:13.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.957+0000 7f2afefbc700 1 Processor -- start 2023-10-10T20:21:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.957+0000 7f2afefbc700 1 -- start start 2023-10-10T20:21:13.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2af80cd6e0 0x7f2af80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cdab0 0x7f2af815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2af815dda0 0x7f2af815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af815e850 con 0x7f2af815dda0 2023-10-10T20:21:13.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80cd030 con 0x7f2af80cd6e0 2023-10-10T20:21:13.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afefbc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cb7c0 con 0x7f2af80cdab0 2023-10-10T20:21:13.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2af815dda0 0x7f2af815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2af815dda0 0x7f2af815cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34526/0 (socket says 172.21.15.148:34526) 2023-10-10T20:21:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 -- 172.21.15.148:0/1537819475 learned_addr learned my addr 172.21.15.148:0/1537819475 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afdfba700 1 --2- 172.21.15.148:0/1537819475 >> v2:172.21.15.161:3300/0 conn(0x7f2af80cd6e0 0x7f2af80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afd7b9700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cdab0 0x7f2af815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 -- 172.21.15.148:0/1537819475 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cdab0 msgr2=0x7f2af815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cdab0 0x7f2af815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 -- 172.21.15.148:0/1537819475 >> v2:172.21.15.161:3300/0 conn(0x7f2af80cd6e0 msgr2=0x7f2af80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/1537819475 >> v2:172.21.15.161:3300/0 conn(0x7f2af80cd6e0 0x7f2af80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 -- 172.21.15.148:0/1537819475 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af815e290 con 0x7f2af815dda0 2023-10-10T20:21:13.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.958+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af815cbd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2af806b730 tx=0x7f2af0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd792d73474d1406 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afcfb8700 1 -- 172.21.15.148:0/1537819475 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000e070 con 0x7f2af815dda0 2023-10-10T20:21:13.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afcfb8700 1 -- 172.21.15.148:0/1537819475 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000ba50 con 0x7f2af815dda0 2023-10-10T20:21:13.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afcfb8700 1 -- 172.21.15.148:0/1537819475 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00092e0 con 0x7f2af815dda0 2023-10-10T20:21:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 -- 172.21.15.148:0/1537819475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 msgr2=0x7f2af815cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af815cbd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2af806b730 tx=0x7f2af0009840 comp rx=0 tx=0).stop 2023-10-10T20:21:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 -- 172.21.15.148:0/1537819475 shutdown_connections 2023-10-10T20:21:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af815cbd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 --2- 172.21.15.148:0/1537819475 >> v2:172.21.15.161:3300/0 conn(0x7f2af80cd6e0 0x7f2af80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 --2- 172.21.15.148:0/1537819475 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cdab0 0x7f2af815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 -- 172.21.15.148:0/1537819475 >> 172.21.15.148:0/1537819475 conn(0x7f2af8067660 msgr2=0x7f2af8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 -- 172.21.15.148:0/1537819475 shutdown_connections 2023-10-10T20:21:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.959+0000 7f2afefbc700 1 -- 172.21.15.148:0/1537819475 wait complete. 2023-10-10T20:21:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.960+0000 7f2afefbc700 1 Processor -- start 2023-10-10T20:21:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.960+0000 7f2afefbc700 1 -- start start 2023-10-10T20:21:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 0x7f2af816a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af80cdab0 0x7f2af814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b68b0 con 0x7f2af815dda0 2023-10-10T20:21:13.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b3b40 con 0x7f2af80cdab0 2023-10-10T20:21:13.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afefbc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af816a7e0 con 0x7f2af80cd6e0 2023-10-10T20:21:13.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34542/0 (socket says 172.21.15.148:34542) 2023-10-10T20:21:13.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 -- 172.21.15.148:0/2806111160 learned_addr learned my addr 172.21.15.148:0/2806111160 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:13.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afd7b9700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af80cdab0 0x7f2af814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afdfba700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 0x7f2af816a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 -- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 msgr2=0x7f2af816a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 0x7f2af816a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 -- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af80cdab0 msgr2=0x7f2af814c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af80cdab0 0x7f2af814c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afe7bb700 1 -- 172.21.15.148:0/2806111160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009ae0 con 0x7f2af815dda0 2023-10-10T20:21:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.961+0000 7f2afdfba700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 0x7f2af816a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2afe7bb700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c7b10 tx=0x7f2af0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00094e0 con 0x7f2af815dda0 2023-10-10T20:21:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af00032f0 con 0x7f2af815dda0 2023-10-10T20:21:13.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af816aac0 con 0x7f2af815dda0 2023-10-10T20:21:13.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00266b0 con 0x7f2af815dda0 2023-10-10T20:21:13.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.962+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af816b080 con 0x7f2af815dda0 2023-10-10T20:21:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.963+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af80b3cf0 con 0x7f2af815dda0 2023-10-10T20:21:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.963+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2af0003040 con 0x7f2af815dda0 2023-10-10T20:21:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.964+0000 7f2aedffb700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.964+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f2af00864f0 con 0x7f2af815dda0 2023-10-10T20:21:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.964+0000 7f2afdfba700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.964+0000 7f2afdfba700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009990 tx=0x7f2af4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:13.968+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af80b3cf0 con 0x7f2af815dda0 2023-10-10T20:21:14.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.152+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af80b3cb0 con 0x7f2af815dda0 2023-10-10T20:21:14.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.153+0000 7f2aedffb700 1 -- 172.21.15.148:0/2806111160 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+15494 (secure 0 0 0) 0x7f2af0053720 con 0x7f2af815dda0 2023-10-10T20:21:14.155 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:14.156 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":54,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:12.559013+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:14.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 msgr2=0x7f2adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009990 tx=0x7f2af4008040 comp rx=0 tx=0).stop 2023-10-10T20:21:14.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 msgr2=0x7f2af80b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c7b10 tx=0x7f2af0018000 comp rx=0 tx=0).stop 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 shutdown_connections 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af815dda0 0x7f2af80b5600 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af80cd6e0 0x7f2af816a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 --2- 172.21.15.148:0/2806111160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af80cdab0 0x7f2af814c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 >> 172.21.15.148:0/2806111160 conn(0x7f2af8067660 msgr2=0x7f2af80d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:14.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 shutdown_connections 2023-10-10T20:21:14.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.156+0000 7f2afefbc700 1 -- 172.21.15.148:0/2806111160 wait complete. 2023-10-10T20:21:14.170 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:21:14.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:14.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 Processor -- start 2023-10-10T20:21:14.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 -- start start 2023-10-10T20:21:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 0x7f14b40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b415c360 con 0x7f14b4157ba0 2023-10-10T20:21:14.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b415b790 con 0x7f14b41577d0 2023-10-10T20:21:14.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.283+0000 7f14ba1ee700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415ba30 con 0x7f14b406ac30 2023-10-10T20:21:14.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b37fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 0x7f14b40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:48376/0 (socket says 172.21.15.148:48376) 2023-10-10T20:21:14.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b37fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 0x7f14b40cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:43188/0 (socket says 172.21.15.148:43188) 2023-10-10T20:21:14.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 -- 172.21.15.148:0/55024736 learned_addr learned my addr 172.21.15.148:0/55024736 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:14.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b2ffd700 1 --1- 172.21.15.148:0/55024736 >> v1:172.21.15.161:6789/0 conn(0x7f14b4157ba0 0x7f14b406a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44256/0 (socket says 172.21.15.148:44256) 2023-10-10T20:21:14.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 -- 172.21.15.148:0/55024736 >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 msgr2=0x7f14b40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 --2- 172.21.15.148:0/55024736 >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 0x7f14b40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 -- 172.21.15.148:0/55024736 >> v1:172.21.15.161:6789/0 conn(0x7f14b4157ba0 legacy=0x7f14b406a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 -- 172.21.15.148:0/55024736 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b415bd10 con 0x7f14b406ac30 2023-10-10T20:21:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b3fff700 1 --2- 172.21.15.148:0/55024736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099b0 tx=0x7f14ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed4fe9487da54f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b27fc700 1 -- 172.21.15.148:0/55024736 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac009ea0 con 0x7f14b406ac30 2023-10-10T20:21:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b27fc700 1 -- 172.21.15.148:0/55024736 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00b860 con 0x7f14b406ac30 2023-10-10T20:21:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.284+0000 7f14b27fc700 1 -- 172.21.15.148:0/55024736 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00bc00 con 0x7f14b406ac30 2023-10-10T20:21:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 -- 172.21.15.148:0/55024736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 msgr2=0x7f14b415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/55024736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099b0 tx=0x7f14ac009250 comp rx=0 tx=0).stop 2023-10-10T20:21:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 -- 172.21.15.148:0/55024736 shutdown_connections 2023-10-10T20:21:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/55024736 >> v2:172.21.15.161:3300/0 conn(0x7f14b41577d0 0x7f14b40cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/55024736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b406ac30 0x7f14b415c7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 -- 172.21.15.148:0/55024736 >> 172.21.15.148:0/55024736 conn(0x7f14b4068ce0 msgr2=0x7f14b40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 -- 172.21.15.148:0/55024736 shutdown_connections 2023-10-10T20:21:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.285+0000 7f14ba1ee700 1 -- 172.21.15.148:0/55024736 wait complete. 2023-10-10T20:21:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.286+0000 7f14ba1ee700 1 Processor -- start 2023-10-10T20:21:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.286+0000 7f14ba1ee700 1 -- start start 2023-10-10T20:21:14.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b41577d0 0x7f14b40a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b4157ba0 0x7f14b415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415cff0 con 0x7f14b406ac30 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415d2d0 con 0x7f14b41577d0 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14ba1ee700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415d5b0 con 0x7f14b4157ba0 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34552/0 (socket says 172.21.15.148:34552) 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 -- 172.21.15.148:0/840633575 learned_addr learned my addr 172.21.15.148:0/840633575 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:14.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b3fff700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b4157ba0 0x7f14b415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b2ffd700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b41577d0 0x7f14b40a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 -- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b4157ba0 msgr2=0x7f14b415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b4157ba0 0x7f14b415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 -- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b41577d0 msgr2=0x7f14b40a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b41577d0 0x7f14b40a8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.287+0000 7f14b37fe700 1 -- 172.21.15.148:0/840633575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0094f0 con 0x7f14b406ac30 2023-10-10T20:21:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14b37fe700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009ef0 tx=0x7f14a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400cb70 con 0x7f14b406ac30 2023-10-10T20:21:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400f040 con 0x7f14b406ac30 2023-10-10T20:21:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b415d890 con 0x7f14b406ac30 2023-10-10T20:21:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4014480 con 0x7f14b406ac30 2023-10-10T20:21:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.288+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b415e090 con 0x7f14b406ac30 2023-10-10T20:21:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.289+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b415dd30 con 0x7f14b406ac30 2023-10-10T20:21:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.289+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14a40147f0 con 0x7f14b406ac30 2023-10-10T20:21:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.289+0000 7f14ab7fe700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.290+0000 7f14b2ffd700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.290+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f14a4084ea0 con 0x7f14b406ac30 2023-10-10T20:21:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.290+0000 7f14b2ffd700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14b415f7b0 tx=0x7f149c007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.294+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14a40520d0 con 0x7f14b406ac30 2023-10-10T20:21:14.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.516+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14b415dee0 con 0x7f14b406ac30 2023-10-10T20:21:14.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.516+0000 7f14ab7fe700 1 -- 172.21.15.148:0/840633575 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f14a40556f0 con 0x7f14b406ac30 2023-10-10T20:21:14.519 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:14.519 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:21:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 msgr2=0x7f14940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 0x7f14940693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14b415f7b0 tx=0x7f149c007eb0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 msgr2=0x7f14b415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009ef0 tx=0x7f14a400b040 comp rx=0 tx=0).stop 2023-10-10T20:21:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 shutdown_connections 2023-10-10T20:21:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1494066f30 0x7f14940693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14b4157ba0 0x7f14b415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b406ac30 0x7f14b415c6f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 --2- 172.21.15.148:0/840633575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b41577d0 0x7f14b40a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 >> 172.21.15.148:0/840633575 conn(0x7f14b4068ce0 msgr2=0x7f14b40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 shutdown_connections 2023-10-10T20:21:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.518+0000 7f14ba1ee700 1 -- 172.21.15.148:0/840633575 wait complete. 2023-10-10T20:21:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:21:14.532 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:14.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 Processor -- start 2023-10-10T20:21:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 -- start start 2023-10-10T20:21:14.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 0x7fa32c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa32c06d590 0x7fa32c067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c0ccaf0 con 0x7fa32c0b0ee0 2023-10-10T20:21:14.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c15a4d0 con 0x7fa32c06d590 2023-10-10T20:21:14.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.646+0000 7fa33133c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c06db50 con 0x7fa32c0b12b0 2023-10-10T20:21:14.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa3227fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 0x7fa32c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa32c06d590 0x7fa32c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32affd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa32c0b0ee0 0x7fa32c15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44260/0 (socket says 172.21.15.148:44260) 2023-10-10T20:21:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32affd700 1 -- 172.21.15.148:0/248966441 learned_addr learned my addr 172.21.15.148:0/248966441 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 -- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 msgr2=0x7fa32c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 0x7fa32c15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 -- 172.21.15.148:0/248966441 >> v1:172.21.15.161:6789/0 conn(0x7fa32c0b0ee0 legacy=0x7fa32c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 -- 172.21.15.148:0/248966441 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c15a790 con 0x7fa32c06d590 2023-10-10T20:21:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa3227fc700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 0x7fa32c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32b7fe700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c06d590 0x7fa32c067660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa324009e60 tx=0x7fa324009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267c4d91bba08b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32a7fc700 1 -- 172.21.15.148:0/248966441 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32400e070 con 0x7fa32c06d590 2023-10-10T20:21:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32a7fc700 1 -- 172.21.15.148:0/248966441 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32400bb60 con 0x7fa32c06d590 2023-10-10T20:21:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.647+0000 7fa32a7fc700 1 -- 172.21.15.148:0/248966441 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324012430 con 0x7fa32c06d590 2023-10-10T20:21:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 -- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c06d590 msgr2=0x7fa32c067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c06d590 0x7fa32c067660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa324009e60 tx=0x7fa324009780 comp rx=0 tx=0).stop 2023-10-10T20:21:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 -- 172.21.15.148:0/248966441 shutdown_connections 2023-10-10T20:21:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c06d590 0x7fa32c067660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 --2- 172.21.15.148:0/248966441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c0b12b0 0x7fa32c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 -- 172.21.15.148:0/248966441 >> 172.21.15.148:0/248966441 conn(0x7fa32c06a310 msgr2=0x7fa32c0cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 -- 172.21.15.148:0/248966441 shutdown_connections 2023-10-10T20:21:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.648+0000 7fa33133c700 1 -- 172.21.15.148:0/248966441 wait complete. 2023-10-10T20:21:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.649+0000 7fa33133c700 1 Processor -- start 2023-10-10T20:21:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.649+0000 7fa33133c700 1 -- start start 2023-10-10T20:21:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c06d590 0x7fa32c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c0b0ee0 0x7fa32c16ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c168020 con 0x7fa32c0b12b0 2023-10-10T20:21:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c16c1c0 con 0x7fa32c0b0ee0 2023-10-10T20:21:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa33133c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c155a10 con 0x7fa32c06d590 2023-10-10T20:21:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34566/0 (socket says 172.21.15.148:34566) 2023-10-10T20:21:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 -- 172.21.15.148:0/2816430201 learned_addr learned my addr 172.21.15.148:0/2816430201 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32affd700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c06d590 0x7fa32c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa3227fc700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c0b0ee0 0x7fa32c16ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 -- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c06d590 msgr2=0x7fa32c155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c06d590 0x7fa32c155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 -- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c0b0ee0 msgr2=0x7fa32c16ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c0b0ee0 0x7fa32c16ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.650+0000 7fa32b7fe700 1 -- 172.21.15.148:0/2816430201 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314009420 con 0x7fa32c0b12b0 2023-10-10T20:21:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa32b7fe700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06e630 tx=0x7fa324019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324027470 con 0x7fa32c0b12b0 2023-10-10T20:21:14.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3240275f0 con 0x7fa32c0b12b0 2023-10-10T20:21:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa324009a20 con 0x7fa32c0b12b0 2023-10-10T20:21:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3240279b0 con 0x7fa32c0b12b0 2023-10-10T20:21:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.651+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c16b890 con 0x7fa32c0b12b0 2023-10-10T20:21:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.652+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c16c330 con 0x7fa32c0b12b0 2023-10-10T20:21:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.652+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa324024070 con 0x7fa32c0b12b0 2023-10-10T20:21:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.653+0000 7fa3237fe700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.653+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fa324087040 con 0x7fa32c0b12b0 2023-10-10T20:21:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.653+0000 7fa32affd700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.653+0000 7fa32affd700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 0x7fa30c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0b4a90 tx=0x7fa31c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.657+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3240b85a0 con 0x7fa32c0b12b0 2023-10-10T20:21:14.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.880+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa32c15aa70 con 0x7fa32c0b12b0 2023-10-10T20:21:14.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.880+0000 7fa3237fe700 1 -- 172.21.15.148:0/2816430201 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fa3240541d0 con 0x7fa32c0b12b0 2023-10-10T20:21:14.883 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:14.883 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:20:39.184854+0000","modified":"2023-10-10T20:20:42.828507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5367":{"gid":5367,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3613275465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3613275465},{"type":"v1","addr":"172.21.15.161:6835","nonce":3613275465}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-10T20:21:14.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 msgr2=0x7fa30c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 0x7fa30c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0b4a90 tx=0x7fa31c009040 comp rx=0 tx=0).stop 2023-10-10T20:21:14.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 msgr2=0x7fa32c068630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:14.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa32c06e630 tx=0x7fa324019000 comp rx=0 tx=0).stop 2023-10-10T20:21:14.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 shutdown_connections 2023-10-10T20:21:14.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa30c066f30 0x7fa30c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa32c0b12b0 0x7fa32c068630 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32c06d590 0x7fa32c155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 --2- 172.21.15.148:0/2816430201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa32c0b0ee0 0x7fa32c16ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:14.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 >> 172.21.15.148:0/2816430201 conn(0x7fa32c06a310 msgr2=0x7fa32c0c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:14.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 shutdown_connections 2023-10-10T20:21:14.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:14.882+0000 7fa33133c700 1 -- 172.21.15.148:0/2816430201 wait complete. 2023-10-10T20:21:14.888 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2023-10-10T20:21:14.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:15.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.008+0000 7f56d31e5700 1 Processor -- start 2023-10-10T20:21:15.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.008+0000 7f56d31e5700 1 -- start start 2023-10-10T20:21:15.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d31e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc157870 0x7f56cc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d31e5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc15cbb0 con 0x7f56cc157c40 2023-10-10T20:21:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d31e5700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc15bfa0 con 0x7f56cc15a620 2023-10-10T20:21:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d31e5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc15c280 con 0x7f56cc157870 2023-10-10T20:21:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d1782700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f56cc157c40 0x7f56cc158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37290/0 (socket says 172.21.15.148:37290) 2023-10-10T20:21:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.009+0000 7f56d1782700 1 -- 172.21.15.148:0/3690628568 learned_addr learned my addr 172.21.15.148:0/3690628568 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933824893 0 0) 0x7f56cc15cbb0 con 0x7f56cc157c40 2023-10-10T20:21:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b4003570 con 0x7f56cc157c40 2023-10-10T20:21:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc157870 0x7f56cc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123731903 0 0) 0x7f56b4003570 con 0x7f56cc157c40 2023-10-10T20:21:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc157870 msgr2=0x7f56cc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc157870 0x7f56cc15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 >> v1:172.21.15.161:6789/0 conn(0x7f56cc15a620 legacy=0x7f56cc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc15c560 con 0x7f56cc157c40 2023-10-10T20:21:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f56c40017f0 con 0x7f56cc157c40 2023-10-10T20:21:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 >> v1:172.21.15.148:6789/0 conn(0x7f56cc157c40 legacy=0x7f56cc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4003a90 0x7f56b4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b4004810 0x7f56b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56b4005230 0x7f56b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc15c280 con 0x7f56b4004810 2023-10-10T20:21:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b400c0d0 con 0x7f56b4003a90 2023-10-10T20:21:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b400c290 con 0x7f56b4005230 2023-10-10T20:21:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc157870 0x7f56cc15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b4004810 0x7f56b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56d1782700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56b4005230 0x7f56b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4003a90 0x7f56b4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 -- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56b4005230 msgr2=0x7f56b400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56b4005230 0x7f56b400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 -- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4003a90 msgr2=0x7f56b4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4003a90 0x7f56b4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.010+0000 7f56c3fff700 1 -- 172.21.15.148:0/3690628568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4003570 con 0x7f56b4004810 2023-10-10T20:21:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b4004810 0x7f56b4004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0018b0 tx=0x7f56bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdb0e22ef8e5e5e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012420 con 0x7f56b4004810 2023-10-10T20:21:15.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc012580 con 0x7f56b4004810 2023-10-10T20:21:15.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56c37fe700 1 -- 172.21.15.148:0/3690628568 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012920 con 0x7f56b4004810 2023-10-10T20:21:15.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 -- 172.21.15.148:0/3690628568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b4004810 msgr2=0x7f56b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3690628568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b4004810 0x7f56b4004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0018b0 tx=0x7f56bc007d30 comp rx=0 tx=0).stop 2023-10-10T20:21:15.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d1782700 1 -- 172.21.15.148:0/3690628568 reap_dead start 2023-10-10T20:21:15.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 -- 172.21.15.148:0/3690628568 shutdown_connections 2023-10-10T20:21:15.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 -- 172.21.15.148:0/3690628568 >> 172.21.15.148:0/3690628568 conn(0x7f56cc068ce0 msgr2=0x7f56cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:15.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 -- 172.21.15.148:0/3690628568 shutdown_connections 2023-10-10T20:21:15.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.011+0000 7f56d31e5700 1 -- 172.21.15.148:0/3690628568 wait complete. 2023-10-10T20:21:15.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.012+0000 7f56d31e5700 1 Processor -- start 2023-10-10T20:21:15.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 -- start start 2023-10-10T20:21:15.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 0x7f56cc14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 0x7f56cc1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc150e00 con 0x7f56cc14f4e0 2023-10-10T20:21:15.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc14bac0 con 0x7f56b4004810 2023-10-10T20:21:15.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.013+0000 7f56d31e5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1525e0 con 0x7f56cc151330 2023-10-10T20:21:15.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34584/0 (socket says 172.21.15.148:34584) 2023-10-10T20:21:15.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 -- 172.21.15.148:0/3585827137 learned_addr learned my addr 172.21.15.148:0/3585827137 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:15.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d1782700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 0x7f56cc1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 0x7f56cc14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 -- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 msgr2=0x7f56cc1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 0x7f56cc1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 -- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 msgr2=0x7f56cc14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 0x7f56cc14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 -- 172.21.15.148:0/3585827137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc150280 con 0x7f56cc14f4e0 2023-10-10T20:21:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d1782700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 0x7f56cc1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 0x7f56cc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c3fff700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00cea0 tx=0x7f56bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc001200 con 0x7f56cc14f4e0 2023-10-10T20:21:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc012c90 con 0x7f56cc14f4e0 2023-10-10T20:21:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc150560 con 0x7f56cc14f4e0 2023-10-10T20:21:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012e00 con 0x7f56cc14f4e0 2023-10-10T20:21:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.014+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc150b20 con 0x7f56cc14f4e0 2023-10-10T20:21:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.015+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc04c3c0 con 0x7f56cc14f4e0 2023-10-10T20:21:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.016+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56bc006a30 con 0x7f56cc14f4e0 2023-10-10T20:21:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.016+0000 7f56c0ff9700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 0x7f56b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.016+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f56bc085c20 con 0x7f56cc14f4e0 2023-10-10T20:21:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.016+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 0x7f56b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.016+0000 7f56d0f81700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 0x7f56b80693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0b37b0 tx=0x7f56c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.020+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56bc052e50 con 0x7f56cc14f4e0 2023-10-10T20:21:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.204+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56cc15c840 con 0x7f56cc14f4e0 2023-10-10T20:21:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.204+0000 7f56c0ff9700 1 -- 172.21.15.148:0/3585827137 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+15494 (secure 0 0 0) 0x7f56bc017090 con 0x7f56cc14f4e0 2023-10-10T20:21:15.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:15.207 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":54,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:12.559013+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:20:37.022631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-10T20:20:38.151667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 msgr2=0x7f56b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 0x7f56b80693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0b37b0 tx=0x7f56c8008040 comp rx=0 tx=0).stop 2023-10-10T20:21:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 msgr2=0x7f56cc14f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00cea0 tx=0x7f56bc015000 comp rx=0 tx=0).stop 2023-10-10T20:21:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 shutdown_connections 2023-10-10T20:21:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f56b8066f30 0x7f56b80693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56cc151330 0x7f56cc1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.206+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56b4004810 0x7f56cc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.207+0000 7f56d31e5700 1 --2- 172.21.15.148:0/3585827137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56cc14f4e0 0x7f56cc14f950 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.207+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 >> 172.21.15.148:0/3585827137 conn(0x7f56cc068ce0 msgr2=0x7f56cc0d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.207+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 shutdown_connections 2023-10-10T20:21:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.207+0000 7f56d31e5700 1 -- 172.21.15.148:0/3585827137 wait complete. 2023-10-10T20:21:15.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:21:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.334+0000 7f15b26cc700 1 Processor -- start 2023-10-10T20:21:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.334+0000 7f15b26cc700 1 -- start start 2023-10-10T20:21:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b26cc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f15ac15a170 0x7f15ac068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b26cc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0694a0 con 0x7f15ac159da0 2023-10-10T20:21:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b26cc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac15ea70 con 0x7f15ac069040 2023-10-10T20:21:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b26cc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0b34b0 con 0x7f15ac15a170 2023-10-10T20:21:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b16ca700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f15ac159da0 0x7f15ac0b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37296/0 (socket says 172.21.15.148:37296) 2023-10-10T20:21:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b16ca700 1 -- 172.21.15.148:0/2593083194 learned_addr learned my addr 172.21.15.148:0/2593083194 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479299689 0 0) 0x7f15a4001110 con 0x7f15ac159da0 2023-10-10T20:21:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> v2:172.21.15.161:3300/0 conn(0x7f15ac15a170 0x7f15ac068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1594003570 con 0x7f15ac159da0 2023-10-10T20:21:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760786129 0 0) 0x7f1594003570 con 0x7f15ac159da0 2023-10-10T20:21:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 >> v2:172.21.15.161:3300/0 conn(0x7f15ac15a170 msgr2=0x7f15ac068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 --2- 172.21.15.148:0/2593083194 >> v2:172.21.15.161:3300/0 conn(0x7f15ac15a170 0x7f15ac068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 >> v1:172.21.15.161:6789/0 conn(0x7f15ac069040 legacy=0x7f15ac0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0b3990 con 0x7f15ac159da0 2023-10-10T20:21:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f15a4001600 con 0x7f15ac159da0 2023-10-10T20:21:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> v2:172.21.15.161:3300/0 conn(0x7f15ac15a170 0x7f15ac068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 >> v1:172.21.15.148:6789/0 conn(0x7f15ac159da0 legacy=0x7f15ac0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1594003a60 0x7f1594003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.335+0000 7f15abfff700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 0x7f1594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15940053c0 0x7f159400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0b34b0 con 0x7f1594004810 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159400c2d0 con 0x7f15940053c0 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159400c490 con 0x7f1594003a60 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 0x7f1594004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b1ecb700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15940053c0 0x7f159400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b16ca700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1594003a60 0x7f1594003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 -- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1594003a60 msgr2=0x7f1594003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1594003a60 0x7f1594003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 -- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15940053c0 msgr2=0x7f159400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15940053c0 0x7f159400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b1ecb700 1 -- 172.21.15.148:0/2593083194 reap_dead start 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 -- 172.21.15.148:0/2593083194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594003570 con 0x7f1594004810 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 0x7f1594004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007580 tx=0x7f15a00097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94015731f8042dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0009ae0 con 0x7f1594004810 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0007dd0 con 0x7f1594004810 2023-10-10T20:21:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.336+0000 7f15abfff700 1 -- 172.21.15.148:0/2593083194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0014630 con 0x7f1594004810 2023-10-10T20:21:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 -- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 msgr2=0x7f1594004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 0x7f1594004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f15a0007580 tx=0x7f15a00097d0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 -- 172.21.15.148:0/2593083194 shutdown_connections 2023-10-10T20:21:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2593083194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1594004810 0x7f1594004c80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 -- 172.21.15.148:0/2593083194 >> 172.21.15.148:0/2593083194 conn(0x7f15ac0d0bc0 msgr2=0x7f15ac0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 -- 172.21.15.148:0/2593083194 shutdown_connections 2023-10-10T20:21:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.337+0000 7f15b26cc700 1 -- 172.21.15.148:0/2593083194 wait complete. 2023-10-10T20:21:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.338+0000 7f15b26cc700 1 Processor -- start 2023-10-10T20:21:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.338+0000 7f15b26cc700 1 -- start start 2023-10-10T20:21:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1594004810 0x7f15ac0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 0x7f15ac0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1586e0 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0b5650 con 0x7f1594004810 2023-10-10T20:21:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b26cc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15bf40 con 0x7f15ac158270 2023-10-10T20:21:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34604/0 (socket says 172.21.15.148:34604) 2023-10-10T20:21:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 -- 172.21.15.148:0/2129570621 learned_addr learned my addr 172.21.15.148:0/2129570621 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b16ca700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1594004810 0x7f15ac0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 0x7f15ac0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 -- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 msgr2=0x7f15ac0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 0x7f15ac0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 -- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1594004810 msgr2=0x7f15ac0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1594004810 0x7f15ac0c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 -- 172.21.15.148:0/2129570621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0cb410 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b0ec9700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 0x7f15ac0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.339+0000 7f15b1ecb700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0c7b60 tx=0x7f159c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.340+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c006b00 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.340+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c006c80 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.340+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac16a540 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.340+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c007030 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.340+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac0c91e0 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.341+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac15c0b0 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.341+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f159c007460 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.341+0000 7f15a97fa700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 0x7f15980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.342+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f159c0834f0 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.342+0000 7f15b16ca700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 0x7f15980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.342+0000 7f15b16ca700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 0x7f15980693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f15a4001ed0 tx=0x7f15a4007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.346+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f159c0507e0 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.573+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f15ac0b0b70 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.575 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:21:15.573+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:21:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.577+0000 7f15a97fa700 1 -- 172.21.15.148:0/2129570621 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f159c019020 con 0x7f15ac0b4ca0 2023-10-10T20:21:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 msgr2=0x7f15980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 0x7f15980693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f15a4001ed0 tx=0x7f15a4007a50 comp rx=0 tx=0).stop 2023-10-10T20:21:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 msgr2=0x7f15ac0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0c7b60 tx=0x7f159c007d30 comp rx=0 tx=0).stop 2023-10-10T20:21:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 shutdown_connections 2023-10-10T20:21:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1598066f30 0x7f15980693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15ac0b4ca0 0x7f15ac0b5110 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1594004810 0x7f15ac0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 --2- 172.21.15.148:0/2129570621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15ac158270 0x7f15ac0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 >> 172.21.15.148:0/2129570621 conn(0x7f15ac0d0bc0 msgr2=0x7f15ac0ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 shutdown_connections 2023-10-10T20:21:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.579+0000 7f15b26cc700 1 -- 172.21.15.148:0/2129570621 wait complete. 2023-10-10T20:21:15.584 INFO:tasks.ceph.mds.b.smithi161.stderr: -9097> 2023-10-10T20:20:44.796+0000 7fe592c76700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:21:15.584 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:21:15.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:21:15.614 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:21:15.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.708+0000 7f5752e63700 1 Processor -- start 2023-10-10T20:21:15.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.709+0000 7f5752e63700 1 -- start start 2023-10-10T20:21:15.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.709+0000 7f5752e63700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f574c159ec0 0x7f574c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5752e63700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0b0790 con 0x7f574c15a290 2023-10-10T20:21:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5752e63700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0ce1f0 con 0x7f574c06acb0 2023-10-10T20:21:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5752e63700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c0b0b20 con 0x7f574c159ec0 2023-10-10T20:21:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f574c159ec0 0x7f574c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f574c15a290 0x7f574c06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37298/0 (socket says 172.21.15.148:37298) 2023-10-10T20:21:15.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574bfff700 1 -- 172.21.15.148:0/4142451476 learned_addr learned my addr 172.21.15.148:0/4142451476 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:15.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 --2- 172.21.15.148:0/4142451476 >> v2:172.21.15.148:3300/0 conn(0x7f574c159ec0 0x7f574c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34618/0 (socket says 172.21.15.148:34618) 2023-10-10T20:21:15.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 -- 172.21.15.148:0/4142451476 >> v1:172.21.15.161:6789/0 conn(0x7f574c06acb0 legacy=0x7f574c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:15.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 -- 172.21.15.148:0/4142451476 >> v1:172.21.15.148:6789/0 conn(0x7f574c15a290 legacy=0x7f574c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574b7fe700 1 -- 172.21.15.148:0/4142451476 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875221084 0 0) 0x7f574c0b0790 con 0x7f574c15a290 2023-10-10T20:21:15.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 -- 172.21.15.148:0/4142451476 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0b04b0 con 0x7f574c159ec0 2023-10-10T20:21:15.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f5750bff700 1 --2- 172.21.15.148:0/4142451476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0af940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57400098e0 tx=0x7f57400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63f7192a5789a92 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574b7fe700 1 -- 172.21.15.148:0/4142451476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740009e80 con 0x7f574c159ec0 2023-10-10T20:21:15.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574b7fe700 1 -- 172.21.15.148:0/4142451476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000b820 con 0x7f574c159ec0 2023-10-10T20:21:15.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.710+0000 7f574b7fe700 1 -- 172.21.15.148:0/4142451476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000bba0 con 0x7f574c159ec0 2023-10-10T20:21:15.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 -- 172.21.15.148:0/4142451476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 msgr2=0x7f574c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 --2- 172.21.15.148:0/4142451476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0af940 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57400098e0 tx=0x7f57400091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 -- 172.21.15.148:0/4142451476 shutdown_connections 2023-10-10T20:21:15.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 --2- 172.21.15.148:0/4142451476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0af940 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 -- 172.21.15.148:0/4142451476 >> 172.21.15.148:0/4142451476 conn(0x7f574c0d1880 msgr2=0x7f574c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:15.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 -- 172.21.15.148:0/4142451476 shutdown_connections 2023-10-10T20:21:15.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.711+0000 7f5752e63700 1 -- 172.21.15.148:0/4142451476 wait complete. 2023-10-10T20:21:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.712+0000 7f5752e63700 1 Processor -- start 2023-10-10T20:21:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.712+0000 7f5752e63700 1 -- start start 2023-10-10T20:21:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 0x7f574c143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f574c15a290 0x7f574c0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c9e70 con 0x7f574c159ec0 2023-10-10T20:21:15.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c9260 con 0x7f574c15a290 2023-10-10T20:21:15.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5752e63700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c9540 con 0x7f574c06acb0 2023-10-10T20:21:15.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f574bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f574bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34620/0 (socket says 172.21.15.148:34620) 2023-10-10T20:21:15.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f574bfff700 1 -- 172.21.15.148:0/2529412905 learned_addr learned my addr 172.21.15.148:0/2529412905 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:15.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5751400700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f574c15a290 0x7f574c0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f5750bff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 0x7f574c143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.713+0000 7f574bfff700 1 -- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 msgr2=0x7f574c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f574bfff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 0x7f574c143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f574bfff700 1 -- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f574c15a290 msgr2=0x7f574c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:15.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f574bfff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f574c15a290 0x7f574c0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:15.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f574bfff700 1 -- 172.21.15.148:0/2529412905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740009420 con 0x7f574c159ec0 2023-10-10T20:21:15.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5750bff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 0x7f574c143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f574bfff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f573800c5b0 tx=0x7f5738001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738005b30 con 0x7f574c159ec0 2023-10-10T20:21:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738005c90 con 0x7f574c159ec0 2023-10-10T20:21:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c0c9820 con 0x7f574c159ec0 2023-10-10T20:21:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738007300 con 0x7f574c159ec0 2023-10-10T20:21:15.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.714+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c144470 con 0x7f574c159ec0 2023-10-10T20:21:15.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.715+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c04c3c0 con 0x7f574c159ec0 2023-10-10T20:21:15.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.715+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f573801b3b0 con 0x7f574c159ec0 2023-10-10T20:21:15.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.715+0000 7f5748ff9700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 0x7f57300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:15.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.716+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f5738083af0 con 0x7f574c159ec0 2023-10-10T20:21:15.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.716+0000 7f5750bff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 0x7f57300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:15.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.716+0000 7f5750bff700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 0x7f57300693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f57400030c0 tx=0x7f5740016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:15.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.720+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5738050cd0 con 0x7f574c159ec0 2023-10-10T20:21:15.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:15.945+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f574c0ceab0 con 0x7f574c159ec0 2023-10-10T20:21:16.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.579+0000 7f5748ff9700 1 -- 172.21.15.148:0/2529412905 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57380170a0 con 0x7f574c159ec0 2023-10-10T20:21:16.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 msgr2=0x7f57300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 0x7f57300693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f57400030c0 tx=0x7f5740016000 comp rx=0 tx=0).stop 2023-10-10T20:21:16.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 msgr2=0x7f574c0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f573800c5b0 tx=0x7f5738001070 comp rx=0 tx=0).stop 2023-10-10T20:21:16.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 shutdown_connections 2023-10-10T20:21:16.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5730066f30 0x7f57300693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f574c15a290 0x7f574c0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f574c06acb0 0x7f574c143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 --2- 172.21.15.148:0/2529412905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c159ec0 0x7f574c0ca300 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 >> 172.21.15.148:0/2529412905 conn(0x7f574c0d1880 msgr2=0x7f574c0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:16.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 shutdown_connections 2023-10-10T20:21:16.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.582+0000 7f5752e63700 1 -- 172.21.15.148:0/2529412905 wait complete. 2023-10-10T20:21:16.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:21:16.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.715+0000 7f72ae532700 1 Processor -- start 2023-10-10T20:21:16.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.715+0000 7f72ae532700 1 -- start start 2023-10-10T20:21:16.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72ae532700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72ae532700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72ae532700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a806af60 con 0x7f72a8157960 2023-10-10T20:21:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72ae532700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a80cfda0 con 0x7f72a80cd7d0 2023-10-10T20:21:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72ae532700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80ce050 con 0x7f72a806a5c0 2023-10-10T20:21:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72acacf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72a77fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72a77fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:43284/0 (socket says 172.21.15.148:43284) 2023-10-10T20:21:16.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72acacf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:48460/0 (socket says 172.21.15.148:48460) 2023-10-10T20:21:16.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72a77fe700 1 -- 172.21.15.148:0/3337954770 learned_addr learned my addr 172.21.15.148:0/3337954770 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:16.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.716+0000 7f72a7fff700 1 --1- 172.21.15.148:0/3337954770 >> v1:172.21.15.161:6789/0 conn(0x7f72a8157960 0x7f72a815ae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44282/0 (socket says 172.21.15.148:44282) 2023-10-10T20:21:16.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72acacf700 1 -- 172.21.15.148:0/3337954770 >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 msgr2=0x7f72a80d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:16.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72acacf700 1 --2- 172.21.15.148:0/3337954770 >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72acacf700 1 -- 172.21.15.148:0/3337954770 >> v1:172.21.15.161:6789/0 conn(0x7f72a8157960 legacy=0x7f72a815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72acacf700 1 -- 172.21.15.148:0/3337954770 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80cf4b0 con 0x7f72a806a5c0 2023-10-10T20:21:16.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72a6ffd700 1 -- 172.21.15.148:0/3337954770 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854197109 0 0) 0x7f7298001110 con 0x7f72a8157960 2023-10-10T20:21:16.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72a77fe700 1 --2- 172.21.15.148:0/3337954770 >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:16.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72acacf700 1 --2- 172.21.15.148:0/3337954770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009e60 tx=0x7f72a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c788ad161e17a785 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:16.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72a6ffd700 1 -- 172.21.15.148:0/3337954770 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a000e070 con 0x7f72a806a5c0 2023-10-10T20:21:16.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72a6ffd700 1 -- 172.21.15.148:0/3337954770 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a000bb60 con 0x7f72a806a5c0 2023-10-10T20:21:16.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.717+0000 7f72a6ffd700 1 -- 172.21.15.148:0/3337954770 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0012430 con 0x7f72a806a5c0 2023-10-10T20:21:16.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 -- 172.21.15.148:0/3337954770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 msgr2=0x7f72a806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 --2- 172.21.15.148:0/3337954770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009e60 tx=0x7f72a0009780 comp rx=0 tx=0).stop 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 -- 172.21.15.148:0/3337954770 shutdown_connections 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 --2- 172.21.15.148:0/3337954770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a806aa20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 --2- 172.21.15.148:0/3337954770 >> v2:172.21.15.161:3300/0 conn(0x7f72a80cd7d0 0x7f72a80d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 -- 172.21.15.148:0/3337954770 >> 172.21.15.148:0/3337954770 conn(0x7f72a80d18a0 msgr2=0x7f72a80b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:16.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 -- 172.21.15.148:0/3337954770 shutdown_connections 2023-10-10T20:21:16.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.718+0000 7f72ae532700 1 -- 172.21.15.148:0/3337954770 wait complete. 2023-10-10T20:21:16.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.719+0000 7f72ae532700 1 Processor -- start 2023-10-10T20:21:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.719+0000 7f72ae532700 1 -- start start 2023-10-10T20:21:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a80cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 0x7f72a80b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8145330 con 0x7f72a8157960 2023-10-10T20:21:16.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8144720 con 0x7f72a80cd7d0 2023-10-10T20:21:16.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8144a00 con 0x7f72a806a5c0 2023-10-10T20:21:16.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34622/0 (socket says 172.21.15.148:34622) 2023-10-10T20:21:16.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 -- 172.21.15.148:0/1937555979 learned_addr learned my addr 172.21.15.148:0/1937555979 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:16.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72a7fff700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a80cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72a77fe700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 0x7f72a80b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 -- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 msgr2=0x7f72a80cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a80cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 -- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 msgr2=0x7f72a80b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:16.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 0x7f72a80b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:16.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 -- 172.21.15.148:0/1937555979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0094f0 con 0x7f72a8157960 2023-10-10T20:21:16.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72a77fe700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 0x7f72a80b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:16.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72acacf700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72a806b4d0 tx=0x7f72a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:16.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0012ea0 con 0x7f72a8157960 2023-10-10T20:21:16.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0027470 con 0x7f72a8157960 2023-10-10T20:21:16.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0009a20 con 0x7f72a8157960 2023-10-10T20:21:16.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.720+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0027830 con 0x7f72a8157960 2023-10-10T20:21:16.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.721+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8144fc0 con 0x7f72a8157960 2023-10-10T20:21:16.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.721+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a804c3c0 con 0x7f72a8157960 2023-10-10T20:21:16.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.722+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72a0024070 con 0x7f72a8157960 2023-10-10T20:21:16.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.722+0000 7f728ffff700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 0x7f72940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:16.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.722+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f72a0086d20 con 0x7f72a8157960 2023-10-10T20:21:16.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.722+0000 7f72a7fff700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 0x7f72940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:16.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.723+0000 7f72a7fff700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 0x7f72940693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7298007180 tx=0x7f7298007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:16.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.727+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72a0053f00 con 0x7f72a8157960 2023-10-10T20:21:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:16.911+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72a80cdd70 con 0x7f72a8157960 2023-10-10T20:21:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.584+0000 7f728ffff700 1 -- 172.21.15.148:0/1937555979 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72a0057520 con 0x7f72a8157960 2023-10-10T20:21:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 msgr2=0x7f72940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 0x7f72940693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7298007180 tx=0x7f7298007ef0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 msgr2=0x7f72a8143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72a806b4d0 tx=0x7f72a0019000 comp rx=0 tx=0).stop 2023-10-10T20:21:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 shutdown_connections 2023-10-10T20:21:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7294066f30 0x7f72940693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8157960 0x7f72a8143d50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a806a5c0 0x7f72a80cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 --2- 172.21.15.148:0/1937555979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a80cd7d0 0x7f72a80b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 >> 172.21.15.148:0/1937555979 conn(0x7f72a80d18a0 msgr2=0x7f72a815cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:17.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 shutdown_connections 2023-10-10T20:21:17.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.587+0000 7f72ae532700 1 -- 172.21.15.148:0/1937555979 wait complete. 2023-10-10T20:21:17.592 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:21:17.602 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:21:17.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.719+0000 7f8d7e283700 1 Processor -- start 2023-10-10T20:21:17.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.719+0000 7f8d7e283700 1 -- start start 2023-10-10T20:21:17.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7e283700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d780b6a10 0x7f8d7815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:17.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7e283700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780b17b0 con 0x7f8d780b46f0 2023-10-10T20:21:17.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7e283700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780b1a60 con 0x7f8d78159ef0 2023-10-10T20:21:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7e283700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780b1d40 con 0x7f8d780b6a10 2023-10-10T20:21:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d777fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d780b6a10 0x7f8d7815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7c820700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8d780b46f0 0x7f8d780b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37300/0 (socket says 172.21.15.148:37300) 2023-10-10T20:21:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d777fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d780b6a10 0x7f8d7815a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34630/0 (socket says 172.21.15.148:34630) 2023-10-10T20:21:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.720+0000 7f8d7c820700 1 -- 172.21.15.148:0/4172811076 learned_addr learned my addr 172.21.15.148:0/4172811076 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d777fe700 1 -- 172.21.15.148:0/4172811076 >> v1:172.21.15.161:6789/0 conn(0x7f8d78159ef0 legacy=0x7f8d780b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d777fe700 1 -- 172.21.15.148:0/4172811076 >> v1:172.21.15.148:6789/0 conn(0x7f8d780b46f0 legacy=0x7f8d780b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d777fe700 1 -- 172.21.15.148:0/4172811076 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780b2020 con 0x7f8d780b6a10 2023-10-10T20:21:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d76ffd700 1 -- 172.21.15.148:0/4172811076 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225931901 0 0) 0x7f8d780b17b0 con 0x7f8d780b46f0 2023-10-10T20:21:17.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d777fe700 1 --2- 172.21.15.148:0/4172811076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d7815a2c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0098e0 tx=0x7f8d6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab1993c38c0ae6f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d76ffd700 1 -- 172.21.15.148:0/4172811076 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c009e80 con 0x7f8d780b6a10 2023-10-10T20:21:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d76ffd700 1 -- 172.21.15.148:0/4172811076 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00b8f0 con 0x7f8d780b6a10 2023-10-10T20:21:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d76ffd700 1 -- 172.21.15.148:0/4172811076 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00bc70 con 0x7f8d780b6a10 2023-10-10T20:21:17.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 -- 172.21.15.148:0/4172811076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 msgr2=0x7f8d7815a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 --2- 172.21.15.148:0/4172811076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d7815a2c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0098e0 tx=0x7f8d6c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 -- 172.21.15.148:0/4172811076 shutdown_connections 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 --2- 172.21.15.148:0/4172811076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d7815a2c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 -- 172.21.15.148:0/4172811076 >> 172.21.15.148:0/4172811076 conn(0x7f8d7809a0a0 msgr2=0x7f8d78067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 -- 172.21.15.148:0/4172811076 shutdown_connections 2023-10-10T20:21:17.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.721+0000 7f8d7e283700 1 -- 172.21.15.148:0/4172811076 wait complete. 2023-10-10T20:21:17.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 Processor -- start 2023-10-10T20:21:17.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 -- start start 2023-10-10T20:21:17.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 0x7f8d78155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:17.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:17.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d78159ef0 0x7f8d78157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:17.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78153850 con 0x7f8d780b6a10 2023-10-10T20:21:17.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78153b30 con 0x7f8d78159ef0 2023-10-10T20:21:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.723+0000 7f8d7e283700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7814fa70 con 0x7f8d780b46f0 2023-10-10T20:21:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34638/0 (socket says 172.21.15.148:34638) 2023-10-10T20:21:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 -- 172.21.15.148:0/442174293 learned_addr learned my addr 172.21.15.148:0/442174293 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d7c820700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d78159ef0 0x7f8d78157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:17.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d77fff700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 0x7f8d78155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:17.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 -- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 msgr2=0x7f8d78155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 0x7f8d78155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 -- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d78159ef0 msgr2=0x7f8d78157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:17.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d78159ef0 0x7f8d78157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:17.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 -- 172.21.15.148:0/442174293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c009420 con 0x7f8d780b6a10 2023-10-10T20:21:17.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d77fff700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 0x7f8d78155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:17.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d777fe700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c013000 tx=0x7f8d6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:17.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c01f070 con 0x7f8d780b6a10 2023-10-10T20:21:17.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00ba60 con 0x7f8d780b6a10 2023-10-10T20:21:17.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7814fd50 con 0x7f8d780b6a10 2023-10-10T20:21:17.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c025690 con 0x7f8d780b6a10 2023-10-10T20:21:17.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.724+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d78153540 con 0x7f8d780b6a10 2023-10-10T20:21:17.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.725+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78150270 con 0x7f8d780b6a10 2023-10-10T20:21:17.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.725+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d6c023070 con 0x7f8d780b6a10 2023-10-10T20:21:17.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.726+0000 7f8d5ffff700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 0x7f8d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:17.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.726+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f8d6c0859c0 con 0x7f8d780b6a10 2023-10-10T20:21:17.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.726+0000 7f8d77fff700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 0x7f8d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:17.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.726+0000 7f8d77fff700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 0x7f8d640693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d680018f0 tx=0x7f8d68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:17.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.730+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d78150270 con 0x7f8d780b6a10 2023-10-10T20:21:17.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:17.916+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d78150270 con 0x7f8d780b6a10 2023-10-10T20:21:18.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.586+0000 7f8d5ffff700 1 -- 172.21.15.148:0/442174293 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d78150270 con 0x7f8d780b6a10 2023-10-10T20:21:18.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 msgr2=0x7f8d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 0x7f8d640693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d680018f0 tx=0x7f8d68009040 comp rx=0 tx=0).stop 2023-10-10T20:21:18.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 msgr2=0x7f8d78153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c013000 tx=0x7f8d6c017000 comp rx=0 tx=0).stop 2023-10-10T20:21:18.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 shutdown_connections 2023-10-10T20:21:18.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d64066f30 0x7f8d640693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d78159ef0 0x7f8d78157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d780b46f0 0x7f8d78155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 --2- 172.21.15.148:0/442174293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d780b6a10 0x7f8d78153000 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 >> 172.21.15.148:0/442174293 conn(0x7f8d7809a0a0 msgr2=0x7f8d7814eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 shutdown_connections 2023-10-10T20:21:18.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.589+0000 7f8d7e283700 1 -- 172.21.15.148:0/442174293 wait complete. 2023-10-10T20:21:18.594 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:21:18.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:21:18.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.719+0000 7f0e26442700 1 Processor -- start 2023-10-10T20:21:18.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.719+0000 7f0e26442700 1 -- start start 2023-10-10T20:21:18.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.720+0000 7f0e26442700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e200af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.720+0000 7f0e26442700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e2015a290 0x7f0e200afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.720+0000 7f0e26442700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e200b0790 con 0x7f0e2006a5a0 2023-10-10T20:21:18.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.720+0000 7f0e26442700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200ce1f0 con 0x7f0e2015a290 2023-10-10T20:21:18.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.720+0000 7f0e26442700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200b0b20 con 0x7f0e20159ec0 2023-10-10T20:21:18.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e2015a290 0x7f0e200afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e200af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e249df700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0e2006a5a0 0x7f0e2015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44308/0 (socket says 172.21.15.148:44308) 2023-10-10T20:21:18.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e249df700 1 -- 172.21.15.148:0/2811588208 learned_addr learned my addr 172.21.15.148:0/2811588208 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:18.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 -- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 msgr2=0x7f0e200af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:18.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e200af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 -- 172.21.15.148:0/2811588208 >> v1:172.21.15.161:6789/0 conn(0x7f0e2006a5a0 legacy=0x7f0e2015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 -- 172.21.15.148:0/2811588208 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200b04b0 con 0x7f0e2015a290 2023-10-10T20:21:18.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1effd700 1 -- 172.21.15.148:0/2811588208 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102552054 0 0) 0x7f0e200b0790 con 0x7f0e2006a5a0 2023-10-10T20:21:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1ffff700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e200af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1f7fe700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e200afe80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098e0 tx=0x7f0e140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfdb23e638ba782e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1effd700 1 -- 172.21.15.148:0/2811588208 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400b640 con 0x7f0e2015a290 2023-10-10T20:21:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.721+0000 7f0e1effd700 1 -- 172.21.15.148:0/2811588208 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400b7a0 con 0x7f0e2015a290 2023-10-10T20:21:18.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e1effd700 1 -- 172.21.15.148:0/2811588208 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400bb40 con 0x7f0e2015a290 2023-10-10T20:21:18.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 -- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 msgr2=0x7f0e200afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e200afe80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e140098e0 tx=0x7f0e140091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 -- 172.21.15.148:0/2811588208 shutdown_connections 2023-10-10T20:21:18.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e200af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 --2- 172.21.15.148:0/2811588208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e200afe80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 -- 172.21.15.148:0/2811588208 >> 172.21.15.148:0/2811588208 conn(0x7f0e200d1880 msgr2=0x7f0e200cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:18.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 -- 172.21.15.148:0/2811588208 shutdown_connections 2023-10-10T20:21:18.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.722+0000 7f0e26442700 1 -- 172.21.15.148:0/2811588208 wait complete. 2023-10-10T20:21:18.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.723+0000 7f0e26442700 1 Processor -- start 2023-10-10T20:21:18.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.723+0000 7f0e26442700 1 -- start start 2023-10-10T20:21:18.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e2014f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e20155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20152120 con 0x7f0e2006a5a0 2023-10-10T20:21:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2014e670 con 0x7f0e2015a290 2023-10-10T20:21:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e26442700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20151b60 con 0x7f0e20159ec0 2023-10-10T20:21:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34648/0 (socket says 172.21.15.148:34648) 2023-10-10T20:21:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 -- 172.21.15.148:0/444790226 learned_addr learned my addr 172.21.15.148:0/444790226 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e249df700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e20155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1f7fe700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e2014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 -- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 msgr2=0x7f0e2014f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e2014f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.724+0000 7f0e1ffff700 1 -- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 msgr2=0x7f0e20155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e1ffff700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e20155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e1ffff700 1 -- 172.21.15.148:0/444790226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10009420 con 0x7f0e2006a5a0 2023-10-10T20:21:18.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e1ffff700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000ecb0 tx=0x7f0e100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:18.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000c920 con 0x7f0e2006a5a0 2023-10-10T20:21:18.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000ca80 con 0x7f0e2006a5a0 2023-10-10T20:21:18.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14009420 con 0x7f0e2006a5a0 2023-10-10T20:21:18.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000ce00 con 0x7f0e2006a5a0 2023-10-10T20:21:18.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.725+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e200c8000 con 0x7f0e2006a5a0 2023-10-10T20:21:18.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.726+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e200c8930 con 0x7f0e2006a5a0 2023-10-10T20:21:18.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.726+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e10013680 con 0x7f0e2006a5a0 2023-10-10T20:21:18.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.726+0000 7f0e07fff700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 0x7f0e0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:18.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.727+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0e10084500 con 0x7f0e2006a5a0 2023-10-10T20:21:18.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.727+0000 7f0e1f7fe700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 0x7f0e0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:18.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.727+0000 7f0e1f7fe700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 0x7f0e0c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e140093f0 tx=0x7f0e14017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:18.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.731+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e10051a70 con 0x7f0e2006a5a0 2023-10-10T20:21:18.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.920+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0e2004c420 con 0x7f0e2006a5a0 2023-10-10T20:21:18.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.924+0000 7f0e07fff700 1 -- 172.21.15.148:0/444790226 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e10055090 con 0x7f0e2006a5a0 2023-10-10T20:21:18.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 msgr2=0x7f0e0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 0x7f0e0c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0e140093f0 tx=0x7f0e14017000 comp rx=0 tx=0).stop 2023-10-10T20:21:18.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 msgr2=0x7f0e2014e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:18.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000ecb0 tx=0x7f0e100091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 shutdown_connections 2023-10-10T20:21:18.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0e0c066f30 0x7f0e0c0693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2015a290 0x7f0e20155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2006a5a0 0x7f0e2014e130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 --2- 172.21.15.148:0/444790226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e20159ec0 0x7f0e2014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:18.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 >> 172.21.15.148:0/444790226 conn(0x7f0e200d1880 msgr2=0x7f0e200b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:18.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 shutdown_connections 2023-10-10T20:21:18.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:18.926+0000 7f0e26442700 1 -- 172.21.15.148:0/444790226 wait complete. 2023-10-10T20:21:18.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-10T20:21:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 Processor -- start 2023-10-10T20:21:19.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 -- start start 2023-10-10T20:21:19.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 0x7f14e4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e415cbb0 con 0x7f14e415a620 2023-10-10T20:21:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e415bfa0 con 0x7f14e4157870 2023-10-10T20:21:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.052+0000 7f14e8f2e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e415bdb0 con 0x7f14e4157c40 2023-10-10T20:21:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14d9d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f14e415a620 0x7f14e415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37306/0 (socket says 172.21.15.148:37306) 2023-10-10T20:21:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14d9d9b700 1 -- 172.21.15.148:0/762162216 learned_addr learned my addr 172.21.15.148:0/762162216 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e2d9d700 1 --2- 172.21.15.148:0/762162216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 0x7f14e4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541178394 0 0) 0x7f14e415cbb0 con 0x7f14e415a620 2023-10-10T20:21:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d0003570 con 0x7f14e415a620 2023-10-10T20:21:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132215890 0 0) 0x7f14e415bfa0 con 0x7f14e4157870 2023-10-10T20:21:19.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e415cbb0 con 0x7f14e4157870 2023-10-10T20:21:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540730392 0 0) 0x7f14d0003570 con 0x7f14e415a620 2023-10-10T20:21:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e2d9d700 1 -- 172.21.15.148:0/762162216 >> v1:172.21.15.161:6789/0 conn(0x7f14e4157870 legacy=0x7f14e415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e2d9d700 1 -- 172.21.15.148:0/762162216 >> v1:172.21.15.148:6789/0 conn(0x7f14e415a620 legacy=0x7f14e415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e2d9d700 1 -- 172.21.15.148:0/762162216 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e415c280 con 0x7f14e4157c40 2023-10-10T20:21:19.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.053+0000 7f14e2d9d700 1 --2- 172.21.15.148:0/762162216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 0x7f14e4158110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e90 tx=0x7f14dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=723316bfa76a4f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00e070 con 0x7f14e4157c40 2023-10-10T20:21:19.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc00b870 con 0x7f14e4157c40 2023-10-10T20:21:19.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e1d9b700 1 -- 172.21.15.148:0/762162216 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc001470 con 0x7f14e4157c40 2023-10-10T20:21:19.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 -- 172.21.15.148:0/762162216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 msgr2=0x7f14e4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/762162216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 0x7f14e4158110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e90 tx=0x7f14dc009780 comp rx=0 tx=0).stop 2023-10-10T20:21:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 -- 172.21.15.148:0/762162216 shutdown_connections 2023-10-10T20:21:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/762162216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e4157c40 0x7f14e4158110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 -- 172.21.15.148:0/762162216 >> 172.21.15.148:0/762162216 conn(0x7f14e4068ce0 msgr2=0x7f14e40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 -- 172.21.15.148:0/762162216 shutdown_connections 2023-10-10T20:21:19.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.054+0000 7f14e8f2e700 1 -- 172.21.15.148:0/762162216 wait complete. 2023-10-10T20:21:19.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.055+0000 7f14e8f2e700 1 Processor -- start 2023-10-10T20:21:19.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.055+0000 7f14e8f2e700 1 -- start start 2023-10-10T20:21:19.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14e4157870 0x7f14e4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e415a620 0x7f14e40b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4153d70 con 0x7f14e4157c40 2023-10-10T20:21:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4153440 con 0x7f14e4157870 2023-10-10T20:21:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e8f2e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4153720 con 0x7f14e415a620 2023-10-10T20:21:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14d9d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14d9d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34650/0 (socket says 172.21.15.148:34650) 2023-10-10T20:21:19.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14d9d9b700 1 -- 172.21.15.148:0/2067363874 learned_addr learned my addr 172.21.15.148:0/2067363874 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:19.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e2d9d700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e415a620 0x7f14e40b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.056+0000 7f14e259c700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14e4157870 0x7f14e4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 -- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e415a620 msgr2=0x7f14e40b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e415a620 0x7f14e40b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 -- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14e4157870 msgr2=0x7f14e4150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14e4157870 0x7f14e4150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 -- 172.21.15.148:0/2067363874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009a20 con 0x7f14e4157c40 2023-10-10T20:21:19.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14d9d9b700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c860 tx=0x7f14cc001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc005b30 con 0x7f14e4157c40 2023-10-10T20:21:19.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc005c90 con 0x7f14e4157c40 2023-10-10T20:21:19.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e4153a00 con 0x7f14e4157c40 2023-10-10T20:21:19.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc007340 con 0x7f14e4157c40 2023-10-10T20:21:19.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.057+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e414d390 con 0x7f14e4157c40 2023-10-10T20:21:19.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.058+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e4153f20 con 0x7f14e4157c40 2023-10-10T20:21:19.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.058+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14cc01b400 con 0x7f14e4157c40 2023-10-10T20:21:19.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.059+0000 7f14daffd700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.059+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f14cc083d80 con 0x7f14e4157c40 2023-10-10T20:21:19.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.059+0000 7f14e259c700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.059+0000 7f14e259c700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007fd0 tx=0x7f14d4007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.063+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14e4153f20 con 0x7f14e4157c40 2023-10-10T20:21:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.320+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f14e40d1760 con 0x7f14e4157c40 2023-10-10T20:21:19.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.593+0000 7f14daffd700 1 -- 172.21.15.148:0/2067363874 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v126) v1 ==== 142+0+0 (secure 0 0 0) 0x7f14cc054e00 con 0x7f14e4157c40 2023-10-10T20:21:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 msgr2=0x7f14c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 0x7f14c40693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007fd0 tx=0x7f14d4007d50 comp rx=0 tx=0).stop 2023-10-10T20:21:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 msgr2=0x7f14e40b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c860 tx=0x7f14cc001930 comp rx=0 tx=0).stop 2023-10-10T20:21:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 shutdown_connections 2023-10-10T20:21:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14c4066f30 0x7f14c40693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14e415a620 0x7f14e40b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14e4157870 0x7f14e4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 --2- 172.21.15.148:0/2067363874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14e4157c40 0x7f14e40b1690 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.595+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 >> 172.21.15.148:0/2067363874 conn(0x7f14e4068ce0 msgr2=0x7f14e40d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.596+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 shutdown_connections 2023-10-10T20:21:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.596+0000 7f14e8f2e700 1 -- 172.21.15.148:0/2067363874 wait complete. 2023-10-10T20:21:19.609 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-10T20:21:19.610 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-10-10T20:21:19.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-10T20:21:19.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.723+0000 7f49ddd66700 1 Processor -- start 2023-10-10T20:21:19.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.723+0000 7f49ddd66700 1 -- start start 2023-10-10T20:21:19.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49ddd66700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d8156700 0x7f49d8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49ddd66700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d81573c0 con 0x7f49d8067d10 2023-10-10T20:21:19.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49ddd66700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80c7e90 con 0x7f49d8067870 2023-10-10T20:21:19.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49ddd66700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d8068140 con 0x7f49d8156700 2023-10-10T20:21:19.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49dcd64700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d8156700 0x7f49d8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49dcd64700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49d8156700 0x7f49d8156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34654/0 (socket says 172.21.15.148:34654) 2023-10-10T20:21:19.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49dd565700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49d8067d10 0x7f49d806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37310/0 (socket says 172.21.15.148:37310) 2023-10-10T20:21:19.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.724+0000 7f49dcd64700 1 -- 172.21.15.148:0/409406081 learned_addr learned my addr 172.21.15.148:0/409406081 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:19.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49dcd64700 1 -- 172.21.15.148:0/409406081 >> v1:172.21.15.161:6789/0 conn(0x7f49d8067870 legacy=0x7f49d80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49dcd64700 1 -- 172.21.15.148:0/409406081 >> v1:172.21.15.148:6789/0 conn(0x7f49d8067d10 legacy=0x7f49d806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49dcd64700 1 -- 172.21.15.148:0/409406081 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80c8450 con 0x7f49d8156700 2023-10-10T20:21:19.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49cf7fe700 1 -- 172.21.15.148:0/409406081 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60036821 0 0) 0x7f49d81573c0 con 0x7f49d8067d10 2023-10-10T20:21:19.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49dcd64700 1 --2- 172.21.15.148:0/409406081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8156ad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f49d400bc50 tx=0x7f49d400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0619499cb423292 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49cf7fe700 1 -- 172.21.15.148:0/409406081 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400be60 con 0x7f49d8156700 2023-10-10T20:21:19.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49cf7fe700 1 -- 172.21.15.148:0/409406081 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400de00 con 0x7f49d8156700 2023-10-10T20:21:19.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49cf7fe700 1 -- 172.21.15.148:0/409406081 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40196b0 con 0x7f49d8156700 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 -- 172.21.15.148:0/409406081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 msgr2=0x7f49d8156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 --2- 172.21.15.148:0/409406081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8156ad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f49d400bc50 tx=0x7f49d400b4f0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 -- 172.21.15.148:0/409406081 shutdown_connections 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 --2- 172.21.15.148:0/409406081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8156ad0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 -- 172.21.15.148:0/409406081 >> 172.21.15.148:0/409406081 conn(0x7f49d80cd8f0 msgr2=0x7f49d80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 -- 172.21.15.148:0/409406081 shutdown_connections 2023-10-10T20:21:19.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.725+0000 7f49ddd66700 1 -- 172.21.15.148:0/409406081 wait complete. 2023-10-10T20:21:19.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.726+0000 7f49ddd66700 1 Processor -- start 2023-10-10T20:21:19.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 -- start start 2023-10-10T20:21:19.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 0x7f49d80ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d8067d10 0x7f49d81549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8155470 con 0x7f49d8156700 2023-10-10T20:21:19.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8166370 con 0x7f49d8067d10 2023-10-10T20:21:19.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.727+0000 7f49ddd66700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80a8210 con 0x7f49d8067870 2023-10-10T20:21:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34670/0 (socket says 172.21.15.148:34670) 2023-10-10T20:21:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 -- 172.21.15.148:0/2548113160 learned_addr learned my addr 172.21.15.148:0/2548113160 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dcd64700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 0x7f49d80ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49cffff700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d8067d10 0x7f49d81549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 -- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 msgr2=0x7f49d80ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 0x7f49d80ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 -- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d8067d10 msgr2=0x7f49d81549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:19.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d8067d10 0x7f49d81549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:19.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 -- 172.21.15.148:0/2548113160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d400b790 con 0x7f49d8156700 2023-10-10T20:21:19.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dcd64700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 0x7f49d80ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:19.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49dd565700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49d000cd00 tx=0x7f49d0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0005df0 con 0x7f49d8156700 2023-10-10T20:21:19.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d000a960 con 0x7f49d8156700 2023-10-10T20:21:19.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80a84f0 con 0x7f49d8156700 2023-10-10T20:21:19.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d000ad20 con 0x7f49d8156700 2023-10-10T20:21:19.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.728+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80a8ab0 con 0x7f49d8156700 2023-10-10T20:21:19.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.729+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8166590 con 0x7f49d8156700 2023-10-10T20:21:19.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.730+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49d0007300 con 0x7f49d8156700 2023-10-10T20:21:19.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.730+0000 7f49ccff9700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 0x7f49c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:19.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.730+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f49d00843c0 con 0x7f49d8156700 2023-10-10T20:21:19.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.730+0000 7f49dcd64700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 0x7f49c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:19.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.730+0000 7f49dcd64700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 0x7f49c40693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49d400bc50 tx=0x7f49d401b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:19.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.734+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49d00519b0 con 0x7f49d8156700 2023-10-10T20:21:19.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:19.991+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f49d806eb70 con 0x7f49d8156700 2023-10-10T20:21:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.594+0000 7f49ccff9700 1 -- 172.21.15.148:0/2548113160 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v127) v1 ==== 137+0+0 (secure 0 0 0) 0x7f49d0054fd0 con 0x7f49d8156700 2023-10-10T20:21:20.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 msgr2=0x7f49c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 0x7f49c40693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49d400bc50 tx=0x7f49d401b000 comp rx=0 tx=0).stop 2023-10-10T20:21:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 msgr2=0x7f49d8154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49d000cd00 tx=0x7f49d0001ac0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 shutdown_connections 2023-10-10T20:21:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f49c4066f30 0x7f49c40693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49d8156700 0x7f49d8154f30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d8067870 0x7f49d80ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 --2- 172.21.15.148:0/2548113160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d8067d10 0x7f49d81549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 >> 172.21.15.148:0/2548113160 conn(0x7f49d80cd8f0 msgr2=0x7f49d8157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 shutdown_connections 2023-10-10T20:21:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.596+0000 7f49ddd66700 1 -- 172.21.15.148:0/2548113160 wait complete. 2023-10-10T20:21:20.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-10T20:21:20.740 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_objectstore":"memstore"} 2023-10-10T20:21:20.740 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-10T20:21:20.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:21:20.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.857+0000 7f3acf7b9700 1 Processor -- start 2023-10-10T20:21:20.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.857+0000 7f3acf7b9700 1 -- start start 2023-10-10T20:21:20.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acf7b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac80caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acf7b9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 0x7f3ac80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acf7b9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80c3430 con 0x7f3ac80ab910 2023-10-10T20:21:20.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acf7b9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac80c9100 con 0x7f3ac80abce0 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acf7b9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8158050 con 0x7f3ac815a730 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3accd54700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac80caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acdd56700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 0x7f3ac80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acd555700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3ac80ab910 0x7f3ac80c2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44342/0 (socket says 172.21.15.148:44342) 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.858+0000 7f3acd555700 1 -- 172.21.15.148:0/4096415232 learned_addr learned my addr 172.21.15.148:0/4096415232 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:20.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3abffff700 1 -- 172.21.15.148:0/4096415232 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430558964 0 0) 0x7f3ac80c3430 con 0x7f3ac80ab910 2023-10-10T20:21:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3abffff700 1 -- 172.21.15.148:0/4096415232 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab4003570 con 0x7f3ac80ab910 2023-10-10T20:21:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3accd54700 1 -- 172.21.15.148:0/4096415232 >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 msgr2=0x7f3ac80cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3accd54700 1 --2- 172.21.15.148:0/4096415232 >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 0x7f3ac80cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3accd54700 1 -- 172.21.15.148:0/4096415232 >> v1:172.21.15.161:6789/0 conn(0x7f3ac80ab910 legacy=0x7f3ac80c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3accd54700 1 -- 172.21.15.148:0/4096415232 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac815abc0 con 0x7f3ac815a730 2023-10-10T20:21:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3acdd56700 1 --2- 172.21.15.148:0/4096415232 >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 0x7f3ac80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3accd54700 1 --2- 172.21.15.148:0/4096415232 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac80caf50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80098e0 tx=0x7f3ab80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cde0129f2ce2d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3abffff700 1 -- 172.21.15.148:0/4096415232 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011630 con 0x7f3ac815a730 2023-10-10T20:21:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3abffff700 1 -- 172.21.15.148:0/4096415232 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8011790 con 0x7f3ac815a730 2023-10-10T20:21:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.859+0000 7f3abffff700 1 -- 172.21.15.148:0/4096415232 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011b10 con 0x7f3ac815a730 2023-10-10T20:21:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 -- 172.21.15.148:0/4096415232 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 msgr2=0x7f3ac80caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/4096415232 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac80caf50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80098e0 tx=0x7f3ab80091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 -- 172.21.15.148:0/4096415232 shutdown_connections 2023-10-10T20:21:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/4096415232 >> v2:172.21.15.161:3300/0 conn(0x7f3ac80abce0 0x7f3ac80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/4096415232 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac80caf50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 -- 172.21.15.148:0/4096415232 >> 172.21.15.148:0/4096415232 conn(0x7f3ac8067660 msgr2=0x7f3ac8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 -- 172.21.15.148:0/4096415232 shutdown_connections 2023-10-10T20:21:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.860+0000 7f3acf7b9700 1 -- 172.21.15.148:0/4096415232 wait complete. 2023-10-10T20:21:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.861+0000 7f3acf7b9700 1 Processor -- start 2023-10-10T20:21:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.861+0000 7f3acf7b9700 1 -- start start 2023-10-10T20:21:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ac80ab910 0x7f3ac80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80ad230 con 0x7f3ac80abce0 2023-10-10T20:21:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8069170 con 0x7f3ac80ab910 2023-10-10T20:21:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acf7b9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8069fd0 con 0x7f3ac815a730 2023-10-10T20:21:20.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acd555700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ac80ab910 0x7f3ac80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34684/0 (socket says 172.21.15.148:34684) 2023-10-10T20:21:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 -- 172.21.15.148:0/2475485840 learned_addr learned my addr 172.21.15.148:0/2475485840 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acdd56700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 -- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 msgr2=0x7f3ac8152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac8152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 -- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ac80ab910 msgr2=0x7f3ac80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ac80ab910 0x7f3ac80b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 -- 172.21.15.148:0/2475485840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0009a20 con 0x7f3ac80abce0 2023-10-10T20:21:20.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3acdd56700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:20.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.862+0000 7f3accd54700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8011f00 tx=0x7f3ab8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:20.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8011b10 con 0x7f3ac80abce0 2023-10-10T20:21:20.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8000d10 con 0x7f3ac80abce0 2023-10-10T20:21:20.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab8009420 con 0x7f3ac80abce0 2023-10-10T20:21:20.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab801a680 con 0x7f3ac80abce0 2023-10-10T20:21:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac8069470 con 0x7f3ac80abce0 2023-10-10T20:21:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.863+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac804c3c0 con 0x7f3ac80abce0 2023-10-10T20:21:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.864+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ab801aa70 con 0x7f3ac80abce0 2023-10-10T20:21:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.864+0000 7f3abd7fa700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.864+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3ab8085cd0 con 0x7f3ac80abce0 2023-10-10T20:21:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.864+0000 7f3acd555700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.865+0000 7f3acd555700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 0x7f3aac0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:20.869+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ab8053300 con 0x7f3ac80abce0 2023-10-10T20:21:21.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.052+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3ac806a110 con 0x7f3ac80abce0 2023-10-10T20:21:21.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.055+0000 7f3abd7fa700 1 -- 172.21.15.148:0/2475485840 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3ab80150c0 con 0x7f3ac80abce0 2023-10-10T20:21:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 msgr2=0x7f3aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 0x7f3aac0693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4009040 comp rx=0 tx=0).stop 2023-10-10T20:21:21.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 msgr2=0x7f3ac806a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8011f00 tx=0x7f3ab8017000 comp rx=0 tx=0).stop 2023-10-10T20:21:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 shutdown_connections 2023-10-10T20:21:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3aac066f30 0x7f3aac0693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ac815a730 0x7f3ac8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ac80ab910 0x7f3ac80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 --2- 172.21.15.148:0/2475485840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac80abce0 0x7f3ac806a460 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 >> 172.21.15.148:0/2475485840 conn(0x7f3ac8067660 msgr2=0x7f3ac8157860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 shutdown_connections 2023-10-10T20:21:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.058+0000 7f3acf7b9700 1 -- 172.21.15.148:0/2475485840 wait complete. 2023-10-10T20:21:21.072 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:21:21.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:21:21.117 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:21:21.162 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:21:21.163 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:21:21.163 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:21.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.275+0000 7fb24e8f8700 1 Processor -- start 2023-10-10T20:21:21.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.275+0000 7fb24e8f8700 1 -- start start 2023-10-10T20:21:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24e8f8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb24815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24e8f8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24815cbb0 con 0x7fb248157c40 2023-10-10T20:21:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24e8f8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24815bfa0 con 0x7fb24815a620 2023-10-10T20:21:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24e8f8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24815c280 con 0x7fb248157870 2023-10-10T20:21:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24ce95700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb248157c40 0x7fb248158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37322/0 (socket says 172.21.15.148:37322) 2023-10-10T20:21:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb24ce95700 1 -- 172.21.15.148:0/2780924914 learned_addr learned my addr 172.21.15.148:0/2780924914 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb247fff700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb24815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461530664 0 0) 0x7fb24815cbb0 con 0x7fb248157c40 2023-10-10T20:21:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.276+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb230003570 con 0x7fb248157c40 2023-10-10T20:21:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2444317388 0 0) 0x7fb230003570 con 0x7fb248157c40 2023-10-10T20:21:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 msgr2=0x7fb24815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb24815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 >> v1:172.21.15.161:6789/0 conn(0x7fb24815a620 legacy=0x7fb24815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24815c560 con 0x7fb248157c40 2023-10-10T20:21:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fb2400017f0 con 0x7fb248157c40 2023-10-10T20:21:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 >> v1:172.21.15.148:6789/0 conn(0x7fb248157c40 legacy=0x7fb248158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb230003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb230003fd0 0x7fb230004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb230004d50 0x7fb2300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24815c280 con 0x7fb230003fd0 2023-10-10T20:21:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23000bb40 con 0x7fb230004d50 2023-10-10T20:21:21.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23000bd00 con 0x7fb248157870 2023-10-10T20:21:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb230003fd0 0x7fb230004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb247fff700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb230003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb24ce95700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb230004d50 0x7fb2300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 -- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 msgr2=0x7fb230003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb248157870 0x7fb230003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 -- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb230004d50 msgr2=0x7fb2300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb230004d50 0x7fb2300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 -- 172.21.15.148:0/2780924914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230003570 con 0x7fb230003fd0 2023-10-10T20:21:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb24ce95700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb230004d50 0x7fb2300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:21.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.277+0000 7fb2477fe700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb230003fd0 0x7fb230004810 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0019a0 tx=0x7fb23c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfd432f7a9a2d66 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00c5d0 con 0x7fb230003fd0 2023-10-10T20:21:21.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c00c750 con 0x7fb230003fd0 2023-10-10T20:21:21.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb246ffd700 1 -- 172.21.15.148:0/2780924914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00cb10 con 0x7fb230003fd0 2023-10-10T20:21:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 -- 172.21.15.148:0/2780924914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb230003fd0 msgr2=0x7fb230004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/2780924914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb230003fd0 0x7fb230004810 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0019a0 tx=0x7fb23c007d10 comp rx=0 tx=0).stop 2023-10-10T20:21:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24ce95700 1 -- 172.21.15.148:0/2780924914 reap_dead start 2023-10-10T20:21:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 -- 172.21.15.148:0/2780924914 shutdown_connections 2023-10-10T20:21:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 -- 172.21.15.148:0/2780924914 >> 172.21.15.148:0/2780924914 conn(0x7fb248068ce0 msgr2=0x7fb2480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:21.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 -- 172.21.15.148:0/2780924914 shutdown_connections 2023-10-10T20:21:21.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.278+0000 7fb24e8f8700 1 -- 172.21.15.148:0/2780924914 wait complete. 2023-10-10T20:21:21.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.279+0000 7fb24e8f8700 1 Processor -- start 2023-10-10T20:21:21.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.279+0000 7fb24e8f8700 1 -- start start 2023-10-10T20:21:21.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb230003fd0 0x7fb2481522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2481502d0 0x7fb248151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb248152000 con 0x7fb24813ee30 2023-10-10T20:21:21.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b1dd0 con 0x7fb2481502d0 2023-10-10T20:21:21.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24e8f8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb248150a30 con 0x7fb230003fd0 2023-10-10T20:21:21.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34706/0 (socket says 172.21.15.148:34706) 2023-10-10T20:21:21.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 -- 172.21.15.148:0/138921921 learned_addr learned my addr 172.21.15.148:0/138921921 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:21.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 -- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb230003fd0 msgr2=0x7fb2481522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb230003fd0 0x7fb2481522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 -- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2481502d0 msgr2=0x7fb248151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2481502d0 0x7fb248151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.280+0000 7fb24ce95700 1 -- 172.21.15.148:0/138921921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480b1780 con 0x7fb24813ee30 2023-10-10T20:21:21.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb24ce95700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb2480b3830 tx=0x7fb240002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000eb60 con 0x7fb24813ee30 2023-10-10T20:21:21.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24000ece0 con 0x7fb24813ee30 2023-10-10T20:21:21.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2480b1a60 con 0x7fb24813ee30 2023-10-10T20:21:21.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240013670 con 0x7fb24813ee30 2023-10-10T20:21:21.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.281+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24813f2b0 con 0x7fb24813ee30 2023-10-10T20:21:21.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.282+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2480b15e0 con 0x7fb24813ee30 2023-10-10T20:21:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.282+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2400139e0 con 0x7fb24813ee30 2023-10-10T20:21:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.282+0000 7fb22ffff700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 0x7fb2340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.282+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb240084840 con 0x7fb24813ee30 2023-10-10T20:21:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.282+0000 7fb247fff700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 0x7fb2340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.283+0000 7fb247fff700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 0x7fb2340693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2480b3150 tx=0x7fb2380091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.287+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2480b15e0 con 0x7fb24813ee30 2023-10-10T20:21:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.513+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2480b15e0 con 0x7fb24813ee30 2023-10-10T20:21:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.514+0000 7fb22ffff700 1 -- 172.21.15.148:0/138921921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fb2480b15e0 con 0x7fb24813ee30 2023-10-10T20:21:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:21.515 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3724611102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3724611102},{"type":"v1","addr":"172.21.15.148:6837","nonce":3724611102}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3355108597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3355108597},{"type":"v1","addr":"172.21.15.161:6834","nonce":3355108597}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5057,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1335120257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1335120257},{"type":"v1","addr":"172.21.15.161:6835","nonce":1335120257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/333908619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":333908619},{"type":"v1","addr":"172.21.15.148:6836","nonce":333908619}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-10-10T20:21:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 msgr2=0x7fb2340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 0x7fb2340693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2480b3150 tx=0x7fb2380091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 msgr2=0x7fb24814d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb2480b3830 tx=0x7fb240002160 comp rx=0 tx=0).stop 2023-10-10T20:21:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 shutdown_connections 2023-10-10T20:21:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb234066f30 0x7fb2340693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb24813ee30 0x7fb24814d8a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb230003fd0 0x7fb2481522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 --2- 172.21.15.148:0/138921921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2481502d0 0x7fb248151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 >> 172.21.15.148:0/138921921 conn(0x7fb248068ce0 msgr2=0x7fb2480b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 shutdown_connections 2023-10-10T20:21:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.516+0000 7fb24e8f8700 1 -- 172.21.15.148:0/138921921 wait complete. 2023-10-10T20:21:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 26 2023-10-10T20:21:21.529 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:21:21.529 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:21:21.530 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:21:21.530 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:21:21.530 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:21:21.531 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:21:21.531 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:21:21.531 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:21:21.531 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:21:21.532 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:21:21.532 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:21:21.532 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:21:21.532 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:21:21.533 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:21:21.533 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:21:21.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:21:21.538 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:21:21.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:21:21.540 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:21:21.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:21:21.542 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:21:21.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:21:21.543 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:21:21.544 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:21:21.582 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:21:21.582 DEBUG:teuthology.parallel:result is None 2023-10-10T20:21:21.583 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:21:21.584 DEBUG:teuthology.parallel:result is None 2023-10-10T20:21:21.586 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:21:21.586 DEBUG:teuthology.parallel:result is None 2023-10-10T20:21:21.588 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:21:21.588 DEBUG:teuthology.parallel:result is None 2023-10-10T20:21:21.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:21:21.656 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:21:21.656 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:21:21.657 INFO:tasks.ceph.mds.c.smithi148.stderr:2023-10-10T20:21:21.655+0000 7fbcfda9f700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-10T20:21:21.666 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:21:21.666 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:21:21.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.710+0000 7f97b0902700 1 Processor -- start 2023-10-10T20:21:21.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.710+0000 7f97b0902700 1 -- start start 2023-10-10T20:21:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97b0902700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97ac15a620 0x7f97ac15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97b0902700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac15cbb0 con 0x7f97ac15d580 2023-10-10T20:21:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97b0902700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac15bfa0 con 0x7f97ac157870 2023-10-10T20:21:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97b0902700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac15bdb0 con 0x7f97ac15a620 2023-10-10T20:21:21.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a1d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97ac15a620 0x7f97ac15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97aad9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97ac15d580 0x7f97ac158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37336/0 (socket says 172.21.15.148:37336) 2023-10-10T20:21:21.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97aad9d700 1 -- 172.21.15.148:0/1451277896 learned_addr learned my addr 172.21.15.148:0/1451277896 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:21.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a1d9b700 1 -- 172.21.15.148:0/1451277896 >> v1:172.21.15.161:6789/0 conn(0x7f97ac157870 legacy=0x7f97ac15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a1d9b700 1 -- 172.21.15.148:0/1451277896 >> v1:172.21.15.148:6789/0 conn(0x7f97ac15d580 legacy=0x7f97ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a1d9b700 1 -- 172.21.15.148:0/1451277896 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac15c280 con 0x7f97ac15a620 2023-10-10T20:21:21.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a9d9b700 1 -- 172.21.15.148:0/1451277896 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413158018 0 0) 0x7f97ac15cbb0 con 0x7f97ac15d580 2023-10-10T20:21:21.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a1d9b700 1 --2- 172.21.15.148:0/1451277896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac15d040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f97940098e0 tx=0x7f97940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730d5d4b989c5895 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a9d9b700 1 -- 172.21.15.148:0/1451277896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794009e80 con 0x7f97ac15a620 2023-10-10T20:21:21.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a9d9b700 1 -- 172.21.15.148:0/1451277896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979400b820 con 0x7f97ac15a620 2023-10-10T20:21:21.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.711+0000 7f97a9d9b700 1 -- 172.21.15.148:0/1451277896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400bba0 con 0x7f97ac15a620 2023-10-10T20:21:21.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 -- 172.21.15.148:0/1451277896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 msgr2=0x7f97ac15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 --2- 172.21.15.148:0/1451277896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac15d040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f97940098e0 tx=0x7f97940091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 -- 172.21.15.148:0/1451277896 shutdown_connections 2023-10-10T20:21:21.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 --2- 172.21.15.148:0/1451277896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac15d040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 -- 172.21.15.148:0/1451277896 >> 172.21.15.148:0/1451277896 conn(0x7f97ac068ce0 msgr2=0x7f97ac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:21.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 -- 172.21.15.148:0/1451277896 shutdown_connections 2023-10-10T20:21:21.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.712+0000 7f97b0902700 1 -- 172.21.15.148:0/1451277896 wait complete. 2023-10-10T20:21:21.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.713+0000 7f97b0902700 1 Processor -- start 2023-10-10T20:21:21.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.713+0000 7f97b0902700 1 -- start start 2023-10-10T20:21:21.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ac157870 0x7f97ac0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97ac15d580 0x7f97ac15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1522c0 con 0x7f97ac15a620 2023-10-10T20:21:21.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0cfed0 con 0x7f97ac157870 2023-10-10T20:21:21.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97b0902700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac152510 con 0x7f97ac15d580 2023-10-10T20:21:21.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34744/0 (socket says 172.21.15.148:34744) 2023-10-10T20:21:21.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 -- 172.21.15.148:0/4058639696 learned_addr learned my addr 172.21.15.148:0/4058639696 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:21.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97aa59c700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ac157870 0x7f97ac0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97aad9d700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97ac15d580 0x7f97ac15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 -- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97ac15d580 msgr2=0x7f97ac15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97ac15d580 0x7f97ac15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 -- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ac157870 msgr2=0x7f97ac0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:21.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ac157870 0x7f97ac0b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:21.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.714+0000 7f97a1d9b700 1 -- 172.21.15.148:0/4058639696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009420 con 0x7f97ac15a620 2023-10-10T20:21:21.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97a1d9b700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97940030c0 tx=0x7f9794016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400bba0 con 0x7f97ac15a620 2023-10-10T20:21:21.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979401b470 con 0x7f97ac15a620 2023-10-10T20:21:21.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac1527f0 con 0x7f97ac15a620 2023-10-10T20:21:21.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979401b7f0 con 0x7f97ac15a620 2023-10-10T20:21:21.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.715+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac0d0360 con 0x7f97ac15a620 2023-10-10T20:21:21.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.716+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac1529e0 con 0x7f97ac15a620 2023-10-10T20:21:21.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.716+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f979401bb60 con 0x7f97ac15a620 2023-10-10T20:21:21.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.716+0000 7f97a2ffd700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:21.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.717+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9794085e80 con 0x7f97ac15a620 2023-10-10T20:21:21.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.717+0000 7f97aa59c700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:21.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.717+0000 7f97aa59c700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 0x7f978c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f979c003a80 tx=0x7f979c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:21.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.721+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97940533f0 con 0x7f97ac15a620 2023-10-10T20:21:21.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:21.919+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97ac04c3c0 con 0x7f97ac15a620 2023-10-10T20:21:22.539 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:21:22.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:21:23.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.076+0000 7f97a2ffd700 1 -- 172.21.15.148:0/4058639696 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7f979402a080 con 0x7f97ac15a620 2023-10-10T20:21:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 msgr2=0x7f978c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 0x7f978c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f979c003a80 tx=0x7f979c007ec0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 msgr2=0x7f97ac14cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97940030c0 tx=0x7f9794016000 comp rx=0 tx=0).stop 2023-10-10T20:21:23.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 shutdown_connections 2023-10-10T20:21:23.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f978c066f30 0x7f978c0693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97ac15d580 0x7f97ac15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ac157870 0x7f97ac0b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 --2- 172.21.15.148:0/4058639696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97ac15a620 0x7f97ac14cb40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 >> 172.21.15.148:0/4058639696 conn(0x7f97ac068ce0 msgr2=0x7f97ac156a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 shutdown_connections 2023-10-10T20:21:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.078+0000 7f97b0902700 1 -- 172.21.15.148:0/4058639696 wait complete. 2023-10-10T20:21:23.082 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:21:23.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:21:23.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.206+0000 7f643cd6a700 1 Processor -- start 2023-10-10T20:21:23.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.206+0000 7f643cd6a700 1 -- start start 2023-10-10T20:21:23.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643cd6a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 0x7f643815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643cd6a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f643815d580 0x7f6438158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643cd6a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643815cbb0 con 0x7f6438157870 2023-10-10T20:21:23.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643cd6a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643815bdb0 con 0x7f643815d580 2023-10-10T20:21:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643cd6a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643815bfa0 con 0x7f643815a620 2023-10-10T20:21:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f6436d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f643815d580 0x7f6438158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6438157870 0x7f643815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58612/0 (socket says 172.21.15.148:58612) 2023-10-10T20:21:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f643659c700 1 -- 172.21.15.148:0/1348476978 learned_addr learned my addr 172.21.15.148:0/1348476978 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.207+0000 7f642dd9b700 1 --2- 172.21.15.148:0/1348476978 >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 0x7f643815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6436d9d700 1 -- 172.21.15.148:0/1348476978 >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 msgr2=0x7f643815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:23.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6436d9d700 1 --2- 172.21.15.148:0/1348476978 >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 0x7f643815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6436d9d700 1 -- 172.21.15.148:0/1348476978 >> v1:172.21.15.148:6789/0 conn(0x7f6438157870 legacy=0x7f643815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6436d9d700 1 -- 172.21.15.148:0/1348476978 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643815c280 con 0x7f643815d580 2023-10-10T20:21:23.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6435d9b700 1 -- 172.21.15.148:0/1348476978 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093085170 0 0) 0x7f643815cbb0 con 0x7f6438157870 2023-10-10T20:21:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f642dd9b700 1 --2- 172.21.15.148:0/1348476978 >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 0x7f643815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6436d9d700 1 --2- 172.21.15.148:0/1348476978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438158110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f643815cb20 tx=0x7f6430009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705680bc3ce325d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6435d9b700 1 -- 172.21.15.148:0/1348476978 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000e070 con 0x7f643815d580 2023-10-10T20:21:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6435d9b700 1 -- 172.21.15.148:0/1348476978 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643000bc30 con 0x7f643815d580 2023-10-10T20:21:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f6435d9b700 1 -- 172.21.15.148:0/1348476978 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430012430 con 0x7f643815d580 2023-10-10T20:21:23.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 -- 172.21.15.148:0/1348476978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 msgr2=0x7f6438158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 --2- 172.21.15.148:0/1348476978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438158110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f643815cb20 tx=0x7f6430009780 comp rx=0 tx=0).stop 2023-10-10T20:21:23.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 -- 172.21.15.148:0/1348476978 shutdown_connections 2023-10-10T20:21:23.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 --2- 172.21.15.148:0/1348476978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438158110 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 --2- 172.21.15.148:0/1348476978 >> v2:172.21.15.161:3300/0 conn(0x7f643815a620 0x7f643815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 -- 172.21.15.148:0/1348476978 >> 172.21.15.148:0/1348476978 conn(0x7f6438068ce0 msgr2=0x7f64380690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 -- 172.21.15.148:0/1348476978 shutdown_connections 2023-10-10T20:21:23.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.208+0000 7f643cd6a700 1 -- 172.21.15.148:0/1348476978 wait complete. 2023-10-10T20:21:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 Processor -- start 2023-10-10T20:21:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 -- start start 2023-10-10T20:21:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 0x7f6438151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 0x7f6438152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438153540 con 0x7f643815d580 2023-10-10T20:21:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64381529c0 con 0x7f6438157870 2023-10-10T20:21:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.210+0000 7f643cd6a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6438152ca0 con 0x7f643815a620 2023-10-10T20:21:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37842/0 (socket says 172.21.15.148:37842) 2023-10-10T20:21:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 -- 172.21.15.148:0/2332955521 learned_addr learned my addr 172.21.15.148:0/2332955521 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f642dd9b700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 0x7f6438152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f643659c700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 0x7f6438151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 -- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 msgr2=0x7f6438152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 0x7f6438152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 -- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 msgr2=0x7f6438151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 0x7f6438151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 -- 172.21.15.148:0/2332955521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009a20 con 0x7f643815d580 2023-10-10T20:21:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f643659c700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 0x7f6438151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f642dd9b700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 0x7f6438152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f6436d9d700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f643806b530 tx=0x7f6430019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643001d630 con 0x7f643815d580 2023-10-10T20:21:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643001d790 con 0x7f643815d580 2023-10-10T20:21:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6438152f80 con 0x7f643815d580 2023-10-10T20:21:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643001db50 con 0x7f643815d580 2023-10-10T20:21:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.211+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643814bf90 con 0x7f643815d580 2023-10-10T20:21:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.212+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643804c3c0 con 0x7f643815d580 2023-10-10T20:21:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.213+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6430024070 con 0x7f643815d580 2023-10-10T20:21:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.213+0000 7f642effd700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 0x7f64180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.213+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6430086a80 con 0x7f643815d580 2023-10-10T20:21:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.213+0000 7f643659c700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 0x7f64180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.213+0000 7f643659c700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 0x7f64180693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64280011e0 tx=0x7f6428009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.217+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6430054040 con 0x7f643815d580 2023-10-10T20:21:23.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.471+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f643815c560 con 0x7f643815d580 2023-10-10T20:21:23.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.472+0000 7f642effd700 1 -- 172.21.15.148:0/2332955521 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6430057660 con 0x7f643815d580 2023-10-10T20:21:23.473 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:21:23.474 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:21:23.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.474 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:21:23.474 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:21:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:21:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:21:23.475 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.476 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:21:23.476 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:21:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:21:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:21:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:21:23.478 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:21:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:21:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.479 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.480 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:21:23.480 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:21:23.480 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.482 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:21:23.482 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:21:23.482 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.482 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.484 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:21:23.484 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:21:23.484 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.484 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.484 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:21:23.485 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:21:23.486 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.486 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.486 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.486 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.487 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:21:23.487 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:21:23.487 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.489 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:21:23.489 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:21:23.489 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.489 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.489 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.490 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.490 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.490 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:21:23.491 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:21:23.491 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.491 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.491 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.491 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.492 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:21:23.492 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:21:23.493 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.493 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.494 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:21:23.494 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:21:23.494 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:21:23.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:21:23.495 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.496 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:21:23.496 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:21:23.496 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.497 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:21:23.497 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:21:23.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:21:23.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:21:23.499 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:21:23.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:21:23.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:21:23.501 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:21:23.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.501 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:21:23.502 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:21:23.502 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:21:23.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:21:23.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:21:23.503 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:21:23.504 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:21:23.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:21:23.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:21:23.504 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:21:23.505 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.506 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:21:23.506 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:21:23.506 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:21:23.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:23.507 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:21:23.507 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:21:23.507 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:21:23.507 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:21:23.508 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:21:23.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 msgr2=0x7f64180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 0x7f64180693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64280011e0 tx=0x7f6428009040 comp rx=0 tx=0).stop 2023-10-10T20:21:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 msgr2=0x7f6438157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f643806b530 tx=0x7f6430019000 comp rx=0 tx=0).stop 2023-10-10T20:21:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 shutdown_connections 2023-10-10T20:21:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6418066f30 0x7f64180693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f643815d580 0x7f6438157dd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6438157870 0x7f6438151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 --2- 172.21.15.148:0/2332955521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f643815a620 0x7f6438152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 >> 172.21.15.148:0/2332955521 conn(0x7f6438068ce0 msgr2=0x7f643815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 shutdown_connections 2023-10-10T20:21:23.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.474+0000 7f643cd6a700 1 -- 172.21.15.148:0/2332955521 wait complete. 2023-10-10T20:21:23.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:23.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.628+0000 7f900697e700 1 Processor -- start 2023-10-10T20:21:23.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.628+0000 7f900697e700 1 -- start start 2023-10-10T20:21:23.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900697e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9000159be0 0x7f9000068db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900697e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 0x7f90000b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900697e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9000067800 con 0x7f9000159fb0 2023-10-10T20:21:23.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900697e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90000701f0 con 0x7f9000159be0 2023-10-10T20:21:23.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900697e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9000067ae0 con 0x7f90000cf060 2023-10-10T20:21:23.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900597c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9000159be0 0x7f9000068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900517b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9000159fb0 0x7f90000ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58628/0 (socket says 172.21.15.148:58628) 2023-10-10T20:21:23.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900517b700 1 -- 172.21.15.148:0/13194537 learned_addr learned my addr 172.21.15.148:0/13194537 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:23.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.629+0000 7f900617d700 1 --2- 172.21.15.148:0/13194537 >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 0x7f90000b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900597c700 1 -- 172.21.15.148:0/13194537 >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 msgr2=0x7f90000b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:23.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900597c700 1 --2- 172.21.15.148:0/13194537 >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 0x7f90000b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900597c700 1 -- 172.21.15.148:0/13194537 >> v1:172.21.15.148:6789/0 conn(0x7f9000159fb0 legacy=0x7f90000ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900597c700 1 -- 172.21.15.148:0/13194537 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000067dc0 con 0x7f9000159be0 2023-10-10T20:21:23.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900497a700 1 -- 172.21.15.148:0/13194537 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251348033 0 0) 0x7f9000067800 con 0x7f9000159fb0 2023-10-10T20:21:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900617d700 1 --2- 172.21.15.148:0/13194537 >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 0x7f90000b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900597c700 1 --2- 172.21.15.148:0/13194537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f9000068db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f900006f9b0 tx=0x7f8ffc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce066e6fd862a032 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900497a700 1 -- 172.21.15.148:0/13194537 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00e070 con 0x7f9000159be0 2023-10-10T20:21:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900497a700 1 -- 172.21.15.148:0/13194537 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00bc30 con 0x7f9000159be0 2023-10-10T20:21:23.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900497a700 1 -- 172.21.15.148:0/13194537 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc012430 con 0x7f9000159be0 2023-10-10T20:21:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 -- 172.21.15.148:0/13194537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 msgr2=0x7f9000068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 --2- 172.21.15.148:0/13194537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f9000068db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f900006f9b0 tx=0x7f8ffc009780 comp rx=0 tx=0).stop 2023-10-10T20:21:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 -- 172.21.15.148:0/13194537 shutdown_connections 2023-10-10T20:21:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 --2- 172.21.15.148:0/13194537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f9000068db0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 --2- 172.21.15.148:0/13194537 >> v2:172.21.15.161:3300/0 conn(0x7f90000cf060 0x7f90000b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 -- 172.21.15.148:0/13194537 >> 172.21.15.148:0/13194537 conn(0x7f900006bc70 msgr2=0x7f900006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 -- 172.21.15.148:0/13194537 shutdown_connections 2023-10-10T20:21:23.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.630+0000 7f900697e700 1 -- 172.21.15.148:0/13194537 wait complete. 2023-10-10T20:21:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 Processor -- start 2023-10-10T20:21:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 -- start start 2023-10-10T20:21:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90000cf060 0x7f9000158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 0x7f90000c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000158860 con 0x7f9000159be0 2023-10-10T20:21:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b2100 con 0x7f9000159fb0 2023-10-10T20:21:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900697e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90000c8480 con 0x7f90000cf060 2023-10-10T20:21:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900517b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900517b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37854/0 (socket says 172.21.15.148:37854) 2023-10-10T20:21:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.632+0000 7f900517b700 1 -- 172.21.15.148:0/1246939360 learned_addr learned my addr 172.21.15.148:0/1246939360 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900617d700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 0x7f90000c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900597c700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90000cf060 0x7f9000158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 -- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90000cf060 msgr2=0x7f9000158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90000cf060 0x7f9000158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 -- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 msgr2=0x7f90000c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 0x7f90000c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 -- 172.21.15.148:0/1246939360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc009a20 con 0x7f9000159be0 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900617d700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 0x7f90000c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900517b700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c7e0 tx=0x7f8ff0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0005c00 con 0x7f9000159be0 2023-10-10T20:21:23.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0005d60 con 0x7f9000159be0 2023-10-10T20:21:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000c8760 con 0x7f9000159be0 2023-10-10T20:21:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00072e0 con 0x7f9000159be0 2023-10-10T20:21:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.633+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90000b1d90 con 0x7f9000159be0 2023-10-10T20:21:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.634+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90000b1bf0 con 0x7f9000159be0 2023-10-10T20:21:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.634+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ff001b3b0 con 0x7f9000159be0 2023-10-10T20:21:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.635+0000 7f8ff5ffb700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 0x7f8fe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.635+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ff0083630 con 0x7f9000159be0 2023-10-10T20:21:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.635+0000 7f900597c700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 0x7f8fe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.635+0000 7f900597c700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 0x7f8fe40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f900006f9b0 tx=0x7f8ffc019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.639+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ff004b5a0 con 0x7f9000159be0 2023-10-10T20:21:23.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.822+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90000c98b0 con 0x7f9000159be0 2023-10-10T20:21:23.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.823+0000 7f8ff5ffb700 1 -- 172.21.15.148:0/1246939360 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f90000b1bf0 con 0x7f9000159be0 2023-10-10T20:21:23.824 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:23.825 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":62,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:23.068848+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:23.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 msgr2=0x7f8fe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 0x7f8fe40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f900006f9b0 tx=0x7f8ffc019000 comp rx=0 tx=0).stop 2023-10-10T20:21:23.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 msgr2=0x7f90000c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c7e0 tx=0x7f8ff0007d90 comp rx=0 tx=0).stop 2023-10-10T20:21:23.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 shutdown_connections 2023-10-10T20:21:23.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8fe4066f30 0x7f8fe40693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9000159fb0 0x7f90000c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90000cf060 0x7f9000158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 --2- 172.21.15.148:0/1246939360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9000159be0 0x7f90000c6320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 >> 172.21.15.148:0/1246939360 conn(0x7f900006bc70 msgr2=0x7f90000ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 shutdown_connections 2023-10-10T20:21:23.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.825+0000 7f900697e700 1 -- 172.21.15.148:0/1246939360 wait complete. 2023-10-10T20:21:23.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:21:23.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:21:23.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.954+0000 7ffbb0f97700 1 Processor -- start 2023-10-10T20:21:23.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.954+0000 7ffbb0f97700 1 -- start start 2023-10-10T20:21:23.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbac159ec0 0x7ffbac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac15a290 0x7ffbac0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 0x7ffbac06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac06af60 con 0x7ffbac159ec0 2023-10-10T20:21:23.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac0b0790 con 0x7ffbac06a5a0 2023-10-10T20:21:23.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbb0f97700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0ce1f0 con 0x7ffbac15a290 2023-10-10T20:21:23.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbac159ec0 0x7ffbac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbac159ec0 0x7ffbac0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37858/0 (socket says 172.21.15.148:37858) 2023-10-10T20:21:23.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 -- 172.21.15.148:0/1228910592 learned_addr learned my addr 172.21.15.148:0/1228910592 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaad9d700 1 --2- 172.21.15.148:0/1228910592 >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 0x7ffbac06aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffba1d9b700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac15a290 0x7ffbac0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 -- 172.21.15.148:0/1228910592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac15a290 msgr2=0x7ffbac0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac15a290 0x7ffbac0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 -- 172.21.15.148:0/1228910592 >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 msgr2=0x7ffbac06aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:23.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/1228910592 >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 0x7ffbac06aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaa59c700 1 -- 172.21.15.148:0/1228910592 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0b0b20 con 0x7ffbac159ec0 2023-10-10T20:21:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.955+0000 7ffbaad9d700 1 --2- 172.21.15.148:0/1228910592 >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 0x7ffbac06aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac159ec0 0x7ffbac0af940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0b03c0 tx=0x7ffb9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1114aeb28593f75 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffba9d9b700 1 -- 172.21.15.148:0/1228910592 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00b860 con 0x7ffbac159ec0 2023-10-10T20:21:23.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffba9d9b700 1 -- 172.21.15.148:0/1228910592 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c00b9c0 con 0x7ffbac159ec0 2023-10-10T20:21:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffba9d9b700 1 -- 172.21.15.148:0/1228910592 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00bd60 con 0x7ffbac159ec0 2023-10-10T20:21:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 -- 172.21.15.148:0/1228910592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac159ec0 msgr2=0x7ffbac0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac159ec0 0x7ffbac0af940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0b03c0 tx=0x7ffb9c009250 comp rx=0 tx=0).stop 2023-10-10T20:21:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 -- 172.21.15.148:0/1228910592 shutdown_connections 2023-10-10T20:21:23.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/1228910592 >> v2:172.21.15.161:3300/0 conn(0x7ffbac06a5a0 0x7ffbac06aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac159ec0 0x7ffbac0af940 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/1228910592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac15a290 0x7ffbac0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 -- 172.21.15.148:0/1228910592 >> 172.21.15.148:0/1228910592 conn(0x7ffbac0d1880 msgr2=0x7ffbac0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 -- 172.21.15.148:0/1228910592 shutdown_connections 2023-10-10T20:21:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.956+0000 7ffbb0f97700 1 -- 172.21.15.148:0/1228910592 wait complete. 2023-10-10T20:21:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.957+0000 7ffbb0f97700 1 Processor -- start 2023-10-10T20:21:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.957+0000 7ffbb0f97700 1 -- start start 2023-10-10T20:21:23.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 0x7ffbac1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbac159ec0 0x7ffbac16ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac16b320 con 0x7ffbac15a290 2023-10-10T20:21:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0c9b80 con 0x7ffbac159ec0 2023-10-10T20:21:23.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbb0f97700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1579d0 con 0x7ffbac06a5a0 2023-10-10T20:21:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbaad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbaad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37866/0 (socket says 172.21.15.148:37866) 2023-10-10T20:21:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbaad9d700 1 -- 172.21.15.148:0/3000083540 learned_addr learned my addr 172.21.15.148:0/3000083540 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffba1d9b700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbac159ec0 0x7ffbac16ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.958+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 0x7ffbac1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 -- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 msgr2=0x7ffbac1521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 0x7ffbac1521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 -- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbac159ec0 msgr2=0x7ffbac16ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbac159ec0 0x7ffbac16ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 -- 172.21.15.148:0/3000083540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0094f0 con 0x7ffbac15a290 2023-10-10T20:21:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 0x7ffbac1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbaad9d700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d180 tx=0x7ffba4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400dd70 con 0x7ffbac15a290 2023-10-10T20:21:23.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400f040 con 0x7ffbac15a290 2023-10-10T20:21:23.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4015690 con 0x7ffbac15a290 2023-10-10T20:21:23.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0ca840 con 0x7ffbac15a290 2023-10-10T20:21:23.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.959+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac157660 con 0x7ffbac15a290 2023-10-10T20:21:23.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.960+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac157b40 con 0x7ffbac15a290 2023-10-10T20:21:23.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.960+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffba401e3b0 con 0x7ffbac15a290 2023-10-10T20:21:23.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.961+0000 7ffba2ffd700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:23.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.961+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ffba40846c0 con 0x7ffbac15a290 2023-10-10T20:21:23.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.961+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:23.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.961+0000 7ffbaa59c700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0b03c0 tx=0x7ffb9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:23.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:23.965+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffba4051c80 con 0x7ffbac15a290 2023-10-10T20:21:24.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.148+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbac04c3c0 con 0x7ffbac15a290 2023-10-10T20:21:24.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.149+0000 7ffba2ffd700 1 -- 172.21.15.148:0/3000083540 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffba401a020 con 0x7ffbac15a290 2023-10-10T20:21:24.151 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:21:24.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 msgr2=0x7ffb8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0b03c0 tx=0x7ffb9c017000 comp rx=0 tx=0).stop 2023-10-10T20:21:24.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 msgr2=0x7ffbac0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d180 tx=0x7ffba4009a50 comp rx=0 tx=0).stop 2023-10-10T20:21:24.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 shutdown_connections 2023-10-10T20:21:24.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbac15a290 0x7ffbac0b4d70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbac06a5a0 0x7ffbac1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 --2- 172.21.15.148:0/3000083540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbac159ec0 0x7ffbac16ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 >> 172.21.15.148:0/3000083540 conn(0x7ffbac0d1880 msgr2=0x7ffbac153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:24.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 shutdown_connections 2023-10-10T20:21:24.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.151+0000 7ffbb0f97700 1 -- 172.21.15.148:0/3000083540 wait complete. 2023-10-10T20:21:24.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:21:24.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.277+0000 7f04bffff700 1 Processor -- start 2023-10-10T20:21:24.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.277+0000 7f04bffff700 1 -- start start 2023-10-10T20:21:24.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04c00cd6e0 0x7f04c00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00cb7c0 con 0x7f04c015cbd0 2023-10-10T20:21:24.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c015e290 con 0x7f04c00cdab0 2023-10-10T20:21:24.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c015e850 con 0x7f04c00cd6e0 2023-10-10T20:21:24.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04beffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04c00cd6e0 0x7f04c00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bf7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f04c015cbd0 0x7f04c015d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58642/0 (socket says 172.21.15.148:58642) 2023-10-10T20:21:24.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bf7fe700 1 -- 172.21.15.148:0/827364040 learned_addr learned my addr 172.21.15.148:0/827364040 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:24.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04beffd700 1 -- 172.21.15.148:0/827364040 >> v1:172.21.15.161:6789/0 conn(0x7f04c00cdab0 legacy=0x7f04c015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:24.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04beffd700 1 -- 172.21.15.148:0/827364040 >> v1:172.21.15.148:6789/0 conn(0x7f04c015cbd0 legacy=0x7f04c015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04bdffb700 1 -- 172.21.15.148:0/827364040 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282097145 0 0) 0x7f04c00cb7c0 con 0x7f04c015cbd0 2023-10-10T20:21:24.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04beffd700 1 -- 172.21.15.148:0/827364040 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c015e570 con 0x7f04c00cd6e0 2023-10-10T20:21:24.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.278+0000 7f04beffd700 1 --2- 172.21.15.148:0/827364040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c00cbc50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009e60 tx=0x7f04b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51feac0585a5e4cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bdffb700 1 -- 172.21.15.148:0/827364040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000e070 con 0x7f04c00cd6e0 2023-10-10T20:21:24.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bdffb700 1 -- 172.21.15.148:0/827364040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b000bc30 con 0x7f04c00cd6e0 2023-10-10T20:21:24.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bdffb700 1 -- 172.21.15.148:0/827364040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0012430 con 0x7f04c00cd6e0 2023-10-10T20:21:24.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 -- 172.21.15.148:0/827364040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 msgr2=0x7f04c00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 --2- 172.21.15.148:0/827364040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c00cbc50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009e60 tx=0x7f04b0009780 comp rx=0 tx=0).stop 2023-10-10T20:21:24.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 -- 172.21.15.148:0/827364040 shutdown_connections 2023-10-10T20:21:24.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 --2- 172.21.15.148:0/827364040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c00cbc50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 -- 172.21.15.148:0/827364040 >> 172.21.15.148:0/827364040 conn(0x7f04c0067660 msgr2=0x7f04c0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:24.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 -- 172.21.15.148:0/827364040 shutdown_connections 2023-10-10T20:21:24.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.279+0000 7f04bffff700 1 -- 172.21.15.148:0/827364040 wait complete. 2023-10-10T20:21:24.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 Processor -- start 2023-10-10T20:21:24.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 -- start start 2023-10-10T20:21:24.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04c00cdab0 0x7f04c00c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c015cbd0 0x7f04c00b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00b5900 con 0x7f04c00cd6e0 2023-10-10T20:21:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014f120 con 0x7f04c015cbd0 2023-10-10T20:21:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04bffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014e510 con 0x7f04c00cdab0 2023-10-10T20:21:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04beffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04beffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37876/0 (socket says 172.21.15.148:37876) 2023-10-10T20:21:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.281+0000 7f04beffd700 1 -- 172.21.15.148:0/2563913311 learned_addr learned my addr 172.21.15.148:0/2563913311 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:24.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04bf7fe700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c015cbd0 0x7f04c00b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 -- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04c00cdab0 msgr2=0x7f04c00c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04c00cdab0 0x7f04c00c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 -- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c015cbd0 msgr2=0x7f04c00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c015cbd0 0x7f04c00b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 -- 172.21.15.148:0/2563913311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0009a20 con 0x7f04c00cd6e0 2023-10-10T20:21:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04beffd700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f04b00099f0 tx=0x7f04b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0027470 con 0x7f04c00cd6e0 2023-10-10T20:21:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00275f0 con 0x7f04c00cd6e0 2023-10-10T20:21:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c014e7f0 con 0x7f04c00cd6e0 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00279b0 con 0x7f04c00cd6e0 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.282+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c014eb80 con 0x7f04c00cd6e0 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.283+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c014d3f0 con 0x7f04c00cd6e0 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.283+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04b0024070 con 0x7f04c00cd6e0 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.284+0000 7f04a6ffd700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 0x7f04ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.284+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f04b0086c90 con 0x7f04c00cd6e0 2023-10-10T20:21:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.284+0000 7f04be7fc700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 0x7f04ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.284+0000 7f04be7fc700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 0x7f04ac0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04c00b4ed0 tx=0x7f04b4007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.288+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04b0054250 con 0x7f04c00cd6e0 2023-10-10T20:21:24.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.471+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f04c004bb80 con 0x7f04c00cd6e0 2023-10-10T20:21:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.471+0000 7f04a6ffd700 1 -- 172.21.15.148:0/2563913311 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+65 (secure 0 0 0) 0x7f04b0057870 con 0x7f04c00cd6e0 2023-10-10T20:21:24.473 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:24.473 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:21:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:21:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:21:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:21:24.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:21:24.474 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:21:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 msgr2=0x7f04ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 0x7f04ac0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04c00b4ed0 tx=0x7f04b4007b00 comp rx=0 tx=0).stop 2023-10-10T20:21:24.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 msgr2=0x7f04c014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f04b00099f0 tx=0x7f04b0019000 comp rx=0 tx=0).stop 2023-10-10T20:21:24.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 shutdown_connections 2023-10-10T20:21:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04ac066f30 0x7f04ac0693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c015cbd0 0x7f04c00b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04c00cd6e0 0x7f04c014ce50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 --2- 172.21.15.148:0/2563913311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04c00cdab0 0x7f04c00c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 >> 172.21.15.148:0/2563913311 conn(0x7f04c0067660 msgr2=0x7f04c0063230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 shutdown_connections 2023-10-10T20:21:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.474+0000 7f04bffff700 1 -- 172.21.15.148:0/2563913311 wait complete. 2023-10-10T20:21:24.487 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:21:24.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:21:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.605+0000 7f928e16d700 1 Processor -- start 2023-10-10T20:21:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.605+0000 7f928e16d700 1 -- start start 2023-10-10T20:21:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f928e16d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9288159ef0 0x7f928815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f928e16d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 0x7f92880b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f928e16d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92880b1a60 con 0x7f92880b15e0 2023-10-10T20:21:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f928e16d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92880b1d40 con 0x7f9288159ef0 2023-10-10T20:21:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f928e16d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92880b2020 con 0x7f92880b6a10 2023-10-10T20:21:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9288159ef0 0x7f928815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9288159ef0 0x7f928815a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37878/0 (socket says 172.21.15.148:37878) 2023-10-10T20:21:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 -- 172.21.15.148:0/2293149997 learned_addr learned my addr 172.21.15.148:0/2293149997 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f9286ffd700 1 --2- 172.21.15.148:0/2293149997 >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 0x7f92880b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 -- 172.21.15.148:0/2293149997 >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 msgr2=0x7f92880b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 --2- 172.21.15.148:0/2293149997 >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 0x7f92880b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 -- 172.21.15.148:0/2293149997 >> v1:172.21.15.161:6789/0 conn(0x7f92880b15e0 legacy=0x7f92880b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 -- 172.21.15.148:0/2293149997 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92880b2300 con 0x7f9288159ef0 2023-10-10T20:21:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f9286ffd700 1 --2- 172.21.15.148:0/2293149997 >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 0x7f92880b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.606+0000 7f92877fe700 1 --2- 172.21.15.148:0/2293149997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288159ef0 0x7f928815a2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9288089550 tx=0x7f9270009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfbb5cff8cc5e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f92867fc700 1 -- 172.21.15.148:0/2293149997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270005c10 con 0x7f9288159ef0 2023-10-10T20:21:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f92867fc700 1 -- 172.21.15.148:0/2293149997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270005d70 con 0x7f9288159ef0 2023-10-10T20:21:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f92867fc700 1 -- 172.21.15.148:0/2293149997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000b880 con 0x7f9288159ef0 2023-10-10T20:21:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 -- 172.21.15.148:0/2293149997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288159ef0 msgr2=0x7f928815a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 --2- 172.21.15.148:0/2293149997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288159ef0 0x7f928815a2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9288089550 tx=0x7f9270009220 comp rx=0 tx=0).stop 2023-10-10T20:21:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 -- 172.21.15.148:0/2293149997 shutdown_connections 2023-10-10T20:21:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 --2- 172.21.15.148:0/2293149997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288159ef0 0x7f928815a2c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 --2- 172.21.15.148:0/2293149997 >> v2:172.21.15.161:3300/0 conn(0x7f92880b6a10 0x7f92880b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 -- 172.21.15.148:0/2293149997 >> 172.21.15.148:0/2293149997 conn(0x7f928809a0a0 msgr2=0x7f9288067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 -- 172.21.15.148:0/2293149997 shutdown_connections 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.608+0000 7f928e16d700 1 -- 172.21.15.148:0/2293149997 wait complete. 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 Processor -- start 2023-10-10T20:21:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 -- start start 2023-10-10T20:21:24.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92880b6a10 0x7f9288152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9288159ef0 0x7f9288153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928809c390 con 0x7f9288155170 2023-10-10T20:21:24.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288157b10 con 0x7f9288159ef0 2023-10-10T20:21:24.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f928e16d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9288157df0 con 0x7f92880b6a10 2023-10-10T20:21:24.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f9287fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f9287fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37884/0 (socket says 172.21.15.148:37884) 2023-10-10T20:21:24.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.610+0000 7f9287fff700 1 -- 172.21.15.148:0/4274876830 learned_addr learned my addr 172.21.15.148:0/4274876830 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:24.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f92877fe700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92880b6a10 0x7f9288152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9286ffd700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9288159ef0 0x7f9288153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 -- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92880b6a10 msgr2=0x7f9288152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92880b6a10 0x7f9288152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 -- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9288159ef0 msgr2=0x7f9288153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:24.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9288159ef0 0x7f9288153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:24.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 -- 172.21.15.148:0/4274876830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270009480 con 0x7f9288155170 2023-10-10T20:21:24.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f9287fff700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f928000c8e0 tx=0x7f9280001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000ce80 con 0x7f9288155170 2023-10-10T20:21:24.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928000a7e0 con 0x7f9288155170 2023-10-10T20:21:24.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000ab80 con 0x7f9288155170 2023-10-10T20:21:24.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92881580d0 con 0x7f9288155170 2023-10-10T20:21:24.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.611+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9288155b50 con 0x7f9288155170 2023-10-10T20:21:24.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.612+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928809c540 con 0x7f9288155170 2023-10-10T20:21:24.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.612+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9280012420 con 0x7f9288155170 2023-10-10T20:21:24.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.613+0000 7f927f7fe700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 0x7f92680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:24.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.613+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9280084500 con 0x7f9288155170 2023-10-10T20:21:24.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.613+0000 7f92877fe700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 0x7f92680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:24.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.613+0000 7f92877fe700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 0x7f92680693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9270009260 tx=0x7f9270016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:24.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.617+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f928809c540 con 0x7f9288155170 2023-10-10T20:21:24.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:24.797+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f928804c3c0 con 0x7f9288155170 2023-10-10T20:21:25.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.616+0000 7f927f7fe700 1 -- 172.21.15.148:0/4274876830 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9280051ac0 con 0x7f9288155170 2023-10-10T20:21:25.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 msgr2=0x7f92680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 0x7f92680693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9270009260 tx=0x7f9270016000 comp rx=0 tx=0).stop 2023-10-10T20:21:25.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 msgr2=0x7f9288155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f928000c8e0 tx=0x7f9280001710 comp rx=0 tx=0).stop 2023-10-10T20:21:25.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 shutdown_connections 2023-10-10T20:21:25.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9268066f30 0x7f92680693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9288155170 0x7f9288155610 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92880b6a10 0x7f9288152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 --2- 172.21.15.148:0/4274876830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9288159ef0 0x7f9288153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 >> 172.21.15.148:0/4274876830 conn(0x7f928809a0a0 msgr2=0x7f928809e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:25.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 shutdown_connections 2023-10-10T20:21:25.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.620+0000 7f928e16d700 1 -- 172.21.15.148:0/4274876830 wait complete. 2023-10-10T20:21:25.624 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:21:25.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:21:25.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.758+0000 7f4826f4e700 1 Processor -- start 2023-10-10T20:21:25.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.758+0000 7f4826f4e700 1 -- start start 2023-10-10T20:21:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4826f4e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 0x7f482015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4826f4e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 0x7f48200af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4826f4e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482015ea90 con 0x7f4820159e60 2023-10-10T20:21:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4826f4e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48200d1cf0 con 0x7f48200697c0 2023-10-10T20:21:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4826f4e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200b3100 con 0x7f482015a230 2023-10-10T20:21:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4817fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 0x7f482015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f48254eb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 0x7f48200af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4824cea700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4820159e60 0x7f48200690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:55372/0 (socket says 172.21.15.148:55372) 2023-10-10T20:21:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.759+0000 7f4824cea700 1 -- 172.21.15.148:0/3131223215 learned_addr learned my addr 172.21.15.148:0/3131223215 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48254eb700 1 -- 172.21.15.148:0/3131223215 >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 msgr2=0x7f482015e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48254eb700 1 --2- 172.21.15.148:0/3131223215 >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 0x7f482015e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48254eb700 1 -- 172.21.15.148:0/3131223215 >> v1:172.21.15.161:6789/0 conn(0x7f4820159e60 legacy=0x7f48200690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48254eb700 1 -- 172.21.15.148:0/3131223215 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200d10e0 con 0x7f482015a230 2023-10-10T20:21:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f4817fff700 1 --2- 172.21.15.148:0/3131223215 >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 0x7f482015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48254eb700 1 --2- 172.21.15.148:0/3131223215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 0x7f48200af730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48180098e0 tx=0x7f48180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc71fcc64d5079f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48177fe700 1 -- 172.21.15.148:0/3131223215 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800b640 con 0x7f482015a230 2023-10-10T20:21:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48177fe700 1 -- 172.21.15.148:0/3131223215 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800b7a0 con 0x7f482015a230 2023-10-10T20:21:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.760+0000 7f48177fe700 1 -- 172.21.15.148:0/3131223215 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800bb40 con 0x7f482015a230 2023-10-10T20:21:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 -- 172.21.15.148:0/3131223215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 msgr2=0x7f48200af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 --2- 172.21.15.148:0/3131223215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 0x7f48200af730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48180098e0 tx=0x7f48180091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 -- 172.21.15.148:0/3131223215 shutdown_connections 2023-10-10T20:21:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 --2- 172.21.15.148:0/3131223215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f482015a230 0x7f48200af730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 --2- 172.21.15.148:0/3131223215 >> v2:172.21.15.161:3300/0 conn(0x7f48200697c0 0x7f482015e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 -- 172.21.15.148:0/3131223215 >> 172.21.15.148:0/3131223215 conn(0x7f4820070700 msgr2=0x7f4820070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 -- 172.21.15.148:0/3131223215 shutdown_connections 2023-10-10T20:21:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.761+0000 7f4826f4e700 1 -- 172.21.15.148:0/3131223215 wait complete. 2023-10-10T20:21:25.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.762+0000 7f4826f4e700 1 Processor -- start 2023-10-10T20:21:25.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.762+0000 7f4826f4e700 1 -- start start 2023-10-10T20:21:25.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48200697c0 0x7f4820158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 0x7f48200c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482015b090 con 0x7f4820159e60 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200cb2f0 con 0x7f482015a230 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4826f4e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4820158c40 con 0x7f48200697c0 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37888/0 (socket says 172.21.15.148:37888) 2023-10-10T20:21:25.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 -- 172.21.15.148:0/61627377 learned_addr learned my addr 172.21.15.148:0/61627377 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:25.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4824cea700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48200697c0 0x7f4820158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f48254eb700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 0x7f48200c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 -- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48200697c0 msgr2=0x7f4820158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48200697c0 0x7f4820158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 -- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 msgr2=0x7f48200c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:25.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 0x7f48200c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:25.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 -- 172.21.15.148:0/61627377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009420 con 0x7f4820159e60 2023-10-10T20:21:25.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f48254eb700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 0x7f48200c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:25.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4817fff700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f481000ecb0 tx=0x7f48100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:25.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.763+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000c920 con 0x7f4820159e60 2023-10-10T20:21:25.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.764+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481000ca80 con 0x7f4820159e60 2023-10-10T20:21:25.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.764+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000ce20 con 0x7f4820159e60 2023-10-10T20:21:25.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.764+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818009420 con 0x7f4820159e60 2023-10-10T20:21:25.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.764+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482015a760 con 0x7f4820159e60 2023-10-10T20:21:25.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.764+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482015b240 con 0x7f4820159e60 2023-10-10T20:21:25.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.765+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4810013680 con 0x7f4820159e60 2023-10-10T20:21:25.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.765+0000 7f4814ff9700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:25.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.765+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f4810084640 con 0x7f4820159e60 2023-10-10T20:21:25.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.765+0000 7f4824cea700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:25.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.766+0000 7f4824cea700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 0x7f480c0693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f481c002250 tx=0x7f481c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:25.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.770+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f482015b240 con 0x7f4820159e60 2023-10-10T20:21:25.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:25.953+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f482004c3c0 con 0x7f4820159e60 2023-10-10T20:21:26.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.617+0000 7f4814ff9700 1 -- 172.21.15.148:0/61627377 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7f48100519f0 con 0x7f4820159e60 2023-10-10T20:21:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.622+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 msgr2=0x7f480c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.622+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 0x7f480c0693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f481c002250 tx=0x7f481c009040 comp rx=0 tx=0).stop 2023-10-10T20:21:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.622+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 msgr2=0x7f48200c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.622+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f481000ecb0 tx=0x7f48100091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 shutdown_connections 2023-10-10T20:21:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f482015a230 0x7f48200c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48200697c0 0x7f4820158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 --2- 172.21.15.148:0/61627377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4820159e60 0x7f48200c5c40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.624+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 >> 172.21.15.148:0/61627377 conn(0x7f4820070700 msgr2=0x7f48201594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:26.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.627+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 shutdown_connections 2023-10-10T20:21:26.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.627+0000 7f4826f4e700 1 -- 172.21.15.148:0/61627377 wait complete. 2023-10-10T20:21:26.630 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:21:26.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:21:26.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.760+0000 7f1c224f0700 1 Processor -- start 2023-10-10T20:21:26.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.760+0000 7f1c224f0700 1 -- start start 2023-10-10T20:21:26.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c0cce30 0x7f1c1c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1c1c06a830 0x7f1c1c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c0b4d70 con 0x7f1c1c0b6580 2023-10-10T20:21:26.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c06ac70 con 0x7f1c1c06a830 2023-10-10T20:21:26.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c224f0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0cd8d0 con 0x7f1c1c0cce30 2023-10-10T20:21:26.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37896/0 (socket says 172.21.15.148:37896) 2023-10-10T20:21:26.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 -- 172.21.15.148:0/842205556 learned_addr learned my addr 172.21.15.148:0/842205556 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:26.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c21cef700 1 --2- 172.21.15.148:0/842205556 >> v2:172.21.15.161:3300/0 conn(0x7f1c1c06a830 0x7f1c1c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 -- 172.21.15.148:0/842205556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c0cce30 msgr2=0x7f1c1c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:26.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 --2- 172.21.15.148:0/842205556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c0cce30 0x7f1c1c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 -- 172.21.15.148:0/842205556 >> v2:172.21.15.161:3300/0 conn(0x7f1c1c06a830 msgr2=0x7f1c1c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:26.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 --2- 172.21.15.148:0/842205556 >> v2:172.21.15.161:3300/0 conn(0x7f1c1c06a830 0x7f1c1c15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 -- 172.21.15.148:0/842205556 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0b4440 con 0x7f1c1c0b6580 2023-10-10T20:21:26.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.761+0000 7f1c20ced700 1 --2- 172.21.15.148:0/842205556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c15a280 tx=0x7f1c10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d0065e5f68042c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:26.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c1bfff700 1 -- 172.21.15.148:0/842205556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000b860 con 0x7f1c1c0b6580 2023-10-10T20:21:26.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c1bfff700 1 -- 172.21.15.148:0/842205556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000b9c0 con 0x7f1c1c0b6580 2023-10-10T20:21:26.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c1bfff700 1 -- 172.21.15.148:0/842205556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000bd60 con 0x7f1c1c0b6580 2023-10-10T20:21:26.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 -- 172.21.15.148:0/842205556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 msgr2=0x7f1c1c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:26.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 --2- 172.21.15.148:0/842205556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c15a280 tx=0x7f1c10009250 comp rx=0 tx=0).stop 2023-10-10T20:21:26.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 -- 172.21.15.148:0/842205556 shutdown_connections 2023-10-10T20:21:26.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 --2- 172.21.15.148:0/842205556 >> v2:172.21.15.161:3300/0 conn(0x7f1c1c06a830 0x7f1c1c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 --2- 172.21.15.148:0/842205556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c0cce30 0x7f1c1c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 --2- 172.21.15.148:0/842205556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c06a2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 -- 172.21.15.148:0/842205556 >> 172.21.15.148:0/842205556 conn(0x7f1c1c068cb0 msgr2=0x7f1c1c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:26.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 -- 172.21.15.148:0/842205556 shutdown_connections 2023-10-10T20:21:26.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.762+0000 7f1c224f0700 1 -- 172.21.15.148:0/842205556 wait complete. 2023-10-10T20:21:26.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.763+0000 7f1c224f0700 1 Processor -- start 2023-10-10T20:21:26.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.763+0000 7f1c224f0700 1 -- start start 2023-10-10T20:21:26.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c06a830 0x7f1c1c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c1c0cce30 0x7f1c1c154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c14e810 con 0x7f1c1c0b6580 2023-10-10T20:21:26.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c153380 con 0x7f1c1c0cce30 2023-10-10T20:21:26.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c224f0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0b11f0 con 0x7f1c1c06a830 2023-10-10T20:21:26.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37902/0 (socket says 172.21.15.148:37902) 2023-10-10T20:21:26.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 -- 172.21.15.148:0/3592649759 learned_addr learned my addr 172.21.15.148:0/3592649759 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:26.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c21cef700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c1c0cce30 0x7f1c1c154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c214ee700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c06a830 0x7f1c1c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 -- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c06a830 msgr2=0x7f1c1c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:26.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c06a830 0x7f1c1c14e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 -- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c1c0cce30 msgr2=0x7f1c1c154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:26.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c1c0cce30 0x7f1c1c154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:26.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.764+0000 7f1c20ced700 1 -- 172.21.15.148:0/3592649759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100094f0 con 0x7f1c1c0b6580 2023-10-10T20:21:26.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c20ced700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009290 tx=0x7f1c10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:26.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10026480 con 0x7f1c1c0b6580 2023-10-10T20:21:26.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1001f070 con 0x7f1c1c0b6580 2023-10-10T20:21:26.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c152a50 con 0x7f1c1c0b6580 2023-10-10T20:21:26.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10026830 con 0x7f1c1c0b6580 2023-10-10T20:21:26.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.765+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c153010 con 0x7f1c1c0b6580 2023-10-10T20:21:26.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.766+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c153530 con 0x7f1c1c0b6580 2023-10-10T20:21:26.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.766+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c10026ba0 con 0x7f1c1c0b6580 2023-10-10T20:21:26.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.766+0000 7f1c197fa700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 0x7f1c080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:26.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.767+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f1c10021070 con 0x7f1c1c0b6580 2023-10-10T20:21:26.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.767+0000 7f1c214ee700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 0x7f1c080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:26.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.767+0000 7f1c214ee700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 0x7f1c080693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007f30 tx=0x7f1c14007420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:26.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.771+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c1c153530 con 0x7f1c1c0b6580 2023-10-10T20:21:26.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:26.993+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1c1c1534f0 con 0x7f1c1c0b6580 2023-10-10T20:21:27.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.651+0000 7f1c197fa700 1 -- 172.21.15.148:0/3592649759 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1c100535f0 con 0x7f1c1c0b6580 2023-10-10T20:21:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 msgr2=0x7f1c080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 0x7f1c080693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007f30 tx=0x7f1c14007420 comp rx=0 tx=0).stop 2023-10-10T20:21:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 msgr2=0x7f1c1c153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009290 tx=0x7f1c10017000 comp rx=0 tx=0).stop 2023-10-10T20:21:27.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 shutdown_connections 2023-10-10T20:21:27.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c08066f30 0x7f1c080693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c1c0cce30 0x7f1c1c154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c1c06a830 0x7f1c1c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 --2- 172.21.15.148:0/3592649759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c1c0b6580 0x7f1c1c153810 secure :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009290 tx=0x7f1c10017000 comp rx=0 tx=0).stop 2023-10-10T20:21:27.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.655+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 >> 172.21.15.148:0/3592649759 conn(0x7f1c1c068cb0 msgr2=0x7f1c1c150590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:27.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.656+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 shutdown_connections 2023-10-10T20:21:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.656+0000 7f1c224f0700 1 -- 172.21.15.148:0/3592649759 wait complete. 2023-10-10T20:21:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 8 and data pool 9 2023-10-10T20:21:27.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.784+0000 7f68569f5700 1 Processor -- start 2023-10-10T20:21:27.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.784+0000 7f68569f5700 1 -- start start 2023-10-10T20:21:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f68500cd6e0 0x7f68500cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68500cdab0 0x7f685015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f685015dda0 0x7f685015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685015e850 con 0x7f68500cdab0 2023-10-10T20:21:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500cd030 con 0x7f68500cd6e0 2023-10-10T20:21:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68569f5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cb7c0 con 0x7f685015dda0 2023-10-10T20:21:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68500cdab0 0x7f685015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f68500cdab0 0x7f685015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37914/0 (socket says 172.21.15.148:37914) 2023-10-10T20:21:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 -- 172.21.15.148:0/923359155 learned_addr learned my addr 172.21.15.148:0/923359155 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68559f3700 1 --2- 172.21.15.148:0/923359155 >> v2:172.21.15.161:3300/0 conn(0x7f68500cd6e0 0x7f68500cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68561f4700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f685015dda0 0x7f685015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 -- 172.21.15.148:0/923359155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f685015dda0 msgr2=0x7f685015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f685015dda0 0x7f685015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 -- 172.21.15.148:0/923359155 >> v2:172.21.15.161:3300/0 conn(0x7f68500cd6e0 msgr2=0x7f68500cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 --2- 172.21.15.148:0/923359155 >> v2:172.21.15.161:3300/0 conn(0x7f68500cd6e0 0x7f68500cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.785+0000 7f68551f2700 1 -- 172.21.15.148:0/923359155 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685015e290 con 0x7f68500cdab0 2023-10-10T20:21:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68551f2700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f685015d860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f685006b730 tx=0x7f6844009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f1e3ca0d7d7f9b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68549f1700 1 -- 172.21.15.148:0/923359155 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400b640 con 0x7f68500cdab0 2023-10-10T20:21:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68549f1700 1 -- 172.21.15.148:0/923359155 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400b7a0 con 0x7f68500cdab0 2023-10-10T20:21:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68549f1700 1 -- 172.21.15.148:0/923359155 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400bb60 con 0x7f68500cdab0 2023-10-10T20:21:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 -- 172.21.15.148:0/923359155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 msgr2=0x7f685015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f685015d860 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f685006b730 tx=0x7f6844009250 comp rx=0 tx=0).stop 2023-10-10T20:21:27.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 -- 172.21.15.148:0/923359155 shutdown_connections 2023-10-10T20:21:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f685015dda0 0x7f685015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 --2- 172.21.15.148:0/923359155 >> v2:172.21.15.161:3300/0 conn(0x7f68500cd6e0 0x7f68500cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 --2- 172.21.15.148:0/923359155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f685015d860 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 -- 172.21.15.148:0/923359155 >> 172.21.15.148:0/923359155 conn(0x7f6850067660 msgr2=0x7f6850067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 -- 172.21.15.148:0/923359155 shutdown_connections 2023-10-10T20:21:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.786+0000 7f68569f5700 1 -- 172.21.15.148:0/923359155 wait complete. 2023-10-10T20:21:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 Processor -- start 2023-10-10T20:21:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 -- start start 2023-10-10T20:21:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68500cd6e0 0x7f6850155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f685015dda0 0x7f68501570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c8c10 con 0x7f68500cdab0 2023-10-10T20:21:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850156150 con 0x7f685015dda0 2023-10-10T20:21:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68569f5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c7d20 con 0x7f68500cd6e0 2023-10-10T20:21:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68551f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68551f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37924/0 (socket says 172.21.15.148:37924) 2023-10-10T20:21:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.788+0000 7f68551f2700 1 -- 172.21.15.148:0/3277642821 learned_addr learned my addr 172.21.15.148:0/3277642821 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68559f3700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68500cd6e0 0x7f6850155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68561f4700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f685015dda0 0x7f68501570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 -- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68500cd6e0 msgr2=0x7f6850155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68500cd6e0 0x7f6850155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 -- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f685015dda0 msgr2=0x7f68501570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f685015dda0 0x7f68501570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 -- 172.21.15.148:0/3277642821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440094f0 con 0x7f68500cdab0 2023-10-10T20:21:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68551f2700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f68500c7b10 tx=0x7f6844017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844000d10 con 0x7f68500cdab0 2023-10-10T20:21:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400bb20 con 0x7f68500cdab0 2023-10-10T20:21:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500c8000 con 0x7f68500cdab0 2023-10-10T20:21:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844011670 con 0x7f68500cdab0 2023-10-10T20:21:27.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.789+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500c85c0 con 0x7f68500cdab0 2023-10-10T20:21:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.790+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685015b400 con 0x7f68500cdab0 2023-10-10T20:21:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.790+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68440119e0 con 0x7f68500cdab0 2023-10-10T20:21:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.791+0000 7f684dffb700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.791+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6844085de0 con 0x7f68500cdab0 2023-10-10T20:21:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.791+0000 7f68559f3700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.791+0000 7f68559f3700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 0x7f683c0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68480076c0 tx=0x7f684800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.795+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6844053050 con 0x7f68500cdab0 2023-10-10T20:21:27.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.979+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f685015e570 con 0x7f68500cdab0 2023-10-10T20:21:27.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.980+0000 7f684dffb700 1 -- 172.21.15.148:0/3277642821 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f6844056670 con 0x7f68500cdab0 2023-10-10T20:21:27.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:27.982 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":65,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:26.993874+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:27.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.981+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 msgr2=0x7f683c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.981+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 0x7f683c0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f68480076c0 tx=0x7f684800b040 comp rx=0 tx=0).stop 2023-10-10T20:21:27.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 msgr2=0x7f6850158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:27.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f68500c7b10 tx=0x7f6844017000 comp rx=0 tx=0).stop 2023-10-10T20:21:27.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 shutdown_connections 2023-10-10T20:21:27.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f685015dda0 0x7f68501570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68500cd6e0 0x7f6850155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 --2- 172.21.15.148:0/3277642821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68500cdab0 0x7f6850158790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:27.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 >> 172.21.15.148:0/3277642821 conn(0x7f6850067660 msgr2=0x7f68500d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:27.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 shutdown_connections 2023-10-10T20:21:27.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:27.982+0000 7f68569f5700 1 -- 172.21.15.148:0/3277642821 wait complete. 2023-10-10T20:21:27.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:28.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.110+0000 7fb89db79700 1 Processor -- start 2023-10-10T20:21:28.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.110+0000 7fb89db79700 1 -- start start 2023-10-10T20:21:28.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb89db79700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb8980cf1c0 0x7fb89815e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb89db79700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb89815a0c0 0x7fb898068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb89db79700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8980cca20 con 0x7fb898159cf0 2023-10-10T20:21:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb89db79700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89815ea40 con 0x7fb89815a0c0 2023-10-10T20:21:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb89db79700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb898067660 con 0x7fb8980cf1c0 2023-10-10T20:21:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb897fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb89815a0c0 0x7fb898068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb8977fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb898159cf0 0x7fb8980ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58650/0 (socket says 172.21.15.148:58650) 2023-10-10T20:21:28.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.111+0000 7fb8977fe700 1 -- 172.21.15.148:0/48432966 learned_addr learned my addr 172.21.15.148:0/48432966 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:28.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb896ffd700 1 --2- 172.21.15.148:0/48432966 >> v2:172.21.15.161:3300/0 conn(0x7fb8980cf1c0 0x7fb89815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb897fff700 1 -- 172.21.15.148:0/48432966 >> v2:172.21.15.161:3300/0 conn(0x7fb8980cf1c0 msgr2=0x7fb89815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:28.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb897fff700 1 --2- 172.21.15.148:0/48432966 >> v2:172.21.15.161:3300/0 conn(0x7fb8980cf1c0 0x7fb89815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb897fff700 1 -- 172.21.15.148:0/48432966 >> v1:172.21.15.148:6789/0 conn(0x7fb898159cf0 legacy=0x7fb8980ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb897fff700 1 -- 172.21.15.148:0/48432966 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898067f00 con 0x7fb89815a0c0 2023-10-10T20:21:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb8967fc700 1 -- 172.21.15.148:0/48432966 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888389618 0 0) 0x7fb8980cca20 con 0x7fb898159cf0 2023-10-10T20:21:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb897fff700 1 --2- 172.21.15.148:0/48432966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89815a0c0 0x7fb898068390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb89806a870 tx=0x7fb8900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9184f40a195209b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb8967fc700 1 -- 172.21.15.148:0/48432966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000b6b0 con 0x7fb89815a0c0 2023-10-10T20:21:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb8967fc700 1 -- 172.21.15.148:0/48432966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000b810 con 0x7fb89815a0c0 2023-10-10T20:21:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb8967fc700 1 -- 172.21.15.148:0/48432966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000bbb0 con 0x7fb89815a0c0 2023-10-10T20:21:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb89db79700 1 -- 172.21.15.148:0/48432966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89815a0c0 msgr2=0x7fb898068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.112+0000 7fb89db79700 1 --2- 172.21.15.148:0/48432966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89815a0c0 0x7fb898068390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb89806a870 tx=0x7fb8900091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 -- 172.21.15.148:0/48432966 shutdown_connections 2023-10-10T20:21:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 --2- 172.21.15.148:0/48432966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb89815a0c0 0x7fb898068390 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 --2- 172.21.15.148:0/48432966 >> v2:172.21.15.161:3300/0 conn(0x7fb8980cf1c0 0x7fb89815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 -- 172.21.15.148:0/48432966 >> 172.21.15.148:0/48432966 conn(0x7fb89806c6e0 msgr2=0x7fb898069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:28.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 -- 172.21.15.148:0/48432966 shutdown_connections 2023-10-10T20:21:28.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.113+0000 7fb89db79700 1 -- 172.21.15.148:0/48432966 wait complete. 2023-10-10T20:21:28.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.114+0000 7fb89db79700 1 Processor -- start 2023-10-10T20:21:28.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.114+0000 7fb89db79700 1 -- start start 2023-10-10T20:21:28.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 0x7fb8981697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 0x7fb8980ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816a850 con 0x7fb8980cf1c0 2023-10-10T20:21:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980b1e00 con 0x7fb898159cf0 2023-10-10T20:21:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89816a290 con 0x7fb89815a0c0 2023-10-10T20:21:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37948/0 (socket says 172.21.15.148:37948) 2023-10-10T20:21:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 -- 172.21.15.148:0/2841631697 learned_addr learned my addr 172.21.15.148:0/2841631697 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb896ffd700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 0x7fb8981697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb897fff700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 0x7fb8980ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 -- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 msgr2=0x7fb8980ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 0x7fb8980ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 -- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 msgr2=0x7fb8981697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 0x7fb8981697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 -- 172.21.15.148:0/2841631697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890009420 con 0x7fb8980cf1c0 2023-10-10T20:21:28.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb897fff700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 0x7fb8980ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:28.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb896ffd700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 0x7fb8981697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:28.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb8977fe700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb88800cd80 tx=0x7fb888001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88800ab10 con 0x7fb8980cf1c0 2023-10-10T20:21:28.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88800ac70 con 0x7fb8980cf1c0 2023-10-10T20:21:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89816a570 con 0x7fb8980cf1c0 2023-10-10T20:21:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8880070f0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.115+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb898155c90 con 0x7fb8980cf1c0 2023-10-10T20:21:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.116+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8980b18f0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.117+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8880074e0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.117+0000 7fb87f7fe700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 0x7fb8840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.117+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb888084b40 con 0x7fb8980cf1c0 2023-10-10T20:21:28.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.117+0000 7fb896ffd700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 0x7fb8840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.117+0000 7fb896ffd700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 0x7fb8840693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.121+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8980b18f0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.303+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8980b18f0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.304+0000 7fb87f7fe700 1 -- 172.21.15.148:0/2841631697 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fb8980b18f0 con 0x7fb8980cf1c0 2023-10-10T20:21:28.305 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:28.306 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":65,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:26.993874+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:28.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 msgr2=0x7fb8840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 0x7fb8840693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c008040 comp rx=0 tx=0).stop 2023-10-10T20:21:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 msgr2=0x7fb898155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb88800cd80 tx=0x7fb888001ae0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 shutdown_connections 2023-10-10T20:21:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb884066f30 0x7fb8840693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb89815a0c0 0x7fb8980ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8980cf1c0 0x7fb898155750 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 --2- 172.21.15.148:0/2841631697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb898159cf0 0x7fb8981697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 >> 172.21.15.148:0/2841631697 conn(0x7fb89806c6e0 msgr2=0x7fb8980c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 shutdown_connections 2023-10-10T20:21:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.306+0000 7fb89db79700 1 -- 172.21.15.148:0/2841631697 wait complete. 2023-10-10T20:21:28.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:21:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.437+0000 7f7b79a54700 1 Processor -- start 2023-10-10T20:21:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.437+0000 7f7b79a54700 1 -- start start 2023-10-10T20:21:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b79a54700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b74152300 0x7f7b7415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b79a54700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7415c8d0 con 0x7f7b74158df0 2023-10-10T20:21:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b79a54700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740c9370 con 0x7f7b7406d370 2023-10-10T20:21:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b79a54700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b740b1400 con 0x7f7b74152300 2023-10-10T20:21:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b72ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7b74152300 0x7f7b7415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b6a7fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7b74158df0 0x7f7b740caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58666/0 (socket says 172.21.15.148:58666) 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.438+0000 7f7b6a7fc700 1 -- 172.21.15.148:0/522186130 learned_addr learned my addr 172.21.15.148:0/522186130 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b72ffd700 1 -- 172.21.15.148:0/522186130 >> v1:172.21.15.161:6789/0 conn(0x7f7b7406d370 legacy=0x7f7b7406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b72ffd700 1 -- 172.21.15.148:0/522186130 >> v1:172.21.15.148:6789/0 conn(0x7f7b74158df0 legacy=0x7f7b740caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b72ffd700 1 -- 172.21.15.148:0/522186130 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7415c030 con 0x7f7b74152300 2023-10-10T20:21:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b727fc700 1 -- 172.21.15.148:0/522186130 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376294383 0 0) 0x7f7b7415c8d0 con 0x7f7b74158df0 2023-10-10T20:21:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b72ffd700 1 --2- 172.21.15.148:0/522186130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b7415b9a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b640098e0 tx=0x7f7b640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4206c3308c11be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b727fc700 1 -- 172.21.15.148:0/522186130 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64009e80 con 0x7f7b74152300 2023-10-10T20:21:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b727fc700 1 -- 172.21.15.148:0/522186130 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6400b860 con 0x7f7b74152300 2023-10-10T20:21:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b727fc700 1 -- 172.21.15.148:0/522186130 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400bbe0 con 0x7f7b74152300 2023-10-10T20:21:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b79a54700 1 -- 172.21.15.148:0/522186130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 msgr2=0x7f7b7415b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b79a54700 1 --2- 172.21.15.148:0/522186130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b7415b9a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b640098e0 tx=0x7f7b640091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b79a54700 1 -- 172.21.15.148:0/522186130 shutdown_connections 2023-10-10T20:21:28.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b79a54700 1 --2- 172.21.15.148:0/522186130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b7415b9a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.439+0000 7f7b79a54700 1 -- 172.21.15.148:0/522186130 >> 172.21.15.148:0/522186130 conn(0x7f7b74069870 msgr2=0x7f7b74069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:28.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.440+0000 7f7b79a54700 1 -- 172.21.15.148:0/522186130 shutdown_connections 2023-10-10T20:21:28.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.440+0000 7f7b79a54700 1 -- 172.21.15.148:0/522186130 wait complete. 2023-10-10T20:21:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.441+0000 7f7b79a54700 1 Processor -- start 2023-10-10T20:21:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.441+0000 7f7b79a54700 1 -- start start 2023-10-10T20:21:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7406d370 0x7f7b740b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 0x7f7b74168030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74168570 con 0x7f7b74152300 2023-10-10T20:21:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b36e0 con 0x7f7b7406d370 2023-10-10T20:21:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b2b10 con 0x7f7b74158df0 2023-10-10T20:21:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37972/0 (socket says 172.21.15.148:37972) 2023-10-10T20:21:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 -- 172.21.15.148:0/3731492369 learned_addr learned my addr 172.21.15.148:0/3731492369 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b72ffd700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7406d370 0x7f7b740b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b737fe700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 0x7f7b74168030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 -- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 msgr2=0x7f7b74168030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 0x7f7b74168030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 -- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7406d370 msgr2=0x7f7b740b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:28.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7406d370 0x7f7b740b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:28.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 -- 172.21.15.148:0/3731492369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009420 con 0x7f7b74152300 2023-10-10T20:21:28.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b737fe700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 0x7f7b74168030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:28.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6a7fc700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c560 tx=0x7f7b5c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c005b30 con 0x7f7b74152300 2023-10-10T20:21:28.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c005c90 con 0x7f7b74152300 2023-10-10T20:21:28.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.442+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740b2db0 con 0x7f7b74152300 2023-10-10T20:21:28.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.443+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007320 con 0x7f7b74152300 2023-10-10T20:21:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.443+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b740b3370 con 0x7f7b74152300 2023-10-10T20:21:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.444+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7404c3c0 con 0x7f7b74152300 2023-10-10T20:21:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.444+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b5c01b3b0 con 0x7f7b74152300 2023-10-10T20:21:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.444+0000 7f7b6b7fe700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.444+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b5c0839c0 con 0x7f7b74152300 2023-10-10T20:21:28.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.444+0000 7f7b72ffd700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:28.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.445+0000 7f7b72ffd700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 0x7f7b540693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b640093f0 tx=0x7f7b64017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:28.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.449+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b5c050c30 con 0x7f7b74152300 2023-10-10T20:21:28.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:28.676+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7b740b3820 con 0x7f7b74152300 2023-10-10T20:21:29.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.669+0000 7f7b6b7fe700 1 -- 172.21.15.148:0/3731492369 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7b5c0170b0 con 0x7f7b74152300 2023-10-10T20:21:29.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 msgr2=0x7f7b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 0x7f7b540693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b640093f0 tx=0x7f7b64017000 comp rx=0 tx=0).stop 2023-10-10T20:21:29.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 msgr2=0x7f7b74167af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c560 tx=0x7f7b5c000fe0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 shutdown_connections 2023-10-10T20:21:29.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b54066f30 0x7f7b540693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b74158df0 0x7f7b74168030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b7406d370 0x7f7b740b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 --2- 172.21.15.148:0/3731492369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b74152300 0x7f7b74167af0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 >> 172.21.15.148:0/3731492369 conn(0x7f7b74069870 msgr2=0x7f7b740c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:29.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 shutdown_connections 2023-10-10T20:21:29.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.672+0000 7f7b79a54700 1 -- 172.21.15.148:0/3731492369 wait complete. 2023-10-10T20:21:29.686 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:21:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.804+0000 7f5c25163700 1 Processor -- start 2023-10-10T20:21:29.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.804+0000 7f5c25163700 1 -- start start 2023-10-10T20:21:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.804+0000 7f5c25163700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c20159ec0 0x7f5c200af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c25163700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200b0790 con 0x7f5c2006acb0 2023-10-10T20:21:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c25163700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200ce1f0 con 0x7f5c2015a290 2023-10-10T20:21:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c25163700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200b0b20 con 0x7f5c20159ec0 2023-10-10T20:21:29.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1f59e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5c2006acb0 0x7f5c200cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58682/0 (socket says 172.21.15.148:58682) 2023-10-10T20:21:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1f59e700 1 -- 172.21.15.148:0/2856402280 learned_addr learned my addr 172.21.15.148:0/2856402280 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/2856402280 >> v2:172.21.15.161:3300/0 conn(0x7f5c20159ec0 0x7f5c200af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687669710 0 0) 0x7f5c200b0790 con 0x7f5c2006acb0 2023-10-10T20:21:29.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c003570 con 0x7f5c2006acb0 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2055690153 0 0) 0x7f5c0c003570 con 0x7f5c2006acb0 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 >> v2:172.21.15.161:3300/0 conn(0x7f5c20159ec0 msgr2=0x7f5c200af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 --2- 172.21.15.148:0/2856402280 >> v2:172.21.15.161:3300/0 conn(0x7f5c20159ec0 0x7f5c200af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 >> v1:172.21.15.161:6789/0 conn(0x7f5c2015a290 legacy=0x7f5c2006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200b04b0 con 0x7f5c2006acb0 2023-10-10T20:21:29.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f5c180017f0 con 0x7f5c2006acb0 2023-10-10T20:21:29.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/2856402280 >> v2:172.21.15.161:3300/0 conn(0x7f5c20159ec0 0x7f5c200af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:29.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 >> v1:172.21.15.148:6789/0 conn(0x7f5c2006acb0 legacy=0x7f5c200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c0c003a60 0x7f5c0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c0051c0 0x7f5c0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b0b20 con 0x7f5c0c004810 2023-10-10T20:21:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c00c160 con 0x7f5c0c0051c0 2023-10-10T20:21:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c00c320 con 0x7f5c0c003a60 2023-10-10T20:21:29.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.805+0000 7f5c17fff700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c0c003a60 0x7f5c0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1f59e700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c0051c0 0x7f5c0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 -- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c0c003a60 msgr2=0x7f5c0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c0c003a60 0x7f5c0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 -- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c0051c0 msgr2=0x7f5c0c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c0051c0 0x7f5c0c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 -- 172.21.15.148:0/2856402280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c003570 con 0x7f5c0c004810 2023-10-10T20:21:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c0c003a60 0x7f5c0c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:29.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1f59e700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c0051c0 0x7f5c0c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1f59e700 1 -- 172.21.15.148:0/2856402280 reap_dead start 2023-10-10T20:21:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c17fff700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c0c004c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c080018b0 tx=0x7f5c08007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d76693f3eff50 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08012420 con 0x7f5c0c004810 2023-10-10T20:21:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c080125a0 con 0x7f5c0c004810 2023-10-10T20:21:29.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c1e59c700 1 -- 172.21.15.148:0/2856402280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08012960 con 0x7f5c0c004810 2023-10-10T20:21:29.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c25163700 1 -- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 msgr2=0x7f5c0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.806+0000 7f5c25163700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c0c004c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c080018b0 tx=0x7f5c08007d30 comp rx=0 tx=0).stop 2023-10-10T20:21:29.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.807+0000 7f5c25163700 1 -- 172.21.15.148:0/2856402280 shutdown_connections 2023-10-10T20:21:29.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.807+0000 7f5c25163700 1 --2- 172.21.15.148:0/2856402280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c0c004c80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.807+0000 7f5c25163700 1 -- 172.21.15.148:0/2856402280 >> 172.21.15.148:0/2856402280 conn(0x7f5c200d1880 msgr2=0x7f5c200cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:29.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.807+0000 7f5c25163700 1 -- 172.21.15.148:0/2856402280 shutdown_connections 2023-10-10T20:21:29.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.807+0000 7f5c25163700 1 -- 172.21.15.148:0/2856402280 wait complete. 2023-10-10T20:21:29.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 Processor -- start 2023-10-10T20:21:29.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 -- start start 2023-10-10T20:21:29.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c2016a3d0 0x7f5c2016a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 0x7f5c20158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20159320 con 0x7f5c0c004810 2023-10-10T20:21:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c8970 con 0x7f5c20158940 2023-10-10T20:21:29.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c25163700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c94f0 con 0x7f5c2016a3d0 2023-10-10T20:21:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c1ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c1ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37984/0 (socket says 172.21.15.148:37984) 2023-10-10T20:21:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.808+0000 7f5c1ed9d700 1 -- 172.21.15.148:0/3407663286 learned_addr learned my addr 172.21.15.148:0/3407663286 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c17fff700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c2016a3d0 0x7f5c2016a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1f59e700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 0x7f5c20158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 -- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c2016a3d0 msgr2=0x7f5c2016a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c2016a3d0 0x7f5c2016a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 -- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 msgr2=0x7f5c20158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 0x7f5c20158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:29.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 -- 172.21.15.148:0/3407663286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200c8bc0 con 0x7f5c0c004810 2023-10-10T20:21:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1f59e700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 0x7f5c20158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c1ed9d700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5c10011890 tx=0x7f5c10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10015670 con 0x7f5c0c004810 2023-10-10T20:21:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100157d0 con 0x7f5c0c004810 2023-10-10T20:21:29.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c200c8ea0 con 0x7f5c0c004810 2023-10-10T20:21:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10015b50 con 0x7f5c0c004810 2023-10-10T20:21:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.809+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c200c9980 con 0x7f5c0c004810 2023-10-10T20:21:29.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.810+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2004c3c0 con 0x7f5c0c004810 2023-10-10T20:21:29.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.810+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c1000f470 con 0x7f5c0c004810 2023-10-10T20:21:29.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.811+0000 7f5c16ffd700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 0x7f5c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:29.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.811+0000 7f5c17fff700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 0x7f5c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:29.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.811+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c10084d40 con 0x7f5c0c004810 2023-10-10T20:21:29.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.811+0000 7f5c17fff700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 0x7f5c000693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c200b03c0 tx=0x7f5c08015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:29.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:29.815+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c10051fb0 con 0x7f5c0c004810 2023-10-10T20:21:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.041+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c200ceab0 con 0x7f5c0c004810 2023-10-10T20:21:30.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.671+0000 7f5c16ffd700 1 -- 172.21.15.148:0/3407663286 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c100555d0 con 0x7f5c0c004810 2023-10-10T20:21:30.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 msgr2=0x7f5c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:30.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 0x7f5c000693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5c200b03c0 tx=0x7f5c08015000 comp rx=0 tx=0).stop 2023-10-10T20:21:30.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 msgr2=0x7f5c2016d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:30.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5c10011890 tx=0x7f5c10008040 comp rx=0 tx=0).stop 2023-10-10T20:21:30.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 shutdown_connections 2023-10-10T20:21:30.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c00066f30 0x7f5c000693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c20158940 0x7f5c20158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c0c004810 0x7f5c2016d2c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 --2- 172.21.15.148:0/3407663286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c2016a3d0 0x7f5c2016a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 >> 172.21.15.148:0/3407663286 conn(0x7f5c200d1880 msgr2=0x7f5c200d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:30.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 shutdown_connections 2023-10-10T20:21:30.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.673+0000 7f5c25163700 1 -- 172.21.15.148:0/3407663286 wait complete. 2023-10-10T20:21:30.687 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:30.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 Processor -- start 2023-10-10T20:21:30.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 -- start start 2023-10-10T20:21:30.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd60157870 0x7fdd6015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd6015a620 0x7fdd6015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdd6015d580 0x7fdd60158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015cbb0 con 0x7fdd6015a620 2023-10-10T20:21:30.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015bfa0 con 0x7fdd6015d580 2023-10-10T20:21:30.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.802+0000 7fdd65679700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6015c280 con 0x7fdd60157870 2023-10-10T20:21:30.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd6015a620 0x7fdd6015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd6015a620 0x7fdd6015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37992/0 (socket says 172.21.15.148:37992) 2023-10-10T20:21:30.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 -- 172.21.15.148:0/1097603257 learned_addr learned my addr 172.21.15.148:0/1097603257 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:30.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd5f7fe700 1 --2- 172.21.15.148:0/1097603257 >> v2:172.21.15.161:3300/0 conn(0x7fdd6015d580 0x7fdd60158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd5effd700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd60157870 0x7fdd6015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 -- 172.21.15.148:0/1097603257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd60157870 msgr2=0x7fdd6015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:30.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd60157870 0x7fdd6015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 -- 172.21.15.148:0/1097603257 >> v2:172.21.15.161:3300/0 conn(0x7fdd6015d580 msgr2=0x7fdd60158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:30.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 --2- 172.21.15.148:0/1097603257 >> v2:172.21.15.161:3300/0 conn(0x7fdd6015d580 0x7fdd60158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 -- 172.21.15.148:0/1097603257 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6015c560 con 0x7fdd6015a620 2023-10-10T20:21:30.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd567fc700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015a620 0x7fdd6015d040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdd600b03b0 tx=0x7fdd48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fae2926a77dfec server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:30.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd5e7fc700 1 -- 172.21.15.148:0/1097603257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800b860 con 0x7fdd6015a620 2023-10-10T20:21:30.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd5e7fc700 1 -- 172.21.15.148:0/1097603257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800b9c0 con 0x7fdd6015a620 2023-10-10T20:21:30.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.803+0000 7fdd5e7fc700 1 -- 172.21.15.148:0/1097603257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800bd60 con 0x7fdd6015a620 2023-10-10T20:21:30.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 -- 172.21.15.148:0/1097603257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015a620 msgr2=0x7fdd6015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:30.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015a620 0x7fdd6015d040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdd600b03b0 tx=0x7fdd48009250 comp rx=0 tx=0).stop 2023-10-10T20:21:30.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 -- 172.21.15.148:0/1097603257 shutdown_connections 2023-10-10T20:21:30.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 --2- 172.21.15.148:0/1097603257 >> v2:172.21.15.161:3300/0 conn(0x7fdd6015d580 0x7fdd60158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd60157870 0x7fdd6015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 --2- 172.21.15.148:0/1097603257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015a620 0x7fdd6015d040 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 -- 172.21.15.148:0/1097603257 >> 172.21.15.148:0/1097603257 conn(0x7fdd60068ce0 msgr2=0x7fdd600690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:30.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 -- 172.21.15.148:0/1097603257 shutdown_connections 2023-10-10T20:21:30.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.804+0000 7fdd65679700 1 -- 172.21.15.148:0/1097603257 wait complete. 2023-10-10T20:21:30.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.805+0000 7fdd65679700 1 Processor -- start 2023-10-10T20:21:30.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.805+0000 7fdd65679700 1 -- start start 2023-10-10T20:21:30.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 0x7fdd601497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd6015a620 0x7fdd601554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b1f10 con 0x7fdd6015d580 2023-10-10T20:21:30.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6016ddd0 con 0x7fdd60157870 2023-10-10T20:21:30.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd65679700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6016e0b0 con 0x7fdd6015a620 2023-10-10T20:21:30.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37994/0 (socket says 172.21.15.148:37994) 2023-10-10T20:21:30.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 -- 172.21.15.148:0/3967164582 learned_addr learned my addr 172.21.15.148:0/3967164582 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:30.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd567fc700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd6015a620 0x7fdd601554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5effd700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 0x7fdd601497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 -- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd6015a620 msgr2=0x7fdd601554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:30.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd6015a620 0x7fdd601554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 -- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 msgr2=0x7fdd601497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:30.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 0x7fdd601497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:30.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 -- 172.21.15.148:0/3967164582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480094f0 con 0x7fdd6015d580 2023-10-10T20:21:30.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5effd700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 0x7fdd601497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:30.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd5f7fe700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdd58007ef0 tx=0x7fdd58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:30.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.806+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800ec10 con 0x7fdd6015d580 2023-10-10T20:21:30.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.807+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800ed70 con 0x7fdd6015d580 2023-10-10T20:21:30.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.807+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6016e390 con 0x7fdd6015d580 2023-10-10T20:21:30.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.807+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58014670 con 0x7fdd6015d580 2023-10-10T20:21:30.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.807+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6016e950 con 0x7fdd6015d580 2023-10-10T20:21:30.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.808+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6016eb40 con 0x7fdd6015d580 2023-10-10T20:21:30.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.808+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd58014a60 con 0x7fdd6015d580 2023-10-10T20:21:30.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.808+0000 7fdd577fe700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 0x7fdd40069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:30.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.808+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd58084d10 con 0x7fdd6015d580 2023-10-10T20:21:30.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.808+0000 7fdd5effd700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 0x7fdd40069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:30.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.809+0000 7fdd5effd700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 0x7fdd40069460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd50006fd0 tx=0x7fdd50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:30.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:30.813+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd58051ee0 con 0x7fdd6015d580 2023-10-10T20:21:31.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.034+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd6004c3c0 con 0x7fdd6015d580 2023-10-10T20:21:31.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.035+0000 7fdd577fe700 1 -- 172.21.15.148:0/3967164582 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fdd58009980 con 0x7fdd6015d580 2023-10-10T20:21:31.036 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:31.036 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:21:31.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 msgr2=0x7fdd40069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 0x7fdd40069460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd50006fd0 tx=0x7fdd50008040 comp rx=0 tx=0).stop 2023-10-10T20:21:31.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 msgr2=0x7fdd6014b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdd58007ef0 tx=0x7fdd58009840 comp rx=0 tx=0).stop 2023-10-10T20:21:31.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 shutdown_connections 2023-10-10T20:21:31.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd40066fb0 0x7fdd40069460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d580 0x7fdd6014b100 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd60157870 0x7fdd601497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 --2- 172.21.15.148:0/3967164582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdd6015a620 0x7fdd601554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 >> 172.21.15.148:0/3967164582 conn(0x7fdd60068ce0 msgr2=0x7fdd60159860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 shutdown_connections 2023-10-10T20:21:31.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.037+0000 7fdd65679700 1 -- 172.21.15.148:0/3967164582 wait complete. 2023-10-10T20:21:31.041 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:21:31.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:21:31.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.163+0000 7fadb7696700 1 Processor -- start 2023-10-10T20:21:31.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.163+0000 7fadb7696700 1 -- start start 2023-10-10T20:21:31.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb7696700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fadb015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb7696700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb015cbb0 con 0x7fadb0157870 2023-10-10T20:21:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb7696700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb015bfa0 con 0x7fadb015d580 2023-10-10T20:21:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb7696700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb015bdb0 con 0x7fadb015a620 2023-10-10T20:21:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb5432700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fadb0157870 0x7fadb015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58696/0 (socket says 172.21.15.148:58696) 2023-10-10T20:21:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb5432700 1 -- 172.21.15.148:0/3382860807 learned_addr learned my addr 172.21.15.148:0/3382860807 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fadb4c31700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fadb015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436692926 0 0) 0x7fadb015cbb0 con 0x7fadb0157870 2023-10-10T20:21:31.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.164+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad9c003570 con 0x7fadb0157870 2023-10-10T20:21:31.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2466833704 0 0) 0x7fad9c003570 con 0x7fadb0157870 2023-10-10T20:21:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 msgr2=0x7fadb015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fadb015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 >> v1:172.21.15.161:6789/0 conn(0x7fadb015d580 legacy=0x7fadb0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb015c280 con 0x7fadb0157870 2023-10-10T20:21:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fadac001470 con 0x7fadb0157870 2023-10-10T20:21:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 >> v1:172.21.15.148:6789/0 conn(0x7fadb0157870 legacy=0x7fadb015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad9c003a90 0x7fad9c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c004810 0x7fad9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fad9c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c00bb60 con 0x7fad9c003a90 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c00bcd0 con 0x7fad9c004810 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c00be90 con 0x7fadb015a620 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad9c003a90 0x7fad9c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb4c31700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c004810 0x7fad9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5c33700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fad9c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 -- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 msgr2=0x7fad9c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:31.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb015a620 0x7fad9c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 -- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c004810 msgr2=0x7fad9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:31.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c004810 0x7fad9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 -- 172.21.15.148:0/3382860807 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c003570 con 0x7fad9c003a90 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.165+0000 7fadb5432700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad9c003a90 0x7fad9c003ea0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c3c0 tx=0x7fadac001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1506eb80b1683 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac00cc90 con 0x7fad9c003a90 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac00cdf0 con 0x7fad9c003a90 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fada7fff700 1 -- 172.21.15.148:0/3382860807 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac005b50 con 0x7fad9c003a90 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 -- 172.21.15.148:0/3382860807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad9c003a90 msgr2=0x7fad9c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 --2- 172.21.15.148:0/3382860807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad9c003a90 0x7fad9c003ea0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c3c0 tx=0x7fadac001e50 comp rx=0 tx=0).stop 2023-10-10T20:21:31.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb5c33700 1 -- 172.21.15.148:0/3382860807 reap_dead start 2023-10-10T20:21:31.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 -- 172.21.15.148:0/3382860807 shutdown_connections 2023-10-10T20:21:31.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 -- 172.21.15.148:0/3382860807 >> 172.21.15.148:0/3382860807 conn(0x7fadb0068ce0 msgr2=0x7fadb00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 -- 172.21.15.148:0/3382860807 shutdown_connections 2023-10-10T20:21:31.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.166+0000 7fadb7696700 1 -- 172.21.15.148:0/3382860807 wait complete. 2023-10-10T20:21:31.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.167+0000 7fadb7696700 1 Processor -- start 2023-10-10T20:21:31.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.167+0000 7fadb7696700 1 -- start start 2023-10-10T20:21:31.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c003a90 0x7fadb00d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb014a680 0x7fadb014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00d01a0 con 0x7fadb0151090 2023-10-10T20:21:31.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00d1620 con 0x7fad9c003a90 2023-10-10T20:21:31.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb7696700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb014aff0 con 0x7fadb014a680 2023-10-10T20:21:31.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38024/0 (socket says 172.21.15.148:38024) 2023-10-10T20:21:31.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 -- 172.21.15.148:0/983830750 learned_addr learned my addr 172.21.15.148:0/983830750 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:31.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb5c33700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb014a680 0x7fadb014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb5432700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c003a90 0x7fadb00d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 -- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb014a680 msgr2=0x7fadb014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb014a680 0x7fadb014aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 -- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c003a90 msgr2=0x7fadb00d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.168+0000 7fadb4c31700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c003a90 0x7fadb00d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fadb4c31700 1 -- 172.21.15.148:0/983830750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00cf2b0 con 0x7fadb0151090 2023-10-10T20:21:31.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fadb4c31700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fada0011b40 tx=0x7fada00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000f940 con 0x7fadb0151090 2023-10-10T20:21:31.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000faa0 con 0x7fadb0151090 2023-10-10T20:21:31.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb00cf590 con 0x7fadb0151090 2023-10-10T20:21:31.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000fe20 con 0x7fadb0151090 2023-10-10T20:21:31.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.169+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb00cfb50 con 0x7fadb0151090 2023-10-10T20:21:31.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.170+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb004c3c0 con 0x7fadb0151090 2023-10-10T20:21:31.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.170+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada0014650 con 0x7fadb0151090 2023-10-10T20:21:31.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.171+0000 7fada57fa700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 0x7fad940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.171+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fada0018030 con 0x7fadb0151090 2023-10-10T20:21:31.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.171+0000 7fadb5432700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 0x7fad940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.171+0000 7fadb5432700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 0x7fad940693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fadac007cd0 tx=0x7fadac015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.175+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada008b080 con 0x7fadb0151090 2023-10-10T20:21:31.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.359+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadb00b19d0 con 0x7fadb0151090 2023-10-10T20:21:31.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.360+0000 7fada57fa700 1 -- 172.21.15.148:0/983830750 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fada0022090 con 0x7fadb0151090 2023-10-10T20:21:31.361 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:31.361 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":66,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:28.629109+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:21:31.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 msgr2=0x7fad940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 0x7fad940693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fadac007cd0 tx=0x7fadac015000 comp rx=0 tx=0).stop 2023-10-10T20:21:31.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 msgr2=0x7fadb0154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fada0011b40 tx=0x7fada00091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 shutdown_connections 2023-10-10T20:21:31.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad94066f30 0x7fad940693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fadb014a680 0x7fadb014aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c003a90 0x7fadb00d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 --2- 172.21.15.148:0/983830750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadb0151090 0x7fadb0154ef0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 >> 172.21.15.148:0/983830750 conn(0x7fadb0068ce0 msgr2=0x7fadb014c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 shutdown_connections 2023-10-10T20:21:31.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.362+0000 7fadb7696700 1 -- 172.21.15.148:0/983830750 wait complete. 2023-10-10T20:21:31.376 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:21:31.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:21:31.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.490+0000 7fa5798c9700 1 Processor -- start 2023-10-10T20:21:31.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.490+0000 7fa5798c9700 1 -- start start 2023-10-10T20:21:31.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5798c9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5740b0040 0x7fa574157900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5798c9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 0x7fa574070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5798c9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa574071070 con 0x7fa574157530 2023-10-10T20:21:31.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5798c9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa574068f50 con 0x7fa5740b0040 2023-10-10T20:21:31.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5798c9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57415a630 con 0x7fa574070700 2023-10-10T20:21:31.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5740b0040 0x7fa574157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5740b0040 0x7fa574157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38028/0 (socket says 172.21.15.148:38028) 2023-10-10T20:21:31.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 -- 172.21.15.148:0/2267339320 learned_addr learned my addr 172.21.15.148:0/2267339320 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:31.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5737fe700 1 --2- 172.21.15.148:0/2267339320 >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 0x7fa574070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 -- 172.21.15.148:0/2267339320 >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 msgr2=0x7fa574070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 --2- 172.21.15.148:0/2267339320 >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 0x7fa574070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 -- 172.21.15.148:0/2267339320 >> v1:172.21.15.161:6789/0 conn(0x7fa574157530 legacy=0x7fa574068840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 -- 172.21.15.148:0/2267339320 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57415a910 con 0x7fa5740b0040 2023-10-10T20:21:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5737fe700 1 --2- 172.21.15.148:0/2267339320 >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 0x7fa574070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:31.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.491+0000 7fa5727fc700 1 --2- 172.21.15.148:0/2267339320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5740b0040 0x7fa574157900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c4900 tx=0x7fa568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3500cd20bb693f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa571ffb700 1 -- 172.21.15.148:0/2267339320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800b860 con 0x7fa5740b0040 2023-10-10T20:21:31.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa571ffb700 1 -- 172.21.15.148:0/2267339320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800b9c0 con 0x7fa5740b0040 2023-10-10T20:21:31.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa571ffb700 1 -- 172.21.15.148:0/2267339320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800bd60 con 0x7fa5740b0040 2023-10-10T20:21:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 -- 172.21.15.148:0/2267339320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5740b0040 msgr2=0x7fa574157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 --2- 172.21.15.148:0/2267339320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5740b0040 0x7fa574157900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c4900 tx=0x7fa568009250 comp rx=0 tx=0).stop 2023-10-10T20:21:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 -- 172.21.15.148:0/2267339320 shutdown_connections 2023-10-10T20:21:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 --2- 172.21.15.148:0/2267339320 >> v2:172.21.15.161:3300/0 conn(0x7fa574070700 0x7fa574070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 --2- 172.21.15.148:0/2267339320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5740b0040 0x7fa574157900 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 -- 172.21.15.148:0/2267339320 >> 172.21.15.148:0/2267339320 conn(0x7fa5740cd8d0 msgr2=0x7fa5740cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 -- 172.21.15.148:0/2267339320 shutdown_connections 2023-10-10T20:21:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.492+0000 7fa5798c9700 1 -- 172.21.15.148:0/2267339320 wait complete. 2023-10-10T20:21:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.493+0000 7fa5798c9700 1 Processor -- start 2023-10-10T20:21:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.493+0000 7fa5798c9700 1 -- start start 2023-10-10T20:21:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5740b0040 0x7fa5740b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa574157530 0x7fa57416df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c5800 con 0x7fa574070700 2023-10-10T20:21:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c4ed0 con 0x7fa574157530 2023-10-10T20:21:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5798c9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c51b0 con 0x7fa5740b0040 2023-10-10T20:21:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa572ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa572ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38044/0 (socket says 172.21.15.148:38044) 2023-10-10T20:21:31.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa572ffd700 1 -- 172.21.15.148:0/3882868693 learned_addr learned my addr 172.21.15.148:0/3882868693 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:31.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5737fe700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa574157530 0x7fa57416df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.494+0000 7fa5727fc700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5740b0040 0x7fa5740b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 -- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5740b0040 msgr2=0x7fa5740b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5740b0040 0x7fa5740b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 -- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa574157530 msgr2=0x7fa57416df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa574157530 0x7fa57416df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 -- 172.21.15.148:0/3882868693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680094f0 con 0x7fa574070700 2023-10-10T20:21:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa572ffd700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c590 tx=0x7fa56400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011590 con 0x7fa574070700 2023-10-10T20:21:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5640116f0 con 0x7fa574070700 2023-10-10T20:21:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011ab0 con 0x7fa574070700 2023-10-10T20:21:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5740c5490 con 0x7fa574070700 2023-10-10T20:21:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.495+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740b23b0 con 0x7fa574070700 2023-10-10T20:21:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.496+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57406d5d0 con 0x7fa574070700 2023-10-10T20:21:31.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.496+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa56401b3b0 con 0x7fa574070700 2023-10-10T20:21:31.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.497+0000 7fa55affd700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.497+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa564083ae0 con 0x7fa574070700 2023-10-10T20:21:31.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.497+0000 7fa5727fc700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.497+0000 7fa5727fc700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 0x7fa5600693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c4900 tx=0x7fa568017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.501+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa564050d50 con 0x7fa574070700 2023-10-10T20:21:31.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.682+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5740c5040 con 0x7fa560066f30 2023-10-10T20:21:31.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.690+0000 7fa55affd700 1 -- 172.21.15.148:0/3882868693 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282424 (secure 0 0 0) 0x7fa5740c5040 con 0x7fa560066f30 2023-10-10T20:21:31.694 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 msgr2=0x7fa5600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 0x7fa5600693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c4900 tx=0x7fa568017000 comp rx=0 tx=0).stop 2023-10-10T20:21:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 msgr2=0x7fa57414c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c590 tx=0x7fa56400c950 comp rx=0 tx=0).stop 2023-10-10T20:21:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 shutdown_connections 2023-10-10T20:21:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa560066f30 0x7fa5600693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa574157530 0x7fa57416df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa574070700 0x7fa57414c010 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 --2- 172.21.15.148:0/3882868693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5740b0040 0x7fa5740b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 >> 172.21.15.148:0/3882868693 conn(0x7fa5740cd8d0 msgr2=0x7fa57415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 shutdown_connections 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.693+0000 7fa5798c9700 1 -- 172.21.15.148:0/3882868693 wait complete. 2023-10-10T20:21:31.699 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:21:31.706 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-10-10T20:21:31.349093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558826,"statfs":{"total":1600000000,"available":1596238050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734008"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622808+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:00.321727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623111+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:53.930618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623423+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626337+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624256+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:53.522969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624593+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621925+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622817+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:05.985305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621295+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619475+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:16.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619830+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:54.893055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621461+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621861+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619679+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619609+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620549+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624645+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623075+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:30.479823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619523+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.627041+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621230+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624226+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623129+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623916+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:50.103031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625076+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624545+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621937+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:00.095689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622502+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622435+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:33.165713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620489+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624622+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.620953+0000","last_change":"2023-10-10T20:21:26.620953+0000","last_active":"2023-10-10T20:21:26.620953+0000","last_peered":"2023-10-10T20:21:26.620953+0000","last_clean":"2023-10-10T20:21:26.620953+0000","last_became_active":"2023-10-10T20:21:26.620376+0000","last_became_peered":"2023-10-10T20:21:26.620376+0000","last_unstale":"2023-10-10T20:21:26.620953+0000","last_undegraded":"2023-10-10T20:21:26.620953+0000","last_fullsized":"2023-10-10T20:21:26.620953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:40.544270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620202+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:32.400694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624607+0000","last_change":"2023-10-10T20:21:26.624607+0000","last_active":"2023-10-10T20:21:26.624607+0000","last_peered":"2023-10-10T20:21:26.624607+0000","last_clean":"2023-10-10T20:21:26.624607+0000","last_became_active":"2023-10-10T20:21:26.624069+0000","last_became_peered":"2023-10-10T20:21:26.624069+0000","last_unstale":"2023-10-10T20:21:26.624607+0000","last_undegraded":"2023-10-10T20:21:26.624607+0000","last_fullsized":"2023-10-10T20:21:26.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:31.294895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619236+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628774+0000","last_change":"2023-10-10T20:21:26.628774+0000","last_active":"2023-10-10T20:21:26.628774+0000","last_peered":"2023-10-10T20:21:26.628774+0000","last_clean":"2023-10-10T20:21:26.628774+0000","last_became_active":"2023-10-10T20:21:26.628427+0000","last_became_peered":"2023-10-10T20:21:26.628427+0000","last_unstale":"2023-10-10T20:21:26.628774+0000","last_undegraded":"2023-10-10T20:21:26.628774+0000","last_fullsized":"2023-10-10T20:21:26.628774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:11.265574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625398+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631546+0000","last_change":"2023-10-10T20:21:26.631546+0000","last_active":"2023-10-10T20:21:26.631546+0000","last_peered":"2023-10-10T20:21:26.631546+0000","last_clean":"2023-10-10T20:21:26.631546+0000","last_became_active":"2023-10-10T20:21:26.631032+0000","last_became_peered":"2023-10-10T20:21:26.631032+0000","last_unstale":"2023-10-10T20:21:26.631546+0000","last_undegraded":"2023-10-10T20:21:26.631546+0000","last_fullsized":"2023-10-10T20:21:26.631546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:36.676962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619863+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626156+0000","last_change":"2023-10-10T20:21:26.626156+0000","last_active":"2023-10-10T20:21:26.626156+0000","last_peered":"2023-10-10T20:21:26.626156+0000","last_clean":"2023-10-10T20:21:26.626156+0000","last_became_active":"2023-10-10T20:21:26.625622+0000","last_became_peered":"2023-10-10T20:21:26.625622+0000","last_unstale":"2023-10-10T20:21:26.626156+0000","last_undegraded":"2023-10-10T20:21:26.626156+0000","last_fullsized":"2023-10-10T20:21:26.626156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620892+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626984+0000","last_change":"2023-10-10T20:21:26.626984+0000","last_active":"2023-10-10T20:21:26.626984+0000","last_peered":"2023-10-10T20:21:26.626984+0000","last_clean":"2023-10-10T20:21:26.626984+0000","last_became_active":"2023-10-10T20:21:26.626672+0000","last_became_peered":"2023-10-10T20:21:26.626672+0000","last_unstale":"2023-10-10T20:21:26.626984+0000","last_undegraded":"2023-10-10T20:21:26.626984+0000","last_fullsized":"2023-10-10T20:21:26.626984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623403+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630463+0000","last_change":"2023-10-10T20:21:26.630463+0000","last_active":"2023-10-10T20:21:26.630463+0000","last_peered":"2023-10-10T20:21:26.630463+0000","last_clean":"2023-10-10T20:21:26.630463+0000","last_became_active":"2023-10-10T20:21:26.630236+0000","last_became_peered":"2023-10-10T20:21:26.630236+0000","last_unstale":"2023-10-10T20:21:26.630463+0000","last_undegraded":"2023-10-10T20:21:26.630463+0000","last_fullsized":"2023-10-10T20:21:26.630463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622001+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625829+0000","last_change":"2023-10-10T20:21:26.625829+0000","last_active":"2023-10-10T20:21:26.625829+0000","last_peered":"2023-10-10T20:21:26.625829+0000","last_clean":"2023-10-10T20:21:26.625829+0000","last_became_active":"2023-10-10T20:21:26.624923+0000","last_became_peered":"2023-10-10T20:21:26.624923+0000","last_unstale":"2023-10-10T20:21:26.625829+0000","last_undegraded":"2023-10-10T20:21:26.625829+0000","last_fullsized":"2023-10-10T20:21:26.625829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620462+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631482+0000","last_change":"2023-10-10T20:21:26.631482+0000","last_active":"2023-10-10T20:21:26.631482+0000","last_peered":"2023-10-10T20:21:26.631482+0000","last_clean":"2023-10-10T20:21:26.631482+0000","last_became_active":"2023-10-10T20:21:26.630913+0000","last_became_peered":"2023-10-10T20:21:26.630913+0000","last_unstale":"2023-10-10T20:21:26.631482+0000","last_undegraded":"2023-10-10T20:21:26.631482+0000","last_fullsized":"2023-10-10T20:21:26.631482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626651+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626142+0000","last_change":"2023-10-10T20:21:26.626142+0000","last_active":"2023-10-10T20:21:26.626142+0000","last_peered":"2023-10-10T20:21:26.626142+0000","last_clean":"2023-10-10T20:21:26.626142+0000","last_became_active":"2023-10-10T20:21:26.625123+0000","last_became_peered":"2023-10-10T20:21:26.625123+0000","last_unstale":"2023-10-10T20:21:26.626142+0000","last_undegraded":"2023-10-10T20:21:26.626142+0000","last_fullsized":"2023-10-10T20:21:26.626142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:07.454724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620902+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:25.135129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628970+0000","last_change":"2023-10-10T20:21:26.628970+0000","last_active":"2023-10-10T20:21:26.628970+0000","last_peered":"2023-10-10T20:21:26.628970+0000","last_clean":"2023-10-10T20:21:26.628970+0000","last_became_active":"2023-10-10T20:21:26.628746+0000","last_became_peered":"2023-10-10T20:21:26.628746+0000","last_unstale":"2023-10-10T20:21:26.628970+0000","last_undegraded":"2023-10-10T20:21:26.628970+0000","last_fullsized":"2023-10-10T20:21:26.628970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623834+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621850+0000","last_change":"2023-10-10T20:21:26.621850+0000","last_active":"2023-10-10T20:21:26.621850+0000","last_peered":"2023-10-10T20:21:26.621850+0000","last_clean":"2023-10-10T20:21:26.621850+0000","last_became_active":"2023-10-10T20:21:26.621603+0000","last_became_peered":"2023-10-10T20:21:26.621603+0000","last_unstale":"2023-10-10T20:21:26.621850+0000","last_undegraded":"2023-10-10T20:21:26.621850+0000","last_fullsized":"2023-10-10T20:21:26.621850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:39.724039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624659+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626666+0000","last_change":"2023-10-10T20:21:26.626666+0000","last_active":"2023-10-10T20:21:26.626666+0000","last_peered":"2023-10-10T20:21:26.626666+0000","last_clean":"2023-10-10T20:21:26.626666+0000","last_became_active":"2023-10-10T20:21:26.626298+0000","last_became_peered":"2023-10-10T20:21:26.626298+0000","last_unstale":"2023-10-10T20:21:26.626666+0000","last_undegraded":"2023-10-10T20:21:26.626666+0000","last_fullsized":"2023-10-10T20:21:26.626666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:35.386841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621312+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630545+0000","last_change":"2023-10-10T20:21:26.630545+0000","last_active":"2023-10-10T20:21:26.630545+0000","last_peered":"2023-10-10T20:21:26.630545+0000","last_clean":"2023-10-10T20:21:26.630545+0000","last_became_active":"2023-10-10T20:21:26.630288+0000","last_became_peered":"2023-10-10T20:21:26.630288+0000","last_unstale":"2023-10-10T20:21:26.630545+0000","last_undegraded":"2023-10-10T20:21:26.630545+0000","last_fullsized":"2023-10-10T20:21:26.630545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:53.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619453+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:04.202003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627411+0000","last_change":"2023-10-10T20:21:26.627411+0000","last_active":"2023-10-10T20:21:26.627411+0000","last_peered":"2023-10-10T20:21:26.627411+0000","last_clean":"2023-10-10T20:21:26.627411+0000","last_became_active":"2023-10-10T20:21:26.627165+0000","last_became_peered":"2023-10-10T20:21:26.627165+0000","last_unstale":"2023-10-10T20:21:26.627411+0000","last_undegraded":"2023-10-10T20:21:26.627411+0000","last_fullsized":"2023-10-10T20:21:26.627411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622569+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630197+0000","last_change":"2023-10-10T20:21:26.630197+0000","last_active":"2023-10-10T20:21:26.630197+0000","last_peered":"2023-10-10T20:21:26.630197+0000","last_clean":"2023-10-10T20:21:26.630197+0000","last_became_active":"2023-10-10T20:21:26.629955+0000","last_became_peered":"2023-10-10T20:21:26.629955+0000","last_unstale":"2023-10-10T20:21:26.630197+0000","last_undegraded":"2023-10-10T20:21:26.630197+0000","last_fullsized":"2023-10-10T20:21:26.630197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620803+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624742+0000","last_change":"2023-10-10T20:21:26.624742+0000","last_active":"2023-10-10T20:21:26.624742+0000","last_peered":"2023-10-10T20:21:26.624742+0000","last_clean":"2023-10-10T20:21:26.624742+0000","last_became_active":"2023-10-10T20:21:26.624396+0000","last_became_peered":"2023-10-10T20:21:26.624396+0000","last_unstale":"2023-10-10T20:21:26.624742+0000","last_undegraded":"2023-10-10T20:21:26.624742+0000","last_fullsized":"2023-10-10T20:21:26.624742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:43.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625945+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628656+0000","last_change":"2023-10-10T20:21:26.628656+0000","last_active":"2023-10-10T20:21:26.628656+0000","last_peered":"2023-10-10T20:21:26.628656+0000","last_clean":"2023-10-10T20:21:26.628656+0000","last_became_active":"2023-10-10T20:21:26.627500+0000","last_became_peered":"2023-10-10T20:21:26.627500+0000","last_unstale":"2023-10-10T20:21:26.628656+0000","last_undegraded":"2023-10-10T20:21:26.628656+0000","last_fullsized":"2023-10-10T20:21:26.628656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:32.116793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631690+0000","last_change":"2023-10-10T20:21:26.631690+0000","last_active":"2023-10-10T20:21:26.631690+0000","last_peered":"2023-10-10T20:21:26.631690+0000","last_clean":"2023-10-10T20:21:26.631690+0000","last_became_active":"2023-10-10T20:21:26.631324+0000","last_became_peered":"2023-10-10T20:21:26.631324+0000","last_unstale":"2023-10-10T20:21:26.631690+0000","last_undegraded":"2023-10-10T20:21:26.631690+0000","last_fullsized":"2023-10-10T20:21:26.631690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:51.891574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624757+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627796+0000","last_change":"2023-10-10T20:21:26.627796+0000","last_active":"2023-10-10T20:21:26.627796+0000","last_peered":"2023-10-10T20:21:26.627796+0000","last_clean":"2023-10-10T20:21:26.627796+0000","last_became_active":"2023-10-10T20:21:26.627566+0000","last_became_peered":"2023-10-10T20:21:26.627566+0000","last_unstale":"2023-10-10T20:21:26.627796+0000","last_undegraded":"2023-10-10T20:21:26.627796+0000","last_fullsized":"2023-10-10T20:21:26.627796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:25.600786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625561+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626981+0000","last_change":"2023-10-10T20:21:26.626981+0000","last_active":"2023-10-10T20:21:26.626981+0000","last_peered":"2023-10-10T20:21:26.626981+0000","last_clean":"2023-10-10T20:21:26.626981+0000","last_became_active":"2023-10-10T20:21:26.626635+0000","last_became_peered":"2023-10-10T20:21:26.626635+0000","last_unstale":"2023-10-10T20:21:26.626981+0000","last_undegraded":"2023-10-10T20:21:26.626981+0000","last_fullsized":"2023-10-10T20:21:26.626981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:32.511186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620109+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624041+0000","last_change":"2023-10-10T20:21:26.624041+0000","last_active":"2023-10-10T20:21:26.624041+0000","last_peered":"2023-10-10T20:21:26.624041+0000","last_clean":"2023-10-10T20:21:26.624041+0000","last_became_active":"2023-10-10T20:21:26.623809+0000","last_became_peered":"2023-10-10T20:21:26.623809+0000","last_unstale":"2023-10-10T20:21:26.624041+0000","last_undegraded":"2023-10-10T20:21:26.624041+0000","last_fullsized":"2023-10-10T20:21:26.624041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:24.387989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619540+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.629529+0000","last_change":"2023-10-10T20:21:26.629529+0000","last_active":"2023-10-10T20:21:26.629529+0000","last_peered":"2023-10-10T20:21:26.629529+0000","last_clean":"2023-10-10T20:21:26.629529+0000","last_became_active":"2023-10-10T20:21:26.629280+0000","last_became_peered":"2023-10-10T20:21:26.629280+0000","last_unstale":"2023-10-10T20:21:26.629529+0000","last_undegraded":"2023-10-10T20:21:26.629529+0000","last_fullsized":"2023-10-10T20:21:26.629529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624689+0000","last_change":"2023-10-10T20:21:26.624689+0000","last_active":"2023-10-10T20:21:26.624689+0000","last_peered":"2023-10-10T20:21:26.624689+0000","last_clean":"2023-10-10T20:21:26.624689+0000","last_became_active":"2023-10-10T20:21:26.624473+0000","last_became_peered":"2023-10-10T20:21:26.624473+0000","last_unstale":"2023-10-10T20:21:26.624689+0000","last_undegraded":"2023-10-10T20:21:26.624689+0000","last_fullsized":"2023-10-10T20:21:26.624689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:30.722252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621024+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626857+0000","last_change":"2023-10-10T20:21:26.626857+0000","last_active":"2023-10-10T20:21:26.626857+0000","last_peered":"2023-10-10T20:21:26.626857+0000","last_clean":"2023-10-10T20:21:26.626857+0000","last_became_active":"2023-10-10T20:21:26.626370+0000","last_became_peered":"2023-10-10T20:21:26.626370+0000","last_unstale":"2023-10-10T20:21:26.626857+0000","last_undegraded":"2023-10-10T20:21:26.626857+0000","last_fullsized":"2023-10-10T20:21:26.626857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619144+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:27.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628016+0000","last_change":"2023-10-10T20:21:26.628016+0000","last_active":"2023-10-10T20:21:26.628016+0000","last_peered":"2023-10-10T20:21:26.628016+0000","last_clean":"2023-10-10T20:21:26.628016+0000","last_became_active":"2023-10-10T20:21:26.627749+0000","last_became_peered":"2023-10-10T20:21:26.627749+0000","last_unstale":"2023-10-10T20:21:26.628016+0000","last_undegraded":"2023-10-10T20:21:26.628016+0000","last_fullsized":"2023-10-10T20:21:26.628016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:31.392459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619667+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624500+0000","last_change":"2023-10-10T20:21:26.624500+0000","last_active":"2023-10-10T20:21:26.624500+0000","last_peered":"2023-10-10T20:21:26.624500+0000","last_clean":"2023-10-10T20:21:26.624500+0000","last_became_active":"2023-10-10T20:21:26.623771+0000","last_became_peered":"2023-10-10T20:21:26.623771+0000","last_unstale":"2023-10-10T20:21:26.624500+0000","last_undegraded":"2023-10-10T20:21:26.624500+0000","last_fullsized":"2023-10-10T20:21:26.624500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620217+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626193+0000","last_change":"2023-10-10T20:21:26.626193+0000","last_active":"2023-10-10T20:21:26.626193+0000","last_peered":"2023-10-10T20:21:26.626193+0000","last_clean":"2023-10-10T20:21:26.626193+0000","last_became_active":"2023-10-10T20:21:26.625852+0000","last_became_peered":"2023-10-10T20:21:26.625852+0000","last_unstale":"2023-10-10T20:21:26.626193+0000","last_undegraded":"2023-10-10T20:21:26.626193+0000","last_fullsized":"2023-10-10T20:21:26.626193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:29.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624370+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628452+0000","last_change":"2023-10-10T20:21:26.628452+0000","last_active":"2023-10-10T20:21:26.628452+0000","last_peered":"2023-10-10T20:21:26.628452+0000","last_clean":"2023-10-10T20:21:26.628452+0000","last_became_active":"2023-10-10T20:21:26.628193+0000","last_became_peered":"2023-10-10T20:21:26.628193+0000","last_unstale":"2023-10-10T20:21:26.628452+0000","last_undegraded":"2023-10-10T20:21:26.628452+0000","last_fullsized":"2023-10-10T20:21:26.628452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:16.104206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620580+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:36.614675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621728+0000","last_change":"2023-10-10T20:21:26.621728+0000","last_active":"2023-10-10T20:21:26.621728+0000","last_peered":"2023-10-10T20:21:26.621728+0000","last_clean":"2023-10-10T20:21:26.621728+0000","last_became_active":"2023-10-10T20:21:26.621493+0000","last_became_peered":"2023-10-10T20:21:26.621493+0000","last_unstale":"2023-10-10T20:21:26.621728+0000","last_undegraded":"2023-10-10T20:21:26.621728+0000","last_fullsized":"2023-10-10T20:21:26.621728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:50.660723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619498+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.623339+0000","last_change":"2023-10-10T20:21:26.623339+0000","last_active":"2023-10-10T20:21:26.623339+0000","last_peered":"2023-10-10T20:21:26.623339+0000","last_clean":"2023-10-10T20:21:26.623339+0000","last_became_active":"2023-10-10T20:21:26.622789+0000","last_became_peered":"2023-10-10T20:21:26.622789+0000","last_unstale":"2023-10-10T20:21:26.623339+0000","last_undegraded":"2023-10-10T20:21:26.623339+0000","last_fullsized":"2023-10-10T20:21:26.623339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:33.771902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619222+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625874+0000","last_change":"2023-10-10T20:21:26.625874+0000","last_active":"2023-10-10T20:21:26.625874+0000","last_peered":"2023-10-10T20:21:26.625874+0000","last_clean":"2023-10-10T20:21:26.625874+0000","last_became_active":"2023-10-10T20:21:26.625638+0000","last_became_peered":"2023-10-10T20:21:26.625638+0000","last_unstale":"2023-10-10T20:21:26.625874+0000","last_undegraded":"2023-10-10T20:21:26.625874+0000","last_fullsized":"2023-10-10T20:21:26.625874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:35.606440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-10T20:21:25.614971+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:21:25.614971+0000","last_peered":"2023-10-10T20:21:25.614971+0000","last_clean":"2023-10-10T20:21:25.614971+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:21:25.614971+0000","last_undegraded":"2023-10-10T20:21:25.614971+0000","last_fullsized":"2023-10-10T20:21:25.614971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47199999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.498}]},{"osd":1,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65500000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.64800000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.64100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38700000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.443,"5min":1.647,"15min":1.647},"last":0.372}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.44600000000000001,"5min":1.524,"15min":1.524},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.432,"15min":0.44700000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.434,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.46200000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.5,"5min":1.673,"15min":1.673},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69399999999999995,"5min":1.633,"15min":1.633},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":1.506,"15min":1.506},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66900000000000004,"5min":1.696,"15min":1.696},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77500000000000002,"5min":1.585,"15min":1.585},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78500000000000003,"5min":1.478,"15min":1.478},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":1.722,"15min":1.722},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":893,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.46000000000000002,"15min":0.48499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40999999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.50600000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.49099999999999999,"15min":0.51300000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53800000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55500000000000005,"5min":1.256,"15min":1.256},"last":0.25700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52400000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.204,"15min":1.204},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.97199999999999998,"15min":1.048},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71899999999999997,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":1.347,"15min":1.347},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":1.173,"15min":1.173},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51100000000000001,"5min":1.389,"15min":1.389},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57299999999999995,"5min":1.256,"15min":1.256},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60099999999999998,"5min":1.135,"15min":1.135},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63400000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55700000000000005,"5min":1.111,"15min":1.111},"last":0.373},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.61699999999999999,"5min":1.413,"15min":1.413},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.083,"15min":1.083},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81699999999999995,"5min":1.363,"15min":1.363},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.224,"15min":1.224},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.29299999999999998,"15min":0.29399999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.47699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46200000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.249},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36199999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63200000000000001,"5min":0.751,"15min":0.751},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46700000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71999999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81599999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.626}]},{"osd":1,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.754},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64200000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:21:31.712 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:21:31.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.859+0000 7f320d814700 1 Processor -- start 2023-10-10T20:21:31.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.859+0000 7f320d814700 1 -- start start 2023-10-10T20:21:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f320d814700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f320806d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f320d814700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3208157050 0x7f320806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f320d814700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3208157c30 con 0x7f320815a330 2023-10-10T20:21:31.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f320d814700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32081584f0 con 0x7f3208157050 2023-10-10T20:21:31.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f320d814700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3208157670 con 0x7f3208156570 2023-10-10T20:21:31.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f32067fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f320815a330 0x7f3208156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58700/0 (socket says 172.21.15.148:58700) 2023-10-10T20:21:31.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.860+0000 7f32067fc700 1 -- 172.21.15.148:0/382437609 learned_addr learned my addr 172.21.15.148:0/382437609 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:31.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 --2- 172.21.15.148:0/382437609 >> v2:172.21.15.161:3300/0 conn(0x7f3208157050 0x7f320806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3206ffd700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f320806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187963561 0 0) 0x7f3208157c30 con 0x7f320815a330 2023-10-10T20:21:31.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f4003570 con 0x7f320815a330 2023-10-10T20:21:31.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 -- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 msgr2=0x7f320806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:31.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f320806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 -- 172.21.15.148:0/382437609 >> v1:172.21.15.148:6789/0 conn(0x7f320815a330 legacy=0x7f3208156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036622666 0 0) 0x7f31f4003570 con 0x7f320815a330 2023-10-10T20:21:31.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 -- 172.21.15.148:0/382437609 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208157950 con 0x7f3208157050 2023-10-10T20:21:31.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3206ffd700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f320806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:31.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f32077fe700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3208157050 0x7f320806d8c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a725433a6245668 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0001470 con 0x7f3208157050 2023-10-10T20:21:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8011420 con 0x7f3208157050 2023-10-10T20:21:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.861+0000 7f3205ffb700 1 -- 172.21.15.148:0/382437609 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80117c0 con 0x7f3208157050 2023-10-10T20:21:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 -- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3208157050 msgr2=0x7f320806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3208157050 0x7f320806d8c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 -- 172.21.15.148:0/382437609 shutdown_connections 2023-10-10T20:21:31.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f320806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 --2- 172.21.15.148:0/382437609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3208157050 0x7f320806d8c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 -- 172.21.15.148:0/382437609 >> 172.21.15.148:0/382437609 conn(0x7f3208067660 msgr2=0x7f3208067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:31.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 -- 172.21.15.148:0/382437609 shutdown_connections 2023-10-10T20:21:31.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.862+0000 7f320d814700 1 -- 172.21.15.148:0/382437609 wait complete. 2023-10-10T20:21:31.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.863+0000 7f320d814700 1 Processor -- start 2023-10-10T20:21:31.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.863+0000 7f320d814700 1 -- start start 2023-10-10T20:21:31.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f3208153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 0x7f32080c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208154240 con 0x7f3208157050 2023-10-10T20:21:31.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208069200 con 0x7f320815a330 2023-10-10T20:21:31.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f320d814700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080694e0 con 0x7f3208156570 2023-10-10T20:21:31.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38060/0 (socket says 172.21.15.148:38060) 2023-10-10T20:21:31.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 -- 172.21.15.148:0/2077680207 learned_addr learned my addr 172.21.15.148:0/2077680207 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:31.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32077fe700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 0x7f32080c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f3206ffd700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f3208153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 -- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 msgr2=0x7f3208153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f3208153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 -- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 msgr2=0x7f32080c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:31.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 0x7f32080c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:31.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32067fc700 1 -- 172.21.15.148:0/2077680207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200009a20 con 0x7f3208157050 2023-10-10T20:21:31.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f3206ffd700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f3208153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:31.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.864+0000 7f32077fe700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 0x7f32080c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:31.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f32067fc700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f31f000cb90 tx=0x7f31f0001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0005b30 con 0x7f3208157050 2023-10-10T20:21:31.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0005cb0 con 0x7f3208157050 2023-10-10T20:21:31.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f8009420 con 0x7f3208157050 2023-10-10T20:21:31.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0007340 con 0x7f3208157050 2023-10-10T20:21:31.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.865+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32080c20e0 con 0x7f3208157050 2023-10-10T20:21:31.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.866+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32080c2300 con 0x7f3208157050 2023-10-10T20:21:31.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.866+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31f000a5d0 con 0x7f3208157050 2023-10-10T20:21:31.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.867+0000 7f31feffd700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 0x7f31e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:31.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.867+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31f0084360 con 0x7f3208157050 2023-10-10T20:21:31.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.867+0000 7f3206ffd700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 0x7f31e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:31.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.867+0000 7f3206ffd700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 0x7f31e80693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32000099f0 tx=0x7f320000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:31.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:31.871+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31f0051650 con 0x7f3208157050 2023-10-10T20:21:32.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.047+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f320804c3c0 con 0x7f31e8066f30 2023-10-10T20:21:32.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.055+0000 7f31feffd700 1 -- 172.21.15.148:0/2077680207 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282424 (secure 0 0 0) 0x7f320804c3c0 con 0x7f31e8066f30 2023-10-10T20:21:32.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 msgr2=0x7f31e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 0x7f31e80693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32000099f0 tx=0x7f320000b040 comp rx=0 tx=0).stop 2023-10-10T20:21:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 msgr2=0x7f32080b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f31f000cb90 tx=0x7f31f0001aa0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 shutdown_connections 2023-10-10T20:21:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31e8066f30 0x7f31e80693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f320815a330 0x7f32080c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3208156570 0x7f3208153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 --2- 172.21.15.148:0/2077680207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3208157050 0x7f32080b10a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 >> 172.21.15.148:0/2077680207 conn(0x7f3208067660 msgr2=0x7f3208155a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 shutdown_connections 2023-10-10T20:21:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.058+0000 7f320d814700 1 -- 172.21.15.148:0/2077680207 wait complete. 2023-10-10T20:21:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:21:32.071 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-10-10T20:21:31.349093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558826,"statfs":{"total":1600000000,"available":1596238050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734008"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622808+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:00.321727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623111+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:53.930618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623423+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626337+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624256+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:53.522969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624593+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621925+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622817+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:05.985305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621295+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619475+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:16.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619830+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:54.893055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621461+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621861+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619679+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619609+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620549+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624645+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623075+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:30.479823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619523+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.627041+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621230+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624226+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623129+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623916+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:50.103031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625076+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624545+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621937+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:00.095689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622502+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622435+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:33.165713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620489+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624622+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.620953+0000","last_change":"2023-10-10T20:21:26.620953+0000","last_active":"2023-10-10T20:21:26.620953+0000","last_peered":"2023-10-10T20:21:26.620953+0000","last_clean":"2023-10-10T20:21:26.620953+0000","last_became_active":"2023-10-10T20:21:26.620376+0000","last_became_peered":"2023-10-10T20:21:26.620376+0000","last_unstale":"2023-10-10T20:21:26.620953+0000","last_undegraded":"2023-10-10T20:21:26.620953+0000","last_fullsized":"2023-10-10T20:21:26.620953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:40.544270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620202+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:32.400694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624607+0000","last_change":"2023-10-10T20:21:26.624607+0000","last_active":"2023-10-10T20:21:26.624607+0000","last_peered":"2023-10-10T20:21:26.624607+0000","last_clean":"2023-10-10T20:21:26.624607+0000","last_became_active":"2023-10-10T20:21:26.624069+0000","last_became_peered":"2023-10-10T20:21:26.624069+0000","last_unstale":"2023-10-10T20:21:26.624607+0000","last_undegraded":"2023-10-10T20:21:26.624607+0000","last_fullsized":"2023-10-10T20:21:26.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:31.294895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619236+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628774+0000","last_change":"2023-10-10T20:21:26.628774+0000","last_active":"2023-10-10T20:21:26.628774+0000","last_peered":"2023-10-10T20:21:26.628774+0000","last_clean":"2023-10-10T20:21:26.628774+0000","last_became_active":"2023-10-10T20:21:26.628427+0000","last_became_peered":"2023-10-10T20:21:26.628427+0000","last_unstale":"2023-10-10T20:21:26.628774+0000","last_undegraded":"2023-10-10T20:21:26.628774+0000","last_fullsized":"2023-10-10T20:21:26.628774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:11.265574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625398+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631546+0000","last_change":"2023-10-10T20:21:26.631546+0000","last_active":"2023-10-10T20:21:26.631546+0000","last_peered":"2023-10-10T20:21:26.631546+0000","last_clean":"2023-10-10T20:21:26.631546+0000","last_became_active":"2023-10-10T20:21:26.631032+0000","last_became_peered":"2023-10-10T20:21:26.631032+0000","last_unstale":"2023-10-10T20:21:26.631546+0000","last_undegraded":"2023-10-10T20:21:26.631546+0000","last_fullsized":"2023-10-10T20:21:26.631546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:36.676962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619863+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626156+0000","last_change":"2023-10-10T20:21:26.626156+0000","last_active":"2023-10-10T20:21:26.626156+0000","last_peered":"2023-10-10T20:21:26.626156+0000","last_clean":"2023-10-10T20:21:26.626156+0000","last_became_active":"2023-10-10T20:21:26.625622+0000","last_became_peered":"2023-10-10T20:21:26.625622+0000","last_unstale":"2023-10-10T20:21:26.626156+0000","last_undegraded":"2023-10-10T20:21:26.626156+0000","last_fullsized":"2023-10-10T20:21:26.626156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620892+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626984+0000","last_change":"2023-10-10T20:21:26.626984+0000","last_active":"2023-10-10T20:21:26.626984+0000","last_peered":"2023-10-10T20:21:26.626984+0000","last_clean":"2023-10-10T20:21:26.626984+0000","last_became_active":"2023-10-10T20:21:26.626672+0000","last_became_peered":"2023-10-10T20:21:26.626672+0000","last_unstale":"2023-10-10T20:21:26.626984+0000","last_undegraded":"2023-10-10T20:21:26.626984+0000","last_fullsized":"2023-10-10T20:21:26.626984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623403+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630463+0000","last_change":"2023-10-10T20:21:26.630463+0000","last_active":"2023-10-10T20:21:26.630463+0000","last_peered":"2023-10-10T20:21:26.630463+0000","last_clean":"2023-10-10T20:21:26.630463+0000","last_became_active":"2023-10-10T20:21:26.630236+0000","last_became_peered":"2023-10-10T20:21:26.630236+0000","last_unstale":"2023-10-10T20:21:26.630463+0000","last_undegraded":"2023-10-10T20:21:26.630463+0000","last_fullsized":"2023-10-10T20:21:26.630463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622001+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625829+0000","last_change":"2023-10-10T20:21:26.625829+0000","last_active":"2023-10-10T20:21:26.625829+0000","last_peered":"2023-10-10T20:21:26.625829+0000","last_clean":"2023-10-10T20:21:26.625829+0000","last_became_active":"2023-10-10T20:21:26.624923+0000","last_became_peered":"2023-10-10T20:21:26.624923+0000","last_unstale":"2023-10-10T20:21:26.625829+0000","last_undegraded":"2023-10-10T20:21:26.625829+0000","last_fullsized":"2023-10-10T20:21:26.625829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620462+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631482+0000","last_change":"2023-10-10T20:21:26.631482+0000","last_active":"2023-10-10T20:21:26.631482+0000","last_peered":"2023-10-10T20:21:26.631482+0000","last_clean":"2023-10-10T20:21:26.631482+0000","last_became_active":"2023-10-10T20:21:26.630913+0000","last_became_peered":"2023-10-10T20:21:26.630913+0000","last_unstale":"2023-10-10T20:21:26.631482+0000","last_undegraded":"2023-10-10T20:21:26.631482+0000","last_fullsized":"2023-10-10T20:21:26.631482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626651+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626142+0000","last_change":"2023-10-10T20:21:26.626142+0000","last_active":"2023-10-10T20:21:26.626142+0000","last_peered":"2023-10-10T20:21:26.626142+0000","last_clean":"2023-10-10T20:21:26.626142+0000","last_became_active":"2023-10-10T20:21:26.625123+0000","last_became_peered":"2023-10-10T20:21:26.625123+0000","last_unstale":"2023-10-10T20:21:26.626142+0000","last_undegraded":"2023-10-10T20:21:26.626142+0000","last_fullsized":"2023-10-10T20:21:26.626142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:07.454724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620902+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:25.135129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628970+0000","last_change":"2023-10-10T20:21:26.628970+0000","last_active":"2023-10-10T20:21:26.628970+0000","last_peered":"2023-10-10T20:21:26.628970+0000","last_clean":"2023-10-10T20:21:26.628970+0000","last_became_active":"2023-10-10T20:21:26.628746+0000","last_became_peered":"2023-10-10T20:21:26.628746+0000","last_unstale":"2023-10-10T20:21:26.628970+0000","last_undegraded":"2023-10-10T20:21:26.628970+0000","last_fullsized":"2023-10-10T20:21:26.628970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623834+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621850+0000","last_change":"2023-10-10T20:21:26.621850+0000","last_active":"2023-10-10T20:21:26.621850+0000","last_peered":"2023-10-10T20:21:26.621850+0000","last_clean":"2023-10-10T20:21:26.621850+0000","last_became_active":"2023-10-10T20:21:26.621603+0000","last_became_peered":"2023-10-10T20:21:26.621603+0000","last_unstale":"2023-10-10T20:21:26.621850+0000","last_undegraded":"2023-10-10T20:21:26.621850+0000","last_fullsized":"2023-10-10T20:21:26.621850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:39.724039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624659+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626666+0000","last_change":"2023-10-10T20:21:26.626666+0000","last_active":"2023-10-10T20:21:26.626666+0000","last_peered":"2023-10-10T20:21:26.626666+0000","last_clean":"2023-10-10T20:21:26.626666+0000","last_became_active":"2023-10-10T20:21:26.626298+0000","last_became_peered":"2023-10-10T20:21:26.626298+0000","last_unstale":"2023-10-10T20:21:26.626666+0000","last_undegraded":"2023-10-10T20:21:26.626666+0000","last_fullsized":"2023-10-10T20:21:26.626666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:35.386841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621312+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630545+0000","last_change":"2023-10-10T20:21:26.630545+0000","last_active":"2023-10-10T20:21:26.630545+0000","last_peered":"2023-10-10T20:21:26.630545+0000","last_clean":"2023-10-10T20:21:26.630545+0000","last_became_active":"2023-10-10T20:21:26.630288+0000","last_became_peered":"2023-10-10T20:21:26.630288+0000","last_unstale":"2023-10-10T20:21:26.630545+0000","last_undegraded":"2023-10-10T20:21:26.630545+0000","last_fullsized":"2023-10-10T20:21:26.630545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:53.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619453+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:04.202003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627411+0000","last_change":"2023-10-10T20:21:26.627411+0000","last_active":"2023-10-10T20:21:26.627411+0000","last_peered":"2023-10-10T20:21:26.627411+0000","last_clean":"2023-10-10T20:21:26.627411+0000","last_became_active":"2023-10-10T20:21:26.627165+0000","last_became_peered":"2023-10-10T20:21:26.627165+0000","last_unstale":"2023-10-10T20:21:26.627411+0000","last_undegraded":"2023-10-10T20:21:26.627411+0000","last_fullsized":"2023-10-10T20:21:26.627411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622569+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630197+0000","last_change":"2023-10-10T20:21:26.630197+0000","last_active":"2023-10-10T20:21:26.630197+0000","last_peered":"2023-10-10T20:21:26.630197+0000","last_clean":"2023-10-10T20:21:26.630197+0000","last_became_active":"2023-10-10T20:21:26.629955+0000","last_became_peered":"2023-10-10T20:21:26.629955+0000","last_unstale":"2023-10-10T20:21:26.630197+0000","last_undegraded":"2023-10-10T20:21:26.630197+0000","last_fullsized":"2023-10-10T20:21:26.630197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620803+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624742+0000","last_change":"2023-10-10T20:21:26.624742+0000","last_active":"2023-10-10T20:21:26.624742+0000","last_peered":"2023-10-10T20:21:26.624742+0000","last_clean":"2023-10-10T20:21:26.624742+0000","last_became_active":"2023-10-10T20:21:26.624396+0000","last_became_peered":"2023-10-10T20:21:26.624396+0000","last_unstale":"2023-10-10T20:21:26.624742+0000","last_undegraded":"2023-10-10T20:21:26.624742+0000","last_fullsized":"2023-10-10T20:21:26.624742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:43.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625945+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628656+0000","last_change":"2023-10-10T20:21:26.628656+0000","last_active":"2023-10-10T20:21:26.628656+0000","last_peered":"2023-10-10T20:21:26.628656+0000","last_clean":"2023-10-10T20:21:26.628656+0000","last_became_active":"2023-10-10T20:21:26.627500+0000","last_became_peered":"2023-10-10T20:21:26.627500+0000","last_unstale":"2023-10-10T20:21:26.628656+0000","last_undegraded":"2023-10-10T20:21:26.628656+0000","last_fullsized":"2023-10-10T20:21:26.628656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:32.116793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631690+0000","last_change":"2023-10-10T20:21:26.631690+0000","last_active":"2023-10-10T20:21:26.631690+0000","last_peered":"2023-10-10T20:21:26.631690+0000","last_clean":"2023-10-10T20:21:26.631690+0000","last_became_active":"2023-10-10T20:21:26.631324+0000","last_became_peered":"2023-10-10T20:21:26.631324+0000","last_unstale":"2023-10-10T20:21:26.631690+0000","last_undegraded":"2023-10-10T20:21:26.631690+0000","last_fullsized":"2023-10-10T20:21:26.631690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:51.891574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624757+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627796+0000","last_change":"2023-10-10T20:21:26.627796+0000","last_active":"2023-10-10T20:21:26.627796+0000","last_peered":"2023-10-10T20:21:26.627796+0000","last_clean":"2023-10-10T20:21:26.627796+0000","last_became_active":"2023-10-10T20:21:26.627566+0000","last_became_peered":"2023-10-10T20:21:26.627566+0000","last_unstale":"2023-10-10T20:21:26.627796+0000","last_undegraded":"2023-10-10T20:21:26.627796+0000","last_fullsized":"2023-10-10T20:21:26.627796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:25.600786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625561+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626981+0000","last_change":"2023-10-10T20:21:26.626981+0000","last_active":"2023-10-10T20:21:26.626981+0000","last_peered":"2023-10-10T20:21:26.626981+0000","last_clean":"2023-10-10T20:21:26.626981+0000","last_became_active":"2023-10-10T20:21:26.626635+0000","last_became_peered":"2023-10-10T20:21:26.626635+0000","last_unstale":"2023-10-10T20:21:26.626981+0000","last_undegraded":"2023-10-10T20:21:26.626981+0000","last_fullsized":"2023-10-10T20:21:26.626981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:32.511186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620109+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624041+0000","last_change":"2023-10-10T20:21:26.624041+0000","last_active":"2023-10-10T20:21:26.624041+0000","last_peered":"2023-10-10T20:21:26.624041+0000","last_clean":"2023-10-10T20:21:26.624041+0000","last_became_active":"2023-10-10T20:21:26.623809+0000","last_became_peered":"2023-10-10T20:21:26.623809+0000","last_unstale":"2023-10-10T20:21:26.624041+0000","last_undegraded":"2023-10-10T20:21:26.624041+0000","last_fullsized":"2023-10-10T20:21:26.624041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:24.387989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619540+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.629529+0000","last_change":"2023-10-10T20:21:26.629529+0000","last_active":"2023-10-10T20:21:26.629529+0000","last_peered":"2023-10-10T20:21:26.629529+0000","last_clean":"2023-10-10T20:21:26.629529+0000","last_became_active":"2023-10-10T20:21:26.629280+0000","last_became_peered":"2023-10-10T20:21:26.629280+0000","last_unstale":"2023-10-10T20:21:26.629529+0000","last_undegraded":"2023-10-10T20:21:26.629529+0000","last_fullsized":"2023-10-10T20:21:26.629529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624689+0000","last_change":"2023-10-10T20:21:26.624689+0000","last_active":"2023-10-10T20:21:26.624689+0000","last_peered":"2023-10-10T20:21:26.624689+0000","last_clean":"2023-10-10T20:21:26.624689+0000","last_became_active":"2023-10-10T20:21:26.624473+0000","last_became_peered":"2023-10-10T20:21:26.624473+0000","last_unstale":"2023-10-10T20:21:26.624689+0000","last_undegraded":"2023-10-10T20:21:26.624689+0000","last_fullsized":"2023-10-10T20:21:26.624689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:30.722252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621024+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626857+0000","last_change":"2023-10-10T20:21:26.626857+0000","last_active":"2023-10-10T20:21:26.626857+0000","last_peered":"2023-10-10T20:21:26.626857+0000","last_clean":"2023-10-10T20:21:26.626857+0000","last_became_active":"2023-10-10T20:21:26.626370+0000","last_became_peered":"2023-10-10T20:21:26.626370+0000","last_unstale":"2023-10-10T20:21:26.626857+0000","last_undegraded":"2023-10-10T20:21:26.626857+0000","last_fullsized":"2023-10-10T20:21:26.626857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619144+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:27.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628016+0000","last_change":"2023-10-10T20:21:26.628016+0000","last_active":"2023-10-10T20:21:26.628016+0000","last_peered":"2023-10-10T20:21:26.628016+0000","last_clean":"2023-10-10T20:21:26.628016+0000","last_became_active":"2023-10-10T20:21:26.627749+0000","last_became_peered":"2023-10-10T20:21:26.627749+0000","last_unstale":"2023-10-10T20:21:26.628016+0000","last_undegraded":"2023-10-10T20:21:26.628016+0000","last_fullsized":"2023-10-10T20:21:26.628016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:31.392459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619667+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624500+0000","last_change":"2023-10-10T20:21:26.624500+0000","last_active":"2023-10-10T20:21:26.624500+0000","last_peered":"2023-10-10T20:21:26.624500+0000","last_clean":"2023-10-10T20:21:26.624500+0000","last_became_active":"2023-10-10T20:21:26.623771+0000","last_became_peered":"2023-10-10T20:21:26.623771+0000","last_unstale":"2023-10-10T20:21:26.624500+0000","last_undegraded":"2023-10-10T20:21:26.624500+0000","last_fullsized":"2023-10-10T20:21:26.624500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620217+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626193+0000","last_change":"2023-10-10T20:21:26.626193+0000","last_active":"2023-10-10T20:21:26.626193+0000","last_peered":"2023-10-10T20:21:26.626193+0000","last_clean":"2023-10-10T20:21:26.626193+0000","last_became_active":"2023-10-10T20:21:26.625852+0000","last_became_peered":"2023-10-10T20:21:26.625852+0000","last_unstale":"2023-10-10T20:21:26.626193+0000","last_undegraded":"2023-10-10T20:21:26.626193+0000","last_fullsized":"2023-10-10T20:21:26.626193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:29.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624370+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628452+0000","last_change":"2023-10-10T20:21:26.628452+0000","last_active":"2023-10-10T20:21:26.628452+0000","last_peered":"2023-10-10T20:21:26.628452+0000","last_clean":"2023-10-10T20:21:26.628452+0000","last_became_active":"2023-10-10T20:21:26.628193+0000","last_became_peered":"2023-10-10T20:21:26.628193+0000","last_unstale":"2023-10-10T20:21:26.628452+0000","last_undegraded":"2023-10-10T20:21:26.628452+0000","last_fullsized":"2023-10-10T20:21:26.628452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:16.104206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620580+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:36.614675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621728+0000","last_change":"2023-10-10T20:21:26.621728+0000","last_active":"2023-10-10T20:21:26.621728+0000","last_peered":"2023-10-10T20:21:26.621728+0000","last_clean":"2023-10-10T20:21:26.621728+0000","last_became_active":"2023-10-10T20:21:26.621493+0000","last_became_peered":"2023-10-10T20:21:26.621493+0000","last_unstale":"2023-10-10T20:21:26.621728+0000","last_undegraded":"2023-10-10T20:21:26.621728+0000","last_fullsized":"2023-10-10T20:21:26.621728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:50.660723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619498+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.623339+0000","last_change":"2023-10-10T20:21:26.623339+0000","last_active":"2023-10-10T20:21:26.623339+0000","last_peered":"2023-10-10T20:21:26.623339+0000","last_clean":"2023-10-10T20:21:26.623339+0000","last_became_active":"2023-10-10T20:21:26.622789+0000","last_became_peered":"2023-10-10T20:21:26.622789+0000","last_unstale":"2023-10-10T20:21:26.623339+0000","last_undegraded":"2023-10-10T20:21:26.623339+0000","last_fullsized":"2023-10-10T20:21:26.623339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:33.771902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619222+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625874+0000","last_change":"2023-10-10T20:21:26.625874+0000","last_active":"2023-10-10T20:21:26.625874+0000","last_peered":"2023-10-10T20:21:26.625874+0000","last_clean":"2023-10-10T20:21:26.625874+0000","last_became_active":"2023-10-10T20:21:26.625638+0000","last_became_peered":"2023-10-10T20:21:26.625638+0000","last_unstale":"2023-10-10T20:21:26.625874+0000","last_undegraded":"2023-10-10T20:21:26.625874+0000","last_fullsized":"2023-10-10T20:21:26.625874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:35.606440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-10T20:21:25.614971+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:21:25.614971+0000","last_peered":"2023-10-10T20:21:25.614971+0000","last_clean":"2023-10-10T20:21:25.614971+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:21:25.614971+0000","last_undegraded":"2023-10-10T20:21:25.614971+0000","last_fullsized":"2023-10-10T20:21:25.614971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47199999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.498}]},{"osd":1,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65500000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.64800000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.64100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38700000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.443,"5min":1.647,"15min":1.647},"last":0.372}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.44600000000000001,"5min":1.524,"15min":1.524},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.432,"15min":0.44700000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.434,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.46200000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.5,"5min":1.673,"15min":1.673},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69399999999999995,"5min":1.633,"15min":1.633},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":1.506,"15min":1.506},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66900000000000004,"5min":1.696,"15min":1.696},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77500000000000002,"5min":1.585,"15min":1.585},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78500000000000003,"5min":1.478,"15min":1.478},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":1.722,"15min":1.722},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":893,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.46000000000000002,"15min":0.48499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40999999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.50600000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.49099999999999999,"15min":0.51300000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53800000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55500000000000005,"5min":1.256,"15min":1.256},"last":0.25700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52400000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.204,"15min":1.204},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.97199999999999998,"15min":1.048},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71899999999999997,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":1.347,"15min":1.347},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":1.173,"15min":1.173},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51100000000000001,"5min":1.389,"15min":1.389},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57299999999999995,"5min":1.256,"15min":1.256},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60099999999999998,"5min":1.135,"15min":1.135},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63400000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55700000000000005,"5min":1.111,"15min":1.111},"last":0.373},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.61699999999999999,"5min":1.413,"15min":1.413},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.083,"15min":1.083},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81699999999999995,"5min":1.363,"15min":1.363},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.224,"15min":1.224},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.29299999999999998,"15min":0.29399999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.47699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46200000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.249},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36199999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63200000000000001,"5min":0.751,"15min":0.751},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46700000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71999999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81599999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.626}]},{"osd":1,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.754},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64200000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:21:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:21:32.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.225+0000 7f7022aaf700 1 Processor -- start 2023-10-10T20:21:32.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.225+0000 7f7022aaf700 1 -- start start 2023-10-10T20:21:32.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7022aaf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f701c15a620 0x7f701c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7022aaf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f701c15d580 0x7f701c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7022aaf700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c15cbb0 con 0x7f701c157870 2023-10-10T20:21:32.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7022aaf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c15bdb0 con 0x7f701c15d580 2023-10-10T20:21:32.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7022aaf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c15bfa0 con 0x7f701c15a620 2023-10-10T20:21:32.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f702104c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f701c15d580 0x7f701c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f702104c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f701c15d580 0x7f701c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38074/0 (socket says 172.21.15.148:38074) 2023-10-10T20:21:32.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f702104c700 1 -- 172.21.15.148:0/31784657 learned_addr learned my addr 172.21.15.148:0/31784657 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:32.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.226+0000 7f7013fff700 1 --2- 172.21.15.148:0/31784657 >> v2:172.21.15.161:3300/0 conn(0x7f701c15a620 0x7f701c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f702104c700 1 -- 172.21.15.148:0/31784657 >> v2:172.21.15.161:3300/0 conn(0x7f701c15a620 msgr2=0x7f701c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f702104c700 1 --2- 172.21.15.148:0/31784657 >> v2:172.21.15.161:3300/0 conn(0x7f701c15a620 0x7f701c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f702104c700 1 -- 172.21.15.148:0/31784657 >> v1:172.21.15.161:6789/0 conn(0x7f701c157870 legacy=0x7f701c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f702104c700 1 -- 172.21.15.148:0/31784657 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c15c280 con 0x7f701c15d580 2023-10-10T20:21:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f702104c700 1 --2- 172.21.15.148:0/31784657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c15d580 0x7f701c158110 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f701c15cb20 tx=0x7f7014009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f1376dd519b004 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f70137fe700 1 -- 172.21.15.148:0/31784657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014009e70 con 0x7f701c15d580 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f70137fe700 1 -- 172.21.15.148:0/31784657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701400b860 con 0x7f701c15d580 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f70137fe700 1 -- 172.21.15.148:0/31784657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400bc00 con 0x7f701c15d580 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f7022aaf700 1 -- 172.21.15.148:0/31784657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c15d580 msgr2=0x7f701c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.227+0000 7f7022aaf700 1 --2- 172.21.15.148:0/31784657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c15d580 0x7f701c158110 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f701c15cb20 tx=0x7f7014009250 comp rx=0 tx=0).stop 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 -- 172.21.15.148:0/31784657 shutdown_connections 2023-10-10T20:21:32.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 --2- 172.21.15.148:0/31784657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c15d580 0x7f701c158110 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 --2- 172.21.15.148:0/31784657 >> v2:172.21.15.161:3300/0 conn(0x7f701c15a620 0x7f701c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 -- 172.21.15.148:0/31784657 >> 172.21.15.148:0/31784657 conn(0x7f701c068ce0 msgr2=0x7f701c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 -- 172.21.15.148:0/31784657 shutdown_connections 2023-10-10T20:21:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.228+0000 7f7022aaf700 1 -- 172.21.15.148:0/31784657 wait complete. 2023-10-10T20:21:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.229+0000 7f7022aaf700 1 Processor -- start 2023-10-10T20:21:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.229+0000 7f7022aaf700 1 -- start start 2023-10-10T20:21:32.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f701c15a620 0x7f701c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15d580 0x7f701c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1584f0 con 0x7f701c157870 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1587d0 con 0x7f701c15d580 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c158ab0 con 0x7f701c15a620 2023-10-10T20:21:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38082/0 (socket says 172.21.15.148:38082) 2023-10-10T20:21:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 -- 172.21.15.148:0/987223336 learned_addr learned my addr 172.21.15.148:0/987223336 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702104c700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15d580 0x7f701c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7013fff700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f701c15a620 0x7f701c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 -- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f701c15a620 msgr2=0x7f701c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f701c15a620 0x7f701c14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 -- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15d580 msgr2=0x7f701c159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15d580 0x7f701c159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 -- 172.21.15.148:0/987223336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140094f0 con 0x7f701c157870 2023-10-10T20:21:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f702084b700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f701800cb10 tx=0x7f70180017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800ce20 con 0x7f701c157870 2023-10-10T20:21:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701800a7e0 con 0x7f701c157870 2023-10-10T20:21:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c158d90 con 0x7f701c157870 2023-10-10T20:21:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800aba0 con 0x7f701c157870 2023-10-10T20:21:32.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.230+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c150be0 con 0x7f701c157870 2023-10-10T20:21:32.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.231+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c15d340 con 0x7f701c157870 2023-10-10T20:21:32.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.232+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7018003030 con 0x7f701c157870 2023-10-10T20:21:32.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.232+0000 7f7010ff9700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 0x7f70080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.232+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70180844c0 con 0x7f701c157870 2023-10-10T20:21:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.232+0000 7f7013fff700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 0x7f70080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.232+0000 7f7013fff700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 0x7f70080693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f701c15aa20 tx=0x7f700c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.236+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7018051730 con 0x7f701c157870 2023-10-10T20:21:32.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.416+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f701c04c3c0 con 0x7f7008066f30 2023-10-10T20:21:32.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.423+0000 7f7010ff9700 1 -- 172.21.15.148:0/987223336 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282424 (secure 0 0 0) 0x7f701c04c3c0 con 0x7f7008066f30 2023-10-10T20:21:32.428 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:32.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 msgr2=0x7f70080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 0x7f70080693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f701c15aa20 tx=0x7f700c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 msgr2=0x7f701c1506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f701800cb10 tx=0x7f70180017b0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 shutdown_connections 2023-10-10T20:21:32.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7008066f30 0x7f70080693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15d580 0x7f701c159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f701c157870 0x7f701c1506a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 --2- 172.21.15.148:0/987223336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f701c15a620 0x7f701c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 >> 172.21.15.148:0/987223336 conn(0x7f701c068ce0 msgr2=0x7f701c15e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:32.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.426+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 shutdown_connections 2023-10-10T20:21:32.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.427+0000 7f7022aaf700 1 -- 172.21.15.148:0/987223336 wait complete. 2023-10-10T20:21:32.433 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:21:32.439 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-10-10T20:21:31.349093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558826,"statfs":{"total":1600000000,"available":1596238050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734008"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622808+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:00.321727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623111+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:53.930618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623423+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626337+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624256+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:53.522969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624593+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621925+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622817+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:05.985305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621295+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619475+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:16.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619830+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:54.893055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621461+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621861+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619679+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619609+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620549+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624645+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623075+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:30.479823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619523+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.627041+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621230+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624226+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623129+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623916+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:50.103031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625076+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624545+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621937+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:00.095689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622502+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622435+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:33.165713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620489+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624622+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.620953+0000","last_change":"2023-10-10T20:21:26.620953+0000","last_active":"2023-10-10T20:21:26.620953+0000","last_peered":"2023-10-10T20:21:26.620953+0000","last_clean":"2023-10-10T20:21:26.620953+0000","last_became_active":"2023-10-10T20:21:26.620376+0000","last_became_peered":"2023-10-10T20:21:26.620376+0000","last_unstale":"2023-10-10T20:21:26.620953+0000","last_undegraded":"2023-10-10T20:21:26.620953+0000","last_fullsized":"2023-10-10T20:21:26.620953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:40.544270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620202+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:32.400694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624607+0000","last_change":"2023-10-10T20:21:26.624607+0000","last_active":"2023-10-10T20:21:26.624607+0000","last_peered":"2023-10-10T20:21:26.624607+0000","last_clean":"2023-10-10T20:21:26.624607+0000","last_became_active":"2023-10-10T20:21:26.624069+0000","last_became_peered":"2023-10-10T20:21:26.624069+0000","last_unstale":"2023-10-10T20:21:26.624607+0000","last_undegraded":"2023-10-10T20:21:26.624607+0000","last_fullsized":"2023-10-10T20:21:26.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:31.294895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619236+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628774+0000","last_change":"2023-10-10T20:21:26.628774+0000","last_active":"2023-10-10T20:21:26.628774+0000","last_peered":"2023-10-10T20:21:26.628774+0000","last_clean":"2023-10-10T20:21:26.628774+0000","last_became_active":"2023-10-10T20:21:26.628427+0000","last_became_peered":"2023-10-10T20:21:26.628427+0000","last_unstale":"2023-10-10T20:21:26.628774+0000","last_undegraded":"2023-10-10T20:21:26.628774+0000","last_fullsized":"2023-10-10T20:21:26.628774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:11.265574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625398+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631546+0000","last_change":"2023-10-10T20:21:26.631546+0000","last_active":"2023-10-10T20:21:26.631546+0000","last_peered":"2023-10-10T20:21:26.631546+0000","last_clean":"2023-10-10T20:21:26.631546+0000","last_became_active":"2023-10-10T20:21:26.631032+0000","last_became_peered":"2023-10-10T20:21:26.631032+0000","last_unstale":"2023-10-10T20:21:26.631546+0000","last_undegraded":"2023-10-10T20:21:26.631546+0000","last_fullsized":"2023-10-10T20:21:26.631546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:36.676962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619863+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626156+0000","last_change":"2023-10-10T20:21:26.626156+0000","last_active":"2023-10-10T20:21:26.626156+0000","last_peered":"2023-10-10T20:21:26.626156+0000","last_clean":"2023-10-10T20:21:26.626156+0000","last_became_active":"2023-10-10T20:21:26.625622+0000","last_became_peered":"2023-10-10T20:21:26.625622+0000","last_unstale":"2023-10-10T20:21:26.626156+0000","last_undegraded":"2023-10-10T20:21:26.626156+0000","last_fullsized":"2023-10-10T20:21:26.626156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620892+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626984+0000","last_change":"2023-10-10T20:21:26.626984+0000","last_active":"2023-10-10T20:21:26.626984+0000","last_peered":"2023-10-10T20:21:26.626984+0000","last_clean":"2023-10-10T20:21:26.626984+0000","last_became_active":"2023-10-10T20:21:26.626672+0000","last_became_peered":"2023-10-10T20:21:26.626672+0000","last_unstale":"2023-10-10T20:21:26.626984+0000","last_undegraded":"2023-10-10T20:21:26.626984+0000","last_fullsized":"2023-10-10T20:21:26.626984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623403+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630463+0000","last_change":"2023-10-10T20:21:26.630463+0000","last_active":"2023-10-10T20:21:26.630463+0000","last_peered":"2023-10-10T20:21:26.630463+0000","last_clean":"2023-10-10T20:21:26.630463+0000","last_became_active":"2023-10-10T20:21:26.630236+0000","last_became_peered":"2023-10-10T20:21:26.630236+0000","last_unstale":"2023-10-10T20:21:26.630463+0000","last_undegraded":"2023-10-10T20:21:26.630463+0000","last_fullsized":"2023-10-10T20:21:26.630463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622001+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625829+0000","last_change":"2023-10-10T20:21:26.625829+0000","last_active":"2023-10-10T20:21:26.625829+0000","last_peered":"2023-10-10T20:21:26.625829+0000","last_clean":"2023-10-10T20:21:26.625829+0000","last_became_active":"2023-10-10T20:21:26.624923+0000","last_became_peered":"2023-10-10T20:21:26.624923+0000","last_unstale":"2023-10-10T20:21:26.625829+0000","last_undegraded":"2023-10-10T20:21:26.625829+0000","last_fullsized":"2023-10-10T20:21:26.625829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620462+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631482+0000","last_change":"2023-10-10T20:21:26.631482+0000","last_active":"2023-10-10T20:21:26.631482+0000","last_peered":"2023-10-10T20:21:26.631482+0000","last_clean":"2023-10-10T20:21:26.631482+0000","last_became_active":"2023-10-10T20:21:26.630913+0000","last_became_peered":"2023-10-10T20:21:26.630913+0000","last_unstale":"2023-10-10T20:21:26.631482+0000","last_undegraded":"2023-10-10T20:21:26.631482+0000","last_fullsized":"2023-10-10T20:21:26.631482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626651+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626142+0000","last_change":"2023-10-10T20:21:26.626142+0000","last_active":"2023-10-10T20:21:26.626142+0000","last_peered":"2023-10-10T20:21:26.626142+0000","last_clean":"2023-10-10T20:21:26.626142+0000","last_became_active":"2023-10-10T20:21:26.625123+0000","last_became_peered":"2023-10-10T20:21:26.625123+0000","last_unstale":"2023-10-10T20:21:26.626142+0000","last_undegraded":"2023-10-10T20:21:26.626142+0000","last_fullsized":"2023-10-10T20:21:26.626142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:07.454724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620902+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:25.135129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628970+0000","last_change":"2023-10-10T20:21:26.628970+0000","last_active":"2023-10-10T20:21:26.628970+0000","last_peered":"2023-10-10T20:21:26.628970+0000","last_clean":"2023-10-10T20:21:26.628970+0000","last_became_active":"2023-10-10T20:21:26.628746+0000","last_became_peered":"2023-10-10T20:21:26.628746+0000","last_unstale":"2023-10-10T20:21:26.628970+0000","last_undegraded":"2023-10-10T20:21:26.628970+0000","last_fullsized":"2023-10-10T20:21:26.628970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623834+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621850+0000","last_change":"2023-10-10T20:21:26.621850+0000","last_active":"2023-10-10T20:21:26.621850+0000","last_peered":"2023-10-10T20:21:26.621850+0000","last_clean":"2023-10-10T20:21:26.621850+0000","last_became_active":"2023-10-10T20:21:26.621603+0000","last_became_peered":"2023-10-10T20:21:26.621603+0000","last_unstale":"2023-10-10T20:21:26.621850+0000","last_undegraded":"2023-10-10T20:21:26.621850+0000","last_fullsized":"2023-10-10T20:21:26.621850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:39.724039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624659+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626666+0000","last_change":"2023-10-10T20:21:26.626666+0000","last_active":"2023-10-10T20:21:26.626666+0000","last_peered":"2023-10-10T20:21:26.626666+0000","last_clean":"2023-10-10T20:21:26.626666+0000","last_became_active":"2023-10-10T20:21:26.626298+0000","last_became_peered":"2023-10-10T20:21:26.626298+0000","last_unstale":"2023-10-10T20:21:26.626666+0000","last_undegraded":"2023-10-10T20:21:26.626666+0000","last_fullsized":"2023-10-10T20:21:26.626666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:35.386841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621312+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630545+0000","last_change":"2023-10-10T20:21:26.630545+0000","last_active":"2023-10-10T20:21:26.630545+0000","last_peered":"2023-10-10T20:21:26.630545+0000","last_clean":"2023-10-10T20:21:26.630545+0000","last_became_active":"2023-10-10T20:21:26.630288+0000","last_became_peered":"2023-10-10T20:21:26.630288+0000","last_unstale":"2023-10-10T20:21:26.630545+0000","last_undegraded":"2023-10-10T20:21:26.630545+0000","last_fullsized":"2023-10-10T20:21:26.630545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:53.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619453+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:04.202003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627411+0000","last_change":"2023-10-10T20:21:26.627411+0000","last_active":"2023-10-10T20:21:26.627411+0000","last_peered":"2023-10-10T20:21:26.627411+0000","last_clean":"2023-10-10T20:21:26.627411+0000","last_became_active":"2023-10-10T20:21:26.627165+0000","last_became_peered":"2023-10-10T20:21:26.627165+0000","last_unstale":"2023-10-10T20:21:26.627411+0000","last_undegraded":"2023-10-10T20:21:26.627411+0000","last_fullsized":"2023-10-10T20:21:26.627411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622569+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630197+0000","last_change":"2023-10-10T20:21:26.630197+0000","last_active":"2023-10-10T20:21:26.630197+0000","last_peered":"2023-10-10T20:21:26.630197+0000","last_clean":"2023-10-10T20:21:26.630197+0000","last_became_active":"2023-10-10T20:21:26.629955+0000","last_became_peered":"2023-10-10T20:21:26.629955+0000","last_unstale":"2023-10-10T20:21:26.630197+0000","last_undegraded":"2023-10-10T20:21:26.630197+0000","last_fullsized":"2023-10-10T20:21:26.630197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620803+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624742+0000","last_change":"2023-10-10T20:21:26.624742+0000","last_active":"2023-10-10T20:21:26.624742+0000","last_peered":"2023-10-10T20:21:26.624742+0000","last_clean":"2023-10-10T20:21:26.624742+0000","last_became_active":"2023-10-10T20:21:26.624396+0000","last_became_peered":"2023-10-10T20:21:26.624396+0000","last_unstale":"2023-10-10T20:21:26.624742+0000","last_undegraded":"2023-10-10T20:21:26.624742+0000","last_fullsized":"2023-10-10T20:21:26.624742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:43.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625945+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628656+0000","last_change":"2023-10-10T20:21:26.628656+0000","last_active":"2023-10-10T20:21:26.628656+0000","last_peered":"2023-10-10T20:21:26.628656+0000","last_clean":"2023-10-10T20:21:26.628656+0000","last_became_active":"2023-10-10T20:21:26.627500+0000","last_became_peered":"2023-10-10T20:21:26.627500+0000","last_unstale":"2023-10-10T20:21:26.628656+0000","last_undegraded":"2023-10-10T20:21:26.628656+0000","last_fullsized":"2023-10-10T20:21:26.628656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:32.116793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631690+0000","last_change":"2023-10-10T20:21:26.631690+0000","last_active":"2023-10-10T20:21:26.631690+0000","last_peered":"2023-10-10T20:21:26.631690+0000","last_clean":"2023-10-10T20:21:26.631690+0000","last_became_active":"2023-10-10T20:21:26.631324+0000","last_became_peered":"2023-10-10T20:21:26.631324+0000","last_unstale":"2023-10-10T20:21:26.631690+0000","last_undegraded":"2023-10-10T20:21:26.631690+0000","last_fullsized":"2023-10-10T20:21:26.631690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:51.891574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624757+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627796+0000","last_change":"2023-10-10T20:21:26.627796+0000","last_active":"2023-10-10T20:21:26.627796+0000","last_peered":"2023-10-10T20:21:26.627796+0000","last_clean":"2023-10-10T20:21:26.627796+0000","last_became_active":"2023-10-10T20:21:26.627566+0000","last_became_peered":"2023-10-10T20:21:26.627566+0000","last_unstale":"2023-10-10T20:21:26.627796+0000","last_undegraded":"2023-10-10T20:21:26.627796+0000","last_fullsized":"2023-10-10T20:21:26.627796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:25.600786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625561+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626981+0000","last_change":"2023-10-10T20:21:26.626981+0000","last_active":"2023-10-10T20:21:26.626981+0000","last_peered":"2023-10-10T20:21:26.626981+0000","last_clean":"2023-10-10T20:21:26.626981+0000","last_became_active":"2023-10-10T20:21:26.626635+0000","last_became_peered":"2023-10-10T20:21:26.626635+0000","last_unstale":"2023-10-10T20:21:26.626981+0000","last_undegraded":"2023-10-10T20:21:26.626981+0000","last_fullsized":"2023-10-10T20:21:26.626981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:32.511186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620109+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624041+0000","last_change":"2023-10-10T20:21:26.624041+0000","last_active":"2023-10-10T20:21:26.624041+0000","last_peered":"2023-10-10T20:21:26.624041+0000","last_clean":"2023-10-10T20:21:26.624041+0000","last_became_active":"2023-10-10T20:21:26.623809+0000","last_became_peered":"2023-10-10T20:21:26.623809+0000","last_unstale":"2023-10-10T20:21:26.624041+0000","last_undegraded":"2023-10-10T20:21:26.624041+0000","last_fullsized":"2023-10-10T20:21:26.624041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:24.387989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619540+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.629529+0000","last_change":"2023-10-10T20:21:26.629529+0000","last_active":"2023-10-10T20:21:26.629529+0000","last_peered":"2023-10-10T20:21:26.629529+0000","last_clean":"2023-10-10T20:21:26.629529+0000","last_became_active":"2023-10-10T20:21:26.629280+0000","last_became_peered":"2023-10-10T20:21:26.629280+0000","last_unstale":"2023-10-10T20:21:26.629529+0000","last_undegraded":"2023-10-10T20:21:26.629529+0000","last_fullsized":"2023-10-10T20:21:26.629529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624689+0000","last_change":"2023-10-10T20:21:26.624689+0000","last_active":"2023-10-10T20:21:26.624689+0000","last_peered":"2023-10-10T20:21:26.624689+0000","last_clean":"2023-10-10T20:21:26.624689+0000","last_became_active":"2023-10-10T20:21:26.624473+0000","last_became_peered":"2023-10-10T20:21:26.624473+0000","last_unstale":"2023-10-10T20:21:26.624689+0000","last_undegraded":"2023-10-10T20:21:26.624689+0000","last_fullsized":"2023-10-10T20:21:26.624689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:30.722252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621024+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626857+0000","last_change":"2023-10-10T20:21:26.626857+0000","last_active":"2023-10-10T20:21:26.626857+0000","last_peered":"2023-10-10T20:21:26.626857+0000","last_clean":"2023-10-10T20:21:26.626857+0000","last_became_active":"2023-10-10T20:21:26.626370+0000","last_became_peered":"2023-10-10T20:21:26.626370+0000","last_unstale":"2023-10-10T20:21:26.626857+0000","last_undegraded":"2023-10-10T20:21:26.626857+0000","last_fullsized":"2023-10-10T20:21:26.626857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619144+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:27.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628016+0000","last_change":"2023-10-10T20:21:26.628016+0000","last_active":"2023-10-10T20:21:26.628016+0000","last_peered":"2023-10-10T20:21:26.628016+0000","last_clean":"2023-10-10T20:21:26.628016+0000","last_became_active":"2023-10-10T20:21:26.627749+0000","last_became_peered":"2023-10-10T20:21:26.627749+0000","last_unstale":"2023-10-10T20:21:26.628016+0000","last_undegraded":"2023-10-10T20:21:26.628016+0000","last_fullsized":"2023-10-10T20:21:26.628016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:31.392459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619667+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624500+0000","last_change":"2023-10-10T20:21:26.624500+0000","last_active":"2023-10-10T20:21:26.624500+0000","last_peered":"2023-10-10T20:21:26.624500+0000","last_clean":"2023-10-10T20:21:26.624500+0000","last_became_active":"2023-10-10T20:21:26.623771+0000","last_became_peered":"2023-10-10T20:21:26.623771+0000","last_unstale":"2023-10-10T20:21:26.624500+0000","last_undegraded":"2023-10-10T20:21:26.624500+0000","last_fullsized":"2023-10-10T20:21:26.624500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620217+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626193+0000","last_change":"2023-10-10T20:21:26.626193+0000","last_active":"2023-10-10T20:21:26.626193+0000","last_peered":"2023-10-10T20:21:26.626193+0000","last_clean":"2023-10-10T20:21:26.626193+0000","last_became_active":"2023-10-10T20:21:26.625852+0000","last_became_peered":"2023-10-10T20:21:26.625852+0000","last_unstale":"2023-10-10T20:21:26.626193+0000","last_undegraded":"2023-10-10T20:21:26.626193+0000","last_fullsized":"2023-10-10T20:21:26.626193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:29.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624370+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628452+0000","last_change":"2023-10-10T20:21:26.628452+0000","last_active":"2023-10-10T20:21:26.628452+0000","last_peered":"2023-10-10T20:21:26.628452+0000","last_clean":"2023-10-10T20:21:26.628452+0000","last_became_active":"2023-10-10T20:21:26.628193+0000","last_became_peered":"2023-10-10T20:21:26.628193+0000","last_unstale":"2023-10-10T20:21:26.628452+0000","last_undegraded":"2023-10-10T20:21:26.628452+0000","last_fullsized":"2023-10-10T20:21:26.628452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:16.104206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620580+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:36.614675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621728+0000","last_change":"2023-10-10T20:21:26.621728+0000","last_active":"2023-10-10T20:21:26.621728+0000","last_peered":"2023-10-10T20:21:26.621728+0000","last_clean":"2023-10-10T20:21:26.621728+0000","last_became_active":"2023-10-10T20:21:26.621493+0000","last_became_peered":"2023-10-10T20:21:26.621493+0000","last_unstale":"2023-10-10T20:21:26.621728+0000","last_undegraded":"2023-10-10T20:21:26.621728+0000","last_fullsized":"2023-10-10T20:21:26.621728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:50.660723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619498+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.623339+0000","last_change":"2023-10-10T20:21:26.623339+0000","last_active":"2023-10-10T20:21:26.623339+0000","last_peered":"2023-10-10T20:21:26.623339+0000","last_clean":"2023-10-10T20:21:26.623339+0000","last_became_active":"2023-10-10T20:21:26.622789+0000","last_became_peered":"2023-10-10T20:21:26.622789+0000","last_unstale":"2023-10-10T20:21:26.623339+0000","last_undegraded":"2023-10-10T20:21:26.623339+0000","last_fullsized":"2023-10-10T20:21:26.623339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:33.771902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619222+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625874+0000","last_change":"2023-10-10T20:21:26.625874+0000","last_active":"2023-10-10T20:21:26.625874+0000","last_peered":"2023-10-10T20:21:26.625874+0000","last_clean":"2023-10-10T20:21:26.625874+0000","last_became_active":"2023-10-10T20:21:26.625638+0000","last_became_peered":"2023-10-10T20:21:26.625638+0000","last_unstale":"2023-10-10T20:21:26.625874+0000","last_undegraded":"2023-10-10T20:21:26.625874+0000","last_fullsized":"2023-10-10T20:21:26.625874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:35.606440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-10T20:21:25.614971+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:21:25.614971+0000","last_peered":"2023-10-10T20:21:25.614971+0000","last_clean":"2023-10-10T20:21:25.614971+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:21:25.614971+0000","last_undegraded":"2023-10-10T20:21:25.614971+0000","last_fullsized":"2023-10-10T20:21:25.614971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47199999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.498}]},{"osd":1,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65500000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.64800000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.64100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38700000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.443,"5min":1.647,"15min":1.647},"last":0.372}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.44600000000000001,"5min":1.524,"15min":1.524},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.432,"15min":0.44700000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.434,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.46200000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.5,"5min":1.673,"15min":1.673},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69399999999999995,"5min":1.633,"15min":1.633},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":1.506,"15min":1.506},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66900000000000004,"5min":1.696,"15min":1.696},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77500000000000002,"5min":1.585,"15min":1.585},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78500000000000003,"5min":1.478,"15min":1.478},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":1.722,"15min":1.722},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":893,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.46000000000000002,"15min":0.48499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40999999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.50600000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.49099999999999999,"15min":0.51300000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53800000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55500000000000005,"5min":1.256,"15min":1.256},"last":0.25700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52400000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.204,"15min":1.204},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.97199999999999998,"15min":1.048},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71899999999999997,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":1.347,"15min":1.347},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":1.173,"15min":1.173},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51100000000000001,"5min":1.389,"15min":1.389},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57299999999999995,"5min":1.256,"15min":1.256},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60099999999999998,"5min":1.135,"15min":1.135},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63400000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55700000000000005,"5min":1.111,"15min":1.111},"last":0.373},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.61699999999999999,"5min":1.413,"15min":1.413},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.083,"15min":1.083},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81699999999999995,"5min":1.363,"15min":1.363},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.224,"15min":1.224},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.29299999999999998,"15min":0.29399999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.47699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46200000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.249},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36199999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63200000000000001,"5min":0.751,"15min":0.751},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46700000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71999999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81599999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.626}]},{"osd":1,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.754},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64200000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:21:32.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-10-10T20:21:32.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622808+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:03:00.321727+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-10-10T20:21:32.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623111+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:03:53.930618+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-10-10T20:21:32.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623423+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-10-10T20:21:32.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.626337+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-10-10T20:21:32.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624256+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:28:53.522969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-10-10T20:21:32.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624593+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-10-10T20:21:32.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621925+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-10-10T20:21:32.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622817+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:13:05.985305+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-10-10T20:21:32.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621295+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-10-10T20:21:32.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619475+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:51:16.491192+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-10-10T20:21:32.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619830+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:47:54.893055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-10-10T20:21:32.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621461+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-10-10T20:21:32.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621861+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:21:32.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-10-10T20:21:32.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619679+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-10-10T20:21:32.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619609+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-10-10T20:21:32.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620549+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-10-10T20:21:32.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621552+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-10-10T20:21:32.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624645+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-10-10T20:21:32.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623075+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:31:30.479823+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-10-10T20:21:32.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619523+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-10-10T20:21:32.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.627041+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-10-10T20:21:32.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621230+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-10-10T20:21:32.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624226+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-10-10T20:21:32.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623129+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:21:32.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-10-10T20:21:32.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623916+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:36:50.103031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-10-10T20:21:32.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.625076+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-10-10T20:21:32.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624545+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-10-10T20:21:32.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621937+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:31:00.095689+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-10-10T20:21:32.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622502+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-10-10T20:21:32.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621704+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-10-10T20:21:32.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622435+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:07:33.165713+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-10T20:21:32.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-10-10T20:21:32.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620489+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-10T20:21:32.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624622+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-10-10T20:21:32.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620202+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:11:32.400694+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-10T20:21:32.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619236+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-10T20:21:32.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.625398+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-10-10T20:21:32.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619863+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-10-10T20:21:32.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620892+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-10T20:21:32.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623403+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-10T20:21:32.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-10-10T20:21:32.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622001+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-10T20:21:32.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620462+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-10-10T20:21:32.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.626651+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-10-10T20:21:32.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620902+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:59:25.135129+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-10-10T20:21:32.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.623834+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-10T20:21:32.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624659+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-10-10T20:21:32.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621312+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-10T20:21:32.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619453+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:21:04.202003+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-10T20:21:32.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-10-10T20:21:32.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.622569+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-10-10T20:21:32.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620803+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-10T20:21:32.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.625945+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-10-10T20:21:32.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620552+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-10-10T20:21:32.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624757+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-10T20:21:32.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.625561+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-10T20:21:32.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620109+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-10T20:21:32.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619540+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-10T20:21:32.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621704+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-10T20:21:32.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.621024+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-10T20:21:32.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-10-10T20:21:32.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619144+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T04:02:27.652495+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-10T20:21:32.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619667+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-10T20:21:32.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-10-10T20:21:32.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620217+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-10-10T20:21:32.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.624370+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-10T20:21:32.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-10-10T20:21:32.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.620580+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T22:31:36.614675+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-10T20:21:32.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-10-10T20:21:32.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619498+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-10-10T20:21:32.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-10T20:21:26.612495+0000', 'last_change': '2023-10-10T20:21:26.619222+0000', 'last_active': '2023-10-10T20:21:26.612495+0000', 'last_peered': '2023-10-10T20:21:26.612495+0000', 'last_clean': '2023-10-10T20:21:26.612495+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-10T20:21:26.612495+0000', 'last_undegraded': '2023-10-10T20:21:26.612495+0000', 'last_fullsized': '2023-10-10T20:21:26.612495+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'last_clean_scrub_stamp': '2023-10-10T20:21:26.612495+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-10T20:21:32.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-10T20:21:32.607 INFO:tasks.ceph.mds.c.smithi148.stderr:2023-10-10T20:21:32.607+0000 7fbcf7a93700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:21:32.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.611+0000 7ff306d82700 1 Processor -- start 2023-10-10T20:21:32.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.611+0000 7ff306d82700 1 -- start start 2023-10-10T20:21:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff306d82700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff300158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff306d82700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015cbb0 con 0x7ff300157870 2023-10-10T20:21:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff306d82700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015bfa0 con 0x7ff30015a620 2023-10-10T20:21:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff306d82700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30015bdb0 con 0x7ff300157c40 2023-10-10T20:21:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff304b1e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff300157870 0x7ff30015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45684/0 (socket says 172.21.15.148:45684) 2023-10-10T20:21:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff304b1e700 1 -- 172.21.15.148:0/947576613 learned_addr learned my addr 172.21.15.148:0/947576613 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff30531f700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff300158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995664572 0 0) 0x7ff30015cbb0 con 0x7ff300157870 2023-10-10T20:21:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e8003570 con 0x7ff300157870 2023-10-10T20:21:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880143395 0 0) 0x7ff30015bfa0 con 0x7ff30015a620 2023-10-10T20:21:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff30015cbb0 con 0x7ff30015a620 2023-10-10T20:21:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1526287318 0 0) 0x7ff2e8003570 con 0x7ff300157870 2023-10-10T20:21:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 msgr2=0x7ff300158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff300158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 >> v1:172.21.15.161:6789/0 conn(0x7ff30015a620 legacy=0x7ff30015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30015c280 con 0x7ff300157870 2023-10-10T20:21:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7ff2fc001470 con 0x7ff300157870 2023-10-10T20:21:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 >> v1:172.21.15.148:6789/0 conn(0x7ff300157870 legacy=0x7ff30015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff2e8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2e80041d0 0x7ff2e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e8004d50 0x7ff2e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e800bbf0 con 0x7ff2e8004d50 2023-10-10T20:21:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e800bd30 con 0x7ff2e80041d0 2023-10-10T20:21:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e800bef0 con 0x7ff300157c40 2023-10-10T20:21:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff30531f700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e8004d50 0x7ff2e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff304b1e700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff2e8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.612+0000 7ff2f7fff700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2e80041d0 0x7ff2e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 -- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 msgr2=0x7ff2e8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff300157c40 0x7ff2e8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 -- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2e80041d0 msgr2=0x7ff2e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2e80041d0 0x7ff2e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 -- 172.21.15.148:0/947576613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8003570 con 0x7ff2e8004d50 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff30531f700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e8004d50 0x7ff2e80051c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3001580c0 tx=0x7ff2f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3f71057732d01 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800ea90 con 0x7ff2e8004d50 2023-10-10T20:21:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f800f040 con 0x7ff2e8004d50 2023-10-10T20:21:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.613+0000 7ff2f77fe700 1 -- 172.21.15.148:0/947576613 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800ee40 con 0x7ff2e8004d50 2023-10-10T20:21:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 -- 172.21.15.148:0/947576613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e8004d50 msgr2=0x7ff2e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 --2- 172.21.15.148:0/947576613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e8004d50 0x7ff2e80051c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3001580c0 tx=0x7ff2f800b040 comp rx=0 tx=0).stop 2023-10-10T20:21:32.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff30531f700 1 -- 172.21.15.148:0/947576613 reap_dead start 2023-10-10T20:21:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 -- 172.21.15.148:0/947576613 shutdown_connections 2023-10-10T20:21:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 -- 172.21.15.148:0/947576613 >> 172.21.15.148:0/947576613 conn(0x7ff300068ce0 msgr2=0x7ff3000690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 -- 172.21.15.148:0/947576613 shutdown_connections 2023-10-10T20:21:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.614+0000 7ff306d82700 1 -- 172.21.15.148:0/947576613 wait complete. 2023-10-10T20:21:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.615+0000 7ff306d82700 1 Processor -- start 2023-10-10T20:21:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.615+0000 7ff306d82700 1 -- start start 2023-10-10T20:21:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff30013f5d0 0x7ff30015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff30013f9a0 0x7ff30015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000b0c40 con 0x7ff30014dab0 2023-10-10T20:21:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300150930 con 0x7ff30013f9a0 2023-10-10T20:21:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff300150c10 con 0x7ff30013f5d0 2023-10-10T20:21:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42618/0 (socket says 172.21.15.148:42618) 2023-10-10T20:21:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 -- 172.21.15.148:0/3561998420 learned_addr learned my addr 172.21.15.148:0/3561998420 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff304b1e700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff30013f5d0 0x7ff30015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff2f7fff700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff30013f9a0 0x7ff30015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 -- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff30013f5d0 msgr2=0x7ff30015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff30013f5d0 0x7ff30015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 -- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff30013f9a0 msgr2=0x7ff30015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff30013f9a0 0x7ff30015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 -- 172.21.15.148:0/3561998420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000b08d0 con 0x7ff30014dab0 2023-10-10T20:21:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff30531f700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800afd0 tx=0x7ff2f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f801d630 con 0x7ff30014dab0 2023-10-10T20:21:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f801d7b0 con 0x7ff30014dab0 2023-10-10T20:21:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30014f530 con 0x7ff30014dab0 2023-10-10T20:21:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f801dbb0 con 0x7ff30014dab0 2023-10-10T20:21:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.616+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30014f9d0 con 0x7ff30014dab0 2023-10-10T20:21:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.617+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3000b0730 con 0x7ff30014dab0 2023-10-10T20:21:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.618+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2f8003040 con 0x7ff30014dab0 2023-10-10T20:21:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.618+0000 7ff2f4ff9700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:32.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.618+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2f8085990 con 0x7ff30014dab0 2023-10-10T20:21:32.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.618+0000 7ff304b1e700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:32.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.618+0000 7ff304b1e700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 0x7ff2ec0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc001bd0 tx=0x7ff2fc001d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:32.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.622+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3000b0730 con 0x7ff30014dab0 2023-10-10T20:21:32.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.880+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff3000b0730 con 0x7ff30014dab0 2023-10-10T20:21:32.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.880+0000 7ff2f4ff9700 1 -- 172.21.15.148:0/3561998420 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1577 (secure 0 0 0) 0x7ff3000b0730 con 0x7ff30014dab0 2023-10-10T20:21:32.882 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:32.882 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":147,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1696969151,"num_in_osds":8,"osd_in_since":1696969148,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":3761950,"bytes_avail":1596238050,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5727}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-10T20:20:15.333667+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-10T20:21:32.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 msgr2=0x7ff2ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 0x7ff2ec0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc001bd0 tx=0x7ff2fc001d40 comp rx=0 tx=0).stop 2023-10-10T20:21:32.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 msgr2=0x7ff30015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:32.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800afd0 tx=0x7ff2f801a000 comp rx=0 tx=0).stop 2023-10-10T20:21:32.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 shutdown_connections 2023-10-10T20:21:32.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2ec066f30 0x7ff2ec0693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff30014dab0 0x7ff30015dac0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff30013f5d0 0x7ff30015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 --2- 172.21.15.148:0/3561998420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff30013f9a0 0x7ff30015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:32.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 >> 172.21.15.148:0/3561998420 conn(0x7ff300068ce0 msgr2=0x7ff3000b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:32.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 shutdown_connections 2023-10-10T20:21:32.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:32.882+0000 7ff306d82700 1 -- 172.21.15.148:0/3561998420 wait complete. 2023-10-10T20:21:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-10T20:21:32.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:21:33.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.009+0000 7fa0a6cf1700 1 Processor -- start 2023-10-10T20:21:33.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.009+0000 7fa0a6cf1700 1 -- start start 2023-10-10T20:21:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a6cf1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 0x7fa0a00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a6cf1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0a01438f0 0x7fa0a006a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a6cf1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a00d0870 con 0x7fa0a00ccf00 2023-10-10T20:21:33.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a6cf1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a00cdfb0 con 0x7fa0a01438f0 2023-10-10T20:21:33.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a6cf1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00cfc60 con 0x7fa0a0143520 2023-10-10T20:21:33.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a54ee700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0a01438f0 0x7fa0a006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a5cef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 0x7fa0a00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a64f0700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa0a00ccf00 0x7fa0a015da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48908/0 (socket says 172.21.15.148:48908) 2023-10-10T20:21:33.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.010+0000 7fa0a64f0700 1 -- 172.21.15.148:0/1467782959 learned_addr learned my addr 172.21.15.148:0/1467782959 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a54ee700 1 -- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 msgr2=0x7fa0a00cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a54ee700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 0x7fa0a00cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a54ee700 1 -- 172.21.15.148:0/1467782959 >> v1:172.21.15.161:6789/0 conn(0x7fa0a00ccf00 legacy=0x7fa0a015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a54ee700 1 -- 172.21.15.148:0/1467782959 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00cff40 con 0x7fa0a01438f0 2023-10-10T20:21:33.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a5cef700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 0x7fa0a00cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:21:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a4ced700 1 -- 172.21.15.148:0/1467782959 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077274124 0 0) 0x7fa0a00d0870 con 0x7fa0a00ccf00 2023-10-10T20:21:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a54ee700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a01438f0 0x7fa0a006a440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0900098e0 tx=0x7fa0900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5bdfbdf60d146c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a4ced700 1 -- 172.21.15.148:0/1467782959 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000b640 con 0x7fa0a01438f0 2023-10-10T20:21:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a4ced700 1 -- 172.21.15.148:0/1467782959 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000b7a0 con 0x7fa0a01438f0 2023-10-10T20:21:33.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.011+0000 7fa0a4ced700 1 -- 172.21.15.148:0/1467782959 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000bb40 con 0x7fa0a01438f0 2023-10-10T20:21:33.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a01438f0 msgr2=0x7fa0a006a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a01438f0 0x7fa0a006a440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0900098e0 tx=0x7fa0900091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/1467782959 shutdown_connections 2023-10-10T20:21:33.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a0143520 0x7fa0a00cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/1467782959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a01438f0 0x7fa0a006a440 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/1467782959 >> 172.21.15.148:0/1467782959 conn(0x7fa0a0069750 msgr2=0x7fa0a00b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:33.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/1467782959 shutdown_connections 2023-10-10T20:21:33.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.012+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/1467782959 wait complete. 2023-10-10T20:21:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 Processor -- start 2023-10-10T20:21:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 -- start start 2023-10-10T20:21:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a00ccf00 0x7fa0a013a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a0143520 0x7fa0a00b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a013c8b0 con 0x7fa0a01438f0 2023-10-10T20:21:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a013bca0 con 0x7fa0a0143520 2023-10-10T20:21:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.013+0000 7fa0a6cf1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a013bf80 con 0x7fa0a00ccf00 2023-10-10T20:21:33.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42628/0 (socket says 172.21.15.148:42628) 2023-10-10T20:21:33.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 -- 172.21.15.148:0/2989692614 learned_addr learned my addr 172.21.15.148:0/2989692614 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:33.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a54ee700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a0143520 0x7fa0a00b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a5cef700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a00ccf00 0x7fa0a013a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 -- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a00ccf00 msgr2=0x7fa0a013a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a00ccf00 0x7fa0a013a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 -- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a0143520 msgr2=0x7fa0a00b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a0143520 0x7fa0a00b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 -- 172.21.15.148:0/2989692614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c009a20 con 0x7fa0a01438f0 2023-10-10T20:21:33.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a64f0700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c7e0 tx=0x7fa098007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:33.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098005c00 con 0x7fa0a01438f0 2023-10-10T20:21:33.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098005d80 con 0x7fa0a01438f0 2023-10-10T20:21:33.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa090009420 con 0x7fa0a01438f0 2023-10-10T20:21:33.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.014+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098007320 con 0x7fa0a01438f0 2023-10-10T20:21:33.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.015+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a013c540 con 0x7fa0a01438f0 2023-10-10T20:21:33.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.015+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0089030 con 0x7fa0a01438f0 2023-10-10T20:21:33.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.016+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa09801b3b0 con 0x7fa0a01438f0 2023-10-10T20:21:33.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.016+0000 7fa095ffb700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 0x7fa0840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:33.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.016+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa098083b50 con 0x7fa0a01438f0 2023-10-10T20:21:33.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.016+0000 7fa0a5cef700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 0x7fa0840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:33.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.017+0000 7fa0a5cef700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 0x7fa0840693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0099f0 tx=0x7fa09c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:33.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.020+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa098050d20 con 0x7fa0a01438f0 2023-10-10T20:21:33.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.198+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0a00d0220 con 0x7fa084066f30 2023-10-10T20:21:33.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.206+0000 7fa095ffb700 1 -- 172.21.15.148:0/2989692614 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282424 (secure 0 0 0) 0x7fa0a00d0220 con 0x7fa084066f30 2023-10-10T20:21:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:33.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 msgr2=0x7fa0840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 0x7fa0840693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0099f0 tx=0x7fa09c00b040 comp rx=0 tx=0).stop 2023-10-10T20:21:33.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 msgr2=0x7fa0a00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:33.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c7e0 tx=0x7fa098007d90 comp rx=0 tx=0).stop 2023-10-10T20:21:33.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 shutdown_connections 2023-10-10T20:21:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa084066f30 0x7fa0840693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a01438f0 0x7fa0a00b3ee0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0a00ccf00 0x7fa0a013a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 --2- 172.21.15.148:0/2989692614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0a0143520 0x7fa0a00b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 >> 172.21.15.148:0/2989692614 conn(0x7fa0a0069750 msgr2=0x7fa0a00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 shutdown_connections 2023-10-10T20:21:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:33.209+0000 7fa0a6cf1700 1 -- 172.21.15.148:0/2989692614 wait complete. 2023-10-10T20:21:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:21:33.223 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-10-10T20:21:31.349093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558826,"statfs":{"total":1600000000,"available":1596238050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734008"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622808+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:00.321727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623111+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:53.930618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623423+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626337+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624256+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:53.522969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624593+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621925+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622817+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:05.985305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621295+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619475+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:16.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619830+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:54.893055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621461+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621861+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619679+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619609+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620549+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624645+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623075+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:30.479823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619523+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.627041+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621230+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624226+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623129+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623916+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:50.103031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625076+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624545+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621937+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:00.095689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622502+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622435+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:33.165713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620489+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624622+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.620953+0000","last_change":"2023-10-10T20:21:26.620953+0000","last_active":"2023-10-10T20:21:26.620953+0000","last_peered":"2023-10-10T20:21:26.620953+0000","last_clean":"2023-10-10T20:21:26.620953+0000","last_became_active":"2023-10-10T20:21:26.620376+0000","last_became_peered":"2023-10-10T20:21:26.620376+0000","last_unstale":"2023-10-10T20:21:26.620953+0000","last_undegraded":"2023-10-10T20:21:26.620953+0000","last_fullsized":"2023-10-10T20:21:26.620953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:40.544270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620202+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:32.400694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624607+0000","last_change":"2023-10-10T20:21:26.624607+0000","last_active":"2023-10-10T20:21:26.624607+0000","last_peered":"2023-10-10T20:21:26.624607+0000","last_clean":"2023-10-10T20:21:26.624607+0000","last_became_active":"2023-10-10T20:21:26.624069+0000","last_became_peered":"2023-10-10T20:21:26.624069+0000","last_unstale":"2023-10-10T20:21:26.624607+0000","last_undegraded":"2023-10-10T20:21:26.624607+0000","last_fullsized":"2023-10-10T20:21:26.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:31.294895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619236+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628774+0000","last_change":"2023-10-10T20:21:26.628774+0000","last_active":"2023-10-10T20:21:26.628774+0000","last_peered":"2023-10-10T20:21:26.628774+0000","last_clean":"2023-10-10T20:21:26.628774+0000","last_became_active":"2023-10-10T20:21:26.628427+0000","last_became_peered":"2023-10-10T20:21:26.628427+0000","last_unstale":"2023-10-10T20:21:26.628774+0000","last_undegraded":"2023-10-10T20:21:26.628774+0000","last_fullsized":"2023-10-10T20:21:26.628774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:11.265574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625398+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631546+0000","last_change":"2023-10-10T20:21:26.631546+0000","last_active":"2023-10-10T20:21:26.631546+0000","last_peered":"2023-10-10T20:21:26.631546+0000","last_clean":"2023-10-10T20:21:26.631546+0000","last_became_active":"2023-10-10T20:21:26.631032+0000","last_became_peered":"2023-10-10T20:21:26.631032+0000","last_unstale":"2023-10-10T20:21:26.631546+0000","last_undegraded":"2023-10-10T20:21:26.631546+0000","last_fullsized":"2023-10-10T20:21:26.631546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:36.676962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619863+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626156+0000","last_change":"2023-10-10T20:21:26.626156+0000","last_active":"2023-10-10T20:21:26.626156+0000","last_peered":"2023-10-10T20:21:26.626156+0000","last_clean":"2023-10-10T20:21:26.626156+0000","last_became_active":"2023-10-10T20:21:26.625622+0000","last_became_peered":"2023-10-10T20:21:26.625622+0000","last_unstale":"2023-10-10T20:21:26.626156+0000","last_undegraded":"2023-10-10T20:21:26.626156+0000","last_fullsized":"2023-10-10T20:21:26.626156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620892+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626984+0000","last_change":"2023-10-10T20:21:26.626984+0000","last_active":"2023-10-10T20:21:26.626984+0000","last_peered":"2023-10-10T20:21:26.626984+0000","last_clean":"2023-10-10T20:21:26.626984+0000","last_became_active":"2023-10-10T20:21:26.626672+0000","last_became_peered":"2023-10-10T20:21:26.626672+0000","last_unstale":"2023-10-10T20:21:26.626984+0000","last_undegraded":"2023-10-10T20:21:26.626984+0000","last_fullsized":"2023-10-10T20:21:26.626984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623403+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630463+0000","last_change":"2023-10-10T20:21:26.630463+0000","last_active":"2023-10-10T20:21:26.630463+0000","last_peered":"2023-10-10T20:21:26.630463+0000","last_clean":"2023-10-10T20:21:26.630463+0000","last_became_active":"2023-10-10T20:21:26.630236+0000","last_became_peered":"2023-10-10T20:21:26.630236+0000","last_unstale":"2023-10-10T20:21:26.630463+0000","last_undegraded":"2023-10-10T20:21:26.630463+0000","last_fullsized":"2023-10-10T20:21:26.630463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622001+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625829+0000","last_change":"2023-10-10T20:21:26.625829+0000","last_active":"2023-10-10T20:21:26.625829+0000","last_peered":"2023-10-10T20:21:26.625829+0000","last_clean":"2023-10-10T20:21:26.625829+0000","last_became_active":"2023-10-10T20:21:26.624923+0000","last_became_peered":"2023-10-10T20:21:26.624923+0000","last_unstale":"2023-10-10T20:21:26.625829+0000","last_undegraded":"2023-10-10T20:21:26.625829+0000","last_fullsized":"2023-10-10T20:21:26.625829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620462+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:37.685197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631482+0000","last_change":"2023-10-10T20:21:26.631482+0000","last_active":"2023-10-10T20:21:26.631482+0000","last_peered":"2023-10-10T20:21:26.631482+0000","last_clean":"2023-10-10T20:21:26.631482+0000","last_became_active":"2023-10-10T20:21:26.630913+0000","last_became_peered":"2023-10-10T20:21:26.630913+0000","last_unstale":"2023-10-10T20:21:26.631482+0000","last_undegraded":"2023-10-10T20:21:26.631482+0000","last_fullsized":"2023-10-10T20:21:26.631482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.626651+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:32.303131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626142+0000","last_change":"2023-10-10T20:21:26.626142+0000","last_active":"2023-10-10T20:21:26.626142+0000","last_peered":"2023-10-10T20:21:26.626142+0000","last_clean":"2023-10-10T20:21:26.626142+0000","last_became_active":"2023-10-10T20:21:26.625123+0000","last_became_peered":"2023-10-10T20:21:26.625123+0000","last_unstale":"2023-10-10T20:21:26.626142+0000","last_undegraded":"2023-10-10T20:21:26.626142+0000","last_fullsized":"2023-10-10T20:21:26.626142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:07.454724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620902+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:25.135129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628970+0000","last_change":"2023-10-10T20:21:26.628970+0000","last_active":"2023-10-10T20:21:26.628970+0000","last_peered":"2023-10-10T20:21:26.628970+0000","last_clean":"2023-10-10T20:21:26.628970+0000","last_became_active":"2023-10-10T20:21:26.628746+0000","last_became_peered":"2023-10-10T20:21:26.628746+0000","last_unstale":"2023-10-10T20:21:26.628970+0000","last_undegraded":"2023-10-10T20:21:26.628970+0000","last_fullsized":"2023-10-10T20:21:26.628970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.623834+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621850+0000","last_change":"2023-10-10T20:21:26.621850+0000","last_active":"2023-10-10T20:21:26.621850+0000","last_peered":"2023-10-10T20:21:26.621850+0000","last_clean":"2023-10-10T20:21:26.621850+0000","last_became_active":"2023-10-10T20:21:26.621603+0000","last_became_peered":"2023-10-10T20:21:26.621603+0000","last_unstale":"2023-10-10T20:21:26.621850+0000","last_undegraded":"2023-10-10T20:21:26.621850+0000","last_fullsized":"2023-10-10T20:21:26.621850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:39.724039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624659+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626666+0000","last_change":"2023-10-10T20:21:26.626666+0000","last_active":"2023-10-10T20:21:26.626666+0000","last_peered":"2023-10-10T20:21:26.626666+0000","last_clean":"2023-10-10T20:21:26.626666+0000","last_became_active":"2023-10-10T20:21:26.626298+0000","last_became_peered":"2023-10-10T20:21:26.626298+0000","last_unstale":"2023-10-10T20:21:26.626666+0000","last_undegraded":"2023-10-10T20:21:26.626666+0000","last_fullsized":"2023-10-10T20:21:26.626666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:35.386841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621312+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:52.899809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630545+0000","last_change":"2023-10-10T20:21:26.630545+0000","last_active":"2023-10-10T20:21:26.630545+0000","last_peered":"2023-10-10T20:21:26.630545+0000","last_clean":"2023-10-10T20:21:26.630545+0000","last_became_active":"2023-10-10T20:21:26.630288+0000","last_became_peered":"2023-10-10T20:21:26.630288+0000","last_unstale":"2023-10-10T20:21:26.630545+0000","last_undegraded":"2023-10-10T20:21:26.630545+0000","last_fullsized":"2023-10-10T20:21:26.630545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:53.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619453+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:04.202003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627411+0000","last_change":"2023-10-10T20:21:26.627411+0000","last_active":"2023-10-10T20:21:26.627411+0000","last_peered":"2023-10-10T20:21:26.627411+0000","last_clean":"2023-10-10T20:21:26.627411+0000","last_became_active":"2023-10-10T20:21:26.627165+0000","last_became_peered":"2023-10-10T20:21:26.627165+0000","last_unstale":"2023-10-10T20:21:26.627411+0000","last_undegraded":"2023-10-10T20:21:26.627411+0000","last_fullsized":"2023-10-10T20:21:26.627411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.622569+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:12.273809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.630197+0000","last_change":"2023-10-10T20:21:26.630197+0000","last_active":"2023-10-10T20:21:26.630197+0000","last_peered":"2023-10-10T20:21:26.630197+0000","last_clean":"2023-10-10T20:21:26.630197+0000","last_became_active":"2023-10-10T20:21:26.629955+0000","last_became_peered":"2023-10-10T20:21:26.629955+0000","last_unstale":"2023-10-10T20:21:26.630197+0000","last_undegraded":"2023-10-10T20:21:26.630197+0000","last_fullsized":"2023-10-10T20:21:26.630197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620803+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:05.720611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624742+0000","last_change":"2023-10-10T20:21:26.624742+0000","last_active":"2023-10-10T20:21:26.624742+0000","last_peered":"2023-10-10T20:21:26.624742+0000","last_clean":"2023-10-10T20:21:26.624742+0000","last_became_active":"2023-10-10T20:21:26.624396+0000","last_became_peered":"2023-10-10T20:21:26.624396+0000","last_unstale":"2023-10-10T20:21:26.624742+0000","last_undegraded":"2023-10-10T20:21:26.624742+0000","last_fullsized":"2023-10-10T20:21:26.624742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:43.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625945+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628656+0000","last_change":"2023-10-10T20:21:26.628656+0000","last_active":"2023-10-10T20:21:26.628656+0000","last_peered":"2023-10-10T20:21:26.628656+0000","last_clean":"2023-10-10T20:21:26.628656+0000","last_became_active":"2023-10-10T20:21:26.627500+0000","last_became_peered":"2023-10-10T20:21:26.627500+0000","last_unstale":"2023-10-10T20:21:26.628656+0000","last_undegraded":"2023-10-10T20:21:26.628656+0000","last_fullsized":"2023-10-10T20:21:26.628656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:32.116793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620552+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.631690+0000","last_change":"2023-10-10T20:21:26.631690+0000","last_active":"2023-10-10T20:21:26.631690+0000","last_peered":"2023-10-10T20:21:26.631690+0000","last_clean":"2023-10-10T20:21:26.631690+0000","last_became_active":"2023-10-10T20:21:26.631324+0000","last_became_peered":"2023-10-10T20:21:26.631324+0000","last_unstale":"2023-10-10T20:21:26.631690+0000","last_undegraded":"2023-10-10T20:21:26.631690+0000","last_fullsized":"2023-10-10T20:21:26.631690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:51.891574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624757+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.627796+0000","last_change":"2023-10-10T20:21:26.627796+0000","last_active":"2023-10-10T20:21:26.627796+0000","last_peered":"2023-10-10T20:21:26.627796+0000","last_clean":"2023-10-10T20:21:26.627796+0000","last_became_active":"2023-10-10T20:21:26.627566+0000","last_became_peered":"2023-10-10T20:21:26.627566+0000","last_unstale":"2023-10-10T20:21:26.627796+0000","last_undegraded":"2023-10-10T20:21:26.627796+0000","last_fullsized":"2023-10-10T20:21:26.627796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:25.600786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.625561+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626981+0000","last_change":"2023-10-10T20:21:26.626981+0000","last_active":"2023-10-10T20:21:26.626981+0000","last_peered":"2023-10-10T20:21:26.626981+0000","last_clean":"2023-10-10T20:21:26.626981+0000","last_became_active":"2023-10-10T20:21:26.626635+0000","last_became_peered":"2023-10-10T20:21:26.626635+0000","last_unstale":"2023-10-10T20:21:26.626981+0000","last_undegraded":"2023-10-10T20:21:26.626981+0000","last_fullsized":"2023-10-10T20:21:26.626981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:32.511186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620109+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:44.836561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624041+0000","last_change":"2023-10-10T20:21:26.624041+0000","last_active":"2023-10-10T20:21:26.624041+0000","last_peered":"2023-10-10T20:21:26.624041+0000","last_clean":"2023-10-10T20:21:26.624041+0000","last_became_active":"2023-10-10T20:21:26.623809+0000","last_became_peered":"2023-10-10T20:21:26.623809+0000","last_unstale":"2023-10-10T20:21:26.624041+0000","last_undegraded":"2023-10-10T20:21:26.624041+0000","last_fullsized":"2023-10-10T20:21:26.624041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:24.387989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619540+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.629529+0000","last_change":"2023-10-10T20:21:26.629529+0000","last_active":"2023-10-10T20:21:26.629529+0000","last_peered":"2023-10-10T20:21:26.629529+0000","last_clean":"2023-10-10T20:21:26.629529+0000","last_became_active":"2023-10-10T20:21:26.629280+0000","last_became_peered":"2023-10-10T20:21:26.629280+0000","last_unstale":"2023-10-10T20:21:26.629529+0000","last_undegraded":"2023-10-10T20:21:26.629529+0000","last_fullsized":"2023-10-10T20:21:26.629529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621704+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624689+0000","last_change":"2023-10-10T20:21:26.624689+0000","last_active":"2023-10-10T20:21:26.624689+0000","last_peered":"2023-10-10T20:21:26.624689+0000","last_clean":"2023-10-10T20:21:26.624689+0000","last_became_active":"2023-10-10T20:21:26.624473+0000","last_became_peered":"2023-10-10T20:21:26.624473+0000","last_unstale":"2023-10-10T20:21:26.624689+0000","last_undegraded":"2023-10-10T20:21:26.624689+0000","last_fullsized":"2023-10-10T20:21:26.624689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:30.722252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.621024+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626857+0000","last_change":"2023-10-10T20:21:26.626857+0000","last_active":"2023-10-10T20:21:26.626857+0000","last_peered":"2023-10-10T20:21:26.626857+0000","last_clean":"2023-10-10T20:21:26.626857+0000","last_became_active":"2023-10-10T20:21:26.626370+0000","last_became_peered":"2023-10-10T20:21:26.626370+0000","last_unstale":"2023-10-10T20:21:26.626857+0000","last_undegraded":"2023-10-10T20:21:26.626857+0000","last_fullsized":"2023-10-10T20:21:26.626857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619144+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:27.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628016+0000","last_change":"2023-10-10T20:21:26.628016+0000","last_active":"2023-10-10T20:21:26.628016+0000","last_peered":"2023-10-10T20:21:26.628016+0000","last_clean":"2023-10-10T20:21:26.628016+0000","last_became_active":"2023-10-10T20:21:26.627749+0000","last_became_peered":"2023-10-10T20:21:26.627749+0000","last_unstale":"2023-10-10T20:21:26.628016+0000","last_undegraded":"2023-10-10T20:21:26.628016+0000","last_fullsized":"2023-10-10T20:21:26.628016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:31.392459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619667+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.624500+0000","last_change":"2023-10-10T20:21:26.624500+0000","last_active":"2023-10-10T20:21:26.624500+0000","last_peered":"2023-10-10T20:21:26.624500+0000","last_clean":"2023-10-10T20:21:26.624500+0000","last_became_active":"2023-10-10T20:21:26.623771+0000","last_became_peered":"2023-10-10T20:21:26.623771+0000","last_unstale":"2023-10-10T20:21:26.624500+0000","last_undegraded":"2023-10-10T20:21:26.624500+0000","last_fullsized":"2023-10-10T20:21:26.624500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620217+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.626193+0000","last_change":"2023-10-10T20:21:26.626193+0000","last_active":"2023-10-10T20:21:26.626193+0000","last_peered":"2023-10-10T20:21:26.626193+0000","last_clean":"2023-10-10T20:21:26.626193+0000","last_became_active":"2023-10-10T20:21:26.625852+0000","last_became_peered":"2023-10-10T20:21:26.625852+0000","last_unstale":"2023-10-10T20:21:26.626193+0000","last_undegraded":"2023-10-10T20:21:26.626193+0000","last_fullsized":"2023-10-10T20:21:26.626193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:29.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.624370+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.628452+0000","last_change":"2023-10-10T20:21:26.628452+0000","last_active":"2023-10-10T20:21:26.628452+0000","last_peered":"2023-10-10T20:21:26.628452+0000","last_clean":"2023-10-10T20:21:26.628452+0000","last_became_active":"2023-10-10T20:21:26.628193+0000","last_became_peered":"2023-10-10T20:21:26.628193+0000","last_unstale":"2023-10-10T20:21:26.628452+0000","last_undegraded":"2023-10-10T20:21:26.628452+0000","last_fullsized":"2023-10-10T20:21:26.628452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:16.104206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.620580+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:36.614675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.621728+0000","last_change":"2023-10-10T20:21:26.621728+0000","last_active":"2023-10-10T20:21:26.621728+0000","last_peered":"2023-10-10T20:21:26.621728+0000","last_clean":"2023-10-10T20:21:26.621728+0000","last_became_active":"2023-10-10T20:21:26.621493+0000","last_became_peered":"2023-10-10T20:21:26.621493+0000","last_unstale":"2023-10-10T20:21:26.621728+0000","last_undegraded":"2023-10-10T20:21:26.621728+0000","last_fullsized":"2023-10-10T20:21:26.621728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:50.660723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619498+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.623339+0000","last_change":"2023-10-10T20:21:26.623339+0000","last_active":"2023-10-10T20:21:26.623339+0000","last_peered":"2023-10-10T20:21:26.623339+0000","last_clean":"2023-10-10T20:21:26.623339+0000","last_became_active":"2023-10-10T20:21:26.622789+0000","last_became_peered":"2023-10-10T20:21:26.622789+0000","last_unstale":"2023-10-10T20:21:26.623339+0000","last_undegraded":"2023-10-10T20:21:26.623339+0000","last_fullsized":"2023-10-10T20:21:26.623339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:33.771902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:21:26.612495+0000","last_change":"2023-10-10T20:21:26.619222+0000","last_active":"2023-10-10T20:21:26.612495+0000","last_peered":"2023-10-10T20:21:26.612495+0000","last_clean":"2023-10-10T20:21:26.612495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:21:26.612495+0000","last_undegraded":"2023-10-10T20:21:26.612495+0000","last_fullsized":"2023-10-10T20:21:26.612495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-10T20:21:26.625874+0000","last_change":"2023-10-10T20:21:26.625874+0000","last_active":"2023-10-10T20:21:26.625874+0000","last_peered":"2023-10-10T20:21:26.625874+0000","last_clean":"2023-10-10T20:21:26.625874+0000","last_became_active":"2023-10-10T20:21:26.625638+0000","last_became_peered":"2023-10-10T20:21:26.625638+0000","last_unstale":"2023-10-10T20:21:26.625874+0000","last_undegraded":"2023-10-10T20:21:26.625874+0000","last_fullsized":"2023-10-10T20:21:26.625874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:35.606440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-10T20:21:25.614971+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:21:25.614971+0000","last_peered":"2023-10-10T20:21:25.614971+0000","last_clean":"2023-10-10T20:21:25.614971+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:21:25.614971+0000","last_undegraded":"2023-10-10T20:21:25.614971+0000","last_fullsized":"2023-10-10T20:21:25.614971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47199999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.498}]},{"osd":1,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65500000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.433}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.64800000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55300000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.64100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38700000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.443,"5min":1.647,"15min":1.647},"last":0.372}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.44600000000000001,"5min":1.524,"15min":1.524},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.432,"15min":0.44700000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.434,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.46200000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.5,"5min":1.673,"15min":1.673},"last":0.32200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69399999999999995,"5min":1.633,"15min":1.633},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":1.506,"15min":1.506},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66900000000000004,"5min":1.696,"15min":1.696},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77500000000000002,"5min":1.585,"15min":1.585},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78500000000000003,"5min":1.478,"15min":1.478},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":1.722,"15min":1.722},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":893,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.46000000000000002,"15min":0.48499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40999999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.50600000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.49099999999999999,"15min":0.51300000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53800000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55500000000000005,"5min":1.256,"15min":1.256},"last":0.25700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52400000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.204,"15min":1.204},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.97199999999999998,"15min":1.048},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71899999999999997,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":1.347,"15min":1.347},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":1.173,"15min":1.173},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51100000000000001,"5min":1.389,"15min":1.389},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57299999999999995,"5min":1.256,"15min":1.256},"last":0.34699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60099999999999998,"5min":1.135,"15min":1.135},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63400000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55700000000000005,"5min":1.111,"15min":1.111},"last":0.373},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.61699999999999999,"5min":1.413,"15min":1.413},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.496},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.083,"15min":1.083},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81699999999999995,"5min":1.363,"15min":1.363},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.224,"15min":1.224},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.29299999999999998,"15min":0.29399999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.47699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.317}]},{"osd":1,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46200000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.249},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36199999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63200000000000001,"5min":0.751,"15min":0.751},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199673968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.505},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46700000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71999999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195000,"statfs":{"total":200000000,"available":199680223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81599999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.626}]},{"osd":1,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.754},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64200000000000002}]}]}],"pool_statfs":[]}} 2023-10-10T20:21:36.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:21:36.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.348+0000 7fcf38c94700 1 Processor -- start 2023-10-10T20:21:36.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.348+0000 7fcf38c94700 1 -- start start 2023-10-10T20:21:36.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf38c94700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf3415a620 0x7fcf3415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf38c94700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf34158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf38c94700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3415cbb0 con 0x7fcf34157870 2023-10-10T20:21:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf38c94700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3415bdb0 con 0x7fcf3415a620 2023-10-10T20:21:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf38c94700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3415bfa0 con 0x7fcf3415d580 2023-10-10T20:21:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf3415a620 0x7fcf3415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf3415a620 0x7fcf3415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42634/0 (socket says 172.21.15.148:42634) 2023-10-10T20:21:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 -- 172.21.15.148:0/3849043325 learned_addr learned my addr 172.21.15.148:0/3849043325 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf32d9d700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf34158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 -- 172.21.15.148:0/3849043325 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 msgr2=0x7fcf34158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf34158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 -- 172.21.15.148:0/3849043325 >> v1:172.21.15.161:6789/0 conn(0x7fcf34157870 legacy=0x7fcf3415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 -- 172.21.15.148:0/3849043325 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3415c280 con 0x7fcf3415a620 2023-10-10T20:21:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf32d9d700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf34158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:36.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf2bfff700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf3415d040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0099b0 tx=0x7fcf1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85882f08c9714678 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf31d9b700 1 -- 172.21.15.148:0/3849043325 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00b860 con 0x7fcf3415a620 2023-10-10T20:21:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf31d9b700 1 -- 172.21.15.148:0/3849043325 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c00b9c0 con 0x7fcf3415a620 2023-10-10T20:21:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.349+0000 7fcf31d9b700 1 -- 172.21.15.148:0/3849043325 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00bd60 con 0x7fcf3415a620 2023-10-10T20:21:36.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 -- 172.21.15.148:0/3849043325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 msgr2=0x7fcf3415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf3415d040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0099b0 tx=0x7fcf1c009250 comp rx=0 tx=0).stop 2023-10-10T20:21:36.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 -- 172.21.15.148:0/3849043325 shutdown_connections 2023-10-10T20:21:36.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf34158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 --2- 172.21.15.148:0/3849043325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf3415d040 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 -- 172.21.15.148:0/3849043325 >> 172.21.15.148:0/3849043325 conn(0x7fcf34068ce0 msgr2=0x7fcf340690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:36.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 -- 172.21.15.148:0/3849043325 shutdown_connections 2023-10-10T20:21:36.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.350+0000 7fcf38c94700 1 -- 172.21.15.148:0/3849043325 wait complete. 2023-10-10T20:21:36.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.351+0000 7fcf38c94700 1 Processor -- start 2023-10-10T20:21:36.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.351+0000 7fcf38c94700 1 -- start start 2023-10-10T20:21:36.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 0x7fcf3414b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf340b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340a9890 con 0x7fcf3415a620 2023-10-10T20:21:36.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340aa130 con 0x7fcf34157870 2023-10-10T20:21:36.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340a8500 con 0x7fcf3415d580 2023-10-10T20:21:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42646/0 (socket says 172.21.15.148:42646) 2023-10-10T20:21:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 -- 172.21.15.148:0/1469790246 learned_addr learned my addr 172.21.15.148:0/1469790246 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf3259c700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 0x7fcf3414b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 -- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 msgr2=0x7fcf340b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf32d9d700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf340b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf340b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 -- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 msgr2=0x7fcf3414b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 0x7fcf3414b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 -- 172.21.15.148:0/1469790246 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0094f0 con 0x7fcf3415a620 2023-10-10T20:21:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf3259c700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 0x7fcf3414b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2bfff700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406b530 tx=0x7fcf1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c024440 con 0x7fcf3415a620 2023-10-10T20:21:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0245a0 con 0x7fcf3415a620 2023-10-10T20:21:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf340a9ae0 con 0x7fcf3415a620 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.352+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c024940 con 0x7fcf3415a620 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.353+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3414bc60 con 0x7fcf3415a620 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.353+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf340aa2c0 con 0x7fcf3415a620 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.354+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf1c01a420 con 0x7fcf3415a620 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.354+0000 7fcf2a7fc700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.354+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf1c0863f0 con 0x7fcf3415a620 2023-10-10T20:21:36.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.354+0000 7fcf3259c700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.355+0000 7fcf3259c700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 0x7fcf140693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf24001840 tx=0x7fcf24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.359+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf1c053660 con 0x7fcf3415a620 2023-10-10T20:21:36.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.537+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf3404c3c0 con 0x7fcf14066f30 2023-10-10T20:21:36.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.545+0000 7fcf2a7fc700 1 -- 172.21.15.148:0/1469790246 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285223 (secure 0 0 0) 0x7fcf3404c3c0 con 0x7fcf14066f30 2023-10-10T20:21:36.550 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:36.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 msgr2=0x7fcf140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 0x7fcf140693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf24001840 tx=0x7fcf24009040 comp rx=0 tx=0).stop 2023-10-10T20:21:36.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 msgr2=0x7fcf340adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406b530 tx=0x7fcf1c017000 comp rx=0 tx=0).stop 2023-10-10T20:21:36.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 shutdown_connections 2023-10-10T20:21:36.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf14066f30 0x7fcf140693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf3415d580 0x7fcf340b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf34157870 0x7fcf3414b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 --2- 172.21.15.148:0/1469790246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf3415a620 0x7fcf340adb70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 >> 172.21.15.148:0/1469790246 conn(0x7fcf34068ce0 msgr2=0x7fcf340690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:36.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 shutdown_connections 2023-10-10T20:21:36.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.548+0000 7fcf38c94700 1 -- 172.21.15.148:0/1469790246 wait complete. 2023-10-10T20:21:36.555 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:21:36.562 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2023-10-10T20:21:35.350215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558695,"statfs":{"total":1600000000,"available":1596104010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001976"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.646343+0000","last_change":"2023-10-10T20:21:27.646281+0000","last_active":"2023-10-10T20:21:27.646343+0000","last_peered":"2023-10-10T20:21:27.646343+0000","last_clean":"2023-10-10T20:21:27.646343+0000","last_became_active":"2023-10-10T20:21:27.646059+0000","last_became_peered":"2023-10-10T20:21:27.646059+0000","last_unstale":"2023-10-10T20:21:27.646343+0000","last_undegraded":"2023-10-10T20:21:27.646343+0000","last_fullsized":"2023-10-10T20:21:27.646343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:07.163311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643668+0000","last_change":"2023-10-10T20:21:27.642590+0000","last_active":"2023-10-10T20:21:27.643668+0000","last_peered":"2023-10-10T20:21:27.643668+0000","last_clean":"2023-10-10T20:21:27.643668+0000","last_became_active":"2023-10-10T20:21:27.641783+0000","last_became_peered":"2023-10-10T20:21:27.641783+0000","last_unstale":"2023-10-10T20:21:27.643668+0000","last_undegraded":"2023-10-10T20:21:27.643668+0000","last_fullsized":"2023-10-10T20:21:27.643668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:01:11.251439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639610+0000","last_change":"2023-10-10T20:21:27.639511+0000","last_active":"2023-10-10T20:21:27.639610+0000","last_peered":"2023-10-10T20:21:27.639610+0000","last_clean":"2023-10-10T20:21:27.639610+0000","last_became_active":"2023-10-10T20:21:27.639169+0000","last_became_peered":"2023-10-10T20:21:27.639169+0000","last_unstale":"2023-10-10T20:21:27.639610+0000","last_undegraded":"2023-10-10T20:21:27.639610+0000","last_fullsized":"2023-10-10T20:21:27.639610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642813+0000","last_change":"2023-10-10T20:21:27.642697+0000","last_active":"2023-10-10T20:21:27.642813+0000","last_peered":"2023-10-10T20:21:27.642813+0000","last_clean":"2023-10-10T20:21:27.642813+0000","last_became_active":"2023-10-10T20:21:27.642467+0000","last_became_peered":"2023-10-10T20:21:27.642467+0000","last_unstale":"2023-10-10T20:21:27.642813+0000","last_undegraded":"2023-10-10T20:21:27.642813+0000","last_fullsized":"2023-10-10T20:21:27.642813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:48.636801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643279+0000","last_change":"2023-10-10T20:21:27.642586+0000","last_active":"2023-10-10T20:21:27.643279+0000","last_peered":"2023-10-10T20:21:27.643279+0000","last_clean":"2023-10-10T20:21:27.643279+0000","last_became_active":"2023-10-10T20:21:27.642291+0000","last_became_peered":"2023-10-10T20:21:27.642291+0000","last_unstale":"2023-10-10T20:21:27.643279+0000","last_undegraded":"2023-10-10T20:21:27.643279+0000","last_fullsized":"2023-10-10T20:21:27.643279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:31.730487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.645121+0000","last_change":"2023-10-10T20:21:27.645021+0000","last_active":"2023-10-10T20:21:27.645121+0000","last_peered":"2023-10-10T20:21:27.645121+0000","last_clean":"2023-10-10T20:21:27.645121+0000","last_became_active":"2023-10-10T20:21:27.644482+0000","last_became_peered":"2023-10-10T20:21:27.644482+0000","last_unstale":"2023-10-10T20:21:27.645121+0000","last_undegraded":"2023-10-10T20:21:27.645121+0000","last_fullsized":"2023-10-10T20:21:27.645121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.633786+0000","last_change":"2023-10-10T20:21:27.633610+0000","last_active":"2023-10-10T20:21:27.633786+0000","last_peered":"2023-10-10T20:21:27.633786+0000","last_clean":"2023-10-10T20:21:27.633786+0000","last_became_active":"2023-10-10T20:21:27.633394+0000","last_became_peered":"2023-10-10T20:21:27.633394+0000","last_unstale":"2023-10-10T20:21:27.633786+0000","last_undegraded":"2023-10-10T20:21:27.633786+0000","last_fullsized":"2023-10-10T20:21:27.633786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641877+0000","last_change":"2023-10-10T20:21:27.640552+0000","last_active":"2023-10-10T20:21:27.641877+0000","last_peered":"2023-10-10T20:21:27.641877+0000","last_clean":"2023-10-10T20:21:27.641877+0000","last_became_active":"2023-10-10T20:21:27.640098+0000","last_became_peered":"2023-10-10T20:21:27.640098+0000","last_unstale":"2023-10-10T20:21:27.641877+0000","last_undegraded":"2023-10-10T20:21:27.641877+0000","last_fullsized":"2023-10-10T20:21:27.641877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:08:13.848460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.634319+0000","last_change":"2023-10-10T20:21:27.634171+0000","last_active":"2023-10-10T20:21:27.634319+0000","last_peered":"2023-10-10T20:21:27.634319+0000","last_clean":"2023-10-10T20:21:27.634319+0000","last_became_active":"2023-10-10T20:21:27.633946+0000","last_became_peered":"2023-10-10T20:21:27.633946+0000","last_unstale":"2023-10-10T20:21:27.634319+0000","last_undegraded":"2023-10-10T20:21:27.634319+0000","last_fullsized":"2023-10-10T20:21:27.634319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.646213+0000","last_change":"2023-10-10T20:21:27.646156+0000","last_active":"2023-10-10T20:21:27.646213+0000","last_peered":"2023-10-10T20:21:27.646213+0000","last_clean":"2023-10-10T20:21:27.646213+0000","last_became_active":"2023-10-10T20:21:27.645525+0000","last_became_peered":"2023-10-10T20:21:27.645525+0000","last_unstale":"2023-10-10T20:21:27.646213+0000","last_undegraded":"2023-10-10T20:21:27.646213+0000","last_fullsized":"2023-10-10T20:21:27.646213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:17.112441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644805+0000","last_change":"2023-10-10T20:21:27.644745+0000","last_active":"2023-10-10T20:21:27.644805+0000","last_peered":"2023-10-10T20:21:27.644805+0000","last_clean":"2023-10-10T20:21:27.644805+0000","last_became_active":"2023-10-10T20:21:27.644505+0000","last_became_peered":"2023-10-10T20:21:27.644505+0000","last_unstale":"2023-10-10T20:21:27.644805+0000","last_undegraded":"2023-10-10T20:21:27.644805+0000","last_fullsized":"2023-10-10T20:21:27.644805+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:05.964593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.638436+0000","last_change":"2023-10-10T20:21:27.637799+0000","last_active":"2023-10-10T20:21:27.638436+0000","last_peered":"2023-10-10T20:21:27.638436+0000","last_clean":"2023-10-10T20:21:27.638436+0000","last_became_active":"2023-10-10T20:21:27.637439+0000","last_became_peered":"2023-10-10T20:21:27.637439+0000","last_unstale":"2023-10-10T20:21:27.638436+0000","last_undegraded":"2023-10-10T20:21:27.638436+0000","last_fullsized":"2023-10-10T20:21:27.638436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642907+0000","last_change":"2023-10-10T20:21:27.642841+0000","last_active":"2023-10-10T20:21:27.642907+0000","last_peered":"2023-10-10T20:21:27.642907+0000","last_clean":"2023-10-10T20:21:27.642907+0000","last_became_active":"2023-10-10T20:21:27.642607+0000","last_became_peered":"2023-10-10T20:21:27.642607+0000","last_unstale":"2023-10-10T20:21:27.642907+0000","last_undegraded":"2023-10-10T20:21:27.642907+0000","last_fullsized":"2023-10-10T20:21:27.642907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:09.042467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644218+0000","last_change":"2023-10-10T20:21:27.644151+0000","last_active":"2023-10-10T20:21:27.644218+0000","last_peered":"2023-10-10T20:21:27.644218+0000","last_clean":"2023-10-10T20:21:27.644218+0000","last_became_active":"2023-10-10T20:21:27.643931+0000","last_became_peered":"2023-10-10T20:21:27.643931+0000","last_unstale":"2023-10-10T20:21:27.644218+0000","last_undegraded":"2023-10-10T20:21:27.644218+0000","last_fullsized":"2023-10-10T20:21:27.644218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640696+0000","last_change":"2023-10-10T20:21:27.640491+0000","last_active":"2023-10-10T20:21:27.640696+0000","last_peered":"2023-10-10T20:21:27.640696+0000","last_clean":"2023-10-10T20:21:27.640696+0000","last_became_active":"2023-10-10T20:21:27.640191+0000","last_became_peered":"2023-10-10T20:21:27.640191+0000","last_unstale":"2023-10-10T20:21:27.640696+0000","last_undegraded":"2023-10-10T20:21:27.640696+0000","last_fullsized":"2023-10-10T20:21:27.640696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:55.217549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639222+0000","last_change":"2023-10-10T20:21:27.639133+0000","last_active":"2023-10-10T20:21:27.639222+0000","last_peered":"2023-10-10T20:21:27.639222+0000","last_clean":"2023-10-10T20:21:27.639222+0000","last_became_active":"2023-10-10T20:21:27.638507+0000","last_became_peered":"2023-10-10T20:21:27.638507+0000","last_unstale":"2023-10-10T20:21:27.639222+0000","last_undegraded":"2023-10-10T20:21:27.639222+0000","last_fullsized":"2023-10-10T20:21:27.639222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:45.965845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643678+0000","last_change":"2023-10-10T20:21:27.643070+0000","last_active":"2023-10-10T20:21:27.643678+0000","last_peered":"2023-10-10T20:21:27.643678+0000","last_clean":"2023-10-10T20:21:27.643678+0000","last_became_active":"2023-10-10T20:21:27.642069+0000","last_became_peered":"2023-10-10T20:21:27.642069+0000","last_unstale":"2023-10-10T20:21:27.643678+0000","last_undegraded":"2023-10-10T20:21:27.643678+0000","last_fullsized":"2023-10-10T20:21:27.643678+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:17:47.764378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644231+0000","last_change":"2023-10-10T20:21:27.644147+0000","last_active":"2023-10-10T20:21:27.644231+0000","last_peered":"2023-10-10T20:21:27.644231+0000","last_clean":"2023-10-10T20:21:27.644231+0000","last_became_active":"2023-10-10T20:21:27.643431+0000","last_became_peered":"2023-10-10T20:21:27.643431+0000","last_unstale":"2023-10-10T20:21:27.644231+0000","last_undegraded":"2023-10-10T20:21:27.644231+0000","last_fullsized":"2023-10-10T20:21:27.644231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642031+0000","last_change":"2023-10-10T20:21:27.641558+0000","last_active":"2023-10-10T20:21:27.642031+0000","last_peered":"2023-10-10T20:21:27.642031+0000","last_clean":"2023-10-10T20:21:27.642031+0000","last_became_active":"2023-10-10T20:21:27.641331+0000","last_became_peered":"2023-10-10T20:21:27.641331+0000","last_unstale":"2023-10-10T20:21:27.642031+0000","last_undegraded":"2023-10-10T20:21:27.642031+0000","last_fullsized":"2023-10-10T20:21:27.642031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:51.668958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.638966+0000","last_change":"2023-10-10T20:21:27.638880+0000","last_active":"2023-10-10T20:21:27.638966+0000","last_peered":"2023-10-10T20:21:27.638966+0000","last_clean":"2023-10-10T20:21:27.638966+0000","last_became_active":"2023-10-10T20:21:27.638649+0000","last_became_peered":"2023-10-10T20:21:27.638649+0000","last_unstale":"2023-10-10T20:21:27.638966+0000","last_undegraded":"2023-10-10T20:21:27.638966+0000","last_fullsized":"2023-10-10T20:21:27.638966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:08:20.755366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640392+0000","last_change":"2023-10-10T20:21:27.640323+0000","last_active":"2023-10-10T20:21:27.640392+0000","last_peered":"2023-10-10T20:21:27.640392+0000","last_clean":"2023-10-10T20:21:27.640392+0000","last_became_active":"2023-10-10T20:21:27.640145+0000","last_became_peered":"2023-10-10T20:21:27.640145+0000","last_unstale":"2023-10-10T20:21:27.640392+0000","last_undegraded":"2023-10-10T20:21:27.640392+0000","last_fullsized":"2023-10-10T20:21:27.640392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:03.198289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644608+0000","last_change":"2023-10-10T20:21:27.644114+0000","last_active":"2023-10-10T20:21:27.644608+0000","last_peered":"2023-10-10T20:21:27.644608+0000","last_clean":"2023-10-10T20:21:27.644608+0000","last_became_active":"2023-10-10T20:21:27.643426+0000","last_became_peered":"2023-10-10T20:21:27.643426+0000","last_unstale":"2023-10-10T20:21:27.644608+0000","last_undegraded":"2023-10-10T20:21:27.644608+0000","last_fullsized":"2023-10-10T20:21:27.644608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:41:31.997996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644664+0000","last_change":"2023-10-10T20:21:27.644463+0000","last_active":"2023-10-10T20:21:27.644664+0000","last_peered":"2023-10-10T20:21:27.644664+0000","last_clean":"2023-10-10T20:21:27.644664+0000","last_became_active":"2023-10-10T20:21:27.643806+0000","last_became_peered":"2023-10-10T20:21:27.643806+0000","last_unstale":"2023-10-10T20:21:27.644664+0000","last_undegraded":"2023-10-10T20:21:27.644664+0000","last_fullsized":"2023-10-10T20:21:27.644664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639894+0000","last_change":"2023-10-10T20:21:27.639832+0000","last_active":"2023-10-10T20:21:27.639894+0000","last_peered":"2023-10-10T20:21:27.639894+0000","last_clean":"2023-10-10T20:21:27.639894+0000","last_became_active":"2023-10-10T20:21:27.639611+0000","last_became_peered":"2023-10-10T20:21:27.639611+0000","last_unstale":"2023-10-10T20:21:27.639894+0000","last_undegraded":"2023-10-10T20:21:27.639894+0000","last_fullsized":"2023-10-10T20:21:27.639894+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:24.116949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640858+0000","last_change":"2023-10-10T20:21:27.640799+0000","last_active":"2023-10-10T20:21:27.640858+0000","last_peered":"2023-10-10T20:21:27.640858+0000","last_clean":"2023-10-10T20:21:27.640858+0000","last_became_active":"2023-10-10T20:21:27.640560+0000","last_became_peered":"2023-10-10T20:21:27.640560+0000","last_unstale":"2023-10-10T20:21:27.640858+0000","last_undegraded":"2023-10-10T20:21:27.640858+0000","last_fullsized":"2023-10-10T20:21:27.640858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:28:06.175064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.634015+0000","last_change":"2023-10-10T20:21:27.633946+0000","last_active":"2023-10-10T20:21:27.634015+0000","last_peered":"2023-10-10T20:21:27.634015+0000","last_clean":"2023-10-10T20:21:27.634015+0000","last_became_active":"2023-10-10T20:21:27.633736+0000","last_became_peered":"2023-10-10T20:21:27.633736+0000","last_unstale":"2023-10-10T20:21:27.634015+0000","last_undegraded":"2023-10-10T20:21:27.634015+0000","last_fullsized":"2023-10-10T20:21:27.634015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:33.343151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.637422+0000","last_change":"2023-10-10T20:21:27.637110+0000","last_active":"2023-10-10T20:21:27.637422+0000","last_peered":"2023-10-10T20:21:27.637422+0000","last_clean":"2023-10-10T20:21:27.637422+0000","last_became_active":"2023-10-10T20:21:27.636892+0000","last_became_peered":"2023-10-10T20:21:27.636892+0000","last_unstale":"2023-10-10T20:21:27.637422+0000","last_undegraded":"2023-10-10T20:21:27.637422+0000","last_fullsized":"2023-10-10T20:21:27.637422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:17.318048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643316+0000","last_change":"2023-10-10T20:21:27.642170+0000","last_active":"2023-10-10T20:21:27.643316+0000","last_peered":"2023-10-10T20:21:27.643316+0000","last_clean":"2023-10-10T20:21:27.643316+0000","last_became_active":"2023-10-10T20:21:27.641944+0000","last_became_peered":"2023-10-10T20:21:27.641944+0000","last_unstale":"2023-10-10T20:21:27.643316+0000","last_undegraded":"2023-10-10T20:21:27.643316+0000","last_fullsized":"2023-10-10T20:21:27.643316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:47.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644624+0000","last_change":"2023-10-10T20:21:27.644195+0000","last_active":"2023-10-10T20:21:27.644624+0000","last_peered":"2023-10-10T20:21:27.644624+0000","last_clean":"2023-10-10T20:21:27.644624+0000","last_became_active":"2023-10-10T20:21:27.643842+0000","last_became_peered":"2023-10-10T20:21:27.643842+0000","last_unstale":"2023-10-10T20:21:27.644624+0000","last_undegraded":"2023-10-10T20:21:27.644624+0000","last_fullsized":"2023-10-10T20:21:27.644624+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:08:39.856473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639593+0000","last_change":"2023-10-10T20:21:27.639372+0000","last_active":"2023-10-10T20:21:27.639593+0000","last_peered":"2023-10-10T20:21:27.639593+0000","last_clean":"2023-10-10T20:21:27.639593+0000","last_became_active":"2023-10-10T20:21:27.638743+0000","last_became_peered":"2023-10-10T20:21:27.638743+0000","last_unstale":"2023-10-10T20:21:27.639593+0000","last_undegraded":"2023-10-10T20:21:27.639593+0000","last_fullsized":"2023-10-10T20:21:27.639593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:33.125028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643300+0000","last_change":"2023-10-10T20:21:27.642722+0000","last_active":"2023-10-10T20:21:27.643300+0000","last_peered":"2023-10-10T20:21:27.643300+0000","last_clean":"2023-10-10T20:21:27.643300+0000","last_became_active":"2023-10-10T20:21:27.642490+0000","last_became_peered":"2023-10-10T20:21:27.642490+0000","last_unstale":"2023-10-10T20:21:27.643300+0000","last_undegraded":"2023-10-10T20:21:27.643300+0000","last_fullsized":"2023-10-10T20:21:27.643300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:16:57.773232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644950+0000","last_change":"2023-10-10T20:21:27.643647+0000","last_active":"2023-10-10T20:21:27.644950+0000","last_peered":"2023-10-10T20:21:27.644950+0000","last_clean":"2023-10-10T20:21:27.644950+0000","last_became_active":"2023-10-10T20:21:27.643291+0000","last_became_peered":"2023-10-10T20:21:27.643291+0000","last_unstale":"2023-10-10T20:21:27.644950+0000","last_undegraded":"2023-10-10T20:21:27.644950+0000","last_fullsized":"2023-10-10T20:21:27.644950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639490+0000","last_change":"2023-10-10T20:21:27.639431+0000","last_active":"2023-10-10T20:21:27.639490+0000","last_peered":"2023-10-10T20:21:27.639490+0000","last_clean":"2023-10-10T20:21:27.639490+0000","last_became_active":"2023-10-10T20:21:27.639207+0000","last_became_peered":"2023-10-10T20:21:27.639207+0000","last_unstale":"2023-10-10T20:21:27.639490+0000","last_undegraded":"2023-10-10T20:21:27.639490+0000","last_fullsized":"2023-10-10T20:21:27.639490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:13:28.395817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.657802+0000","last_change":"2023-10-10T20:21:26.620953+0000","last_active":"2023-10-10T20:21:27.657802+0000","last_peered":"2023-10-10T20:21:27.657802+0000","last_clean":"2023-10-10T20:21:27.657802+0000","last_became_active":"2023-10-10T20:21:26.620376+0000","last_became_peered":"2023-10-10T20:21:26.620376+0000","last_unstale":"2023-10-10T20:21:27.657802+0000","last_undegraded":"2023-10-10T20:21:27.657802+0000","last_fullsized":"2023-10-10T20:21:27.657802+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:40.544270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641527+0000","last_change":"2023-10-10T20:21:27.641457+0000","last_active":"2023-10-10T20:21:27.641527+0000","last_peered":"2023-10-10T20:21:27.641527+0000","last_clean":"2023-10-10T20:21:27.641527+0000","last_became_active":"2023-10-10T20:21:27.641233+0000","last_became_peered":"2023-10-10T20:21:27.641233+0000","last_unstale":"2023-10-10T20:21:27.641527+0000","last_undegraded":"2023-10-10T20:21:27.641527+0000","last_fullsized":"2023-10-10T20:21:27.641527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:24.116949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.654863+0000","last_change":"2023-10-10T20:21:26.624607+0000","last_active":"2023-10-10T20:21:27.654863+0000","last_peered":"2023-10-10T20:21:27.654863+0000","last_clean":"2023-10-10T20:21:27.654863+0000","last_became_active":"2023-10-10T20:21:26.624069+0000","last_became_peered":"2023-10-10T20:21:26.624069+0000","last_unstale":"2023-10-10T20:21:27.654863+0000","last_undegraded":"2023-10-10T20:21:27.654863+0000","last_fullsized":"2023-10-10T20:21:27.654863+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:31.294895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644968+0000","last_change":"2023-10-10T20:21:27.644877+0000","last_active":"2023-10-10T20:21:27.644968+0000","last_peered":"2023-10-10T20:21:27.644968+0000","last_clean":"2023-10-10T20:21:27.644968+0000","last_became_active":"2023-10-10T20:21:27.644186+0000","last_became_peered":"2023-10-10T20:21:27.644186+0000","last_unstale":"2023-10-10T20:21:27.644968+0000","last_undegraded":"2023-10-10T20:21:27.644968+0000","last_fullsized":"2023-10-10T20:21:27.644968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.657022+0000","last_change":"2023-10-10T20:21:26.628774+0000","last_active":"2023-10-10T20:21:27.657022+0000","last_peered":"2023-10-10T20:21:27.657022+0000","last_clean":"2023-10-10T20:21:27.657022+0000","last_became_active":"2023-10-10T20:21:26.628427+0000","last_became_peered":"2023-10-10T20:21:26.628427+0000","last_unstale":"2023-10-10T20:21:27.657022+0000","last_undegraded":"2023-10-10T20:21:27.657022+0000","last_fullsized":"2023-10-10T20:21:27.657022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:11.265574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.634065+0000","last_change":"2023-10-10T20:21:27.633988+0000","last_active":"2023-10-10T20:21:27.634065+0000","last_peered":"2023-10-10T20:21:27.634065+0000","last_clean":"2023-10-10T20:21:27.634065+0000","last_became_active":"2023-10-10T20:21:27.633803+0000","last_became_peered":"2023-10-10T20:21:27.633803+0000","last_unstale":"2023-10-10T20:21:27.634065+0000","last_undegraded":"2023-10-10T20:21:27.634065+0000","last_fullsized":"2023-10-10T20:21:27.634065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:14.822522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.655285+0000","last_change":"2023-10-10T20:21:26.631546+0000","last_active":"2023-10-10T20:21:27.655285+0000","last_peered":"2023-10-10T20:21:27.655285+0000","last_clean":"2023-10-10T20:21:27.655285+0000","last_became_active":"2023-10-10T20:21:26.631032+0000","last_became_peered":"2023-10-10T20:21:26.631032+0000","last_unstale":"2023-10-10T20:21:27.655285+0000","last_undegraded":"2023-10-10T20:21:27.655285+0000","last_fullsized":"2023-10-10T20:21:27.655285+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:36.676962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644897+0000","last_change":"2023-10-10T20:21:27.644836+0000","last_active":"2023-10-10T20:21:27.644897+0000","last_peered":"2023-10-10T20:21:27.644897+0000","last_clean":"2023-10-10T20:21:27.644897+0000","last_became_active":"2023-10-10T20:21:27.644519+0000","last_became_peered":"2023-10-10T20:21:27.644519+0000","last_unstale":"2023-10-10T20:21:27.644897+0000","last_undegraded":"2023-10-10T20:21:27.644897+0000","last_fullsized":"2023-10-10T20:21:27.644897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:00:39.591932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.656561+0000","last_change":"2023-10-10T20:21:26.626156+0000","last_active":"2023-10-10T20:21:27.656561+0000","last_peered":"2023-10-10T20:21:27.656561+0000","last_clean":"2023-10-10T20:21:27.656561+0000","last_became_active":"2023-10-10T20:21:26.625622+0000","last_became_peered":"2023-10-10T20:21:26.625622+0000","last_unstale":"2023-10-10T20:21:27.656561+0000","last_undegraded":"2023-10-10T20:21:27.656561+0000","last_fullsized":"2023-10-10T20:21:27.656561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643689+0000","last_change":"2023-10-10T20:21:27.643209+0000","last_active":"2023-10-10T20:21:27.643689+0000","last_peered":"2023-10-10T20:21:27.643689+0000","last_clean":"2023-10-10T20:21:27.643689+0000","last_became_active":"2023-10-10T20:21:27.642366+0000","last_became_peered":"2023-10-10T20:21:27.642366+0000","last_unstale":"2023-10-10T20:21:27.643689+0000","last_undegraded":"2023-10-10T20:21:27.643689+0000","last_fullsized":"2023-10-10T20:21:27.643689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:58:42.001781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.658106+0000","last_change":"2023-10-10T20:21:26.626984+0000","last_active":"2023-10-10T20:21:27.658106+0000","last_peered":"2023-10-10T20:21:27.658106+0000","last_clean":"2023-10-10T20:21:27.658106+0000","last_became_active":"2023-10-10T20:21:26.626672+0000","last_became_peered":"2023-10-10T20:21:26.626672+0000","last_unstale":"2023-10-10T20:21:27.658106+0000","last_undegraded":"2023-10-10T20:21:27.658106+0000","last_fullsized":"2023-10-10T20:21:27.658106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641886+0000","last_change":"2023-10-10T20:21:27.641816+0000","last_active":"2023-10-10T20:21:27.641886+0000","last_peered":"2023-10-10T20:21:27.641886+0000","last_clean":"2023-10-10T20:21:27.641886+0000","last_became_active":"2023-10-10T20:21:27.641032+0000","last_became_peered":"2023-10-10T20:21:27.641032+0000","last_unstale":"2023-10-10T20:21:27.641886+0000","last_undegraded":"2023-10-10T20:21:27.641886+0000","last_fullsized":"2023-10-10T20:21:27.641886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:17.112441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.630762+0000","last_change":"2023-10-10T20:21:26.630463+0000","last_active":"2023-10-10T20:21:27.630762+0000","last_peered":"2023-10-10T20:21:27.630762+0000","last_clean":"2023-10-10T20:21:27.630762+0000","last_became_active":"2023-10-10T20:21:26.630236+0000","last_became_peered":"2023-10-10T20:21:26.630236+0000","last_unstale":"2023-10-10T20:21:27.630762+0000","last_undegraded":"2023-10-10T20:21:27.630762+0000","last_fullsized":"2023-10-10T20:21:27.630762+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639792+0000","last_change":"2023-10-10T20:21:27.639715+0000","last_active":"2023-10-10T20:21:27.639792+0000","last_peered":"2023-10-10T20:21:27.639792+0000","last_clean":"2023-10-10T20:21:27.639792+0000","last_became_active":"2023-10-10T20:21:27.639483+0000","last_became_peered":"2023-10-10T20:21:27.639483+0000","last_unstale":"2023-10-10T20:21:27.639792+0000","last_undegraded":"2023-10-10T20:21:27.639792+0000","last_fullsized":"2023-10-10T20:21:27.639792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:22:17.057987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.632354+0000","last_change":"2023-10-10T20:21:26.625829+0000","last_active":"2023-10-10T20:21:27.632354+0000","last_peered":"2023-10-10T20:21:27.632354+0000","last_clean":"2023-10-10T20:21:27.632354+0000","last_became_active":"2023-10-10T20:21:26.624923+0000","last_became_peered":"2023-10-10T20:21:26.624923+0000","last_unstale":"2023-10-10T20:21:27.632354+0000","last_undegraded":"2023-10-10T20:21:27.632354+0000","last_fullsized":"2023-10-10T20:21:27.632354+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641384+0000","last_change":"2023-10-10T20:21:27.641305+0000","last_active":"2023-10-10T20:21:27.641384+0000","last_peered":"2023-10-10T20:21:27.641384+0000","last_clean":"2023-10-10T20:21:27.641384+0000","last_became_active":"2023-10-10T20:21:27.641015+0000","last_became_peered":"2023-10-10T20:21:27.641015+0000","last_unstale":"2023-10-10T20:21:27.641384+0000","last_undegraded":"2023-10-10T20:21:27.641384+0000","last_fullsized":"2023-10-10T20:21:27.641384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:40:41.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.638978+0000","last_change":"2023-10-10T20:21:26.631482+0000","last_active":"2023-10-10T20:21:27.638978+0000","last_peered":"2023-10-10T20:21:27.638978+0000","last_clean":"2023-10-10T20:21:27.638978+0000","last_became_active":"2023-10-10T20:21:26.630913+0000","last_became_peered":"2023-10-10T20:21:26.630913+0000","last_unstale":"2023-10-10T20:21:27.638978+0000","last_undegraded":"2023-10-10T20:21:27.638978+0000","last_fullsized":"2023-10-10T20:21:27.638978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:08.034232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643313+0000","last_change":"2023-10-10T20:21:27.643250+0000","last_active":"2023-10-10T20:21:27.643313+0000","last_peered":"2023-10-10T20:21:27.643313+0000","last_clean":"2023-10-10T20:21:27.643313+0000","last_became_active":"2023-10-10T20:21:27.642698+0000","last_became_peered":"2023-10-10T20:21:27.642698+0000","last_unstale":"2023-10-10T20:21:27.643313+0000","last_undegraded":"2023-10-10T20:21:27.643313+0000","last_fullsized":"2023-10-10T20:21:27.643313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:27:18.483407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640975+0000","last_change":"2023-10-10T20:21:26.626142+0000","last_active":"2023-10-10T20:21:27.640975+0000","last_peered":"2023-10-10T20:21:27.640975+0000","last_clean":"2023-10-10T20:21:27.640975+0000","last_became_active":"2023-10-10T20:21:26.625123+0000","last_became_peered":"2023-10-10T20:21:26.625123+0000","last_unstale":"2023-10-10T20:21:27.640975+0000","last_undegraded":"2023-10-10T20:21:27.640975+0000","last_fullsized":"2023-10-10T20:21:27.640975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:07.454724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639989+0000","last_change":"2023-10-10T20:21:27.639770+0000","last_active":"2023-10-10T20:21:27.639989+0000","last_peered":"2023-10-10T20:21:27.639989+0000","last_clean":"2023-10-10T20:21:27.639989+0000","last_became_active":"2023-10-10T20:21:27.639556+0000","last_became_peered":"2023-10-10T20:21:27.639556+0000","last_unstale":"2023-10-10T20:21:27.639989+0000","last_undegraded":"2023-10-10T20:21:27.639989+0000","last_fullsized":"2023-10-10T20:21:27.639989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:24.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.659229+0000","last_change":"2023-10-10T20:21:26.628970+0000","last_active":"2023-10-10T20:21:27.659229+0000","last_peered":"2023-10-10T20:21:27.659229+0000","last_clean":"2023-10-10T20:21:27.659229+0000","last_became_active":"2023-10-10T20:21:26.628746+0000","last_became_peered":"2023-10-10T20:21:26.628746+0000","last_unstale":"2023-10-10T20:21:27.659229+0000","last_undegraded":"2023-10-10T20:21:27.659229+0000","last_fullsized":"2023-10-10T20:21:27.659229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640716+0000","last_change":"2023-10-10T20:21:27.640580+0000","last_active":"2023-10-10T20:21:27.640716+0000","last_peered":"2023-10-10T20:21:27.640716+0000","last_clean":"2023-10-10T20:21:27.640716+0000","last_became_active":"2023-10-10T20:21:27.640348+0000","last_became_peered":"2023-10-10T20:21:27.640348+0000","last_unstale":"2023-10-10T20:21:27.640716+0000","last_undegraded":"2023-10-10T20:21:27.640716+0000","last_fullsized":"2023-10-10T20:21:27.640716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"65'6","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.659342+0000","last_change":"2023-10-10T20:21:26.621850+0000","last_active":"2023-10-10T20:21:27.659342+0000","last_peered":"2023-10-10T20:21:27.659342+0000","last_clean":"2023-10-10T20:21:27.659342+0000","last_became_active":"2023-10-10T20:21:26.621603+0000","last_became_peered":"2023-10-10T20:21:26.621603+0000","last_unstale":"2023-10-10T20:21:27.659342+0000","last_undegraded":"2023-10-10T20:21:27.659342+0000","last_fullsized":"2023-10-10T20:21:27.659342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:39.724039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641096+0000","last_change":"2023-10-10T20:21:27.641012+0000","last_active":"2023-10-10T20:21:27.641096+0000","last_peered":"2023-10-10T20:21:27.641096+0000","last_clean":"2023-10-10T20:21:27.641096+0000","last_became_active":"2023-10-10T20:21:27.640566+0000","last_became_peered":"2023-10-10T20:21:27.640566+0000","last_unstale":"2023-10-10T20:21:27.641096+0000","last_undegraded":"2023-10-10T20:21:27.641096+0000","last_fullsized":"2023-10-10T20:21:27.641096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:36.395076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.654987+0000","last_change":"2023-10-10T20:21:26.626666+0000","last_active":"2023-10-10T20:21:27.654987+0000","last_peered":"2023-10-10T20:21:27.654987+0000","last_clean":"2023-10-10T20:21:27.654987+0000","last_became_active":"2023-10-10T20:21:26.626298+0000","last_became_peered":"2023-10-10T20:21:26.626298+0000","last_unstale":"2023-10-10T20:21:27.654987+0000","last_undegraded":"2023-10-10T20:21:27.654987+0000","last_fullsized":"2023-10-10T20:21:27.654987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:35.386841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639926+0000","last_change":"2023-10-10T20:21:27.639715+0000","last_active":"2023-10-10T20:21:27.639926+0000","last_peered":"2023-10-10T20:21:27.639926+0000","last_clean":"2023-10-10T20:21:27.639926+0000","last_became_active":"2023-10-10T20:21:27.639041+0000","last_became_peered":"2023-10-10T20:21:27.639041+0000","last_unstale":"2023-10-10T20:21:27.639926+0000","last_undegraded":"2023-10-10T20:21:27.639926+0000","last_fullsized":"2023-10-10T20:21:27.639926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:33.519421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.635983+0000","last_change":"2023-10-10T20:21:26.630545+0000","last_active":"2023-10-10T20:21:27.635983+0000","last_peered":"2023-10-10T20:21:27.635983+0000","last_clean":"2023-10-10T20:21:27.635983+0000","last_became_active":"2023-10-10T20:21:26.630288+0000","last_became_peered":"2023-10-10T20:21:26.630288+0000","last_unstale":"2023-10-10T20:21:27.635983+0000","last_undegraded":"2023-10-10T20:21:27.635983+0000","last_fullsized":"2023-10-10T20:21:27.635983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:47:53.884820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642951+0000","last_change":"2023-10-10T20:21:27.642785+0000","last_active":"2023-10-10T20:21:27.642951+0000","last_peered":"2023-10-10T20:21:27.642951+0000","last_clean":"2023-10-10T20:21:27.642951+0000","last_became_active":"2023-10-10T20:21:27.642148+0000","last_became_peered":"2023-10-10T20:21:27.642148+0000","last_unstale":"2023-10-10T20:21:27.642951+0000","last_undegraded":"2023-10-10T20:21:27.642951+0000","last_fullsized":"2023-10-10T20:21:27.642951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:04:50.099557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.631465+0000","last_change":"2023-10-10T20:21:26.627411+0000","last_active":"2023-10-10T20:21:27.631465+0000","last_peered":"2023-10-10T20:21:27.631465+0000","last_clean":"2023-10-10T20:21:27.631465+0000","last_became_active":"2023-10-10T20:21:26.627165+0000","last_became_peered":"2023-10-10T20:21:26.627165+0000","last_unstale":"2023-10-10T20:21:27.631465+0000","last_undegraded":"2023-10-10T20:21:27.631465+0000","last_fullsized":"2023-10-10T20:21:27.631465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:32:04.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.646137+0000","last_change":"2023-10-10T20:21:27.646079+0000","last_active":"2023-10-10T20:21:27.646137+0000","last_peered":"2023-10-10T20:21:27.646137+0000","last_clean":"2023-10-10T20:21:27.646137+0000","last_became_active":"2023-10-10T20:21:27.645863+0000","last_became_peered":"2023-10-10T20:21:27.645863+0000","last_unstale":"2023-10-10T20:21:27.646137+0000","last_undegraded":"2023-10-10T20:21:27.646137+0000","last_fullsized":"2023-10-10T20:21:27.646137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:34.780137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"65'8","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.658245+0000","last_change":"2023-10-10T20:21:26.630197+0000","last_active":"2023-10-10T20:21:27.658245+0000","last_peered":"2023-10-10T20:21:27.658245+0000","last_clean":"2023-10-10T20:21:27.658245+0000","last_became_active":"2023-10-10T20:21:26.629955+0000","last_became_peered":"2023-10-10T20:21:26.629955+0000","last_unstale":"2023-10-10T20:21:27.658245+0000","last_undegraded":"2023-10-10T20:21:27.658245+0000","last_fullsized":"2023-10-10T20:21:27.658245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:22:38.101485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644257+0000","last_change":"2023-10-10T20:21:27.644178+0000","last_active":"2023-10-10T20:21:27.644257+0000","last_peered":"2023-10-10T20:21:27.644257+0000","last_clean":"2023-10-10T20:21:27.644257+0000","last_became_active":"2023-10-10T20:21:27.643716+0000","last_became_peered":"2023-10-10T20:21:27.643716+0000","last_unstale":"2023-10-10T20:21:27.644257+0000","last_undegraded":"2023-10-10T20:21:27.644257+0000","last_fullsized":"2023-10-10T20:21:27.644257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:26.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.654780+0000","last_change":"2023-10-10T20:21:26.624742+0000","last_active":"2023-10-10T20:21:27.654780+0000","last_peered":"2023-10-10T20:21:27.654780+0000","last_clean":"2023-10-10T20:21:27.654780+0000","last_became_active":"2023-10-10T20:21:26.624396+0000","last_became_peered":"2023-10-10T20:21:26.624396+0000","last_unstale":"2023-10-10T20:21:27.654780+0000","last_undegraded":"2023-10-10T20:21:27.654780+0000","last_fullsized":"2023-10-10T20:21:27.654780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:43.828326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642335+0000","last_change":"2023-10-10T20:21:27.642231+0000","last_active":"2023-10-10T20:21:27.642335+0000","last_peered":"2023-10-10T20:21:27.642335+0000","last_clean":"2023-10-10T20:21:27.642335+0000","last_became_active":"2023-10-10T20:21:27.641842+0000","last_became_peered":"2023-10-10T20:21:27.641842+0000","last_unstale":"2023-10-10T20:21:27.642335+0000","last_undegraded":"2023-10-10T20:21:27.642335+0000","last_fullsized":"2023-10-10T20:21:27.642335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:42:04.675636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.657940+0000","last_change":"2023-10-10T20:21:26.628656+0000","last_active":"2023-10-10T20:21:27.657940+0000","last_peered":"2023-10-10T20:21:27.657940+0000","last_clean":"2023-10-10T20:21:27.657940+0000","last_became_active":"2023-10-10T20:21:26.627500+0000","last_became_peered":"2023-10-10T20:21:26.627500+0000","last_unstale":"2023-10-10T20:21:27.657940+0000","last_undegraded":"2023-10-10T20:21:27.657940+0000","last_fullsized":"2023-10-10T20:21:27.657940+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:26:32.116793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644591+0000","last_change":"2023-10-10T20:21:27.643933+0000","last_active":"2023-10-10T20:21:27.644591+0000","last_peered":"2023-10-10T20:21:27.644591+0000","last_clean":"2023-10-10T20:21:27.644591+0000","last_became_active":"2023-10-10T20:21:27.642834+0000","last_became_peered":"2023-10-10T20:21:27.642834+0000","last_unstale":"2023-10-10T20:21:27.644591+0000","last_undegraded":"2023-10-10T20:21:27.644591+0000","last_fullsized":"2023-10-10T20:21:27.644591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:46.104445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.655417+0000","last_change":"2023-10-10T20:21:26.631690+0000","last_active":"2023-10-10T20:21:27.655417+0000","last_peered":"2023-10-10T20:21:27.655417+0000","last_clean":"2023-10-10T20:21:27.655417+0000","last_became_active":"2023-10-10T20:21:26.631324+0000","last_became_peered":"2023-10-10T20:21:26.631324+0000","last_unstale":"2023-10-10T20:21:27.655417+0000","last_undegraded":"2023-10-10T20:21:27.655417+0000","last_fullsized":"2023-10-10T20:21:27.655417+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:07:51.891574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643517+0000","last_change":"2023-10-10T20:21:27.643440+0000","last_active":"2023-10-10T20:21:27.643517+0000","last_peered":"2023-10-10T20:21:27.643517+0000","last_clean":"2023-10-10T20:21:27.643517+0000","last_became_active":"2023-10-10T20:21:27.642983+0000","last_became_peered":"2023-10-10T20:21:27.642983+0000","last_unstale":"2023-10-10T20:21:27.643517+0000","last_undegraded":"2023-10-10T20:21:27.643517+0000","last_fullsized":"2023-10-10T20:21:27.643517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.633473+0000","last_change":"2023-10-10T20:21:26.627796+0000","last_active":"2023-10-10T20:21:27.633473+0000","last_peered":"2023-10-10T20:21:27.633473+0000","last_clean":"2023-10-10T20:21:27.633473+0000","last_became_active":"2023-10-10T20:21:26.627566+0000","last_became_peered":"2023-10-10T20:21:26.627566+0000","last_unstale":"2023-10-10T20:21:27.633473+0000","last_undegraded":"2023-10-10T20:21:27.633473+0000","last_fullsized":"2023-10-10T20:21:27.633473+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:25.600786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642166+0000","last_change":"2023-10-10T20:21:27.642079+0000","last_active":"2023-10-10T20:21:27.642166+0000","last_peered":"2023-10-10T20:21:27.642166+0000","last_clean":"2023-10-10T20:21:27.642166+0000","last_became_active":"2023-10-10T20:21:27.641515+0000","last_became_peered":"2023-10-10T20:21:27.641515+0000","last_unstale":"2023-10-10T20:21:27.642166+0000","last_undegraded":"2023-10-10T20:21:27.642166+0000","last_fullsized":"2023-10-10T20:21:27.642166+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:38.515478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.655045+0000","last_change":"2023-10-10T20:21:26.626981+0000","last_active":"2023-10-10T20:21:27.655045+0000","last_peered":"2023-10-10T20:21:27.655045+0000","last_clean":"2023-10-10T20:21:27.655045+0000","last_became_active":"2023-10-10T20:21:26.626635+0000","last_became_peered":"2023-10-10T20:21:26.626635+0000","last_unstale":"2023-10-10T20:21:27.655045+0000","last_undegraded":"2023-10-10T20:21:27.655045+0000","last_fullsized":"2023-10-10T20:21:27.655045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:21:32.511186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.641620+0000","last_change":"2023-10-10T20:21:27.641493+0000","last_active":"2023-10-10T20:21:27.641620+0000","last_peered":"2023-10-10T20:21:27.641620+0000","last_clean":"2023-10-10T20:21:27.641620+0000","last_became_active":"2023-10-10T20:21:27.640865+0000","last_became_peered":"2023-10-10T20:21:27.640865+0000","last_unstale":"2023-10-10T20:21:27.641620+0000","last_undegraded":"2023-10-10T20:21:27.641620+0000","last_fullsized":"2023-10-10T20:21:27.641620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.636840+0000","last_change":"2023-10-10T20:21:26.624041+0000","last_active":"2023-10-10T20:21:27.636840+0000","last_peered":"2023-10-10T20:21:27.636840+0000","last_clean":"2023-10-10T20:21:27.636840+0000","last_became_active":"2023-10-10T20:21:26.623809+0000","last_became_peered":"2023-10-10T20:21:26.623809+0000","last_unstale":"2023-10-10T20:21:27.636840+0000","last_undegraded":"2023-10-10T20:21:27.636840+0000","last_fullsized":"2023-10-10T20:21:27.636840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:36:24.387989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639300+0000","last_change":"2023-10-10T20:21:27.639234+0000","last_active":"2023-10-10T20:21:27.639300+0000","last_peered":"2023-10-10T20:21:27.639300+0000","last_clean":"2023-10-10T20:21:27.639300+0000","last_became_active":"2023-10-10T20:21:27.639007+0000","last_became_peered":"2023-10-10T20:21:27.639007+0000","last_unstale":"2023-10-10T20:21:27.639300+0000","last_undegraded":"2023-10-10T20:21:27.639300+0000","last_fullsized":"2023-10-10T20:21:27.639300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:08.983333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.633225+0000","last_change":"2023-10-10T20:21:26.629529+0000","last_active":"2023-10-10T20:21:27.633225+0000","last_peered":"2023-10-10T20:21:27.633225+0000","last_clean":"2023-10-10T20:21:27.633225+0000","last_became_active":"2023-10-10T20:21:26.629280+0000","last_became_peered":"2023-10-10T20:21:26.629280+0000","last_unstale":"2023-10-10T20:21:27.633225+0000","last_undegraded":"2023-10-10T20:21:27.633225+0000","last_fullsized":"2023-10-10T20:21:27.633225+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:39:23.108714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.642833+0000","last_change":"2023-10-10T20:21:27.642708+0000","last_active":"2023-10-10T20:21:27.642833+0000","last_peered":"2023-10-10T20:21:27.642833+0000","last_clean":"2023-10-10T20:21:27.642833+0000","last_became_active":"2023-10-10T20:21:27.642308+0000","last_became_peered":"2023-10-10T20:21:27.642308+0000","last_unstale":"2023-10-10T20:21:27.642833+0000","last_undegraded":"2023-10-10T20:21:27.642833+0000","last_fullsized":"2023-10-10T20:21:27.642833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:04.285745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.657404+0000","last_change":"2023-10-10T20:21:26.624689+0000","last_active":"2023-10-10T20:21:27.657404+0000","last_peered":"2023-10-10T20:21:27.657404+0000","last_clean":"2023-10-10T20:21:27.657404+0000","last_became_active":"2023-10-10T20:21:26.624473+0000","last_became_peered":"2023-10-10T20:21:26.624473+0000","last_unstale":"2023-10-10T20:21:27.657404+0000","last_undegraded":"2023-10-10T20:21:27.657404+0000","last_fullsized":"2023-10-10T20:21:27.657404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:45:30.722252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.638416+0000","last_change":"2023-10-10T20:21:27.637661+0000","last_active":"2023-10-10T20:21:27.638416+0000","last_peered":"2023-10-10T20:21:27.638416+0000","last_clean":"2023-10-10T20:21:27.638416+0000","last_became_active":"2023-10-10T20:21:27.637131+0000","last_became_peered":"2023-10-10T20:21:27.637131+0000","last_unstale":"2023-10-10T20:21:27.638416+0000","last_undegraded":"2023-10-10T20:21:27.638416+0000","last_fullsized":"2023-10-10T20:21:27.638416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:48:08.462959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-10T20:21:28.634481+0000","last_change":"2023-10-10T20:21:26.626857+0000","last_active":"2023-10-10T20:21:28.634481+0000","last_peered":"2023-10-10T20:21:28.634481+0000","last_clean":"2023-10-10T20:21:28.634481+0000","last_became_active":"2023-10-10T20:21:26.626370+0000","last_became_peered":"2023-10-10T20:21:26.626370+0000","last_unstale":"2023-10-10T20:21:28.634481+0000","last_undegraded":"2023-10-10T20:21:28.634481+0000","last_fullsized":"2023-10-10T20:21:28.634481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.646368+0000","last_change":"2023-10-10T20:21:27.646284+0000","last_active":"2023-10-10T20:21:27.646368+0000","last_peered":"2023-10-10T20:21:27.646368+0000","last_clean":"2023-10-10T20:21:27.646368+0000","last_became_active":"2023-10-10T20:21:27.645907+0000","last_became_peered":"2023-10-10T20:21:27.645907+0000","last_unstale":"2023-10-10T20:21:27.646368+0000","last_undegraded":"2023-10-10T20:21:27.646368+0000","last_fullsized":"2023-10-10T20:21:27.646368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:36:40.732274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.635858+0000","last_change":"2023-10-10T20:21:26.628016+0000","last_active":"2023-10-10T20:21:27.635858+0000","last_peered":"2023-10-10T20:21:27.635858+0000","last_clean":"2023-10-10T20:21:27.635858+0000","last_became_active":"2023-10-10T20:21:26.627749+0000","last_became_peered":"2023-10-10T20:21:26.627749+0000","last_unstale":"2023-10-10T20:21:27.635858+0000","last_undegraded":"2023-10-10T20:21:27.635858+0000","last_fullsized":"2023-10-10T20:21:27.635858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:11:31.392459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.637393+0000","last_change":"2023-10-10T20:21:27.635618+0000","last_active":"2023-10-10T20:21:27.637393+0000","last_peered":"2023-10-10T20:21:27.637393+0000","last_clean":"2023-10-10T20:21:27.637393+0000","last_became_active":"2023-10-10T20:21:27.635406+0000","last_became_peered":"2023-10-10T20:21:27.635406+0000","last_unstale":"2023-10-10T20:21:27.637393+0000","last_undegraded":"2023-10-10T20:21:27.637393+0000","last_fullsized":"2023-10-10T20:21:27.637393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:39.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.630885+0000","last_change":"2023-10-10T20:21:26.624500+0000","last_active":"2023-10-10T20:21:27.630885+0000","last_peered":"2023-10-10T20:21:27.630885+0000","last_clean":"2023-10-10T20:21:27.630885+0000","last_became_active":"2023-10-10T20:21:26.623771+0000","last_became_peered":"2023-10-10T20:21:26.623771+0000","last_unstale":"2023-10-10T20:21:27.630885+0000","last_undegraded":"2023-10-10T20:21:27.630885+0000","last_fullsized":"2023-10-10T20:21:27.630885+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:03.193768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644639+0000","last_change":"2023-10-10T20:21:27.644300+0000","last_active":"2023-10-10T20:21:27.644639+0000","last_peered":"2023-10-10T20:21:27.644639+0000","last_clean":"2023-10-10T20:21:27.644639+0000","last_became_active":"2023-10-10T20:21:27.643710+0000","last_became_peered":"2023-10-10T20:21:27.643710+0000","last_unstale":"2023-10-10T20:21:27.644639+0000","last_undegraded":"2023-10-10T20:21:27.644639+0000","last_fullsized":"2023-10-10T20:21:27.644639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:21.480558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.639454+0000","last_change":"2023-10-10T20:21:26.626193+0000","last_active":"2023-10-10T20:21:27.639454+0000","last_peered":"2023-10-10T20:21:27.639454+0000","last_clean":"2023-10-10T20:21:27.639454+0000","last_became_active":"2023-10-10T20:21:26.625852+0000","last_became_peered":"2023-10-10T20:21:26.625852+0000","last_unstale":"2023-10-10T20:21:27.639454+0000","last_undegraded":"2023-10-10T20:21:27.639454+0000","last_fullsized":"2023-10-10T20:21:27.639454+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:29.404480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.644441+0000","last_change":"2023-10-10T20:21:27.644375+0000","last_active":"2023-10-10T20:21:27.644441+0000","last_peered":"2023-10-10T20:21:27.644441+0000","last_clean":"2023-10-10T20:21:27.644441+0000","last_became_active":"2023-10-10T20:21:27.644144+0000","last_became_peered":"2023-10-10T20:21:27.644144+0000","last_unstale":"2023-10-10T20:21:27.644441+0000","last_undegraded":"2023-10-10T20:21:27.644441+0000","last_fullsized":"2023-10-10T20:21:27.644441+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:48.517658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.632107+0000","last_change":"2023-10-10T20:21:26.628452+0000","last_active":"2023-10-10T20:21:27.632107+0000","last_peered":"2023-10-10T20:21:27.632107+0000","last_clean":"2023-10-10T20:21:27.632107+0000","last_became_active":"2023-10-10T20:21:26.628193+0000","last_became_peered":"2023-10-10T20:21:26.628193+0000","last_unstale":"2023-10-10T20:21:27.632107+0000","last_undegraded":"2023-10-10T20:21:27.632107+0000","last_fullsized":"2023-10-10T20:21:27.632107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:56:16.104206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643340+0000","last_change":"2023-10-10T20:21:27.643216+0000","last_active":"2023-10-10T20:21:27.643340+0000","last_peered":"2023-10-10T20:21:27.643340+0000","last_clean":"2023-10-10T20:21:27.643340+0000","last_became_active":"2023-10-10T20:21:27.642703+0000","last_became_peered":"2023-10-10T20:21:27.642703+0000","last_unstale":"2023-10-10T20:21:27.643340+0000","last_undegraded":"2023-10-10T20:21:27.643340+0000","last_fullsized":"2023-10-10T20:21:27.643340+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:09.042467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.637026+0000","last_change":"2023-10-10T20:21:26.621728+0000","last_active":"2023-10-10T20:21:27.637026+0000","last_peered":"2023-10-10T20:21:27.637026+0000","last_clean":"2023-10-10T20:21:27.637026+0000","last_became_active":"2023-10-10T20:21:26.621493+0000","last_became_peered":"2023-10-10T20:21:26.621493+0000","last_unstale":"2023-10-10T20:21:27.637026+0000","last_undegraded":"2023-10-10T20:21:27.637026+0000","last_fullsized":"2023-10-10T20:21:27.637026+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:50.660723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.640642+0000","last_change":"2023-10-10T20:21:27.640385+0000","last_active":"2023-10-10T20:21:27.640642+0000","last_peered":"2023-10-10T20:21:27.640642+0000","last_clean":"2023-10-10T20:21:27.640642+0000","last_became_active":"2023-10-10T20:21:27.639816+0000","last_became_peered":"2023-10-10T20:21:27.639816+0000","last_unstale":"2023-10-10T20:21:27.640642+0000","last_undegraded":"2023-10-10T20:21:27.640642+0000","last_fullsized":"2023-10-10T20:21:27.640642+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:47.767852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.636604+0000","last_change":"2023-10-10T20:21:26.623339+0000","last_active":"2023-10-10T20:21:27.636604+0000","last_peered":"2023-10-10T20:21:27.636604+0000","last_clean":"2023-10-10T20:21:27.636604+0000","last_became_active":"2023-10-10T20:21:26.622789+0000","last_became_peered":"2023-10-10T20:21:26.622789+0000","last_unstale":"2023-10-10T20:21:27.636604+0000","last_undegraded":"2023-10-10T20:21:27.636604+0000","last_fullsized":"2023-10-10T20:21:27.636604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:33.771902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.643647+0000","last_change":"2023-10-10T20:21:27.641585+0000","last_active":"2023-10-10T20:21:27.643647+0000","last_peered":"2023-10-10T20:21:27.643647+0000","last_clean":"2023-10-10T20:21:27.643647+0000","last_became_active":"2023-10-10T20:21:27.641370+0000","last_became_peered":"2023-10-10T20:21:27.641370+0000","last_unstale":"2023-10-10T20:21:27.643647+0000","last_undegraded":"2023-10-10T20:21:27.643647+0000","last_fullsized":"2023-10-10T20:21:27.643647+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:26.612495+0000","last_clean_scrub_stamp":"2023-10-10T20:21:26.612495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:23.777715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.636039+0000","last_change":"2023-10-10T20:21:26.625874+0000","last_active":"2023-10-10T20:21:27.636039+0000","last_peered":"2023-10-10T20:21:27.636039+0000","last_clean":"2023-10-10T20:21:27.636039+0000","last_became_active":"2023-10-10T20:21:26.625638+0000","last_became_peered":"2023-10-10T20:21:26.625638+0000","last_unstale":"2023-10-10T20:21:27.636039+0000","last_undegraded":"2023-10-10T20:21:27.636039+0000","last_fullsized":"2023-10-10T20:21:27.636039+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:21:25.604260+0000","last_clean_scrub_stamp":"2023-10-10T20:21:25.604260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:35.606440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-10T20:21:27.636069+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:21:27.636069+0000","last_peered":"2023-10-10T20:21:27.636069+0000","last_clean":"2023-10-10T20:21:27.636069+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:21:27.636069+0000","last_undegraded":"2023-10-10T20:21:27.636069+0000","last_fullsized":"2023-10-10T20:21:27.636069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":907,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194404,"statfs":{"total":200000000,"available":199070389,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47199999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.60699999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65500000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.378},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.47299999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199660917,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50600000000000001,"15min":0.504},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70999999999999996,"5min":0.879,"15min":0.879},"last":0.495}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.64800000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.373}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.64100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.317},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.39100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199660827,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38700000000000001,"15min":0.39700000000000002},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48399999999999999,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.38,"15min":0.39200000000000002},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.443,"5min":1.647,"15min":1.647},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39600000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.44600000000000001,"5min":1.524,"15min":1.524},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.432,"15min":0.44700000000000001},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.434,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.27000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.29399999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.46200000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.5,"5min":1.673,"15min":1.673},"last":0.312}]},{"osd":4,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.69399999999999995,"5min":1.633,"15min":1.633},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":1.506,"15min":1.506},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.66900000000000004,"5min":1.696,"15min":1.696},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72099999999999997,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77500000000000002,"5min":1.585,"15min":1.585},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78500000000000003,"5min":1.478,"15min":1.478},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":1.722,"15min":1.722},"last":0.53700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":908,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194403,"statfs":{"total":200000000,"available":199069470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.46000000000000002,"15min":0.48499999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40999999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.47799999999999998,"15min":0.497},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.50600000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.49099999999999999,"15min":0.51300000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53800000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40300000000000002,"15min":0.41099999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.55500000000000005,"5min":1.256,"15min":1.256},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52400000000000002,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.58099999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.29299999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.204,"15min":1.204},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.97199999999999998,"15min":1.048},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71899999999999997,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82299999999999995,"5min":1.347,"15min":1.347},"last":0.5}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78400000000000003,"5min":1.173,"15min":1.173},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73199999999999998,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194982,"statfs":{"total":200000000,"available":199661872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.51100000000000001,"5min":1.389,"15min":1.389},"last":0.214},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40699999999999997,"15min":0.41499999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57299999999999995,"5min":1.256,"15min":1.256},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.60099999999999998,"5min":1.135,"15min":1.135},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63400000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.41199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55700000000000005,"5min":1.111,"15min":1.111},"last":0.318},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.61699999999999999,"5min":1.413,"15min":1.413},"last":0.255}]},{"osd":4,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.752},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.61399999999999999},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74399999999999999,"5min":1.083,"15min":1.083},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81699999999999995,"5min":1.363,"15min":1.363},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.224,"15min":1.224},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83999999999999997,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199660861,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.28499999999999998,"5min":0.29299999999999998,"15min":0.29399999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30199999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.47699999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46200000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36199999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63200000000000001,"5min":0.751,"15min":0.751},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51300000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.307},{"interface":"front","average":{"1min":0.378,"5min":0.379,"15min":0.379},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59299999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.753,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72299999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194979,"statfs":{"total":200000000,"available":199659381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50800000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.58599999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67000000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.623}]},{"osd":5,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46700000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71999999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56200000000000006}]},{"osd":6,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.753,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.51300000000000001},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74099999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69099999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199660293,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.498,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81599999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.502},"min":{"1min":0.36599999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68500000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.432,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60699999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69999999999999996,"5min":0.754,"15min":0.754},"last":0.34799999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.34399999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65600000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.439}]},{"osd":7,"last update":"Tue Oct 10 20:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54100000000000004}]}]}],"pool_statfs":[]}} 2023-10-10T20:21:36.568 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:21:36.568 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:21:36.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.721+0000 7f4367049700 1 Processor -- start 2023-10-10T20:21:36.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.721+0000 7f4367049700 1 -- start start 2023-10-10T20:21:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4367049700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4360159ec0 0x7f43600af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4367049700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43600afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4367049700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600b0790 con 0x7f436006a5a0 2023-10-10T20:21:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4367049700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43600ce1f0 con 0x7f4360159ec0 2023-10-10T20:21:36.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4367049700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b0b20 con 0x7f436015a290 2023-10-10T20:21:36.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4360159ec0 0x7f43600af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4360159ec0 0x7f43600af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42652/0 (socket says 172.21.15.148:42652) 2023-10-10T20:21:36.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 -- 172.21.15.148:0/1402629288 learned_addr learned my addr 172.21.15.148:0/1402629288 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:36.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f435ffff700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43600afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 -- 172.21.15.148:0/1402629288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 msgr2=0x7f43600afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43600afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 -- 172.21.15.148:0/1402629288 >> v1:172.21.15.161:6789/0 conn(0x7f436006a5a0 legacy=0x7f436015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 -- 172.21.15.148:0/1402629288 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600b04b0 con 0x7f4360159ec0 2023-10-10T20:21:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.722+0000 7f4364de5700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f43600af940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f43540099b0 tx=0x7f4354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a04e1f7c66252 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f435f7fe700 1 -- 172.21.15.148:0/1402629288 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354009ea0 con 0x7f4360159ec0 2023-10-10T20:21:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f435f7fe700 1 -- 172.21.15.148:0/1402629288 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435400b860 con 0x7f4360159ec0 2023-10-10T20:21:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f435f7fe700 1 -- 172.21.15.148:0/1402629288 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400bc00 con 0x7f4360159ec0 2023-10-10T20:21:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 -- 172.21.15.148:0/1402629288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 msgr2=0x7f43600af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f43600af940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f43540099b0 tx=0x7f4354009250 comp rx=0 tx=0).stop 2023-10-10T20:21:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 -- 172.21.15.148:0/1402629288 shutdown_connections 2023-10-10T20:21:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f43600af940 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 --2- 172.21.15.148:0/1402629288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43600afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 -- 172.21.15.148:0/1402629288 >> 172.21.15.148:0/1402629288 conn(0x7f43600d1880 msgr2=0x7f43600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 -- 172.21.15.148:0/1402629288 shutdown_connections 2023-10-10T20:21:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.723+0000 7f4367049700 1 -- 172.21.15.148:0/1402629288 wait complete. 2023-10-10T20:21:36.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.724+0000 7f4367049700 1 Processor -- start 2023-10-10T20:21:36.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 -- start start 2023-10-10T20:21:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f436006a5a0 0x7f4360158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43601601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436015ad90 con 0x7f4360159ec0 2023-10-10T20:21:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436015b070 con 0x7f436006a5a0 2023-10-10T20:21:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4367049700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436015b350 con 0x7f436015a290 2023-10-10T20:21:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f435ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f435ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42668/0 (socket says 172.21.15.148:42668) 2023-10-10T20:21:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f435ffff700 1 -- 172.21.15.148:0/1075793839 learned_addr learned my addr 172.21.15.148:0/1075793839 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f4364de5700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f436006a5a0 0x7f4360158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.725+0000 7f43655e6700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43601601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 -- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 msgr2=0x7f43601601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43601601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 -- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f436006a5a0 msgr2=0x7f4360158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f436006a5a0 0x7f4360158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 -- 172.21.15.148:0/1075793839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540094f0 con 0x7f4360159ec0 2023-10-10T20:21:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f43655e6700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43601601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435ffff700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f434c00ec80 tx=0x7f434c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00c850 con 0x7f4360159ec0 2023-10-10T20:21:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c00c9b0 con 0x7f4360159ec0 2023-10-10T20:21:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436015b630 con 0x7f4360159ec0 2023-10-10T20:21:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00cd50 con 0x7f4360159ec0 2023-10-10T20:21:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.726+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4360159040 con 0x7f4360159ec0 2023-10-10T20:21:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.727+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436004c3c0 con 0x7f4360159ec0 2023-10-10T20:21:36.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.727+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f434c013680 con 0x7f4360159ec0 2023-10-10T20:21:36.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.728+0000 7f435cff9700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 0x7f43440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:36.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.728+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f434c0847f0 con 0x7f4360159ec0 2023-10-10T20:21:36.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.728+0000 7f4364de5700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 0x7f43440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:36.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.728+0000 7f4364de5700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 0x7f43440693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43540099b0 tx=0x7f4354017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:36.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.732+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f434c051a60 con 0x7f4360159ec0 2023-10-10T20:21:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.988+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43600ceab0 con 0x7f4360159ec0 2023-10-10T20:21:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.991+0000 7f435cff9700 1 -- 172.21.15.148:0/1075793839 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f434c055080 con 0x7f4360159ec0 2023-10-10T20:21:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 msgr2=0x7f43440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 0x7f43440693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43540099b0 tx=0x7f4354017000 comp rx=0 tx=0).stop 2023-10-10T20:21:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 msgr2=0x7f436015be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f434c00ec80 tx=0x7f434c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 shutdown_connections 2023-10-10T20:21:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4344066f30 0x7f43440693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f436015a290 0x7f43601601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f436006a5a0 0x7f4360158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 --2- 172.21.15.148:0/1075793839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4360159ec0 0x7f436015be30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:36.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 >> 172.21.15.148:0/1075793839 conn(0x7f43600d1880 msgr2=0x7f43600b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:36.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.993+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 shutdown_connections 2023-10-10T20:21:36.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:36.994+0000 7f4367049700 1 -- 172.21.15.148:0/1075793839 wait complete. 2023-10-10T20:21:36.998 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:21:37.007 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:37.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.121+0000 7fa2bd5a3700 1 Processor -- start 2023-10-10T20:21:37.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.121+0000 7fa2bd5a3700 1 -- start start 2023-10-10T20:21:37.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2bd5a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa2b815a620 0x7fa2b815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2bd5a3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b815cbb0 con 0x7fa2b8157870 2023-10-10T20:21:37.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2bd5a3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b815bfa0 con 0x7fa2b815d580 2023-10-10T20:21:37.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2bd5a3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b815bdb0 con 0x7fa2b815a620 2023-10-10T20:21:37.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2b6ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa2b8157870 0x7fa2b815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45688/0 (socket says 172.21.15.148:45688) 2023-10-10T20:21:37.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2b6ffd700 1 -- 172.21.15.148:0/2143679517 learned_addr learned my addr 172.21.15.148:0/2143679517 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:37.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.122+0000 7fa2b67fc700 1 --2- 172.21.15.148:0/2143679517 >> v2:172.21.15.161:3300/0 conn(0x7fa2b815a620 0x7fa2b815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923851507 0 0) 0x7fa2b815cbb0 con 0x7fa2b8157870 2023-10-10T20:21:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a0003570 con 0x7fa2b8157870 2023-10-10T20:21:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b67fc700 1 -- 172.21.15.148:0/2143679517 >> v1:172.21.15.161:6789/0 conn(0x7fa2b815d580 legacy=0x7fa2b8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b67fc700 1 -- 172.21.15.148:0/2143679517 >> v1:172.21.15.148:6789/0 conn(0x7fa2b8157870 legacy=0x7fa2b815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2765389158 0 0) 0x7fa2a0003570 con 0x7fa2b8157870 2023-10-10T20:21:37.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b67fc700 1 -- 172.21.15.148:0/2143679517 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b815c280 con 0x7fa2b815a620 2023-10-10T20:21:37.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b67fc700 1 --2- 172.21.15.148:0/2143679517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b815a620 0x7fa2b815d040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0098e0 tx=0x7fa2ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9062ff8665621007 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80017f0 con 0x7fa2b815a620 2023-10-10T20:21:37.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac011420 con 0x7fa2b815a620 2023-10-10T20:21:37.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.123+0000 7fa2b5ffb700 1 -- 172.21.15.148:0/2143679517 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0117c0 con 0x7fa2b815a620 2023-10-10T20:21:37.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/2143679517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b815a620 msgr2=0x7fa2b815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/2143679517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b815a620 0x7fa2b815d040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0098e0 tx=0x7fa2ac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/2143679517 shutdown_connections 2023-10-10T20:21:37.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/2143679517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b815a620 0x7fa2b815d040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/2143679517 >> 172.21.15.148:0/2143679517 conn(0x7fa2b8068ce0 msgr2=0x7fa2b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/2143679517 shutdown_connections 2023-10-10T20:21:37.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.124+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/2143679517 wait complete. 2023-10-10T20:21:37.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.125+0000 7fa2bd5a3700 1 Processor -- start 2023-10-10T20:21:37.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.125+0000 7fa2bd5a3700 1 -- start start 2023-10-10T20:21:37.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b8157870 0x7fa2b814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2b815d580 0x7fa2b816b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b816bae0 con 0x7fa2b815a620 2023-10-10T20:21:37.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8155ed0 con 0x7fa2b8157870 2023-10-10T20:21:37.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80d1660 con 0x7fa2b815d580 2023-10-10T20:21:37.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42674/0 (socket says 172.21.15.148:42674) 2023-10-10T20:21:37.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 -- 172.21.15.148:0/1748093932 learned_addr learned my addr 172.21.15.148:0/1748093932 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b6ffd700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b8157870 0x7fa2b814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b77fe700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2b815d580 0x7fa2b816b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 -- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2b815d580 msgr2=0x7fa2b816b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2b815d580 0x7fa2b816b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 -- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b8157870 msgr2=0x7fa2b814b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b8157870 0x7fa2b814b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 -- 172.21.15.148:0/1748093932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac009420 con 0x7fa2b815a620 2023-10-10T20:21:37.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2b67fc700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0091e0 tx=0x7fa2ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac01cb30 con 0x7fa2b815a620 2023-10-10T20:21:37.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac01ccb0 con 0x7fa2b815a620 2023-10-10T20:21:37.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b8153d10 con 0x7fa2b815a620 2023-10-10T20:21:37.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac01a6a0 con 0x7fa2b815a620 2023-10-10T20:21:37.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.126+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b81542d0 con 0x7fa2b815a620 2023-10-10T20:21:37.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.127+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b804c3c0 con 0x7fa2b815a620 2023-10-10T20:21:37.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.128+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2ac01aa50 con 0x7fa2b815a620 2023-10-10T20:21:37.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.128+0000 7fa29effd700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 0x7fa2a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.128+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2ac0860f0 con 0x7fa2b815a620 2023-10-10T20:21:37.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.128+0000 7fa2b6ffd700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 0x7fa2a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.129+0000 7fa2b6ffd700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 0x7fa2a40693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fd0 tx=0x7fa2a8007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.133+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2ac053360 con 0x7fa2b815a620 2023-10-10T20:21:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.355+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2b815c560 con 0x7fa2b815a620 2023-10-10T20:21:37.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.356+0000 7fa29effd700 1 -- 172.21.15.148:0/1748093932 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fa2ac056980 con 0x7fa2b815a620 2023-10-10T20:21:37.358 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:37.358 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:21:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 msgr2=0x7fa2a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 0x7fa2a40693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8007fd0 tx=0x7fa2a8007c70 comp rx=0 tx=0).stop 2023-10-10T20:21:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 msgr2=0x7fa2b816b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0091e0 tx=0x7fa2ac016000 comp rx=0 tx=0).stop 2023-10-10T20:21:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 shutdown_connections 2023-10-10T20:21:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa2a4066f30 0x7fa2a40693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2b815d580 0x7fa2b816b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2b8157870 0x7fa2b814b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 --2- 172.21.15.148:0/1748093932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2b815a620 0x7fa2b816b060 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 >> 172.21.15.148:0/1748093932 conn(0x7fa2b8068ce0 msgr2=0x7fa2b8159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 shutdown_connections 2023-10-10T20:21:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.358+0000 7fa2bd5a3700 1 -- 172.21.15.148:0/1748093932 wait complete. 2023-10-10T20:21:37.363 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:21:37.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:21:26.994181+0000', 'modified': '2023-10-10T20:21:30.668566+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5727': {'gid': 5727, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/1666843566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1666843566}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1666843566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:21:37.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:21:37.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-10T20:21:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.890+0000 7fd8b4a6f700 1 Processor -- start 2023-10-10T20:21:37.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.890+0000 7fd8b4a6f700 1 -- start start 2023-10-10T20:21:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8b4a6f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8b00cdfa0 0x7fd8b00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8b4a6f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 0x7fd8b0159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8b4a6f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b015db30 con 0x7fd8b00693b0 2023-10-10T20:21:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8b4a6f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b015ddd0 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8b4a6f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b015e0b0 con 0x7fd8b0068f10 2023-10-10T20:21:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8b00cdfa0 0x7fd8b00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8affff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd8b00693b0 0x7fd8b0159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45702/0 (socket says 172.21.15.148:45702) 2023-10-10T20:21:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8affff700 1 -- 172.21.15.148:0/3304825197 learned_addr learned my addr 172.21.15.148:0/3304825197 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/3304825197 >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 0x7fd8b0159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3304825197 >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 msgr2=0x7fd8b0159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/3304825197 >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 0x7fd8b0159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3304825197 >> v1:172.21.15.148:6789/0 conn(0x7fd8b00693b0 legacy=0x7fd8b0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3304825197 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b015e390 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8ae7fc700 1 -- 172.21.15.148:0/3304825197 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627716089 0 0) 0x7fd8b015db30 con 0x7fd8b00693b0 2023-10-10T20:21:37.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/3304825197 >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 0x7fd8b0159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:37.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.891+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/3304825197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b00ce370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00cb330 tx=0x7fd8a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65826b6c94ad433b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8ae7fc700 1 -- 172.21.15.148:0/3304825197 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000e070 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8ae7fc700 1 -- 172.21.15.148:0/3304825197 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000bc30 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8ae7fc700 1 -- 172.21.15.148:0/3304825197 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0012430 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/3304825197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00cdfa0 msgr2=0x7fd8b00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/3304825197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b00ce370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00cb330 tx=0x7fd8a0009780 comp rx=0 tx=0).stop 2023-10-10T20:21:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/3304825197 shutdown_connections 2023-10-10T20:21:37.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/3304825197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b00ce370 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/3304825197 >> v2:172.21.15.161:3300/0 conn(0x7fd8b0068f10 0x7fd8b0159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/3304825197 >> 172.21.15.148:0/3304825197 conn(0x7fd8b006c700 msgr2=0x7fd8b0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/3304825197 shutdown_connections 2023-10-10T20:21:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.892+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/3304825197 wait complete. 2023-10-10T20:21:37.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.893+0000 7fd8b4a6f700 1 Processor -- start 2023-10-10T20:21:37.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 -- start start 2023-10-10T20:21:37.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8b0068f10 0x7fd8b00b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00b4840 con 0x7fd8b00693b0 2023-10-10T20:21:37.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b015c990 con 0x7fd8b00cdfa0 2023-10-10T20:21:37.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8b4a6f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00ca820 con 0x7fd8b0068f10 2023-10-10T20:21:37.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42692/0 (socket says 172.21.15.148:42692) 2023-10-10T20:21:37.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.894+0000 7fd8aeffd700 1 -- 172.21.15.148:0/1000277739 learned_addr learned my addr 172.21.15.148:0/1000277739 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 -- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8b0068f10 msgr2=0x7fd8b00b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8b0068f10 0x7fd8b00b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 -- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b00cdfa0 msgr2=0x7fd8b015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 -- 172.21.15.148:0/1000277739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009a20 con 0x7fd8b00693b0 2023-10-10T20:21:37.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d860 tx=0x7fd8a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8977fe700 1 -- 172.21.15.148:0/1000277739 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40125f0 con 0x7fd8b00693b0 2023-10-10T20:21:37.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8977fe700 1 -- 172.21.15.148:0/1000277739 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4012750 con 0x7fd8b00693b0 2023-10-10T20:21:37.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b015bca0 con 0x7fd8b00693b0 2023-10-10T20:21:37.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8977fe700 1 -- 172.21.15.148:0/1000277739 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4012ad0 con 0x7fd8b00693b0 2023-10-10T20:21:37.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.895+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b015c620 con 0x7fd8b00693b0 2023-10-10T20:21:37.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.896+0000 7fd8977fe700 1 -- 172.21.15.148:0/1000277739 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8a4014430 con 0x7fd8b00693b0 2023-10-10T20:21:37.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.897+0000 7fd8977fe700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.897+0000 7fd8977fe700 1 -- 172.21.15.148:0/1000277739 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8a4084980 con 0x7fd8b00693b0 2023-10-10T20:21:37.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.897+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.897+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 0x7fd89c0693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00cb330 tx=0x7fd8a0019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.899+0000 7fd8befcab80 1 Processor -- start 2023-10-10T20:21:37.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.899+0000 7fd8befcab80 1 -- start start 2023-10-10T20:21:37.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.899+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.899+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd45f0 0x564c8ce21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8cdd6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8befcab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd2da70 con 0x564c8ce21480 2023-10-10T20:21:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd2b270 con 0x564c8cdd6b80 2023-10-10T20:21:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd27060 con 0x564c8cdd45f0 2023-10-10T20:21:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42706/0 (socket says 172.21.15.148:42706) 2023-10-10T20:21:37.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2360367587 learned_addr learned my addr 172.21.15.148:0/2360367587 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd45f0 msgr2=0x564c8ce21850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd45f0 0x564c8ce21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 msgr2=0x564c8cdd6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8cdd6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2360367587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8cbc1270 con 0x564c8ce21480 2023-10-10T20:21:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd8b006cf80 tx=0x7fd8a0020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c346bd57a55cacc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd894ff9700 1 -- 172.21.15.148:0/2360367587 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a001e420 con 0x564c8ce21480 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd894ff9700 1 -- 172.21.15.148:0/2360367587 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0022070 con 0x564c8ce21480 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.900+0000 7fd894ff9700 1 -- 172.21.15.148:0/2360367587 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a001e790 con 0x564c8ce21480 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 -- 172.21.15.148:0/2360367587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 msgr2=0x564c8ce007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd8b006cf80 tx=0x7fd8a0020070 comp rx=0 tx=0).stop 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 -- 172.21.15.148:0/2360367587 shutdown_connections 2023-10-10T20:21:37.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8cdd6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8ce21480 0x564c8ce007d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2360367587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd45f0 0x564c8ce21850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 -- 172.21.15.148:0/2360367587 >> 172.21.15.148:0/2360367587 conn(0x564c8cdd80c0 msgr2=0x564c8cdd88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 -- 172.21.15.148:0/2360367587 shutdown_connections 2023-10-10T20:21:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.901+0000 7fd8befcab80 1 -- 172.21.15.148:0/2360367587 wait complete. 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 -- start start 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 Processor -- start 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce20220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8ccd2890 con 0x564c8cdd45f0 2023-10-10T20:21:37.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8ccedea0 con 0x564c8ce21480 2023-10-10T20:21:37.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c8cba9e30 con 0x564c8cdd6b80 2023-10-10T20:21:37.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8befcab80 10 client.0 _ll_register_callbacks cb 0x564c8cd9a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:21:37.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42714/0 (socket says 172.21.15.148:42714) 2023-10-10T20:21:37.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.902+0000 7fd8af7fe700 1 -- 172.21.15.148:0/1588640316 learned_addr learned my addr 172.21.15.148:0/1588640316 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce20220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8affff700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 -- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 msgr2=0x564c8ce20220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce20220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 -- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 msgr2=0x564c8ce20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 -- 172.21.15.148:0/1588640316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b015e390 con 0x564c8cdd45f0 2023-10-10T20:21:37.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce20220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:37.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd8a001efc0 tx=0x7fd8a002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16134c42419cfb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:21:37.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0033620 con 0x564c8cdd45f0 2023-10-10T20:21:37.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8a0024070 con 0x564c8cdd45f0 2023-10-10T20:21:37.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c8cdcbcd0 con 0x564c8cdd45f0 2023-10-10T20:21:37.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a0033990 con 0x564c8cdd45f0 2023-10-10T20:21:37.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 10 client.5901 fetch_fsmap learned FSMap version 32 2023-10-10T20:21:37.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x564c8cb63fe0 con 0x564c8cdd45f0 2023-10-10T20:21:37.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c8cdcbcd0 con 0x564c8cdd45f0 2023-10-10T20:21:37.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd8a0033af0 con 0x564c8cdd45f0 2023-10-10T20:21:37.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 10 client.5901 fetch_fsmap finished waiting for FSMap version 32 2023-10-10T20:21:37.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 10 client.5901 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:21:37.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 20 client.5901 populate_metadata read hostname 'smithi148' 2023-10-10T20:21:37.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8cdfaff0 0x564c8cdff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 4 client.5901 mds_command: new command op to 5727 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:21:37.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.903+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 --> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c8cda8aa0 con 0x564c8cdfaff0 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8cdfaff0 0x564c8cdff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8cdfaff0 0x564c8cdff8e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105c5c5dfdf71f9b server_cookie=1e9bdf7c1c6bb08a in_seq=0 out_seq=0 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd88dffb700 10 client.5901 ms_handle_connect on v2:172.21.15.148:6835/1666843566 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd88dffb700 1 -- 172.21.15.148:0/1588640316 <== mds.0 v2:172.21.15.148:6835/1666843566 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564c8cda8aa0 con 0x564c8cdfaff0 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd88dffb700 10 client.5901 handle_command_reply: tid=0 2023-10-10T20:21:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd8befcab80 1 client.5901 shutdown 2023-10-10T20:21:37.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 msgr2=0x564c8ce1d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.904+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd8a001efc0 tx=0x7fd8a002a000 comp rx=0 tx=0).stop 2023-10-10T20:21:37.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 shutdown_connections 2023-10-10T20:21:37.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8cdfaff0 0x564c8cdff8e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd88dffb700 0 client.5901 ms_handle_reset on v2:172.21.15.148:6835/1666843566 2023-10-10T20:21:37.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8ce1d1d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 --2- 172.21.15.148:0/1588640316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce20220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 >> 172.21.15.148:0/1588640316 conn(0x564c8cdd80c0 msgr2=0x564c8cdd3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 shutdown_connections 2023-10-10T20:21:37.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 1 -- 172.21.15.148:0/1588640316 wait complete. 2023-10-10T20:21:37.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.905+0000 7fd8befcab80 20 client.5901 trim_cache size 0 max 16384 2023-10-10T20:21:37.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.921+0000 7fd8befcab80 1 Processor -- start 2023-10-10T20:21:37.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.921+0000 7fd8befcab80 1 -- start start 2023-10-10T20:21:37.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8ce1e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 0x564c8ce1e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cdd69f0 con 0x564c8cdd45f0 2023-10-10T20:21:37.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8ce1fb70 con 0x564c8cdd6b80 2023-10-10T20:21:37.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c8ce2ff10 con 0x564c8ce21480 2023-10-10T20:21:37.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42728/0 (socket says 172.21.15.148:42728) 2023-10-10T20:21:37.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2915208956 learned_addr learned my addr 172.21.15.148:0/2915208956 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8ce1e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8affff700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 0x564c8ce1e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 msgr2=0x564c8ce1e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 0x564c8ce1e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 msgr2=0x564c8ce1e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8ce1e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 -- 172.21.15.148:0/2915208956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8ccf5780 con 0x564c8cdd45f0 2023-10-10T20:21:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8affff700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 0x564c8ce1e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:21:37.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.922+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000beb0 tx=0x7fd8a0037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bc14be06fc9776 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8957fa700 1 -- 172.21.15.148:0/2915208956 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000b750 con 0x564c8cdd45f0 2023-10-10T20:21:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8957fa700 1 -- 172.21.15.148:0/2915208956 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a001e630 con 0x564c8cdd45f0 2023-10-10T20:21:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8957fa700 1 -- 172.21.15.148:0/2915208956 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0033bc0 con 0x564c8cdd45f0 2023-10-10T20:21:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 -- 172.21.15.148:0/2915208956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 msgr2=0x564c8cdffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000beb0 tx=0x7fd8a0037000 comp rx=0 tx=0).stop 2023-10-10T20:21:37.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 -- 172.21.15.148:0/2915208956 shutdown_connections 2023-10-10T20:21:37.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8ce21480 0x564c8ce1e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdffab0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 --2- 172.21.15.148:0/2915208956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8cdd6b80 0x564c8ce1e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 -- 172.21.15.148:0/2915208956 >> 172.21.15.148:0/2915208956 conn(0x564c8cdd80c0 msgr2=0x564c8ce1cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 -- 172.21.15.148:0/2915208956 shutdown_connections 2023-10-10T20:21:37.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.923+0000 7fd8befcab80 1 -- 172.21.15.148:0/2915208956 wait complete. 2023-10-10T20:21:37.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 -- start start 2023-10-10T20:21:37.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 Processor -- start 2023-10-10T20:21:37.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce2f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8ce00a50 con 0x564c8cdd45f0 2023-10-10T20:21:37.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8ce83700 con 0x564c8ce21480 2023-10-10T20:21:37.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c8ce00bc0 con 0x564c8cdd6b80 2023-10-10T20:21:37.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8befcab80 10 client.0 _ll_register_callbacks cb 0x564c8cd9a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:21:37.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8af7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42744/0 (socket says 172.21.15.148:42744) 2023-10-10T20:21:37.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.924+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3048103649 learned_addr learned my addr 172.21.15.148:0/3048103649 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:37.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8affff700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce2f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 msgr2=0x564c8ce1c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce1c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 msgr2=0x564c8ce2f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce2f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 -- 172.21.15.148:0/3048103649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c8cb63fe0 con 0x564c8cdd45f0 2023-10-10T20:21:37.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8affff700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce2f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:37.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8af7fe700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0049040 tx=0x7fd8a0035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c15c17e40856ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:37.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:21:37.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0033ba0 con 0x564c8cdd45f0 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8a0024050 con 0x564c8cdd45f0 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8a0022070 con 0x564c8cdd45f0 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a003b610 con 0x564c8cdd45f0 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 10 client.5907 fetch_fsmap learned FSMap version 32 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x564c8ccdc470 con 0x564c8cdd45f0 2023-10-10T20:21:37.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0022070 con 0x564c8cdd45f0 2023-10-10T20:21:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd8a0059070 con 0x564c8cdd45f0 2023-10-10T20:21:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 10 client.5907 fetch_fsmap finished waiting for FSMap version 32 2023-10-10T20:21:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 10 client.5907 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:21:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 20 client.5907 populate_metadata read hostname 'smithi148' 2023-10-10T20:21:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8ce2fb90 0x564c8ce82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 4 client.5907 mds_command: new command op to 5727 tid=0[{"prefix": "status"}] 2023-10-10T20:21:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.925+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 --> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c8ccc9e00 con 0x564c8ce2fb90 2023-10-10T20:21:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8ce2fb90 0x564c8ce82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd88dffb700 10 client.5907 ms_handle_connect on v2:172.21.15.148:6835/1666843566 2023-10-10T20:21:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8aeffd700 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8ce2fb90 0x564c8ce82890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bc615e4e47e92d server_cookie=25c440c0d7dde32e in_seq=0 out_seq=0 2023-10-10T20:21:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd88dffb700 1 -- 172.21.15.148:0/3048103649 <== mds.0 v2:172.21.15.148:6835/1666843566 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564c8ccc9e00 con 0x564c8ce2fb90 2023-10-10T20:21:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd88dffb700 10 client.5907 handle_command_reply: tid=0 2023-10-10T20:21:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 client.5907 shutdown 2023-10-10T20:21:37.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 msgr2=0x564c8cdfbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0049040 tx=0x7fd8a0035040 comp rx=0 tx=0).stop 2023-10-10T20:21:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 shutdown_connections 2023-10-10T20:21:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:6835/1666843566,v1:172.21.15.148:6837/1666843566] conn(0x564c8ce2fb90 0x564c8ce82890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564c8ce21480 0x564c8ce2f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd88dffb700 0 client.5907 ms_handle_reset on v2:172.21.15.148:6835/1666843566 2023-10-10T20:21:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c8cdd45f0 0x564c8cdfbf40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 --2- 172.21.15.148:0/3048103649 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x564c8cdd6b80 0x564c8ce1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 >> 172.21.15.148:0/3048103649 conn(0x564c8cdd80c0 msgr2=0x564c8cdffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 shutdown_connections 2023-10-10T20:21:37.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 1 -- 172.21.15.148:0/3048103649 wait complete. 2023-10-10T20:21:37.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.926+0000 7fd8befcab80 20 client.5907 trim_cache size 0 max 16384 2023-10-10T20:21:37.955 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:21:37.955 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:21:37.955 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:21:37.956 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5727, 2023-10-10T20:21:37.956 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:21:37.956 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:21:37.956 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:21:37.956 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 10.279885645, 2023-10-10T20:21:37.957 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 32, 2023-10-10T20:21:37.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 66, 2023-10-10T20:21:37.957 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 66, 2023-10-10T20:21:37.957 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 16.270845983000001 2023-10-10T20:21:37.957 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:21:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 msgr2=0x7fd89c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 0x7fd89c0693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00cb330 tx=0x7fd8a0019000 comp rx=0 tx=0).stop 2023-10-10T20:21:37.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 msgr2=0x7fd8b00ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d860 tx=0x7fd8a4008040 comp rx=0 tx=0).stop 2023-10-10T20:21:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 shutdown_connections 2023-10-10T20:21:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b00cdfa0 0x7fd8b015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8b0068f10 0x7fd8b00b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 --2- 172.21.15.148:0/1000277739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8b00693b0 0x7fd8b00ca2e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 >> 172.21.15.148:0/1000277739 conn(0x7fd8b006c700 msgr2=0x7fd8b0153430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 shutdown_connections 2023-10-10T20:21:37.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:37.927+0000 7fd8b4a6f700 1 -- 172.21.15.148:0/1000277739 wait complete. 2023-10-10T20:21:37.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:21:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.073+0000 7fbd77744700 1 Processor -- start 2023-10-10T20:21:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.074+0000 7fbd77744700 1 -- start start 2023-10-10T20:21:38.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd77744700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd70158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd77744700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7015cbb0 con 0x7fbd70157870 2023-10-10T20:21:38.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd77744700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7015bfa0 con 0x7fbd7015a620 2023-10-10T20:21:38.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd77744700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7015bdb0 con 0x7fbd70157c40 2023-10-10T20:21:38.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd754e0700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbd70157870 0x7fbd7015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45708/0 (socket says 172.21.15.148:45708) 2023-10-10T20:21:38.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd754e0700 1 -- 172.21.15.148:0/2359253482 learned_addr learned my addr 172.21.15.148:0/2359253482 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:21:38.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435954068 0 0) 0x7fbd7015cbb0 con 0x7fbd70157870 2023-10-10T20:21:38.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd70158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5c003570 con 0x7fbd70157870 2023-10-10T20:21:38.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176857437 0 0) 0x7fbd7015bfa0 con 0x7fbd7015a620 2023-10-10T20:21:38.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7015cbb0 con 0x7fbd7015a620 2023-10-10T20:21:38.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311245871 0 0) 0x7fbd5c003570 con 0x7fbd70157870 2023-10-10T20:21:38.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 msgr2=0x7fbd70158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:38.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd70158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 >> v1:172.21.15.161:6789/0 conn(0x7fbd7015a620 legacy=0x7fbd7015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7015c280 con 0x7fbd70157870 2023-10-10T20:21:38.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fbd6c001470 con 0x7fbd70157870 2023-10-10T20:21:38.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 >> v1:172.21.15.148:6789/0 conn(0x7fbd70157870 legacy=0x7fbd7015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5c003c20 0x7fbd5c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5c004d50 0x7fbd5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00bbf0 con 0x7fbd5c003c20 2023-10-10T20:21:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00bd30 con 0x7fbd5c004d50 2023-10-10T20:21:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00bea0 con 0x7fbd70157c40 2023-10-10T20:21:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.075+0000 7fbd754e0700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5c003c20 0x7fbd5c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd74cdf700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5c004d50 0x7fbd5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 -- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 msgr2=0x7fbd5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd70157c40 0x7fbd5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 -- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5c004d50 msgr2=0x7fbd5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5c004d50 0x7fbd5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 -- 172.21.15.148:0/2359253482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c003570 con 0x7fbd5c003c20 2023-10-10T20:21:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd5c004d50 0x7fbd5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:38.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd754e0700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5c003c20 0x7fbd5c004010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007fd0 tx=0x7fbd6c001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fab9551f51be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:38.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00cae0 con 0x7fbd5c003c20 2023-10-10T20:21:38.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00cc60 con 0x7fbd5c003c20 2023-10-10T20:21:38.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.076+0000 7fbd67fff700 1 -- 172.21.15.148:0/2359253482 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005920 con 0x7fbd5c003c20 2023-10-10T20:21:38.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 -- 172.21.15.148:0/2359253482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5c003c20 msgr2=0x7fbd5c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 --2- 172.21.15.148:0/2359253482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd5c003c20 0x7fbd5c004010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c007fd0 tx=0x7fbd6c001d40 comp rx=0 tx=0).stop 2023-10-10T20:21:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd75ce1700 1 -- 172.21.15.148:0/2359253482 reap_dead start 2023-10-10T20:21:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 -- 172.21.15.148:0/2359253482 shutdown_connections 2023-10-10T20:21:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 -- 172.21.15.148:0/2359253482 >> 172.21.15.148:0/2359253482 conn(0x7fbd70068ce0 msgr2=0x7fbd700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 -- 172.21.15.148:0/2359253482 shutdown_connections 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.077+0000 7fbd77744700 1 -- 172.21.15.148:0/2359253482 wait complete. 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.078+0000 7fbd77744700 1 Processor -- start 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.078+0000 7fbd77744700 1 -- start start 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd5c003c20 0x7fbd70153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70151160 0x7fbd701515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70151b10 con 0x7fbd700b1820 2023-10-10T20:21:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70154520 con 0x7fbd70151160 2023-10-10T20:21:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700b2550 con 0x7fbd5c003c20 2023-10-10T20:21:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42766/0 (socket says 172.21.15.148:42766) 2023-10-10T20:21:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 -- 172.21.15.148:0/299677186 learned_addr learned my addr 172.21.15.148:0/299677186 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd754e0700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd5c003c20 0x7fbd70153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd74cdf700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70151160 0x7fbd701515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 -- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd5c003c20 msgr2=0x7fbd70153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd5c003c20 0x7fbd70153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 -- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70151160 msgr2=0x7fbd701515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70151160 0x7fbd701515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 -- 172.21.15.148:0/299677186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700b2830 con 0x7fbd700b1820 2023-10-10T20:21:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd75ce1700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbd700b3830 tx=0x7fbd6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:38.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801b070 con 0x7fbd700b1820 2023-10-10T20:21:38.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6800f040 con 0x7fbd700b1820 2023-10-10T20:21:38.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd700b2b10 con 0x7fbd700b1820 2023-10-10T20:21:38.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800e050 con 0x7fbd700b1820 2023-10-10T20:21:38.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.079+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd700b21b0 con 0x7fbd700b1820 2023-10-10T20:21:38.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.080+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7004c3c0 con 0x7fbd700b1820 2023-10-10T20:21:38.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.081+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd68003020 con 0x7fbd700b1820 2023-10-10T20:21:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.081+0000 7fbd657fa700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 0x7fbd540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.081+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd68084940 con 0x7fbd700b1820 2023-10-10T20:21:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.081+0000 7fbd754e0700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 0x7fbd540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.082+0000 7fbd754e0700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 0x7fbd540693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c920 tx=0x7fbd6c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:38.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.085+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd68051bb0 con 0x7fbd700b1820 2023-10-10T20:21:38.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.309+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd7015c560 con 0x7fbd700b1820 2023-10-10T20:21:38.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.309+0000 7fbd657fa700 1 -- 172.21.15.148:0/299677186 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd680551d0 con 0x7fbd700b1820 2023-10-10T20:21:38.312 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:21:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 msgr2=0x7fbd540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 0x7fbd540693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c920 tx=0x7fbd6c015000 comp rx=0 tx=0).stop 2023-10-10T20:21:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 msgr2=0x7fbd700b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbd700b3830 tx=0x7fbd6800b040 comp rx=0 tx=0).stop 2023-10-10T20:21:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 shutdown_connections 2023-10-10T20:21:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbd54066f30 0x7fbd540693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd700b1820 0x7fbd700b1c70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd5c003c20 0x7fbd70153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 --2- 172.21.15.148:0/299677186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd70151160 0x7fbd701515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 >> 172.21.15.148:0/299677186 conn(0x7fbd70068ce0 msgr2=0x7fbd700b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 shutdown_connections 2023-10-10T20:21:38.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:38.311+0000 7fbd77744700 1 -- 172.21.15.148:0/299677186 wait complete. 2023-10-10T20:21:38.325 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:21:38.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:21:38.326 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:21:38.326 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:21:38.326 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:21:38.327 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:21:38.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:21:38.327 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:21:38.328 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:21:38.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.329 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:21:38.344 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:21:38.344 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:21:38.345 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:21:38.345 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.345 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:21:38.345 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.345 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:38.346 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:38.346 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:21:38.346 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:38.347 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:38.347 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:21:38.347 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:38.348 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:38.348 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:21:38.348 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:38.348 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:21:38.348 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42413sec preferred_lft 42413sec 2023-10-10T20:21:38.349 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:21:38.349 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.349 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:21:38.349 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:38.349 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:21:38.350 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.350 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:21:38.350 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.350 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:21:38.350 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:21:38.351 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:21:38.351 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.351 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:21:38.351 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:21:38.351 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:21:38.352 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:38.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.353 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:21:38.407 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:21:38.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.409 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:21:38.464 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:21:38.465 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:21:38.465 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:21:38.465 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:21:38.465 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:21:38.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.466 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:21:38.467 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:21:38.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:21:38.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:21:38.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:21:38.468 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:21:38.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.565 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:21:38.580 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:21:38.580 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:21:38.580 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:21:38.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:21:38.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:21:38.644 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:21:38.699 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:21:38.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:21:38.699 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:21:38.753 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:21:38.753 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:21:38.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:38.833 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:38.833 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:38.852 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:38.852 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:38.852 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:21:38.852 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:21:38.853 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:21:38.853 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:21:38.853 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:21:38.853 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:21:38.874 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:38.874 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:38.874 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:21:38.875 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:21:38.875 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:21:38.875 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:21:38.875 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:21:38.875 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:21:38.876 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:21:38.876 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:21:38.876 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:21:38.877 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:21:38.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:38.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:21:38.942 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:21:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.095+0000 7f0fdf454700 1 Processor -- start 2023-10-10T20:21:39.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.095+0000 7f0fdf454700 1 -- start start 2023-10-10T20:21:39.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdf454700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fd815a620 0x7f0fd815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdf454700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0fd815d580 0x7f0fd8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdf454700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd815cbb0 con 0x7f0fd8157870 2023-10-10T20:21:39.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdf454700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd815bdb0 con 0x7f0fd815a620 2023-10-10T20:21:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdf454700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd815bfa0 con 0x7f0fd815d580 2023-10-10T20:21:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fd815a620 0x7f0fd815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fd815a620 0x7f0fd815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42782/0 (socket says 172.21.15.148:42782) 2023-10-10T20:21:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 -- 172.21.15.148:0/1451605240 learned_addr learned my addr 172.21.15.148:0/1451605240 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:39.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdd9f1700 1 --2- 172.21.15.148:0/1451605240 >> v2:172.21.15.161:3300/0 conn(0x7f0fd815d580 0x7f0fd8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 -- 172.21.15.148:0/1451605240 >> v2:172.21.15.161:3300/0 conn(0x7f0fd815d580 msgr2=0x7f0fd8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 --2- 172.21.15.148:0/1451605240 >> v2:172.21.15.161:3300/0 conn(0x7f0fd815d580 0x7f0fd8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 -- 172.21.15.148:0/1451605240 >> v1:172.21.15.161:6789/0 conn(0x7f0fd8157870 legacy=0x7f0fd815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.096+0000 7f0fdc9ef700 1 -- 172.21.15.148:0/1451605240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd815c280 con 0x7f0fd815a620 2023-10-10T20:21:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdc9ef700 1 --2- 172.21.15.148:0/1451605240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815a620 0x7f0fd815d040 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0fd806b4e0 tx=0x7f0fc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7202198e0b6f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fcffff700 1 -- 172.21.15.148:0/1451605240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8009e70 con 0x7f0fd815a620 2023-10-10T20:21:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fcffff700 1 -- 172.21.15.148:0/1451605240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800b860 con 0x7f0fd815a620 2023-10-10T20:21:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fcffff700 1 -- 172.21.15.148:0/1451605240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800bc00 con 0x7f0fd815a620 2023-10-10T20:21:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 -- 172.21.15.148:0/1451605240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815a620 msgr2=0x7f0fd815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 --2- 172.21.15.148:0/1451605240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815a620 0x7f0fd815d040 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0fd806b4e0 tx=0x7f0fc8009250 comp rx=0 tx=0).stop 2023-10-10T20:21:39.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 -- 172.21.15.148:0/1451605240 shutdown_connections 2023-10-10T20:21:39.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 --2- 172.21.15.148:0/1451605240 >> v2:172.21.15.161:3300/0 conn(0x7f0fd815d580 0x7f0fd8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 --2- 172.21.15.148:0/1451605240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815a620 0x7f0fd815d040 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 -- 172.21.15.148:0/1451605240 >> 172.21.15.148:0/1451605240 conn(0x7f0fd8068ce0 msgr2=0x7f0fd80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:39.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 -- 172.21.15.148:0/1451605240 shutdown_connections 2023-10-10T20:21:39.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.097+0000 7f0fdf454700 1 -- 172.21.15.148:0/1451605240 wait complete. 2023-10-10T20:21:39.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.098+0000 7f0fdf454700 1 Processor -- start 2023-10-10T20:21:39.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 -- start start 2023-10-10T20:21:39.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 0x7f0fd814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fd815a620 0x7f0fd80cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80c7ca0 con 0x7f0fd815d580 2023-10-10T20:21:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80ca960 con 0x7f0fd8157870 2023-10-10T20:21:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.099+0000 7f0fdf454700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd814bf30 con 0x7f0fd815a620 2023-10-10T20:21:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42788/0 (socket says 172.21.15.148:42788) 2023-10-10T20:21:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 -- 172.21.15.148:0/579285086 learned_addr learned my addr 172.21.15.148:0/579285086 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:39.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdc9ef700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fd815a620 0x7f0fd80cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd1f0700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 0x7f0fd814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 -- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fd815a620 msgr2=0x7f0fd80cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fd815a620 0x7f0fd80cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 -- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 msgr2=0x7f0fd814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 0x7f0fd814b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 -- 172.21.15.148:0/579285086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80094f0 con 0x7f0fd815d580 2023-10-10T20:21:39.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd1f0700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 0x7f0fd814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:21:39.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdd9f1700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e930 tx=0x7f0fd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:39.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0013680 con 0x7f0fd815d580 2023-10-10T20:21:39.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0013800 con 0x7f0fd815d580 2023-10-10T20:21:39.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0013bc0 con 0x7f0fd815d580 2023-10-10T20:21:39.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd80c7eb0 con 0x7f0fd815d580 2023-10-10T20:21:39.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.100+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd80c8470 con 0x7f0fd815d580 2023-10-10T20:21:39.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.101+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd815a2d0 con 0x7f0fd815d580 2023-10-10T20:21:39.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.101+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fd00123d0 con 0x7f0fd815d580 2023-10-10T20:21:39.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.102+0000 7f0fcd7fa700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 0x7f0fbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:39.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.102+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0fd0084750 con 0x7f0fd815d580 2023-10-10T20:21:39.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.102+0000 7f0fdd1f0700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 0x7f0fbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:39.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.102+0000 7f0fdd1f0700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 0x7f0fbc0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007fd0 tx=0x7f0fd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:39.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.106+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fd0051920 con 0x7f0fd815d580 2023-10-10T20:21:39.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.331+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0fd804c3c0 con 0x7f0fd815d580 2023-10-10T20:21:39.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.331+0000 7f0fcd7fa700 1 -- 172.21.15.148:0/579285086 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0fd0054f40 con 0x7f0fd815d580 2023-10-10T20:21:39.334 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:21:39.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 msgr2=0x7f0fbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 0x7f0fbc0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007fd0 tx=0x7f0fd4009040 comp rx=0 tx=0).stop 2023-10-10T20:21:39.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 msgr2=0x7f0fd80b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:39.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e930 tx=0x7f0fd00091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 shutdown_connections 2023-10-10T20:21:39.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0fbc066f30 0x7f0fbc0693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fd815d580 0x7f0fd80b2ad0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fd8157870 0x7f0fd814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 --2- 172.21.15.148:0/579285086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fd815a620 0x7f0fd80cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:39.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 >> 172.21.15.148:0/579285086 conn(0x7f0fd8068ce0 msgr2=0x7f0fd81592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:39.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 shutdown_connections 2023-10-10T20:21:39.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:39.333+0000 7f0fdf454700 1 -- 172.21.15.148:0/579285086 wait complete. 2023-10-10T20:21:39.347 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:21:39.347 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:21:39.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:21:39.348 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:21:39.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:21:39.349 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:21:39.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:21:39.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:21:39.350 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:21:39.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.351 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:21:39.372 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:21:39.373 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:21:39.373 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:21:39.373 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.373 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:21:39.373 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.374 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:39.374 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:39.374 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:21:39.374 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:39.374 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:39.375 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:21:39.375 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:21:39.375 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:39.376 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:21:39.376 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:39.376 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:21:39.376 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-10-10T20:21:39.376 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:21:39.377 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.377 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:21:39.377 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:21:39.377 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:21:39.377 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.378 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:21:39.378 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.378 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:21:39.378 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:21:39.378 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:21:39.379 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.379 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:21:39.379 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:21:39.379 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:21:39.379 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:21:39.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.380 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:21:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:21:39.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.442 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:21:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:21:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:21:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:21:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:21:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:21:39.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.502 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:21:39.503 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:21:39.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:21:39.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:21:39.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:21:39.504 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:21:39.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.607 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:21:39.626 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:21:39.627 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:21:39.627 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:21:39.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.628 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:21:39.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:21:39.689 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:21:39.744 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:21:39.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:21:39.745 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:21:39.798 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:21:39.798 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:21:39.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:39.877 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:39.878 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:39.896 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:39.896 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:39.897 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:21:39.897 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:21:39.897 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:21:39.897 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:21:39.897 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:21:39.898 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:21:39.918 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:21:39.918 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:21:39.918 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:21:39.919 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:21:39.920 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:21:39.920 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:21:39.921 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:21:39.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:21:39.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:21:39.985 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:21:40.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.099+0000 7f1a34263700 1 Processor -- start 2023-10-10T20:21:40.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.100+0000 7f1a34263700 1 -- start start 2023-10-10T20:21:40.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.100+0000 7f1a34263700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1a2c157870 0x7f1a2c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.100+0000 7f1a34263700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a2c157c40 0x7f1a2c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.100+0000 7f1a34263700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c15cbb0 con 0x7f1a2c15a620 2023-10-10T20:21:40.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a34263700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c15bfa0 con 0x7f1a2c157c40 2023-10-10T20:21:40.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a34263700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c15c280 con 0x7f1a2c157870 2023-10-10T20:21:40.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a2c157c40 0x7f1a2c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a2c157c40 0x7f1a2c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42794/0 (socket says 172.21.15.148:42794) 2023-10-10T20:21:40.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 -- 172.21.15.148:0/206079065 learned_addr learned my addr 172.21.15.148:0/206079065 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:40.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a31fff700 1 --2- 172.21.15.148:0/206079065 >> v2:172.21.15.161:3300/0 conn(0x7f1a2c157870 0x7f1a2c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 -- 172.21.15.148:0/206079065 >> v2:172.21.15.161:3300/0 conn(0x7f1a2c157870 msgr2=0x7f1a2c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:21:40.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 --2- 172.21.15.148:0/206079065 >> v2:172.21.15.161:3300/0 conn(0x7f1a2c157870 0x7f1a2c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 -- 172.21.15.148:0/206079065 >> v1:172.21.15.161:6789/0 conn(0x7f1a2c15a620 legacy=0x7f1a2c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 -- 172.21.15.148:0/206079065 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c15c560 con 0x7f1a2c157c40 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a32800700 1 --2- 172.21.15.148:0/206079065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c157c40 0x7f1a2c158110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c069d80 tx=0x7f1a24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f9fda2bd0f7a5a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a30ffd700 1 -- 172.21.15.148:0/206079065 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400e070 con 0x7f1a2c157c40 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a30ffd700 1 -- 172.21.15.148:0/206079065 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2400ba20 con 0x7f1a2c157c40 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.101+0000 7f1a30ffd700 1 -- 172.21.15.148:0/206079065 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240092e0 con 0x7f1a2c157c40 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 -- 172.21.15.148:0/206079065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c157c40 msgr2=0x7f1a2c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 --2- 172.21.15.148:0/206079065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c157c40 0x7f1a2c158110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c069d80 tx=0x7f1a24009840 comp rx=0 tx=0).stop 2023-10-10T20:21:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 -- 172.21.15.148:0/206079065 shutdown_connections 2023-10-10T20:21:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 --2- 172.21.15.148:0/206079065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c157c40 0x7f1a2c158110 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 --2- 172.21.15.148:0/206079065 >> v2:172.21.15.161:3300/0 conn(0x7f1a2c157870 0x7f1a2c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 -- 172.21.15.148:0/206079065 >> 172.21.15.148:0/206079065 conn(0x7f1a2c068ce0 msgr2=0x7f1a2c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 -- 172.21.15.148:0/206079065 shutdown_connections 2023-10-10T20:21:40.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.102+0000 7f1a34263700 1 -- 172.21.15.148:0/206079065 wait complete. 2023-10-10T20:21:40.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.103+0000 7f1a34263700 1 Processor -- start 2023-10-10T20:21:40.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.103+0000 7f1a34263700 1 -- start start 2023-10-10T20:21:40.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a2c157870 0x7f1a2c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2c157c40 0x7f1a2c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c14e830 con 0x7f1a2c15a620 2023-10-10T20:21:40.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b1b50 con 0x7f1a2c157870 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a34263700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0b1e30 con 0x7f1a2c157c40 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42808/0 (socket says 172.21.15.148:42808) 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 -- 172.21.15.148:0/1564445457 learned_addr learned my addr 172.21.15.148:0/1564445457 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a31fff700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a2c157870 0x7f1a2c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a317fe700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2c157c40 0x7f1a2c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 -- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2c157c40 msgr2=0x7f1a2c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2c157c40 0x7f1a2c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 -- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a2c157870 msgr2=0x7f1a2c14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a2c157870 0x7f1a2c14b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.104+0000 7f1a32800700 1 -- 172.21.15.148:0/1564445457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009ae0 con 0x7f1a2c15a620 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a32800700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009880 tx=0x7f1a24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24008a80 con 0x7f1a2c15a620 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24008be0 con 0x7f1a2c15a620 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0b2110 con 0x7f1a2c15a620 2023-10-10T20:21:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24026460 con 0x7f1a2c15a620 2023-10-10T20:21:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.105+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c14e450 con 0x7f1a2c15a620 2023-10-10T20:21:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.106+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c14e9e0 con 0x7f1a2c15a620 2023-10-10T20:21:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.106+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a24003040 con 0x7f1a2c15a620 2023-10-10T20:21:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.107+0000 7f1a21ffb700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 0x7f1a100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:21:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.107+0000 7f1a31fff700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 0x7f1a100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:21:40.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.107+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a240863a0 con 0x7f1a2c15a620 2023-10-10T20:21:40.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.107+0000 7f1a31fff700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 0x7f1a100693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009db0 tx=0x7f1a280091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:21:40.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.111+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a2c14e9e0 con 0x7f1a2c15a620 2023-10-10T20:21:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.333+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a2c0d1760 con 0x7f1a2c15a620 2023-10-10T20:21:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.334+0000 7f1a21ffb700 1 -- 172.21.15.148:0/1564445457 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f1a240536c0 con 0x7f1a2c15a620 2023-10-10T20:21:40.337 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:21:40.337 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:21:40.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 msgr2=0x7f1a100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 0x7f1a100693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009db0 tx=0x7f1a280091a0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 msgr2=0x7f1a2c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:21:40.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009880 tx=0x7f1a24018000 comp rx=0 tx=0).stop 2023-10-10T20:21:40.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 shutdown_connections 2023-10-10T20:21:40.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a10066f30 0x7f1a100693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a2c15a620 0x7f1a2c152c50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a2c157870 0x7f1a2c14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 --2- 172.21.15.148:0/1564445457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2c157c40 0x7f1a2c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:21:40.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 >> 172.21.15.148:0/1564445457 conn(0x7f1a2c068ce0 msgr2=0x7f1a2c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:21:40.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 shutdown_connections 2023-10-10T20:21:40.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:21:40.336+0000 7f1a34263700 1 -- 172.21.15.148:0/1564445457 wait complete. 2023-10-10T20:21:40.341 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:21:40.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-10T20:21:40.485 INFO:teuthology.orchestra.run.smithi148.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":5727,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.826992148,"mdsmap_epoch":32,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":18.817955571999999} 2023-10-10T20:21:40.486 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 5727, "mdsmap_epoch": 32, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 12.826992148, "state": "up:active", "uptime": 18.817955572, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:21:40.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:21:40.615 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:21:40.615 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:21:40.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-10T20:21:40.740 INFO:teuthology.orchestra.run.smithi148.stdout:{"mon_tick_interval":"5"} 2023-10-10T20:21:40.740 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-10T20:21:40.741 INFO:tasks.cephfs.test_full:pool capacity 557102656, 531MB should be enough to fill it 2023-10-10T20:21:40.741 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:21:40.742 DEBUG:teuthology.orchestra.run.smithi148:> import time 2023-10-10T20:21:40.742 DEBUG:teuthology.orchestra.run.smithi148:> import datetime 2023-10-10T20:21:40.742 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-10-10T20:21:40.743 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:21:40.743 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.744 DEBUG:teuthology.orchestra.run.smithi148:> # Write some buffered data through before going full, all should be well 2023-10-10T20:21:40.744 DEBUG:teuthology.orchestra.run.smithi148:> print("writing some data through which we expect to succeed") 2023-10-10T20:21:40.744 DEBUG:teuthology.orchestra.run.smithi148:> bytes = 0 2023-10-10T20:21:40.745 DEBUG:teuthology.orchestra.run.smithi148:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-10T20:21:40.745 DEBUG:teuthology.orchestra.run.smithi148:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-10T20:21:40.745 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-10T20:21:40.746 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:21:40.746 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-10T20:21:40.746 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.747 DEBUG:teuthology.orchestra.run.smithi148:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-10T20:21:40.747 DEBUG:teuthology.orchestra.run.smithi148:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-10T20:21:40.747 DEBUG:teuthology.orchestra.run.smithi148:> # buffered IO, the error should always only appear from fsync and not 2023-10-10T20:21:40.748 DEBUG:teuthology.orchestra.run.smithi148:> # from write 2023-10-10T20:21:40.748 DEBUG:teuthology.orchestra.run.smithi148:> full = False 2023-10-10T20:21:40.748 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.749 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(531 * 0.9)): 2023-10-10T20:21:40.749 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:21:40.749 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-10T20:21:40.750 DEBUG:teuthology.orchestra.run.smithi148:> print("done writing {0} bytes".format(bytes)) 2023-10-10T20:21:40.750 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.750 DEBUG:teuthology.orchestra.run.smithi148:> # OK, now we should sneak in under the full condition 2023-10-10T20:21:40.751 DEBUG:teuthology.orchestra.run.smithi148:> # due to the time it takes the OSDs to report to the 2023-10-10T20:21:40.751 DEBUG:teuthology.orchestra.run.smithi148:> # mons, and get a successful fsync on our full-making data 2023-10-10T20:21:40.751 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:21:40.752 DEBUG:teuthology.orchestra.run.smithi148:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-10T20:21:40.752 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.752 DEBUG:teuthology.orchestra.run.smithi148:> # buffered write, add more dirty data to the buffer 2023-10-10T20:21:40.753 DEBUG:teuthology.orchestra.run.smithi148:> print("starting buffered write") 2023-10-10T20:21:40.753 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:21:40.754 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(531 * 0.2)): 2023-10-10T20:21:40.754 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:21:40.754 DEBUG:teuthology.orchestra.run.smithi148:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-10T20:21:40.755 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(15.0) 2023-10-10T20:21:40.755 DEBUG:teuthology.orchestra.run.smithi148:> except OSError: 2023-10-10T20:21:40.755 DEBUG:teuthology.orchestra.run.smithi148:> pass; 2023-10-10T20:21:40.756 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.756 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-10T20:21:40.756 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.757 DEBUG:teuthology.orchestra.run.smithi148:> # Wait long enough for a background flush that should fail 2023-10-10T20:21:40.757 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(30) 2023-10-10T20:21:40.757 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.758 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2023-10-10T20:21:40.758 DEBUG:teuthology.orchestra.run.smithi148:> # ...and check that the failed background flush is reflected in fclose 2023-10-10T20:21:40.758 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:21:40.759 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:21:40.759 DEBUG:teuthology.orchestra.run.smithi148:> except OSError: 2023-10-10T20:21:40.759 DEBUG:teuthology.orchestra.run.smithi148:> print("close() returned an error as expected") 2023-10-10T20:21:40.760 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:21:40.760 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("close() failed to raise error") 2023-10-10T20:21:40.760 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:21:40.761 DEBUG:teuthology.orchestra.run.smithi148:> # The kernel cephfs client does not raise errors on fclose 2023-10-10T20:21:40.761 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:21:40.761 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:21:40.762 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-10T20:21:40.762 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:21:41.722 INFO:teuthology.orchestra.run.smithi148.stdout:writing some data through which we expect to succeed 2023-10-10T20:21:41.723 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-10T20:21:41.723 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-10T20:21:41.723 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-10T20:21:41.723 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-10T20:21:41.724 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-10T20:21:41.724 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-10T20:21:41.724 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-10T20:21:41.724 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-10T20:21:41.724 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-10T20:21:41.725 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-10T20:21:41.725 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-10T20:21:41.725 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-10T20:21:41.725 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-10T20:21:41.725 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-10T20:21:41.726 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-10T20:21:41.726 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-10T20:21:41.726 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-10T20:21:41.726 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-10T20:21:41.726 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-10T20:21:41.727 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-10T20:21:41.727 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-10T20:21:41.727 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-10T20:21:41.727 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-10T20:21:41.727 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-10T20:21:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-10T20:21:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-10T20:21:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-10T20:21:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-10T20:21:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-10T20:21:41.729 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-10-10T20:21:41.729 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-10-10T20:21:41.729 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-10-10T20:21:41.729 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-10-10T20:21:41.730 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-10-10T20:21:41.730 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-10-10T20:21:41.730 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-10-10T20:21:41.730 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-10-10T20:21:41.730 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-10-10T20:21:41.731 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-10-10T20:21:41.731 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-10-10T20:21:41.731 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-10-10T20:21:41.731 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-10-10T20:21:41.731 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-10-10T20:21:41.732 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-10-10T20:21:41.732 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-10-10T20:21:41.732 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-10-10T20:21:41.732 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-10-10T20:21:41.732 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-10-10T20:21:41.733 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-10-10T20:21:41.733 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-10-10T20:21:41.733 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-10-10T20:21:41.733 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-10-10T20:21:41.733 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-10-10T20:21:41.734 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-10-10T20:21:41.734 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-10-10T20:21:41.734 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-10-10T20:21:41.734 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-10-10T20:21:41.734 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-10-10T20:21:41.735 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-10-10T20:21:41.736 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-10-10T20:21:41.736 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-10-10T20:21:41.736 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-10-10T20:21:41.736 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-10-10T20:21:41.736 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-10-10T20:21:41.737 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-10-10T20:21:41.737 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-10-10T20:21:41.737 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-10-10T20:21:41.737 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-10-10T20:21:41.737 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-10-10T20:21:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-10-10T20:21:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-10-10T20:21:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-10-10T20:21:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-10-10T20:21:41.738 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-10-10T20:21:41.739 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-10-10T20:21:41.739 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-10-10T20:21:41.739 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-10-10T20:21:41.740 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-10-10T20:21:41.742 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-10-10T20:21:41.742 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-10-10T20:21:41.743 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-10-10T20:21:41.743 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-10-10T20:21:41.743 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-10-10T20:21:41.743 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-10-10T20:21:41.743 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-10-10T20:21:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-10-10T20:21:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-10-10T20:21:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-10-10T20:21:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-10-10T20:21:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-10-10T20:21:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-10-10T20:21:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-10-10T20:21:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-10-10T20:21:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-10-10T20:21:41.745 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-10-10T20:21:41.746 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-10-10T20:21:41.746 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-10-10T20:21:41.746 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-10-10T20:21:41.746 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-10-10T20:21:41.746 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-10-10T20:21:41.747 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-10-10T20:21:41.748 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-10-10T20:21:41.748 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-10-10T20:21:41.748 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-10-10T20:21:41.748 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-10-10T20:21:41.749 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-10-10T20:21:41.749 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-10-10T20:21:41.749 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-10-10T20:21:41.749 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-10-10T20:21:41.749 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-10-10T20:21:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-10-10T20:21:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-10-10T20:21:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-10-10T20:21:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-10-10T20:21:41.750 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-10-10T20:21:41.751 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-10-10T20:21:41.751 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-10-10T20:21:41.751 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-10-10T20:21:41.751 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-10-10T20:21:41.751 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-10-10T20:21:41.752 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-10-10T20:21:41.753 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-10-10T20:21:41.753 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-10-10T20:21:41.753 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-10-10T20:21:41.753 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-10-10T20:21:41.753 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-10-10T20:21:41.754 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-10-10T20:21:41.754 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-10-10T20:21:41.754 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-10-10T20:21:41.754 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-10-10T20:21:41.754 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-10-10T20:21:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-10-10T20:21:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-10-10T20:21:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-10-10T20:21:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-10-10T20:21:41.755 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-10-10T20:21:41.756 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-10-10T20:21:41.756 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-10-10T20:21:41.756 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-10-10T20:21:41.756 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-10-10T20:21:41.771 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-10-10T20:21:41.772 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-10-10T20:21:41.772 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-10-10T20:21:41.772 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-10-10T20:21:41.772 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-10-10T20:21:41.772 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-10-10T20:21:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-10-10T20:21:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-10-10T20:21:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-10-10T20:21:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-10-10T20:21:41.773 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-10-10T20:21:41.774 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-10-10T20:21:41.774 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-10-10T20:21:41.774 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-10-10T20:21:41.774 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-10-10T20:21:41.774 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-10-10T20:21:41.775 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-10-10T20:21:41.775 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-10-10T20:21:41.775 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-10-10T20:21:41.775 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-10-10T20:21:41.775 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-10-10T20:21:41.776 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-10-10T20:21:41.776 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-10-10T20:21:41.776 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-10-10T20:21:41.776 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-10-10T20:21:41.776 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-10-10T20:21:41.777 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-10-10T20:21:41.777 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-10-10T20:21:41.777 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-10-10T20:21:41.777 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-10-10T20:21:41.777 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-10-10T20:21:41.778 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-10-10T20:21:41.778 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-10-10T20:21:41.778 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-10-10T20:21:41.778 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-10-10T20:21:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-10-10T20:21:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-10-10T20:21:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-10-10T20:21:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-10-10T20:21:41.779 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-10-10T20:21:41.780 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-10-10T20:21:41.780 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-10-10T20:21:41.780 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-10-10T20:21:41.780 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-10-10T20:21:41.780 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-10-10T20:21:41.781 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-10-10T20:21:41.782 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-10-10T20:21:41.782 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-10-10T20:21:41.782 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-10-10T20:21:41.782 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-10-10T20:21:41.782 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-10-10T20:21:41.783 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-10-10T20:21:41.784 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-10-10T20:21:41.784 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-10-10T20:21:41.784 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-10-10T20:21:41.784 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-10-10T20:21:41.784 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-10-10T20:21:41.785 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-10-10T20:21:41.785 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-10-10T20:21:41.785 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-10-10T20:21:41.785 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-10-10T20:21:41.785 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-10-10T20:21:41.786 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-10-10T20:21:41.787 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-10-10T20:21:41.787 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-10-10T20:21:41.787 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-10-10T20:21:41.787 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-10-10T20:21:41.787 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-10-10T20:21:41.788 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-10-10T20:21:41.788 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-10-10T20:21:41.788 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-10-10T20:21:41.788 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-10-10T20:21:41.788 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-10-10T20:21:41.789 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-10-10T20:21:41.790 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-10-10T20:21:41.790 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-10-10T20:21:41.790 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-10-10T20:21:41.790 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-10-10T20:21:41.790 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-10-10T20:21:41.791 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-10-10T20:21:41.792 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-10-10T20:21:41.792 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-10-10T20:21:41.792 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-10-10T20:21:41.792 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-10-10T20:21:41.792 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-10-10T20:21:41.793 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-10-10T20:21:41.793 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-10-10T20:21:41.793 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-10-10T20:21:41.793 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-10-10T20:21:41.793 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-10-10T20:21:41.794 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-10-10T20:21:41.795 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-10-10T20:21:41.795 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-10-10T20:21:41.795 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-10-10T20:21:41.795 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-10-10T20:21:41.795 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-10-10T20:21:41.796 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-10-10T20:21:41.796 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-10-10T20:21:41.796 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-10-10T20:21:41.796 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-10-10T20:21:41.796 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-10-10T20:21:41.797 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-10-10T20:21:41.797 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-10-10T20:21:41.797 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-10-10T20:21:41.797 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-10-10T20:21:41.797 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-10-10T20:21:41.798 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-10-10T20:21:41.798 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-10-10T20:21:41.798 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-10-10T20:21:41.798 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-10-10T20:21:41.798 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-10-10T20:21:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-10-10T20:21:41.800 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-10-10T20:21:41.800 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-10-10T20:21:41.800 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-10-10T20:21:41.800 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-10-10T20:21:41.800 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-10-10T20:21:41.801 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-10-10T20:21:41.801 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-10-10T20:21:41.801 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-10-10T20:21:41.819 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-10-10T20:21:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-10-10T20:21:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-10-10T20:21:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-10-10T20:21:41.820 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-10-10T20:21:41.821 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-10-10T20:21:41.822 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-10-10T20:21:41.822 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-10-10T20:21:41.822 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-10-10T20:21:41.822 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-10-10T20:21:41.822 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-10-10T20:21:41.823 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-10-10T20:21:41.823 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-10-10T20:21:41.823 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-10-10T20:21:41.823 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-10-10T20:21:41.823 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-10-10T20:21:41.824 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-10-10T20:21:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-10-10T20:21:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-10-10T20:21:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-10-10T20:21:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-10-10T20:21:41.825 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-10-10T20:21:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-10-10T20:21:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-10-10T20:21:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-10-10T20:21:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-10-10T20:21:41.826 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-10-10T20:21:41.827 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-10-10T20:21:41.828 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-10-10T20:21:41.828 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-10-10T20:21:41.828 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-10-10T20:21:41.828 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-10-10T20:21:41.828 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-10-10T20:21:41.829 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-10-10T20:21:41.829 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-10-10T20:21:41.829 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-10-10T20:21:41.829 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-10-10T20:21:41.829 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-10-10T20:21:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-10-10T20:21:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-10-10T20:21:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-10-10T20:21:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-10-10T20:21:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-10-10T20:21:41.831 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-10-10T20:21:41.831 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-10-10T20:21:41.831 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-10-10T20:21:41.831 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-10-10T20:21:41.831 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-10-10T20:21:41.832 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-10-10T20:21:41.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-10-10T20:21:41.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-10-10T20:21:41.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-10-10T20:21:41.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-10-10T20:21:41.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-10-10T20:21:41.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-10-10T20:21:41.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-10-10T20:21:41.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-10-10T20:21:41.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-10-10T20:21:41.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-10-10T20:21:41.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-10-10T20:21:41.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-10-10T20:21:41.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-10-10T20:21:41.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-10-10T20:21:41.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-10-10T20:21:41.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-10-10T20:21:41.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-10-10T20:21:41.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-10-10T20:21:41.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-10-10T20:21:41.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-10-10T20:21:41.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-10-10T20:21:41.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-10-10T20:21:41.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-10-10T20:21:41.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-10-10T20:21:41.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-10-10T20:21:41.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-10-10T20:21:41.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-10-10T20:21:41.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-10-10T20:21:41.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-10-10T20:21:41.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-10-10T20:21:41.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-10-10T20:21:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-10-10T20:21:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-10-10T20:21:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-10-10T20:21:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-10-10T20:21:41.840 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-10-10T20:21:41.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-10-10T20:21:41.842 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-10-10T20:21:41.842 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-10-10T20:21:41.842 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-10-10T20:21:41.842 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-10-10T20:21:41.842 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-10-10T20:21:41.843 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-10-10T20:21:41.843 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-10-10T20:21:41.843 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-10-10T20:21:41.843 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-10-10T20:21:41.843 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-10-10T20:21:41.844 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-10-10T20:21:41.845 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-10-10T20:21:41.845 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-10-10T20:21:41.845 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-10-10T20:21:41.845 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-10-10T20:21:41.845 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-10-10T20:21:41.846 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-10-10T20:21:41.846 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-10-10T20:21:41.846 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-10-10T20:21:41.846 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-10-10T20:21:41.846 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-10-10T20:21:41.847 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-10-10T20:21:41.847 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-10-10T20:21:41.847 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-10-10T20:21:41.847 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-10-10T20:21:41.847 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-10-10T20:21:41.848 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-10-10T20:21:41.849 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-10-10T20:21:41.849 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-10-10T20:21:43.696 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:21:43.695+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-10T20:21:50.489 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:21:50.487+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-10T20:21:52.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:21:52.620 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:22:22.685 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:22:22.688 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:22:52.748 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:22:52.751 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:22:57.276 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-10-10T20:22:57.277 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-10-10T20:22:57.277 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-10-10T20:22:57.277 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-10-10T20:22:57.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-10-10T20:22:57.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-10-10T20:22:57.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-10-10T20:22:57.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-10-10T20:22:57.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-10-10T20:22:57.279 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-10-10T20:22:57.279 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-10-10T20:22:57.279 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-10-10T20:22:57.279 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-10-10T20:22:57.279 INFO:teuthology.orchestra.run.smithi148.stdout:done writing 500695040 bytes 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:successfully fsync'ed prior to getting full state reported 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:starting buffered write 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:22:57.280 INFO:teuthology.orchestra.run.smithi148.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-10T20:22:57.321 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:22:57.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:22:57.371 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:22:57.410 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:22:57.411 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:22:57.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:22:57.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:22:57.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:22:57.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:22:57.454 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:22:57.509 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:22:57.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:22:57.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:22:57.551 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:22:57.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:22:57.552 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:22:57.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:22:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.755+0000 7f9d1ce7d700 1 Processor -- start 2023-10-10T20:22:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.755+0000 7f9d1ce7d700 1 -- start start 2023-10-10T20:22:57.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1ce7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 0x7f9d18162a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:57.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1ce7d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180c8ce0 con 0x7f9d1806f2c0 2023-10-10T20:22:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1ce7d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d18069e20 con 0x7f9d1806f690 2023-10-10T20:22:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1ce7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1806a6c0 con 0x7f9d1815e4a0 2023-10-10T20:22:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9d1806f2c0 0x7f9d18069640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37186/0 (socket says 172.21.15.148:37186) 2023-10-10T20:22:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d1659c700 1 -- 172.21.15.148:0/2371581793 learned_addr learned my addr 172.21.15.148:0/2371581793 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:22:57.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.756+0000 7f9d16d9d700 1 --2- 172.21.15.148:0/2371581793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 0x7f9d18162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:57.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d15d9b700 1 -- 172.21.15.148:0/2371581793 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247120174 0 0) 0x7f9d180c8ce0 con 0x7f9d1806f2c0 2023-10-10T20:22:57.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d15d9b700 1 -- 172.21.15.148:0/2371581793 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d04003570 con 0x7f9d1806f2c0 2023-10-10T20:22:57.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d16d9d700 1 -- 172.21.15.148:0/2371581793 >> v1:172.21.15.161:6789/0 conn(0x7f9d1806f690 legacy=0x7f9d1815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:57.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d16d9d700 1 -- 172.21.15.148:0/2371581793 >> v1:172.21.15.148:6789/0 conn(0x7f9d1806f2c0 legacy=0x7f9d18069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:57.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d16d9d700 1 -- 172.21.15.148:0/2371581793 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1806a100 con 0x7f9d1815e4a0 2023-10-10T20:22:57.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.757+0000 7f9d16d9d700 1 --2- 172.21.15.148:0/2371581793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 0x7f9d18162a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009910 tx=0x7f9d100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc9796eb561890 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:57.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.758+0000 7f9d15d9b700 1 -- 172.21.15.148:0/2371581793 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000b680 con 0x7f9d1815e4a0 2023-10-10T20:22:57.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.758+0000 7f9d15d9b700 1 -- 172.21.15.148:0/2371581793 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000b7e0 con 0x7f9d1815e4a0 2023-10-10T20:22:57.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.758+0000 7f9d15d9b700 1 -- 172.21.15.148:0/2371581793 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000bb80 con 0x7f9d1815e4a0 2023-10-10T20:22:57.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.759+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/2371581793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 msgr2=0x7f9d18162a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:57.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.759+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/2371581793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 0x7f9d18162a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009910 tx=0x7f9d100091a0 comp rx=0 tx=0).stop 2023-10-10T20:22:57.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.759+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/2371581793 shutdown_connections 2023-10-10T20:22:57.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.759+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/2371581793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1815e4a0 0x7f9d18162a20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:57.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.759+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/2371581793 >> 172.21.15.148:0/2371581793 conn(0x7f9d18067660 msgr2=0x7f9d18067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:57.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.760+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/2371581793 shutdown_connections 2023-10-10T20:22:57.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.760+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/2371581793 wait complete. 2023-10-10T20:22:57.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.762+0000 7f9d1ce7d700 1 Processor -- start 2023-10-10T20:22:57.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.762+0000 7f9d1ce7d700 1 -- start start 2023-10-10T20:22:57.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:57.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 0x7f9d180b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:57.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d1815e4a0 0x7f9d180b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:57.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c06f0 con 0x7f9d1815e4a0 2023-10-10T20:22:57.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c0f90 con 0x7f9d1806f2c0 2023-10-10T20:22:57.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1ce7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1816eda0 con 0x7f9d1806f690 2023-10-10T20:22:57.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:57.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:53314/0 (socket says 172.21.15.148:53314) 2023-10-10T20:22:57.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d1659c700 1 -- 172.21.15.148:0/3365705245 learned_addr learned my addr 172.21.15.148:0/3365705245 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:57.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d16d9d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d1815e4a0 0x7f9d180b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:57.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.763+0000 7f9d0dd9b700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 0x7f9d180b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:57.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 -- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 msgr2=0x7f9d180b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:57.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 0x7f9d180b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:57.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 -- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d1815e4a0 msgr2=0x7f9d180b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d1815e4a0 0x7f9d180b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009420 con 0x7f9d1806f2c0 2023-10-10T20:22:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d0dd9b700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 0x7f9d180b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:22:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1659c700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800cd30 tx=0x7f9d08001b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:57.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800ce90 con 0x7f9d1806f2c0 2023-10-10T20:22:57.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800aac0 con 0x7f9d1806f2c0 2023-10-10T20:22:57.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180c0940 con 0x7f9d1806f2c0 2023-10-10T20:22:57.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.764+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800ae80 con 0x7f9d1806f2c0 2023-10-10T20:22:57.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.765+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d18171140 con 0x7f9d1806f2c0 2023-10-10T20:22:57.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.766+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1804c3c0 con 0x7f9d1806f2c0 2023-10-10T20:22:57.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.766+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d08007300 con 0x7f9d1806f2c0 2023-10-10T20:22:57.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.767+0000 7f9d0effd700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 0x7f9cf8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:57.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.767+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d08084890 con 0x7f9d1806f2c0 2023-10-10T20:22:57.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.767+0000 7f9d0dd9b700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 0x7f9cf8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:57.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.767+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f9cf806ead0 con 0x7f9d1806f2c0 2023-10-10T20:22:57.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.768+0000 7f9d0dd9b700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 0x7f9cf8069460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c04a0 tx=0x7f9d00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:57.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:57.774+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d08084030 con 0x7f9d1806f2c0 2023-10-10T20:22:58.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.015+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1806a3e0 con 0x7f9d1806f2c0 2023-10-10T20:22:58.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.016+0000 7f9d0effd700 1 -- 172.21.15.148:0/3365705245 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f9d08051ae0 con 0x7f9d1806f2c0 2023-10-10T20:22:58.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:22:58.019 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:22:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 msgr2=0x7f9cf8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 0x7f9cf8069460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9d180c04a0 tx=0x7f9d00009040 comp rx=0 tx=0).stop 2023-10-10T20:22:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 msgr2=0x7f9d1816f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800cd30 tx=0x7f9d08001b10 comp rx=0 tx=0).stop 2023-10-10T20:22:58.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 shutdown_connections 2023-10-10T20:22:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9cf8066fb0 0x7f9cf8069460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d1815e4a0 0x7f9d180b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1806f2c0 0x7f9d1816f160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 --2- 172.21.15.148:0/3365705245 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1806f690 0x7f9d180b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 >> 172.21.15.148:0/3365705245 conn(0x7f9d18067660 msgr2=0x7f9d1815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 shutdown_connections 2023-10-10T20:22:58.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.018+0000 7f9d1ce7d700 1 -- 172.21.15.148:0/3365705245 wait complete. 2023-10-10T20:22:58.023 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:22:58.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:22:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.146+0000 7f529f1ec700 1 Processor -- start 2023-10-10T20:22:58.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.146+0000 7f529f1ec700 1 -- start start 2023-10-10T20:22:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f529f1ec700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529815a620 0x7f529815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f529f1ec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 0x7f5298158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f529f1ec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529815cbb0 con 0x7f5298157870 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f529f1ec700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529815bdb0 con 0x7f529815a620 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f529f1ec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529815bfa0 con 0x7f529815d580 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f528ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529815a620 0x7f529815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f528ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529815a620 0x7f529815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35842/0 (socket says 172.21.15.148:35842) 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.147+0000 7f528ffff700 1 -- 172.21.15.148:0/3566862287 learned_addr learned my addr 172.21.15.148:0/3566862287 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529d789700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 0x7f5298158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528ffff700 1 -- 172.21.15.148:0/3566862287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 msgr2=0x7f5298158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528ffff700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 0x7f5298158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528ffff700 1 -- 172.21.15.148:0/3566862287 >> v1:172.21.15.161:6789/0 conn(0x7f5298157870 legacy=0x7f529815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528ffff700 1 -- 172.21.15.148:0/3566862287 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529815c280 con 0x7f529815a620 2023-10-10T20:22:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529d789700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 0x7f5298158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:22:58.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528ffff700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f529815d040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52880099b0 tx=0x7f5288009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9062cfb84cddd47b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528f7fe700 1 -- 172.21.15.148:0/3566862287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528800b860 con 0x7f529815a620 2023-10-10T20:22:58.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528f7fe700 1 -- 172.21.15.148:0/3566862287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528800b9c0 con 0x7f529815a620 2023-10-10T20:22:58.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f528f7fe700 1 -- 172.21.15.148:0/3566862287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528800bd60 con 0x7f529815a620 2023-10-10T20:22:58.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 -- 172.21.15.148:0/3566862287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 msgr2=0x7f529815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f529815d040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52880099b0 tx=0x7f5288009250 comp rx=0 tx=0).stop 2023-10-10T20:22:58.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 -- 172.21.15.148:0/3566862287 shutdown_connections 2023-10-10T20:22:58.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f529815d580 0x7f5298158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 --2- 172.21.15.148:0/3566862287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f529815d040 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 -- 172.21.15.148:0/3566862287 >> 172.21.15.148:0/3566862287 conn(0x7f5298068ce0 msgr2=0x7f52980690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.148+0000 7f529f1ec700 1 -- 172.21.15.148:0/3566862287 shutdown_connections 2023-10-10T20:22:58.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.149+0000 7f529f1ec700 1 -- 172.21.15.148:0/3566862287 wait complete. 2023-10-10T20:22:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.150+0000 7f529f1ec700 1 Processor -- start 2023-10-10T20:22:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.150+0000 7f529f1ec700 1 -- start start 2023-10-10T20:22:58.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.150+0000 7f529f1ec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5298157870 0x7f529814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.150+0000 7f529f1ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.150+0000 7f529f1ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529815d580 0x7f52980b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529f1ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980a9890 con 0x7f529815a620 2023-10-10T20:22:58.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529f1ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980aa130 con 0x7f529815d580 2023-10-10T20:22:58.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529f1ec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980a8500 con 0x7f5298157870 2023-10-10T20:22:58.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35856/0 (socket says 172.21.15.148:35856) 2023-10-10T20:22:58.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 -- 172.21.15.148:0/2228862234 learned_addr learned my addr 172.21.15.148:0/2228862234 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:58.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529d789700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529815d580 0x7f52980b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529cf88700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5298157870 0x7f529814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 -- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5298157870 msgr2=0x7f529814b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5298157870 0x7f529814b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 -- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529815d580 msgr2=0x7f52980b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529815d580 0x7f52980b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880094f0 con 0x7f529815a620 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528ffff700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f529806b530 tx=0x7f5288017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288011e00 con 0x7f529815a620 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288024440 con 0x7f529815a620 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980a9ae0 con 0x7f529815a620 2023-10-10T20:22:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880247e0 con 0x7f529815a620 2023-10-10T20:22:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.151+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529814bc60 con 0x7f529815a620 2023-10-10T20:22:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.152+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52980aa2c0 con 0x7f529815a620 2023-10-10T20:22:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.153+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f528801a420 con 0x7f529815a620 2023-10-10T20:22:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.153+0000 7f528cff9700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 0x7f52840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.153+0000 7f529cf88700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 0x7f52840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.153+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52880863f0 con 0x7f529815a620 2023-10-10T20:22:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.153+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f528406ea50 con 0x7f529815a620 2023-10-10T20:22:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.154+0000 7f529cf88700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 0x7f52840693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5294007fd0 tx=0x7f52940016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.158+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5288053660 con 0x7f529815a620 2023-10-10T20:22:58.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.340+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f529804c3c0 con 0x7f529815a620 2023-10-10T20:22:58.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.341+0000 7f528cff9700 1 -- 172.21.15.148:0/2228862234 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15456 (secure 0 0 0) 0x7f5288015020 con 0x7f529815a620 2023-10-10T20:22:58.343 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:22:58.343 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":69,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:47.767303+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:22:58.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 msgr2=0x7f52840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 0x7f52840693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5294007fd0 tx=0x7f52940016f0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 msgr2=0x7f52980adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f529806b530 tx=0x7f5288017000 comp rx=0 tx=0).stop 2023-10-10T20:22:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 shutdown_connections 2023-10-10T20:22:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5284066f30 0x7f52840693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f529815d580 0x7f52980b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5298157870 0x7f529814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 --2- 172.21.15.148:0/2228862234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529815a620 0x7f52980adb70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 >> 172.21.15.148:0/2228862234 conn(0x7f5298068ce0 msgr2=0x7f52980690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 shutdown_connections 2023-10-10T20:22:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.343+0000 7f529f1ec700 1 -- 172.21.15.148:0/2228862234 wait complete. 2023-10-10T20:22:58.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:21:24.814992+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:21:25.970062+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:22:58.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:22:58.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.476+0000 7fa1e0273700 1 Processor -- start 2023-10-10T20:22:58.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.476+0000 7fa1e0273700 1 -- start start 2023-10-10T20:22:58.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1e0273700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1d8159ec0 0x7fa1d80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1e0273700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1d806acb0 0x7fa1d80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1e0273700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d80b0790 con 0x7fa1d815a290 2023-10-10T20:22:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1e0273700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d80ce1f0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1e0273700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d80b0b20 con 0x7fa1d806acb0 2023-10-10T20:22:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1d8159ec0 0x7fa1d80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1d8159ec0 0x7fa1d80af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35868/0 (socket says 172.21.15.148:35868) 2023-10-10T20:22:58.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 -- 172.21.15.148:0/2750945103 learned_addr learned my addr 172.21.15.148:0/2750945103 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:58.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de810700 1 --2- 172.21.15.148:0/2750945103 >> v2:172.21.15.161:3300/0 conn(0x7fa1d806acb0 0x7fa1d80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 -- 172.21.15.148:0/2750945103 >> v2:172.21.15.161:3300/0 conn(0x7fa1d806acb0 msgr2=0x7fa1d80afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:58.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 --2- 172.21.15.148:0/2750945103 >> v2:172.21.15.161:3300/0 conn(0x7fa1d806acb0 0x7fa1d80afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 -- 172.21.15.148:0/2750945103 >> v1:172.21.15.161:6789/0 conn(0x7fa1d815a290 legacy=0x7fa1d806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 -- 172.21.15.148:0/2750945103 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80b04b0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1de00f700 1 --2- 172.21.15.148:0/2750945103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80af940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80711f0 tx=0x7fa1d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1248f1469cb9c52 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1dd00d700 1 -- 172.21.15.148:0/2750945103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000b820 con 0x7fa1d8159ec0 2023-10-10T20:22:58.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1dd00d700 1 -- 172.21.15.148:0/2750945103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000b980 con 0x7fa1d8159ec0 2023-10-10T20:22:58.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.477+0000 7fa1dd00d700 1 -- 172.21.15.148:0/2750945103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000bd20 con 0x7fa1d8159ec0 2023-10-10T20:22:58.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 -- 172.21.15.148:0/2750945103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 msgr2=0x7fa1d80af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 --2- 172.21.15.148:0/2750945103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80af940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80711f0 tx=0x7fa1d0009250 comp rx=0 tx=0).stop 2023-10-10T20:22:58.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 -- 172.21.15.148:0/2750945103 shutdown_connections 2023-10-10T20:22:58.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 --2- 172.21.15.148:0/2750945103 >> v2:172.21.15.161:3300/0 conn(0x7fa1d806acb0 0x7fa1d80afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 --2- 172.21.15.148:0/2750945103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80af940 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 -- 172.21.15.148:0/2750945103 >> 172.21.15.148:0/2750945103 conn(0x7fa1d80d1880 msgr2=0x7fa1d80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 -- 172.21.15.148:0/2750945103 shutdown_connections 2023-10-10T20:22:58.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.478+0000 7fa1e0273700 1 -- 172.21.15.148:0/2750945103 wait complete. 2023-10-10T20:22:58.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.479+0000 7fa1e0273700 1 Processor -- start 2023-10-10T20:22:58.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.479+0000 7fa1e0273700 1 -- start start 2023-10-10T20:22:58.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d806acb0 0x7fa1d81497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1d815a290 0x7fa1d80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d816a020 con 0x7fa1d8159ec0 2023-10-10T20:22:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d816a2d0 con 0x7fa1d806acb0 2023-10-10T20:22:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1e0273700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d8156ff0 con 0x7fa1d815a290 2023-10-10T20:22:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35874/0 (socket says 172.21.15.148:35874) 2023-10-10T20:22:58.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 -- 172.21.15.148:0/677503845 learned_addr learned my addr 172.21.15.148:0/677503845 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1de00f700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d806acb0 0x7fa1d81497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1de810700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1d815a290 0x7fa1d80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 -- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1d815a290 msgr2=0x7fa1d80b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1d815a290 0x7fa1d80b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 -- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d806acb0 msgr2=0x7fa1d81497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d806acb0 0x7fa1d81497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d00094f0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.480+0000 7fa1dd80e700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00bc60 tx=0x7fa1cc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00c840 con 0x7fa1d8159ec0 2023-10-10T20:22:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00c9a0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d81572d0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00cda0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d8149d10 con 0x7fa1d8159ec0 2023-10-10T20:22:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.481+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d816a460 con 0x7fa1d8159ec0 2023-10-10T20:22:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.482+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1cc014620 con 0x7fa1d8159ec0 2023-10-10T20:22:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.482+0000 7fa1d67fc700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.482+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1cc0844c0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.482+0000 7fa1de00f700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.483+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fa1c406ea50 con 0x7fa1d8159ec0 2023-10-10T20:22:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.483+0000 7fa1de00f700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80711f0 tx=0x7fa1d0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.487+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1cc04c5a0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.675+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1d804c3c0 con 0x7fa1d8159ec0 2023-10-10T20:22:58.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.675+0000 7fa1d67fc700 1 -- 172.21.15.148:0/677503845 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1cc051730 con 0x7fa1d8159ec0 2023-10-10T20:22:58.677 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:22:58.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 msgr2=0x7fa1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80711f0 tx=0x7fa1d0016000 comp rx=0 tx=0).stop 2023-10-10T20:22:58.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 msgr2=0x7fa1d80b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00bc60 tx=0x7fa1cc009820 comp rx=0 tx=0).stop 2023-10-10T20:22:58.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 shutdown_connections 2023-10-10T20:22:58.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1d815a290 0x7fa1d80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1d806acb0 0x7fa1d81497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 --2- 172.21.15.148:0/677503845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1d8159ec0 0x7fa1d80b2b70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 >> 172.21.15.148:0/677503845 conn(0x7fa1d80d1880 msgr2=0x7fa1d81558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 shutdown_connections 2023-10-10T20:22:58.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.677+0000 7fa1e0273700 1 -- 172.21.15.148:0/677503845 wait complete. 2023-10-10T20:22:58.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:22:58.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.808+0000 7f8d7524e700 1 Processor -- start 2023-10-10T20:22:58.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.808+0000 7f8d7524e700 1 -- start start 2023-10-10T20:22:58.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d7524e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d700cbf10 0x7f8d7015e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d7524e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700c9390 0x7f8d7015a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d7524e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700cae70 con 0x7f8d700678b0 2023-10-10T20:22:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d7524e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700cb150 con 0x7f8d700cbf10 2023-10-10T20:22:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d7524e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700cb430 con 0x7f8d700c9390 2023-10-10T20:22:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d6659c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8d700cbf10 0x7f8d7015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d6ed9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8d700678b0 0x7f8d7015db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37192/0 (socket says 172.21.15.148:37192) 2023-10-10T20:22:58.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.809+0000 7f8d6ed9d700 1 -- 172.21.15.148:0/392292237 learned_addr learned my addr 172.21.15.148:0/392292237 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:22:58.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6f59e700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700c9390 0x7f8d7015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6659c700 1 -- 172.21.15.148:0/392292237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700c9390 msgr2=0x7f8d7015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:58.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6659c700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700c9390 0x7f8d7015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6659c700 1 -- 172.21.15.148:0/392292237 >> v1:172.21.15.148:6789/0 conn(0x7f8d700678b0 legacy=0x7f8d7015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6659c700 1 -- 172.21.15.148:0/392292237 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700cb710 con 0x7f8d700cbf10 2023-10-10T20:22:58.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6e59c700 1 -- 172.21.15.148:0/392292237 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751013453 0 0) 0x7f8d700cae70 con 0x7f8d700678b0 2023-10-10T20:22:58.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6659c700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700cbf10 0x7f8d7015e290 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8d580098e0 tx=0x7f8d580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98348ca6e540b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6e59c700 1 -- 172.21.15.148:0/392292237 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58009e80 con 0x7f8d700cbf10 2023-10-10T20:22:58.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6e59c700 1 -- 172.21.15.148:0/392292237 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5800b860 con 0x7f8d700cbf10 2023-10-10T20:22:58.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d6e59c700 1 -- 172.21.15.148:0/392292237 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800bbe0 con 0x7f8d700cbf10 2023-10-10T20:22:58.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 -- 172.21.15.148:0/392292237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700cbf10 msgr2=0x7f8d7015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700cbf10 0x7f8d7015e290 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8d580098e0 tx=0x7f8d580091a0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 -- 172.21.15.148:0/392292237 shutdown_connections 2023-10-10T20:22:58.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700c9390 0x7f8d7015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 --2- 172.21.15.148:0/392292237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700cbf10 0x7f8d7015e290 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 -- 172.21.15.148:0/392292237 >> 172.21.15.148:0/392292237 conn(0x7f8d70069870 msgr2=0x7f8d70069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:58.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 -- 172.21.15.148:0/392292237 shutdown_connections 2023-10-10T20:22:58.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.810+0000 7f8d7524e700 1 -- 172.21.15.148:0/392292237 wait complete. 2023-10-10T20:22:58.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 Processor -- start 2023-10-10T20:22:58.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 -- start start 2023-10-10T20:22:58.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d700678b0 0x7f8d7015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700b1f60 0x7f8d7014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015b5f0 con 0x7f8d700c9390 2023-10-10T20:22:58.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b24f0 con 0x7f8d700678b0 2023-10-10T20:22:58.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.812+0000 7f8d7524e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b27d0 con 0x7f8d700b1f60 2023-10-10T20:22:58.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35884/0 (socket says 172.21.15.148:35884) 2023-10-10T20:22:58.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 -- 172.21.15.148:0/2957922535 learned_addr learned my addr 172.21.15.148:0/2957922535 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:58.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6f59e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700b1f60 0x7f8d7014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6ed9d700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d700678b0 0x7f8d7015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 -- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700b1f60 msgr2=0x7f8d7014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700b1f60 0x7f8d7014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 -- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d700678b0 msgr2=0x7f8d7015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:58.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d700678b0 0x7f8d7015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:58.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d58009420 con 0x7f8d700c9390 2023-10-10T20:22:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d6659c700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d58013000 tx=0x7f8d58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800bbe0 con 0x7f8d700c9390 2023-10-10T20:22:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58011420 con 0x7f8d700c9390 2023-10-10T20:22:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d700b2ab0 con 0x7f8d700c9390 2023-10-10T20:22:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58011800 con 0x7f8d700c9390 2023-10-10T20:22:58.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.813+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700b3070 con 0x7f8d700c9390 2023-10-10T20:22:58.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.814+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700b3260 con 0x7f8d700c9390 2023-10-10T20:22:58.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.814+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d58011bf0 con 0x7f8d700c9390 2023-10-10T20:22:58.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.815+0000 7f8d677fe700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:58.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.815+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d58085f00 con 0x7f8d700c9390 2023-10-10T20:22:58.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.815+0000 7f8d6ed9d700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:58.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.815+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f8d5006ea50 con 0x7f8d700c9390 2023-10-10T20:22:58.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.815+0000 7f8d6ed9d700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 0x7f8d500693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d600011e0 tx=0x7f8d60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:58.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.819+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d58053170 con 0x7f8d700c9390 2023-10-10T20:22:59.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:58.999+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d7004c3c0 con 0x7f8d700c9390 2023-10-10T20:22:59.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.000+0000 7f8d677fe700 1 -- 172.21.15.148:0/2957922535 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d58056790 con 0x7f8d700c9390 2023-10-10T20:22:59.002 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:22:59.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 msgr2=0x7f8d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 0x7f8d500693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d600011e0 tx=0x7f8d60009040 comp rx=0 tx=0).stop 2023-10-10T20:22:59.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 msgr2=0x7f8d701539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d58013000 tx=0x7f8d58017000 comp rx=0 tx=0).stop 2023-10-10T20:22:59.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 shutdown_connections 2023-10-10T20:22:59.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d50066f30 0x7f8d500693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d700b1f60 0x7f8d7014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d700678b0 0x7f8d7015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 --2- 172.21.15.148:0/2957922535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d700c9390 0x7f8d701539d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.001+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 >> 172.21.15.148:0/2957922535 conn(0x7f8d70069870 msgr2=0x7f8d70150890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.002+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 shutdown_connections 2023-10-10T20:22:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.002+0000 7f8d7524e700 1 -- 172.21.15.148:0/2957922535 wait complete. 2023-10-10T20:22:59.015 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:22:59.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.131+0000 7fe249798700 1 Processor -- start 2023-10-10T20:22:59.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.131+0000 7fe249798700 1 -- start start 2023-10-10T20:22:59.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe249798700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe2440b6580 0x7fe24406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe249798700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe24406a830 0x7fe2440b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe249798700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24406ac90 con 0x7fe2440cce30 2023-10-10T20:22:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe249798700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2440b4d70 con 0x7fe24406a830 2023-10-10T20:22:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe249798700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24415b3b0 con 0x7fe2440b6580 2023-10-10T20:22:59.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe24406a830 0x7fe2440b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe243fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe2440cce30 0x7fe24415aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37208/0 (socket says 172.21.15.148:37208) 2023-10-10T20:22:59.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe243fff700 1 -- 172.21.15.148:0/787590000 learned_addr learned my addr 172.21.15.148:0/787590000 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:22:59.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe2437fe700 1 --2- 172.21.15.148:0/787590000 >> v2:172.21.15.161:3300/0 conn(0x7fe2440b6580 0x7fe24406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 -- 172.21.15.148:0/787590000 >> v2:172.21.15.161:3300/0 conn(0x7fe2440b6580 msgr2=0x7fe24406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:59.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 --2- 172.21.15.148:0/787590000 >> v2:172.21.15.161:3300/0 conn(0x7fe2440b6580 0x7fe24406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 -- 172.21.15.148:0/787590000 >> v1:172.21.15.148:6789/0 conn(0x7fe2440cce30 legacy=0x7fe24415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 -- 172.21.15.148:0/787590000 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2440cd8d0 con 0x7fe24406a830 2023-10-10T20:22:59.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe242ffd700 1 -- 172.21.15.148:0/787590000 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798495438 0 0) 0x7fe234001110 con 0x7fe2440cce30 2023-10-10T20:22:59.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.132+0000 7fe248f97700 1 --2- 172.21.15.148:0/787590000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b5200 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe244089bc0 tx=0x7fe23c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f643c621ddfe7a92 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe242ffd700 1 -- 172.21.15.148:0/787590000 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00e070 con 0x7fe24406a830 2023-10-10T20:22:59.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe242ffd700 1 -- 172.21.15.148:0/787590000 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00bc30 con 0x7fe24406a830 2023-10-10T20:22:59.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe242ffd700 1 -- 172.21.15.148:0/787590000 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c012430 con 0x7fe24406a830 2023-10-10T20:22:59.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 -- 172.21.15.148:0/787590000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 msgr2=0x7fe2440b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 --2- 172.21.15.148:0/787590000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b5200 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe244089bc0 tx=0x7fe23c009780 comp rx=0 tx=0).stop 2023-10-10T20:22:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 -- 172.21.15.148:0/787590000 shutdown_connections 2023-10-10T20:22:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 --2- 172.21.15.148:0/787590000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b5200 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 --2- 172.21.15.148:0/787590000 >> v2:172.21.15.161:3300/0 conn(0x7fe2440b6580 0x7fe24406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 -- 172.21.15.148:0/787590000 >> 172.21.15.148:0/787590000 conn(0x7fe244068cb0 msgr2=0x7fe2440690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 -- 172.21.15.148:0/787590000 shutdown_connections 2023-10-10T20:22:59.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.133+0000 7fe249798700 1 -- 172.21.15.148:0/787590000 wait complete. 2023-10-10T20:22:59.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.134+0000 7fe249798700 1 Processor -- start 2023-10-10T20:22:59.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.134+0000 7fe249798700 1 -- start start 2023-10-10T20:22:59.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 0x7fe24413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 0x7fe244151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24413c270 con 0x7fe24406a830 2023-10-10T20:22:59.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440b24d0 con 0x7fe2440cce30 2023-10-10T20:22:59.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe249798700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe244154150 con 0x7fe2440b6580 2023-10-10T20:22:59.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe243fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe243fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35906/0 (socket says 172.21.15.148:35906) 2023-10-10T20:22:59.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe243fff700 1 -- 172.21.15.148:0/4191847072 learned_addr learned my addr 172.21.15.148:0/4191847072 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:59.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe248f97700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 0x7fe244151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe2437fe700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 0x7fe24413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.135+0000 7fe243fff700 1 -- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 msgr2=0x7fe24413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe243fff700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 0x7fe24413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe243fff700 1 -- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 msgr2=0x7fe244151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe243fff700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 0x7fe244151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe243fff700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009a20 con 0x7fe24406a830 2023-10-10T20:22:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe248f97700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 0x7fe244151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:22:59.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe2437fe700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 0x7fe24413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:22:59.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe243fff700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c5b0 tx=0x7fe234001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234005b30 con 0x7fe24406a830 2023-10-10T20:22:59.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234005c90 con 0x7fe24406a830 2023-10-10T20:22:59.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234007300 con 0x7fe24406a830 2023-10-10T20:22:59.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe244154430 con 0x7fe24406a830 2023-10-10T20:22:59.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.136+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe244151b80 con 0x7fe24406a830 2023-10-10T20:22:59.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.137+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe244151dd0 con 0x7fe24406a830 2023-10-10T20:22:59.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.137+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe23401b3b0 con 0x7fe24406a830 2023-10-10T20:22:59.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.138+0000 7fe22bfff700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.138+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe234083910 con 0x7fe24406a830 2023-10-10T20:22:59.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.138+0000 7fe2437fe700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.138+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fe23006ea50 con 0x7fe24406a830 2023-10-10T20:22:59.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.138+0000 7fe2437fe700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe24416d440 tx=0x7fe238008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.142+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe234017060 con 0x7fe24406a830 2023-10-10T20:22:59.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.322+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe24406ac90 con 0x7fe24406a830 2023-10-10T20:22:59.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.323+0000 7fe22bfff700 1 -- 172.21.15.148:0/4191847072 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe24406ac90 con 0x7fe24406a830 2023-10-10T20:22:59.326 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:22:59.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 msgr2=0x7fe2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 0x7fe2300693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe24416d440 tx=0x7fe238008040 comp rx=0 tx=0).stop 2023-10-10T20:22:59.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 msgr2=0x7fe2440b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c5b0 tx=0x7fe234001070 comp rx=0 tx=0).stop 2023-10-10T20:22:59.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 shutdown_connections 2023-10-10T20:22:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe230066f30 0x7fe2300693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2440cce30 0x7fe244151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24406a830 0x7fe2440b1e30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 --2- 172.21.15.148:0/4191847072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2440b6580 0x7fe24413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 >> 172.21.15.148:0/4191847072 conn(0x7fe244068cb0 msgr2=0x7fe2440cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 shutdown_connections 2023-10-10T20:22:59.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.325+0000 7fe249798700 1 -- 172.21.15.148:0/4191847072 wait complete. 2023-10-10T20:22:59.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:22:59.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.453+0000 7ffbbdb82700 1 Processor -- start 2023-10-10T20:22:59.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 -- start start 2023-10-10T20:22:59.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8152010 0x7ffbb815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbb8158e50 0x7ffbb80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb806c010 con 0x7ffbb81523e0 2023-10-10T20:22:59.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb806bd10 con 0x7ffbb8158e50 2023-10-10T20:22:59.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbbdb82700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb815c060 con 0x7ffbb8152010 2023-10-10T20:22:59.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.454+0000 7ffbb7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbb8158e50 0x7ffbb80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbb8158e50 0x7ffbb80cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35918/0 (socket says 172.21.15.148:35918) 2023-10-10T20:22:59.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 -- 172.21.15.148:0/3518946834 learned_addr learned my addr 172.21.15.148:0/3518946834 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:59.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb77fe700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8152010 0x7ffbb815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 -- 172.21.15.148:0/3518946834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8152010 msgr2=0x7ffbb815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8152010 0x7ffbb815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 -- 172.21.15.148:0/3518946834 >> v1:172.21.15.161:6789/0 conn(0x7ffbb81523e0 legacy=0x7ffbb806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 -- 172.21.15.148:0/3518946834 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb815c340 con 0x7ffbb8158e50 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb7fff700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8158e50 0x7ffbb80cdf30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00099b0 tx=0x7ffbb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dba94ce92175e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb67fc700 1 -- 172.21.15.148:0/3518946834 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0009ea0 con 0x7ffbb8158e50 2023-10-10T20:22:59.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb67fc700 1 -- 172.21.15.148:0/3518946834 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000b860 con 0x7ffbb8158e50 2023-10-10T20:22:59.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.455+0000 7ffbb67fc700 1 -- 172.21.15.148:0/3518946834 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000bc00 con 0x7ffbb8158e50 2023-10-10T20:22:59.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 -- 172.21.15.148:0/3518946834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8158e50 msgr2=0x7ffbb80cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8158e50 0x7ffbb80cdf30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00099b0 tx=0x7ffbb0009250 comp rx=0 tx=0).stop 2023-10-10T20:22:59.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 -- 172.21.15.148:0/3518946834 shutdown_connections 2023-10-10T20:22:59.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8158e50 0x7ffbb80cdf30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/3518946834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8152010 0x7ffbb815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 -- 172.21.15.148:0/3518946834 >> 172.21.15.148:0/3518946834 conn(0x7ffbb8069870 msgr2=0x7ffbb8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 -- 172.21.15.148:0/3518946834 shutdown_connections 2023-10-10T20:22:59.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.456+0000 7ffbbdb82700 1 -- 172.21.15.148:0/3518946834 wait complete. 2023-10-10T20:22:59.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.457+0000 7ffbbdb82700 1 Processor -- start 2023-10-10T20:22:59.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.457+0000 7ffbbdb82700 1 -- start start 2023-10-10T20:22:59.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb81523e0 0x7ffbb816ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8158e50 0x7ffbb80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8067810 con 0x7ffbb8152010 2023-10-10T20:22:59.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80af9b0 con 0x7ffbb81523e0 2023-10-10T20:22:59.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80696a0 con 0x7ffbb8158e50 2023-10-10T20:22:59.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35928/0 (socket says 172.21.15.148:35928) 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 -- 172.21.15.148:0/2152905731 learned_addr learned my addr 172.21.15.148:0/2152905731 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb6ffd700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb81523e0 0x7ffbb816ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb7fff700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8158e50 0x7ffbb80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 -- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8158e50 msgr2=0x7ffbb80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8158e50 0x7ffbb80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 -- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb81523e0 msgr2=0x7ffbb816ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb81523e0 0x7ffbb816ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00094f0 con 0x7ffbb8152010 2023-10-10T20:22:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbb77fe700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009e80 tx=0x7ffba800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800caa0 con 0x7ffbb8152010 2023-10-10T20:22:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800f040 con 0x7ffbb8152010 2023-10-10T20:22:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb80aeda0 con 0x7ffbb8152010 2023-10-10T20:22:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800ce50 con 0x7ffbb8152010 2023-10-10T20:22:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.458+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80af360 con 0x7ffbb8152010 2023-10-10T20:22:59.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.459+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb804c3c0 con 0x7ffbb8152010 2023-10-10T20:22:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.460+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffba8014690 con 0x7ffbb8152010 2023-10-10T20:22:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.460+0000 7ffb9f7fe700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.460+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffba8084d40 con 0x7ffbb8152010 2023-10-10T20:22:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.460+0000 7ffbb6ffd700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.460+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7ffba406ea50 con 0x7ffbb8152010 2023-10-10T20:22:59.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.461+0000 7ffbb6ffd700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8151ad0 tx=0x7ffbac007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.464+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffba8051fb0 con 0x7ffbb8152010 2023-10-10T20:22:59.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.692+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbb80c8f00 con 0x7ffbb8152010 2023-10-10T20:22:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.693+0000 7ffb9f7fe700 1 -- 172.21.15.148:0/2152905731 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ffba80555d0 con 0x7ffbb8152010 2023-10-10T20:22:59.695 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:22:59.696 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:22:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 msgr2=0x7ffba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8151ad0 tx=0x7ffbac007ec0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 msgr2=0x7ffbb8069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009e80 tx=0x7ffba800b040 comp rx=0 tx=0).stop 2023-10-10T20:22:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 shutdown_connections 2023-10-10T20:22:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbb8158e50 0x7ffbb80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb8152010 0x7ffbb8069160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 --2- 172.21.15.148:0/2152905731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb81523e0 0x7ffbb816ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 >> 172.21.15.148:0/2152905731 conn(0x7ffbb8069870 msgr2=0x7ffbb8069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 shutdown_connections 2023-10-10T20:22:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.695+0000 7ffbbdb82700 1 -- 172.21.15.148:0/2152905731 wait complete. 2023-10-10T20:22:59.700 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:22:59.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:22:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.821+0000 7f32cc1f5700 1 Processor -- start 2023-10-10T20:22:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.821+0000 7f32cc1f5700 1 -- start start 2023-10-10T20:22:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32cc1f5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32c406abc0 0x7f32c40af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32cc1f5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c40b0580 con 0x7f32c40afd90 2023-10-10T20:22:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32cc1f5700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c415b0c0 con 0x7f32c40cc3c0 2023-10-10T20:22:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32cc1f5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c415b550 con 0x7f32c406abc0 2023-10-10T20:22:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32ca792700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32c40afd90 0x7f32c4163c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37224/0 (socket says 172.21.15.148:37224) 2023-10-10T20:22:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32ca792700 1 -- 172.21.15.148:0/4022904191 learned_addr learned my addr 172.21.15.148:0/4022904191 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:22:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.822+0000 7f32c9790700 1 --2- 172.21.15.148:0/4022904191 >> v2:172.21.15.161:3300/0 conn(0x7f32c406abc0 0x7f32c40af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057497253 0 0) 0x7f32c40b0580 con 0x7f32c40afd90 2023-10-10T20:22:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b0003570 con 0x7f32c40afd90 2023-10-10T20:22:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554258527 0 0) 0x7f32c415b0c0 con 0x7f32c40cc3c0 2023-10-10T20:22:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c40b0580 con 0x7f32c40cc3c0 2023-10-10T20:22:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c9790700 1 -- 172.21.15.148:0/4022904191 >> v1:172.21.15.161:6789/0 conn(0x7f32c40cc3c0 legacy=0x7f32c406a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c9790700 1 -- 172.21.15.148:0/4022904191 >> v1:172.21.15.148:6789/0 conn(0x7f32c40afd90 legacy=0x7f32c4163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c9790700 1 -- 172.21.15.148:0/4022904191 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40cc790 con 0x7f32c406abc0 2023-10-10T20:22:59.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c9790700 1 --2- 172.21.15.148:0/4022904191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c406abc0 0x7f32c40af850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32b40098e0 tx=0x7f32b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed937d11a5a5e163 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4011630 con 0x7f32c406abc0 2023-10-10T20:22:59.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4011790 con 0x7f32c406abc0 2023-10-10T20:22:59.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.823+0000 7f32c8f8f700 1 -- 172.21.15.148:0/4022904191 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4011b30 con 0x7f32c406abc0 2023-10-10T20:22:59.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4022904191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c406abc0 msgr2=0x7f32c40af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4022904191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c406abc0 0x7f32c40af850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32b40098e0 tx=0x7f32b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4022904191 shutdown_connections 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4022904191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c406abc0 0x7f32c40af850 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4022904191 >> 172.21.15.148:0/4022904191 conn(0x7f32c4069800 msgr2=0x7f32c40d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4022904191 shutdown_connections 2023-10-10T20:22:59.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.824+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4022904191 wait complete. 2023-10-10T20:22:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.825+0000 7f32cc1f5700 1 Processor -- start 2023-10-10T20:22:59.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.825+0000 7f32cc1f5700 1 -- start start 2023-10-10T20:22:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c40afd90 0x7f32c415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c40cc3c0 0x7f32c415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c7dc0 con 0x7f32c406abc0 2023-10-10T20:22:59.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c8380 con 0x7f32c40cc3c0 2023-10-10T20:22:59.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41467f0 con 0x7f32c40afd90 2023-10-10T20:22:59.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35934/0 (socket says 172.21.15.148:35934) 2023-10-10T20:22:59.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 -- 172.21.15.148:0/4253153497 learned_addr learned my addr 172.21.15.148:0/4253153497 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:22:59.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32ca792700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c40cc3c0 0x7f32c415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9790700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c40afd90 0x7f32c415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 -- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c40afd90 msgr2=0x7f32c415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c40afd90 0x7f32c415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 -- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c40cc3c0 msgr2=0x7f32c415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:22:59.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c40cc3c0 0x7f32c415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:22:59.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4009420 con 0x7f32c406abc0 2023-10-10T20:22:59.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32c9f91700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c5b0 tx=0x7f32c0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010510 con 0x7f32c406abc0 2023-10-10T20:22:59.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0010670 con 0x7f32c406abc0 2023-10-10T20:22:59.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c40b66f0 con 0x7f32c406abc0 2023-10-10T20:22:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010a30 con 0x7f32c406abc0 2023-10-10T20:22:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.826+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c40c8010 con 0x7f32c406abc0 2023-10-10T20:22:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.827+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4089110 con 0x7f32c406abc0 2023-10-10T20:22:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.828+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32c001b400 con 0x7f32c406abc0 2023-10-10T20:22:59.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.828+0000 7f32b9ffb700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:22:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.828+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32c0084060 con 0x7f32c406abc0 2023-10-10T20:22:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.828+0000 7f32c9790700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:22:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.828+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f32a806ea50 con 0x7f32c406abc0 2023-10-10T20:22:59.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.829+0000 7f32c9790700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32c406afc0 tx=0x7f32b4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:22:59.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:22:59.832+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c00512d0 con 0x7f32c406abc0 2023-10-10T20:23:00.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.017+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c404c3c0 con 0x7f32c406abc0 2023-10-10T20:23:00.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.018+0000 7f32b9ffb700 1 -- 172.21.15.148:0/4253153497 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15456 (secure 0 0 0) 0x7f32c00548f0 con 0x7f32c406abc0 2023-10-10T20:23:00.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:00.021 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":69,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:47.767303+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:00.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 msgr2=0x7f32a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f32c406afc0 tx=0x7f32b4017000 comp rx=0 tx=0).stop 2023-10-10T20:23:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 msgr2=0x7f32c41462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c5b0 tx=0x7f32c0007e10 comp rx=0 tx=0).stop 2023-10-10T20:23:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 shutdown_connections 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c40cc3c0 0x7f32c415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c406abc0 0x7f32c41462b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 --2- 172.21.15.148:0/4253153497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c40afd90 0x7f32c415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 >> 172.21.15.148:0/4253153497 conn(0x7f32c4069800 msgr2=0x7f32c40b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 shutdown_connections 2023-10-10T20:23:00.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.020+0000 7f32cc1f5700 1 -- 172.21.15.148:0/4253153497 wait complete. 2023-10-10T20:23:00.034 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:23:00.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:00.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.147+0000 7fd71e01d700 1 Processor -- start 2023-10-10T20:23:00.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.147+0000 7fd71e01d700 1 -- start start 2023-10-10T20:23:00.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd718157870 0x7fd71815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd71815a620 0x7fd71815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 0x7fd718158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71815cbb0 con 0x7fd71815a620 2023-10-10T20:23:00.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71815bfa0 con 0x7fd71815d580 2023-10-10T20:23:00.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd71e01d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815c280 con 0x7fd718157870 2023-10-10T20:23:00.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.148+0000 7fd716ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd71815a620 0x7fd71815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd71815a620 0x7fd71815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35940/0 (socket says 172.21.15.148:35940) 2023-10-10T20:23:00.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 -- 172.21.15.148:0/792631133 learned_addr learned my addr 172.21.15.148:0/792631133 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd717fff700 1 --2- 172.21.15.148:0/792631133 >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 0x7fd718158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd7177fe700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd718157870 0x7fd71815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 -- 172.21.15.148:0/792631133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd718157870 msgr2=0x7fd71815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd718157870 0x7fd71815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 -- 172.21.15.148:0/792631133 >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 msgr2=0x7fd718158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 --2- 172.21.15.148:0/792631133 >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 0x7fd718158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 -- 172.21.15.148:0/792631133 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71815c560 con 0x7fd71815a620 2023-10-10T20:23:00.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd717fff700 1 --2- 172.21.15.148:0/792631133 >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 0x7fd718158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:00.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd716ffd700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71815d040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7180b03b0 tx=0x7fd70c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4caa179f336e1e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd7167fc700 1 -- 172.21.15.148:0/792631133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00b860 con 0x7fd71815a620 2023-10-10T20:23:00.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd7167fc700 1 -- 172.21.15.148:0/792631133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00b9c0 con 0x7fd71815a620 2023-10-10T20:23:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.149+0000 7fd7167fc700 1 -- 172.21.15.148:0/792631133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00bd60 con 0x7fd71815a620 2023-10-10T20:23:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 -- 172.21.15.148:0/792631133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 msgr2=0x7fd71815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71815d040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7180b03b0 tx=0x7fd70c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 -- 172.21.15.148:0/792631133 shutdown_connections 2023-10-10T20:23:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 --2- 172.21.15.148:0/792631133 >> v2:172.21.15.161:3300/0 conn(0x7fd71815d580 0x7fd718158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd718157870 0x7fd71815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 --2- 172.21.15.148:0/792631133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71815d040 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 -- 172.21.15.148:0/792631133 >> 172.21.15.148:0/792631133 conn(0x7fd718068ce0 msgr2=0x7fd7180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 -- 172.21.15.148:0/792631133 shutdown_connections 2023-10-10T20:23:00.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.150+0000 7fd71e01d700 1 -- 172.21.15.148:0/792631133 wait complete. 2023-10-10T20:23:00.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.151+0000 7fd71e01d700 1 Processor -- start 2023-10-10T20:23:00.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.151+0000 7fd71e01d700 1 -- start start 2023-10-10T20:23:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd718157870 0x7fd718151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd71815d580 0x7fd718152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71813cf80 con 0x7fd71815a620 2023-10-10T20:23:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71813fea0 con 0x7fd718157870 2023-10-10T20:23:00.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd71e01d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71813d1d0 con 0x7fd71815d580 2023-10-10T20:23:00.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35954/0 (socket says 172.21.15.148:35954) 2023-10-10T20:23:00.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 -- 172.21.15.148:0/211843474 learned_addr learned my addr 172.21.15.148:0/211843474 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd7177fe700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd718157870 0x7fd718151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd717fff700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd71815d580 0x7fd718152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 -- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd71815d580 msgr2=0x7fd718152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd71815d580 0x7fd718152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 -- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd718157870 msgr2=0x7fd718151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd718157870 0x7fd718151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0094f0 con 0x7fd71815a620 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.152+0000 7fd716ffd700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd71806b530 tx=0x7fd70c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.153+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c024440 con 0x7fd71815a620 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.153+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c0245a0 con 0x7fd71815a620 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.153+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71813d4b0 con 0x7fd71815a620 2023-10-10T20:23:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.153+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c024960 con 0x7fd71815a620 2023-10-10T20:23:00.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.153+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71813da70 con 0x7fd71815a620 2023-10-10T20:23:00.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.154+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71813dc90 con 0x7fd71815a620 2023-10-10T20:23:00.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.154+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd70c01a420 con 0x7fd71815a620 2023-10-10T20:23:00.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.154+0000 7fd6ff7fe700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 0x7fd7040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.154+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd70c086440 con 0x7fd71815a620 2023-10-10T20:23:00.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.154+0000 7fd7177fe700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 0x7fd7040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.155+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fd70406ea50 con 0x7fd71815a620 2023-10-10T20:23:00.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.155+0000 7fd7177fe700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 0x7fd7040693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e4d0 tx=0x7fd708007680 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.159+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd70c0536b0 con 0x7fd71815a620 2023-10-10T20:23:00.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.380+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd71804c3c0 con 0x7fd71815a620 2023-10-10T20:23:00.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.381+0000 7fd6ff7fe700 1 -- 172.21.15.148:0/211843474 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fd70c015020 con 0x7fd71815a620 2023-10-10T20:23:00.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:00.384 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:23:00.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 msgr2=0x7fd7040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 0x7fd7040693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e4d0 tx=0x7fd708007680 comp rx=0 tx=0).stop 2023-10-10T20:23:00.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 msgr2=0x7fd71813f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd71806b530 tx=0x7fd70c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:00.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 shutdown_connections 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd704066f30 0x7fd7040693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd71815d580 0x7fd718152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd718157870 0x7fd718151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 --2- 172.21.15.148:0/211843474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd71815a620 0x7fd71813f960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 >> 172.21.15.148:0/211843474 conn(0x7fd718068ce0 msgr2=0x7fd71815e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 shutdown_connections 2023-10-10T20:23:00.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.383+0000 7fd71e01d700 1 -- 172.21.15.148:0/211843474 wait complete. 2023-10-10T20:23:00.388 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:23:00.398 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:00.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.511+0000 7f96008d9700 1 Processor -- start 2023-10-10T20:23:00.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.511+0000 7f96008d9700 1 -- start start 2023-10-10T20:23:00.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f96008d9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95fc157870 0x7f95fc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f96008d9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f95fc15a620 0x7f95fc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f96008d9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc15cbb0 con 0x7f95fc15d580 2023-10-10T20:23:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f96008d9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc15bfa0 con 0x7f95fc157870 2023-10-10T20:23:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f96008d9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc15c280 con 0x7f95fc15a620 2023-10-10T20:23:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95fc157870 0x7f95fc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f95fc157870 0x7f95fc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35970/0 (socket says 172.21.15.148:35970) 2023-10-10T20:23:00.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 -- 172.21.15.148:0/2107965000 learned_addr learned my addr 172.21.15.148:0/2107965000 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 -- 172.21.15.148:0/2107965000 >> v2:172.21.15.161:3300/0 conn(0x7f95fc15a620 msgr2=0x7f95fc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 --2- 172.21.15.148:0/2107965000 >> v2:172.21.15.161:3300/0 conn(0x7f95fc15a620 0x7f95fc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 -- 172.21.15.148:0/2107965000 >> v1:172.21.15.161:6789/0 conn(0x7f95fc15d580 legacy=0x7f95fc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 -- 172.21.15.148:0/2107965000 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc15c560 con 0x7f95fc157870 2023-10-10T20:23:00.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.512+0000 7f95fa59c700 1 --2- 172.21.15.148:0/2107965000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc157870 0x7f95fc15b6a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cb20 tx=0x7f95ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba504df6f31ebab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f95f9d9b700 1 -- 172.21.15.148:0/2107965000 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00b820 con 0x7f95fc157870 2023-10-10T20:23:00.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f95f9d9b700 1 -- 172.21.15.148:0/2107965000 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec00b980 con 0x7f95fc157870 2023-10-10T20:23:00.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f95f9d9b700 1 -- 172.21.15.148:0/2107965000 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00bd20 con 0x7f95fc157870 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 -- 172.21.15.148:0/2107965000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc157870 msgr2=0x7f95fc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 --2- 172.21.15.148:0/2107965000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc157870 0x7f95fc15b6a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cb20 tx=0x7f95ec009250 comp rx=0 tx=0).stop 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 -- 172.21.15.148:0/2107965000 shutdown_connections 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 --2- 172.21.15.148:0/2107965000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc157870 0x7f95fc15b6a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 --2- 172.21.15.148:0/2107965000 >> v2:172.21.15.161:3300/0 conn(0x7f95fc15a620 0x7f95fc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 -- 172.21.15.148:0/2107965000 >> 172.21.15.148:0/2107965000 conn(0x7f95fc068ce0 msgr2=0x7f95fc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 -- 172.21.15.148:0/2107965000 shutdown_connections 2023-10-10T20:23:00.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.513+0000 7f96008d9700 1 -- 172.21.15.148:0/2107965000 wait complete. 2023-10-10T20:23:00.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.514+0000 7f96008d9700 1 Processor -- start 2023-10-10T20:23:00.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.514+0000 7f96008d9700 1 -- start start 2023-10-10T20:23:00.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95fc157870 0x7f95fc16b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95fc15a620 0x7f95fc14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c6c70 con 0x7f95fc15d580 2023-10-10T20:23:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc16ba90 con 0x7f95fc157870 2023-10-10T20:23:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f96008d9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc14bd80 con 0x7f95fc15a620 2023-10-10T20:23:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95fad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95fad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35974/0 (socket says 172.21.15.148:35974) 2023-10-10T20:23:00.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95fad9d700 1 -- 172.21.15.148:0/2343757336 learned_addr learned my addr 172.21.15.148:0/2343757336 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95fa59c700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95fc157870 0x7f95fc16b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95f1d9b700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95fc15a620 0x7f95fc14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.515+0000 7f95fad9d700 1 -- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95fc15a620 msgr2=0x7f95fc14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95fad9d700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95fc15a620 0x7f95fc14b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95fad9d700 1 -- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95fc157870 msgr2=0x7f95fc16b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95fad9d700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95fc157870 0x7f95fc16b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95fad9d700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0094f0 con 0x7f95fc15d580 2023-10-10T20:23:00.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95fad9d700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95f400bb80 tx=0x7f95f4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400c970 con 0x7f95fc15d580 2023-10-10T20:23:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f400cad0 con 0x7f95fc15d580 2023-10-10T20:23:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0c6060 con 0x7f95fc15d580 2023-10-10T20:23:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400ce90 con 0x7f95fc15d580 2023-10-10T20:23:00.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.516+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc0c6620 con 0x7f95fc15d580 2023-10-10T20:23:00.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.517+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc04c3c0 con 0x7f95fc15d580 2023-10-10T20:23:00.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.517+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95f4014620 con 0x7f95fc15d580 2023-10-10T20:23:00.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.518+0000 7f95f2ffd700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.518+0000 7f95fa59c700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.518+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95f4084630 con 0x7f95fc15d580 2023-10-10T20:23:00.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.518+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f95dc06ea50 con 0x7f95fc15d580 2023-10-10T20:23:00.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.518+0000 7f95fa59c700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 0x7f95dc0693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cb20 tx=0x7f95ec016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.522+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95f404c5a0 con 0x7f95fc15d580 2023-10-10T20:23:00.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.748+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95fc15c840 con 0x7f95fc15d580 2023-10-10T20:23:00.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.748+0000 7f95f2ffd700 1 -- 172.21.15.148:0/2343757336 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f95f40518a0 con 0x7f95fc15d580 2023-10-10T20:23:00.751 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:00.751 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:21:26.994181+0000","modified":"2023-10-10T20:21:30.668566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1666843566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1666843566},{"type":"v1","addr":"172.21.15.148:6837","nonce":1666843566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-10T20:23:00.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 msgr2=0x7f95dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 0x7f95dc0693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15cb20 tx=0x7f95ec016000 comp rx=0 tx=0).stop 2023-10-10T20:23:00.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 msgr2=0x7f95fc0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95f400bb80 tx=0x7f95f4009810 comp rx=0 tx=0).stop 2023-10-10T20:23:00.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 shutdown_connections 2023-10-10T20:23:00.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f95dc066f30 0x7f95dc0693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95fc15d580 0x7f95fc0b2760 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95fc157870 0x7f95fc16b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 --2- 172.21.15.148:0/2343757336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95fc15a620 0x7f95fc14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 >> 172.21.15.148:0/2343757336 conn(0x7f95fc068ce0 msgr2=0x7f95fc0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 shutdown_connections 2023-10-10T20:23:00.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.750+0000 7f96008d9700 1 -- 172.21.15.148:0/2343757336 wait complete. 2023-10-10T20:23:00.755 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 32 2023-10-10T20:23:00.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:00.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 Processor -- start 2023-10-10T20:23:00.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 -- start start 2023-10-10T20:23:00.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf4159cf0 0x7f7bf415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bf415a0c0 0x7f7bf40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7bf40ce1a0 0x7f7bf40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf40cca20 con 0x7f7bf415a0c0 2023-10-10T20:23:00.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf415ea40 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.878+0000 7f7bf9e2a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4067660 con 0x7f7bf4159cf0 2023-10-10T20:23:00.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bf415a0c0 0x7f7bf40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bf415a0c0 0x7f7bf40ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35982/0 (socket says 172.21.15.148:35982) 2023-10-10T20:23:00.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 -- 172.21.15.148:0/2011784539 learned_addr learned my addr 172.21.15.148:0/2011784539 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf37fe700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf4159cf0 0x7f7bf415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf3fff700 1 --2- 172.21.15.148:0/2011784539 >> v2:172.21.15.161:3300/0 conn(0x7f7bf40ce1a0 0x7f7bf40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 -- 172.21.15.148:0/2011784539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf4159cf0 msgr2=0x7f7bf415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf4159cf0 0x7f7bf415e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 -- 172.21.15.148:0/2011784539 >> v2:172.21.15.161:3300/0 conn(0x7f7bf40ce1a0 msgr2=0x7f7bf40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/2011784539 >> v2:172.21.15.161:3300/0 conn(0x7f7bf40ce1a0 0x7f7bf40ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 -- 172.21.15.148:0/2011784539 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4067f00 con 0x7f7bf415a0c0 2023-10-10T20:23:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf415a0c0 0x7f7bf40ceab0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4071760 tx=0x7f7be8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa561f666f2361d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf27fc700 1 -- 172.21.15.148:0/2011784539 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be800b640 con 0x7f7bf415a0c0 2023-10-10T20:23:00.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf27fc700 1 -- 172.21.15.148:0/2011784539 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be800b7a0 con 0x7f7bf415a0c0 2023-10-10T20:23:00.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.879+0000 7f7bf27fc700 1 -- 172.21.15.148:0/2011784539 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be800bb60 con 0x7f7bf415a0c0 2023-10-10T20:23:00.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/2011784539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf415a0c0 msgr2=0x7f7bf40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf415a0c0 0x7f7bf40ceab0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4071760 tx=0x7f7be8009250 comp rx=0 tx=0).stop 2023-10-10T20:23:00.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/2011784539 shutdown_connections 2023-10-10T20:23:00.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/2011784539 >> v2:172.21.15.161:3300/0 conn(0x7f7bf40ce1a0 0x7f7bf40ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf4159cf0 0x7f7bf415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/2011784539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf415a0c0 0x7f7bf40ceab0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/2011784539 >> 172.21.15.148:0/2011784539 conn(0x7f7bf406c6e0 msgr2=0x7f7bf4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:00.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/2011784539 shutdown_connections 2023-10-10T20:23:00.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.880+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/2011784539 wait complete. 2023-10-10T20:23:00.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.881+0000 7f7bf9e2a700 1 Processor -- start 2023-10-10T20:23:00.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.881+0000 7f7bf9e2a700 1 -- start start 2023-10-10T20:23:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 0x7f7bf414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 0x7f7bf40c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4153ad0 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40c8a50 con 0x7f7bf4159cf0 2023-10-10T20:23:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf9e2a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40c81b0 con 0x7f7bf415a0c0 2023-10-10T20:23:00.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35996/0 (socket says 172.21.15.148:35996) 2023-10-10T20:23:00.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 -- 172.21.15.148:0/3149119082 learned_addr learned my addr 172.21.15.148:0/3149119082 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:00.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf3fff700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 0x7f7bf40c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 0x7f7bf414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 -- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 msgr2=0x7f7bf40c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 0x7f7bf40c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 -- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 msgr2=0x7f7bf414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:00.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 0x7f7bf414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:00.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf37fe700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80094f0 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf3fff700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 0x7f7bf40c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:00.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.882+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 0x7f7bf414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:00.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bf37fe700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ec80 tx=0x7f7be40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be400c9d0 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be400cb50 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf40c8490 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4013470 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.883+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf414fdf0 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.884+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf404c3c0 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.884+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7be4013860 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.884+0000 7f7bdb7fe700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 0x7f7be00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.884+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7be4084900 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.884+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 0x7f7be00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.885+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f7be006ea50 con 0x7f7bf40ce1a0 2023-10-10T20:23:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.885+0000 7f7bf2ffd700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 0x7f7be00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bf415a530 tx=0x7f7be8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:00.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:00.889+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7be4017060 con 0x7f7bf40ce1a0 2023-10-10T20:23:01.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.070+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf40cef00 con 0x7f7bf40ce1a0 2023-10-10T20:23:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.071+0000 7f7bdb7fe700 1 -- 172.21.15.148:0/3149119082 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15456 (secure 0 0 0) 0x7f7be4017090 con 0x7f7bf40ce1a0 2023-10-10T20:23:01.073 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:01.074 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":69,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:21:47.767303+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:21:24.814992+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-10T20:21:25.970062+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:01.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 msgr2=0x7f7be00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 0x7f7be00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bf415a530 tx=0x7f7be8017000 comp rx=0 tx=0).stop 2023-10-10T20:23:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 msgr2=0x7f7bf40c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ec80 tx=0x7f7be40091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 shutdown_connections 2023-10-10T20:23:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7be0066f30 0x7f7be00693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bf415a0c0 0x7f7bf40c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ce1a0 0x7f7bf40c7ad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 --2- 172.21.15.148:0/3149119082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bf4159cf0 0x7f7bf414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 >> 172.21.15.148:0/3149119082 conn(0x7f7bf406c6e0 msgr2=0x7f7bf40c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 shutdown_connections 2023-10-10T20:23:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.073+0000 7f7bf9e2a700 1 -- 172.21.15.148:0/3149119082 wait complete. 2023-10-10T20:23:01.088 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:23:01.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.202+0000 7f13c2230700 1 Processor -- start 2023-10-10T20:23:01.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.202+0000 7f13c2230700 1 -- start start 2023-10-10T20:23:01.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13c2230700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13bc15a2b0 0x7f13bc0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13c2230700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 0x7f13bc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13c2230700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13bc0c5d80 con 0x7f13bc0d1c30 2023-10-10T20:23:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13c2230700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13bc068470 con 0x7f13bc15a2b0 2023-10-10T20:23:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13c2230700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13bc0c6060 con 0x7f13bc0cfe20 2023-10-10T20:23:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13bc15a2b0 0x7f13bc0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13bc15a2b0 0x7f13bc0b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36002/0 (socket says 172.21.15.148:36002) 2023-10-10T20:23:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 -- 172.21.15.148:0/2268358352 learned_addr learned my addr 172.21.15.148:0/2268358352 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bbfff700 1 --2- 172.21.15.148:0/2268358352 >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 0x7f13bc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 -- 172.21.15.148:0/2268358352 >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 msgr2=0x7f13bc0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 --2- 172.21.15.148:0/2268358352 >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 0x7f13bc0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 -- 172.21.15.148:0/2268358352 >> v1:172.21.15.161:6789/0 conn(0x7f13bc0d1c30 legacy=0x7f13bc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bb7fe700 1 -- 172.21.15.148:0/2268358352 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc0c6340 con 0x7f13bc15a2b0 2023-10-10T20:23:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.203+0000 7f13bbfff700 1 --2- 172.21.15.148:0/2268358352 >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 0x7f13bc0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13bb7fe700 1 --2- 172.21.15.148:0/2268358352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0b3da0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13bc064970 tx=0x7f13ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd4bbee4fac3067 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13ba7fc700 1 -- 172.21.15.148:0/2268358352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00e070 con 0x7f13bc15a2b0 2023-10-10T20:23:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13ba7fc700 1 -- 172.21.15.148:0/2268358352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac00bad0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13ba7fc700 1 -- 172.21.15.148:0/2268358352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0092e0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 -- 172.21.15.148:0/2268358352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 msgr2=0x7f13bc0b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 --2- 172.21.15.148:0/2268358352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0b3da0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13bc064970 tx=0x7f13ac009840 comp rx=0 tx=0).stop 2023-10-10T20:23:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 -- 172.21.15.148:0/2268358352 shutdown_connections 2023-10-10T20:23:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 --2- 172.21.15.148:0/2268358352 >> v2:172.21.15.161:3300/0 conn(0x7f13bc0cfe20 0x7f13bc0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 --2- 172.21.15.148:0/2268358352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0b3da0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 -- 172.21.15.148:0/2268358352 >> 172.21.15.148:0/2268358352 conn(0x7f13bc067e40 msgr2=0x7f13bc0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:01.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 -- 172.21.15.148:0/2268358352 shutdown_connections 2023-10-10T20:23:01.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.204+0000 7f13c2230700 1 -- 172.21.15.148:0/2268358352 wait complete. 2023-10-10T20:23:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.205+0000 7f13c2230700 1 Processor -- start 2023-10-10T20:23:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.205+0000 7f13c2230700 1 -- start start 2023-10-10T20:23:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13bc0cfe20 0x7f13bc156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13bc0d1c30 0x7f13bc153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc157a80 con 0x7f13bc15a2b0 2023-10-10T20:23:01.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc154490 con 0x7f13bc0cfe20 2023-10-10T20:23:01.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13c2230700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc0c9d20 con 0x7f13bc0d1c30 2023-10-10T20:23:01.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13bbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13bbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36018/0 (socket says 172.21.15.148:36018) 2023-10-10T20:23:01.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13bbfff700 1 -- 172.21.15.148:0/3947676421 learned_addr learned my addr 172.21.15.148:0/3947676421 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:01.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13bb7fe700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13bc0cfe20 0x7f13bc156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.206+0000 7f13baffd700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13bc0d1c30 0x7f13bc153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 -- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13bc0d1c30 msgr2=0x7f13bc153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13bc0d1c30 0x7f13bc153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 -- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13bc0cfe20 msgr2=0x7f13bc156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13bc0cfe20 0x7f13bc156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac009ae0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13bbfff700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13b400d890 tx=0x7f13b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b40125f0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b4012770 con 0x7f13bc15a2b0 2023-10-10T20:23:01.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc15f190 con 0x7f13bc15a2b0 2023-10-10T20:23:01.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4012b30 con 0x7f13bc15a2b0 2023-10-10T20:23:01.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.207+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc1574c0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.208+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13bc15eff0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.208+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13b4013480 con 0x7f13bc15a2b0 2023-10-10T20:23:01.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.209+0000 7f13a37fe700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 0x7f13a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.209+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13b4084860 con 0x7f13bc15a2b0 2023-10-10T20:23:01.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.209+0000 7f13bb7fe700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 0x7f13a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.209+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f13a806ea50 con 0x7f13bc15a2b0 2023-10-10T20:23:01.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.209+0000 7f13bb7fe700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 0x7f13a80693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009fd0 tx=0x7f13ac018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.213+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13bc15eff0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.444+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f13bc15eff0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.446 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:01.444+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-10T20:23:01.447 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:01.444+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:23:01.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.446+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(70..70 src has 1..70) v4 ==== 382+0+0 (secure 0 0 0) 0x7f13b4055e90 con 0x7f13bc15a2b0 2023-10-10T20:23:01.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.446+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f13a806f3a0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.449+0000 7f13a37fe700 1 -- 172.21.15.148:0/3947676421 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13bc15eff0 con 0x7f13bc15a2b0 2023-10-10T20:23:01.453 INFO:tasks.ceph.mds.c.smithi148.stderr: -9279> 2023-10-10T20:21:32.607+0000 7fbcf7a93700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:23:01.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.451+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 msgr2=0x7f13a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.451+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 0x7f13a80693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009fd0 tx=0x7f13ac018000 comp rx=0 tx=0).stop 2023-10-10T20:23:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.451+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 msgr2=0x7f13bc0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.451+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13b400d890 tx=0x7f13b4008040 comp rx=0 tx=0).stop 2023-10-10T20:23:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 shutdown_connections 2023-10-10T20:23:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13a8066f30 0x7f13a80693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13bc15a2b0 0x7f13bc0c97e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13bc0cfe20 0x7f13bc156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 --2- 172.21.15.148:0/3947676421 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13bc0d1c30 0x7f13bc153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 >> 172.21.15.148:0/3947676421 conn(0x7f13bc067e40 msgr2=0x7f13bc0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:01.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 shutdown_connections 2023-10-10T20:23:01.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.452+0000 7f13c2230700 1 -- 172.21.15.148:0/3947676421 wait complete. 2023-10-10T20:23:01.457 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:23:01.466 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:23:01.483 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:23:01.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.583+0000 7f94734fc700 1 Processor -- start 2023-10-10T20:23:01.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.583+0000 7f94734fc700 1 -- start start 2023-10-10T20:23:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94734fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f946c070d20 0x7f946c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94734fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 0x7f946c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94734fc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c068120 con 0x7f946c070950 2023-10-10T20:23:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94734fc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c068400 con 0x7f946c070d20 2023-10-10T20:23:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94734fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c15a600 con 0x7f946c159700 2023-10-10T20:23:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f946c070d20 0x7f946c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f946c070d20 0x7f946c15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36048/0 (socket says 172.21.15.148:36048) 2023-10-10T20:23:01.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 -- 172.21.15.148:0/2264361343 learned_addr learned my addr 172.21.15.148:0/2264361343 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:01.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9472cfb700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 0x7f946c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 -- 172.21.15.148:0/2264361343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 msgr2=0x7f946c159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:01.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 0x7f946c159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 -- 172.21.15.148:0/2264361343 >> v1:172.21.15.161:6789/0 conn(0x7f946c070950 legacy=0x7f946c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 -- 172.21.15.148:0/2264361343 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c15a8e0 con 0x7f946c070d20 2023-10-10T20:23:01.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9472cfb700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 0x7f946c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:01.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f9471cf9700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c15ae00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f945c0099b0 tx=0x7f945c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d0856aec8cfd0b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94714f8700 1 -- 172.21.15.148:0/2264361343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00b860 con 0x7f946c070d20 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94714f8700 1 -- 172.21.15.148:0/2264361343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00b9c0 con 0x7f946c070d20 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.584+0000 7f94714f8700 1 -- 172.21.15.148:0/2264361343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00bd60 con 0x7f946c070d20 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 -- 172.21.15.148:0/2264361343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 msgr2=0x7f946c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c15ae00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f945c0099b0 tx=0x7f945c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:01.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 -- 172.21.15.148:0/2264361343 shutdown_connections 2023-10-10T20:23:01.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c159700 0x7f946c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 --2- 172.21.15.148:0/2264361343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c15ae00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 -- 172.21.15.148:0/2264361343 >> 172.21.15.148:0/2264361343 conn(0x7f946c0cd940 msgr2=0x7f946c0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:01.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 -- 172.21.15.148:0/2264361343 shutdown_connections 2023-10-10T20:23:01.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.585+0000 7f94734fc700 1 -- 172.21.15.148:0/2264361343 wait complete. 2023-10-10T20:23:01.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.586+0000 7f94734fc700 1 Processor -- start 2023-10-10T20:23:01.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.586+0000 7f94734fc700 1 -- start start 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c070950 0x7f946c16a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c159700 0x7f946c0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c4fb0 con 0x7f946c070d20 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0ca400 con 0x7f946c159700 2023-10-10T20:23:01.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c9b10 con 0x7f946c070950 2023-10-10T20:23:01.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36062/0 (socket says 172.21.15.148:36062) 2023-10-10T20:23:01.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 -- 172.21.15.148:0/3409001286 learned_addr learned my addr 172.21.15.148:0/3409001286 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:01.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94724fa700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c070950 0x7f946c16a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9472cfb700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c159700 0x7f946c0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 -- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c070950 msgr2=0x7f946c16a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c070950 0x7f946c16a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 -- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c159700 msgr2=0x7f946c0c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c159700 0x7f946c0c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:01.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c0094f0 con 0x7f946c070d20 2023-10-10T20:23:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f9471cf9700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f945c009290 tx=0x7f945c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c011e00 con 0x7f946c070d20 2023-10-10T20:23:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c01f070 con 0x7f946c070d20 2023-10-10T20:23:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.587+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c16af10 con 0x7f946c070d20 2023-10-10T20:23:01.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.588+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0266d0 con 0x7f946c070d20 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.588+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0c9db0 con 0x7f946c070d20 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.588+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0ca5b0 con 0x7f946c070d20 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.589+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f945c026a40 con 0x7f946c070d20 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.589+0000 7f94627fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 0x7f9450069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.589+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f945c021070 con 0x7f946c070d20 2023-10-10T20:23:01.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.589+0000 7f94724fa700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 0x7f9450069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:01.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.589+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f945006eb20 con 0x7f946c070d20 2023-10-10T20:23:01.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.590+0000 7f94724fa700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 0x7f9450069410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f946c1566a0 tx=0x7f9468009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:01.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.594+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f946c0ca5b0 con 0x7f946c070d20 2023-10-10T20:23:01.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:01.820+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f946c0ca570 con 0x7f946c070d20 2023-10-10T20:23:02.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.447+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(71..71 src has 1..71) v4 ==== 296+0+0 (secure 0 0 0) 0x7f945c057360 con 0x7f946c070d20 2023-10-10T20:23:02.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.447+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f945006f470 con 0x7f946c070d20 2023-10-10T20:23:02.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.453+0000 7f94627fc700 1 -- 172.21.15.148:0/3409001286 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f945c052fa0 con 0x7f946c070d20 2023-10-10T20:23:02.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 msgr2=0x7f9450069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 0x7f9450069410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f946c1566a0 tx=0x7f9468009040 comp rx=0 tx=0).stop 2023-10-10T20:23:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 msgr2=0x7f946c153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f945c009290 tx=0x7f945c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 shutdown_connections 2023-10-10T20:23:02.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9450066f60 0x7f9450069410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c159700 0x7f946c0c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c070950 0x7f946c16a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 --2- 172.21.15.148:0/3409001286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f946c070d20 0x7f946c153e50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 >> 172.21.15.148:0/3409001286 conn(0x7f946c0cd940 msgr2=0x7f946c067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:02.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 shutdown_connections 2023-10-10T20:23:02.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.456+0000 7f94734fc700 1 -- 172.21.15.148:0/3409001286 wait complete. 2023-10-10T20:23:02.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:23:02.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.629+0000 7ff0ba11f700 1 Processor -- start 2023-10-10T20:23:02.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.629+0000 7ff0ba11f700 1 -- start start 2023-10-10T20:23:02.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.629+0000 7ff0ba11f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff0b4141320 0x7ff0b41416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0ba11f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b40c9a50 0x7ff0b4070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0ba11f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9ec0 0x7ff0b4070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0ba11f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b415c850 con 0x7ff0b40c9a50 2023-10-10T20:23:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0ba11f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b415bc40 con 0x7ff0b4141320 2023-10-10T20:23:02.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0ba11f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415bf20 con 0x7ff0b40c9ec0 2023-10-10T20:23:02.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b40c9a50 0x7ff0b4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b40c9a50 0x7ff0b4070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44360/0 (socket says 172.21.15.148:44360) 2023-10-10T20:23:02.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1385531588 learned_addr learned my addr 172.21.15.148:0/1385531588 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:02.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b3fff700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9ec0 0x7ff0b4070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b37fe700 1 --2- 172.21.15.148:0/1385531588 >> v2:172.21.15.161:3300/0 conn(0x7ff0b4141320 0x7ff0b41416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1385531588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9ec0 msgr2=0x7ff0b4070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:02.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9ec0 0x7ff0b4070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1385531588 >> v2:172.21.15.161:3300/0 conn(0x7ff0b4141320 msgr2=0x7ff0b41416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:02.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/1385531588 >> v2:172.21.15.161:3300/0 conn(0x7ff0b4141320 0x7ff0b41416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 -- 172.21.15.148:0/1385531588 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b415c200 con 0x7ff0b40c9a50 2023-10-10T20:23:02.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b40c9a50 0x7ff0b4070700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff0b406e010 tx=0x7ff09c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa23457cbac38732 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:02.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b27fc700 1 -- 172.21.15.148:0/1385531588 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00b640 con 0x7ff0b40c9a50 2023-10-10T20:23:02.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b27fc700 1 -- 172.21.15.148:0/1385531588 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00b7a0 con 0x7ff0b40c9a50 2023-10-10T20:23:02.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.630+0000 7ff0b27fc700 1 -- 172.21.15.148:0/1385531588 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00bb60 con 0x7ff0b40c9a50 2023-10-10T20:23:02.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 -- 172.21.15.148:0/1385531588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b40c9a50 msgr2=0x7ff0b4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:02.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b40c9a50 0x7ff0b4070700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff0b406e010 tx=0x7ff09c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:02.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 -- 172.21.15.148:0/1385531588 shutdown_connections 2023-10-10T20:23:02.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9ec0 0x7ff0b4070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/1385531588 >> v2:172.21.15.161:3300/0 conn(0x7ff0b4141320 0x7ff0b41416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/1385531588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b40c9a50 0x7ff0b4070700 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 -- 172.21.15.148:0/1385531588 >> 172.21.15.148:0/1385531588 conn(0x7ff0b40cdae0 msgr2=0x7ff0b40cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:02.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 -- 172.21.15.148:0/1385531588 shutdown_connections 2023-10-10T20:23:02.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.631+0000 7ff0ba11f700 1 -- 172.21.15.148:0/1385531588 wait complete. 2023-10-10T20:23:02.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.632+0000 7ff0ba11f700 1 Processor -- start 2023-10-10T20:23:02.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.632+0000 7ff0ba11f700 1 -- start start 2023-10-10T20:23:02.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9a50 0x7ff0b40afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 0x7ff0b40c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c5df0 con 0x7ff0b4141320 2023-10-10T20:23:02.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c60d0 con 0x7ff0b40c9ec0 2023-10-10T20:23:02.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0ba11f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40c63b0 con 0x7ff0b40c9a50 2023-10-10T20:23:02.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44364/0 (socket says 172.21.15.148:44364) 2023-10-10T20:23:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 -- 172.21.15.148:0/2206141111 learned_addr learned my addr 172.21.15.148:0/2206141111 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 0x7ff0b40c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b37fe700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9a50 0x7ff0b40afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 -- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9a50 msgr2=0x7ff0b40afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:02.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9a50 0x7ff0b40afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 -- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 msgr2=0x7ff0b40c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 0x7ff0b40c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0094f0 con 0x7ff0b4141320 2023-10-10T20:23:02.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b2ffd700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 0x7ff0b40c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.633+0000 7ff0b3fff700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ea50 tx=0x7ff0a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.634+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400c9b0 con 0x7ff0b4141320 2023-10-10T20:23:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.634+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a400cb30 con 0x7ff0b4141320 2023-10-10T20:23:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.634+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40c6690 con 0x7ff0b4141320 2023-10-10T20:23:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.634+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4013470 con 0x7ff0b4141320 2023-10-10T20:23:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.634+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40c5b00 con 0x7ff0b4141320 2023-10-10T20:23:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b4167da0 con 0x7ff0b4141320 2023-10-10T20:23:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0a4013820 con 0x7ff0b4141320 2023-10-10T20:23:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0ab7fe700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff0a4084aa0 con 0x7ff0b4141320 2023-10-10T20:23:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0b37fe700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.635+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7ff09406eaf0 con 0x7ff0b4141320 2023-10-10T20:23:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.636+0000 7ff0b37fe700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 0x7ff0940693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac007600 tx=0x7ff0ac00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:02.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.640+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0a4017060 con 0x7ff0b4141320 2023-10-10T20:23:02.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:02.824+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0b404c3c0 con 0x7ff0b4141320 2023-10-10T20:23:03.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.588+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(72..72 src has 1..72) v4 ==== 288+0+0 (secure 0 0 0) 0x7ff0a4056080 con 0x7ff0b4141320 2023-10-10T20:23:03.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.588+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7ff09406cf20 con 0x7ff0b4141320 2023-10-10T20:23:03.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.591+0000 7ff0ab7fe700 1 -- 172.21.15.148:0/2206141111 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff0a4021070 con 0x7ff0b4141320 2023-10-10T20:23:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 msgr2=0x7ff0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 0x7ff0940693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac007600 tx=0x7ff0ac00b040 comp rx=0 tx=0).stop 2023-10-10T20:23:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 msgr2=0x7ff0b415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ea50 tx=0x7ff0a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 shutdown_connections 2023-10-10T20:23:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff094066f30 0x7ff0940693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b4141320 0x7ff0b415a020 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0b40c9a50 0x7ff0b40afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 --2- 172.21.15.148:0/2206141111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0b40c9ec0 0x7ff0b40c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.597+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 >> 172.21.15.148:0/2206141111 conn(0x7ff0b40cdae0 msgr2=0x7ff0b406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:03.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.598+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 shutdown_connections 2023-10-10T20:23:03.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.598+0000 7ff0ba11f700 1 -- 172.21.15.148:0/2206141111 wait complete. 2023-10-10T20:23:03.602 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:23:03.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:23:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.731+0000 7fe2985df700 1 Processor -- start 2023-10-10T20:23:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.731+0000 7fe2985df700 1 -- start start 2023-10-10T20:23:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe2985df700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe290159e50 0x7fe2900b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe2985df700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe2900b3370 0x7fe2900b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe2985df700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900b0d60 con 0x7fe2900b1730 2023-10-10T20:23:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe2985df700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900d15b0 con 0x7fe290159e50 2023-10-10T20:23:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe2985df700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900d1890 con 0x7fe2900b3370 2023-10-10T20:23:03.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe29637b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe290159e50 0x7fe2900b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:03.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.732+0000 7fe29637b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe290159e50 0x7fe2900b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44372/0 (socket says 172.21.15.148:44372) 2023-10-10T20:23:03.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 -- 172.21.15.148:0/4022788853 learned_addr learned my addr 172.21.15.148:0/4022788853 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:03.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 -- 172.21.15.148:0/4022788853 >> v2:172.21.15.161:3300/0 conn(0x7fe2900b3370 msgr2=0x7fe2900b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:03.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 --2- 172.21.15.148:0/4022788853 >> v2:172.21.15.161:3300/0 conn(0x7fe2900b3370 0x7fe2900b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 -- 172.21.15.148:0/4022788853 >> v1:172.21.15.161:6789/0 conn(0x7fe2900b1730 legacy=0x7fe290068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:03.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 -- 172.21.15.148:0/4022788853 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900b09f0 con 0x7fe290159e50 2023-10-10T20:23:03.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe29637b700 1 --2- 172.21.15.148:0/4022788853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900b2e30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe2900cb2a0 tx=0x7fe28c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27691f222f7b9ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe295379700 1 -- 172.21.15.148:0/4022788853 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00e070 con 0x7fe290159e50 2023-10-10T20:23:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe295379700 1 -- 172.21.15.148:0/4022788853 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00ba10 con 0x7fe290159e50 2023-10-10T20:23:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.733+0000 7fe295379700 1 -- 172.21.15.148:0/4022788853 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0092e0 con 0x7fe290159e50 2023-10-10T20:23:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 -- 172.21.15.148:0/4022788853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 msgr2=0x7fe2900b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 --2- 172.21.15.148:0/4022788853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900b2e30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe2900cb2a0 tx=0x7fe28c009840 comp rx=0 tx=0).stop 2023-10-10T20:23:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 -- 172.21.15.148:0/4022788853 shutdown_connections 2023-10-10T20:23:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 --2- 172.21.15.148:0/4022788853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900b2e30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 --2- 172.21.15.148:0/4022788853 >> v2:172.21.15.161:3300/0 conn(0x7fe2900b3370 0x7fe2900b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 -- 172.21.15.148:0/4022788853 >> 172.21.15.148:0/4022788853 conn(0x7fe290068130 msgr2=0x7fe2900d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 -- 172.21.15.148:0/4022788853 shutdown_connections 2023-10-10T20:23:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.734+0000 7fe2985df700 1 -- 172.21.15.148:0/4022788853 wait complete. 2023-10-10T20:23:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 Processor -- start 2023-10-10T20:23:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 -- start start 2023-10-10T20:23:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2900b1730 0x7fe2900ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 0x7fe290157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29016a490 con 0x7fe290159e50 2023-10-10T20:23:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900ca5d0 con 0x7fe2900b3370 2023-10-10T20:23:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.735+0000 7fe2985df700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe290158400 con 0x7fe2900b1730 2023-10-10T20:23:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44388/0 (socket says 172.21.15.148:44388) 2023-10-10T20:23:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 -- 172.21.15.148:0/749065021 learned_addr learned my addr 172.21.15.148:0/749065021 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:03.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe29637b700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2900b1730 0x7fe2900ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:03.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe295b7a700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 0x7fe290157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:03.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 -- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2900b1730 msgr2=0x7fe2900ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:03.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2900b1730 0x7fe2900ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 -- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 msgr2=0x7fe290157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 0x7fe290157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c009ae0 con 0x7fe290159e50 2023-10-10T20:23:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe295b7a700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 0x7fe290157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe296b7c700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b510 tx=0x7fe288006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f510 con 0x7fe290159e50 2023-10-10T20:23:03.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800f670 con 0x7fe290159e50 2023-10-10T20:23:03.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800fa30 con 0x7fe290159e50 2023-10-10T20:23:03.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900cb0b0 con 0x7fe290159e50 2023-10-10T20:23:03.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.736+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2900ca810 con 0x7fe290159e50 2023-10-10T20:23:03.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.737+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29016a600 con 0x7fe290159e50 2023-10-10T20:23:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe28801a400 con 0x7fe290159e50 2023-10-10T20:23:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe2867fc700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 0x7fe2740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fe288083d60 con 0x7fe290159e50 2023-10-10T20:23:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe29637b700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 0x7fe2740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fe27406dcf0 con 0x7fe290159e50 2023-10-10T20:23:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.738+0000 7fe29637b700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 0x7fe2740693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009880 tx=0x7fe28c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.742+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe29016a600 con 0x7fe290159e50 2023-10-10T20:23:03.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:03.925+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe29016a5d0 con 0x7fe290159e50 2023-10-10T20:23:04.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.590+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(73..73 src has 1..73) v4 ==== 666+0+0 (secure 0 0 0) 0x7fe288055570 con 0x7fe290159e50 2023-10-10T20:23:04.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.590+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fe27406ca80 con 0x7fe290159e50 2023-10-10T20:23:04.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.598+0000 7fe2867fc700 1 -- 172.21.15.148:0/749065021 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe2880511b0 con 0x7fe290159e50 2023-10-10T20:23:04.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 msgr2=0x7fe2740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 0x7fe2740693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009880 tx=0x7fe28c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:04.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 msgr2=0x7fe2900a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe28800b510 tx=0x7fe288006f90 comp rx=0 tx=0).stop 2023-10-10T20:23:04.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 shutdown_connections 2023-10-10T20:23:04.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe274066f30 0x7fe2740693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe290159e50 0x7fe2900a8a50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2900b1730 0x7fe2900ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 --2- 172.21.15.148:0/749065021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2900b3370 0x7fe290157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.604+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 >> 172.21.15.148:0/749065021 conn(0x7fe290068130 msgr2=0x7fe2900cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.605+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 shutdown_connections 2023-10-10T20:23:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.605+0000 7fe2985df700 1 -- 172.21.15.148:0/749065021 wait complete. 2023-10-10T20:23:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:23:04.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:23:04.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.735+0000 7f7d670f3700 1 Processor -- start 2023-10-10T20:23:04.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.735+0000 7f7d670f3700 1 -- start start 2023-10-10T20:23:04.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d670f3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7d600d05a0 0x7f7d60164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:04.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d670f3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d600b0790 con 0x7f7d6015a290 2023-10-10T20:23:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d670f3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d600ce1f0 con 0x7f7d60159ec0 2023-10-10T20:23:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d670f3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d600b0b20 con 0x7f7d600d05a0 2023-10-10T20:23:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d57fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7d6015a290 0x7f7d6006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39200/0 (socket says 172.21.15.148:39200) 2023-10-10T20:23:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d57fff700 1 -- 172.21.15.148:0/1649000567 learned_addr learned my addr 172.21.15.148:0/1649000567 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.736+0000 7f7d65690700 1 --2- 172.21.15.148:0/1649000567 >> v2:172.21.15.161:3300/0 conn(0x7f7d600d05a0 0x7f7d60164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425045717 0 0) 0x7f7d600b0790 con 0x7f7d6015a290 2023-10-10T20:23:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d48003570 con 0x7f7d6015a290 2023-10-10T20:23:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408949527 0 0) 0x7f7d600ce1f0 con 0x7f7d60159ec0 2023-10-10T20:23:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d600b0790 con 0x7f7d60159ec0 2023-10-10T20:23:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449789573 0 0) 0x7f7d48003570 con 0x7f7d6015a290 2023-10-10T20:23:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d65690700 1 -- 172.21.15.148:0/1649000567 >> v1:172.21.15.161:6789/0 conn(0x7f7d60159ec0 legacy=0x7f7d600af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d65690700 1 -- 172.21.15.148:0/1649000567 >> v1:172.21.15.148:6789/0 conn(0x7f7d6015a290 legacy=0x7f7d6006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d65690700 1 -- 172.21.15.148:0/1649000567 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600b04b0 con 0x7f7d600d05a0 2023-10-10T20:23:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d65690700 1 --2- 172.21.15.148:0/1649000567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d60164ed0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d580098e0 tx=0x7f7d580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2823d7fdf215d5c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50001470 con 0x7f7d600d05a0 2023-10-10T20:23:04.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58011420 con 0x7f7d600d05a0 2023-10-10T20:23:04.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.737+0000 7f7d577fe700 1 -- 172.21.15.148:0/1649000567 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580117c0 con 0x7f7d600d05a0 2023-10-10T20:23:04.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 -- 172.21.15.148:0/1649000567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 msgr2=0x7f7d60164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 --2- 172.21.15.148:0/1649000567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d60164ed0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d580098e0 tx=0x7f7d580091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 -- 172.21.15.148:0/1649000567 shutdown_connections 2023-10-10T20:23:04.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 --2- 172.21.15.148:0/1649000567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d60164ed0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 -- 172.21.15.148:0/1649000567 >> 172.21.15.148:0/1649000567 conn(0x7f7d600d1880 msgr2=0x7f7d600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:04.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 -- 172.21.15.148:0/1649000567 shutdown_connections 2023-10-10T20:23:04.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.738+0000 7f7d670f3700 1 -- 172.21.15.148:0/1649000567 wait complete. 2023-10-10T20:23:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.739+0000 7f7d670f3700 1 Processor -- start 2023-10-10T20:23:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.739+0000 7f7d670f3700 1 -- start start 2023-10-10T20:23:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d600c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d60159ec0 0x7f7d6016abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:04.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:04.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600c8420 con 0x7f7d6015a290 2023-10-10T20:23:04.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600b4670 con 0x7f7d600d05a0 2023-10-10T20:23:04.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d670f3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6016b120 con 0x7f7d60159ec0 2023-10-10T20:23:04.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44398/0 (socket says 172.21.15.148:44398) 2023-10-10T20:23:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 -- 172.21.15.148:0/368882407 learned_addr learned my addr 172.21.15.148:0/368882407 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d64e8f700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d600c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d57fff700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d60159ec0 0x7f7d6016abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 -- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d60159ec0 msgr2=0x7f7d6016abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d60159ec0 0x7f7d6016abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 -- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 msgr2=0x7f7d600c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d600c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 -- 172.21.15.148:0/368882407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58009420 con 0x7f7d6015a290 2023-10-10T20:23:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.740+0000 7f7d65690700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7d580093f0 tx=0x7f7d58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:04.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5801cca0 con 0x7f7d6015a290 2023-10-10T20:23:04.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5801ce00 con 0x7f7d6015a290 2023-10-10T20:23:04.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5801a6a0 con 0x7f7d6015a290 2023-10-10T20:23:04.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d600c7e60 con 0x7f7d6015a290 2023-10-10T20:23:04.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d600b3fb0 con 0x7f7d6015a290 2023-10-10T20:23:04.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.741+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d600b3e10 con 0x7f7d6015a290 2023-10-10T20:23:04.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.742+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d5801aa10 con 0x7f7d6015a290 2023-10-10T20:23:04.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.742+0000 7f7d54ff9700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 0x7f7d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:04.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.742+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7d58085f20 con 0x7f7d6015a290 2023-10-10T20:23:04.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.742+0000 7f7d64e8f700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 0x7f7d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:04.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.743+0000 7f7d64e8f700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 0x7f7d4c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c004140 tx=0x7f7d5c007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:04.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.747+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d600b3e10 con 0x7f7d6015a290 2023-10-10T20:23:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.928+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7d600b3e10 con 0x7f7d6015a290 2023-10-10T20:23:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.932+0000 7f7d54ff9700 1 -- 172.21.15.148:0/368882407 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d600b3e10 con 0x7f7d6015a290 2023-10-10T20:23:04.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 msgr2=0x7f7d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 0x7f7d4c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c004140 tx=0x7f7d5c007f90 comp rx=0 tx=0).stop 2023-10-10T20:23:04.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 msgr2=0x7f7d6015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:04.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7d580093f0 tx=0x7f7d58016000 comp rx=0 tx=0).stop 2023-10-10T20:23:04.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 shutdown_connections 2023-10-10T20:23:04.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d4c066f30 0x7f7d4c0693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6015a290 0x7f7d6015e3b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d600d05a0 0x7f7d600c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 --2- 172.21.15.148:0/368882407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d60159ec0 0x7f7d6016abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 >> 172.21.15.148:0/368882407 conn(0x7f7d600d1880 msgr2=0x7f7d6006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:04.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 shutdown_connections 2023-10-10T20:23:04.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:04.934+0000 7f7d670f3700 1 -- 172.21.15.148:0/368882407 wait complete. 2023-10-10T20:23:04.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-10T20:23:05.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.060+0000 7f7a19065700 1 Processor -- start 2023-10-10T20:23:05.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.060+0000 7f7a19065700 1 -- start start 2023-10-10T20:23:05.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a19065700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 0x7f7a14158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a19065700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1415cbb0 con 0x7f7a1415a620 2023-10-10T20:23:05.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a19065700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1415bfa0 con 0x7f7a14157870 2023-10-10T20:23:05.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a19065700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1415bdb0 con 0x7f7a14157c40 2023-10-10T20:23:05.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1259c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7a1415a620 0x7f7a1415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39216/0 (socket says 172.21.15.148:39216) 2023-10-10T20:23:05.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1259c700 1 -- 172.21.15.148:0/3051539292 learned_addr learned my addr 172.21.15.148:0/3051539292 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:05.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1359e700 1 --2- 172.21.15.148:0/3051539292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 0x7f7a14158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121767245 0 0) 0x7f7a1415cbb0 con 0x7f7a1415a620 2023-10-10T20:23:05.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79fc003570 con 0x7f7a1415a620 2023-10-10T20:23:05.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149032056 0 0) 0x7f7a1415bfa0 con 0x7f7a14157870 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a1415cbb0 con 0x7f7a14157870 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4290623025 0 0) 0x7f79fc003570 con 0x7f7a1415a620 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1359e700 1 -- 172.21.15.148:0/3051539292 >> v1:172.21.15.161:6789/0 conn(0x7f7a14157870 legacy=0x7f7a1415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1359e700 1 -- 172.21.15.148:0/3051539292 >> v1:172.21.15.148:6789/0 conn(0x7f7a1415a620 legacy=0x7f7a1415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1359e700 1 -- 172.21.15.148:0/3051539292 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1415c280 con 0x7f7a14157c40 2023-10-10T20:23:05.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.061+0000 7f7a1359e700 1 --2- 172.21.15.148:0/3051539292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 0x7f7a14158110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009e90 tx=0x7f7a0c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ebae6ac464fc5f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00e070 con 0x7f7a14157c40 2023-10-10T20:23:05.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00b870 con 0x7f7a14157c40 2023-10-10T20:23:05.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a11d9b700 1 -- 172.21.15.148:0/3051539292 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08001470 con 0x7f7a14157c40 2023-10-10T20:23:05.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 -- 172.21.15.148:0/3051539292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 msgr2=0x7f7a14158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 --2- 172.21.15.148:0/3051539292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 0x7f7a14158110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009e90 tx=0x7f7a0c009780 comp rx=0 tx=0).stop 2023-10-10T20:23:05.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 -- 172.21.15.148:0/3051539292 shutdown_connections 2023-10-10T20:23:05.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 --2- 172.21.15.148:0/3051539292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a14157c40 0x7f7a14158110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 -- 172.21.15.148:0/3051539292 >> 172.21.15.148:0/3051539292 conn(0x7f7a14068ce0 msgr2=0x7f7a140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:05.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 -- 172.21.15.148:0/3051539292 shutdown_connections 2023-10-10T20:23:05.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.062+0000 7f7a19065700 1 -- 172.21.15.148:0/3051539292 wait complete. 2023-10-10T20:23:05.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.063+0000 7f7a19065700 1 Processor -- start 2023-10-10T20:23:05.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 -- start start 2023-10-10T20:23:05.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 0x7f7a1414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a1415a620 0x7f7a14152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1414b8a0 con 0x7f7a14157870 2023-10-10T20:23:05.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140b0760 con 0x7f7a14157c40 2023-10-10T20:23:05.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a19065700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14151d70 con 0x7f7a1415a620 2023-10-10T20:23:05.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a12d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a12d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44406/0 (socket says 172.21.15.148:44406) 2023-10-10T20:23:05.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a12d9d700 1 -- 172.21.15.148:0/1311693841 learned_addr learned my addr 172.21.15.148:0/1311693841 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:05.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a1259c700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 0x7f7a1414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.064+0000 7f7a1359e700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a1415a620 0x7f7a14152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 -- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a1415a620 msgr2=0x7f7a14152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a1415a620 0x7f7a14152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 -- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 msgr2=0x7f7a1414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 0x7f7a1414b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 -- 172.21.15.148:0/1311693841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c009a20 con 0x7f7a14157870 2023-10-10T20:23:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a1259c700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 0x7f7a1414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a12d9d700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c790 tx=0x7f7a04007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04010510 con 0x7f7a14157870 2023-10-10T20:23:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04010670 con 0x7f7a14157870 2023-10-10T20:23:05.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a14152050 con 0x7f7a14157870 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04010a10 con 0x7f7a14157870 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.065+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a14152330 con 0x7f7a14157870 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.066+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a140b0c40 con 0x7f7a14157870 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.066+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a0401b400 con 0x7f7a14157870 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.067+0000 7f79faffd700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 0x7f7a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.067+0000 7f7a1259c700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 0x7f7a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.067+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7a04084000 con 0x7f7a14157870 2023-10-10T20:23:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.067+0000 7f7a1259c700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 0x7f7a000693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a080015a0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.071+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a04051570 con 0x7f7a14157870 2023-10-10T20:23:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.334+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f7a1404c3c0 con 0x7f7a14157870 2023-10-10T20:23:05.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.588+0000 7f79faffd700 1 -- 172.21.15.148:0/1311693841 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v195) v1 ==== 134+0+0 (secure 0 0 0) 0x7f7a04054b90 con 0x7f7a14157870 2023-10-10T20:23:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 msgr2=0x7f7a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 0x7f7a000693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a080015a0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2023-10-10T20:23:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 msgr2=0x7f7a140b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c790 tx=0x7f7a04007e10 comp rx=0 tx=0).stop 2023-10-10T20:23:05.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 shutdown_connections 2023-10-10T20:23:05.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7a00066f30 0x7f7a000693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a1415a620 0x7f7a14152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a14157870 0x7f7a140b3140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 --2- 172.21.15.148:0/1311693841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a14157c40 0x7f7a1414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.590+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 >> 172.21.15.148:0/1311693841 conn(0x7f7a14068ce0 msgr2=0x7f7a140690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:05.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.591+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 shutdown_connections 2023-10-10T20:23:05.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.591+0000 7f7a19065700 1 -- 172.21.15.148:0/1311693841 wait complete. 2023-10-10T20:23:05.605 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-10T20:23:05.605 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-10-10T20:23:05.606 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-10-10T20:23:05.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-10T20:23:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.719+0000 7fa202aa0700 1 Processor -- start 2023-10-10T20:23:05.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.719+0000 7fa202aa0700 1 -- start start 2023-10-10T20:23:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1fc0b1220 0x7fa1fc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 0x7fa1fc0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc15de60 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc15e140 con 0x7fa1fc0b1220 2023-10-10T20:23:05.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa202aa0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15e420 con 0x7fa1fc06dad0 2023-10-10T20:23:05.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa20083c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa20083c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44414/0 (socket says 172.21.15.148:44414) 2023-10-10T20:23:05.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.720+0000 7fa20083c700 1 -- 172.21.15.148:0/2858866279 learned_addr learned my addr 172.21.15.148:0/2858866279 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa1fbfff700 1 --2- 172.21.15.148:0/2858866279 >> v2:172.21.15.161:3300/0 conn(0x7fa1fc0b1220 0x7fa1fc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20103d700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 0x7fa1fc0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 -- 172.21.15.148:0/2858866279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 msgr2=0x7fa1fc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 0x7fa1fc0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 -- 172.21.15.148:0/2858866279 >> v2:172.21.15.161:3300/0 conn(0x7fa1fc0b1220 msgr2=0x7fa1fc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:05.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 --2- 172.21.15.148:0/2858866279 >> v2:172.21.15.161:3300/0 conn(0x7fa1fc0b1220 0x7fa1fc06d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 -- 172.21.15.148:0/2858866279 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc15e700 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20103d700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 0x7fa1fc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:05.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa20083c700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06cd30 tx=0x7fa1ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e06cd3128f90bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa1fb7fe700 1 -- 172.21.15.148:0/2858866279 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00e070 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa1fb7fe700 1 -- 172.21.15.148:0/2858866279 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00ba20 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa1fb7fe700 1 -- 172.21.15.148:0/2858866279 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0092e0 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa202aa0700 1 -- 172.21.15.148:0/2858866279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc0b0e50 msgr2=0x7fa1fc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.721+0000 7fa202aa0700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06cd30 tx=0x7fa1ec009840 comp rx=0 tx=0).stop 2023-10-10T20:23:05.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 -- 172.21.15.148:0/2858866279 shutdown_connections 2023-10-10T20:23:05.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc06dad0 0x7fa1fc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 --2- 172.21.15.148:0/2858866279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc0b0e50 0x7fa1fc0c9180 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 --2- 172.21.15.148:0/2858866279 >> v2:172.21.15.161:3300/0 conn(0x7fa1fc0b1220 0x7fa1fc06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 -- 172.21.15.148:0/2858866279 >> 172.21.15.148:0/2858866279 conn(0x7fa1fc069870 msgr2=0x7fa1fc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 -- 172.21.15.148:0/2858866279 shutdown_connections 2023-10-10T20:23:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.722+0000 7fa202aa0700 1 -- 172.21.15.148:0/2858866279 wait complete. 2023-10-10T20:23:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.723+0000 7fa202aa0700 1 Processor -- start 2023-10-10T20:23:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.723+0000 7fa202aa0700 1 -- start start 2023-10-10T20:23:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc0b0e50 0x7fa1fc0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 0x7fa1fc156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c75c0 con 0x7fa1fc06dad0 2023-10-10T20:23:05.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc153af0 con 0x7fa1fc0b1220 2023-10-10T20:23:05.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc16e0b0 con 0x7fa1fc0b0e50 2023-10-10T20:23:05.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44426/0 (socket says 172.21.15.148:44426) 2023-10-10T20:23:05.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 -- 172.21.15.148:0/1646315941 learned_addr learned my addr 172.21.15.148:0/1646315941 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20103d700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 0x7fa1fc156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa1fbfff700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc0b0e50 0x7fa1fc0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 -- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc0b0e50 msgr2=0x7fa1fc0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc0b0e50 0x7fa1fc0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 -- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 msgr2=0x7fa1fc156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:05.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 0x7fa1fc156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:05.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 -- 172.21.15.148:0/1646315941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009ae0 con 0x7fa1fc06dad0 2023-10-10T20:23:05.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20103d700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 0x7fa1fc156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:05.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa20083c700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b6c0 tx=0x7fa1ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec007820 con 0x7fa1fc06dad0 2023-10-10T20:23:05.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0079a0 con 0x7fa1fc06dad0 2023-10-10T20:23:05.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc16e390 con 0x7fa1fc06dad0 2023-10-10T20:23:05.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec007d60 con 0x7fa1fc06dad0 2023-10-10T20:23:05.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.724+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc16e950 con 0x7fa1fc06dad0 2023-10-10T20:23:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.725+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc04c3c0 con 0x7fa1fc06dad0 2023-10-10T20:23:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.726+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1ec01d420 con 0x7fa1fc06dad0 2023-10-10T20:23:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.726+0000 7fa1f8ff9700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 0x7fa1e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.726+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa1ec085e00 con 0x7fa1fc06dad0 2023-10-10T20:23:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.726+0000 7fa1fbfff700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 0x7fa1e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.726+0000 7fa1fbfff700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 0x7fa1e80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc16dc50 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.730+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1ec0533b0 con 0x7fa1fc06dad0 2023-10-10T20:23:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:05.986+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7fa1fc157530 con 0x7fa1fc06dad0 2023-10-10T20:23:06.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.590+0000 7fa1f8ff9700 1 -- 172.21.15.148:0/1646315941 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v196) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa1ec056990 con 0x7fa1fc06dad0 2023-10-10T20:23:06.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 msgr2=0x7fa1e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:06.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 0x7fa1e80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc16dc50 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2023-10-10T20:23:06.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 msgr2=0x7fa1fc145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:06.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b6c0 tx=0x7fa1ec018000 comp rx=0 tx=0).stop 2023-10-10T20:23:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 shutdown_connections 2023-10-10T20:23:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa1e8066f30 0x7fa1e80693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1fc0b1220 0x7fa1fc156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1fc06dad0 0x7fa1fc145540 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 --2- 172.21.15.148:0/1646315941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1fc0b0e50 0x7fa1fc0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 >> 172.21.15.148:0/1646315941 conn(0x7fa1fc069870 msgr2=0x7fa1fc0c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 shutdown_connections 2023-10-10T20:23:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.592+0000 7fa202aa0700 1 -- 172.21.15.148:0/1646315941 wait complete. 2023-10-10T20:23:06.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-10T20:23:06.739 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_objectstore":"memstore"} 2023-10-10T20:23:06.740 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-10T20:23:06.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:23:06.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.855+0000 7f029fce7700 1 Processor -- start 2023-10-10T20:23:06.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.855+0000 7f029fce7700 1 -- start start 2023-10-10T20:23:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0298159ec0 0x7f02980af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f029815a290 0x7f02980afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029806a5a0 0x7f029806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029806af60 con 0x7f029815a290 2023-10-10T20:23:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980b0790 con 0x7f0298159ec0 2023-10-10T20:23:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029fce7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980ce1f0 con 0x7f029806a5a0 2023-10-10T20:23:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029d282700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f029815a290 0x7f02980afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029d282700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f029815a290 0x7f02980afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44430/0 (socket says 172.21.15.148:44430) 2023-10-10T20:23:06.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.856+0000 7f029d282700 1 -- 172.21.15.148:0/3370911645 learned_addr learned my addr 172.21.15.148:0/3370911645 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029e284700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029806a5a0 0x7f029806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029da83700 1 --2- 172.21.15.148:0/3370911645 >> v2:172.21.15.161:3300/0 conn(0x7f0298159ec0 0x7f02980af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 -- 172.21.15.148:0/3370911645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029806a5a0 msgr2=0x7f029806aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029806a5a0 0x7f029806aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 -- 172.21.15.148:0/3370911645 >> v2:172.21.15.161:3300/0 conn(0x7f0298159ec0 msgr2=0x7f02980af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:06.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 --2- 172.21.15.148:0/3370911645 >> v2:172.21.15.161:3300/0 conn(0x7f0298159ec0 0x7f02980af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 -- 172.21.15.148:0/3370911645 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980b0b20 con 0x7f029815a290 2023-10-10T20:23:06.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029d282700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029815a290 0x7f02980afe80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02980898e0 tx=0x7f0288009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b0d3558b423024 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:06.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029ca81700 1 -- 172.21.15.148:0/3370911645 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800b860 con 0x7f029815a290 2023-10-10T20:23:06.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029ca81700 1 -- 172.21.15.148:0/3370911645 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028800b9c0 con 0x7f029815a290 2023-10-10T20:23:06.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029ca81700 1 -- 172.21.15.148:0/3370911645 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800bd60 con 0x7f029815a290 2023-10-10T20:23:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029fce7700 1 -- 172.21.15.148:0/3370911645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029815a290 msgr2=0x7f02980afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.857+0000 7f029fce7700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029815a290 0x7f02980afe80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f02980898e0 tx=0x7f0288009250 comp rx=0 tx=0).stop 2023-10-10T20:23:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 -- 172.21.15.148:0/3370911645 shutdown_connections 2023-10-10T20:23:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029806a5a0 0x7f029806aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 --2- 172.21.15.148:0/3370911645 >> v2:172.21.15.161:3300/0 conn(0x7f0298159ec0 0x7f02980af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 --2- 172.21.15.148:0/3370911645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f029815a290 0x7f02980afe80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 -- 172.21.15.148:0/3370911645 >> 172.21.15.148:0/3370911645 conn(0x7f02980d1880 msgr2=0x7f02980cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:06.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 -- 172.21.15.148:0/3370911645 shutdown_connections 2023-10-10T20:23:06.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.858+0000 7f029fce7700 1 -- 172.21.15.148:0/3370911645 wait complete. 2023-10-10T20:23:06.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 Processor -- start 2023-10-10T20:23:06.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 -- start start 2023-10-10T20:23:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029806a5a0 0x7f02980b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029815a290 0x7f02980b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298153d00 con 0x7f0298159ec0 2023-10-10T20:23:06.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029815abc0 con 0x7f029806a5a0 2023-10-10T20:23:06.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029fce7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980b4260 con 0x7f029815a290 2023-10-10T20:23:06.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.859+0000 7f029d282700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44432/0 (socket says 172.21.15.148:44432) 2023-10-10T20:23:06.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 -- 172.21.15.148:0/70308534 learned_addr learned my addr 172.21.15.148:0/70308534 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:06.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029da83700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029806a5a0 0x7f02980b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029e284700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029815a290 0x7f02980b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 -- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029815a290 msgr2=0x7f02980b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:06.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029815a290 0x7f02980b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 -- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029806a5a0 msgr2=0x7f02980b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:06.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029806a5a0 0x7f02980b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:06.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 -- 172.21.15.148:0/70308534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880094f0 con 0x7f0298159ec0 2023-10-10T20:23:06.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029d282700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0298071eb0 tx=0x7f0288017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:06.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288011e40 con 0x7f0298159ec0 2023-10-10T20:23:06.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028801a450 con 0x7f0298159ec0 2023-10-10T20:23:06.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029815d5c0 con 0x7f0298159ec0 2023-10-10T20:23:06.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028801a810 con 0x7f0298159ec0 2023-10-10T20:23:06.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.860+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029815d2e0 con 0x7f0298159ec0 2023-10-10T20:23:06.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.861+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029804c3c0 con 0x7f0298159ec0 2023-10-10T20:23:06.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.862+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f028801abc0 con 0x7f0298159ec0 2023-10-10T20:23:06.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.862+0000 7f028dffb700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:06.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.862+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0288085d30 con 0x7f0298159ec0 2023-10-10T20:23:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.862+0000 7f029da83700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.863+0000 7f029da83700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0294009fd0 tx=0x7f0294009880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:06.866+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02880532a0 con 0x7f0298159ec0 2023-10-10T20:23:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.048+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02980b04b0 con 0x7f0298159ec0 2023-10-10T20:23:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.051+0000 7f028dffb700 1 -- 172.21.15.148:0/70308534 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02880568c0 con 0x7f0298159ec0 2023-10-10T20:23:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 msgr2=0x7f027c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0294009fd0 tx=0x7f0294009880 comp rx=0 tx=0).stop 2023-10-10T20:23:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 msgr2=0x7f029815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0298071eb0 tx=0x7f0288017000 comp rx=0 tx=0).stop 2023-10-10T20:23:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 shutdown_connections 2023-10-10T20:23:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f029815a290 0x7f02980b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029806a5a0 0x7f02980b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 --2- 172.21.15.148:0/70308534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0298159ec0 0x7f029815a680 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 >> 172.21.15.148:0/70308534 conn(0x7f02980d1880 msgr2=0x7f02981541f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 shutdown_connections 2023-10-10T20:23:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.054+0000 7f029fce7700 1 -- 172.21.15.148:0/70308534 wait complete. 2023-10-10T20:23:07.068 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:23:07.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:23:07.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:23:07.150 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:23:07.150 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:23:07.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:07.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.263+0000 7f7bbf466700 1 Processor -- start 2023-10-10T20:23:07.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.263+0000 7f7bbf466700 1 -- start start 2023-10-10T20:23:07.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbf466700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbf466700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7bb815a620 0x7f7bb815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbf466700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb815cbb0 con 0x7f7bb815d580 2023-10-10T20:23:07.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbf466700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb815bfa0 con 0x7f7bb815a620 2023-10-10T20:23:07.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbf466700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb815c280 con 0x7f7bb8157870 2023-10-10T20:23:07.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbd202700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbca01700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7bb815a620 0x7f7bb815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbda03700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7bb815d580 0x7f7bb8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48416/0 (socket says 172.21.15.148:48416) 2023-10-10T20:23:07.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbda03700 1 -- 172.21.15.148:0/2799687155 learned_addr learned my addr 172.21.15.148:0/2799687155 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.264+0000 7f7bbca01700 1 -- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 msgr2=0x7f7bb815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7bbca01700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7bbca01700 1 -- 172.21.15.148:0/2799687155 >> v1:172.21.15.161:6789/0 conn(0x7f7bb815d580 legacy=0x7f7bb8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7baffff700 1 -- 172.21.15.148:0/2799687155 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962061792 0 0) 0x7f7bb815cbb0 con 0x7f7bb815d580 2023-10-10T20:23:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7bbca01700 1 -- 172.21.15.148:0/2799687155 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb815c560 con 0x7f7bb815a620 2023-10-10T20:23:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7bbd202700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:07.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7bbca01700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb815d040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d23e76debce5e19c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7baffff700 1 -- 172.21.15.148:0/2799687155 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800b640 con 0x7f7bb815a620 2023-10-10T20:23:07.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7baffff700 1 -- 172.21.15.148:0/2799687155 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800b7a0 con 0x7f7bb815a620 2023-10-10T20:23:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.265+0000 7f7baffff700 1 -- 172.21.15.148:0/2799687155 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800bb40 con 0x7f7bb815a620 2023-10-10T20:23:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 -- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 msgr2=0x7f7bb815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb815d040 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 -- 172.21.15.148:0/2799687155 shutdown_connections 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2799687155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb815d040 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 -- 172.21.15.148:0/2799687155 >> 172.21.15.148:0/2799687155 conn(0x7f7bb8068ce0 msgr2=0x7f7bb80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 -- 172.21.15.148:0/2799687155 shutdown_connections 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.266+0000 7f7bbf466700 1 -- 172.21.15.148:0/2799687155 wait complete. 2023-10-10T20:23:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.267+0000 7f7bbf466700 1 Processor -- start 2023-10-10T20:23:07.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.267+0000 7f7bbf466700 1 -- start start 2023-10-10T20:23:07.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb80b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b63b0 con 0x7f7bb815d580 2023-10-10T20:23:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b57a0 con 0x7f7bb815a620 2023-10-10T20:23:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbf466700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b5a80 con 0x7f7bb8157870 2023-10-10T20:23:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44434/0 (socket says 172.21.15.148:44434) 2023-10-10T20:23:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 -- 172.21.15.148:0/2850239087 learned_addr learned my addr 172.21.15.148:0/2850239087 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbd202700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb80b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbca01700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 -- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 msgr2=0x7f7bb80b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb80b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 -- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 msgr2=0x7f7bb80b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb80b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 -- 172.21.15.148:0/2850239087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4009420 con 0x7f7bb815d580 2023-10-10T20:23:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.268+0000 7f7bbda03700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000cbc0 tx=0x7f7bb0001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.269+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000a7e0 con 0x7f7bb815d580 2023-10-10T20:23:07.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.269+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb000a940 con 0x7f7bb815d580 2023-10-10T20:23:07.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.269+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8009420 con 0x7f7bb815d580 2023-10-10T20:23:07.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.269+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000ad00 con 0x7f7bb815d580 2023-10-10T20:23:07.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.269+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80b6040 con 0x7f7bb815d580 2023-10-10T20:23:07.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.270+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb80b6520 con 0x7f7bb815d580 2023-10-10T20:23:07.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.270+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bb0007300 con 0x7f7bb815d580 2023-10-10T20:23:07.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.270+0000 7f7bad7fa700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.271+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7bb00844d0 con 0x7f7bb815d580 2023-10-10T20:23:07.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.271+0000 7f7bbd202700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.271+0000 7f7bbd202700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 0x7f7b9c0693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009f40 tx=0x7f7bb40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.275+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bb0051ac0 con 0x7f7bb815d580 2023-10-10T20:23:07.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.500+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb804c3c0 con 0x7f7bb815d580 2023-10-10T20:23:07.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.501+0000 7f7bad7fa700 1 -- 172.21.15.148:0/2850239087 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f7bb000ae60 con 0x7f7bb815d580 2023-10-10T20:23:07.504 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:07.504 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5108,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2784712018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2784712018},{"type":"v1","addr":"172.21.15.161:6834","nonce":2784712018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1723524486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1723524486},{"type":"v1","addr":"172.21.15.161:6835","nonce":1723524486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2490125425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2490125425},{"type":"v1","addr":"172.21.15.148:6836","nonce":2490125425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1783852029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1783852029},{"type":"v1","addr":"172.21.15.148:6837","nonce":1783852029}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-10-10T20:23:07.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 msgr2=0x7f7b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 0x7f7b9c0693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009f40 tx=0x7f7bb40091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 msgr2=0x7f7bb80d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000cbc0 tx=0x7f7bb0001980 comp rx=0 tx=0).stop 2023-10-10T20:23:07.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 shutdown_connections 2023-10-10T20:23:07.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b9c066f30 0x7f7b9c0693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb815d580 0x7f7bb80d01a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bb8157870 0x7f7bb80b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 --2- 172.21.15.148:0/2850239087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb815a620 0x7f7bb80b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 >> 172.21.15.148:0/2850239087 conn(0x7f7bb8068ce0 msgr2=0x7f7bb814c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 shutdown_connections 2023-10-10T20:23:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.503+0000 7f7bbf466700 1 -- 172.21.15.148:0/2850239087 wait complete. 2023-10-10T20:23:07.508 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 34 2023-10-10T20:23:07.518 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:23:07.518 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:23:07.518 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:23:07.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:23:07.519 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:23:07.519 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:23:07.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:23:07.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:23:07.520 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:23:07.520 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:23:07.521 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:23:07.521 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:23:07.521 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:23:07.521 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:23:07.522 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:23:07.522 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:23:07.527 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:23:07.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:23:07.528 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:23:07.529 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:23:07.531 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:23:07.531 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:23:07.532 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:23:07.532 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:23:07.571 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:23:07.571 DEBUG:teuthology.parallel:result is None 2023-10-10T20:23:07.573 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:23:07.573 DEBUG:teuthology.parallel:result is None 2023-10-10T20:23:07.574 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:23:07.574 DEBUG:teuthology.parallel:result is None 2023-10-10T20:23:07.575 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:23:07.575 DEBUG:teuthology.parallel:result is None 2023-10-10T20:23:07.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:23:07.644 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:23:07.645 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:23:07.653 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:23:07.653 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:23:07.654 INFO:tasks.ceph.mds.d.smithi161.stderr:2023-10-10T20:23:07.653+0000 7fdf142a0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:23:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.697+0000 7f1678e94700 1 Processor -- start 2023-10-10T20:23:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.697+0000 7f1678e94700 1 -- start start 2023-10-10T20:23:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.698+0000 7f1678e94700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16740a7b20 0x7f16740b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.698+0000 7f1678e94700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 0x7f1674099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.698+0000 7f1678e94700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740990a0 con 0x7f1674063cf0 2023-10-10T20:23:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.698+0000 7f1678e94700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1674064120 con 0x7f16740a7b20 2023-10-10T20:23:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.698+0000 7f1678e94700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740984b0 con 0x7f167406f1c0 2023-10-10T20:23:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1671d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1674063cf0 0x7f167406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39222/0 (socket says 172.21.15.148:39222) 2023-10-10T20:23:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1671d9b700 1 -- 172.21.15.148:0/1841103750 learned_addr learned my addr 172.21.15.148:0/1841103750 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 --2- 172.21.15.148:0/1841103750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 0x7f1674099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167259c700 1 --2- 172.21.15.148:0/1841103750 >> v2:172.21.15.161:3300/0 conn(0x7f16740a7b20 0x7f16740b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167159a700 1 -- 172.21.15.148:0/1841103750 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885326030 0 0) 0x7f16740990a0 con 0x7f1674063cf0 2023-10-10T20:23:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167159a700 1 -- 172.21.15.148:0/1841103750 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1660003570 con 0x7f1674063cf0 2023-10-10T20:23:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 -- 172.21.15.148:0/1841103750 >> v2:172.21.15.161:3300/0 conn(0x7f16740a7b20 msgr2=0x7f16740b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 --2- 172.21.15.148:0/1841103750 >> v2:172.21.15.161:3300/0 conn(0x7f16740a7b20 0x7f16740b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 -- 172.21.15.148:0/1841103750 >> v1:172.21.15.148:6789/0 conn(0x7f1674063cf0 legacy=0x7f167406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 -- 172.21.15.148:0/1841103750 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1674098770 con 0x7f167406f1c0 2023-10-10T20:23:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f1672d9d700 1 --2- 172.21.15.148:0/1841103750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 0x7f1674099530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16640098e0 tx=0x7f16640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2676c3bd647ef662 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167159a700 1 -- 172.21.15.148:0/1841103750 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664011630 con 0x7f167406f1c0 2023-10-10T20:23:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167159a700 1 -- 172.21.15.148:0/1841103750 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664011790 con 0x7f167406f1c0 2023-10-10T20:23:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.699+0000 7f167159a700 1 -- 172.21.15.148:0/1841103750 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664011b30 con 0x7f167406f1c0 2023-10-10T20:23:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 -- 172.21.15.148:0/1841103750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 msgr2=0x7f1674099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 --2- 172.21.15.148:0/1841103750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 0x7f1674099530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16640098e0 tx=0x7f16640091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 -- 172.21.15.148:0/1841103750 shutdown_connections 2023-10-10T20:23:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 --2- 172.21.15.148:0/1841103750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f167406f1c0 0x7f1674099530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 --2- 172.21.15.148:0/1841103750 >> v2:172.21.15.161:3300/0 conn(0x7f16740a7b20 0x7f16740b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 -- 172.21.15.148:0/1841103750 >> 172.21.15.148:0/1841103750 conn(0x7f16740cd1e0 msgr2=0x7f16740cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 -- 172.21.15.148:0/1841103750 shutdown_connections 2023-10-10T20:23:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.700+0000 7f1678e94700 1 -- 172.21.15.148:0/1841103750 wait complete. 2023-10-10T20:23:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.701+0000 7f1678e94700 1 Processor -- start 2023-10-10T20:23:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.701+0000 7f1678e94700 1 -- start start 2023-10-10T20:23:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1674063cf0 0x7f167416a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16740a7b20 0x7f167415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167416d470 con 0x7f167406f1c0 2023-10-10T20:23:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740c4150 con 0x7f16740a7b20 2023-10-10T20:23:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167415b8f0 con 0x7f1674063cf0 2023-10-10T20:23:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44482/0 (socket says 172.21.15.148:44482) 2023-10-10T20:23:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 -- 172.21.15.148:0/1411284785 learned_addr learned my addr 172.21.15.148:0/1411284785 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f167259c700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1674063cf0 0x7f167416a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1672d9d700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16740a7b20 0x7f167415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 -- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1674063cf0 msgr2=0x7f167416a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1674063cf0 0x7f167416a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 -- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16740a7b20 msgr2=0x7f167415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:07.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16740a7b20 0x7f167415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:07.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 -- 172.21.15.148:0/1411284785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664009420 con 0x7f167406f1c0 2023-10-10T20:23:07.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1671d9b700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c7c0 tx=0x7f165c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c005cd0 con 0x7f167406f1c0 2023-10-10T20:23:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c005e30 con 0x7f167406f1c0 2023-10-10T20:23:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c007320 con 0x7f167406f1c0 2023-10-10T20:23:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167415bbd0 con 0x7f167406f1c0 2023-10-10T20:23:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.702+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16740c4430 con 0x7f167406f1c0 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.703+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167404c3c0 con 0x7f167406f1c0 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.704+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f165c01b3b0 con 0x7f167406f1c0 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.704+0000 7f166a7fc700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 0x7f16540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.704+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f165c0838d0 con 0x7f167406f1c0 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.704+0000 7f167259c700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 0x7f16540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.704+0000 7f167259c700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 0x7f16540693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f166c009fd0 tx=0x7f166c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:07.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.708+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f165c050e40 con 0x7f167406f1c0 2023-10-10T20:23:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:07.905+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1674098a50 con 0x7f167406f1c0 2023-10-10T20:23:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.071+0000 7f166a7fc700 1 -- 172.21.15.148:0/1411284785 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) v1 ==== 97+0+0 (secure 0 0 0) 0x7f165c017020 con 0x7f167406f1c0 2023-10-10T20:23:09.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 msgr2=0x7f16540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 0x7f16540693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f166c009fd0 tx=0x7f166c00b040 comp rx=0 tx=0).stop 2023-10-10T20:23:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 msgr2=0x7f167416cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c7c0 tx=0x7f165c007e00 comp rx=0 tx=0).stop 2023-10-10T20:23:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 shutdown_connections 2023-10-10T20:23:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1654066f30 0x7f16540693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16740a7b20 0x7f167415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1674063cf0 0x7f167416a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 --2- 172.21.15.148:0/1411284785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f167406f1c0 0x7f167416cf30 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 >> 172.21.15.148:0/1411284785 conn(0x7f16740cd1e0 msgr2=0x7f16740695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 shutdown_connections 2023-10-10T20:23:09.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.074+0000 7f1678e94700 1 -- 172.21.15.148:0/1411284785 wait complete. 2023-10-10T20:23:09.079 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:23:09.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:23:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.203+0000 7f64d7064700 1 Processor -- start 2023-10-10T20:23:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.203+0000 7f64d7064700 1 -- start start 2023-10-10T20:23:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d7064700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64d0157c40 0x7f64d0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d7064700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d015cbb0 con 0x7f64d0157870 2023-10-10T20:23:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d7064700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d015bfa0 con 0x7f64d015a620 2023-10-10T20:23:09.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d7064700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d015bdb0 con 0x7f64d0157c40 2023-10-10T20:23:09.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d4e00700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f64d0157870 0x7f64d015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39238/0 (socket says 172.21.15.148:39238) 2023-10-10T20:23:09.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d4e00700 1 -- 172.21.15.148:0/4173109248 learned_addr learned my addr 172.21.15.148:0/4173109248 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:09.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.204+0000 7f64d5601700 1 --2- 172.21.15.148:0/4173109248 >> v2:172.21.15.148:3300/0 conn(0x7f64d0157c40 0x7f64d0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113055091 0 0) 0x7f64d015cbb0 con 0x7f64d0157870 2023-10-10T20:23:09.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c0003570 con 0x7f64d0157870 2023-10-10T20:23:09.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64d5601700 1 -- 172.21.15.148:0/4173109248 >> v1:172.21.15.161:6789/0 conn(0x7f64d015a620 legacy=0x7f64d015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64d5601700 1 -- 172.21.15.148:0/4173109248 >> v1:172.21.15.148:6789/0 conn(0x7f64d0157870 legacy=0x7f64d015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481103741 0 0) 0x7f64d015bfa0 con 0x7f64d015a620 2023-10-10T20:23:09.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64d5601700 1 -- 172.21.15.148:0/4173109248 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d015c280 con 0x7f64d0157c40 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64d5601700 1 --2- 172.21.15.148:0/4173109248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157c40 0x7f64d0158110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009e60 tx=0x7f64c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61540998237076d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800e070 con 0x7f64d0157c40 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800bb60 con 0x7f64d0157c40 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.205+0000 7f64cf7fe700 1 -- 172.21.15.148:0/4173109248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8012430 con 0x7f64d0157c40 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 -- 172.21.15.148:0/4173109248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157c40 msgr2=0x7f64d0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 --2- 172.21.15.148:0/4173109248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157c40 0x7f64d0158110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009e60 tx=0x7f64c8009780 comp rx=0 tx=0).stop 2023-10-10T20:23:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 -- 172.21.15.148:0/4173109248 shutdown_connections 2023-10-10T20:23:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 --2- 172.21.15.148:0/4173109248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157c40 0x7f64d0158110 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 -- 172.21.15.148:0/4173109248 >> 172.21.15.148:0/4173109248 conn(0x7f64d0068ce0 msgr2=0x7f64d00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 -- 172.21.15.148:0/4173109248 shutdown_connections 2023-10-10T20:23:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.206+0000 7f64d7064700 1 -- 172.21.15.148:0/4173109248 wait complete. 2023-10-10T20:23:09.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.207+0000 7f64d7064700 1 Processor -- start 2023-10-10T20:23:09.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.207+0000 7f64d7064700 1 -- start start 2023-10-10T20:23:09.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64d0157c40 0x7f64d016ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d015a620 0x7f64d006e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d016f490 con 0x7f64d0157870 2023-10-10T20:23:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00b2580 con 0x7f64d015a620 2023-10-10T20:23:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d7064700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00b2860 con 0x7f64d0157c40 2023-10-10T20:23:09.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44494/0 (socket says 172.21.15.148:44494) 2023-10-10T20:23:09.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 -- 172.21.15.148:0/1242221108 learned_addr learned my addr 172.21.15.148:0/1242221108 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:09.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d5601700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d015a620 0x7f64d006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64cffff700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64d0157c40 0x7f64d016ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 -- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64d0157c40 msgr2=0x7f64d016ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64d0157c40 0x7f64d016ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 -- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d015a620 msgr2=0x7f64d006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d015a620 0x7f64d006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.208+0000 7f64d4e00700 1 -- 172.21.15.148:0/1242221108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009a20 con 0x7f64d0157870 2023-10-10T20:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64d4e00700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64c40011e0 tx=0x7f64c4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4005b30 con 0x7f64d0157870 2023-10-10T20:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4005c90 con 0x7f64d0157870 2023-10-10T20:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00b2b40 con 0x7f64d0157870 2023-10-10T20:23:09.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4007320 con 0x7f64d0157870 2023-10-10T20:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.209+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d006ecc0 con 0x7f64d0157870 2023-10-10T20:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.210+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d004c3c0 con 0x7f64d0157870 2023-10-10T20:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.210+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64c401b3b0 con 0x7f64d0157870 2023-10-10T20:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.210+0000 7f64ccff9700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.210+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f64c4083590 con 0x7f64d0157870 2023-10-10T20:23:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.210+0000 7f64cffff700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.211+0000 7f64cffff700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 0x7f64b40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64d0171820 tx=0x7f64bc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.215+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64c4050b50 con 0x7f64d0157870 2023-10-10T20:23:09.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.471+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f64d015c560 con 0x7f64d0157870 2023-10-10T20:23:09.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.471+0000 7f64ccff9700 1 -- 172.21.15.148:0/1242221108 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f64c4017090 con 0x7f64d0157870 2023-10-10T20:23:09.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:09.474 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:23:09.474 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:23:09.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.475 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:23:09.475 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:23:09.475 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.477 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:23:09.477 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:23:09.477 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.477 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.478 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:23:09.478 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:23:09.478 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.479 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:23:09.479 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:23:09.479 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.480 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:23:09.480 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:23:09.480 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:23:09.482 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:23:09.482 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.484 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:23:09.484 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:23:09.485 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.485 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.485 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.485 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.485 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.486 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:23:09.486 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:23:09.486 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.488 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:23:09.488 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:23:09.488 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.488 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.488 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.489 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.489 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:23:09.490 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:23:09.490 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.490 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.490 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.490 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.491 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:23:09.491 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.493 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:23:09.493 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:23:09.493 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.494 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:23:09.495 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.496 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:23:09.496 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:23:09.497 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.497 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.497 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:23:09.497 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:23:09.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.498 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:23:09.498 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:23:09.498 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:23:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:23:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:23:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:23:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:23:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:23:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:23:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:23:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:23:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:23:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:23:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:23:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:23:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:23:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:23:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:23:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:23:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:23:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:23:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:23:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:23:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:23:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:23:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:23:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:23:09.509 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:23:09.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 msgr2=0x7f64b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 0x7f64b40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64d0171820 tx=0x7f64bc007ec0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 msgr2=0x7f64d016e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64c40011e0 tx=0x7f64c4007df0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 shutdown_connections 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f64b4066f30 0x7f64b40693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d015a620 0x7f64d006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64d0157870 0x7f64d016e940 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 --2- 172.21.15.148:0/1242221108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64d0157c40 0x7f64d016ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 >> 172.21.15.148:0/1242221108 conn(0x7f64d0068ce0 msgr2=0x7f64d00b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 shutdown_connections 2023-10-10T20:23:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.473+0000 7f64d7064700 1 -- 172.21.15.148:0/1242221108 wait complete. 2023-10-10T20:23:09.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:09.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.627+0000 7f139ac45700 1 Processor -- start 2023-10-10T20:23:09.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.627+0000 7f139ac45700 1 -- start start 2023-10-10T20:23:09.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f139ac45700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1394152020 0x7f13940ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f139ac45700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 0x7f139406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f139ac45700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940cbb10 con 0x7f13941523f0 2023-10-10T20:23:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f139ac45700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13940cbfa0 con 0x7f1394152020 2023-10-10T20:23:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f139ac45700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13940caca0 con 0x7f139415bf00 2023-10-10T20:23:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f138bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f13941523f0 0x7f139415b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39254/0 (socket says 172.21.15.148:39254) 2023-10-10T20:23:09.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f138bfff700 1 -- 172.21.15.148:0/3605756629 learned_addr learned my addr 172.21.15.148:0/3605756629 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:09.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f13991e2700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 0x7f139406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.628+0000 7f13989e1700 1 --2- 172.21.15.148:0/3605756629 >> v2:172.21.15.161:3300/0 conn(0x7f1394152020 0x7f13940ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445303047 0 0) 0x7f13940cbb10 con 0x7f13941523f0 2023-10-10T20:23:09.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f137c003570 con 0x7f13941523f0 2023-10-10T20:23:09.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13989e1700 1 -- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 msgr2=0x7f139406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:09.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 331224959 0 0) 0x7f137c003570 con 0x7f13941523f0 2023-10-10T20:23:09.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13989e1700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 0x7f139406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13989e1700 1 -- 172.21.15.148:0/3605756629 >> v1:172.21.15.148:6789/0 conn(0x7f13941523f0 legacy=0x7f139415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13989e1700 1 -- 172.21.15.148:0/3605756629 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139406cd30 con 0x7f1394152020 2023-10-10T20:23:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13991e2700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 0x7f139406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f13989e1700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1394152020 0x7f13940ca760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f139000fad0 tx=0x7f13900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f18becaa6fcf4a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384001470 con 0x7f1394152020 2023-10-10T20:23:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139000f140 con 0x7f1394152020 2023-10-10T20:23:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.629+0000 7f138b7fe700 1 -- 172.21.15.148:0/3605756629 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390009770 con 0x7f1394152020 2023-10-10T20:23:09.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 -- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1394152020 msgr2=0x7f13940ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1394152020 0x7f13940ca760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f139000fad0 tx=0x7f13900091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 -- 172.21.15.148:0/3605756629 shutdown_connections 2023-10-10T20:23:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f139415bf00 0x7f139406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 --2- 172.21.15.148:0/3605756629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1394152020 0x7f13940ca760 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 -- 172.21.15.148:0/3605756629 >> 172.21.15.148:0/3605756629 conn(0x7f1394069870 msgr2=0x7f1394069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 -- 172.21.15.148:0/3605756629 shutdown_connections 2023-10-10T20:23:09.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.630+0000 7f139ac45700 1 -- 172.21.15.148:0/3605756629 wait complete. 2023-10-10T20:23:09.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.631+0000 7f139ac45700 1 Processor -- start 2023-10-10T20:23:09.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.631+0000 7f139ac45700 1 -- start start 2023-10-10T20:23:09.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 0x7f13940c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 0x7f139414cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940c7e50 con 0x7f13941523f0 2023-10-10T20:23:09.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940c7240 con 0x7f139415bf00 2023-10-10T20:23:09.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f139ac45700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13940c7520 con 0x7f1394152020 2023-10-10T20:23:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44496/0 (socket says 172.21.15.148:44496) 2023-10-10T20:23:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 -- 172.21.15.148:0/608973347 learned_addr learned my addr 172.21.15.148:0/608973347 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f13989e1700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 0x7f13940c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f13991e2700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 0x7f139414cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 -- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 msgr2=0x7f13940c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 0x7f13940c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 -- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 msgr2=0x7f139414cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 0x7f139414cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 -- 172.21.15.148:0/608973347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c009a20 con 0x7f13941523f0 2023-10-10T20:23:09.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f13989e1700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 0x7f13940c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:09.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f13991e2700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 0x7f139414cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:09.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.632+0000 7f138bfff700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f138400c930 tx=0x7f1384001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400ce70 con 0x7f13941523f0 2023-10-10T20:23:09.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384005b30 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139000f610 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840070f0 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13940c7ae0 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.633+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13940c8000 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.634+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1384007460 con 0x7f13941523f0 2023-10-10T20:23:09.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.634+0000 7f1388ff9700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 0x7f1380069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.634+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1384083890 con 0x7f13941523f0 2023-10-10T20:23:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.634+0000 7f13989e1700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 0x7f1380069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.635+0000 7f13989e1700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 0x7f1380069410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13900030c0 tx=0x7f1390015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.639+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f138404b5a0 con 0x7f13941523f0 2023-10-10T20:23:09.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.821+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f139404c3c0 con 0x7f13941523f0 2023-10-10T20:23:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.822+0000 7f1388ff9700 1 -- 172.21.15.148:0/608973347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f1384017070 con 0x7f13941523f0 2023-10-10T20:23:09.825 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:09.825 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:09.066841+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.824+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 msgr2=0x7f1380069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.824+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 0x7f1380069410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f13900030c0 tx=0x7f1390015000 comp rx=0 tx=0).stop 2023-10-10T20:23:09.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.824+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 msgr2=0x7f139413d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f138400c930 tx=0x7f1384001a10 comp rx=0 tx=0).stop 2023-10-10T20:23:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 shutdown_connections 2023-10-10T20:23:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1380066f60 0x7f1380069410 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f139415bf00 0x7f139414cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1394152020 0x7f13940c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 --2- 172.21.15.148:0/608973347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13941523f0 0x7f139413d420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 >> 172.21.15.148:0/608973347 conn(0x7f1394069870 msgr2=0x7f1394150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 shutdown_connections 2023-10-10T20:23:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.825+0000 7f139ac45700 1 -- 172.21.15.148:0/608973347 wait complete. 2023-10-10T20:23:09.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:23:09.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:23:09.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.955+0000 7f945eb66700 1 Processor -- start 2023-10-10T20:23:09.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.955+0000 7f945eb66700 1 -- start start 2023-10-10T20:23:09.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945eb66700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94580b6580 0x7f945806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945eb66700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f945806a830 0x7f94580b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945eb66700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94580b4d70 con 0x7f94580cce30 2023-10-10T20:23:09.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945eb66700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945806ac70 con 0x7f945806a830 2023-10-10T20:23:09.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945eb66700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945815b3b0 con 0x7f94580b6580 2023-10-10T20:23:09.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945db64700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f94580cce30 0x7f945815aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39256/0 (socket says 172.21.15.148:39256) 2023-10-10T20:23:09.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945db64700 1 -- 172.21.15.148:0/643124027 learned_addr learned my addr 172.21.15.148:0/643124027 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:09.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945e365700 1 --2- 172.21.15.148:0/643124027 >> v2:172.21.15.148:3300/0 conn(0x7f945806a830 0x7f94580b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945d363700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94580b6580 0x7f945806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945cb62700 1 -- 172.21.15.148:0/643124027 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225383995 0 0) 0x7f94580b4d70 con 0x7f94580cce30 2023-10-10T20:23:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945cb62700 1 -- 172.21.15.148:0/643124027 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9444003570 con 0x7f94580cce30 2023-10-10T20:23:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945e365700 1 -- 172.21.15.148:0/643124027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94580b6580 msgr2=0x7f945806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945e365700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94580b6580 0x7f945806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945e365700 1 -- 172.21.15.148:0/643124027 >> v1:172.21.15.148:6789/0 conn(0x7f94580cce30 legacy=0x7f945815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.956+0000 7f945e365700 1 -- 172.21.15.148:0/643124027 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580cd8d0 con 0x7f945806a830 2023-10-10T20:23:09.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945e365700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945806a830 0x7f94580b5200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9450009e60 tx=0x7f9450009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edee2ddda878c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945cb62700 1 -- 172.21.15.148:0/643124027 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000e070 con 0x7f945806a830 2023-10-10T20:23:09.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945cb62700 1 -- 172.21.15.148:0/643124027 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000bab0 con 0x7f945806a830 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945cb62700 1 -- 172.21.15.148:0/643124027 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000be70 con 0x7f945806a830 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 -- 172.21.15.148:0/643124027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945806a830 msgr2=0x7f94580b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945806a830 0x7f94580b5200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9450009e60 tx=0x7f9450009780 comp rx=0 tx=0).stop 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 -- 172.21.15.148:0/643124027 shutdown_connections 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945806a830 0x7f94580b5200 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 --2- 172.21.15.148:0/643124027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94580b6580 0x7f945806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 -- 172.21.15.148:0/643124027 >> 172.21.15.148:0/643124027 conn(0x7f9458068cb0 msgr2=0x7f94580690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:09.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 -- 172.21.15.148:0/643124027 shutdown_connections 2023-10-10T20:23:09.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.957+0000 7f945eb66700 1 -- 172.21.15.148:0/643124027 wait complete. 2023-10-10T20:23:09.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.958+0000 7f945eb66700 1 Processor -- start 2023-10-10T20:23:09.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.958+0000 7f945eb66700 1 -- start start 2023-10-10T20:23:09.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f945806a830 0x7f94580b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 0x7f9458147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94581488c0 con 0x7f94580b6580 2023-10-10T20:23:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458152430 con 0x7f94580cce30 2023-10-10T20:23:09.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945eb66700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9458152140 con 0x7f945806a830 2023-10-10T20:23:09.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945d363700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945d363700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44526/0 (socket says 172.21.15.148:44526) 2023-10-10T20:23:09.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945d363700 1 -- 172.21.15.148:0/212751884 learned_addr learned my addr 172.21.15.148:0/212751884 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:09.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945db64700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f945806a830 0x7f94580b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945e365700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 0x7f9458147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945d363700 1 -- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f945806a830 msgr2=0x7f94580b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.959+0000 7f945d363700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f945806a830 0x7f94580b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945d363700 1 -- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 msgr2=0x7f9458147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945d363700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 0x7f9458147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:09.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945d363700 1 -- 172.21.15.148:0/212751884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009a20 con 0x7f94580b6580 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945e365700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 0x7f9458147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945d363700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f944800ec50 tx=0x7f94480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800c850 con 0x7f94580b6580 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944800c9b0 con 0x7f94580b6580 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9458148020 con 0x7f94580b6580 2023-10-10T20:23:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800cd70 con 0x7f94580b6580 2023-10-10T20:23:09.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.960+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94581485e0 con 0x7f94580b6580 2023-10-10T20:23:09.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.961+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9458159af0 con 0x7f94580b6580 2023-10-10T20:23:09.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.961+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9448013680 con 0x7f94580b6580 2023-10-10T20:23:09.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.962+0000 7f944dffb700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:09.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.962+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f94480844b0 con 0x7f94580b6580 2023-10-10T20:23:09.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.962+0000 7f945db64700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:09.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.962+0000 7f945db64700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 0x7f943c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9454007fd0 tx=0x7f9454009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:09.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:09.966+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9448051a70 con 0x7f94580b6580 2023-10-10T20:23:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.151+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94581525a0 con 0x7f94580b6580 2023-10-10T20:23:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.151+0000 7f944dffb700 1 -- 172.21.15.148:0/212751884 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9448055090 con 0x7f94580b6580 2023-10-10T20:23:10.155 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:23:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 msgr2=0x7f943c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 0x7f943c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9454007fd0 tx=0x7f9454009040 comp rx=0 tx=0).stop 2023-10-10T20:23:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 msgr2=0x7f9458151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f944800ec50 tx=0x7f94480091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 shutdown_connections 2023-10-10T20:23:10.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94580cce30 0x7f9458147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f945806a830 0x7f94580b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 --2- 172.21.15.148:0/212751884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94580b6580 0x7f9458151c00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 >> 172.21.15.148:0/212751884 conn(0x7f9458068cb0 msgr2=0x7f945815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.153+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 shutdown_connections 2023-10-10T20:23:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.154+0000 7f945eb66700 1 -- 172.21.15.148:0/212751884 wait complete. 2023-10-10T20:23:10.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:23:10.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.284+0000 7f32ad18d700 1 Processor -- start 2023-10-10T20:23:10.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.284+0000 7f32ad18d700 1 -- start start 2023-10-10T20:23:10.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32ad18d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32a80abae0 0x7f32a80c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32ad18d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80c8b10 0x7f32a80c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32ad18d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a80c9cb0 con 0x7f32a80abeb0 2023-10-10T20:23:10.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32ad18d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a80c90a0 con 0x7f32a80abae0 2023-10-10T20:23:10.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32ad18d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80c9380 con 0x7f32a80c8b10 2023-10-10T20:23:10.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32a6d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32a80abae0 0x7f32a80c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32a759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80c8b10 0x7f32a80c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f329e59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f32a80abeb0 0x7f32a80c8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48452/0 (socket says 172.21.15.148:48452) 2023-10-10T20:23:10.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f32a6d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32a80abae0 0x7f32a80c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:47868/0 (socket says 172.21.15.148:47868) 2023-10-10T20:23:10.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.285+0000 7f329e59c700 1 -- 172.21.15.148:0/369344872 learned_addr learned my addr 172.21.15.148:0/369344872 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:10.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a6d9d700 1 -- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80c8b10 msgr2=0x7f32a80c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:10.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a6d9d700 1 --2- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80c8b10 0x7f32a80c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a6d9d700 1 -- 172.21.15.148:0/369344872 >> v1:172.21.15.161:6789/0 conn(0x7f32a80abeb0 legacy=0x7f32a80c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a6d9d700 1 -- 172.21.15.148:0/369344872 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80c9660 con 0x7f32a80abae0 2023-10-10T20:23:10.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a659c700 1 -- 172.21.15.148:0/369344872 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837403400 0 0) 0x7f32a80c9cb0 con 0x7f32a80abeb0 2023-10-10T20:23:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a6d9d700 1 --2- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80abae0 0x7f32a80c6eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32980098e0 tx=0x7f32980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b57c57df16f402e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a659c700 1 -- 172.21.15.148:0/369344872 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800b640 con 0x7f32a80abae0 2023-10-10T20:23:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a659c700 1 -- 172.21.15.148:0/369344872 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800b7a0 con 0x7f32a80abae0 2023-10-10T20:23:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.286+0000 7f32a659c700 1 -- 172.21.15.148:0/369344872 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800bb40 con 0x7f32a80abae0 2023-10-10T20:23:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 -- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80abae0 msgr2=0x7f32a80c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 --2- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80abae0 0x7f32a80c6eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32980098e0 tx=0x7f32980091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 -- 172.21.15.148:0/369344872 shutdown_connections 2023-10-10T20:23:10.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 --2- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80abae0 0x7f32a80c6eb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 --2- 172.21.15.148:0/369344872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80c8b10 0x7f32a80c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 -- 172.21.15.148:0/369344872 >> 172.21.15.148:0/369344872 conn(0x7f32a8067660 msgr2=0x7f32a8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:10.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 -- 172.21.15.148:0/369344872 shutdown_connections 2023-10-10T20:23:10.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.287+0000 7f32ad18d700 1 -- 172.21.15.148:0/369344872 wait complete. 2023-10-10T20:23:10.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.288+0000 7f32ad18d700 1 Processor -- start 2023-10-10T20:23:10.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.288+0000 7f32ad18d700 1 -- start start 2023-10-10T20:23:10.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80abae0 0x7f32a816dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80c8b10 0x7f32a80c1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a816aa60 con 0x7f32a80abeb0 2023-10-10T20:23:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a816a710 con 0x7f32a80c8b10 2023-10-10T20:23:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32ad18d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a816e250 con 0x7f32a80abae0 2023-10-10T20:23:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f329e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f329e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44536/0 (socket says 172.21.15.148:44536) 2023-10-10T20:23:10.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f329e59c700 1 -- 172.21.15.148:0/398946881 learned_addr learned my addr 172.21.15.148:0/398946881 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:10.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32a6d9d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80abae0 0x7f32a816dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.289+0000 7f32a759e700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80c8b10 0x7f32a80c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 -- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80abae0 msgr2=0x7f32a816dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80abae0 0x7f32a816dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 -- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80c8b10 msgr2=0x7f32a80c1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80c8b10 0x7f32a80c1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 -- 172.21.15.148:0/398946881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298009420 con 0x7f32a80abeb0 2023-10-10T20:23:10.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329e59c700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f329000c810 tx=0x7f3290007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290005c00 con 0x7f32a80abeb0 2023-10-10T20:23:10.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3290005d60 con 0x7f32a80abeb0 2023-10-10T20:23:10.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a816e520 con 0x7f32a80abeb0 2023-10-10T20:23:10.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290007300 con 0x7f32a80abeb0 2023-10-10T20:23:10.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.290+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a816eae0 con 0x7f32a80abeb0 2023-10-10T20:23:10.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.291+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a816ac10 con 0x7f32a80abeb0 2023-10-10T20:23:10.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.291+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f329001b3b0 con 0x7f32a80abeb0 2023-10-10T20:23:10.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.291+0000 7f329f7fe700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 0x7f32880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.292+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f32900836a0 con 0x7f32a80abeb0 2023-10-10T20:23:10.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.292+0000 7f32a6d9d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 0x7f32880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.292+0000 7f32a6d9d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 0x7f32880693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32980093f0 tx=0x7f3298017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.296+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32a816ac10 con 0x7f32a80abeb0 2023-10-10T20:23:10.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.480+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f32a806b2d0 con 0x7f32a80abeb0 2023-10-10T20:23:10.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.480+0000 7f329f7fe700 1 -- 172.21.15.148:0/398946881 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3290017090 con 0x7f32a80abeb0 2023-10-10T20:23:10.483 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:10.483 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:23:10.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:23:10.483 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:23:10.484 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:23:10.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:23:10.484 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:23:10.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 msgr2=0x7f32880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 0x7f32880693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32980093f0 tx=0x7f3298017000 comp rx=0 tx=0).stop 2023-10-10T20:23:10.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 msgr2=0x7f32a816aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f329000c810 tx=0x7f3290007db0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 shutdown_connections 2023-10-10T20:23:10.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3288066f30 0x7f32880693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a80c8b10 0x7f32a80c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32a80abae0 0x7f32a816dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 --2- 172.21.15.148:0/398946881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32a80abeb0 0x7f32a816aef0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.482+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 >> 172.21.15.148:0/398946881 conn(0x7f32a8067660 msgr2=0x7f32a80ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.483+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 shutdown_connections 2023-10-10T20:23:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.483+0000 7f32ad18d700 1 -- 172.21.15.148:0/398946881 wait complete. 2023-10-10T20:23:10.497 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:23:10.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:23:10.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.616+0000 7f551eded700 1 Processor -- start 2023-10-10T20:23:10.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.616+0000 7f551eded700 1 -- start start 2023-10-10T20:23:10.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551eded700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5518159ef0 0x7f551815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551eded700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55180b46f0 0x7f55180b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551eded700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55180b5070 con 0x7f55180b6a10 2023-10-10T20:23:10.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551eded700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180b17b0 con 0x7f5518159ef0 2023-10-10T20:23:10.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551eded700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180b1a60 con 0x7f55180b46f0 2023-10-10T20:23:10.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5518159ef0 0x7f551815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5518159ef0 0x7f551815a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44548/0 (socket says 172.21.15.148:44548) 2023-10-10T20:23:10.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 -- 172.21.15.148:0/382349218 learned_addr learned my addr 172.21.15.148:0/382349218 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:10.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551d38a700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55180b46f0 0x7f55180b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 -- 172.21.15.148:0/382349218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55180b46f0 msgr2=0x7f55180b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:10.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55180b46f0 0x7f55180b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 -- 172.21.15.148:0/382349218 >> v1:172.21.15.161:6789/0 conn(0x7f55180b6a10 legacy=0x7f55180b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 -- 172.21.15.148:0/382349218 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55180b1d40 con 0x7f5518159ef0 2023-10-10T20:23:10.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f551cb89700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5518159ef0 0x7f551815a2c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5510009990 tx=0x7f5510009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df45d04a43c7ca8c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.617+0000 7f55177fe700 1 -- 172.21.15.148:0/382349218 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510005c10 con 0x7f5518159ef0 2023-10-10T20:23:10.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f55177fe700 1 -- 172.21.15.148:0/382349218 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510005d70 con 0x7f5518159ef0 2023-10-10T20:23:10.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f55177fe700 1 -- 172.21.15.148:0/382349218 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000b880 con 0x7f5518159ef0 2023-10-10T20:23:10.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 -- 172.21.15.148:0/382349218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5518159ef0 msgr2=0x7f551815a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5518159ef0 0x7f551815a2c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5510009990 tx=0x7f5510009220 comp rx=0 tx=0).stop 2023-10-10T20:23:10.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 -- 172.21.15.148:0/382349218 shutdown_connections 2023-10-10T20:23:10.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5518159ef0 0x7f551815a2c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 --2- 172.21.15.148:0/382349218 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55180b46f0 0x7f55180b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 -- 172.21.15.148:0/382349218 >> 172.21.15.148:0/382349218 conn(0x7f551809a0a0 msgr2=0x7f5518067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:10.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 -- 172.21.15.148:0/382349218 shutdown_connections 2023-10-10T20:23:10.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.618+0000 7f551eded700 1 -- 172.21.15.148:0/382349218 wait complete. 2023-10-10T20:23:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.619+0000 7f551eded700 1 Processor -- start 2023-10-10T20:23:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.619+0000 7f551eded700 1 -- start start 2023-10-10T20:23:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55180b6a10 0x7f55181517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5518159ef0 0x7f551814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551814d960 con 0x7f55180b46f0 2023-10-10T20:23:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5518150eb0 con 0x7f55180b6a10 2023-10-10T20:23:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551eded700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5518150610 con 0x7f5518159ef0 2023-10-10T20:23:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44564/0 (socket says 172.21.15.148:44564) 2023-10-10T20:23:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 -- 172.21.15.148:0/3177061379 learned_addr learned my addr 172.21.15.148:0/3177061379 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551d38a700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5518159ef0 0x7f551814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f5517fff700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55180b6a10 0x7f55181517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 -- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5518159ef0 msgr2=0x7f551814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5518159ef0 0x7f551814d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 -- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55180b6a10 msgr2=0x7f55181517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55180b6a10 0x7f55181517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.620+0000 7f551cb89700 1 -- 172.21.15.148:0/3177061379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510009480 con 0x7f55180b46f0 2023-10-10T20:23:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f551cb89700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5510005d40 tx=0x7f5510016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510011a60 con 0x7f55180b46f0 2023-10-10T20:23:10.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510011bc0 con 0x7f55180b46f0 2023-10-10T20:23:10.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55181508f0 con 0x7f55180b46f0 2023-10-10T20:23:10.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551001b430 con 0x7f55180b46f0 2023-10-10T20:23:10.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.621+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55181547a0 con 0x7f55180b46f0 2023-10-10T20:23:10.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.622+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518154990 con 0x7f55180b46f0 2023-10-10T20:23:10.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.622+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f551001b860 con 0x7f55180b46f0 2023-10-10T20:23:10.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.623+0000 7f5514ff9700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 0x7f54f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:10.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.623+0000 7f5517fff700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 0x7f54f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:10.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.623+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5510085ba0 con 0x7f55180b46f0 2023-10-10T20:23:10.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.623+0000 7f5517fff700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 0x7f54f80693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5518156a90 tx=0x7f5508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:10.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.627+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5510053160 con 0x7f55180b46f0 2023-10-10T20:23:10.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:10.811+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f551804c3c0 con 0x7f55180b46f0 2023-10-10T20:23:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.609+0000 7f5514ff9700 1 -- 172.21.15.148:0/3177061379 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f551002a0b0 con 0x7f55180b46f0 2023-10-10T20:23:11.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 msgr2=0x7f54f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 0x7f54f80693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5518156a90 tx=0x7f5508009040 comp rx=0 tx=0).stop 2023-10-10T20:23:11.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 msgr2=0x7f551814ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5510005d40 tx=0x7f5510016000 comp rx=0 tx=0).stop 2023-10-10T20:23:11.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 shutdown_connections 2023-10-10T20:23:11.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f54f8066f30 0x7f54f80693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5518159ef0 0x7f551814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55180b46f0 0x7f551814ff30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 --2- 172.21.15.148:0/3177061379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55180b6a10 0x7f55181517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 >> 172.21.15.148:0/3177061379 conn(0x7f551809a0a0 msgr2=0x7f5518158160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 shutdown_connections 2023-10-10T20:23:11.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.613+0000 7f551eded700 1 -- 172.21.15.148:0/3177061379 wait complete. 2023-10-10T20:23:11.618 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:23:11.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:23:11.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.749+0000 7faa4a5c7700 1 Processor -- start 2023-10-10T20:23:11.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.749+0000 7faa4a5c7700 1 -- start start 2023-10-10T20:23:11.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.750+0000 7faa4a5c7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 0x7faa4415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.750+0000 7faa4a5c7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.750+0000 7faa4a5c7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa441513f0 con 0x7faa440d12a0 2023-10-10T20:23:11.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.750+0000 7faa4a5c7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4415e7a0 con 0x7faa440cd300 2023-10-10T20:23:11.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.750+0000 7faa4a5c7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440d1700 con 0x7faa440b6760 2023-10-10T20:23:11.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa43fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 0x7faa4415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa48b64700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7faa440d12a0 0x7faa4406a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48470/0 (socket says 172.21.15.148:48470) 2023-10-10T20:23:11.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:45638/0 (socket says 172.21.15.148:45638) 2023-10-10T20:23:11.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa48b64700 1 -- 172.21.15.148:0/2367765814 learned_addr learned my addr 172.21.15.148:0/2367765814 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:11.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 -- 172.21.15.148:0/2367765814 >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 msgr2=0x7faa4415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 --2- 172.21.15.148:0/2367765814 >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 0x7faa4415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 -- 172.21.15.148:0/2367765814 >> v1:172.21.15.161:6789/0 conn(0x7faa440d12a0 legacy=0x7faa4406a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 -- 172.21.15.148:0/2367765814 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4415db20 con 0x7faa440b6760 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa43fff700 1 --2- 172.21.15.148:0/2367765814 >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 0x7faa4415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa437fe700 1 --2- 172.21.15.148:0/2367765814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6147e7b1ea5a79d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:11.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa42ffd700 1 -- 172.21.15.148:0/2367765814 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800b640 con 0x7faa440b6760 2023-10-10T20:23:11.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa42ffd700 1 -- 172.21.15.148:0/2367765814 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3800b7a0 con 0x7faa440b6760 2023-10-10T20:23:11.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.751+0000 7faa42ffd700 1 -- 172.21.15.148:0/2367765814 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800bb40 con 0x7faa440b6760 2023-10-10T20:23:11.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 -- 172.21.15.148:0/2367765814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 msgr2=0x7faa440d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/2367765814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa380091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 -- 172.21.15.148:0/2367765814 shutdown_connections 2023-10-10T20:23:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/2367765814 >> v2:172.21.15.161:3300/0 conn(0x7faa440cd300 0x7faa4415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/2367765814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440b6760 0x7faa440d0d60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 -- 172.21.15.148:0/2367765814 >> 172.21.15.148:0/2367765814 conn(0x7faa44068d90 msgr2=0x7faa44069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 -- 172.21.15.148:0/2367765814 shutdown_connections 2023-10-10T20:23:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.752+0000 7faa4a5c7700 1 -- 172.21.15.148:0/2367765814 wait complete. 2023-10-10T20:23:11.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.753+0000 7faa4a5c7700 1 Processor -- start 2023-10-10T20:23:11.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.753+0000 7faa4a5c7700 1 -- start start 2023-10-10T20:23:11.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 0x7faa4414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440d12a0 0x7faa4415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441588e0 con 0x7faa440cd300 2023-10-10T20:23:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44157cd0 con 0x7faa440b6760 2023-10-10T20:23:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa4a5c7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa44157fb0 con 0x7faa440d12a0 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44572/0 (socket says 172.21.15.148:44572) 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa43fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 0x7faa4414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 -- 172.21.15.148:0/555434786 learned_addr learned my addr 172.21.15.148:0/555434786 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa48b64700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440d12a0 0x7faa4415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 -- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440d12a0 msgr2=0x7faa4415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440d12a0 0x7faa4415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 -- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 msgr2=0x7faa4414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:11.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 0x7faa4414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:11.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 -- 172.21.15.148:0/555434786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa34009420 con 0x7faa440cd300 2023-10-10T20:23:11.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa43fff700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 0x7faa4414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:11.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.754+0000 7faa437fe700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faa4406b520 tx=0x7faa38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:11.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801bc40 con 0x7faa440cd300 2023-10-10T20:23:11.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3801bde0 con 0x7faa440cd300 2023-10-10T20:23:11.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa38009420 con 0x7faa440cd300 2023-10-10T20:23:11.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38011670 con 0x7faa440cd300 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44158570 con 0x7faa440cd300 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.755+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa44158a90 con 0x7faa440cd300 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.756+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa380119e0 con 0x7faa440cd300 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.756+0000 7faa2bfff700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 0x7faa300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.756+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5223+0+0 (secure 0 0 0) 0x7faa380858a0 con 0x7faa440cd300 2023-10-10T20:23:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.756+0000 7faa43fff700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 0x7faa300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:11.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.757+0000 7faa43fff700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 0x7faa300693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faa34011560 tx=0x7faa340091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:11.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.761+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa44158a90 con 0x7faa440cd300 2023-10-10T20:23:11.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:11.942+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faa44158a90 con 0x7faa440cd300 2023-10-10T20:23:12.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.613+0000 7faa2bfff700 1 -- 172.21.15.148:0/555434786 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7faa44158a90 con 0x7faa440cd300 2023-10-10T20:23:12.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 msgr2=0x7faa300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 0x7faa300693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faa34011560 tx=0x7faa340091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 msgr2=0x7faa44159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faa4406b520 tx=0x7faa38017000 comp rx=0 tx=0).stop 2023-10-10T20:23:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 shutdown_connections 2023-10-10T20:23:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faa30066f30 0x7faa300693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa440d12a0 0x7faa4415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa440b6760 0x7faa4414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 --2- 172.21.15.148:0/555434786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa440cd300 0x7faa44159320 secure :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faa4406b520 tx=0x7faa38017000 comp rx=0 tx=0).stop 2023-10-10T20:23:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.621+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 >> 172.21.15.148:0/555434786 conn(0x7faa44068d90 msgr2=0x7faa4414f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.622+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 shutdown_connections 2023-10-10T20:23:12.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.623+0000 7faa4a5c7700 1 -- 172.21.15.148:0/555434786 wait complete. 2023-10-10T20:23:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:23:12.638 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:23:12.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.757+0000 7f5eb118b700 1 Processor -- start 2023-10-10T20:23:12.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.757+0000 7f5eb118b700 1 -- start start 2023-10-10T20:23:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eb118b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eac159ec0 0x7f5eac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eb118b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5eac15a290 0x7f5eac0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eb118b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0b0790 con 0x7f5eac06a5a0 2023-10-10T20:23:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eb118b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0ce1f0 con 0x7f5eac159ec0 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eb118b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0b0b20 con 0x7f5eac15a290 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eac159ec0 0x7f5eac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eab59e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5eac06a5a0 0x7f5eac15db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50182/0 (socket says 172.21.15.148:50182) 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eac159ec0 0x7f5eac0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54650/0 (socket says 172.21.15.148:54650) 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eab59e700 1 -- 172.21.15.148:0/2075601247 learned_addr learned my addr 172.21.15.148:0/2075601247 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:12.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5ea259c700 1 --2- 172.21.15.148:0/2075601247 >> v2:172.21.15.161:3300/0 conn(0x7f5eac15a290 0x7f5eac0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 -- 172.21.15.148:0/2075601247 >> v2:172.21.15.161:3300/0 conn(0x7f5eac15a290 msgr2=0x7f5eac0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 --2- 172.21.15.148:0/2075601247 >> v2:172.21.15.161:3300/0 conn(0x7f5eac15a290 0x7f5eac0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 -- 172.21.15.148:0/2075601247 >> v1:172.21.15.148:6789/0 conn(0x7f5eac06a5a0 legacy=0x7f5eac15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaa59c700 1 -- 172.21.15.148:0/2075601247 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149437646 0 0) 0x7f5eac0b0790 con 0x7f5eac06a5a0 2023-10-10T20:23:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.758+0000 7f5eaad9d700 1 -- 172.21.15.148:0/2075601247 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0b04b0 con 0x7f5eac159ec0 2023-10-10T20:23:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eaad9d700 1 --2- 172.21.15.148:0/2075601247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac0af940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5eac089910 tx=0x7f5e9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1448fc8b367dc11 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eaa59c700 1 -- 172.21.15.148:0/2075601247 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011630 con 0x7f5eac159ec0 2023-10-10T20:23:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eaa59c700 1 -- 172.21.15.148:0/2075601247 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c011790 con 0x7f5eac159ec0 2023-10-10T20:23:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eaa59c700 1 -- 172.21.15.148:0/2075601247 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011b30 con 0x7f5eac159ec0 2023-10-10T20:23:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 -- 172.21.15.148:0/2075601247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 msgr2=0x7f5eac0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 --2- 172.21.15.148:0/2075601247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac0af940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5eac089910 tx=0x7f5e9c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 -- 172.21.15.148:0/2075601247 shutdown_connections 2023-10-10T20:23:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 --2- 172.21.15.148:0/2075601247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac0af940 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 --2- 172.21.15.148:0/2075601247 >> v2:172.21.15.161:3300/0 conn(0x7f5eac15a290 0x7f5eac0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 -- 172.21.15.148:0/2075601247 >> 172.21.15.148:0/2075601247 conn(0x7f5eac0d1880 msgr2=0x7f5eac0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 -- 172.21.15.148:0/2075601247 shutdown_connections 2023-10-10T20:23:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.759+0000 7f5eb118b700 1 -- 172.21.15.148:0/2075601247 wait complete. 2023-10-10T20:23:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.760+0000 7f5eb118b700 1 Processor -- start 2023-10-10T20:23:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 -- start start 2023-10-10T20:23:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5eac06a5a0 0x7f5eac0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5eac15a290 0x7f5eac156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac158ec0 con 0x7f5eac159ec0 2023-10-10T20:23:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0c88c0 con 0x7f5eac15a290 2023-10-10T20:23:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5eb118b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac158bf0 con 0x7f5eac06a5a0 2023-10-10T20:23:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5ea259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5ea259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54666/0 (socket says 172.21.15.148:54666) 2023-10-10T20:23:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.761+0000 7f5ea259c700 1 -- 172.21.15.148:0/1500254608 learned_addr learned my addr 172.21.15.148:0/1500254608 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5eaad9d700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5eac06a5a0 0x7f5eac0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5eab59e700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5eac15a290 0x7f5eac156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 -- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5eac06a5a0 msgr2=0x7f5eac0cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5eac06a5a0 0x7f5eac0cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 -- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5eac15a290 msgr2=0x7f5eac156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5eac15a290 0x7f5eac156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 -- 172.21.15.148:0/1500254608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c009420 con 0x7f5eac159ec0 2023-10-10T20:23:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea259c700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400e930 tx=0x7f5e940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400eed0 con 0x7f5eac159ec0 2023-10-10T20:23:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9400c850 con 0x7f5eac159ec0 2023-10-10T20:23:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac0c7f90 con 0x7f5eac159ec0 2023-10-10T20:23:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400cbf0 con 0x7f5eac159ec0 2023-10-10T20:23:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.762+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac0c8550 con 0x7f5eac159ec0 2023-10-10T20:23:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.763+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0c8a70 con 0x7f5eac159ec0 2023-10-10T20:23:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.763+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e94013470 con 0x7f5eac159ec0 2023-10-10T20:23:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.764+0000 7f5ea37fe700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 0x7f5e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.764+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f5e94084710 con 0x7f5eac159ec0 2023-10-10T20:23:12.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.764+0000 7f5eaad9d700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 0x7f5e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:12.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.764+0000 7f5eaad9d700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 0x7f5e8c0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c003080 tx=0x7f5e9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:12.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.768+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5eac0c8a70 con 0x7f5eac159ec0 2023-10-10T20:23:12.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:12.994+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5eac0c8a30 con 0x7f5eac159ec0 2023-10-10T20:23:13.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.643+0000 7f5ea37fe700 1 -- 172.21.15.148:0/1500254608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5e94055470 con 0x7f5eac159ec0 2023-10-10T20:23:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 msgr2=0x7f5e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 0x7f5e8c0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c003080 tx=0x7f5e9c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:13.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 msgr2=0x7f5eac1586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400e930 tx=0x7f5e940091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 shutdown_connections 2023-10-10T20:23:13.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e8c066f30 0x7f5e8c0693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5eac15a290 0x7f5eac156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5eac06a5a0 0x7f5eac0cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 --2- 172.21.15.148:0/1500254608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eac159ec0 0x7f5eac1586b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.647+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 >> 172.21.15.148:0/1500254608 conn(0x7f5eac0d1880 msgr2=0x7f5eac0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:13.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.649+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 shutdown_connections 2023-10-10T20:23:13.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.649+0000 7f5eb118b700 1 -- 172.21.15.148:0/1500254608 wait complete. 2023-10-10T20:23:13.652 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 10 and data pool 11 2023-10-10T20:23:13.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:13.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.778+0000 7f383e0fe700 1 Processor -- start 2023-10-10T20:23:13.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.778+0000 7f383e0fe700 1 -- start start 2023-10-10T20:23:13.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f383e0fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f383815a6e0 0x7f3838068950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:13.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f383e0fe700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3838067660 con 0x7f383815b1c0 2023-10-10T20:23:13.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f383e0fe700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380684c0 con 0x7f383806f630 2023-10-10T20:23:13.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f383e0fe700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380678b0 con 0x7f383815a6e0 2023-10-10T20:23:13.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f38377fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f383815a6e0 0x7f3838068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:13.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f3837fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f383815b1c0 0x7f3838070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50192/0 (socket says 172.21.15.148:50192) 2023-10-10T20:23:13.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f38377fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f383815a6e0 0x7f3838068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54668/0 (socket says 172.21.15.148:54668) 2023-10-10T20:23:13.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.779+0000 7f3837fff700 1 -- 172.21.15.148:0/3624571420 learned_addr learned my addr 172.21.15.148:0/3624571420 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:13.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38377fe700 1 -- 172.21.15.148:0/3624571420 >> v1:172.21.15.161:6789/0 conn(0x7f383806f630 legacy=0x7f383815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:13.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38377fe700 1 -- 172.21.15.148:0/3624571420 >> v1:172.21.15.148:6789/0 conn(0x7f383815b1c0 legacy=0x7f3838070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38367fc700 1 -- 172.21.15.148:0/3624571420 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419306988 0 0) 0x7f3838067660 con 0x7f383815b1c0 2023-10-10T20:23:13.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38377fe700 1 -- 172.21.15.148:0/3624571420 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838067b90 con 0x7f383815a6e0 2023-10-10T20:23:13.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38377fe700 1 --2- 172.21.15.148:0/3624571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f3838068950 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3828009e60 tx=0x7f3828009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fccc7e59b933ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:13.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38367fc700 1 -- 172.21.15.148:0/3624571420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800e070 con 0x7f383815a6e0 2023-10-10T20:23:13.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38367fc700 1 -- 172.21.15.148:0/3624571420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800bbf0 con 0x7f383815a6e0 2023-10-10T20:23:13.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.780+0000 7f38367fc700 1 -- 172.21.15.148:0/3624571420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828012430 con 0x7f383815a6e0 2023-10-10T20:23:13.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 -- 172.21.15.148:0/3624571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 msgr2=0x7f3838068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 --2- 172.21.15.148:0/3624571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f3838068950 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3828009e60 tx=0x7f3828009780 comp rx=0 tx=0).stop 2023-10-10T20:23:13.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 -- 172.21.15.148:0/3624571420 shutdown_connections 2023-10-10T20:23:13.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 --2- 172.21.15.148:0/3624571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f3838068950 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 -- 172.21.15.148:0/3624571420 >> 172.21.15.148:0/3624571420 conn(0x7f383806a250 msgr2=0x7f383806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:13.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 -- 172.21.15.148:0/3624571420 shutdown_connections 2023-10-10T20:23:13.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.781+0000 7f383e0fe700 1 -- 172.21.15.148:0/3624571420 wait complete. 2023-10-10T20:23:13.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.782+0000 7f383e0fe700 1 Processor -- start 2023-10-10T20:23:13.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.782+0000 7f383e0fe700 1 -- start start 2023-10-10T20:23:13.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383806f630 0x7f383816dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:13.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:13.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815b1c0 0x7f3838152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:13.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383816e520 con 0x7f383815a6e0 2023-10-10T20:23:13.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380ad420 con 0x7f383806f630 2023-10-10T20:23:13.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380c4470 con 0x7f383815b1c0 2023-10-10T20:23:13.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:13.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54684/0 (socket says 172.21.15.148:54684) 2023-10-10T20:23:13.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 -- 172.21.15.148:0/526122233 learned_addr learned my addr 172.21.15.148:0/526122233 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:13.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3837fff700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815b1c0 0x7f3838152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 -- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815b1c0 msgr2=0x7f3838152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f38377fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383806f630 0x7f383816dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815b1c0 0x7f3838152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 -- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383806f630 msgr2=0x7f383816dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383806f630 0x7f383816dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 -- 172.21.15.148:0/526122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828009a20 con 0x7f383815a6e0 2023-10-10T20:23:13.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f3836ffd700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b5e0 tx=0x7f382c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:13.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00f510 con 0x7f383815a6e0 2023-10-10T20:23:13.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c00f6b0 con 0x7f383815a6e0 2023-10-10T20:23:13.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00fa70 con 0x7f383815a6e0 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38380ad670 con 0x7f383815a6e0 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.783+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38380adc30 con 0x7f383815a6e0 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.784+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38380ade50 con 0x7f383815a6e0 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.785+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f382c01a400 con 0x7f383815a6e0 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.785+0000 7f381f7fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:13.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.785+0000 7f38377fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:13.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.785+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f382c084080 con 0x7f383815a6e0 2023-10-10T20:23:13.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.786+0000 7f38377fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38280097c0 tx=0x7f3828018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:13.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.790+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f382c051370 con 0x7f383815a6e0 2023-10-10T20:23:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.973+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f383804c3c0 con 0x7f383815a6e0 2023-10-10T20:23:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.974+0000 7f381f7fe700 1 -- 172.21.15.148:0/526122233 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f382c054990 con 0x7f383815a6e0 2023-10-10T20:23:13.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:13.976 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":81,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:12.996420+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:13.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 msgr2=0x7f38240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38280097c0 tx=0x7f3828018000 comp rx=0 tx=0).stop 2023-10-10T20:23:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 msgr2=0x7f38380af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b5e0 tx=0x7f382c0090f0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 shutdown_connections 2023-10-10T20:23:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815b1c0 0x7f3838152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383806f630 0x7f383816dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 --2- 172.21.15.148:0/526122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383815a6e0 0x7f38380af700 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 >> 172.21.15.148:0/526122233 conn(0x7f383806a250 msgr2=0x7f38380c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 shutdown_connections 2023-10-10T20:23:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:13.976+0000 7f383e0fe700 1 -- 172.21.15.148:0/526122233 wait complete. 2023-10-10T20:23:13.990 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:14.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.103+0000 7f434ffff700 1 Processor -- start 2023-10-10T20:23:14.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.103+0000 7f434ffff700 1 -- start start 2023-10-10T20:23:14.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 0x7f435015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434ffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4350153e50 0x7f4350154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500cf0c0 con 0x7f435015aea0 2023-10-10T20:23:14.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434ffff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435015e8f0 con 0x7f4350153e50 2023-10-10T20:23:14.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435015b340 con 0x7f43500d0970 2023-10-10T20:23:14.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434effd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 0x7f435015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f43467fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f435015aea0 0x7f43500d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50196/0 (socket says 172.21.15.148:50196) 2023-10-10T20:23:14.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f43467fc700 1 -- 172.21.15.148:0/475595519 learned_addr learned my addr 172.21.15.148:0/475595519 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:14.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.104+0000 7f434f7fe700 1 --2- 172.21.15.148:0/475595519 >> v2:172.21.15.161:3300/0 conn(0x7f4350153e50 0x7f4350154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434e7fc700 1 -- 172.21.15.148:0/475595519 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286946885 0 0) 0x7f43500cf0c0 con 0x7f435015aea0 2023-10-10T20:23:14.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434e7fc700 1 -- 172.21.15.148:0/475595519 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f433c003570 con 0x7f435015aea0 2023-10-10T20:23:14.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434effd700 1 -- 172.21.15.148:0/475595519 >> v2:172.21.15.161:3300/0 conn(0x7f4350153e50 msgr2=0x7f4350154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:14.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434effd700 1 --2- 172.21.15.148:0/475595519 >> v2:172.21.15.161:3300/0 conn(0x7f4350153e50 0x7f4350154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434effd700 1 -- 172.21.15.148:0/475595519 >> v1:172.21.15.148:6789/0 conn(0x7f435015aea0 legacy=0x7f43500d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434effd700 1 -- 172.21.15.148:0/475595519 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435015e050 con 0x7f43500d0970 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434effd700 1 --2- 172.21.15.148:0/475595519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 0x7f435015b540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43400098e0 tx=0x7f43400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62dddd461c9491c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434e7fc700 1 -- 172.21.15.148:0/475595519 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340011630 con 0x7f43500d0970 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434e7fc700 1 -- 172.21.15.148:0/475595519 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340011790 con 0x7f43500d0970 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.105+0000 7f434e7fc700 1 -- 172.21.15.148:0/475595519 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340011b30 con 0x7f43500d0970 2023-10-10T20:23:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 -- 172.21.15.148:0/475595519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 msgr2=0x7f435015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 --2- 172.21.15.148:0/475595519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 0x7f435015b540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43400098e0 tx=0x7f43400091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 -- 172.21.15.148:0/475595519 shutdown_connections 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 --2- 172.21.15.148:0/475595519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43500d0970 0x7f435015b540 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 --2- 172.21.15.148:0/475595519 >> v2:172.21.15.161:3300/0 conn(0x7f4350153e50 0x7f4350154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 -- 172.21.15.148:0/475595519 >> 172.21.15.148:0/475595519 conn(0x7f4350068c00 msgr2=0x7f4350069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 -- 172.21.15.148:0/475595519 shutdown_connections 2023-10-10T20:23:14.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.106+0000 7f434ffff700 1 -- 172.21.15.148:0/475595519 wait complete. 2023-10-10T20:23:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.107+0000 7f434ffff700 1 Processor -- start 2023-10-10T20:23:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.107+0000 7f434ffff700 1 -- start start 2023-10-10T20:23:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4350153e50 0x7f435014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435015aea0 0x7f4350147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500b4b20 con 0x7f43500d0970 2023-10-10T20:23:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435014b9e0 con 0x7f435015aea0 2023-10-10T20:23:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500b41f0 con 0x7f4350153e50 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54694/0 (socket says 172.21.15.148:54694) 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 -- 172.21.15.148:0/3415535024 learned_addr learned my addr 172.21.15.148:0/3415535024 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f43467fc700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4350153e50 0x7f435014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434f7fe700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435015aea0 0x7f4350147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 -- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4350153e50 msgr2=0x7f435014b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4350153e50 0x7f435014b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 -- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435015aea0 msgr2=0x7f4350147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435015aea0 0x7f4350147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 -- 172.21.15.148:0/3415535024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009420 con 0x7f43500d0970 2023-10-10T20:23:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434effd700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43500899e0 tx=0x7f4340017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340011900 con 0x7f43500d0970 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340011a60 con 0x7f43500d0970 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500b44d0 con 0x7f43500d0970 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340025640 con 0x7f43500d0970 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.108+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4350147870 con 0x7f43500d0970 2023-10-10T20:23:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.109+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350089290 con 0x7f43500d0970 2023-10-10T20:23:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.110+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4340025a30 con 0x7f43500d0970 2023-10-10T20:23:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.110+0000 7f43477fe700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.110+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4340085c40 con 0x7f43500d0970 2023-10-10T20:23:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.110+0000 7f43467fc700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.111+0000 7f43467fc700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43500b62d0 tx=0x7f4338009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.114+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4340052eb0 con 0x7f43500d0970 2023-10-10T20:23:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.298+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f435015e330 con 0x7f43500d0970 2023-10-10T20:23:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.299+0000 7f43477fe700 1 -- 172.21.15.148:0/3415535024 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f43400564d0 con 0x7f43500d0970 2023-10-10T20:23:14.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:14.301 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":81,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:12.996420+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 msgr2=0x7f43300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43500b62d0 tx=0x7f4338009040 comp rx=0 tx=0).stop 2023-10-10T20:23:14.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 msgr2=0x7f435016acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43500899e0 tx=0x7f4340017000 comp rx=0 tx=0).stop 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 shutdown_connections 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435015aea0 0x7f4350147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43500d0970 0x7f435016acd0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 --2- 172.21.15.148:0/3415535024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4350153e50 0x7f435014b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.300+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 >> 172.21.15.148:0/3415535024 conn(0x7f4350068c00 msgr2=0x7f43500d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.301+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 shutdown_connections 2023-10-10T20:23:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.301+0000 7f434ffff700 1 -- 172.21.15.148:0/3415535024 wait complete. 2023-10-10T20:23:14.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:23:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.431+0000 7fe9aabf7700 1 Processor -- start 2023-10-10T20:23:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.431+0000 7fe9aabf7700 1 -- start start 2023-10-10T20:23:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9aabf7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a4159ec0 0x7fe9a40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9aabf7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9a415a290 0x7fe9a40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9aabf7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a40b0790 con 0x7fe9a406a5a0 2023-10-10T20:23:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9aabf7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a40ce1f0 con 0x7fe9a415a290 2023-10-10T20:23:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9aabf7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a40b0b20 con 0x7fe9a4159ec0 2023-10-10T20:23:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9a415a290 0x7fe9a40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9a415a290 0x7fe9a40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54700/0 (socket says 172.21.15.148:54700) 2023-10-10T20:23:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 -- 172.21.15.148:0/2989231732 learned_addr learned my addr 172.21.15.148:0/2989231732 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a8993700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a4159ec0 0x7fe9a40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 -- 172.21.15.148:0/2989231732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a4159ec0 msgr2=0x7fe9a40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a4159ec0 0x7fe9a40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 -- 172.21.15.148:0/2989231732 >> v1:172.21.15.161:6789/0 conn(0x7fe9a406a5a0 legacy=0x7fe9a415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 -- 172.21.15.148:0/2989231732 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a40b04b0 con 0x7fe9a415a290 2023-10-10T20:23:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.432+0000 7fe9a3fff700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a415a290 0x7fe9a40afe80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe9900099b0 tx=0x7fe990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46947ff1e466df4d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9a37fe700 1 -- 172.21.15.148:0/2989231732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990009ea0 con 0x7fe9a415a290 2023-10-10T20:23:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9a37fe700 1 -- 172.21.15.148:0/2989231732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99000b860 con 0x7fe9a415a290 2023-10-10T20:23:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9a37fe700 1 -- 172.21.15.148:0/2989231732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000bc00 con 0x7fe9a415a290 2023-10-10T20:23:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 -- 172.21.15.148:0/2989231732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a415a290 msgr2=0x7fe9a40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a415a290 0x7fe9a40afe80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe9900099b0 tx=0x7fe990009250 comp rx=0 tx=0).stop 2023-10-10T20:23:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 -- 172.21.15.148:0/2989231732 shutdown_connections 2023-10-10T20:23:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a4159ec0 0x7fe9a40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/2989231732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a415a290 0x7fe9a40afe80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 -- 172.21.15.148:0/2989231732 >> 172.21.15.148:0/2989231732 conn(0x7fe9a40d1880 msgr2=0x7fe9a40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 -- 172.21.15.148:0/2989231732 shutdown_connections 2023-10-10T20:23:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.433+0000 7fe9aabf7700 1 -- 172.21.15.148:0/2989231732 wait complete. 2023-10-10T20:23:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.434+0000 7fe9aabf7700 1 Processor -- start 2023-10-10T20:23:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.434+0000 7fe9aabf7700 1 -- start start 2023-10-10T20:23:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a4159ec0 0x7fe9a415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a415a290 0x7fe9a4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415ec10 con 0x7fe9a406a5a0 2023-10-10T20:23:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415eef0 con 0x7fe9a4159ec0 2023-10-10T20:23:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9aabf7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a415f1d0 con 0x7fe9a415a290 2023-10-10T20:23:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54710/0 (socket says 172.21.15.148:54710) 2023-10-10T20:23:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 -- 172.21.15.148:0/3043894850 learned_addr learned my addr 172.21.15.148:0/3043894850 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 -- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a415a290 msgr2=0x7fe9a4156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a9194700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a415a290 0x7fe9a4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a415a290 0x7fe9a4156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a8993700 1 -- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a4159ec0 msgr2=0x7fe9a415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:14.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.435+0000 7fe9a3fff700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a4159ec0 0x7fe9a415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a8993700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a4159ec0 0x7fe9a415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a8993700 1 -- 172.21.15.148:0/3043894850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900094f0 con 0x7fe9a406a5a0 2023-10-10T20:23:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a8993700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe99800ec80 tx=0x7fe9980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99800c9d0 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99800cb30 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a415f4b0 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998013470 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.436+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a415e870 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.437+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a415f640 con 0x7fe9a406a5a0 2023-10-10T20:23:14.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.437+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe998013820 con 0x7fe9a406a5a0 2023-10-10T20:23:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.438+0000 7fe9a0ff9700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 0x7fe9880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.438+0000 7fe9a3fff700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 0x7fe9880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.438+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe998084920 con 0x7fe9a406a5a0 2023-10-10T20:23:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.438+0000 7fe9a3fff700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 0x7fe9880693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe9900099b0 tx=0x7fe990017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.442+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe998051b90 con 0x7fe9a406a5a0 2023-10-10T20:23:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:14.668+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe9a40c8ff0 con 0x7fe9a406a5a0 2023-10-10T20:23:15.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.652+0000 7fe9a0ff9700 1 -- 172.21.15.148:0/3043894850 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9980551b0 con 0x7fe9a406a5a0 2023-10-10T20:23:15.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 msgr2=0x7fe9880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 0x7fe9880693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe9900099b0 tx=0x7fe990017000 comp rx=0 tx=0).stop 2023-10-10T20:23:15.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 msgr2=0x7fe9a415e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe99800ec80 tx=0x7fe9980091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 shutdown_connections 2023-10-10T20:23:15.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe988066f30 0x7fe9880693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9a415a290 0x7fe9a4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9a406a5a0 0x7fe9a415e330 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 --2- 172.21.15.148:0/3043894850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9a4159ec0 0x7fe9a415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.654+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 >> 172.21.15.148:0/3043894850 conn(0x7fe9a40d1880 msgr2=0x7fe9a40b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:15.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.655+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 shutdown_connections 2023-10-10T20:23:15.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.655+0000 7fe9aabf7700 1 -- 172.21.15.148:0/3043894850 wait complete. 2023-10-10T20:23:15.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:23:15.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.786+0000 7f677ece1700 1 Processor -- start 2023-10-10T20:23:15.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.786+0000 7f677ece1700 1 -- start start 2023-10-10T20:23:15.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677ece1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677ece1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67780cb9a0 0x7f67780af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677ece1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67780af8e0 con 0x7f67780c4a70 2023-10-10T20:23:15.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677ece1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780c9960 con 0x7f67780cb9a0 2023-10-10T20:23:15.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677ece1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780c3610 con 0x7f67780cb5d0 2023-10-10T20:23:15.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67780cb9a0 0x7f67780af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:15.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67780cb9a0 0x7f67780af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54714/0 (socket says 172.21.15.148:54714) 2023-10-10T20:23:15.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 -- 172.21.15.148:0/1017799087 learned_addr learned my addr 172.21.15.148:0/1017799087 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f677d27e700 1 --1- 172.21.15.148:0/1017799087 >> v1:172.21.15.148:6789/0 conn(0x7f67780c4a70 0x7f67780c4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50212/0 (socket says 172.21.15.148:50212) 2023-10-10T20:23:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 -- 172.21.15.148:0/1017799087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 msgr2=0x7f67780b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 --2- 172.21.15.148:0/1017799087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 -- 172.21.15.148:0/1017799087 >> v1:172.21.15.148:6789/0 conn(0x7f67780c4a70 legacy=0x7f67780c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 -- 172.21.15.148:0/1017799087 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780af6b0 con 0x7f67780cb9a0 2023-10-10T20:23:15.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f67777fe700 1 -- 172.21.15.148:0/1017799087 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079736878 0 0) 0x7f67780af8e0 con 0x7f67780c4a70 2023-10-10T20:23:15.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.787+0000 7f6777fff700 1 --2- 172.21.15.148:0/1017799087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780cb9a0 0x7f67780af170 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f67640099b0 tx=0x7f6764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360e21455b1ce956 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:15.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f67777fe700 1 -- 172.21.15.148:0/1017799087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400b8f0 con 0x7f67780cb9a0 2023-10-10T20:23:15.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f67777fe700 1 -- 172.21.15.148:0/1017799087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400ba50 con 0x7f67780cb9a0 2023-10-10T20:23:15.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f67777fe700 1 -- 172.21.15.148:0/1017799087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400bdf0 con 0x7f67780cb9a0 2023-10-10T20:23:15.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 -- 172.21.15.148:0/1017799087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780cb9a0 msgr2=0x7f67780af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 --2- 172.21.15.148:0/1017799087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780cb9a0 0x7f67780af170 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f67640099b0 tx=0x7f6764009250 comp rx=0 tx=0).stop 2023-10-10T20:23:15.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 -- 172.21.15.148:0/1017799087 shutdown_connections 2023-10-10T20:23:15.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 --2- 172.21.15.148:0/1017799087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 --2- 172.21.15.148:0/1017799087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780cb9a0 0x7f67780af170 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 -- 172.21.15.148:0/1017799087 >> 172.21.15.148:0/1017799087 conn(0x7f677806f3e0 msgr2=0x7f677806f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:15.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 -- 172.21.15.148:0/1017799087 shutdown_connections 2023-10-10T20:23:15.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.788+0000 7f677ece1700 1 -- 172.21.15.148:0/1017799087 wait complete. 2023-10-10T20:23:15.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.789+0000 7f677ece1700 1 Processor -- start 2023-10-10T20:23:15.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.789+0000 7f677ece1700 1 -- start start 2023-10-10T20:23:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780cb9a0 0x7f67780b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780c89d0 con 0x7f67780c4a70 2023-10-10T20:23:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677816db20 con 0x7f67780cb9a0 2023-10-10T20:23:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ece1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780c7dc0 con 0x7f67780cb5d0 2023-10-10T20:23:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54718/0 (socket says 172.21.15.148:54718) 2023-10-10T20:23:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 -- 172.21.15.148:0/1183768180 learned_addr learned my addr 172.21.15.148:0/1183768180 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 -- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 msgr2=0x7f67780b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 -- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780cb9a0 msgr2=0x7f67780b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780cb9a0 0x7f67780b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.790+0000 7f677ca7d700 1 -- 172.21.15.148:0/1183768180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640094f0 con 0x7f67780c4a70 2023-10-10T20:23:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f677ca7d700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f676c00b810 tx=0x7f676c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:15.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00f5f0 con 0x7f67780c4a70 2023-10-10T20:23:15.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c00f770 con 0x7f67780c4a70 2023-10-10T20:23:15.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00fb30 con 0x7f67780c4a70 2023-10-10T20:23:15.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67780c80a0 con 0x7f67780c4a70 2023-10-10T20:23:15.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.791+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67780c8660 con 0x7f67780c4a70 2023-10-10T20:23:15.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.792+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67780ae140 con 0x7f67780c4a70 2023-10-10T20:23:15.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.792+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f676c010480 con 0x7f67780c4a70 2023-10-10T20:23:15.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.792+0000 7f6774ff9700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 0x7f675c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:15.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.793+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f676c084490 con 0x7f67780c4a70 2023-10-10T20:23:15.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.793+0000 7f6777fff700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 0x7f675c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:15.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.793+0000 7f6777fff700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 0x7f675c0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67640099b0 tx=0x7f6764017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:15.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:15.797+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f676c051780 con 0x7f67780c4a70 2023-10-10T20:23:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.020+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f677816a800 con 0x7f67780c4a70 2023-10-10T20:23:16.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.653+0000 7f6774ff9700 1 -- 172.21.15.148:0/1183768180 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f676c054da0 con 0x7f67780c4a70 2023-10-10T20:23:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 msgr2=0x7f675c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 0x7f675c0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67640099b0 tx=0x7f6764017000 comp rx=0 tx=0).stop 2023-10-10T20:23:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 msgr2=0x7f677816d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f676c00b810 tx=0x7f676c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 shutdown_connections 2023-10-10T20:23:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f675c066f30 0x7f675c0693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780cb9a0 0x7f67780b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67780c4a70 0x7f677816d5e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 --2- 172.21.15.148:0/1183768180 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780cb5d0 0x7f67780b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 >> 172.21.15.148:0/1183768180 conn(0x7f677806f3e0 msgr2=0x7f67780c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 shutdown_connections 2023-10-10T20:23:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.656+0000 7f677ece1700 1 -- 172.21.15.148:0/1183768180 wait complete. 2023-10-10T20:23:16.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:16.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.785+0000 7fb95ccf6700 1 Processor -- start 2023-10-10T20:23:16.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.785+0000 7fb95ccf6700 1 -- start start 2023-10-10T20:23:16.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb95ccf6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb95ccf6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb95ccf6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9580c4300 con 0x7fb958157bc0 2023-10-10T20:23:16.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb95ccf6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580c57f0 con 0x7fb958157f90 2023-10-10T20:23:16.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb95ccf6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580c37c0 con 0x7fb958069ff0 2023-10-10T20:23:16.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb957fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb956ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb956ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:50710/0 (socket says 172.21.15.148:50710) 2023-10-10T20:23:16.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb956ffd700 1 -- 172.21.15.148:0/1561475200 learned_addr learned my addr 172.21.15.148:0/1561475200 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb957fff700 1 --2- 172.21.15.148:0/1561475200 >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:34000/0 (socket says 172.21.15.148:34000) 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.786+0000 7fb9577fe700 1 --1- 172.21.15.148:0/1561475200 >> v1:172.21.15.161:6789/0 conn(0x7fb958157bc0 0x7fb9580c50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48950/0 (socket says 172.21.15.148:48950) 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb956ffd700 1 -- 172.21.15.148:0/1561475200 >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 msgr2=0x7fb9580adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb956ffd700 1 --2- 172.21.15.148:0/1561475200 >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb956ffd700 1 -- 172.21.15.148:0/1561475200 >> v1:172.21.15.161:6789/0 conn(0x7fb958157bc0 legacy=0x7fb9580c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb956ffd700 1 -- 172.21.15.148:0/1561475200 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580c39d0 con 0x7fb958069ff0 2023-10-10T20:23:16.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb957fff700 1 --2- 172.21.15.148:0/1561475200 >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:16.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb956ffd700 1 --2- 172.21.15.148:0/1561475200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0099b0 tx=0x7fb94c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7db92ef81acd81b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:16.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb9567fc700 1 -- 172.21.15.148:0/1561475200 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c009ea0 con 0x7fb958069ff0 2023-10-10T20:23:16.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb9567fc700 1 -- 172.21.15.148:0/1561475200 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00b860 con 0x7fb958069ff0 2023-10-10T20:23:16.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb9567fc700 1 -- 172.21.15.148:0/1561475200 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00bc20 con 0x7fb958069ff0 2023-10-10T20:23:16.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb95ccf6700 1 -- 172.21.15.148:0/1561475200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 msgr2=0x7fb9580ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/1561475200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0099b0 tx=0x7fb94c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:16.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb95ccf6700 1 -- 172.21.15.148:0/1561475200 shutdown_connections 2023-10-10T20:23:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/1561475200 >> v2:172.21.15.161:3300/0 conn(0x7fb958157f90 0x7fb9580adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.787+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/1561475200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958069ff0 0x7fb9580ad6e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.788+0000 7fb95ccf6700 1 -- 172.21.15.148:0/1561475200 >> 172.21.15.148:0/1561475200 conn(0x7fb95806c700 msgr2=0x7fb958069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.788+0000 7fb95ccf6700 1 -- 172.21.15.148:0/1561475200 shutdown_connections 2023-10-10T20:23:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.788+0000 7fb95ccf6700 1 -- 172.21.15.148:0/1561475200 wait complete. 2023-10-10T20:23:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.789+0000 7fb95ccf6700 1 Processor -- start 2023-10-10T20:23:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.789+0000 7fb95ccf6700 1 -- start start 2023-10-10T20:23:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958157bc0 0x7fb958159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb958158690 con 0x7fb958157f90 2023-10-10T20:23:16.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb958155ac0 con 0x7fb958069ff0 2023-10-10T20:23:16.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb958155d40 con 0x7fb958157bc0 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54720/0 (socket says 172.21.15.148:54720) 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb956ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958157bc0 0x7fb958159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:34008/0 (socket says 172.21.15.148:34008) 2023-10-10T20:23:16.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 -- 172.21.15.148:0/2232310823 learned_addr learned my addr 172.21.15.148:0/2232310823 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:16.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 -- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958157bc0 msgr2=0x7fb958159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958157bc0 0x7fb958159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 -- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 msgr2=0x7fb958169110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:16.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:16.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 -- 172.21.15.148:0/2232310823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009a20 con 0x7fb958157f90 2023-10-10T20:23:16.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb9577fe700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:16.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb957fff700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c0f0 tx=0x7fb95000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:16.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000cc30 con 0x7fb958157f90 2023-10-10T20:23:16.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000f040 con 0x7fb958157f90 2023-10-10T20:23:16.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0094f0 con 0x7fb958157f90 2023-10-10T20:23:16.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950014480 con 0x7fb958157f90 2023-10-10T20:23:16.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.790+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb958156300 con 0x7fb958157f90 2023-10-10T20:23:16.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.791+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958158460 con 0x7fb958157f90 2023-10-10T20:23:16.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.792+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb950003020 con 0x7fb958157f90 2023-10-10T20:23:16.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.792+0000 7fb93f7fe700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 0x7fb9440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:16.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.792+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb950084eb0 con 0x7fb958157f90 2023-10-10T20:23:16.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.792+0000 7fb9577fe700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 0x7fb9440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:16.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.793+0000 7fb9577fe700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 0x7fb9440693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9581529a0 tx=0x7fb948007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:16.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:16.797+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb958158460 con 0x7fb958157f90 2023-10-10T20:23:17.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.020+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb958158460 con 0x7fb958157f90 2023-10-10T20:23:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.021+0000 7fb93f7fe700 1 -- 172.21.15.148:0/2232310823 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb958158460 con 0x7fb958157f90 2023-10-10T20:23:17.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:17.024 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:17.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 msgr2=0x7fb9440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 0x7fb9440693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9581529a0 tx=0x7fb948007eb0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 msgr2=0x7fb958155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c0f0 tx=0x7fb95000b040 comp rx=0 tx=0).stop 2023-10-10T20:23:17.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 shutdown_connections 2023-10-10T20:23:17.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb944066f30 0x7fb9440693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb958157f90 0x7fb958155580 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb958069ff0 0x7fb958169110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 --2- 172.21.15.148:0/2232310823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb958157bc0 0x7fb958159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 >> 172.21.15.148:0/2232310823 conn(0x7fb95806c700 msgr2=0x7fb9580cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 shutdown_connections 2023-10-10T20:23:17.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.023+0000 7fb95ccf6700 1 -- 172.21.15.148:0/2232310823 wait complete. 2023-10-10T20:23:17.029 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:17.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:17.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.150+0000 7f1c6d7f5700 1 Processor -- start 2023-10-10T20:23:17.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.150+0000 7f1c6d7f5700 1 -- start start 2023-10-10T20:23:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c6d7f5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c680cd6e0 0x7f1c680cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c6d7f5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 0x7f1c6815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c6d7f5700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680cb7c0 con 0x7f1c6815dda0 2023-10-10T20:23:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c6d7f5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6815e850 con 0x7f1c680cd6e0 2023-10-10T20:23:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c6d7f5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c680cd030 con 0x7f1c680cdab0 2023-10-10T20:23:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c680cd6e0 0x7f1c680cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c680cd6e0 0x7f1c680cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54724/0 (socket says 172.21.15.148:54724) 2023-10-10T20:23:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 -- 172.21.15.148:0/67815240 learned_addr learned my addr 172.21.15.148:0/67815240 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c677fe700 1 --2- 172.21.15.148:0/67815240 >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 0x7f1c6815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 -- 172.21.15.148:0/67815240 >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 msgr2=0x7f1c6815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 --2- 172.21.15.148:0/67815240 >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 0x7f1c6815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 -- 172.21.15.148:0/67815240 >> v1:172.21.15.161:6789/0 conn(0x7f1c6815dda0 legacy=0x7f1c6815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c67fff700 1 -- 172.21.15.148:0/67815240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6815e290 con 0x7f1c680cd6e0 2023-10-10T20:23:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.151+0000 7f1c677fe700 1 --2- 172.21.15.148:0/67815240 >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 0x7f1c6815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c67fff700 1 --2- 172.21.15.148:0/67815240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c680cbc50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1c680b6c70 tx=0x7f1c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf98f67c088790b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c66ffd700 1 -- 172.21.15.148:0/67815240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800b860 con 0x7f1c680cd6e0 2023-10-10T20:23:17.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c66ffd700 1 -- 172.21.15.148:0/67815240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800b9c0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c66ffd700 1 -- 172.21.15.148:0/67815240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800bd60 con 0x7f1c680cd6e0 2023-10-10T20:23:17.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/67815240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 msgr2=0x7f1c680cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/67815240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c680cbc50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1c680b6c70 tx=0x7f1c58009250 comp rx=0 tx=0).stop 2023-10-10T20:23:17.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/67815240 shutdown_connections 2023-10-10T20:23:17.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/67815240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c680cbc50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/67815240 >> v2:172.21.15.161:3300/0 conn(0x7f1c680cdab0 0x7f1c6815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/67815240 >> 172.21.15.148:0/67815240 conn(0x7f1c68067660 msgr2=0x7f1c68067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/67815240 shutdown_connections 2023-10-10T20:23:17.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.152+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/67815240 wait complete. 2023-10-10T20:23:17.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.153+0000 7f1c6d7f5700 1 Processor -- start 2023-10-10T20:23:17.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 -- start start 2023-10-10T20:23:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c680cdab0 0x7f1c6816b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6815dda0 0x7f1c680c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6816b4b0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680c95d0 con 0x7f1c6815dda0 2023-10-10T20:23:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c6d7f5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6816a9b0 con 0x7f1c680cdab0 2023-10-10T20:23:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54726/0 (socket says 172.21.15.148:54726) 2023-10-10T20:23:17.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.154+0000 7f1c67fff700 1 -- 172.21.15.148:0/1581195587 learned_addr learned my addr 172.21.15.148:0/1581195587 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c677fe700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c680cdab0 0x7f1c6816b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c6cff4700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6815dda0 0x7f1c680c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 -- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c680cdab0 msgr2=0x7f1c6816b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c680cdab0 0x7f1c6816b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 -- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6815dda0 msgr2=0x7f1c680c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6815dda0 0x7f1c680c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 -- 172.21.15.148:0/1581195587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580094f0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c67fff700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009290 tx=0x7f1c58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58011e60 con 0x7f1c680cd6e0 2023-10-10T20:23:17.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5801f070 con 0x7f1c680cd6e0 2023-10-10T20:23:17.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6816ac10 con 0x7f1c680cd6e0 2023-10-10T20:23:17.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5801a870 con 0x7f1c680cd6e0 2023-10-10T20:23:17.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.155+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6816b1d0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.156+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c680c98b0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.156+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c5801abe0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.157+0000 7f1c4ffff700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.157+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c58021070 con 0x7f1c680cd6e0 2023-10-10T20:23:17.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.157+0000 7f1c677fe700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.157+0000 7f1c677fe700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 0x7f1c540693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006f00 tx=0x7f1c5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.161+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c58053800 con 0x7f1c680cd6e0 2023-10-10T20:23:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.345+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c680c93a0 con 0x7f1c680cd6e0 2023-10-10T20:23:17.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.346+0000 7f1c4ffff700 1 -- 172.21.15.148:0/1581195587 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f1c58056e20 con 0x7f1c680cd6e0 2023-10-10T20:23:17.349 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:17.349 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":82,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:14.625780+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:17.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.348+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 msgr2=0x7f1c540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 0x7f1c540693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006f00 tx=0x7f1c5c008040 comp rx=0 tx=0).stop 2023-10-10T20:23:17.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 msgr2=0x7f1c6816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009290 tx=0x7f1c58017000 comp rx=0 tx=0).stop 2023-10-10T20:23:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 shutdown_connections 2023-10-10T20:23:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c54066f30 0x7f1c540693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c6815dda0 0x7f1c680c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c680cd6e0 0x7f1c6816a470 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 --2- 172.21.15.148:0/1581195587 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c680cdab0 0x7f1c6816b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 >> 172.21.15.148:0/1581195587 conn(0x7f1c68067660 msgr2=0x7f1c68157090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 shutdown_connections 2023-10-10T20:23:17.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.349+0000 7f1c6d7f5700 1 -- 172.21.15.148:0/1581195587 wait complete. 2023-10-10T20:23:17.363 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:23:17.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:23:17.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.475+0000 7f11329bf700 1 Processor -- start 2023-10-10T20:23:17.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.475+0000 7f11329bf700 1 -- start start 2023-10-10T20:23:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f11329bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c0b0cd0 0x7f112c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f11329bf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f112c15b820 0x7f112c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f11329bf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c0c9450 con 0x7f112c0b10a0 2023-10-10T20:23:17.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f11329bf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c0ce960 con 0x7f112c15b820 2023-10-10T20:23:17.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f11329bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0cc520 con 0x7f112c0b0cd0 2023-10-10T20:23:17.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f1130f5c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f112c15b820 0x7f112c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f112b7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f112c0b10a0 0x7f112c0ce250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50220/0 (socket says 172.21.15.148:50220) 2023-10-10T20:23:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f112b7fe700 1 -- 172.21.15.148:0/3829613387 learned_addr learned my addr 172.21.15.148:0/3829613387 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.476+0000 7f112bfff700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c0b0cd0 0x7f112c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f1130f5c700 1 -- 172.21.15.148:0/3829613387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c0b0cd0 msgr2=0x7f112c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f1130f5c700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c0b0cd0 0x7f112c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f1130f5c700 1 -- 172.21.15.148:0/3829613387 >> v1:172.21.15.148:6789/0 conn(0x7f112c0b10a0 legacy=0x7f112c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f112affd700 1 -- 172.21.15.148:0/3829613387 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490079649 0 0) 0x7f112c0c9450 con 0x7f112c0b10a0 2023-10-10T20:23:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f1130f5c700 1 -- 172.21.15.148:0/3829613387 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0cc910 con 0x7f112c15b820 2023-10-10T20:23:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f1130f5c700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c15b820 0x7f112c0ca930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e60 tx=0x7f1124009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9327333398c2714 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f112affd700 1 -- 172.21.15.148:0/3829613387 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400e070 con 0x7f112c15b820 2023-10-10T20:23:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f112affd700 1 -- 172.21.15.148:0/3829613387 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400bb60 con 0x7f112c15b820 2023-10-10T20:23:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f112affd700 1 -- 172.21.15.148:0/3829613387 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124012430 con 0x7f112c15b820 2023-10-10T20:23:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 -- 172.21.15.148:0/3829613387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c15b820 msgr2=0x7f112c0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c15b820 0x7f112c0ca930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e60 tx=0x7f1124009780 comp rx=0 tx=0).stop 2023-10-10T20:23:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 -- 172.21.15.148:0/3829613387 shutdown_connections 2023-10-10T20:23:17.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c15b820 0x7f112c0ca930 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 --2- 172.21.15.148:0/3829613387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c0b0cd0 0x7f112c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.477+0000 7f11329bf700 1 -- 172.21.15.148:0/3829613387 >> 172.21.15.148:0/3829613387 conn(0x7f112c06fdc0 msgr2=0x7f112c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.478+0000 7f11329bf700 1 -- 172.21.15.148:0/3829613387 shutdown_connections 2023-10-10T20:23:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.478+0000 7f11329bf700 1 -- 172.21.15.148:0/3829613387 wait complete. 2023-10-10T20:23:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 Processor -- start 2023-10-10T20:23:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 -- start start 2023-10-10T20:23:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c0b10a0 0x7f112c067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c15b820 0x7f112c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c068600 con 0x7f112c0b0cd0 2023-10-10T20:23:17.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0b38c0 con 0x7f112c0b10a0 2023-10-10T20:23:17.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.479+0000 7f11329bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c068040 con 0x7f112c15b820 2023-10-10T20:23:17.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54742/0 (socket says 172.21.15.148:54742) 2023-10-10T20:23:17.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 -- 172.21.15.148:0/261359279 learned_addr learned my addr 172.21.15.148:0/261359279 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:17.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f1130f5c700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c15b820 0x7f112c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112b7fe700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c0b10a0 0x7f112c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 -- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c15b820 msgr2=0x7f112c0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c15b820 0x7f112c0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 -- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c0b10a0 msgr2=0x7f112c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c0b10a0 0x7f112c067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 -- 172.21.15.148:0/261359279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124009a20 con 0x7f112c0b0cd0 2023-10-10T20:23:17.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f112bfff700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e930 tx=0x7f111c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c009510 con 0x7f112c0b0cd0 2023-10-10T20:23:17.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c00c850 con 0x7f112c0b0cd0 2023-10-10T20:23:17.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c068320 con 0x7f112c0b0cd0 2023-10-10T20:23:17.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c00cc10 con 0x7f112c0b0cd0 2023-10-10T20:23:17.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.480+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c0c8a00 con 0x7f112c0b0cd0 2023-10-10T20:23:17.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.481+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c04c3c0 con 0x7f112c0b0cd0 2023-10-10T20:23:17.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.482+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f111c013470 con 0x7f112c0b0cd0 2023-10-10T20:23:17.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.482+0000 7f1113fff700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.482+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f111c084670 con 0x7f112c0b0cd0 2023-10-10T20:23:17.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.482+0000 7f112b7fe700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.482+0000 7f112b7fe700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f112c167660 tx=0x7f1120009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.486+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f111c0518e0 con 0x7f112c0b0cd0 2023-10-10T20:23:17.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.667+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f112c0cf940 con 0x7f1118066f30 2023-10-10T20:23:17.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.675+0000 7f1113fff700 1 -- 172.21.15.148:0/261359279 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284998 (secure 0 0 0) 0x7f112c0cf940 con 0x7f1118066f30 2023-10-10T20:23:17.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:17.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 msgr2=0x7f11180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f112c167660 tx=0x7f1120009040 comp rx=0 tx=0).stop 2023-10-10T20:23:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 msgr2=0x7f112c0c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e930 tx=0x7f111c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 shutdown_connections 2023-10-10T20:23:17.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f112c15b820 0x7f112c0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f112c0b0cd0 0x7f112c0c84c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 --2- 172.21.15.148:0/261359279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c0b10a0 0x7f112c067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 >> 172.21.15.148:0/261359279 conn(0x7f112c06fdc0 msgr2=0x7f112c0c3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 shutdown_connections 2023-10-10T20:23:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.679+0000 7f11329bf700 1 -- 172.21.15.148:0/261359279 wait complete. 2023-10-10T20:23:17.686 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:23:17.693 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-10-10T20:23:17.374735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557963,"statfs":{"total":1600000000,"available":1595354674,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.765215"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640540+0000","last_change":"2023-10-10T20:23:13.640125+0000","last_active":"2023-10-10T20:23:13.640540+0000","last_peered":"2023-10-10T20:23:13.640540+0000","last_clean":"2023-10-10T20:23:13.640540+0000","last_became_active":"2023-10-10T20:23:13.639159+0000","last_became_peered":"2023-10-10T20:23:13.639159+0000","last_unstale":"2023-10-10T20:23:13.640540+0000","last_undegraded":"2023-10-10T20:23:13.640540+0000","last_fullsized":"2023-10-10T20:23:13.640540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:48.390723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644118+0000","last_change":"2023-10-10T20:23:13.644031+0000","last_active":"2023-10-10T20:23:13.644118+0000","last_peered":"2023-10-10T20:23:13.644118+0000","last_clean":"2023-10-10T20:23:13.644118+0000","last_became_active":"2023-10-10T20:23:13.643577+0000","last_became_peered":"2023-10-10T20:23:13.643577+0000","last_unstale":"2023-10-10T20:23:13.644118+0000","last_undegraded":"2023-10-10T20:23:13.644118+0000","last_fullsized":"2023-10-10T20:23:13.644118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:04.422064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636469+0000","last_change":"2023-10-10T20:23:13.635996+0000","last_active":"2023-10-10T20:23:13.636469+0000","last_peered":"2023-10-10T20:23:13.636469+0000","last_clean":"2023-10-10T20:23:13.636469+0000","last_became_active":"2023-10-10T20:23:13.635469+0000","last_became_peered":"2023-10-10T20:23:13.635469+0000","last_unstale":"2023-10-10T20:23:13.636469+0000","last_undegraded":"2023-10-10T20:23:13.636469+0000","last_fullsized":"2023-10-10T20:23:13.636469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:03:14.495529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637889+0000","last_change":"2023-10-10T20:23:13.637825+0000","last_active":"2023-10-10T20:23:13.637889+0000","last_peered":"2023-10-10T20:23:13.637889+0000","last_clean":"2023-10-10T20:23:13.637889+0000","last_became_active":"2023-10-10T20:23:13.637187+0000","last_became_peered":"2023-10-10T20:23:13.637187+0000","last_unstale":"2023-10-10T20:23:13.637889+0000","last_undegraded":"2023-10-10T20:23:13.637889+0000","last_fullsized":"2023-10-10T20:23:13.637889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:39.980831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636412+0000","last_change":"2023-10-10T20:23:13.636053+0000","last_active":"2023-10-10T20:23:13.636412+0000","last_peered":"2023-10-10T20:23:13.636412+0000","last_clean":"2023-10-10T20:23:13.636412+0000","last_became_active":"2023-10-10T20:23:13.635820+0000","last_became_peered":"2023-10-10T20:23:13.635820+0000","last_unstale":"2023-10-10T20:23:13.636412+0000","last_undegraded":"2023-10-10T20:23:13.636412+0000","last_fullsized":"2023-10-10T20:23:13.636412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:31.739028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644050+0000","last_change":"2023-10-10T20:23:13.643400+0000","last_active":"2023-10-10T20:23:13.644050+0000","last_peered":"2023-10-10T20:23:13.644050+0000","last_clean":"2023-10-10T20:23:13.644050+0000","last_became_active":"2023-10-10T20:23:13.642807+0000","last_became_peered":"2023-10-10T20:23:13.642807+0000","last_unstale":"2023-10-10T20:23:13.644050+0000","last_undegraded":"2023-10-10T20:23:13.644050+0000","last_fullsized":"2023-10-10T20:23:13.644050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636812+0000","last_change":"2023-10-10T20:23:13.636613+0000","last_active":"2023-10-10T20:23:13.636812+0000","last_peered":"2023-10-10T20:23:13.636812+0000","last_clean":"2023-10-10T20:23:13.636812+0000","last_became_active":"2023-10-10T20:23:13.636212+0000","last_became_peered":"2023-10-10T20:23:13.636212+0000","last_unstale":"2023-10-10T20:23:13.636812+0000","last_undegraded":"2023-10-10T20:23:13.636812+0000","last_fullsized":"2023-10-10T20:23:13.636812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:45.481112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645093+0000","last_change":"2023-10-10T20:23:13.644891+0000","last_active":"2023-10-10T20:23:13.645093+0000","last_peered":"2023-10-10T20:23:13.645093+0000","last_clean":"2023-10-10T20:23:13.645093+0000","last_became_active":"2023-10-10T20:23:13.644416+0000","last_became_peered":"2023-10-10T20:23:13.644416+0000","last_unstale":"2023-10-10T20:23:13.645093+0000","last_undegraded":"2023-10-10T20:23:13.645093+0000","last_fullsized":"2023-10-10T20:23:13.645093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:50.584878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633495+0000","last_change":"2023-10-10T20:23:13.632978+0000","last_active":"2023-10-10T20:23:13.633495+0000","last_peered":"2023-10-10T20:23:13.633495+0000","last_clean":"2023-10-10T20:23:13.633495+0000","last_became_active":"2023-10-10T20:23:13.632611+0000","last_became_peered":"2023-10-10T20:23:13.632611+0000","last_unstale":"2023-10-10T20:23:13.633495+0000","last_undegraded":"2023-10-10T20:23:13.633495+0000","last_fullsized":"2023-10-10T20:23:13.633495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:24.198632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640918+0000","last_change":"2023-10-10T20:23:13.640092+0000","last_active":"2023-10-10T20:23:13.640918+0000","last_peered":"2023-10-10T20:23:13.640918+0000","last_clean":"2023-10-10T20:23:13.640918+0000","last_became_active":"2023-10-10T20:23:13.639289+0000","last_became_peered":"2023-10-10T20:23:13.639289+0000","last_unstale":"2023-10-10T20:23:13.640918+0000","last_undegraded":"2023-10-10T20:23:13.640918+0000","last_fullsized":"2023-10-10T20:23:13.640918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:09.836472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640556+0000","last_change":"2023-10-10T20:23:13.640302+0000","last_active":"2023-10-10T20:23:13.640556+0000","last_peered":"2023-10-10T20:23:13.640556+0000","last_clean":"2023-10-10T20:23:13.640556+0000","last_became_active":"2023-10-10T20:23:13.639528+0000","last_became_peered":"2023-10-10T20:23:13.639528+0000","last_unstale":"2023-10-10T20:23:13.640556+0000","last_undegraded":"2023-10-10T20:23:13.640556+0000","last_fullsized":"2023-10-10T20:23:13.640556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:20.262895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638138+0000","last_change":"2023-10-10T20:23:13.638086+0000","last_active":"2023-10-10T20:23:13.638138+0000","last_peered":"2023-10-10T20:23:13.638138+0000","last_clean":"2023-10-10T20:23:13.638138+0000","last_became_active":"2023-10-10T20:23:13.637739+0000","last_became_peered":"2023-10-10T20:23:13.637739+0000","last_unstale":"2023-10-10T20:23:13.638138+0000","last_undegraded":"2023-10-10T20:23:13.638138+0000","last_fullsized":"2023-10-10T20:23:13.638138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:28.215742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.646091+0000","last_change":"2023-10-10T20:23:13.646034+0000","last_active":"2023-10-10T20:23:13.646091+0000","last_peered":"2023-10-10T20:23:13.646091+0000","last_clean":"2023-10-10T20:23:13.646091+0000","last_became_active":"2023-10-10T20:23:13.645721+0000","last_became_peered":"2023-10-10T20:23:13.645721+0000","last_unstale":"2023-10-10T20:23:13.646091+0000","last_undegraded":"2023-10-10T20:23:13.646091+0000","last_fullsized":"2023-10-10T20:23:13.646091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:20.011991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633478+0000","last_change":"2023-10-10T20:23:13.632835+0000","last_active":"2023-10-10T20:23:13.633478+0000","last_peered":"2023-10-10T20:23:13.633478+0000","last_clean":"2023-10-10T20:23:13.633478+0000","last_became_active":"2023-10-10T20:23:13.632310+0000","last_became_peered":"2023-10-10T20:23:13.632310+0000","last_unstale":"2023-10-10T20:23:13.633478+0000","last_undegraded":"2023-10-10T20:23:13.633478+0000","last_fullsized":"2023-10-10T20:23:13.633478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:18.021296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634701+0000","last_change":"2023-10-10T20:23:13.634500+0000","last_active":"2023-10-10T20:23:13.634701+0000","last_peered":"2023-10-10T20:23:13.634701+0000","last_clean":"2023-10-10T20:23:13.634701+0000","last_became_active":"2023-10-10T20:23:13.633969+0000","last_became_peered":"2023-10-10T20:23:13.633969+0000","last_unstale":"2023-10-10T20:23:13.634701+0000","last_undegraded":"2023-10-10T20:23:13.634701+0000","last_fullsized":"2023-10-10T20:23:13.634701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:45.958778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.641227+0000","last_change":"2023-10-10T20:23:13.641071+0000","last_active":"2023-10-10T20:23:13.641227+0000","last_peered":"2023-10-10T20:23:13.641227+0000","last_clean":"2023-10-10T20:23:13.641227+0000","last_became_active":"2023-10-10T20:23:13.640573+0000","last_became_peered":"2023-10-10T20:23:13.640573+0000","last_unstale":"2023-10-10T20:23:13.641227+0000","last_undegraded":"2023-10-10T20:23:13.641227+0000","last_fullsized":"2023-10-10T20:23:13.641227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634547+0000","last_change":"2023-10-10T20:23:13.634489+0000","last_active":"2023-10-10T20:23:13.634547+0000","last_peered":"2023-10-10T20:23:13.634547+0000","last_clean":"2023-10-10T20:23:13.634547+0000","last_became_active":"2023-10-10T20:23:13.634350+0000","last_became_peered":"2023-10-10T20:23:13.634350+0000","last_unstale":"2023-10-10T20:23:13.634547+0000","last_undegraded":"2023-10-10T20:23:13.634547+0000","last_fullsized":"2023-10-10T20:23:13.634547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:12.830771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.646025+0000","last_change":"2023-10-10T20:23:13.645916+0000","last_active":"2023-10-10T20:23:13.646025+0000","last_peered":"2023-10-10T20:23:13.646025+0000","last_clean":"2023-10-10T20:23:13.646025+0000","last_became_active":"2023-10-10T20:23:13.645599+0000","last_became_peered":"2023-10-10T20:23:13.645599+0000","last_unstale":"2023-10-10T20:23:13.646025+0000","last_undegraded":"2023-10-10T20:23:13.646025+0000","last_fullsized":"2023-10-10T20:23:13.646025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:37.648089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644071+0000","last_change":"2023-10-10T20:23:13.643503+0000","last_active":"2023-10-10T20:23:13.644071+0000","last_peered":"2023-10-10T20:23:13.644071+0000","last_clean":"2023-10-10T20:23:13.644071+0000","last_became_active":"2023-10-10T20:23:13.641271+0000","last_became_peered":"2023-10-10T20:23:13.641271+0000","last_unstale":"2023-10-10T20:23:13.644071+0000","last_undegraded":"2023-10-10T20:23:13.644071+0000","last_fullsized":"2023-10-10T20:23:13.644071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634718+0000","last_change":"2023-10-10T20:23:13.634630+0000","last_active":"2023-10-10T20:23:13.634718+0000","last_peered":"2023-10-10T20:23:13.634718+0000","last_clean":"2023-10-10T20:23:13.634718+0000","last_became_active":"2023-10-10T20:23:13.634285+0000","last_became_peered":"2023-10-10T20:23:13.634285+0000","last_unstale":"2023-10-10T20:23:13.634718+0000","last_undegraded":"2023-10-10T20:23:13.634718+0000","last_fullsized":"2023-10-10T20:23:13.634718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:01.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638990+0000","last_change":"2023-10-10T20:23:13.638823+0000","last_active":"2023-10-10T20:23:13.638990+0000","last_peered":"2023-10-10T20:23:13.638990+0000","last_clean":"2023-10-10T20:23:13.638990+0000","last_became_active":"2023-10-10T20:23:13.638569+0000","last_became_peered":"2023-10-10T20:23:13.638569+0000","last_unstale":"2023-10-10T20:23:13.638990+0000","last_undegraded":"2023-10-10T20:23:13.638990+0000","last_fullsized":"2023-10-10T20:23:13.638990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:06.922604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642351+0000","last_change":"2023-10-10T20:23:13.642217+0000","last_active":"2023-10-10T20:23:13.642351+0000","last_peered":"2023-10-10T20:23:13.642351+0000","last_clean":"2023-10-10T20:23:13.642351+0000","last_became_active":"2023-10-10T20:23:13.641215+0000","last_became_peered":"2023-10-10T20:23:13.641215+0000","last_unstale":"2023-10-10T20:23:13.642351+0000","last_undegraded":"2023-10-10T20:23:13.642351+0000","last_fullsized":"2023-10-10T20:23:13.642351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:04.422064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637491+0000","last_change":"2023-10-10T20:23:13.637344+0000","last_active":"2023-10-10T20:23:13.637491+0000","last_peered":"2023-10-10T20:23:13.637491+0000","last_clean":"2023-10-10T20:23:13.637491+0000","last_became_active":"2023-10-10T20:23:13.636655+0000","last_became_peered":"2023-10-10T20:23:13.636655+0000","last_unstale":"2023-10-10T20:23:13.637491+0000","last_undegraded":"2023-10-10T20:23:13.637491+0000","last_fullsized":"2023-10-10T20:23:13.637491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:35.052103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644861+0000","last_change":"2023-10-10T20:23:13.644806+0000","last_active":"2023-10-10T20:23:13.644861+0000","last_peered":"2023-10-10T20:23:13.644861+0000","last_clean":"2023-10-10T20:23:13.644861+0000","last_became_active":"2023-10-10T20:23:13.644598+0000","last_became_peered":"2023-10-10T20:23:13.644598+0000","last_unstale":"2023-10-10T20:23:13.644861+0000","last_undegraded":"2023-10-10T20:23:13.644861+0000","last_fullsized":"2023-10-10T20:23:13.644861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:32.514800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640940+0000","last_change":"2023-10-10T20:23:13.640240+0000","last_active":"2023-10-10T20:23:13.640940+0000","last_peered":"2023-10-10T20:23:13.640940+0000","last_clean":"2023-10-10T20:23:13.640940+0000","last_became_active":"2023-10-10T20:23:13.639834+0000","last_became_peered":"2023-10-10T20:23:13.639834+0000","last_unstale":"2023-10-10T20:23:13.640940+0000","last_undegraded":"2023-10-10T20:23:13.640940+0000","last_fullsized":"2023-10-10T20:23:13.640940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:26.348972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.639511+0000","last_change":"2023-10-10T20:23:13.639366+0000","last_active":"2023-10-10T20:23:13.639511+0000","last_peered":"2023-10-10T20:23:13.639511+0000","last_clean":"2023-10-10T20:23:13.639511+0000","last_became_active":"2023-10-10T20:23:13.639148+0000","last_became_peered":"2023-10-10T20:23:13.639148+0000","last_unstale":"2023-10-10T20:23:13.639511+0000","last_undegraded":"2023-10-10T20:23:13.639511+0000","last_fullsized":"2023-10-10T20:23:13.639511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:07.345694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636514+0000","last_change":"2023-10-10T20:23:13.636405+0000","last_active":"2023-10-10T20:23:13.636514+0000","last_peered":"2023-10-10T20:23:13.636514+0000","last_clean":"2023-10-10T20:23:13.636514+0000","last_became_active":"2023-10-10T20:23:13.635928+0000","last_became_peered":"2023-10-10T20:23:13.635928+0000","last_unstale":"2023-10-10T20:23:13.636514+0000","last_undegraded":"2023-10-10T20:23:13.636514+0000","last_fullsized":"2023-10-10T20:23:13.636514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:35.808252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638956+0000","last_change":"2023-10-10T20:23:13.638742+0000","last_active":"2023-10-10T20:23:13.638956+0000","last_peered":"2023-10-10T20:23:13.638956+0000","last_clean":"2023-10-10T20:23:13.638956+0000","last_became_active":"2023-10-10T20:23:13.638507+0000","last_became_peered":"2023-10-10T20:23:13.638507+0000","last_unstale":"2023-10-10T20:23:13.638956+0000","last_undegraded":"2023-10-10T20:23:13.638956+0000","last_fullsized":"2023-10-10T20:23:13.638956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:50.584878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633898+0000","last_change":"2023-10-10T20:23:13.633528+0000","last_active":"2023-10-10T20:23:13.633898+0000","last_peered":"2023-10-10T20:23:13.633898+0000","last_clean":"2023-10-10T20:23:13.633898+0000","last_became_active":"2023-10-10T20:23:13.633278+0000","last_became_peered":"2023-10-10T20:23:13.633278+0000","last_unstale":"2023-10-10T20:23:13.633898+0000","last_undegraded":"2023-10-10T20:23:13.633898+0000","last_fullsized":"2023-10-10T20:23:13.633898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:08.588110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638431+0000","last_change":"2023-10-10T20:23:13.638060+0000","last_active":"2023-10-10T20:23:13.638431+0000","last_peered":"2023-10-10T20:23:13.638431+0000","last_clean":"2023-10-10T20:23:13.638431+0000","last_became_active":"2023-10-10T20:23:13.637833+0000","last_became_peered":"2023-10-10T20:23:13.637833+0000","last_unstale":"2023-10-10T20:23:13.638431+0000","last_undegraded":"2023-10-10T20:23:13.638431+0000","last_fullsized":"2023-10-10T20:23:13.638431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638514+0000","last_change":"2023-10-10T20:23:13.638389+0000","last_active":"2023-10-10T20:23:13.638514+0000","last_peered":"2023-10-10T20:23:13.638514+0000","last_clean":"2023-10-10T20:23:13.638514+0000","last_became_active":"2023-10-10T20:23:13.638119+0000","last_became_peered":"2023-10-10T20:23:13.638119+0000","last_unstale":"2023-10-10T20:23:13.638514+0000","last_undegraded":"2023-10-10T20:23:13.638514+0000","last_fullsized":"2023-10-10T20:23:13.638514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644101+0000","last_change":"2023-10-10T20:23:13.643872+0000","last_active":"2023-10-10T20:23:13.644101+0000","last_peered":"2023-10-10T20:23:13.644101+0000","last_clean":"2023-10-10T20:23:13.644101+0000","last_became_active":"2023-10-10T20:23:13.643211+0000","last_became_peered":"2023-10-10T20:23:13.643211+0000","last_unstale":"2023-10-10T20:23:13.644101+0000","last_undegraded":"2023-10-10T20:23:13.644101+0000","last_fullsized":"2023-10-10T20:23:13.644101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:08:54.872779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.641116+0000","last_change":"2023-10-10T20:23:13.640903+0000","last_active":"2023-10-10T20:23:13.641116+0000","last_peered":"2023-10-10T20:23:13.641116+0000","last_clean":"2023-10-10T20:23:13.641116+0000","last_became_active":"2023-10-10T20:23:13.640509+0000","last_became_peered":"2023-10-10T20:23:13.640509+0000","last_unstale":"2023-10-10T20:23:13.641116+0000","last_undegraded":"2023-10-10T20:23:13.641116+0000","last_fullsized":"2023-10-10T20:23:13.641116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:26.034146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"81'4","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.652897+0000","last_change":"2023-10-10T20:23:12.622894+0000","last_active":"2023-10-10T20:23:13.652897+0000","last_peered":"2023-10-10T20:23:13.652897+0000","last_clean":"2023-10-10T20:23:13.652897+0000","last_became_active":"2023-10-10T20:23:12.622138+0000","last_became_peered":"2023-10-10T20:23:12.622138+0000","last_unstale":"2023-10-10T20:23:13.652897+0000","last_undegraded":"2023-10-10T20:23:13.652897+0000","last_fullsized":"2023-10-10T20:23:13.652897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:49.578349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634153+0000","last_change":"2023-10-10T20:23:13.634111+0000","last_active":"2023-10-10T20:23:13.634153+0000","last_peered":"2023-10-10T20:23:13.634153+0000","last_clean":"2023-10-10T20:23:13.634153+0000","last_became_active":"2023-10-10T20:23:13.633989+0000","last_became_peered":"2023-10-10T20:23:13.633989+0000","last_unstale":"2023-10-10T20:23:13.634153+0000","last_undegraded":"2023-10-10T20:23:13.634153+0000","last_fullsized":"2023-10-10T20:23:13.634153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:31:47.064488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649535+0000","last_change":"2023-10-10T20:23:12.617961+0000","last_active":"2023-10-10T20:23:13.649535+0000","last_peered":"2023-10-10T20:23:13.649535+0000","last_clean":"2023-10-10T20:23:13.649535+0000","last_became_active":"2023-10-10T20:23:12.616517+0000","last_became_peered":"2023-10-10T20:23:12.616517+0000","last_unstale":"2023-10-10T20:23:13.649535+0000","last_undegraded":"2023-10-10T20:23:13.649535+0000","last_fullsized":"2023-10-10T20:23:13.649535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:17.507798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644019+0000","last_change":"2023-10-10T20:23:13.641062+0000","last_active":"2023-10-10T20:23:13.644019+0000","last_peered":"2023-10-10T20:23:13.644019+0000","last_clean":"2023-10-10T20:23:13.644019+0000","last_became_active":"2023-10-10T20:23:13.640495+0000","last_became_peered":"2023-10-10T20:23:13.640495+0000","last_unstale":"2023-10-10T20:23:13.644019+0000","last_undegraded":"2023-10-10T20:23:13.644019+0000","last_fullsized":"2023-10-10T20:23:13.644019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:09.836472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651036+0000","last_change":"2023-10-10T20:23:12.619201+0000","last_active":"2023-10-10T20:23:13.651036+0000","last_peered":"2023-10-10T20:23:13.651036+0000","last_clean":"2023-10-10T20:23:13.651036+0000","last_became_active":"2023-10-10T20:23:12.618675+0000","last_became_peered":"2023-10-10T20:23:12.618675+0000","last_unstale":"2023-10-10T20:23:13.651036+0000","last_undegraded":"2023-10-10T20:23:13.651036+0000","last_fullsized":"2023-10-10T20:23:13.651036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:37:23.530954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640522+0000","last_change":"2023-10-10T20:23:13.639501+0000","last_active":"2023-10-10T20:23:13.640522+0000","last_peered":"2023-10-10T20:23:13.640522+0000","last_clean":"2023-10-10T20:23:13.640522+0000","last_became_active":"2023-10-10T20:23:13.639048+0000","last_became_peered":"2023-10-10T20:23:13.639048+0000","last_unstale":"2023-10-10T20:23:13.640522+0000","last_undegraded":"2023-10-10T20:23:13.640522+0000","last_fullsized":"2023-10-10T20:23:13.640522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:31.942566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650862+0000","last_change":"2023-10-10T20:23:12.622333+0000","last_active":"2023-10-10T20:23:13.650862+0000","last_peered":"2023-10-10T20:23:13.650862+0000","last_clean":"2023-10-10T20:23:13.650862+0000","last_became_active":"2023-10-10T20:23:12.621956+0000","last_became_peered":"2023-10-10T20:23:12.621956+0000","last_unstale":"2023-10-10T20:23:13.650862+0000","last_undegraded":"2023-10-10T20:23:13.650862+0000","last_fullsized":"2023-10-10T20:23:13.650862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:27.659618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636657+0000","last_change":"2023-10-10T20:23:13.636510+0000","last_active":"2023-10-10T20:23:13.636657+0000","last_peered":"2023-10-10T20:23:13.636657+0000","last_clean":"2023-10-10T20:23:13.636657+0000","last_became_active":"2023-10-10T20:23:13.636249+0000","last_became_peered":"2023-10-10T20:23:13.636249+0000","last_unstale":"2023-10-10T20:23:13.636657+0000","last_undegraded":"2023-10-10T20:23:13.636657+0000","last_fullsized":"2023-10-10T20:23:13.636657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649603+0000","last_change":"2023-10-10T20:23:12.616615+0000","last_active":"2023-10-10T20:23:13.649603+0000","last_peered":"2023-10-10T20:23:13.649603+0000","last_clean":"2023-10-10T20:23:13.649603+0000","last_became_active":"2023-10-10T20:23:12.616369+0000","last_became_peered":"2023-10-10T20:23:12.616369+0000","last_unstale":"2023-10-10T20:23:13.649603+0000","last_undegraded":"2023-10-10T20:23:13.649603+0000","last_fullsized":"2023-10-10T20:23:13.649603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:06:47.117125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642430+0000","last_change":"2023-10-10T20:23:13.642376+0000","last_active":"2023-10-10T20:23:13.642430+0000","last_peered":"2023-10-10T20:23:13.642430+0000","last_clean":"2023-10-10T20:23:13.642430+0000","last_became_active":"2023-10-10T20:23:13.642025+0000","last_became_peered":"2023-10-10T20:23:13.642025+0000","last_unstale":"2023-10-10T20:23:13.642430+0000","last_undegraded":"2023-10-10T20:23:13.642430+0000","last_fullsized":"2023-10-10T20:23:13.642430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649265+0000","last_change":"2023-10-10T20:23:12.621158+0000","last_active":"2023-10-10T20:23:13.649265+0000","last_peered":"2023-10-10T20:23:13.649265+0000","last_clean":"2023-10-10T20:23:13.649265+0000","last_became_active":"2023-10-10T20:23:12.620832+0000","last_became_peered":"2023-10-10T20:23:12.620832+0000","last_unstale":"2023-10-10T20:23:13.649265+0000","last_undegraded":"2023-10-10T20:23:13.649265+0000","last_fullsized":"2023-10-10T20:23:13.649265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:36.641560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637972+0000","last_change":"2023-10-10T20:23:13.637564+0000","last_active":"2023-10-10T20:23:13.637972+0000","last_peered":"2023-10-10T20:23:13.637972+0000","last_clean":"2023-10-10T20:23:13.637972+0000","last_became_active":"2023-10-10T20:23:13.637323+0000","last_became_peered":"2023-10-10T20:23:13.637323+0000","last_unstale":"2023-10-10T20:23:13.637972+0000","last_undegraded":"2023-10-10T20:23:13.637972+0000","last_fullsized":"2023-10-10T20:23:13.637972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:34.172505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.627733+0000","last_change":"2023-10-10T20:23:12.620901+0000","last_active":"2023-10-10T20:23:13.627733+0000","last_peered":"2023-10-10T20:23:13.627733+0000","last_clean":"2023-10-10T20:23:13.627733+0000","last_became_active":"2023-10-10T20:23:12.620517+0000","last_became_peered":"2023-10-10T20:23:12.620517+0000","last_unstale":"2023-10-10T20:23:13.627733+0000","last_undegraded":"2023-10-10T20:23:13.627733+0000","last_fullsized":"2023-10-10T20:23:13.627733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:00.531731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638626+0000","last_change":"2023-10-10T20:23:13.638562+0000","last_active":"2023-10-10T20:23:13.638626+0000","last_peered":"2023-10-10T20:23:13.638626+0000","last_clean":"2023-10-10T20:23:13.638626+0000","last_became_active":"2023-10-10T20:23:13.638325+0000","last_became_peered":"2023-10-10T20:23:13.638325+0000","last_unstale":"2023-10-10T20:23:13.638626+0000","last_undegraded":"2023-10-10T20:23:13.638626+0000","last_fullsized":"2023-10-10T20:23:13.638626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:54.970810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630230+0000","last_change":"2023-10-10T20:23:12.615528+0000","last_active":"2023-10-10T20:23:13.630230+0000","last_peered":"2023-10-10T20:23:13.630230+0000","last_clean":"2023-10-10T20:23:13.630230+0000","last_became_active":"2023-10-10T20:23:12.615300+0000","last_became_peered":"2023-10-10T20:23:12.615300+0000","last_unstale":"2023-10-10T20:23:13.630230+0000","last_undegraded":"2023-10-10T20:23:13.630230+0000","last_fullsized":"2023-10-10T20:23:13.630230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:40.542969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637371+0000","last_change":"2023-10-10T20:23:13.637251+0000","last_active":"2023-10-10T20:23:13.637371+0000","last_peered":"2023-10-10T20:23:13.637371+0000","last_clean":"2023-10-10T20:23:13.637371+0000","last_became_active":"2023-10-10T20:23:13.636998+0000","last_became_peered":"2023-10-10T20:23:13.636998+0000","last_unstale":"2023-10-10T20:23:13.637371+0000","last_undegraded":"2023-10-10T20:23:13.637371+0000","last_fullsized":"2023-10-10T20:23:13.637371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:18.584835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632048+0000","last_change":"2023-10-10T20:23:12.616413+0000","last_active":"2023-10-10T20:23:13.632048+0000","last_peered":"2023-10-10T20:23:13.632048+0000","last_clean":"2023-10-10T20:23:13.632048+0000","last_became_active":"2023-10-10T20:23:12.615631+0000","last_became_peered":"2023-10-10T20:23:12.615631+0000","last_unstale":"2023-10-10T20:23:13.632048+0000","last_undegraded":"2023-10-10T20:23:13.632048+0000","last_fullsized":"2023-10-10T20:23:13.632048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:42.574789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644751+0000","last_change":"2023-10-10T20:23:13.644679+0000","last_active":"2023-10-10T20:23:13.644751+0000","last_peered":"2023-10-10T20:23:13.644751+0000","last_clean":"2023-10-10T20:23:13.644751+0000","last_became_active":"2023-10-10T20:23:13.644452+0000","last_became_peered":"2023-10-10T20:23:13.644452+0000","last_unstale":"2023-10-10T20:23:13.644751+0000","last_undegraded":"2023-10-10T20:23:13.644751+0000","last_fullsized":"2023-10-10T20:23:13.644751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:45.292120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635331+0000","last_change":"2023-10-10T20:23:12.618793+0000","last_active":"2023-10-10T20:23:13.635331+0000","last_peered":"2023-10-10T20:23:13.635331+0000","last_clean":"2023-10-10T20:23:13.635331+0000","last_became_active":"2023-10-10T20:23:12.618260+0000","last_became_peered":"2023-10-10T20:23:12.618260+0000","last_unstale":"2023-10-10T20:23:13.635331+0000","last_undegraded":"2023-10-10T20:23:13.635331+0000","last_fullsized":"2023-10-10T20:23:13.635331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:39.205941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644034+0000","last_change":"2023-10-10T20:23:13.643033+0000","last_active":"2023-10-10T20:23:13.644034+0000","last_peered":"2023-10-10T20:23:13.644034+0000","last_clean":"2023-10-10T20:23:13.644034+0000","last_became_active":"2023-10-10T20:23:13.640933+0000","last_became_peered":"2023-10-10T20:23:13.640933+0000","last_unstale":"2023-10-10T20:23:13.644034+0000","last_undegraded":"2023-10-10T20:23:13.644034+0000","last_fullsized":"2023-10-10T20:23:13.644034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:10.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"81'3","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649755+0000","last_change":"2023-10-10T20:23:12.620984+0000","last_active":"2023-10-10T20:23:13.649755+0000","last_peered":"2023-10-10T20:23:13.649755+0000","last_clean":"2023-10-10T20:23:13.649755+0000","last_became_active":"2023-10-10T20:23:12.620522+0000","last_became_peered":"2023-10-10T20:23:12.620522+0000","last_unstale":"2023-10-10T20:23:13.649755+0000","last_undegraded":"2023-10-10T20:23:13.649755+0000","last_fullsized":"2023-10-10T20:23:13.649755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:30.936037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633431+0000","last_change":"2023-10-10T20:23:13.631640+0000","last_active":"2023-10-10T20:23:13.633431+0000","last_peered":"2023-10-10T20:23:13.633431+0000","last_clean":"2023-10-10T20:23:13.633431+0000","last_became_active":"2023-10-10T20:23:13.631362+0000","last_became_peered":"2023-10-10T20:23:13.631362+0000","last_unstale":"2023-10-10T20:23:13.633431+0000","last_undegraded":"2023-10-10T20:23:13.633431+0000","last_fullsized":"2023-10-10T20:23:13.633431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.332035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"81'6","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650847+0000","last_change":"2023-10-10T20:23:12.616388+0000","last_active":"2023-10-10T20:23:13.650847+0000","last_peered":"2023-10-10T20:23:13.650847+0000","last_clean":"2023-10-10T20:23:13.650847+0000","last_became_active":"2023-10-10T20:23:12.616163+0000","last_became_peered":"2023-10-10T20:23:12.616163+0000","last_unstale":"2023-10-10T20:23:13.650847+0000","last_undegraded":"2023-10-10T20:23:13.650847+0000","last_fullsized":"2023-10-10T20:23:13.650847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:19.435023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640572+0000","last_change":"2023-10-10T20:23:13.640444+0000","last_active":"2023-10-10T20:23:13.640572+0000","last_peered":"2023-10-10T20:23:13.640572+0000","last_clean":"2023-10-10T20:23:13.640572+0000","last_became_active":"2023-10-10T20:23:13.639877+0000","last_became_peered":"2023-10-10T20:23:13.639877+0000","last_unstale":"2023-10-10T20:23:13.640572+0000","last_undegraded":"2023-10-10T20:23:13.640572+0000","last_fullsized":"2023-10-10T20:23:13.640572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:59.753904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649244+0000","last_change":"2023-10-10T20:23:12.621577+0000","last_active":"2023-10-10T20:23:13.649244+0000","last_peered":"2023-10-10T20:23:13.649244+0000","last_clean":"2023-10-10T20:23:13.649244+0000","last_became_active":"2023-10-10T20:23:12.621356+0000","last_became_peered":"2023-10-10T20:23:12.621356+0000","last_unstale":"2023-10-10T20:23:13.649244+0000","last_undegraded":"2023-10-10T20:23:13.649244+0000","last_fullsized":"2023-10-10T20:23:13.649244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634825+0000","last_change":"2023-10-10T20:23:13.634723+0000","last_active":"2023-10-10T20:23:13.634825+0000","last_peered":"2023-10-10T20:23:13.634825+0000","last_clean":"2023-10-10T20:23:13.634825+0000","last_became_active":"2023-10-10T20:23:13.634506+0000","last_became_peered":"2023-10-10T20:23:13.634506+0000","last_unstale":"2023-10-10T20:23:13.634825+0000","last_undegraded":"2023-10-10T20:23:13.634825+0000","last_fullsized":"2023-10-10T20:23:13.634825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:57.019239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637360+0000","last_change":"2023-10-10T20:23:12.622168+0000","last_active":"2023-10-10T20:23:13.637360+0000","last_peered":"2023-10-10T20:23:13.637360+0000","last_clean":"2023-10-10T20:23:13.637360+0000","last_became_active":"2023-10-10T20:23:12.621333+0000","last_became_peered":"2023-10-10T20:23:12.621333+0000","last_unstale":"2023-10-10T20:23:13.637360+0000","last_undegraded":"2023-10-10T20:23:13.637360+0000","last_fullsized":"2023-10-10T20:23:13.637360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:38.974301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645950+0000","last_change":"2023-10-10T20:23:13.645900+0000","last_active":"2023-10-10T20:23:13.645950+0000","last_peered":"2023-10-10T20:23:13.645950+0000","last_clean":"2023-10-10T20:23:13.645950+0000","last_became_active":"2023-10-10T20:23:13.645423+0000","last_became_peered":"2023-10-10T20:23:13.645423+0000","last_unstale":"2023-10-10T20:23:13.645950+0000","last_undegraded":"2023-10-10T20:23:13.645950+0000","last_fullsized":"2023-10-10T20:23:13.645950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:32.514800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630791+0000","last_change":"2023-10-10T20:23:12.617061+0000","last_active":"2023-10-10T20:23:13.630791+0000","last_peered":"2023-10-10T20:23:13.630791+0000","last_clean":"2023-10-10T20:23:13.630791+0000","last_became_active":"2023-10-10T20:23:12.616463+0000","last_became_peered":"2023-10-10T20:23:12.616463+0000","last_unstale":"2023-10-10T20:23:13.630791+0000","last_undegraded":"2023-10-10T20:23:13.630791+0000","last_fullsized":"2023-10-10T20:23:13.630791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:36.302180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635875+0000","last_change":"2023-10-10T20:23:13.635799+0000","last_active":"2023-10-10T20:23:13.635875+0000","last_peered":"2023-10-10T20:23:13.635875+0000","last_clean":"2023-10-10T20:23:13.635875+0000","last_became_active":"2023-10-10T20:23:13.635434+0000","last_became_peered":"2023-10-10T20:23:13.635434+0000","last_unstale":"2023-10-10T20:23:13.635875+0000","last_undegraded":"2023-10-10T20:23:13.635875+0000","last_fullsized":"2023-10-10T20:23:13.635875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:20.441553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"81'8","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.652091+0000","last_change":"2023-10-10T20:23:12.622460+0000","last_active":"2023-10-10T20:23:13.652091+0000","last_peered":"2023-10-10T20:23:13.652091+0000","last_clean":"2023-10-10T20:23:13.652091+0000","last_became_active":"2023-10-10T20:23:12.622122+0000","last_became_peered":"2023-10-10T20:23:12.622122+0000","last_unstale":"2023-10-10T20:23:13.652091+0000","last_undegraded":"2023-10-10T20:23:13.652091+0000","last_fullsized":"2023-10-10T20:23:13.652091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:27.209212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632780+0000","last_change":"2023-10-10T20:23:13.632694+0000","last_active":"2023-10-10T20:23:13.632780+0000","last_peered":"2023-10-10T20:23:13.632780+0000","last_clean":"2023-10-10T20:23:13.632780+0000","last_became_active":"2023-10-10T20:23:13.632424+0000","last_became_peered":"2023-10-10T20:23:13.632424+0000","last_unstale":"2023-10-10T20:23:13.632780+0000","last_undegraded":"2023-10-10T20:23:13.632780+0000","last_fullsized":"2023-10-10T20:23:13.632780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650524+0000","last_change":"2023-10-10T20:23:12.618132+0000","last_active":"2023-10-10T20:23:13.650524+0000","last_peered":"2023-10-10T20:23:13.650524+0000","last_clean":"2023-10-10T20:23:13.650524+0000","last_became_active":"2023-10-10T20:23:12.617831+0000","last_became_peered":"2023-10-10T20:23:12.617831+0000","last_unstale":"2023-10-10T20:23:13.650524+0000","last_undegraded":"2023-10-10T20:23:13.650524+0000","last_fullsized":"2023-10-10T20:23:13.650524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:42.574789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640839+0000","last_change":"2023-10-10T20:23:13.640723+0000","last_active":"2023-10-10T20:23:13.640839+0000","last_peered":"2023-10-10T20:23:13.640839+0000","last_clean":"2023-10-10T20:23:13.640839+0000","last_became_active":"2023-10-10T20:23:13.640001+0000","last_became_peered":"2023-10-10T20:23:13.640001+0000","last_unstale":"2023-10-10T20:23:13.640839+0000","last_undegraded":"2023-10-10T20:23:13.640839+0000","last_fullsized":"2023-10-10T20:23:13.640839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:45.958778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651368+0000","last_change":"2023-10-10T20:23:12.619338+0000","last_active":"2023-10-10T20:23:13.651368+0000","last_peered":"2023-10-10T20:23:13.651368+0000","last_clean":"2023-10-10T20:23:13.651368+0000","last_became_active":"2023-10-10T20:23:12.618979+0000","last_became_peered":"2023-10-10T20:23:12.618979+0000","last_unstale":"2023-10-10T20:23:13.651368+0000","last_undegraded":"2023-10-10T20:23:13.651368+0000","last_fullsized":"2023-10-10T20:23:13.651368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:36.641560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638010+0000","last_change":"2023-10-10T20:23:13.637953+0000","last_active":"2023-10-10T20:23:13.638010+0000","last_peered":"2023-10-10T20:23:13.638010+0000","last_clean":"2023-10-10T20:23:13.638010+0000","last_became_active":"2023-10-10T20:23:13.637441+0000","last_became_peered":"2023-10-10T20:23:13.637441+0000","last_unstale":"2023-10-10T20:23:13.638010+0000","last_undegraded":"2023-10-10T20:23:13.638010+0000","last_fullsized":"2023-10-10T20:23:13.638010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:52.682236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651387+0000","last_change":"2023-10-10T20:23:12.623238+0000","last_active":"2023-10-10T20:23:13.651387+0000","last_peered":"2023-10-10T20:23:13.651387+0000","last_clean":"2023-10-10T20:23:13.651387+0000","last_became_active":"2023-10-10T20:23:12.622992+0000","last_became_peered":"2023-10-10T20:23:12.622992+0000","last_unstale":"2023-10-10T20:23:13.651387+0000","last_undegraded":"2023-10-10T20:23:13.651387+0000","last_fullsized":"2023-10-10T20:23:13.651387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:33.165975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637989+0000","last_change":"2023-10-10T20:23:13.637902+0000","last_active":"2023-10-10T20:23:13.637989+0000","last_peered":"2023-10-10T20:23:13.637989+0000","last_clean":"2023-10-10T20:23:13.637989+0000","last_became_active":"2023-10-10T20:23:13.637674+0000","last_became_peered":"2023-10-10T20:23:13.637674+0000","last_unstale":"2023-10-10T20:23:13.637989+0000","last_undegraded":"2023-10-10T20:23:13.637989+0000","last_fullsized":"2023-10-10T20:23:13.637989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:40.212470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634073+0000","last_change":"2023-10-10T20:23:12.624278+0000","last_active":"2023-10-10T20:23:13.634073+0000","last_peered":"2023-10-10T20:23:13.634073+0000","last_clean":"2023-10-10T20:23:13.634073+0000","last_became_active":"2023-10-10T20:23:12.624050+0000","last_became_peered":"2023-10-10T20:23:12.624050+0000","last_unstale":"2023-10-10T20:23:13.634073+0000","last_undegraded":"2023-10-10T20:23:13.634073+0000","last_fullsized":"2023-10-10T20:23:13.634073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:05.916075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636489+0000","last_change":"2023-10-10T20:23:13.636165+0000","last_active":"2023-10-10T20:23:13.636489+0000","last_peered":"2023-10-10T20:23:13.636489+0000","last_clean":"2023-10-10T20:23:13.636489+0000","last_became_active":"2023-10-10T20:23:13.635760+0000","last_became_peered":"2023-10-10T20:23:13.635760+0000","last_unstale":"2023-10-10T20:23:13.636489+0000","last_undegraded":"2023-10-10T20:23:13.636489+0000","last_fullsized":"2023-10-10T20:23:13.636489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:04:38.578668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636449+0000","last_change":"2023-10-10T20:23:12.620878+0000","last_active":"2023-10-10T20:23:13.636449+0000","last_peered":"2023-10-10T20:23:13.636449+0000","last_clean":"2023-10-10T20:23:13.636449+0000","last_became_active":"2023-10-10T20:23:12.620629+0000","last_became_peered":"2023-10-10T20:23:12.620629+0000","last_unstale":"2023-10-10T20:23:13.636449+0000","last_undegraded":"2023-10-10T20:23:13.636449+0000","last_fullsized":"2023-10-10T20:23:13.636449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:51.675707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.639286+0000","last_change":"2023-10-10T20:23:13.639235+0000","last_active":"2023-10-10T20:23:13.639286+0000","last_peered":"2023-10-10T20:23:13.639286+0000","last_clean":"2023-10-10T20:23:13.639286+0000","last_became_active":"2023-10-10T20:23:13.639052+0000","last_became_peered":"2023-10-10T20:23:13.639052+0000","last_unstale":"2023-10-10T20:23:13.639286+0000","last_undegraded":"2023-10-10T20:23:13.639286+0000","last_fullsized":"2023-10-10T20:23:13.639286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:17.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632659+0000","last_change":"2023-10-10T20:23:12.615450+0000","last_active":"2023-10-10T20:23:13.632659+0000","last_peered":"2023-10-10T20:23:13.632659+0000","last_clean":"2023-10-10T20:23:13.632659+0000","last_became_active":"2023-10-10T20:23:12.615222+0000","last_became_peered":"2023-10-10T20:23:12.615222+0000","last_unstale":"2023-10-10T20:23:13.632659+0000","last_undegraded":"2023-10-10T20:23:13.632659+0000","last_fullsized":"2023-10-10T20:23:13.632659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:43:39.728186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638584+0000","last_change":"2023-10-10T20:23:13.638543+0000","last_active":"2023-10-10T20:23:13.638584+0000","last_peered":"2023-10-10T20:23:13.638584+0000","last_clean":"2023-10-10T20:23:13.638584+0000","last_became_active":"2023-10-10T20:23:13.638380+0000","last_became_peered":"2023-10-10T20:23:13.638380+0000","last_unstale":"2023-10-10T20:23:13.638584+0000","last_undegraded":"2023-10-10T20:23:13.638584+0000","last_fullsized":"2023-10-10T20:23:13.638584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:14.272395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.629398+0000","last_change":"2023-10-10T20:23:12.617234+0000","last_active":"2023-10-10T20:23:13.629398+0000","last_peered":"2023-10-10T20:23:13.629398+0000","last_clean":"2023-10-10T20:23:13.629398+0000","last_became_active":"2023-10-10T20:23:12.616114+0000","last_became_peered":"2023-10-10T20:23:12.616114+0000","last_unstale":"2023-10-10T20:23:13.629398+0000","last_undegraded":"2023-10-10T20:23:13.629398+0000","last_fullsized":"2023-10-10T20:23:13.629398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:08.829943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636170+0000","last_change":"2023-10-10T20:23:13.635745+0000","last_active":"2023-10-10T20:23:13.636170+0000","last_peered":"2023-10-10T20:23:13.636170+0000","last_clean":"2023-10-10T20:23:13.636170+0000","last_became_active":"2023-10-10T20:23:13.635463+0000","last_became_peered":"2023-10-10T20:23:13.635463+0000","last_unstale":"2023-10-10T20:23:13.636170+0000","last_undegraded":"2023-10-10T20:23:13.636170+0000","last_fullsized":"2023-10-10T20:23:13.636170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:26.034146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650690+0000","last_change":"2023-10-10T20:23:12.618029+0000","last_active":"2023-10-10T20:23:13.650690+0000","last_peered":"2023-10-10T20:23:13.650690+0000","last_clean":"2023-10-10T20:23:13.650690+0000","last_became_active":"2023-10-10T20:23:12.617619+0000","last_became_peered":"2023-10-10T20:23:12.617619+0000","last_unstale":"2023-10-10T20:23:13.650690+0000","last_undegraded":"2023-10-10T20:23:13.650690+0000","last_fullsized":"2023-10-10T20:23:13.650690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636638+0000","last_change":"2023-10-10T20:23:13.636446+0000","last_active":"2023-10-10T20:23:13.636638+0000","last_peered":"2023-10-10T20:23:13.636638+0000","last_clean":"2023-10-10T20:23:13.636638+0000","last_became_active":"2023-10-10T20:23:13.635891+0000","last_became_peered":"2023-10-10T20:23:13.635891+0000","last_unstale":"2023-10-10T20:23:13.636638+0000","last_undegraded":"2023-10-10T20:23:13.636638+0000","last_fullsized":"2023-10-10T20:23:13.636638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:59:39.740113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630127+0000","last_change":"2023-10-10T20:23:12.615104+0000","last_active":"2023-10-10T20:23:13.630127+0000","last_peered":"2023-10-10T20:23:13.630127+0000","last_clean":"2023-10-10T20:23:13.630127+0000","last_became_active":"2023-10-10T20:23:12.614865+0000","last_became_peered":"2023-10-10T20:23:12.614865+0000","last_unstale":"2023-10-10T20:23:13.630127+0000","last_undegraded":"2023-10-10T20:23:13.630127+0000","last_fullsized":"2023-10-10T20:23:13.630127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:43:24.949892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644780+0000","last_change":"2023-10-10T20:23:13.644714+0000","last_active":"2023-10-10T20:23:13.644780+0000","last_peered":"2023-10-10T20:23:13.644780+0000","last_clean":"2023-10-10T20:23:13.644780+0000","last_became_active":"2023-10-10T20:23:13.644507+0000","last_became_peered":"2023-10-10T20:23:13.644507+0000","last_unstale":"2023-10-10T20:23:13.644780+0000","last_undegraded":"2023-10-10T20:23:13.644780+0000","last_fullsized":"2023-10-10T20:23:13.644780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:06:48.123654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630897+0000","last_change":"2023-10-10T20:23:12.615708+0000","last_active":"2023-10-10T20:23:13.630897+0000","last_peered":"2023-10-10T20:23:13.630897+0000","last_clean":"2023-10-10T20:23:13.630897+0000","last_became_active":"2023-10-10T20:23:12.615407+0000","last_became_peered":"2023-10-10T20:23:12.615407+0000","last_unstale":"2023-10-10T20:23:13.630897+0000","last_undegraded":"2023-10-10T20:23:13.630897+0000","last_fullsized":"2023-10-10T20:23:13.630897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:58.489753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642154+0000","last_change":"2023-10-10T20:23:13.642030+0000","last_active":"2023-10-10T20:23:13.642154+0000","last_peered":"2023-10-10T20:23:13.642154+0000","last_clean":"2023-10-10T20:23:13.642154+0000","last_became_active":"2023-10-10T20:23:13.641721+0000","last_became_peered":"2023-10-10T20:23:13.641721+0000","last_unstale":"2023-10-10T20:23:13.642154+0000","last_undegraded":"2023-10-10T20:23:13.642154+0000","last_fullsized":"2023-10-10T20:23:13.642154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:20.011991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630057+0000","last_change":"2023-10-10T20:23:12.620430+0000","last_active":"2023-10-10T20:23:13.630057+0000","last_peered":"2023-10-10T20:23:13.630057+0000","last_clean":"2023-10-10T20:23:13.630057+0000","last_became_active":"2023-10-10T20:23:12.620200+0000","last_became_peered":"2023-10-10T20:23:12.620200+0000","last_unstale":"2023-10-10T20:23:13.630057+0000","last_undegraded":"2023-10-10T20:23:13.630057+0000","last_fullsized":"2023-10-10T20:23:13.630057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:38.509700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635859+0000","last_change":"2023-10-10T20:23:13.635653+0000","last_active":"2023-10-10T20:23:13.635859+0000","last_peered":"2023-10-10T20:23:13.635859+0000","last_clean":"2023-10-10T20:23:13.635859+0000","last_became_active":"2023-10-10T20:23:13.635170+0000","last_became_peered":"2023-10-10T20:23:13.635170+0000","last_unstale":"2023-10-10T20:23:13.635859+0000","last_undegraded":"2023-10-10T20:23:13.635859+0000","last_fullsized":"2023-10-10T20:23:13.635859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:01.538260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.629529+0000","last_change":"2023-10-10T20:23:12.616491+0000","last_active":"2023-10-10T20:23:13.629529+0000","last_peered":"2023-10-10T20:23:13.629529+0000","last_clean":"2023-10-10T20:23:13.629529+0000","last_became_active":"2023-10-10T20:23:12.616271+0000","last_became_peered":"2023-10-10T20:23:12.616271+0000","last_unstale":"2023-10-10T20:23:13.629529+0000","last_undegraded":"2023-10-10T20:23:13.629529+0000","last_fullsized":"2023-10-10T20:23:13.629529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:52:28.130767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644084+0000","last_change":"2023-10-10T20:23:13.643582+0000","last_active":"2023-10-10T20:23:13.644084+0000","last_peered":"2023-10-10T20:23:13.644084+0000","last_clean":"2023-10-10T20:23:13.644084+0000","last_became_active":"2023-10-10T20:23:13.643157+0000","last_became_peered":"2023-10-10T20:23:13.643157+0000","last_unstale":"2023-10-10T20:23:13.644084+0000","last_undegraded":"2023-10-10T20:23:13.644084+0000","last_fullsized":"2023-10-10T20:23:13.644084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.627726+0000","last_change":"2023-10-10T20:23:12.621840+0000","last_active":"2023-10-10T20:23:13.627726+0000","last_peered":"2023-10-10T20:23:13.627726+0000","last_clean":"2023-10-10T20:23:13.627726+0000","last_became_active":"2023-10-10T20:23:12.621620+0000","last_became_peered":"2023-10-10T20:23:12.621620+0000","last_unstale":"2023-10-10T20:23:13.627726+0000","last_undegraded":"2023-10-10T20:23:13.627726+0000","last_fullsized":"2023-10-10T20:23:13.627726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:24.325506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632552+0000","last_change":"2023-10-10T20:23:13.632445+0000","last_active":"2023-10-10T20:23:13.632552+0000","last_peered":"2023-10-10T20:23:13.632552+0000","last_clean":"2023-10-10T20:23:13.632552+0000","last_became_active":"2023-10-10T20:23:13.632215+0000","last_became_peered":"2023-10-10T20:23:13.632215+0000","last_unstale":"2023-10-10T20:23:13.632552+0000","last_undegraded":"2023-10-10T20:23:13.632552+0000","last_fullsized":"2023-10-10T20:23:13.632552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:43.581318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.631439+0000","last_change":"2023-10-10T20:23:12.622318+0000","last_active":"2023-10-10T20:23:13.631439+0000","last_peered":"2023-10-10T20:23:13.631439+0000","last_clean":"2023-10-10T20:23:13.631439+0000","last_became_active":"2023-10-10T20:23:12.621849+0000","last_became_peered":"2023-10-10T20:23:12.621849+0000","last_unstale":"2023-10-10T20:23:13.631439+0000","last_undegraded":"2023-10-10T20:23:13.631439+0000","last_fullsized":"2023-10-10T20:23:13.631439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:51.675707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645076+0000","last_change":"2023-10-10T20:23:13.644714+0000","last_active":"2023-10-10T20:23:13.645076+0000","last_peered":"2023-10-10T20:23:13.645076+0000","last_clean":"2023-10-10T20:23:13.645076+0000","last_became_active":"2023-10-10T20:23:13.644128+0000","last_became_peered":"2023-10-10T20:23:13.644128+0000","last_unstale":"2023-10-10T20:23:13.645076+0000","last_undegraded":"2023-10-10T20:23:13.645076+0000","last_fullsized":"2023-10-10T20:23:13.645076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:12:42.699985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636582+0000","last_change":"2023-10-10T20:23:12.623803+0000","last_active":"2023-10-10T20:23:13.636582+0000","last_peered":"2023-10-10T20:23:13.636582+0000","last_clean":"2023-10-10T20:23:13.636582+0000","last_became_active":"2023-10-10T20:23:12.623545+0000","last_became_peered":"2023-10-10T20:23:12.623545+0000","last_unstale":"2023-10-10T20:23:13.636582+0000","last_undegraded":"2023-10-10T20:23:13.636582+0000","last_fullsized":"2023-10-10T20:23:13.636582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:59.225495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638096+0000","last_change":"2023-10-10T20:23:13.637951+0000","last_active":"2023-10-10T20:23:13.638096+0000","last_peered":"2023-10-10T20:23:13.638096+0000","last_clean":"2023-10-10T20:23:13.638096+0000","last_became_active":"2023-10-10T20:23:13.637744+0000","last_became_peered":"2023-10-10T20:23:13.637744+0000","last_unstale":"2023-10-10T20:23:13.638096+0000","last_undegraded":"2023-10-10T20:23:13.638096+0000","last_fullsized":"2023-10-10T20:23:13.638096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:50.090841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649503+0000","last_change":"2023-10-10T20:23:12.619360+0000","last_active":"2023-10-10T20:23:13.649503+0000","last_peered":"2023-10-10T20:23:13.649503+0000","last_clean":"2023-10-10T20:23:13.649503+0000","last_became_active":"2023-10-10T20:23:12.619053+0000","last_became_peered":"2023-10-10T20:23:12.619053+0000","last_unstale":"2023-10-10T20:23:13.649503+0000","last_undegraded":"2023-10-10T20:23:13.649503+0000","last_fullsized":"2023-10-10T20:23:13.649503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:30.732499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637460+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:23:13.637460+0000","last_peered":"2023-10-10T20:23:13.637460+0000","last_clean":"2023-10-10T20:23:13.637460+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:23:13.637460+0000","last_undegraded":"2023-10-10T20:23:13.637460+0000","last_fullsized":"2023-10-10T20:23:13.637460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":999,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198976294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66900000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.373,"5min":0.316,"15min":0.316},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73799999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68400000000000005,"5min":0.754,"15min":0.754},"last":0.52700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39100000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.377},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73899999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567158,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.56499999999999995,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.4219999999999997,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.435}]},{"osd":1,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0430000000000001,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.298,"5min":0.68500000000000005,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":15.862,"5min":15.862,"15min":15.862},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.29099999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.29299999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.39800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194887,"statfs":{"total":200000000,"available":199565126,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.50700000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.37,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54800000000000004,"5min":1.647,"15min":1.647},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56899999999999995,"5min":1.524,"15min":1.524},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.58799999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.25900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42499999999999999,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61199999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.311},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.317,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63,"5min":1.673,"15min":1.673},"last":0.33600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.633,"15min":1.633},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":1.506,"15min":1.506},"last":0.36199999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":1.696,"15min":1.696},"last":0.374},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.627,"15min":0.68300000000000005},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":1.585,"15min":1.585},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80500000000000005,"5min":1.478,"15min":1.478},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.722,"15min":1.722},"last":0.46000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":998,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194314,"statfs":{"total":200000000,"available":198977893,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.42499999999999999,"15min":0.47299999999999998},"min":{"1min":0.21099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55100000000000005,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.223},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.52200000000000002,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.45800000000000002,"15min":0.502},"min":{"1min":0.23599999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.46999999999999997,"5min":1.256,"15min":1.256},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.375,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41999999999999998,"15min":0.436},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.505,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.28299999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":1.204,"15min":1.204},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.875,"15min":1.0149999999999999},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81399999999999995,"5min":1.347,"15min":1.347},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70299999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78100000000000003,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76100000000000001,"5min":1.173,"15min":1.173},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.745}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":423,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199566725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.47299999999999998,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.497,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.51900000000000002,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48699999999999999,"5min":11.815,"15min":11.815},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.58399999999999996,"15min":0.46899999999999997},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58399999999999996,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.316}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74299999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72399999999999998,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":6.399,"15min":6.399},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.39000000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.66900000000000004,"15min":0.60299999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.60899999999999999},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66500000000000004,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76900000000000002,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.52000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40200000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48699999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.505,"5min":0.751,"15min":0.751},"last":0.371}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53100000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53700000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.496,"15min":0.47499999999999998},"min":{"1min":0.317,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.437}]},{"osd":1,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.44900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.46600000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.48499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.436,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.54000000000000004,"15min":0.51700000000000002},"min":{"1min":0.54000000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.499}]},{"osd":5,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.39000000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.39300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567160,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.505,"15min":0.497},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68799999999999994,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68799999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.875,"15min":0.875},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627}]}]}],"pool_statfs":[]}} 2023-10-10T20:23:17.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:23:17.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.844+0000 7fd9613c8700 1 Processor -- start 2023-10-10T20:23:17.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd9613c8700 1 -- start start 2023-10-10T20:23:17.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd9613c8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd95c15a620 0x7fd95c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd9613c8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c15cbb0 con 0x7fd95c157870 2023-10-10T20:23:17.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd9613c8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c15bfa0 con 0x7fd95c15d580 2023-10-10T20:23:17.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd9613c8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95c15bdb0 con 0x7fd95c15a620 2023-10-10T20:23:17.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd95affd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd95c157870 0x7fd95c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50226/0 (socket says 172.21.15.148:50226) 2023-10-10T20:23:17.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.845+0000 7fd95affd700 1 -- 172.21.15.148:0/3169838741 learned_addr learned my addr 172.21.15.148:0/3169838741 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:17.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd9527fc700 1 --2- 172.21.15.148:0/3169838741 >> v2:172.21.15.161:3300/0 conn(0x7fd95c15a620 0x7fd95c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737684118 0 0) 0x7fd95c15cbb0 con 0x7fd95c157870 2023-10-10T20:23:17.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003570 con 0x7fd95c157870 2023-10-10T20:23:17.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608754675 0 0) 0x7fd948003570 con 0x7fd95c157870 2023-10-10T20:23:17.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 >> v2:172.21.15.161:3300/0 conn(0x7fd95c15a620 msgr2=0x7fd95c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:17.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 --2- 172.21.15.148:0/3169838741 >> v2:172.21.15.161:3300/0 conn(0x7fd95c15a620 0x7fd95c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 >> v1:172.21.15.161:6789/0 conn(0x7fd95c15d580 legacy=0x7fd95c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c15c280 con 0x7fd95c157870 2023-10-10T20:23:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fd94c001470 con 0x7fd95c157870 2023-10-10T20:23:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 >> v1:172.21.15.148:6789/0 conn(0x7fd95c157870 legacy=0x7fd95c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 0x7fd948003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd948004810 0x7fd948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9480051c0 0x7fd94800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800c0b0 con 0x7fd948003a60 2023-10-10T20:23:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800c220 con 0x7fd9480051c0 2023-10-10T20:23:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800c3e0 con 0x7fd948004810 2023-10-10T20:23:17.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 0x7fd948003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95b7fe700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9480051c0 0x7fd94800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd9527fc700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd948004810 0x7fd948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 -- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd948004810 msgr2=0x7fd948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd948004810 0x7fd948004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 -- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9480051c0 msgr2=0x7fd94800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:17.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9480051c0 0x7fd94800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95affd700 1 -- 172.21.15.148:0/3169838741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948003570 con 0x7fd948003a60 2023-10-10T20:23:17.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95b7fe700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9480051c0 0x7fd94800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:17.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.846+0000 7fd95b7fe700 1 -- 172.21.15.148:0/3169838741 reap_dead start 2023-10-10T20:23:17.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd95affd700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 0x7fd948003e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c3c0 tx=0x7fd94c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d4a06dd3900769 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00cc90 con 0x7fd948003a60 2023-10-10T20:23:17.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00ce10 con 0x7fd948003a60 2023-10-10T20:23:17.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd95a7fc700 1 -- 172.21.15.148:0/3169838741 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c005b70 con 0x7fd948003a60 2023-10-10T20:23:17.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 -- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 msgr2=0x7fd948003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 0x7fd948003e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c3c0 tx=0x7fd94c001e70 comp rx=0 tx=0).stop 2023-10-10T20:23:17.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 -- 172.21.15.148:0/3169838741 shutdown_connections 2023-10-10T20:23:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 --2- 172.21.15.148:0/3169838741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd948003a60 0x7fd948003e30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 -- 172.21.15.148:0/3169838741 >> 172.21.15.148:0/3169838741 conn(0x7fd95c068ce0 msgr2=0x7fd95c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 -- 172.21.15.148:0/3169838741 shutdown_connections 2023-10-10T20:23:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.847+0000 7fd9613c8700 1 -- 172.21.15.148:0/3169838741 wait complete. 2023-10-10T20:23:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 Processor -- start 2023-10-10T20:23:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 -- start start 2023-10-10T20:23:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd948003a60 0x7fd95c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 0x7fd95c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c14d8f0 con 0x7fd95c16a6b0 2023-10-10T20:23:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b4860 con 0x7fd948003a60 2023-10-10T20:23:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9613c8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c14fe10 con 0x7fd95c14f0c0 2023-10-10T20:23:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9527fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9527fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54770/0 (socket says 172.21.15.148:54770) 2023-10-10T20:23:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd9527fc700 1 -- 172.21.15.148:0/1906847242 learned_addr learned my addr 172.21.15.148:0/1906847242 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.849+0000 7fd95affd700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd948003a60 0x7fd95c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd95b7fe700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 0x7fd95c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 -- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 msgr2=0x7fd95c14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 0x7fd95c14f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 -- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd948003a60 msgr2=0x7fd95c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:17.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd948003a60 0x7fd95c14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 -- 172.21.15.148:0/1906847242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c1500f0 con 0x7fd95c16a6b0 2023-10-10T20:23:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd95b7fe700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 0x7fd95c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9527fc700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd944011990 tx=0x7fd944008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94400f6d0 con 0x7fd95c16a6b0 2023-10-10T20:23:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94400f830 con 0x7fd95c16a6b0 2023-10-10T20:23:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94400fbf0 con 0x7fd95c16a6b0 2023-10-10T20:23:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0b4b00 con 0x7fd95c16a6b0 2023-10-10T20:23:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.850+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c16b040 con 0x7fd95c16a6b0 2023-10-10T20:23:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.851+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c0b5020 con 0x7fd95c16a6b0 2023-10-10T20:23:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.851+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd944015460 con 0x7fd95c16a6b0 2023-10-10T20:23:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.852+0000 7fd9537fe700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 0x7fd93c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.852+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd944084690 con 0x7fd95c16a6b0 2023-10-10T20:23:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.852+0000 7fd95affd700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 0x7fd93c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.852+0000 7fd95affd700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 0x7fd93c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007cf0 tx=0x7fd94c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:17.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:17.856+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd95c0b5020 con 0x7fd95c16a6b0 2023-10-10T20:23:18.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.034+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd95c0b5020 con 0x7fd93c066f30 2023-10-10T20:23:18.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.042+0000 7fd9537fe700 1 -- 172.21.15.148:0/1906847242 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284998 (secure 0 0 0) 0x7fd95c0b5020 con 0x7fd93c066f30 2023-10-10T20:23:18.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:18.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 msgr2=0x7fd93c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 0x7fd93c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007cf0 tx=0x7fd94c015000 comp rx=0 tx=0).stop 2023-10-10T20:23:18.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 msgr2=0x7fd95c16ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd944011990 tx=0x7fd944008040 comp rx=0 tx=0).stop 2023-10-10T20:23:18.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 shutdown_connections 2023-10-10T20:23:18.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd93c066f30 0x7fd93c0693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd95c14f0c0 0x7fd95c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd948003a60 0x7fd95c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 --2- 172.21.15.148:0/1906847242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c16a6b0 0x7fd95c16ab00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 >> 172.21.15.148:0/1906847242 conn(0x7fd95c068ce0 msgr2=0x7fd95c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 shutdown_connections 2023-10-10T20:23:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.045+0000 7fd9613c8700 1 -- 172.21.15.148:0/1906847242 wait complete. 2023-10-10T20:23:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:23:18.059 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-10-10T20:23:17.374735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557963,"statfs":{"total":1600000000,"available":1595354674,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.765215"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640540+0000","last_change":"2023-10-10T20:23:13.640125+0000","last_active":"2023-10-10T20:23:13.640540+0000","last_peered":"2023-10-10T20:23:13.640540+0000","last_clean":"2023-10-10T20:23:13.640540+0000","last_became_active":"2023-10-10T20:23:13.639159+0000","last_became_peered":"2023-10-10T20:23:13.639159+0000","last_unstale":"2023-10-10T20:23:13.640540+0000","last_undegraded":"2023-10-10T20:23:13.640540+0000","last_fullsized":"2023-10-10T20:23:13.640540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:48.390723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644118+0000","last_change":"2023-10-10T20:23:13.644031+0000","last_active":"2023-10-10T20:23:13.644118+0000","last_peered":"2023-10-10T20:23:13.644118+0000","last_clean":"2023-10-10T20:23:13.644118+0000","last_became_active":"2023-10-10T20:23:13.643577+0000","last_became_peered":"2023-10-10T20:23:13.643577+0000","last_unstale":"2023-10-10T20:23:13.644118+0000","last_undegraded":"2023-10-10T20:23:13.644118+0000","last_fullsized":"2023-10-10T20:23:13.644118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:04.422064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636469+0000","last_change":"2023-10-10T20:23:13.635996+0000","last_active":"2023-10-10T20:23:13.636469+0000","last_peered":"2023-10-10T20:23:13.636469+0000","last_clean":"2023-10-10T20:23:13.636469+0000","last_became_active":"2023-10-10T20:23:13.635469+0000","last_became_peered":"2023-10-10T20:23:13.635469+0000","last_unstale":"2023-10-10T20:23:13.636469+0000","last_undegraded":"2023-10-10T20:23:13.636469+0000","last_fullsized":"2023-10-10T20:23:13.636469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:03:14.495529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637889+0000","last_change":"2023-10-10T20:23:13.637825+0000","last_active":"2023-10-10T20:23:13.637889+0000","last_peered":"2023-10-10T20:23:13.637889+0000","last_clean":"2023-10-10T20:23:13.637889+0000","last_became_active":"2023-10-10T20:23:13.637187+0000","last_became_peered":"2023-10-10T20:23:13.637187+0000","last_unstale":"2023-10-10T20:23:13.637889+0000","last_undegraded":"2023-10-10T20:23:13.637889+0000","last_fullsized":"2023-10-10T20:23:13.637889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:39.980831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636412+0000","last_change":"2023-10-10T20:23:13.636053+0000","last_active":"2023-10-10T20:23:13.636412+0000","last_peered":"2023-10-10T20:23:13.636412+0000","last_clean":"2023-10-10T20:23:13.636412+0000","last_became_active":"2023-10-10T20:23:13.635820+0000","last_became_peered":"2023-10-10T20:23:13.635820+0000","last_unstale":"2023-10-10T20:23:13.636412+0000","last_undegraded":"2023-10-10T20:23:13.636412+0000","last_fullsized":"2023-10-10T20:23:13.636412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:31.739028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644050+0000","last_change":"2023-10-10T20:23:13.643400+0000","last_active":"2023-10-10T20:23:13.644050+0000","last_peered":"2023-10-10T20:23:13.644050+0000","last_clean":"2023-10-10T20:23:13.644050+0000","last_became_active":"2023-10-10T20:23:13.642807+0000","last_became_peered":"2023-10-10T20:23:13.642807+0000","last_unstale":"2023-10-10T20:23:13.644050+0000","last_undegraded":"2023-10-10T20:23:13.644050+0000","last_fullsized":"2023-10-10T20:23:13.644050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636812+0000","last_change":"2023-10-10T20:23:13.636613+0000","last_active":"2023-10-10T20:23:13.636812+0000","last_peered":"2023-10-10T20:23:13.636812+0000","last_clean":"2023-10-10T20:23:13.636812+0000","last_became_active":"2023-10-10T20:23:13.636212+0000","last_became_peered":"2023-10-10T20:23:13.636212+0000","last_unstale":"2023-10-10T20:23:13.636812+0000","last_undegraded":"2023-10-10T20:23:13.636812+0000","last_fullsized":"2023-10-10T20:23:13.636812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:45.481112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645093+0000","last_change":"2023-10-10T20:23:13.644891+0000","last_active":"2023-10-10T20:23:13.645093+0000","last_peered":"2023-10-10T20:23:13.645093+0000","last_clean":"2023-10-10T20:23:13.645093+0000","last_became_active":"2023-10-10T20:23:13.644416+0000","last_became_peered":"2023-10-10T20:23:13.644416+0000","last_unstale":"2023-10-10T20:23:13.645093+0000","last_undegraded":"2023-10-10T20:23:13.645093+0000","last_fullsized":"2023-10-10T20:23:13.645093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:50.584878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633495+0000","last_change":"2023-10-10T20:23:13.632978+0000","last_active":"2023-10-10T20:23:13.633495+0000","last_peered":"2023-10-10T20:23:13.633495+0000","last_clean":"2023-10-10T20:23:13.633495+0000","last_became_active":"2023-10-10T20:23:13.632611+0000","last_became_peered":"2023-10-10T20:23:13.632611+0000","last_unstale":"2023-10-10T20:23:13.633495+0000","last_undegraded":"2023-10-10T20:23:13.633495+0000","last_fullsized":"2023-10-10T20:23:13.633495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:24.198632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640918+0000","last_change":"2023-10-10T20:23:13.640092+0000","last_active":"2023-10-10T20:23:13.640918+0000","last_peered":"2023-10-10T20:23:13.640918+0000","last_clean":"2023-10-10T20:23:13.640918+0000","last_became_active":"2023-10-10T20:23:13.639289+0000","last_became_peered":"2023-10-10T20:23:13.639289+0000","last_unstale":"2023-10-10T20:23:13.640918+0000","last_undegraded":"2023-10-10T20:23:13.640918+0000","last_fullsized":"2023-10-10T20:23:13.640918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:09.836472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640556+0000","last_change":"2023-10-10T20:23:13.640302+0000","last_active":"2023-10-10T20:23:13.640556+0000","last_peered":"2023-10-10T20:23:13.640556+0000","last_clean":"2023-10-10T20:23:13.640556+0000","last_became_active":"2023-10-10T20:23:13.639528+0000","last_became_peered":"2023-10-10T20:23:13.639528+0000","last_unstale":"2023-10-10T20:23:13.640556+0000","last_undegraded":"2023-10-10T20:23:13.640556+0000","last_fullsized":"2023-10-10T20:23:13.640556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:55:20.262895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638138+0000","last_change":"2023-10-10T20:23:13.638086+0000","last_active":"2023-10-10T20:23:13.638138+0000","last_peered":"2023-10-10T20:23:13.638138+0000","last_clean":"2023-10-10T20:23:13.638138+0000","last_became_active":"2023-10-10T20:23:13.637739+0000","last_became_peered":"2023-10-10T20:23:13.637739+0000","last_unstale":"2023-10-10T20:23:13.638138+0000","last_undegraded":"2023-10-10T20:23:13.638138+0000","last_fullsized":"2023-10-10T20:23:13.638138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:28.215742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.646091+0000","last_change":"2023-10-10T20:23:13.646034+0000","last_active":"2023-10-10T20:23:13.646091+0000","last_peered":"2023-10-10T20:23:13.646091+0000","last_clean":"2023-10-10T20:23:13.646091+0000","last_became_active":"2023-10-10T20:23:13.645721+0000","last_became_peered":"2023-10-10T20:23:13.645721+0000","last_unstale":"2023-10-10T20:23:13.646091+0000","last_undegraded":"2023-10-10T20:23:13.646091+0000","last_fullsized":"2023-10-10T20:23:13.646091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:20.011991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633478+0000","last_change":"2023-10-10T20:23:13.632835+0000","last_active":"2023-10-10T20:23:13.633478+0000","last_peered":"2023-10-10T20:23:13.633478+0000","last_clean":"2023-10-10T20:23:13.633478+0000","last_became_active":"2023-10-10T20:23:13.632310+0000","last_became_peered":"2023-10-10T20:23:13.632310+0000","last_unstale":"2023-10-10T20:23:13.633478+0000","last_undegraded":"2023-10-10T20:23:13.633478+0000","last_fullsized":"2023-10-10T20:23:13.633478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:18.021296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634701+0000","last_change":"2023-10-10T20:23:13.634500+0000","last_active":"2023-10-10T20:23:13.634701+0000","last_peered":"2023-10-10T20:23:13.634701+0000","last_clean":"2023-10-10T20:23:13.634701+0000","last_became_active":"2023-10-10T20:23:13.633969+0000","last_became_peered":"2023-10-10T20:23:13.633969+0000","last_unstale":"2023-10-10T20:23:13.634701+0000","last_undegraded":"2023-10-10T20:23:13.634701+0000","last_fullsized":"2023-10-10T20:23:13.634701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:45.958778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.641227+0000","last_change":"2023-10-10T20:23:13.641071+0000","last_active":"2023-10-10T20:23:13.641227+0000","last_peered":"2023-10-10T20:23:13.641227+0000","last_clean":"2023-10-10T20:23:13.641227+0000","last_became_active":"2023-10-10T20:23:13.640573+0000","last_became_peered":"2023-10-10T20:23:13.640573+0000","last_unstale":"2023-10-10T20:23:13.641227+0000","last_undegraded":"2023-10-10T20:23:13.641227+0000","last_fullsized":"2023-10-10T20:23:13.641227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634547+0000","last_change":"2023-10-10T20:23:13.634489+0000","last_active":"2023-10-10T20:23:13.634547+0000","last_peered":"2023-10-10T20:23:13.634547+0000","last_clean":"2023-10-10T20:23:13.634547+0000","last_became_active":"2023-10-10T20:23:13.634350+0000","last_became_peered":"2023-10-10T20:23:13.634350+0000","last_unstale":"2023-10-10T20:23:13.634547+0000","last_undegraded":"2023-10-10T20:23:13.634547+0000","last_fullsized":"2023-10-10T20:23:13.634547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:12.830771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.646025+0000","last_change":"2023-10-10T20:23:13.645916+0000","last_active":"2023-10-10T20:23:13.646025+0000","last_peered":"2023-10-10T20:23:13.646025+0000","last_clean":"2023-10-10T20:23:13.646025+0000","last_became_active":"2023-10-10T20:23:13.645599+0000","last_became_peered":"2023-10-10T20:23:13.645599+0000","last_unstale":"2023-10-10T20:23:13.646025+0000","last_undegraded":"2023-10-10T20:23:13.646025+0000","last_fullsized":"2023-10-10T20:23:13.646025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:37.648089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644071+0000","last_change":"2023-10-10T20:23:13.643503+0000","last_active":"2023-10-10T20:23:13.644071+0000","last_peered":"2023-10-10T20:23:13.644071+0000","last_clean":"2023-10-10T20:23:13.644071+0000","last_became_active":"2023-10-10T20:23:13.641271+0000","last_became_peered":"2023-10-10T20:23:13.641271+0000","last_unstale":"2023-10-10T20:23:13.644071+0000","last_undegraded":"2023-10-10T20:23:13.644071+0000","last_fullsized":"2023-10-10T20:23:13.644071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634718+0000","last_change":"2023-10-10T20:23:13.634630+0000","last_active":"2023-10-10T20:23:13.634718+0000","last_peered":"2023-10-10T20:23:13.634718+0000","last_clean":"2023-10-10T20:23:13.634718+0000","last_became_active":"2023-10-10T20:23:13.634285+0000","last_became_peered":"2023-10-10T20:23:13.634285+0000","last_unstale":"2023-10-10T20:23:13.634718+0000","last_undegraded":"2023-10-10T20:23:13.634718+0000","last_fullsized":"2023-10-10T20:23:13.634718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:01.540350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638990+0000","last_change":"2023-10-10T20:23:13.638823+0000","last_active":"2023-10-10T20:23:13.638990+0000","last_peered":"2023-10-10T20:23:13.638990+0000","last_clean":"2023-10-10T20:23:13.638990+0000","last_became_active":"2023-10-10T20:23:13.638569+0000","last_became_peered":"2023-10-10T20:23:13.638569+0000","last_unstale":"2023-10-10T20:23:13.638990+0000","last_undegraded":"2023-10-10T20:23:13.638990+0000","last_fullsized":"2023-10-10T20:23:13.638990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:06.922604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642351+0000","last_change":"2023-10-10T20:23:13.642217+0000","last_active":"2023-10-10T20:23:13.642351+0000","last_peered":"2023-10-10T20:23:13.642351+0000","last_clean":"2023-10-10T20:23:13.642351+0000","last_became_active":"2023-10-10T20:23:13.641215+0000","last_became_peered":"2023-10-10T20:23:13.641215+0000","last_unstale":"2023-10-10T20:23:13.642351+0000","last_undegraded":"2023-10-10T20:23:13.642351+0000","last_fullsized":"2023-10-10T20:23:13.642351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:04.422064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637491+0000","last_change":"2023-10-10T20:23:13.637344+0000","last_active":"2023-10-10T20:23:13.637491+0000","last_peered":"2023-10-10T20:23:13.637491+0000","last_clean":"2023-10-10T20:23:13.637491+0000","last_became_active":"2023-10-10T20:23:13.636655+0000","last_became_peered":"2023-10-10T20:23:13.636655+0000","last_unstale":"2023-10-10T20:23:13.637491+0000","last_undegraded":"2023-10-10T20:23:13.637491+0000","last_fullsized":"2023-10-10T20:23:13.637491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:35.052103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644861+0000","last_change":"2023-10-10T20:23:13.644806+0000","last_active":"2023-10-10T20:23:13.644861+0000","last_peered":"2023-10-10T20:23:13.644861+0000","last_clean":"2023-10-10T20:23:13.644861+0000","last_became_active":"2023-10-10T20:23:13.644598+0000","last_became_peered":"2023-10-10T20:23:13.644598+0000","last_unstale":"2023-10-10T20:23:13.644861+0000","last_undegraded":"2023-10-10T20:23:13.644861+0000","last_fullsized":"2023-10-10T20:23:13.644861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:32.514800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640940+0000","last_change":"2023-10-10T20:23:13.640240+0000","last_active":"2023-10-10T20:23:13.640940+0000","last_peered":"2023-10-10T20:23:13.640940+0000","last_clean":"2023-10-10T20:23:13.640940+0000","last_became_active":"2023-10-10T20:23:13.639834+0000","last_became_peered":"2023-10-10T20:23:13.639834+0000","last_unstale":"2023-10-10T20:23:13.640940+0000","last_undegraded":"2023-10-10T20:23:13.640940+0000","last_fullsized":"2023-10-10T20:23:13.640940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:26.348972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.639511+0000","last_change":"2023-10-10T20:23:13.639366+0000","last_active":"2023-10-10T20:23:13.639511+0000","last_peered":"2023-10-10T20:23:13.639511+0000","last_clean":"2023-10-10T20:23:13.639511+0000","last_became_active":"2023-10-10T20:23:13.639148+0000","last_became_peered":"2023-10-10T20:23:13.639148+0000","last_unstale":"2023-10-10T20:23:13.639511+0000","last_undegraded":"2023-10-10T20:23:13.639511+0000","last_fullsized":"2023-10-10T20:23:13.639511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:44:07.345694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636514+0000","last_change":"2023-10-10T20:23:13.636405+0000","last_active":"2023-10-10T20:23:13.636514+0000","last_peered":"2023-10-10T20:23:13.636514+0000","last_clean":"2023-10-10T20:23:13.636514+0000","last_became_active":"2023-10-10T20:23:13.635928+0000","last_became_peered":"2023-10-10T20:23:13.635928+0000","last_unstale":"2023-10-10T20:23:13.636514+0000","last_undegraded":"2023-10-10T20:23:13.636514+0000","last_fullsized":"2023-10-10T20:23:13.636514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:35:35.808252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638956+0000","last_change":"2023-10-10T20:23:13.638742+0000","last_active":"2023-10-10T20:23:13.638956+0000","last_peered":"2023-10-10T20:23:13.638956+0000","last_clean":"2023-10-10T20:23:13.638956+0000","last_became_active":"2023-10-10T20:23:13.638507+0000","last_became_peered":"2023-10-10T20:23:13.638507+0000","last_unstale":"2023-10-10T20:23:13.638956+0000","last_undegraded":"2023-10-10T20:23:13.638956+0000","last_fullsized":"2023-10-10T20:23:13.638956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:50.584878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633898+0000","last_change":"2023-10-10T20:23:13.633528+0000","last_active":"2023-10-10T20:23:13.633898+0000","last_peered":"2023-10-10T20:23:13.633898+0000","last_clean":"2023-10-10T20:23:13.633898+0000","last_became_active":"2023-10-10T20:23:13.633278+0000","last_became_peered":"2023-10-10T20:23:13.633278+0000","last_unstale":"2023-10-10T20:23:13.633898+0000","last_undegraded":"2023-10-10T20:23:13.633898+0000","last_fullsized":"2023-10-10T20:23:13.633898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:09:08.588110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638431+0000","last_change":"2023-10-10T20:23:13.638060+0000","last_active":"2023-10-10T20:23:13.638431+0000","last_peered":"2023-10-10T20:23:13.638431+0000","last_clean":"2023-10-10T20:23:13.638431+0000","last_became_active":"2023-10-10T20:23:13.637833+0000","last_became_peered":"2023-10-10T20:23:13.637833+0000","last_unstale":"2023-10-10T20:23:13.638431+0000","last_undegraded":"2023-10-10T20:23:13.638431+0000","last_fullsized":"2023-10-10T20:23:13.638431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638514+0000","last_change":"2023-10-10T20:23:13.638389+0000","last_active":"2023-10-10T20:23:13.638514+0000","last_peered":"2023-10-10T20:23:13.638514+0000","last_clean":"2023-10-10T20:23:13.638514+0000","last_became_active":"2023-10-10T20:23:13.638119+0000","last_became_peered":"2023-10-10T20:23:13.638119+0000","last_unstale":"2023-10-10T20:23:13.638514+0000","last_undegraded":"2023-10-10T20:23:13.638514+0000","last_fullsized":"2023-10-10T20:23:13.638514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:39.516230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644101+0000","last_change":"2023-10-10T20:23:13.643872+0000","last_active":"2023-10-10T20:23:13.644101+0000","last_peered":"2023-10-10T20:23:13.644101+0000","last_clean":"2023-10-10T20:23:13.644101+0000","last_became_active":"2023-10-10T20:23:13.643211+0000","last_became_peered":"2023-10-10T20:23:13.643211+0000","last_unstale":"2023-10-10T20:23:13.644101+0000","last_undegraded":"2023-10-10T20:23:13.644101+0000","last_fullsized":"2023-10-10T20:23:13.644101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:08:54.872779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.641116+0000","last_change":"2023-10-10T20:23:13.640903+0000","last_active":"2023-10-10T20:23:13.641116+0000","last_peered":"2023-10-10T20:23:13.641116+0000","last_clean":"2023-10-10T20:23:13.641116+0000","last_became_active":"2023-10-10T20:23:13.640509+0000","last_became_peered":"2023-10-10T20:23:13.640509+0000","last_unstale":"2023-10-10T20:23:13.641116+0000","last_undegraded":"2023-10-10T20:23:13.641116+0000","last_fullsized":"2023-10-10T20:23:13.641116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:26.034146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"81'4","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.652897+0000","last_change":"2023-10-10T20:23:12.622894+0000","last_active":"2023-10-10T20:23:13.652897+0000","last_peered":"2023-10-10T20:23:13.652897+0000","last_clean":"2023-10-10T20:23:13.652897+0000","last_became_active":"2023-10-10T20:23:12.622138+0000","last_became_peered":"2023-10-10T20:23:12.622138+0000","last_unstale":"2023-10-10T20:23:13.652897+0000","last_undegraded":"2023-10-10T20:23:13.652897+0000","last_fullsized":"2023-10-10T20:23:13.652897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:38:49.578349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634153+0000","last_change":"2023-10-10T20:23:13.634111+0000","last_active":"2023-10-10T20:23:13.634153+0000","last_peered":"2023-10-10T20:23:13.634153+0000","last_clean":"2023-10-10T20:23:13.634153+0000","last_became_active":"2023-10-10T20:23:13.633989+0000","last_became_peered":"2023-10-10T20:23:13.633989+0000","last_unstale":"2023-10-10T20:23:13.634153+0000","last_undegraded":"2023-10-10T20:23:13.634153+0000","last_fullsized":"2023-10-10T20:23:13.634153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:31:47.064488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649535+0000","last_change":"2023-10-10T20:23:12.617961+0000","last_active":"2023-10-10T20:23:13.649535+0000","last_peered":"2023-10-10T20:23:13.649535+0000","last_clean":"2023-10-10T20:23:13.649535+0000","last_became_active":"2023-10-10T20:23:12.616517+0000","last_became_peered":"2023-10-10T20:23:12.616517+0000","last_unstale":"2023-10-10T20:23:13.649535+0000","last_undegraded":"2023-10-10T20:23:13.649535+0000","last_fullsized":"2023-10-10T20:23:13.649535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:17.507798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644019+0000","last_change":"2023-10-10T20:23:13.641062+0000","last_active":"2023-10-10T20:23:13.644019+0000","last_peered":"2023-10-10T20:23:13.644019+0000","last_clean":"2023-10-10T20:23:13.644019+0000","last_became_active":"2023-10-10T20:23:13.640495+0000","last_became_peered":"2023-10-10T20:23:13.640495+0000","last_unstale":"2023-10-10T20:23:13.644019+0000","last_undegraded":"2023-10-10T20:23:13.644019+0000","last_fullsized":"2023-10-10T20:23:13.644019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:09.836472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651036+0000","last_change":"2023-10-10T20:23:12.619201+0000","last_active":"2023-10-10T20:23:13.651036+0000","last_peered":"2023-10-10T20:23:13.651036+0000","last_clean":"2023-10-10T20:23:13.651036+0000","last_became_active":"2023-10-10T20:23:12.618675+0000","last_became_peered":"2023-10-10T20:23:12.618675+0000","last_unstale":"2023-10-10T20:23:13.651036+0000","last_undegraded":"2023-10-10T20:23:13.651036+0000","last_fullsized":"2023-10-10T20:23:13.651036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:37:23.530954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640522+0000","last_change":"2023-10-10T20:23:13.639501+0000","last_active":"2023-10-10T20:23:13.640522+0000","last_peered":"2023-10-10T20:23:13.640522+0000","last_clean":"2023-10-10T20:23:13.640522+0000","last_became_active":"2023-10-10T20:23:13.639048+0000","last_became_peered":"2023-10-10T20:23:13.639048+0000","last_unstale":"2023-10-10T20:23:13.640522+0000","last_undegraded":"2023-10-10T20:23:13.640522+0000","last_fullsized":"2023-10-10T20:23:13.640522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:31.942566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650862+0000","last_change":"2023-10-10T20:23:12.622333+0000","last_active":"2023-10-10T20:23:13.650862+0000","last_peered":"2023-10-10T20:23:13.650862+0000","last_clean":"2023-10-10T20:23:13.650862+0000","last_became_active":"2023-10-10T20:23:12.621956+0000","last_became_peered":"2023-10-10T20:23:12.621956+0000","last_unstale":"2023-10-10T20:23:13.650862+0000","last_undegraded":"2023-10-10T20:23:13.650862+0000","last_fullsized":"2023-10-10T20:23:13.650862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:27.659618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636657+0000","last_change":"2023-10-10T20:23:13.636510+0000","last_active":"2023-10-10T20:23:13.636657+0000","last_peered":"2023-10-10T20:23:13.636657+0000","last_clean":"2023-10-10T20:23:13.636657+0000","last_became_active":"2023-10-10T20:23:13.636249+0000","last_became_peered":"2023-10-10T20:23:13.636249+0000","last_unstale":"2023-10-10T20:23:13.636657+0000","last_undegraded":"2023-10-10T20:23:13.636657+0000","last_fullsized":"2023-10-10T20:23:13.636657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649603+0000","last_change":"2023-10-10T20:23:12.616615+0000","last_active":"2023-10-10T20:23:13.649603+0000","last_peered":"2023-10-10T20:23:13.649603+0000","last_clean":"2023-10-10T20:23:13.649603+0000","last_became_active":"2023-10-10T20:23:12.616369+0000","last_became_peered":"2023-10-10T20:23:12.616369+0000","last_unstale":"2023-10-10T20:23:13.649603+0000","last_undegraded":"2023-10-10T20:23:13.649603+0000","last_fullsized":"2023-10-10T20:23:13.649603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:06:47.117125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642430+0000","last_change":"2023-10-10T20:23:13.642376+0000","last_active":"2023-10-10T20:23:13.642430+0000","last_peered":"2023-10-10T20:23:13.642430+0000","last_clean":"2023-10-10T20:23:13.642430+0000","last_became_active":"2023-10-10T20:23:13.642025+0000","last_became_peered":"2023-10-10T20:23:13.642025+0000","last_unstale":"2023-10-10T20:23:13.642430+0000","last_undegraded":"2023-10-10T20:23:13.642430+0000","last_fullsized":"2023-10-10T20:23:13.642430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:31:22.561639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649265+0000","last_change":"2023-10-10T20:23:12.621158+0000","last_active":"2023-10-10T20:23:13.649265+0000","last_peered":"2023-10-10T20:23:13.649265+0000","last_clean":"2023-10-10T20:23:13.649265+0000","last_became_active":"2023-10-10T20:23:12.620832+0000","last_became_peered":"2023-10-10T20:23:12.620832+0000","last_unstale":"2023-10-10T20:23:13.649265+0000","last_undegraded":"2023-10-10T20:23:13.649265+0000","last_fullsized":"2023-10-10T20:23:13.649265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:36.641560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637972+0000","last_change":"2023-10-10T20:23:13.637564+0000","last_active":"2023-10-10T20:23:13.637972+0000","last_peered":"2023-10-10T20:23:13.637972+0000","last_clean":"2023-10-10T20:23:13.637972+0000","last_became_active":"2023-10-10T20:23:13.637323+0000","last_became_peered":"2023-10-10T20:23:13.637323+0000","last_unstale":"2023-10-10T20:23:13.637972+0000","last_undegraded":"2023-10-10T20:23:13.637972+0000","last_fullsized":"2023-10-10T20:23:13.637972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:34.172505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.627733+0000","last_change":"2023-10-10T20:23:12.620901+0000","last_active":"2023-10-10T20:23:13.627733+0000","last_peered":"2023-10-10T20:23:13.627733+0000","last_clean":"2023-10-10T20:23:13.627733+0000","last_became_active":"2023-10-10T20:23:12.620517+0000","last_became_peered":"2023-10-10T20:23:12.620517+0000","last_unstale":"2023-10-10T20:23:13.627733+0000","last_undegraded":"2023-10-10T20:23:13.627733+0000","last_fullsized":"2023-10-10T20:23:13.627733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:00.531731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638626+0000","last_change":"2023-10-10T20:23:13.638562+0000","last_active":"2023-10-10T20:23:13.638626+0000","last_peered":"2023-10-10T20:23:13.638626+0000","last_clean":"2023-10-10T20:23:13.638626+0000","last_became_active":"2023-10-10T20:23:13.638325+0000","last_became_peered":"2023-10-10T20:23:13.638325+0000","last_unstale":"2023-10-10T20:23:13.638626+0000","last_undegraded":"2023-10-10T20:23:13.638626+0000","last_fullsized":"2023-10-10T20:23:13.638626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:54.970810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630230+0000","last_change":"2023-10-10T20:23:12.615528+0000","last_active":"2023-10-10T20:23:13.630230+0000","last_peered":"2023-10-10T20:23:13.630230+0000","last_clean":"2023-10-10T20:23:13.630230+0000","last_became_active":"2023-10-10T20:23:12.615300+0000","last_became_peered":"2023-10-10T20:23:12.615300+0000","last_unstale":"2023-10-10T20:23:13.630230+0000","last_undegraded":"2023-10-10T20:23:13.630230+0000","last_fullsized":"2023-10-10T20:23:13.630230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:03:40.542969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637371+0000","last_change":"2023-10-10T20:23:13.637251+0000","last_active":"2023-10-10T20:23:13.637371+0000","last_peered":"2023-10-10T20:23:13.637371+0000","last_clean":"2023-10-10T20:23:13.637371+0000","last_became_active":"2023-10-10T20:23:13.636998+0000","last_became_peered":"2023-10-10T20:23:13.636998+0000","last_unstale":"2023-10-10T20:23:13.637371+0000","last_undegraded":"2023-10-10T20:23:13.637371+0000","last_fullsized":"2023-10-10T20:23:13.637371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:18.584835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632048+0000","last_change":"2023-10-10T20:23:12.616413+0000","last_active":"2023-10-10T20:23:13.632048+0000","last_peered":"2023-10-10T20:23:13.632048+0000","last_clean":"2023-10-10T20:23:13.632048+0000","last_became_active":"2023-10-10T20:23:12.615631+0000","last_became_peered":"2023-10-10T20:23:12.615631+0000","last_unstale":"2023-10-10T20:23:13.632048+0000","last_undegraded":"2023-10-10T20:23:13.632048+0000","last_fullsized":"2023-10-10T20:23:13.632048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:42.574789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644751+0000","last_change":"2023-10-10T20:23:13.644679+0000","last_active":"2023-10-10T20:23:13.644751+0000","last_peered":"2023-10-10T20:23:13.644751+0000","last_clean":"2023-10-10T20:23:13.644751+0000","last_became_active":"2023-10-10T20:23:13.644452+0000","last_became_peered":"2023-10-10T20:23:13.644452+0000","last_unstale":"2023-10-10T20:23:13.644751+0000","last_undegraded":"2023-10-10T20:23:13.644751+0000","last_fullsized":"2023-10-10T20:23:13.644751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:31:45.292120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635331+0000","last_change":"2023-10-10T20:23:12.618793+0000","last_active":"2023-10-10T20:23:13.635331+0000","last_peered":"2023-10-10T20:23:13.635331+0000","last_clean":"2023-10-10T20:23:13.635331+0000","last_became_active":"2023-10-10T20:23:12.618260+0000","last_became_peered":"2023-10-10T20:23:12.618260+0000","last_unstale":"2023-10-10T20:23:13.635331+0000","last_undegraded":"2023-10-10T20:23:13.635331+0000","last_fullsized":"2023-10-10T20:23:13.635331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:39.205941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644034+0000","last_change":"2023-10-10T20:23:13.643033+0000","last_active":"2023-10-10T20:23:13.644034+0000","last_peered":"2023-10-10T20:23:13.644034+0000","last_clean":"2023-10-10T20:23:13.644034+0000","last_became_active":"2023-10-10T20:23:13.640933+0000","last_became_peered":"2023-10-10T20:23:13.640933+0000","last_unstale":"2023-10-10T20:23:13.644034+0000","last_undegraded":"2023-10-10T20:23:13.644034+0000","last_fullsized":"2023-10-10T20:23:13.644034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:49:10.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"81'3","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649755+0000","last_change":"2023-10-10T20:23:12.620984+0000","last_active":"2023-10-10T20:23:13.649755+0000","last_peered":"2023-10-10T20:23:13.649755+0000","last_clean":"2023-10-10T20:23:13.649755+0000","last_became_active":"2023-10-10T20:23:12.620522+0000","last_became_peered":"2023-10-10T20:23:12.620522+0000","last_unstale":"2023-10-10T20:23:13.649755+0000","last_undegraded":"2023-10-10T20:23:13.649755+0000","last_fullsized":"2023-10-10T20:23:13.649755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:30.936037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.633431+0000","last_change":"2023-10-10T20:23:13.631640+0000","last_active":"2023-10-10T20:23:13.633431+0000","last_peered":"2023-10-10T20:23:13.633431+0000","last_clean":"2023-10-10T20:23:13.633431+0000","last_became_active":"2023-10-10T20:23:13.631362+0000","last_became_peered":"2023-10-10T20:23:13.631362+0000","last_unstale":"2023-10-10T20:23:13.633431+0000","last_undegraded":"2023-10-10T20:23:13.633431+0000","last_fullsized":"2023-10-10T20:23:13.633431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:25.332035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"81'6","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650847+0000","last_change":"2023-10-10T20:23:12.616388+0000","last_active":"2023-10-10T20:23:13.650847+0000","last_peered":"2023-10-10T20:23:13.650847+0000","last_clean":"2023-10-10T20:23:13.650847+0000","last_became_active":"2023-10-10T20:23:12.616163+0000","last_became_peered":"2023-10-10T20:23:12.616163+0000","last_unstale":"2023-10-10T20:23:13.650847+0000","last_undegraded":"2023-10-10T20:23:13.650847+0000","last_fullsized":"2023-10-10T20:23:13.650847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:19.435023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640572+0000","last_change":"2023-10-10T20:23:13.640444+0000","last_active":"2023-10-10T20:23:13.640572+0000","last_peered":"2023-10-10T20:23:13.640572+0000","last_clean":"2023-10-10T20:23:13.640572+0000","last_became_active":"2023-10-10T20:23:13.639877+0000","last_became_peered":"2023-10-10T20:23:13.639877+0000","last_unstale":"2023-10-10T20:23:13.640572+0000","last_undegraded":"2023-10-10T20:23:13.640572+0000","last_fullsized":"2023-10-10T20:23:13.640572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:35:59.753904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649244+0000","last_change":"2023-10-10T20:23:12.621577+0000","last_active":"2023-10-10T20:23:13.649244+0000","last_peered":"2023-10-10T20:23:13.649244+0000","last_clean":"2023-10-10T20:23:13.649244+0000","last_became_active":"2023-10-10T20:23:12.621356+0000","last_became_peered":"2023-10-10T20:23:12.621356+0000","last_unstale":"2023-10-10T20:23:13.649244+0000","last_undegraded":"2023-10-10T20:23:13.649244+0000","last_fullsized":"2023-10-10T20:23:13.649244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634825+0000","last_change":"2023-10-10T20:23:13.634723+0000","last_active":"2023-10-10T20:23:13.634825+0000","last_peered":"2023-10-10T20:23:13.634825+0000","last_clean":"2023-10-10T20:23:13.634825+0000","last_became_active":"2023-10-10T20:23:13.634506+0000","last_became_peered":"2023-10-10T20:23:13.634506+0000","last_unstale":"2023-10-10T20:23:13.634825+0000","last_undegraded":"2023-10-10T20:23:13.634825+0000","last_fullsized":"2023-10-10T20:23:13.634825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:39:57.019239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637360+0000","last_change":"2023-10-10T20:23:12.622168+0000","last_active":"2023-10-10T20:23:13.637360+0000","last_peered":"2023-10-10T20:23:13.637360+0000","last_clean":"2023-10-10T20:23:13.637360+0000","last_became_active":"2023-10-10T20:23:12.621333+0000","last_became_peered":"2023-10-10T20:23:12.621333+0000","last_unstale":"2023-10-10T20:23:13.637360+0000","last_undegraded":"2023-10-10T20:23:13.637360+0000","last_fullsized":"2023-10-10T20:23:13.637360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:42:38.974301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645950+0000","last_change":"2023-10-10T20:23:13.645900+0000","last_active":"2023-10-10T20:23:13.645950+0000","last_peered":"2023-10-10T20:23:13.645950+0000","last_clean":"2023-10-10T20:23:13.645950+0000","last_became_active":"2023-10-10T20:23:13.645423+0000","last_became_peered":"2023-10-10T20:23:13.645423+0000","last_unstale":"2023-10-10T20:23:13.645950+0000","last_undegraded":"2023-10-10T20:23:13.645950+0000","last_fullsized":"2023-10-10T20:23:13.645950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:47:32.514800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630791+0000","last_change":"2023-10-10T20:23:12.617061+0000","last_active":"2023-10-10T20:23:13.630791+0000","last_peered":"2023-10-10T20:23:13.630791+0000","last_clean":"2023-10-10T20:23:13.630791+0000","last_became_active":"2023-10-10T20:23:12.616463+0000","last_became_peered":"2023-10-10T20:23:12.616463+0000","last_unstale":"2023-10-10T20:23:13.630791+0000","last_undegraded":"2023-10-10T20:23:13.630791+0000","last_fullsized":"2023-10-10T20:23:13.630791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:21:36.302180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635875+0000","last_change":"2023-10-10T20:23:13.635799+0000","last_active":"2023-10-10T20:23:13.635875+0000","last_peered":"2023-10-10T20:23:13.635875+0000","last_clean":"2023-10-10T20:23:13.635875+0000","last_became_active":"2023-10-10T20:23:13.635434+0000","last_became_peered":"2023-10-10T20:23:13.635434+0000","last_unstale":"2023-10-10T20:23:13.635875+0000","last_undegraded":"2023-10-10T20:23:13.635875+0000","last_fullsized":"2023-10-10T20:23:13.635875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:45:20.441553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"81'8","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.652091+0000","last_change":"2023-10-10T20:23:12.622460+0000","last_active":"2023-10-10T20:23:13.652091+0000","last_peered":"2023-10-10T20:23:13.652091+0000","last_clean":"2023-10-10T20:23:13.652091+0000","last_became_active":"2023-10-10T20:23:12.622122+0000","last_became_peered":"2023-10-10T20:23:12.622122+0000","last_unstale":"2023-10-10T20:23:13.652091+0000","last_undegraded":"2023-10-10T20:23:13.652091+0000","last_fullsized":"2023-10-10T20:23:13.652091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:27.209212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632780+0000","last_change":"2023-10-10T20:23:13.632694+0000","last_active":"2023-10-10T20:23:13.632780+0000","last_peered":"2023-10-10T20:23:13.632780+0000","last_clean":"2023-10-10T20:23:13.632780+0000","last_became_active":"2023-10-10T20:23:13.632424+0000","last_became_peered":"2023-10-10T20:23:13.632424+0000","last_unstale":"2023-10-10T20:23:13.632780+0000","last_undegraded":"2023-10-10T20:23:13.632780+0000","last_fullsized":"2023-10-10T20:23:13.632780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650524+0000","last_change":"2023-10-10T20:23:12.618132+0000","last_active":"2023-10-10T20:23:13.650524+0000","last_peered":"2023-10-10T20:23:13.650524+0000","last_clean":"2023-10-10T20:23:13.650524+0000","last_became_active":"2023-10-10T20:23:12.617831+0000","last_became_peered":"2023-10-10T20:23:12.617831+0000","last_unstale":"2023-10-10T20:23:13.650524+0000","last_undegraded":"2023-10-10T20:23:13.650524+0000","last_fullsized":"2023-10-10T20:23:13.650524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:42.574789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.640839+0000","last_change":"2023-10-10T20:23:13.640723+0000","last_active":"2023-10-10T20:23:13.640839+0000","last_peered":"2023-10-10T20:23:13.640839+0000","last_clean":"2023-10-10T20:23:13.640839+0000","last_became_active":"2023-10-10T20:23:13.640001+0000","last_became_peered":"2023-10-10T20:23:13.640001+0000","last_unstale":"2023-10-10T20:23:13.640839+0000","last_undegraded":"2023-10-10T20:23:13.640839+0000","last_fullsized":"2023-10-10T20:23:13.640839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:45.958778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651368+0000","last_change":"2023-10-10T20:23:12.619338+0000","last_active":"2023-10-10T20:23:13.651368+0000","last_peered":"2023-10-10T20:23:13.651368+0000","last_clean":"2023-10-10T20:23:13.651368+0000","last_became_active":"2023-10-10T20:23:12.618979+0000","last_became_peered":"2023-10-10T20:23:12.618979+0000","last_unstale":"2023-10-10T20:23:13.651368+0000","last_undegraded":"2023-10-10T20:23:13.651368+0000","last_fullsized":"2023-10-10T20:23:13.651368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:37:36.641560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638010+0000","last_change":"2023-10-10T20:23:13.637953+0000","last_active":"2023-10-10T20:23:13.638010+0000","last_peered":"2023-10-10T20:23:13.638010+0000","last_clean":"2023-10-10T20:23:13.638010+0000","last_became_active":"2023-10-10T20:23:13.637441+0000","last_became_peered":"2023-10-10T20:23:13.637441+0000","last_unstale":"2023-10-10T20:23:13.638010+0000","last_undegraded":"2023-10-10T20:23:13.638010+0000","last_fullsized":"2023-10-10T20:23:13.638010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:52.682236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.651387+0000","last_change":"2023-10-10T20:23:12.623238+0000","last_active":"2023-10-10T20:23:13.651387+0000","last_peered":"2023-10-10T20:23:13.651387+0000","last_clean":"2023-10-10T20:23:13.651387+0000","last_became_active":"2023-10-10T20:23:12.622992+0000","last_became_peered":"2023-10-10T20:23:12.622992+0000","last_unstale":"2023-10-10T20:23:13.651387+0000","last_undegraded":"2023-10-10T20:23:13.651387+0000","last_fullsized":"2023-10-10T20:23:13.651387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:33.165975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637989+0000","last_change":"2023-10-10T20:23:13.637902+0000","last_active":"2023-10-10T20:23:13.637989+0000","last_peered":"2023-10-10T20:23:13.637989+0000","last_clean":"2023-10-10T20:23:13.637989+0000","last_became_active":"2023-10-10T20:23:13.637674+0000","last_became_peered":"2023-10-10T20:23:13.637674+0000","last_unstale":"2023-10-10T20:23:13.637989+0000","last_undegraded":"2023-10-10T20:23:13.637989+0000","last_fullsized":"2023-10-10T20:23:13.637989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:40.212470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.634073+0000","last_change":"2023-10-10T20:23:12.624278+0000","last_active":"2023-10-10T20:23:13.634073+0000","last_peered":"2023-10-10T20:23:13.634073+0000","last_clean":"2023-10-10T20:23:13.634073+0000","last_became_active":"2023-10-10T20:23:12.624050+0000","last_became_peered":"2023-10-10T20:23:12.624050+0000","last_unstale":"2023-10-10T20:23:13.634073+0000","last_undegraded":"2023-10-10T20:23:13.634073+0000","last_fullsized":"2023-10-10T20:23:13.634073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:20:05.916075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636489+0000","last_change":"2023-10-10T20:23:13.636165+0000","last_active":"2023-10-10T20:23:13.636489+0000","last_peered":"2023-10-10T20:23:13.636489+0000","last_clean":"2023-10-10T20:23:13.636489+0000","last_became_active":"2023-10-10T20:23:13.635760+0000","last_became_peered":"2023-10-10T20:23:13.635760+0000","last_unstale":"2023-10-10T20:23:13.636489+0000","last_undegraded":"2023-10-10T20:23:13.636489+0000","last_fullsized":"2023-10-10T20:23:13.636489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:04:38.578668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636449+0000","last_change":"2023-10-10T20:23:12.620878+0000","last_active":"2023-10-10T20:23:13.636449+0000","last_peered":"2023-10-10T20:23:13.636449+0000","last_clean":"2023-10-10T20:23:13.636449+0000","last_became_active":"2023-10-10T20:23:12.620629+0000","last_became_peered":"2023-10-10T20:23:12.620629+0000","last_unstale":"2023-10-10T20:23:13.636449+0000","last_undegraded":"2023-10-10T20:23:13.636449+0000","last_fullsized":"2023-10-10T20:23:13.636449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:51.675707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.639286+0000","last_change":"2023-10-10T20:23:13.639235+0000","last_active":"2023-10-10T20:23:13.639286+0000","last_peered":"2023-10-10T20:23:13.639286+0000","last_clean":"2023-10-10T20:23:13.639286+0000","last_became_active":"2023-10-10T20:23:13.639052+0000","last_became_peered":"2023-10-10T20:23:13.639052+0000","last_unstale":"2023-10-10T20:23:13.639286+0000","last_undegraded":"2023-10-10T20:23:13.639286+0000","last_fullsized":"2023-10-10T20:23:13.639286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:32:17.000182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632659+0000","last_change":"2023-10-10T20:23:12.615450+0000","last_active":"2023-10-10T20:23:13.632659+0000","last_peered":"2023-10-10T20:23:13.632659+0000","last_clean":"2023-10-10T20:23:13.632659+0000","last_became_active":"2023-10-10T20:23:12.615222+0000","last_became_peered":"2023-10-10T20:23:12.615222+0000","last_unstale":"2023-10-10T20:23:13.632659+0000","last_undegraded":"2023-10-10T20:23:13.632659+0000","last_fullsized":"2023-10-10T20:23:13.632659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:43:39.728186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638584+0000","last_change":"2023-10-10T20:23:13.638543+0000","last_active":"2023-10-10T20:23:13.638584+0000","last_peered":"2023-10-10T20:23:13.638584+0000","last_clean":"2023-10-10T20:23:13.638584+0000","last_became_active":"2023-10-10T20:23:13.638380+0000","last_became_peered":"2023-10-10T20:23:13.638380+0000","last_unstale":"2023-10-10T20:23:13.638584+0000","last_undegraded":"2023-10-10T20:23:13.638584+0000","last_fullsized":"2023-10-10T20:23:13.638584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:14.272395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.629398+0000","last_change":"2023-10-10T20:23:12.617234+0000","last_active":"2023-10-10T20:23:13.629398+0000","last_peered":"2023-10-10T20:23:13.629398+0000","last_clean":"2023-10-10T20:23:13.629398+0000","last_became_active":"2023-10-10T20:23:12.616114+0000","last_became_peered":"2023-10-10T20:23:12.616114+0000","last_unstale":"2023-10-10T20:23:13.629398+0000","last_undegraded":"2023-10-10T20:23:13.629398+0000","last_fullsized":"2023-10-10T20:23:13.629398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:54:08.829943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636170+0000","last_change":"2023-10-10T20:23:13.635745+0000","last_active":"2023-10-10T20:23:13.636170+0000","last_peered":"2023-10-10T20:23:13.636170+0000","last_clean":"2023-10-10T20:23:13.636170+0000","last_became_active":"2023-10-10T20:23:13.635463+0000","last_became_peered":"2023-10-10T20:23:13.635463+0000","last_unstale":"2023-10-10T20:23:13.636170+0000","last_undegraded":"2023-10-10T20:23:13.636170+0000","last_fullsized":"2023-10-10T20:23:13.636170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:26.034146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.650690+0000","last_change":"2023-10-10T20:23:12.618029+0000","last_active":"2023-10-10T20:23:13.650690+0000","last_peered":"2023-10-10T20:23:13.650690+0000","last_clean":"2023-10-10T20:23:13.650690+0000","last_became_active":"2023-10-10T20:23:12.617619+0000","last_became_peered":"2023-10-10T20:23:12.617619+0000","last_unstale":"2023-10-10T20:23:13.650690+0000","last_undegraded":"2023-10-10T20:23:13.650690+0000","last_fullsized":"2023-10-10T20:23:13.650690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636638+0000","last_change":"2023-10-10T20:23:13.636446+0000","last_active":"2023-10-10T20:23:13.636638+0000","last_peered":"2023-10-10T20:23:13.636638+0000","last_clean":"2023-10-10T20:23:13.636638+0000","last_became_active":"2023-10-10T20:23:13.635891+0000","last_became_peered":"2023-10-10T20:23:13.635891+0000","last_unstale":"2023-10-10T20:23:13.636638+0000","last_undegraded":"2023-10-10T20:23:13.636638+0000","last_fullsized":"2023-10-10T20:23:13.636638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:59:39.740113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630127+0000","last_change":"2023-10-10T20:23:12.615104+0000","last_active":"2023-10-10T20:23:13.630127+0000","last_peered":"2023-10-10T20:23:13.630127+0000","last_clean":"2023-10-10T20:23:13.630127+0000","last_became_active":"2023-10-10T20:23:12.614865+0000","last_became_peered":"2023-10-10T20:23:12.614865+0000","last_unstale":"2023-10-10T20:23:13.630127+0000","last_undegraded":"2023-10-10T20:23:13.630127+0000","last_fullsized":"2023-10-10T20:23:13.630127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:43:24.949892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644780+0000","last_change":"2023-10-10T20:23:13.644714+0000","last_active":"2023-10-10T20:23:13.644780+0000","last_peered":"2023-10-10T20:23:13.644780+0000","last_clean":"2023-10-10T20:23:13.644780+0000","last_became_active":"2023-10-10T20:23:13.644507+0000","last_became_peered":"2023-10-10T20:23:13.644507+0000","last_unstale":"2023-10-10T20:23:13.644780+0000","last_undegraded":"2023-10-10T20:23:13.644780+0000","last_fullsized":"2023-10-10T20:23:13.644780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:06:48.123654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630897+0000","last_change":"2023-10-10T20:23:12.615708+0000","last_active":"2023-10-10T20:23:13.630897+0000","last_peered":"2023-10-10T20:23:13.630897+0000","last_clean":"2023-10-10T20:23:13.630897+0000","last_became_active":"2023-10-10T20:23:12.615407+0000","last_became_peered":"2023-10-10T20:23:12.615407+0000","last_unstale":"2023-10-10T20:23:13.630897+0000","last_undegraded":"2023-10-10T20:23:13.630897+0000","last_fullsized":"2023-10-10T20:23:13.630897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:58.489753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.642154+0000","last_change":"2023-10-10T20:23:13.642030+0000","last_active":"2023-10-10T20:23:13.642154+0000","last_peered":"2023-10-10T20:23:13.642154+0000","last_clean":"2023-10-10T20:23:13.642154+0000","last_became_active":"2023-10-10T20:23:13.641721+0000","last_became_peered":"2023-10-10T20:23:13.641721+0000","last_unstale":"2023-10-10T20:23:13.642154+0000","last_undegraded":"2023-10-10T20:23:13.642154+0000","last_fullsized":"2023-10-10T20:23:13.642154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:20.011991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.630057+0000","last_change":"2023-10-10T20:23:12.620430+0000","last_active":"2023-10-10T20:23:13.630057+0000","last_peered":"2023-10-10T20:23:13.630057+0000","last_clean":"2023-10-10T20:23:13.630057+0000","last_became_active":"2023-10-10T20:23:12.620200+0000","last_became_peered":"2023-10-10T20:23:12.620200+0000","last_unstale":"2023-10-10T20:23:13.630057+0000","last_undegraded":"2023-10-10T20:23:13.630057+0000","last_fullsized":"2023-10-10T20:23:13.630057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:28:38.509700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.635859+0000","last_change":"2023-10-10T20:23:13.635653+0000","last_active":"2023-10-10T20:23:13.635859+0000","last_peered":"2023-10-10T20:23:13.635859+0000","last_clean":"2023-10-10T20:23:13.635859+0000","last_became_active":"2023-10-10T20:23:13.635170+0000","last_became_peered":"2023-10-10T20:23:13.635170+0000","last_unstale":"2023-10-10T20:23:13.635859+0000","last_undegraded":"2023-10-10T20:23:13.635859+0000","last_fullsized":"2023-10-10T20:23:13.635859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:01.538260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.629529+0000","last_change":"2023-10-10T20:23:12.616491+0000","last_active":"2023-10-10T20:23:13.629529+0000","last_peered":"2023-10-10T20:23:13.629529+0000","last_clean":"2023-10-10T20:23:13.629529+0000","last_became_active":"2023-10-10T20:23:12.616271+0000","last_became_peered":"2023-10-10T20:23:12.616271+0000","last_unstale":"2023-10-10T20:23:13.629529+0000","last_undegraded":"2023-10-10T20:23:13.629529+0000","last_fullsized":"2023-10-10T20:23:13.629529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:52:28.130767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.644084+0000","last_change":"2023-10-10T20:23:13.643582+0000","last_active":"2023-10-10T20:23:13.644084+0000","last_peered":"2023-10-10T20:23:13.644084+0000","last_clean":"2023-10-10T20:23:13.644084+0000","last_became_active":"2023-10-10T20:23:13.643157+0000","last_became_peered":"2023-10-10T20:23:13.643157+0000","last_unstale":"2023-10-10T20:23:13.644084+0000","last_undegraded":"2023-10-10T20:23:13.644084+0000","last_fullsized":"2023-10-10T20:23:13.644084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:17:18.514327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.627726+0000","last_change":"2023-10-10T20:23:12.621840+0000","last_active":"2023-10-10T20:23:13.627726+0000","last_peered":"2023-10-10T20:23:13.627726+0000","last_clean":"2023-10-10T20:23:13.627726+0000","last_became_active":"2023-10-10T20:23:12.621620+0000","last_became_peered":"2023-10-10T20:23:12.621620+0000","last_unstale":"2023-10-10T20:23:13.627726+0000","last_undegraded":"2023-10-10T20:23:13.627726+0000","last_fullsized":"2023-10-10T20:23:13.627726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:00:24.325506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.632552+0000","last_change":"2023-10-10T20:23:13.632445+0000","last_active":"2023-10-10T20:23:13.632552+0000","last_peered":"2023-10-10T20:23:13.632552+0000","last_clean":"2023-10-10T20:23:13.632552+0000","last_became_active":"2023-10-10T20:23:13.632215+0000","last_became_peered":"2023-10-10T20:23:13.632215+0000","last_unstale":"2023-10-10T20:23:13.632552+0000","last_undegraded":"2023-10-10T20:23:13.632552+0000","last_fullsized":"2023-10-10T20:23:13.632552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:43.581318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.631439+0000","last_change":"2023-10-10T20:23:12.622318+0000","last_active":"2023-10-10T20:23:13.631439+0000","last_peered":"2023-10-10T20:23:13.631439+0000","last_clean":"2023-10-10T20:23:13.631439+0000","last_became_active":"2023-10-10T20:23:12.621849+0000","last_became_peered":"2023-10-10T20:23:12.621849+0000","last_unstale":"2023-10-10T20:23:13.631439+0000","last_undegraded":"2023-10-10T20:23:13.631439+0000","last_fullsized":"2023-10-10T20:23:13.631439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:51.675707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.645076+0000","last_change":"2023-10-10T20:23:13.644714+0000","last_active":"2023-10-10T20:23:13.645076+0000","last_peered":"2023-10-10T20:23:13.645076+0000","last_clean":"2023-10-10T20:23:13.645076+0000","last_became_active":"2023-10-10T20:23:13.644128+0000","last_became_peered":"2023-10-10T20:23:13.644128+0000","last_unstale":"2023-10-10T20:23:13.645076+0000","last_undegraded":"2023-10-10T20:23:13.645076+0000","last_fullsized":"2023-10-10T20:23:13.645076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:12:42.699985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.636582+0000","last_change":"2023-10-10T20:23:12.623803+0000","last_active":"2023-10-10T20:23:13.636582+0000","last_peered":"2023-10-10T20:23:13.636582+0000","last_clean":"2023-10-10T20:23:13.636582+0000","last_became_active":"2023-10-10T20:23:12.623545+0000","last_became_peered":"2023-10-10T20:23:12.623545+0000","last_unstale":"2023-10-10T20:23:13.636582+0000","last_undegraded":"2023-10-10T20:23:13.636582+0000","last_fullsized":"2023-10-10T20:23:13.636582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:21:59.225495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.638096+0000","last_change":"2023-10-10T20:23:13.637951+0000","last_active":"2023-10-10T20:23:13.638096+0000","last_peered":"2023-10-10T20:23:13.638096+0000","last_clean":"2023-10-10T20:23:13.638096+0000","last_became_active":"2023-10-10T20:23:13.637744+0000","last_became_peered":"2023-10-10T20:23:13.637744+0000","last_unstale":"2023-10-10T20:23:13.638096+0000","last_undegraded":"2023-10-10T20:23:13.638096+0000","last_fullsized":"2023-10-10T20:23:13.638096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:12.606888+0000","last_clean_scrub_stamp":"2023-10-10T20:23:12.606888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:26:50.090841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"81'2","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.649503+0000","last_change":"2023-10-10T20:23:12.619360+0000","last_active":"2023-10-10T20:23:13.649503+0000","last_peered":"2023-10-10T20:23:13.649503+0000","last_clean":"2023-10-10T20:23:13.649503+0000","last_became_active":"2023-10-10T20:23:12.619053+0000","last_became_peered":"2023-10-10T20:23:12.619053+0000","last_unstale":"2023-10-10T20:23:13.649503+0000","last_undegraded":"2023-10-10T20:23:13.649503+0000","last_fullsized":"2023-10-10T20:23:13.649503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:23:11.600359+0000","last_clean_scrub_stamp":"2023-10-10T20:23:11.600359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:30:30.732499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":219,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-10-10T20:23:13.637460+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:23:13.637460+0000","last_peered":"2023-10-10T20:23:13.637460+0000","last_clean":"2023-10-10T20:23:13.637460+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:23:13.637460+0000","last_undegraded":"2023-10-10T20:23:13.637460+0000","last_fullsized":"2023-10-10T20:23:13.637460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":999,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198976294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.48399999999999999,"15min":0.46899999999999997},"min":{"1min":0.35299999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66900000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.373,"5min":0.316,"15min":0.316},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48599999999999999},"min":{"1min":0.36299999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73799999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68400000000000005,"5min":0.754,"15min":0.754},"last":0.52700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39100000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46400000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.377},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73899999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567158,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.56499999999999995,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.4219999999999997,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.435}]},{"osd":1,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.56200000000000006,"15min":0.52800000000000002},"min":{"1min":0.374,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.308,"15min":0.308},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0430000000000001,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.298,"5min":0.68500000000000005,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":15.862,"5min":15.862,"15min":15.862},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.42999999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.29099999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.29299999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.39800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194887,"statfs":{"total":200000000,"available":199565126,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.50700000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.37,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54800000000000004,"5min":1.647,"15min":1.647},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56899999999999995,"5min":1.524,"15min":1.524},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.58799999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.25900000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42499999999999999,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61199999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.311},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.317,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63,"5min":1.673,"15min":1.673},"last":0.33600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.633,"15min":1.633},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":1.506,"15min":1.506},"last":0.36199999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":1.696,"15min":1.696},"last":0.374},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.627,"15min":0.68300000000000005},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":1.585,"15min":1.585},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80500000000000005,"5min":1.478,"15min":1.478},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.722,"15min":1.722},"last":0.46000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":998,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194314,"statfs":{"total":200000000,"available":198977893,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.42499999999999999,"15min":0.47299999999999998},"min":{"1min":0.21099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55100000000000005,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.223},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.52200000000000002,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.308}]},{"osd":2,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.45800000000000002,"15min":0.502},"min":{"1min":0.23599999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.25700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.46999999999999997,"5min":1.256,"15min":1.256},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.375,"15min":0.38900000000000001},"min":{"1min":0.25700000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41999999999999998,"15min":0.436},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.505,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.28299999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":1.204,"15min":1.204},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.875,"15min":1.0149999999999999},"min":{"1min":0.40600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81399999999999995,"5min":1.347,"15min":1.347},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70299999999999996,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.503},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.39600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78100000000000003,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.624}]},{"osd":7,"last update":"Tue Oct 10 20:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76100000000000001,"5min":1.173,"15min":1.173},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.745}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":423,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199566725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.47299999999999998,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.497,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.51900000000000002,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48699999999999999,"5min":11.815,"15min":11.815},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.58399999999999996,"15min":0.46899999999999997},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58399999999999996,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.316}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74299999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72399999999999998,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":6.399,"15min":6.399},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.39000000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.66900000000000004,"15min":0.60299999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.60899999999999999},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66500000000000004,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76900000000000002,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.52000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40200000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48699999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.505,"5min":0.751,"15min":0.751},"last":0.371}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53100000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.379},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53700000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.496,"15min":0.47499999999999998},"min":{"1min":0.317,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.437}]},{"osd":1,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.44900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.46600000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.49399999999999999},"min":{"1min":0.48499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.51600000000000001},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.501,"15min":0.48099999999999998},"min":{"1min":0.436,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.54000000000000004,"15min":0.51700000000000002},"min":{"1min":0.54000000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.499}]},{"osd":5,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.35899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.39000000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.39300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194889,"statfs":{"total":200000000,"available":199567160,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.505,"15min":0.497},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68799999999999994,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68799999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.875,"15min":0.875},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627}]}]}],"pool_statfs":[]}} 2023-10-10T20:23:18.067 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:23:18.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:23:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.217+0000 7f675cd7b700 1 Processor -- start 2023-10-10T20:23:18.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.217+0000 7f675cd7b700 1 -- start start 2023-10-10T20:23:18.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f675cd7b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f67580d05a0 0x7f6758164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f675cd7b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580b0790 con 0x7f675815a290 2023-10-10T20:23:18.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f675cd7b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580ce1f0 con 0x7f6758159ec0 2023-10-10T20:23:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f675cd7b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b0b20 con 0x7f67580d05a0 2023-10-10T20:23:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f674dd9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f675815a290 0x7f675806a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50230/0 (socket says 172.21.15.148:50230) 2023-10-10T20:23:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f674dd9b700 1 -- 172.21.15.148:0/3339493984 learned_addr learned my addr 172.21.15.148:0/3339493984 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:18.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6756d9d700 1 --2- 172.21.15.148:0/3339493984 >> v2:172.21.15.161:3300/0 conn(0x7f67580d05a0 0x7f6758164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967444072 0 0) 0x7f67580b0790 con 0x7f675815a290 2023-10-10T20:23:18.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6744003570 con 0x7f675815a290 2023-10-10T20:23:18.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944150547 0 0) 0x7f6744003570 con 0x7f675815a290 2023-10-10T20:23:18.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 >> v2:172.21.15.161:3300/0 conn(0x7f67580d05a0 msgr2=0x7f6758164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 --2- 172.21.15.148:0/3339493984 >> v2:172.21.15.161:3300/0 conn(0x7f67580d05a0 0x7f6758164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 >> v1:172.21.15.161:6789/0 conn(0x7f6758159ec0 legacy=0x7f67580af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580b04b0 con 0x7f675815a290 2023-10-10T20:23:18.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f6740001470 con 0x7f675815a290 2023-10-10T20:23:18.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6756d9d700 1 --2- 172.21.15.148:0/3339493984 >> v2:172.21.15.161:3300/0 conn(0x7f67580d05a0 0x7f6758164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:18.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 >> v1:172.21.15.148:6789/0 conn(0x7f675815a290 legacy=0x7f675806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744003a60 0x7f6744003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f6744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67440051c0 0x7f674400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.218+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b0b20 con 0x7f6744004810 2023-10-10T20:23:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674400c0e0 con 0x7f6744003a60 2023-10-10T20:23:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674400c2a0 con 0x7f67440051c0 2023-10-10T20:23:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f6744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f675659c700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744003a60 0x7f6744003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6756d9d700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67440051c0 0x7f674400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 -- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67440051c0 msgr2=0x7f674400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67440051c0 0x7f674400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 -- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744003a60 msgr2=0x7f6744003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6744003a60 0x7f6744003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 -- 172.21.15.148:0/3339493984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6744003570 con 0x7f6744004810 2023-10-10T20:23:18.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6756d9d700 1 -- 172.21.15.148:0/3339493984 reap_dead start 2023-10-10T20:23:18.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f674dd9b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f6744004c80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f674000c2d0 tx=0x7f6740001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a728f38449c804 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000cbc0 con 0x7f6744004810 2023-10-10T20:23:18.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674000cd20 con 0x7f6744004810 2023-10-10T20:23:18.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.219+0000 7f6755d9b700 1 -- 172.21.15.148:0/3339493984 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740005b70 con 0x7f6744004810 2023-10-10T20:23:18.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 -- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 msgr2=0x7f6744004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f6744004c80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f674000c2d0 tx=0x7f6740001df0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 -- 172.21.15.148:0/3339493984 shutdown_connections 2023-10-10T20:23:18.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 --2- 172.21.15.148:0/3339493984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f6744004c80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 -- 172.21.15.148:0/3339493984 >> 172.21.15.148:0/3339493984 conn(0x7f67580d1880 msgr2=0x7f67580cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:18.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 -- 172.21.15.148:0/3339493984 shutdown_connections 2023-10-10T20:23:18.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.220+0000 7f675cd7b700 1 -- 172.21.15.148:0/3339493984 wait complete. 2023-10-10T20:23:18.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.221+0000 7f675cd7b700 1 Processor -- start 2023-10-10T20:23:18.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.221+0000 7f675cd7b700 1 -- start start 2023-10-10T20:23:18.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67580b4b20 0x7f6758154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f675815d150 0x7f675815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b5c30 con 0x7f6744004810 2023-10-10T20:23:18.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758160b50 con 0x7f67580b4b20 2023-10-10T20:23:18.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b5020 con 0x7f675815d150 2023-10-10T20:23:18.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54790/0 (socket says 172.21.15.148:54790) 2023-10-10T20:23:18.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 -- 172.21.15.148:0/1511511868 learned_addr learned my addr 172.21.15.148:0/1511511868 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:18.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f674dd9b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67580b4b20 0x7f6758154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f6756d9d700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f675815d150 0x7f675815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 -- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f675815d150 msgr2=0x7f675815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f675815d150 0x7f675815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 -- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67580b4b20 msgr2=0x7f6758154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67580b4b20 0x7f6758154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 -- 172.21.15.148:0/1511511868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580b5300 con 0x7f6744004810 2023-10-10T20:23:18.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675659c700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6748002b80 tx=0x7f6748007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800f850 con 0x7f6744004810 2023-10-10T20:23:18.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674800f9d0 con 0x7f6744004810 2023-10-10T20:23:18.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67580b55e0 con 0x7f6744004810 2023-10-10T20:23:18.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800fd90 con 0x7f6744004810 2023-10-10T20:23:18.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.222+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6758155400 con 0x7f6744004810 2023-10-10T20:23:18.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.223+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675804c3c0 con 0x7f6744004810 2023-10-10T20:23:18.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.224+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6748013600 con 0x7f6744004810 2023-10-10T20:23:18.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.224+0000 7f674effd700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.224+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6748084ac0 con 0x7f6744004810 2023-10-10T20:23:18.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.224+0000 7f674dd9b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.225+0000 7f674dd9b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 0x7f67380693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6740008fd0 tx=0x7f6740015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.228+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6748051d30 con 0x7f6744004810 2023-10-10T20:23:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.485+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f67580ceab0 con 0x7f6744004810 2023-10-10T20:23:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.489+0000 7f674effd700 1 -- 172.21.15.148:0/1511511868 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f67480170e0 con 0x7f6744004810 2023-10-10T20:23:18.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 msgr2=0x7f67380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 0x7f67380693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6740008fd0 tx=0x7f6740015000 comp rx=0 tx=0).stop 2023-10-10T20:23:18.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 msgr2=0x7f67580b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6748002b80 tx=0x7f6748007d10 comp rx=0 tx=0).stop 2023-10-10T20:23:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 shutdown_connections 2023-10-10T20:23:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6738066f30 0x7f67380693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f675815d150 0x7f675815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6744004810 0x7f67580b60c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 --2- 172.21.15.148:0/1511511868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67580b4b20 0x7f6758154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 >> 172.21.15.148:0/1511511868 conn(0x7f67580d1880 msgr2=0x7f67580c7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:18.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 shutdown_connections 2023-10-10T20:23:18.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.491+0000 7f675cd7b700 1 -- 172.21.15.148:0/1511511868 wait complete. 2023-10-10T20:23:18.496 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:23:18.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:18.602 INFO:tasks.ceph.mds.c.smithi148.stderr:2023-10-10T20:23:18.600+0000 7fb22e3bb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:23:18.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.617+0000 7f7e1ffff700 1 Processor -- start 2023-10-10T20:23:18.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.617+0000 7f7e1ffff700 1 -- start start 2023-10-10T20:23:18.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e2015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1ffff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e200cb7c0 con 0x7f7e200cdab0 2023-10-10T20:23:18.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1ffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2015e290 con 0x7f7e200cd6e0 2023-10-10T20:23:18.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015e850 con 0x7f7e2015cbd0 2023-10-10T20:23:18.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e167fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7e200cdab0 0x7f7e2015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50244/0 (socket says 172.21.15.148:50244) 2023-10-10T20:23:18.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e167fc700 1 -- 172.21.15.148:0/356433313 learned_addr learned my addr 172.21.15.148:0/356433313 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:18.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1f7fe700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e2015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869172889 0 0) 0x7f7e200cb7c0 con 0x7f7e200cdab0 2023-10-10T20:23:18.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.618+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e0c003570 con 0x7f7e200cdab0 2023-10-10T20:23:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246619030 0 0) 0x7f7e0c003570 con 0x7f7e200cdab0 2023-10-10T20:23:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 msgr2=0x7f7e2015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e2015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 >> v1:172.21.15.161:6789/0 conn(0x7f7e200cd6e0 legacy=0x7f7e200cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2015e570 con 0x7f7e200cdab0 2023-10-10T20:23:18.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7e08001470 con 0x7f7e200cdab0 2023-10-10T20:23:18.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 >> v1:172.21.15.148:6789/0 conn(0x7f7e200cdab0 legacy=0x7f7e2015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e0c003a90 0x7f7e0c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c004d50 0x7f7e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015e850 con 0x7f7e0c003a90 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c00bb70 con 0x7f7e0c004d50 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c00bd30 con 0x7f7e2015cbd0 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e0c003a90 0x7f7e0c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1f7fe700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c004d50 0x7f7e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e167fc700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 -- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 msgr2=0x7f7e0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2015cbd0 0x7f7e0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 -- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c004d50 msgr2=0x7f7e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:18.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c004d50 0x7f7e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 -- 172.21.15.148:0/356433313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c003570 con 0x7f7e0c003a90 2023-10-10T20:23:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1effd700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e0c003a90 0x7f7e0c003ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7e100018b0 tx=0x7f7e10007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40159b8594d5cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10012420 con 0x7f7e0c003a90 2023-10-10T20:23:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10012580 con 0x7f7e0c003a90 2023-10-10T20:23:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.619+0000 7f7e1e7fc700 1 -- 172.21.15.148:0/356433313 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10012920 con 0x7f7e0c003a90 2023-10-10T20:23:18.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 -- 172.21.15.148:0/356433313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e0c003a90 msgr2=0x7f7e0c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/356433313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e0c003a90 0x7f7e0c003ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7e100018b0 tx=0x7f7e10007d30 comp rx=0 tx=0).stop 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1f7fe700 1 -- 172.21.15.148:0/356433313 reap_dead start 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 -- 172.21.15.148:0/356433313 shutdown_connections 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 -- 172.21.15.148:0/356433313 >> 172.21.15.148:0/356433313 conn(0x7f7e20067660 msgr2=0x7f7e20067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 -- 172.21.15.148:0/356433313 shutdown_connections 2023-10-10T20:23:18.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.620+0000 7f7e1ffff700 1 -- 172.21.15.148:0/356433313 wait complete. 2023-10-10T20:23:18.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.621+0000 7f7e1ffff700 1 Processor -- start 2023-10-10T20:23:18.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.621+0000 7f7e1ffff700 1 -- start start 2023-10-10T20:23:18.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 0x7f7e20156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 0x7f7e20159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200cc360 con 0x7f7e2015a460 2023-10-10T20:23:18.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20159360 con 0x7f7e0c003a90 2023-10-10T20:23:18.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015aee0 con 0x7f7e201597f0 2023-10-10T20:23:18.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54808/0 (socket says 172.21.15.148:54808) 2023-10-10T20:23:18.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 -- 172.21.15.148:0/1046981897 learned_addr learned my addr 172.21.15.148:0/1046981897 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:18.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1effd700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 0x7f7e20156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e167fc700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 0x7f7e20159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 -- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 msgr2=0x7f7e20159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 0x7f7e20159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 -- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 msgr2=0x7f7e20156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 0x7f7e20156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 -- 172.21.15.148:0/1046981897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20158a30 con 0x7f7e2015a460 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e167fc700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 0x7f7e20159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1effd700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 0x7f7e20156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.622+0000 7f7e1f7fe700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7e200c7cd0 tx=0x7f7e1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800e070 con 0x7f7e2015a460 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18011ca0 con 0x7f7e2015a460 2023-10-10T20:23:18.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18016470 con 0x7f7e2015a460 2023-10-10T20:23:18.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e20158d10 con 0x7f7e2015a460 2023-10-10T20:23:18.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200b0140 con 0x7f7e2015a460 2023-10-10T20:23:18.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.623+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e200cbb50 con 0x7f7e2015a460 2023-10-10T20:23:18.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.624+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e180168a0 con 0x7f7e2015a460 2023-10-10T20:23:18.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.624+0000 7f7e177fe700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 0x7f7e00069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:18.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.624+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e18085210 con 0x7f7e2015a460 2023-10-10T20:23:18.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.624+0000 7f7e1effd700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 0x7f7e00069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:18.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.625+0000 7f7e1effd700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 0x7f7e00069460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e100018b0 tx=0x7f7e10015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:18.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.629+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e180523e0 con 0x7f7e2015a460 2023-10-10T20:23:18.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.856+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e201594f0 con 0x7f7e2015a460 2023-10-10T20:23:18.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.857+0000 7f7e177fe700 1 -- 172.21.15.148:0/1046981897 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f7e18055a00 con 0x7f7e2015a460 2023-10-10T20:23:18.859 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:18.859 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 msgr2=0x7f7e00069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 0x7f7e00069460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e100018b0 tx=0x7f7e10015000 comp rx=0 tx=0).stop 2023-10-10T20:23:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 msgr2=0x7f7e2015a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7e200c7cd0 tx=0x7f7e1800b040 comp rx=0 tx=0).stop 2023-10-10T20:23:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 shutdown_connections 2023-10-10T20:23:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7e00066fb0 0x7f7e00069460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e2015a460 0x7f7e2015a8d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e0c003a90 0x7f7e20156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 --2- 172.21.15.148:0/1046981897 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e201597f0 0x7f7e20159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 >> 172.21.15.148:0/1046981897 conn(0x7f7e20067660 msgr2=0x7f7e200c6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 shutdown_connections 2023-10-10T20:23:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:18.859+0000 7f7e1ffff700 1 -- 172.21.15.148:0/1046981897 wait complete. 2023-10-10T20:23:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:19.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:23:12.996722+0000', 'modified': '2023-10-10T20:23:16.652575+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6060}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6060': {'gid': 6060, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/934736302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 934736302}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 934736302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:23:19.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:23:19.277 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-10T20:23:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.392+0000 7f9d2e67d700 1 Processor -- start 2023-10-10T20:23:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.392+0000 7f9d2e67d700 1 -- start start 2023-10-10T20:23:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d28157870 0x7f9d2815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 0x7f9d2815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d2815d580 0x7f9d28158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2815cbb0 con 0x7f9d28157870 2023-10-10T20:23:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2815bfa0 con 0x7f9d2815a620 2023-10-10T20:23:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d2e67d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2815c280 con 0x7f9d2815d580 2023-10-10T20:23:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d27fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d28157870 0x7f9d2815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d27fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d28157870 0x7f9d2815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54812/0 (socket says 172.21.15.148:54812) 2023-10-10T20:23:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d27fff700 1 -- 172.21.15.148:0/2637062818 learned_addr learned my addr 172.21.15.148:0/2637062818 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.393+0000 7f9d277fe700 1 --2- 172.21.15.148:0/2637062818 >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 0x7f9d2815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d2815d580 0x7f9d28158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 -- 172.21.15.148:0/2637062818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d2815d580 msgr2=0x7f9d28158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d2815d580 0x7f9d28158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 -- 172.21.15.148:0/2637062818 >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 msgr2=0x7f9d2815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 --2- 172.21.15.148:0/2637062818 >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 0x7f9d2815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 -- 172.21.15.148:0/2637062818 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2815c560 con 0x7f9d28157870 2023-10-10T20:23:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d277fe700 1 --2- 172.21.15.148:0/2637062818 >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 0x7f9d2815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d27fff700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d28157870 0x7f9d2815b6a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9d2815cb20 tx=0x7f9d18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4f20d162ae2281 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d26ffd700 1 -- 172.21.15.148:0/2637062818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800b860 con 0x7f9d28157870 2023-10-10T20:23:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d26ffd700 1 -- 172.21.15.148:0/2637062818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1800b9c0 con 0x7f9d28157870 2023-10-10T20:23:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d26ffd700 1 -- 172.21.15.148:0/2637062818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800bd60 con 0x7f9d28157870 2023-10-10T20:23:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 -- 172.21.15.148:0/2637062818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d28157870 msgr2=0x7f9d2815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d28157870 0x7f9d2815b6a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9d2815cb20 tx=0x7f9d18009250 comp rx=0 tx=0).stop 2023-10-10T20:23:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 -- 172.21.15.148:0/2637062818 shutdown_connections 2023-10-10T20:23:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d2815d580 0x7f9d28158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/2637062818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d28157870 0x7f9d2815b6a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/2637062818 >> v2:172.21.15.161:3300/0 conn(0x7f9d2815a620 0x7f9d2815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.394+0000 7f9d2e67d700 1 -- 172.21.15.148:0/2637062818 >> 172.21.15.148:0/2637062818 conn(0x7f9d28068ce0 msgr2=0x7f9d280690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.395+0000 7f9d2e67d700 1 -- 172.21.15.148:0/2637062818 shutdown_connections 2023-10-10T20:23:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.395+0000 7f9d2e67d700 1 -- 172.21.15.148:0/2637062818 wait complete. 2023-10-10T20:23:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 Processor -- start 2023-10-10T20:23:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 -- start start 2023-10-10T20:23:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d28157870 0x7f9d280b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2815d580 0x7f9d2814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b2be0 con 0x7f9d2815a620 2023-10-10T20:23:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b1fd0 con 0x7f9d2815d580 2023-10-10T20:23:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.396+0000 7f9d2e67d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280b22b0 con 0x7f9d28157870 2023-10-10T20:23:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54814/0 (socket says 172.21.15.148:54814) 2023-10-10T20:23:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 -- 172.21.15.148:0/3719917620 learned_addr learned my addr 172.21.15.148:0/3719917620 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d27fff700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d28157870 0x7f9d280b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2815d580 0x7f9d2814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 -- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d28157870 msgr2=0x7f9d280b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d28157870 0x7f9d280b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 -- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2815d580 msgr2=0x7f9d2814b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2815d580 0x7f9d2814b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 -- 172.21.15.148:0/3719917620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180094f0 con 0x7f9d2815a620 2023-10-10T20:23:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d277fe700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c880 tx=0x7f9d1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d0ffff700 1 -- 172.21.15.148:0/3719917620 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c011600 con 0x7f9d2815a620 2023-10-10T20:23:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d0ffff700 1 -- 172.21.15.148:0/3719917620 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c011760 con 0x7f9d2815a620 2023-10-10T20:23:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d280b2590 con 0x7f9d2815a620 2023-10-10T20:23:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d0ffff700 1 -- 172.21.15.148:0/3719917620 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c011ae0 con 0x7f9d2815a620 2023-10-10T20:23:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.397+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d28151b00 con 0x7f9d2815a620 2023-10-10T20:23:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.399+0000 7f9d0ffff700 1 -- 172.21.15.148:0/3719917620 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d1c0193b0 con 0x7f9d2815a620 2023-10-10T20:23:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.399+0000 7f9d0ffff700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 0x7f9d140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.399+0000 7f9d0ffff700 1 -- 172.21.15.148:0/3719917620 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d1c084480 con 0x7f9d2815a620 2023-10-10T20:23:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.399+0000 7f9d27fff700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 0x7f9d140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.399+0000 7f9d27fff700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 0x7f9d140693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d2815cb20 tx=0x7f9d18017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.401+0000 7f9d37976b80 1 Processor -- start 2023-10-10T20:23:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.401+0000 7f9d37976b80 1 -- start start 2023-10-10T20:23:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec717a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec645a70 con 0x5606ec717f60 2023-10-10T20:23:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec643270 con 0x5606ec6eea50 2023-10-10T20:23:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606ec63f060 con 0x5606ec738d00 2023-10-10T20:23:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54820/0 (socket says 172.21.15.148:54820) 2023-10-10T20:23:19.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/268482441 learned_addr learned my addr 172.21.15.148:0/268482441 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d27fff700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d277fe700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec717a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 msgr2=0x5606ec717a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec717a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 msgr2=0x5606ec6eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/268482441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ec4d9270 con 0x5606ec717f60 2023-10-10T20:23:19.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d277fe700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec717a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d27fff700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000f450 tx=0x7f9d20007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1c262af1abbb00 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d0d7fa700 1 -- 172.21.15.148:0/268482441 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2000fee0 con 0x5606ec717f60 2023-10-10T20:23:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d0d7fa700 1 -- 172.21.15.148:0/268482441 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2001a480 con 0x5606ec717f60 2023-10-10T20:23:19.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.402+0000 7f9d0d7fa700 1 -- 172.21.15.148:0/268482441 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2001a800 con 0x5606ec717f60 2023-10-10T20:23:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 -- 172.21.15.148:0/268482441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 msgr2=0x5606ec6eacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000f450 tx=0x7f9d20007710 comp rx=0 tx=0).stop 2023-10-10T20:23:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 -- 172.21.15.148:0/268482441 shutdown_connections 2023-10-10T20:23:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec6eacf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 --2- 172.21.15.148:0/268482441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec717a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 -- 172.21.15.148:0/268482441 >> 172.21.15.148:0/268482441 conn(0x5606ec6f0370 msgr2=0x5606ec6f0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 -- 172.21.15.148:0/268482441 shutdown_connections 2023-10-10T20:23:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.403+0000 7f9d37976b80 1 -- 172.21.15.148:0/268482441 wait complete. 2023-10-10T20:23:19.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 -- start start 2023-10-10T20:23:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 Processor -- start 2023-10-10T20:23:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec713ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec5ea890 con 0x5606ec717f60 2023-10-10T20:23:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec605ea0 con 0x5606ec6eea50 2023-10-10T20:23:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606ec4c1e30 con 0x5606ec738d00 2023-10-10T20:23:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.404+0000 7f9d37976b80 10 client.0 _ll_register_callbacks cb 0x5606ec6b2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:23:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54828/0 (socket says 172.21.15.148:54828) 2023-10-10T20:23:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 -- 172.21.15.148:0/1245344402 learned_addr learned my addr 172.21.15.148:0/1245344402 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec713ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 -- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 msgr2=0x5606ec713ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec713ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 -- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 msgr2=0x5606ec72f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 -- 172.21.15.148:0/1245344402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d20007d10 con 0x5606ec717f60 2023-10-10T20:23:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c082a50 tx=0x7f9d1c082900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64410f2045b841a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:23:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c058600 con 0x5606ec717f60 2023-10-10T20:23:19.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d1c05c350 con 0x5606ec717f60 2023-10-10T20:23:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606ec6e3b70 con 0x5606ec717f60 2023-10-10T20:23:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d1c07f050 con 0x5606ec717f60 2023-10-10T20:23:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 10 client.6201 fetch_fsmap learned FSMap version 40 2023-10-10T20:23:19.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5606ec47bfe0 con 0x5606ec717f60 2023-10-10T20:23:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606ec6e3b70 con 0x5606ec717f60 2023-10-10T20:23:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9d1c082020 con 0x5606ec717f60 2023-10-10T20:23:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 10 client.6201 fetch_fsmap finished waiting for FSMap version 40 2023-10-10T20:23:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 10 client.6201 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:23:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 20 client.6201 populate_metadata read hostname 'smithi148' 2023-10-10T20:23:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.405+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec730680 0x5606ec730a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d37976b80 4 client.6201 mds_command: new command op to 6060 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:23:19.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 --> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606ec6c0aa0 con 0x5606ec730680 2023-10-10T20:23:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec730680 0x5606ec730a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d067fc700 10 client.6201 ms_handle_connect on v2:172.21.15.148:6835/934736302 2023-10-10T20:23:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec730680 0x5606ec730a50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc524ce18d246c0b server_cookie=c95c62227aef6482 in_seq=0 out_seq=0 2023-10-10T20:23:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d067fc700 1 -- 172.21.15.148:0/1245344402 <== mds.0 v2:172.21.15.148:6835/934736302 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5606ec6c0aa0 con 0x5606ec730680 2023-10-10T20:23:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d067fc700 10 client.6201 handle_command_reply: tid=0 2023-10-10T20:23:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.406+0000 7f9d37976b80 1 client.6201 shutdown 2023-10-10T20:23:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 msgr2=0x5606ec737760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c082a50 tx=0x7f9d1c082900 comp rx=0 tx=0).stop 2023-10-10T20:23:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 shutdown_connections 2023-10-10T20:23:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec730680 0x5606ec730a50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec713ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d067fc700 0 client.6201 ms_handle_reset on v2:172.21.15.148:6835/934736302 2023-10-10T20:23:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1245344402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec737760 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 >> 172.21.15.148:0/1245344402 conn(0x5606ec6f0370 msgr2=0x5606ec6eb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 shutdown_connections 2023-10-10T20:23:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 1 -- 172.21.15.148:0/1245344402 wait complete. 2023-10-10T20:23:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.407+0000 7f9d37976b80 20 client.6201 trim_cache size 0 max 16384 2023-10-10T20:23:19.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.423+0000 7f9d37976b80 1 Processor -- start 2023-10-10T20:23:19.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 -- start start 2023-10-10T20:23:19.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec717f60 0x5606ec734420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec6ebae0 con 0x5606ec738d00 2023-10-10T20:23:19.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec72fd80 con 0x5606ec6eea50 2023-10-10T20:23:19.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606ec71e490 con 0x5606ec717f60 2023-10-10T20:23:19.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d2cc1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d2cc1a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54844/0 (socket says 172.21.15.148:54844) 2023-10-10T20:23:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.424+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/3448387030 learned_addr learned my addr 172.21.15.148:0/3448387030 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec717f60 msgr2=0x5606ec734420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d27fff700 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d277fe700 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec717f60 0x5606ec734420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec717f60 0x5606ec734420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 msgr2=0x5606ec72f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 -- 172.21.15.148:0/3448387030 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ec60d780 con 0x5606ec738d00 2023-10-10T20:23:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000f420 tx=0x7f9d2001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc8af4139b18b96 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d0dffb700 1 -- 172.21.15.148:0/3448387030 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2002b070 con 0x5606ec738d00 2023-10-10T20:23:19.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d0dffb700 1 -- 172.21.15.148:0/3448387030 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200032f0 con 0x5606ec738d00 2023-10-10T20:23:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.425+0000 7f9d0dffb700 1 -- 172.21.15.148:0/3448387030 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2001a5f0 con 0x5606ec738d00 2023-10-10T20:23:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- 172.21.15.148:0/3448387030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 msgr2=0x5606ec734960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000f420 tx=0x7f9d2001c000 comp rx=0 tx=0).stop 2023-10-10T20:23:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- 172.21.15.148:0/3448387030 shutdown_connections 2023-10-10T20:23:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec738d00 0x5606ec734960 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec72f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 --2- 172.21.15.148:0/3448387030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec717f60 0x5606ec734420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- 172.21.15.148:0/3448387030 >> 172.21.15.148:0/3448387030 conn(0x5606ec6f0370 msgr2=0x5606ec7374b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- 172.21.15.148:0/3448387030 shutdown_connections 2023-10-10T20:23:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- 172.21.15.148:0/3448387030 wait complete. 2023-10-10T20:23:19.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 -- start start 2023-10-10T20:23:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.426+0000 7f9d37976b80 1 Processor -- start 2023-10-10T20:23:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec72c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec734740 con 0x5606ec717f60 2023-10-10T20:23:19.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ec6e9e30 con 0x5606ec6eea50 2023-10-10T20:23:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606ec7334b0 con 0x5606ec738d00 2023-10-10T20:23:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d37976b80 10 client.0 _ll_register_callbacks cb 0x5606ec6b2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:23:19.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54852/0 (socket says 172.21.15.148:54852) 2023-10-10T20:23:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 -- 172.21.15.148:0/1121510191 learned_addr learned my addr 172.21.15.148:0/1121510191 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d2cc1a700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec72c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 -- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 msgr2=0x5606ec72c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec72c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 -- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 msgr2=0x5606ec6eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.427+0000 7f9d277fe700 1 -- 172.21.15.148:0/1121510191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606ec4d9270 con 0x5606ec717f60 2023-10-10T20:23:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d277fe700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00ce70 tx=0x7f9d1c071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14b629913423266 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:23:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c07f040 con 0x5606ec717f60 2023-10-10T20:23:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d1c05c350 con 0x5606ec717f60 2023-10-10T20:23:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606ec5e1e00 con 0x5606ec717f60 2023-10-10T20:23:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d1c058600 con 0x5606ec717f60 2023-10-10T20:23:19.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 10 client.6207 fetch_fsmap learned FSMap version 40 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5606ec5f4470 con 0x5606ec717f60 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606ec5e1e00 con 0x5606ec717f60 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9d1c063020 con 0x5606ec717f60 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 10 client.6207 fetch_fsmap finished waiting for FSMap version 40 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 10 client.6207 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-10T20:23:19.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 20 client.6207 populate_metadata read hostname 'smithi148' 2023-10-10T20:23:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec79bb50 0x5606ec79bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 4 client.6207 mds_command: new command op to 6060 tid=0[{"prefix": "status"}] 2023-10-10T20:23:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 --> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5606ec5e1b20 con 0x5606ec79bb50 2023-10-10T20:23:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec79bb50 0x5606ec79bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d067fc700 10 client.6207 ms_handle_connect on v2:172.21.15.148:6835/934736302 2023-10-10T20:23:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.428+0000 7f9d27fff700 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec79bb50 0x5606ec79bf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c7d6285a5db8df server_cookie=93a4791357825d20 in_seq=0 out_seq=0 2023-10-10T20:23:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d067fc700 1 -- 172.21.15.148:0/1121510191 <== mds.0 v2:172.21.15.148:6835/934736302 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5606ec5e1b20 con 0x5606ec79bb50 2023-10-10T20:23:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d067fc700 10 client.6207 handle_command_reply: tid=0 2023-10-10T20:23:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 client.6207 shutdown 2023-10-10T20:23:19.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 msgr2=0x5606ec72bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00ce70 tx=0x7f9d1c071000 comp rx=0 tx=0).stop 2023-10-10T20:23:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 shutdown_connections 2023-10-10T20:23:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:6835/934736302,v1:172.21.15.148:6837/934736302] conn(0x5606ec79bb50 0x5606ec79bf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5606ec738d00 0x5606ec72c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5606ec6eea50 0x5606ec6eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d067fc700 0 client.6207 ms_handle_reset on v2:172.21.15.148:6835/934736302 2023-10-10T20:23:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 --2- 172.21.15.148:0/1121510191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606ec717f60 0x5606ec72bf90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 >> 172.21.15.148:0/1121510191 conn(0x5606ec6f0370 msgr2=0x5606ec735040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 shutdown_connections 2023-10-10T20:23:19.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 1 -- 172.21.15.148:0/1121510191 wait complete. 2023-10-10T20:23:19.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d37976b80 20 client.6207 trim_cache size 0 max 16384 2023-10-10T20:23:19.481 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:23:19.481 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:23:19.482 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:23:19.482 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 6060, 2023-10-10T20:23:19.482 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:23:19.482 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:23:19.482 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.788487248, 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 40, 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 82, 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 82, 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 11.784383041 2023-10-10T20:23:19.483 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:23:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 msgr2=0x7f9d140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 0x7f9d140693e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d2815cb20 tx=0x7f9d18017000 comp rx=0 tx=0).stop 2023-10-10T20:23:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 msgr2=0x7f9d281515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00c880 tx=0x7f9d1c008040 comp rx=0 tx=0).stop 2023-10-10T20:23:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 shutdown_connections 2023-10-10T20:23:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d14066f30 0x7f9d140693e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2815d580 0x7f9d2814b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d28157870 0x7f9d280b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 --2- 172.21.15.148:0/3719917620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d2815a620 0x7f9d281515c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.429+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 >> 172.21.15.148:0/3719917620 conn(0x7f9d28068ce0 msgr2=0x7f9d28159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.430+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 shutdown_connections 2023-10-10T20:23:19.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.430+0000 7f9d2e67d700 1 -- 172.21.15.148:0/3719917620 wait complete. 2023-10-10T20:23:19.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:23:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.600+0000 7fdef0147700 1 Processor -- start 2023-10-10T20:23:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.601+0000 7fdef0147700 1 -- start start 2023-10-10T20:23:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.601+0000 7fdef0147700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 0x7fdee80b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.601+0000 7fdef0147700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdee80cde50 0x7fdee80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdef0147700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee815db80 con 0x7fdee815a1e0 2023-10-10T20:23:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdef0147700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815e800 con 0x7fdee80cde50 2023-10-10T20:23:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdef0147700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815de60 con 0x7fdee8159e10 2023-10-10T20:23:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdee80cde50 0x7fdee80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdee80cde50 0x7fdee80b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54866/0 (socket says 172.21.15.148:54866) 2023-10-10T20:23:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 -- 172.21.15.148:0/2553623211 learned_addr learned my addr 172.21.15.148:0/2553623211 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeedee3700 1 --2- 172.21.15.148:0/2553623211 >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 0x7fdee80b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 -- 172.21.15.148:0/2553623211 >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 msgr2=0x7fdee80b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 --2- 172.21.15.148:0/2553623211 >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 0x7fdee80b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 -- 172.21.15.148:0/2553623211 >> v1:172.21.15.161:6789/0 conn(0x7fdee815a1e0 legacy=0x7fdee80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 -- 172.21.15.148:0/2553623211 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee815e140 con 0x7fdee80cde50 2023-10-10T20:23:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeedee3700 1 --2- 172.21.15.148:0/2553623211 >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 0x7fdee80b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeee6e4700 1 --2- 172.21.15.148:0/2553623211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee80cde50 0x7fdee80b3260 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdee8068a60 tx=0x7fdee0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab36b91c7e98db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeecee1700 1 -- 172.21.15.148:0/2553623211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000b860 con 0x7fdee80cde50 2023-10-10T20:23:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeecee1700 1 -- 172.21.15.148:0/2553623211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee000b9c0 con 0x7fdee80cde50 2023-10-10T20:23:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.602+0000 7fdeecee1700 1 -- 172.21.15.148:0/2553623211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000bd60 con 0x7fdee80cde50 2023-10-10T20:23:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 -- 172.21.15.148:0/2553623211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee80cde50 msgr2=0x7fdee80b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 --2- 172.21.15.148:0/2553623211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee80cde50 0x7fdee80b3260 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdee8068a60 tx=0x7fdee0009250 comp rx=0 tx=0).stop 2023-10-10T20:23:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 -- 172.21.15.148:0/2553623211 shutdown_connections 2023-10-10T20:23:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 --2- 172.21.15.148:0/2553623211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee80cde50 0x7fdee80b3260 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 --2- 172.21.15.148:0/2553623211 >> v2:172.21.15.161:3300/0 conn(0x7fdee8159e10 0x7fdee80b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 -- 172.21.15.148:0/2553623211 >> 172.21.15.148:0/2553623211 conn(0x7fdee8067660 msgr2=0x7fdee8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 -- 172.21.15.148:0/2553623211 shutdown_connections 2023-10-10T20:23:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.603+0000 7fdef0147700 1 -- 172.21.15.148:0/2553623211 wait complete. 2023-10-10T20:23:19.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.604+0000 7fdef0147700 1 Processor -- start 2023-10-10T20:23:19.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.604+0000 7fdef0147700 1 -- start start 2023-10-10T20:23:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee80cde50 0x7fdee81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee8159e10 0x7fdee80b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c5730 con 0x7fdee815a1e0 2023-10-10T20:23:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c62b0 con 0x7fdee8159e10 2023-10-10T20:23:19.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c5980 con 0x7fdee80cde50 2023-10-10T20:23:19.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54868/0 (socket says 172.21.15.148:54868) 2023-10-10T20:23:19.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 -- 172.21.15.148:0/1126471861 learned_addr learned my addr 172.21.15.148:0/1126471861 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:19.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeedee3700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee80cde50 0x7fdee81582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeed6e2700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee8159e10 0x7fdee80b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 -- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee80cde50 msgr2=0x7fdee81582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee80cde50 0x7fdee81582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 -- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee8159e10 msgr2=0x7fdee80b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee8159e10 0x7fdee80b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 -- 172.21.15.148:0/1126471861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00094f0 con 0x7fdee815a1e0 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeee6e4700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdee00090b0 tx=0x7fdee0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0011e80 con 0x7fdee815a1e0 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee001a660 con 0x7fdee815a1e0 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.605+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80c5c60 con 0x7fdee815a1e0 2023-10-10T20:23:19.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.606+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee001aa00 con 0x7fdee815a1e0 2023-10-10T20:23:19.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.606+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee8158810 con 0x7fdee815a1e0 2023-10-10T20:23:19.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.607+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee804c3c0 con 0x7fdee815a1e0 2023-10-10T20:23:19.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.607+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdee001b430 con 0x7fdee815a1e0 2023-10-10T20:23:19.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.607+0000 7fdeddffb700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 0x7fdecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:19.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.607+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdee0085cd0 con 0x7fdee815a1e0 2023-10-10T20:23:19.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.608+0000 7fdeedee3700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 0x7fdecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:19.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.608+0000 7fdeedee3700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 0x7fdecc0693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007740 tx=0x7fdee40099d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:19.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.612+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdee0052f40 con 0x7fdee815a1e0 2023-10-10T20:23:19.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.839+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdee80c63f0 con 0x7fdee815a1e0 2023-10-10T20:23:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.839+0000 7fdeddffb700 1 -- 172.21.15.148:0/1126471861 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdee0015060 con 0x7fdee815a1e0 2023-10-10T20:23:19.842 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:23:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 msgr2=0x7fdecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 0x7fdecc0693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007740 tx=0x7fdee40099d0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 msgr2=0x7fdee80b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdee00090b0 tx=0x7fdee0017000 comp rx=0 tx=0).stop 2023-10-10T20:23:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 shutdown_connections 2023-10-10T20:23:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdecc066f30 0x7fdecc0693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdee815a1e0 0x7fdee80b4030 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee80cde50 0x7fdee81582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 --2- 172.21.15.148:0/1126471861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee8159e10 0x7fdee80b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 >> 172.21.15.148:0/1126471861 conn(0x7fdee8067660 msgr2=0x7fdee8159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 shutdown_connections 2023-10-10T20:23:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:19.842+0000 7fdef0147700 1 -- 172.21.15.148:0/1126471861 wait complete. 2023-10-10T20:23:19.856 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:23:19.856 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:23:19.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:23:19.857 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:23:19.857 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:23:19.857 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:23:19.858 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:23:19.858 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:23:19.858 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:23:19.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:19.859 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:23:19.875 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:23:19.875 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:23:19.875 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:23:19.875 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.876 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:23:19.876 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.876 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:19.876 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:19.876 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:23:19.877 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:19.877 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:19.877 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:23:19.877 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:19.877 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:19.878 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:23:19.878 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:19.878 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:23:19.878 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42311sec preferred_lft 42311sec 2023-10-10T20:23:19.878 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:23:19.879 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.879 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:23:19.879 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:19.879 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:23:19.879 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:23:19.880 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.881 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:23:19.881 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:23:19.881 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:23:19.881 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:19.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:19.882 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:23:19.938 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:23:19.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:19.940 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:23:19.996 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:23:19.996 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:23:19.997 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:23:19.997 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:23:19.997 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:23:19.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:19.998 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:23:19.998 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:23:19.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:23:19.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:23:19.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:23:20.000 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:23:20.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.096 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:23:20.111 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:23:20.112 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:23:20.112 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:23:20.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:23:20.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:23:20.179 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:23:20.233 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:23:20.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:23:20.234 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:23:20.287 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:23:20.288 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:23:20.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:20.368 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:20.368 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:20.387 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:20.387 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:20.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:23:20.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:23:20.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:23:20.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:23:20.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:23:20.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:23:20.409 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:20.409 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:20.409 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:23:20.410 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:23:20.410 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:23:20.410 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:23:20.410 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:23:20.410 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:23:20.411 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:23:20.411 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:23:20.411 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:23:20.412 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:23:20.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:23:20.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:23:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 Processor -- start 2023-10-10T20:23:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 -- start start 2023-10-10T20:23:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4b40ade00 0x7ff4b4161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40c9cb0 con 0x7ff4b40abae0 2023-10-10T20:23:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40c90a0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.631+0000 7ff4ba756700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40c9380 con 0x7ff4b40ade00 2023-10-10T20:23:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4b40ade00 0x7ff4b4161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4b40ade00 0x7ff4b4161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54870/0 (socket says 172.21.15.148:54870) 2023-10-10T20:23:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff4b40abae0 0x7ff4b40c6eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50246/0 (socket says 172.21.15.148:50246) 2023-10-10T20:23:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/4120048916 learned_addr learned my addr 172.21.15.148:0/4120048916 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/4120048916 >> v1:172.21.15.161:6789/0 conn(0x7ff4b40c75c0 legacy=0x7ff4b40c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/4120048916 >> v1:172.21.15.148:6789/0 conn(0x7ff4b40abae0 legacy=0x7ff4b40c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/4120048916 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40c9660 con 0x7ff4b40ade00 2023-10-10T20:23:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b2ffd700 1 -- 172.21.15.148:0/4120048916 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226694341 0 0) 0x7ff4b40c9cb0 con 0x7ff4b40abae0 2023-10-10T20:23:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b8cf3700 1 --2- 172.21.15.148:0/4120048916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40ade00 0x7ff4b4161b40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d1a0 tx=0x7ff4ac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd66e449a9713d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b2ffd700 1 -- 172.21.15.148:0/4120048916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac010070 con 0x7ff4b40ade00 2023-10-10T20:23:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b2ffd700 1 -- 172.21.15.148:0/4120048916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00de80 con 0x7ff4b40ade00 2023-10-10T20:23:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.632+0000 7ff4b2ffd700 1 -- 172.21.15.148:0/4120048916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00b320 con 0x7ff4b40ade00 2023-10-10T20:23:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 -- 172.21.15.148:0/4120048916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40ade00 msgr2=0x7ff4b4161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 --2- 172.21.15.148:0/4120048916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40ade00 0x7ff4b4161b40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d1a0 tx=0x7ff4ac00ba50 comp rx=0 tx=0).stop 2023-10-10T20:23:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 -- 172.21.15.148:0/4120048916 shutdown_connections 2023-10-10T20:23:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 --2- 172.21.15.148:0/4120048916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40ade00 0x7ff4b4161b40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 -- 172.21.15.148:0/4120048916 >> 172.21.15.148:0/4120048916 conn(0x7ff4b4067660 msgr2=0x7ff4b4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 -- 172.21.15.148:0/4120048916 shutdown_connections 2023-10-10T20:23:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.633+0000 7ff4ba756700 1 -- 172.21.15.148:0/4120048916 wait complete. 2023-10-10T20:23:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.634+0000 7ff4ba756700 1 Processor -- start 2023-10-10T20:23:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.634+0000 7ff4ba756700 1 -- start start 2023-10-10T20:23:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 0x7ff4b40af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4b40ade00 0x7ff4b4156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c5410 con 0x7ff4b40c75c0 2023-10-10T20:23:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41578d0 con 0x7ff4b40abae0 2023-10-10T20:23:20.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4157030 con 0x7ff4b40ade00 2023-10-10T20:23:20.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:20.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54882/0 (socket says 172.21.15.148:54882) 2023-10-10T20:23:20.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/1312232995 learned_addr learned my addr 172.21.15.148:0/1312232995 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:20.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b3fff700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 0x7ff4b40af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b37fe700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4b40ade00 0x7ff4b4156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4b40ade00 msgr2=0x7ff4b4156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4b40ade00 0x7ff4b4156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 msgr2=0x7ff4b40af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 0x7ff4b40af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 -- 172.21.15.148:0/1312232995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac00bcf0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b3fff700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 0x7ff4b40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:20.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4b8cf3700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00bcc0 tx=0x7ff4ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:20.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00b4d0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac020430 con 0x7ff4b40c75c0 2023-10-10T20:23:20.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b4157310 con 0x7ff4b40c75c0 2023-10-10T20:23:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0207f0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.635+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b40c5640 con 0x7ff4b40c75c0 2023-10-10T20:23:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.636+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b404c3c0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.637+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4ac020b60 con 0x7ff4b40c75c0 2023-10-10T20:23:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.637+0000 7ff49bfff700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 0x7ff4a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.637+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4ac088290 con 0x7ff4b40c75c0 2023-10-10T20:23:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.637+0000 7ff4b3fff700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 0x7ff4a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.637+0000 7ff4b3fff700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 0x7ff4a00693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40c2fa0 tx=0x7ff4a4007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.642+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4ac055500 con 0x7ff4b40c75c0 2023-10-10T20:23:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.871+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4b406aeb0 con 0x7ff4b40c75c0 2023-10-10T20:23:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.871+0000 7ff49bfff700 1 -- 172.21.15.148:0/1312232995 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4ac058b20 con 0x7ff4b40c75c0 2023-10-10T20:23:20.874 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:23:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 msgr2=0x7ff4a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 0x7ff4a00693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40c2fa0 tx=0x7ff4a4007cb0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 msgr2=0x7ff4b41690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:20.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00bcc0 tx=0x7ff4ac019000 comp rx=0 tx=0).stop 2023-10-10T20:23:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 shutdown_connections 2023-10-10T20:23:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4a0066f30 0x7ff4a00693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4b40c75c0 0x7ff4b41690d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b40abae0 0x7ff4b40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 --2- 172.21.15.148:0/1312232995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4b40ade00 0x7ff4b4156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:20.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 >> 172.21.15.148:0/1312232995 conn(0x7ff4b4067660 msgr2=0x7ff4b40c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:20.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 shutdown_connections 2023-10-10T20:23:20.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:20.873+0000 7ff4ba756700 1 -- 172.21.15.148:0/1312232995 wait complete. 2023-10-10T20:23:20.887 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:23:20.888 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:23:20.888 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:23:20.888 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:23:20.889 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:23:20.889 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:23:20.889 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:23:20.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:23:20.890 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:23:20.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.891 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:23:20.910 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:23:20.910 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:23:20.911 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:23:20.911 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.911 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:23:20.912 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.912 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:20.912 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:20.912 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:23:20.912 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:20.913 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:20.913 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:23:20.913 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:23:20.913 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:20.913 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:23:20.914 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:20.914 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:23:20.914 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42307sec preferred_lft 42307sec 2023-10-10T20:23:20.914 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:23:20.914 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:23:20.915 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.916 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:23:20.916 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:23:20.916 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:23:20.916 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.916 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:23:20.917 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:23:20.917 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:23:20.917 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:23:20.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.918 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:23:20.980 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:23:20.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:20.983 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:23:21.040 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:23:21.041 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:23:21.041 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:23:21.041 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:23:21.041 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:23:21.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:21.042 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:23:21.043 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:23:21.043 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:23:21.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:23:21.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:23:21.044 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:23:21.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:21.152 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:23:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:23:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:23:21.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:23:21.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:21.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:23:21.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:23:21.233 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:23:21.287 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:23:21.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:23:21.288 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:23:21.342 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:23:21.343 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:23:21.343 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:21.425 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:21.425 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:21.444 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:21.444 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:21.444 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:23:21.445 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:23:21.445 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:23:21.445 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:23:21.445 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:23:21.445 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:23:21.466 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:23:21.466 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:23:21.466 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:23:21.467 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:23:21.467 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:23:21.467 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:23:21.467 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:23:21.467 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:23:21.468 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:23:21.468 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:23:21.468 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:23:21.469 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:23:21.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:21.470 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:23:21.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:21.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.666+0000 7f434cd36700 1 Processor -- start 2023-10-10T20:23:21.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.666+0000 7f434cd36700 1 -- start start 2023-10-10T20:23:21.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43480cd6e0 0x7f43480cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f434815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f434815dda0 0x7f434815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434815e850 con 0x7f43480cd6e0 2023-10-10T20:23:21.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43480cd030 con 0x7f434815dda0 2023-10-10T20:23:21.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480cb7c0 con 0x7f43480cdab0 2023-10-10T20:23:21.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43480cd6e0 0x7f43480cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43480cd6e0 0x7f43480cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54888/0 (socket says 172.21.15.148:54888) 2023-10-10T20:23:21.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 -- 172.21.15.148:0/2469655106 learned_addr learned my addr 172.21.15.148:0/2469655106 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:21.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f4347fff700 1 --2- 172.21.15.148:0/2469655106 >> v2:172.21.15.161:3300/0 conn(0x7f434815dda0 0x7f434815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f4346ffd700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f434815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 -- 172.21.15.148:0/2469655106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 msgr2=0x7f434815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:21.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f434815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 -- 172.21.15.148:0/2469655106 >> v2:172.21.15.161:3300/0 conn(0x7f434815dda0 msgr2=0x7f434815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:21.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 --2- 172.21.15.148:0/2469655106 >> v2:172.21.15.161:3300/0 conn(0x7f434815dda0 0x7f434815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 -- 172.21.15.148:0/2469655106 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434815e290 con 0x7f43480cd6e0 2023-10-10T20:23:21.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43477fe700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43480cd6e0 0x7f43480cbc50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4348089ac0 tx=0x7f4338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee7c8ed2ad20eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:21.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43467fc700 1 -- 172.21.15.148:0/2469655106 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800b640 con 0x7f43480cd6e0 2023-10-10T20:23:21.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43467fc700 1 -- 172.21.15.148:0/2469655106 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800b7a0 con 0x7f43480cd6e0 2023-10-10T20:23:21.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f43467fc700 1 -- 172.21.15.148:0/2469655106 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800bb60 con 0x7f43480cd6e0 2023-10-10T20:23:21.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- 172.21.15.148:0/2469655106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43480cd6e0 msgr2=0x7f43480cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:21.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43480cd6e0 0x7f43480cbc50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4348089ac0 tx=0x7f4338009250 comp rx=0 tx=0).stop 2023-10-10T20:23:21.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- 172.21.15.148:0/2469655106 shutdown_connections 2023-10-10T20:23:21.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- 172.21.15.148:0/2469655106 >> v2:172.21.15.161:3300/0 conn(0x7f434815dda0 0x7f434815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43480cd6e0 0x7f43480cbc50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 --2- 172.21.15.148:0/2469655106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f434815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.668+0000 7f434cd36700 1 -- 172.21.15.148:0/2469655106 >> 172.21.15.148:0/2469655106 conn(0x7f4348067660 msgr2=0x7f4348067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:21.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.669+0000 7f434cd36700 1 -- 172.21.15.148:0/2469655106 shutdown_connections 2023-10-10T20:23:21.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.669+0000 7f434cd36700 1 -- 172.21.15.148:0/2469655106 wait complete. 2023-10-10T20:23:21.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.670+0000 7f434cd36700 1 Processor -- start 2023-10-10T20:23:21.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.670+0000 7f434cd36700 1 -- start start 2023-10-10T20:23:21.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43480cd6e0 0x7f4348147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f43480b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348147cb0 con 0x7f434815dda0 2023-10-10T20:23:21.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480c8be0 con 0x7f43480cd6e0 2023-10-10T20:23:21.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480c91a0 con 0x7f43480cdab0 2023-10-10T20:23:21.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54896/0 (socket says 172.21.15.148:54896) 2023-10-10T20:23:21.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 -- 172.21.15.148:0/695867027 learned_addr learned my addr 172.21.15.148:0/695867027 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:21.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f43477fe700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43480cd6e0 0x7f4348147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4346ffd700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f43480b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 -- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 msgr2=0x7f43480b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:21.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f43480b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 -- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43480cd6e0 msgr2=0x7f4348147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43480cd6e0 0x7f4348147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 -- 172.21.15.148:0/695867027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380094f0 con 0x7f434815dda0 2023-10-10T20:23:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4346ffd700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f43480b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:21.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f4347fff700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4340009ec0 tx=0x7f434000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000cc20 con 0x7f434815dda0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000f040 con 0x7f434815dda0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340014480 con 0x7f434815dda0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43480b2690 con 0x7f434815dda0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.671+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43480c8e30 con 0x7f434815dda0 2023-10-10T20:23:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.672+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4348089280 con 0x7f434815dda0 2023-10-10T20:23:21.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.673+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43400147f0 con 0x7f434815dda0 2023-10-10T20:23:21.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.673+0000 7f432f7fe700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 0x7f43340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:21.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.673+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43400851c0 con 0x7f434815dda0 2023-10-10T20:23:21.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.673+0000 7f43477fe700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 0x7f43340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:21.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.674+0000 7f43477fe700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 0x7f43340693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4348089ac0 tx=0x7f4338017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:21.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.678+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4340052430 con 0x7f434815dda0 2023-10-10T20:23:21.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.903+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f434804bb80 con 0x7f434815dda0 2023-10-10T20:23:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.903+0000 7f432f7fe700 1 -- 172.21.15.148:0/695867027 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4340055a50 con 0x7f434815dda0 2023-10-10T20:23:21.906 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:21.906 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 msgr2=0x7f43340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 0x7f43340693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4348089ac0 tx=0x7f4338017000 comp rx=0 tx=0).stop 2023-10-10T20:23:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 msgr2=0x7f43481590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4340009ec0 tx=0x7f434000b040 comp rx=0 tx=0).stop 2023-10-10T20:23:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 shutdown_connections 2023-10-10T20:23:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4334066f30 0x7f43340693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f434815dda0 0x7f43481590a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43480cd6e0 0x7f4348147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 --2- 172.21.15.148:0/695867027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43480cdab0 0x7f43480b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 >> 172.21.15.148:0/695867027 conn(0x7f4348067660 msgr2=0x7f4348156850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 shutdown_connections 2023-10-10T20:23:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:21.905+0000 7f434cd36700 1 -- 172.21.15.148:0/695867027 wait complete. 2023-10-10T20:23:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:21.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-10T20:23:22.053 INFO:teuthology.orchestra.run.smithi148.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":6060,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4008707010000006,"mdsmap_epoch":40,"osdmap_epoch":82,"osdmap_epoch_barrier":82,"uptime":14.396770463999999} 2023-10-10T20:23:22.054 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 6060, "mdsmap_epoch": 40, "osdmap_epoch": 82, "osdmap_epoch_barrier": 82, "rank_uptime": 8.400870701, "state": "up:active", "uptime": 14.396770464, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:23:22.054 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:23:22.185 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:23:22.185 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:23:22.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-10T20:23:22.309 INFO:teuthology.orchestra.run.smithi148.stdout:{"mon_tick_interval":"5"} 2023-10-10T20:23:22.310 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-10T20:23:22.310 INFO:tasks.cephfs.test_full:pool capacity 557102656, 531MB should be enough to fill it 2023-10-10T20:23:22.311 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:23:22.311 DEBUG:teuthology.orchestra.run.smithi148:> import time 2023-10-10T20:23:22.311 DEBUG:teuthology.orchestra.run.smithi148:> import datetime 2023-10-10T20:23:22.312 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-10-10T20:23:22.312 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:23:22.312 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.313 DEBUG:teuthology.orchestra.run.smithi148:> # Write some buffered data through before going full, all should be well 2023-10-10T20:23:22.313 DEBUG:teuthology.orchestra.run.smithi148:> print("writing some data through which we expect to succeed") 2023-10-10T20:23:22.313 DEBUG:teuthology.orchestra.run.smithi148:> bytes = 0 2023-10-10T20:23:22.314 DEBUG:teuthology.orchestra.run.smithi148:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-10T20:23:22.314 DEBUG:teuthology.orchestra.run.smithi148:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-10T20:23:22.315 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-10T20:23:22.315 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:23:22.315 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-10T20:23:22.316 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.316 DEBUG:teuthology.orchestra.run.smithi148:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-10T20:23:22.316 DEBUG:teuthology.orchestra.run.smithi148:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-10T20:23:22.317 DEBUG:teuthology.orchestra.run.smithi148:> # buffered IO, the error should always only appear from fsync and not 2023-10-10T20:23:22.317 DEBUG:teuthology.orchestra.run.smithi148:> # from write 2023-10-10T20:23:22.317 DEBUG:teuthology.orchestra.run.smithi148:> full = False 2023-10-10T20:23:22.318 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.318 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(531 * 1.1)): 2023-10-10T20:23:22.318 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:23:22.319 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:23:22.319 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-10T20:23:22.319 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2023-10-10T20:23:22.320 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2023-10-10T20:23:22.320 DEBUG:teuthology.orchestra.run.smithi148:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-10T20:23:22.320 DEBUG:teuthology.orchestra.run.smithi148:> raise 2023-10-10T20:23:22.321 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:23:22.321 DEBUG:teuthology.orchestra.run.smithi148:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:23:22.321 DEBUG:teuthology.orchestra.run.smithi148:> full = True 2023-10-10T20:23:22.322 DEBUG:teuthology.orchestra.run.smithi148:> break 2023-10-10T20:23:22.322 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.323 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:23:22.323 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:23:22.323 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed successfully") 2023-10-10T20:23:22.324 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2023-10-10T20:23:22.324 DEBUG:teuthology.orchestra.run.smithi148:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:23:22.324 DEBUG:teuthology.orchestra.run.smithi148:> full = True 2023-10-10T20:23:22.325 DEBUG:teuthology.orchestra.run.smithi148:> break 2023-10-10T20:23:22.325 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:23:22.325 DEBUG:teuthology.orchestra.run.smithi148:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:23:22.326 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.326 DEBUG:teuthology.orchestra.run.smithi148:> if n > 531 * 0.9: 2023-10-10T20:23:22.326 DEBUG:teuthology.orchestra.run.smithi148:> # Be cautious in the last region where we expect to hit 2023-10-10T20:23:22.327 DEBUG:teuthology.orchestra.run.smithi148:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-10T20:23:22.327 DEBUG:teuthology.orchestra.run.smithi148:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-10T20:23:22.327 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(15.0) 2023-10-10T20:23:22.328 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.328 DEBUG:teuthology.orchestra.run.smithi148:> if not full: 2023-10-10T20:23:22.328 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-10T20:23:22.329 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.329 DEBUG:teuthology.orchestra.run.smithi148:> # close() should not raise an error because we already caught it in 2023-10-10T20:23:22.329 DEBUG:teuthology.orchestra.run.smithi148:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-10T20:23:22.329 DEBUG:teuthology.orchestra.run.smithi148:> # since then because all IOs got cancelled on the full flag. 2023-10-10T20:23:22.330 DEBUG:teuthology.orchestra.run.smithi148:> print("calling close") 2023-10-10T20:23:22.330 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:23:22.331 DEBUG:teuthology.orchestra.run.smithi148:> print("close() did not raise error") 2023-10-10T20:23:22.331 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:23:22.331 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-10T20:23:22.332 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:23:22.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:23:22.821 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:23:22.868 INFO:teuthology.orchestra.run.smithi148.stdout:writing some data through which we expect to succeed 2023-10-10T20:23:22.869 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-10T20:23:22.869 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.869 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.869 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 1.00 MB 2023-10-10T20:23:22.870 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.870 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.870 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 2.00 MB 2023-10-10T20:23:22.871 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.871 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.871 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 3.00 MB 2023-10-10T20:23:22.872 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.872 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.872 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 4.00 MB 2023-10-10T20:23:22.872 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.872 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.873 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 5.00 MB 2023-10-10T20:23:22.873 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.873 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.874 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 6.00 MB 2023-10-10T20:23:22.874 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.874 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.874 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 7.00 MB 2023-10-10T20:23:22.875 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.875 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.875 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 8.00 MB 2023-10-10T20:23:22.875 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.876 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.876 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 9.00 MB 2023-10-10T20:23:22.876 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.877 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.877 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 10.00 MB 2023-10-10T20:23:22.877 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.877 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.878 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 11.00 MB 2023-10-10T20:23:22.878 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.878 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.878 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 12.00 MB 2023-10-10T20:23:22.879 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.879 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.879 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 13.00 MB 2023-10-10T20:23:22.879 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.879 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 14.00 MB 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 15.00 MB 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.880 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.881 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 16.00 MB 2023-10-10T20:23:22.881 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.881 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.881 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 17.00 MB 2023-10-10T20:23:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 18.00 MB 2023-10-10T20:23:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.882 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 19.00 MB 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 20.00 MB 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.883 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.884 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 21.00 MB 2023-10-10T20:23:22.884 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.884 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.884 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 22.00 MB 2023-10-10T20:23:22.884 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.885 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.885 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 23.00 MB 2023-10-10T20:23:22.885 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.885 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.885 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 24.00 MB 2023-10-10T20:23:22.886 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.886 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.886 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 25.00 MB 2023-10-10T20:23:22.886 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.886 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 26.00 MB 2023-10-10T20:23:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 27.00 MB 2023-10-10T20:23:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 28.00 MB 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 29.00 MB 2023-10-10T20:23:22.888 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.889 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.889 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 30.00 MB 2023-10-10T20:23:22.889 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.889 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.889 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 31.00 MB 2023-10-10T20:23:22.890 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.890 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.890 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 32.00 MB 2023-10-10T20:23:22.890 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.890 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.891 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 33.00 MB 2023-10-10T20:23:22.891 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.891 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.891 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 34.00 MB 2023-10-10T20:23:22.891 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.892 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.892 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 35.00 MB 2023-10-10T20:23:22.892 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.892 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.892 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 36.00 MB 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 37.00 MB 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.893 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 38.00 MB 2023-10-10T20:23:22.894 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.894 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.894 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 39.00 MB 2023-10-10T20:23:22.894 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 40.00 MB 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 41.00 MB 2023-10-10T20:23:22.895 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.896 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.896 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 42.00 MB 2023-10-10T20:23:22.896 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.896 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.896 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 43.00 MB 2023-10-10T20:23:22.897 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.897 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.897 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 44.00 MB 2023-10-10T20:23:22.897 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.898 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.898 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 45.00 MB 2023-10-10T20:23:22.898 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.898 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.898 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 46.00 MB 2023-10-10T20:23:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 47.00 MB 2023-10-10T20:23:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 48.00 MB 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 49.00 MB 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.900 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.901 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 50.00 MB 2023-10-10T20:23:22.901 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.901 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.901 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 51.00 MB 2023-10-10T20:23:22.901 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.902 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.902 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 52.00 MB 2023-10-10T20:23:22.902 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 53.00 MB 2023-10-10T20:23:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.903 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 54.00 MB 2023-10-10T20:23:22.904 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.904 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.904 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 55.00 MB 2023-10-10T20:23:22.904 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.904 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.905 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 56.00 MB 2023-10-10T20:23:22.905 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.905 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.905 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 57.00 MB 2023-10-10T20:23:22.905 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.906 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.906 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 58.00 MB 2023-10-10T20:23:22.906 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.906 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.906 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 59.00 MB 2023-10-10T20:23:22.907 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.907 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.907 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 60.00 MB 2023-10-10T20:23:22.907 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.907 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.908 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 61.00 MB 2023-10-10T20:23:22.908 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.908 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.908 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 62.00 MB 2023-10-10T20:23:22.908 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.909 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 63.00 MB 2023-10-10T20:23:22.909 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.909 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 64.00 MB 2023-10-10T20:23:22.910 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.910 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.910 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 65.00 MB 2023-10-10T20:23:22.910 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.911 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.911 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 66.00 MB 2023-10-10T20:23:22.911 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.911 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.911 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 67.00 MB 2023-10-10T20:23:22.912 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.912 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.912 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 68.00 MB 2023-10-10T20:23:22.912 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.912 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.913 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 69.00 MB 2023-10-10T20:23:22.913 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.913 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.914 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 70.00 MB 2023-10-10T20:23:22.914 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.914 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.915 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 71.00 MB 2023-10-10T20:23:22.915 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.915 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.916 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 72.00 MB 2023-10-10T20:23:22.916 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.916 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.917 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 73.00 MB 2023-10-10T20:23:22.917 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.918 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.918 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 74.00 MB 2023-10-10T20:23:22.918 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.919 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.919 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 75.00 MB 2023-10-10T20:23:22.919 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.920 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.920 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 76.00 MB 2023-10-10T20:23:22.920 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.921 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.921 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 77.00 MB 2023-10-10T20:23:22.921 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.922 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.922 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 78.00 MB 2023-10-10T20:23:22.923 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.923 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.923 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 79.00 MB 2023-10-10T20:23:22.924 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:22.924 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:22.924 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 80.00 MB 2023-10-10T20:23:23.069 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.070 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.070 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 81.00 MB 2023-10-10T20:23:23.070 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.070 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.070 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 82.00 MB 2023-10-10T20:23:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 83.00 MB 2023-10-10T20:23:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 84.00 MB 2023-10-10T20:23:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 85.00 MB 2023-10-10T20:23:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.073 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.073 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 86.00 MB 2023-10-10T20:23:23.073 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.073 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.073 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 87.00 MB 2023-10-10T20:23:23.074 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.074 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.074 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 88.00 MB 2023-10-10T20:23:23.074 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.074 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.075 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 89.00 MB 2023-10-10T20:23:23.075 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.075 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.075 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 90.00 MB 2023-10-10T20:23:23.075 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 91.00 MB 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 92.00 MB 2023-10-10T20:23:23.076 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.077 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.077 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 93.00 MB 2023-10-10T20:23:23.077 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.077 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.077 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 94.00 MB 2023-10-10T20:23:23.078 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.078 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.078 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 95.00 MB 2023-10-10T20:23:23.078 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.079 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.079 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 96.00 MB 2023-10-10T20:23:23.079 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.079 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.079 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 97.00 MB 2023-10-10T20:23:23.080 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.080 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.080 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 98.00 MB 2023-10-10T20:23:23.080 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.080 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 99.00 MB 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 100.00 MB 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.081 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.082 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 101.00 MB 2023-10-10T20:23:23.082 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.082 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.082 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 102.00 MB 2023-10-10T20:23:23.083 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.083 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.083 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 103.00 MB 2023-10-10T20:23:23.083 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.083 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 104.00 MB 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 105.00 MB 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.084 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.085 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 106.00 MB 2023-10-10T20:23:23.085 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.085 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.085 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 107.00 MB 2023-10-10T20:23:23.085 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.086 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.086 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 108.00 MB 2023-10-10T20:23:23.086 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.086 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.086 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 109.00 MB 2023-10-10T20:23:23.087 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.087 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.087 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 110.00 MB 2023-10-10T20:23:23.087 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.087 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.088 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 111.00 MB 2023-10-10T20:23:23.088 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.088 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.088 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 112.00 MB 2023-10-10T20:23:23.088 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.089 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.089 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 113.00 MB 2023-10-10T20:23:23.089 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.089 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.089 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 114.00 MB 2023-10-10T20:23:23.090 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.090 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.090 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 115.00 MB 2023-10-10T20:23:23.090 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.090 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.091 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 116.00 MB 2023-10-10T20:23:23.091 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.091 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.091 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 117.00 MB 2023-10-10T20:23:23.091 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.092 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.092 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 118.00 MB 2023-10-10T20:23:23.092 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.093 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.093 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 119.00 MB 2023-10-10T20:23:23.093 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.093 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 120.00 MB 2023-10-10T20:23:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 121.00 MB 2023-10-10T20:23:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.095 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.095 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 122.00 MB 2023-10-10T20:23:23.095 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.095 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.095 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 123.00 MB 2023-10-10T20:23:23.096 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.096 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.096 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 124.00 MB 2023-10-10T20:23:23.096 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.096 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.097 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 125.00 MB 2023-10-10T20:23:23.097 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.097 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.097 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 126.00 MB 2023-10-10T20:23:23.097 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.098 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.098 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 127.00 MB 2023-10-10T20:23:23.098 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.098 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.098 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 128.00 MB 2023-10-10T20:23:23.099 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.099 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.099 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 129.00 MB 2023-10-10T20:23:23.099 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.100 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.100 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 130.00 MB 2023-10-10T20:23:23.100 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.100 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.101 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 131.00 MB 2023-10-10T20:23:23.101 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.101 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.102 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 132.00 MB 2023-10-10T20:23:23.102 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.102 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.103 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 133.00 MB 2023-10-10T20:23:23.103 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.103 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.104 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 134.00 MB 2023-10-10T20:23:23.104 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.104 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.105 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 135.00 MB 2023-10-10T20:23:23.105 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.105 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.106 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 136.00 MB 2023-10-10T20:23:23.106 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.106 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.107 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 137.00 MB 2023-10-10T20:23:23.107 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.108 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.108 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 138.00 MB 2023-10-10T20:23:23.108 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.109 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.109 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 139.00 MB 2023-10-10T20:23:23.109 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.109 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.110 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 140.00 MB 2023-10-10T20:23:23.110 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.110 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.110 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 141.00 MB 2023-10-10T20:23:23.110 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.111 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.111 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 142.00 MB 2023-10-10T20:23:23.111 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.111 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 143.00 MB 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 144.00 MB 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.112 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.113 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 145.00 MB 2023-10-10T20:23:23.113 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.113 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.113 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 146.00 MB 2023-10-10T20:23:23.113 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.114 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.114 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 147.00 MB 2023-10-10T20:23:23.114 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.114 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.115 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 148.00 MB 2023-10-10T20:23:23.115 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.115 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.115 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 149.00 MB 2023-10-10T20:23:23.116 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.116 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.116 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 150.00 MB 2023-10-10T20:23:23.117 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.117 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.117 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 151.00 MB 2023-10-10T20:23:23.118 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.118 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.119 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 152.00 MB 2023-10-10T20:23:23.119 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.119 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.120 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 153.00 MB 2023-10-10T20:23:23.120 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.120 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.121 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 154.00 MB 2023-10-10T20:23:23.121 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.121 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.121 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 155.00 MB 2023-10-10T20:23:23.122 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.122 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.122 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 156.00 MB 2023-10-10T20:23:23.123 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.123 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.124 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 157.00 MB 2023-10-10T20:23:23.124 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.124 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.125 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 158.00 MB 2023-10-10T20:23:23.125 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.125 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.126 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 159.00 MB 2023-10-10T20:23:23.126 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.126 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.126 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 160.00 MB 2023-10-10T20:23:23.127 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.127 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.127 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 161.00 MB 2023-10-10T20:23:23.266 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.266 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.267 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 162.00 MB 2023-10-10T20:23:23.267 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.267 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.268 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 163.00 MB 2023-10-10T20:23:23.268 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.268 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.268 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 164.00 MB 2023-10-10T20:23:23.269 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.269 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.269 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 165.00 MB 2023-10-10T20:23:23.269 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.270 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.270 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 166.00 MB 2023-10-10T20:23:23.270 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.271 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.271 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 167.00 MB 2023-10-10T20:23:23.271 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.271 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.271 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 168.00 MB 2023-10-10T20:23:23.272 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.272 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.272 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 169.00 MB 2023-10-10T20:23:23.272 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.272 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.273 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 170.00 MB 2023-10-10T20:23:23.273 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.273 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.273 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 171.00 MB 2023-10-10T20:23:23.273 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.274 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.274 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 172.00 MB 2023-10-10T20:23:23.274 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.274 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.274 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 173.00 MB 2023-10-10T20:23:23.275 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.275 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.275 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 174.00 MB 2023-10-10T20:23:23.275 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.275 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.276 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 175.00 MB 2023-10-10T20:23:23.276 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.276 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.276 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 176.00 MB 2023-10-10T20:23:23.276 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.277 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.277 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 177.00 MB 2023-10-10T20:23:23.277 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.277 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.278 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 178.00 MB 2023-10-10T20:23:23.278 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.278 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.279 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 179.00 MB 2023-10-10T20:23:23.279 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.279 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.280 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 180.00 MB 2023-10-10T20:23:23.280 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.281 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.281 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 181.00 MB 2023-10-10T20:23:23.281 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.282 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.282 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 182.00 MB 2023-10-10T20:23:23.282 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.283 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.283 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 183.00 MB 2023-10-10T20:23:23.283 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.284 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.284 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 184.00 MB 2023-10-10T20:23:23.284 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.285 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.285 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 185.00 MB 2023-10-10T20:23:23.285 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.286 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.286 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 186.00 MB 2023-10-10T20:23:23.286 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.287 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.287 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 187.00 MB 2023-10-10T20:23:23.287 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.287 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.287 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 188.00 MB 2023-10-10T20:23:23.288 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.288 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.288 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 189.00 MB 2023-10-10T20:23:23.288 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.288 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.289 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 190.00 MB 2023-10-10T20:23:23.289 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.289 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.289 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 191.00 MB 2023-10-10T20:23:23.289 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.290 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.290 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 192.00 MB 2023-10-10T20:23:23.290 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.290 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.290 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 193.00 MB 2023-10-10T20:23:23.291 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.291 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.291 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 194.00 MB 2023-10-10T20:23:23.291 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.291 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.292 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 195.00 MB 2023-10-10T20:23:23.292 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.292 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.292 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 196.00 MB 2023-10-10T20:23:23.292 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 197.00 MB 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 198.00 MB 2023-10-10T20:23:23.293 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.294 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.294 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 199.00 MB 2023-10-10T20:23:23.294 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.294 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.294 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 200.00 MB 2023-10-10T20:23:23.295 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.295 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.295 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 201.00 MB 2023-10-10T20:23:23.295 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.295 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.296 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 202.00 MB 2023-10-10T20:23:23.296 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.296 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.296 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 203.00 MB 2023-10-10T20:23:23.296 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.297 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.297 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 204.00 MB 2023-10-10T20:23:23.297 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.297 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.297 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 205.00 MB 2023-10-10T20:23:23.298 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.298 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.298 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 206.00 MB 2023-10-10T20:23:23.298 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.298 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.299 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 207.00 MB 2023-10-10T20:23:23.299 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.299 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.299 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 208.00 MB 2023-10-10T20:23:23.299 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 209.00 MB 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 210.00 MB 2023-10-10T20:23:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.301 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.301 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 211.00 MB 2023-10-10T20:23:23.301 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.301 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.301 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 212.00 MB 2023-10-10T20:23:23.302 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.302 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.302 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 213.00 MB 2023-10-10T20:23:23.302 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.302 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.303 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 214.00 MB 2023-10-10T20:23:23.303 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.303 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.303 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 215.00 MB 2023-10-10T20:23:23.303 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.304 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.304 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 216.00 MB 2023-10-10T20:23:23.304 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.304 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.304 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 217.00 MB 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 218.00 MB 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.305 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 219.00 MB 2023-10-10T20:23:23.306 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.306 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.306 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 220.00 MB 2023-10-10T20:23:23.306 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.306 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.307 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 221.00 MB 2023-10-10T20:23:23.307 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.307 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.307 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 222.00 MB 2023-10-10T20:23:23.307 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.308 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.308 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 223.00 MB 2023-10-10T20:23:23.308 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.308 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.308 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 224.00 MB 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 225.00 MB 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.309 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 226.00 MB 2023-10-10T20:23:23.310 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.310 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.310 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 227.00 MB 2023-10-10T20:23:23.310 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.310 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.311 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 228.00 MB 2023-10-10T20:23:23.311 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.311 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.311 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 229.00 MB 2023-10-10T20:23:23.311 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.312 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.312 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 230.00 MB 2023-10-10T20:23:23.312 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.312 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.312 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 231.00 MB 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 232.00 MB 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.313 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 233.00 MB 2023-10-10T20:23:23.314 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.314 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.314 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 234.00 MB 2023-10-10T20:23:23.314 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.314 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.315 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 235.00 MB 2023-10-10T20:23:23.315 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.315 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.315 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 236.00 MB 2023-10-10T20:23:23.315 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.316 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.316 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 237.00 MB 2023-10-10T20:23:23.316 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.316 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.316 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 238.00 MB 2023-10-10T20:23:23.317 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.317 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.317 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 239.00 MB 2023-10-10T20:23:23.317 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.317 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.318 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 240.00 MB 2023-10-10T20:23:23.318 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.318 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.318 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 241.00 MB 2023-10-10T20:23:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 242.00 MB 2023-10-10T20:23:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 243.00 MB 2023-10-10T20:23:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 244.00 MB 2023-10-10T20:23:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.462 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.462 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 245.00 MB 2023-10-10T20:23:23.462 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.462 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.463 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 246.00 MB 2023-10-10T20:23:23.463 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.463 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.463 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 247.00 MB 2023-10-10T20:23:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 248.00 MB 2023-10-10T20:23:23.464 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 249.00 MB 2023-10-10T20:23:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.465 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 250.00 MB 2023-10-10T20:23:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 251.00 MB 2023-10-10T20:23:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.466 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 252.00 MB 2023-10-10T20:23:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 253.00 MB 2023-10-10T20:23:23.467 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 254.00 MB 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 255.00 MB 2023-10-10T20:23:23.468 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 256.00 MB 2023-10-10T20:23:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.469 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 257.00 MB 2023-10-10T20:23:23.470 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.470 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.470 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 258.00 MB 2023-10-10T20:23:23.470 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.470 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 259.00 MB 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 260.00 MB 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.471 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 261.00 MB 2023-10-10T20:23:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 262.00 MB 2023-10-10T20:23:23.472 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 263.00 MB 2023-10-10T20:23:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.473 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 264.00 MB 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 265.00 MB 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.474 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 266.00 MB 2023-10-10T20:23:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 267.00 MB 2023-10-10T20:23:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.475 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.476 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 268.00 MB 2023-10-10T20:23:23.476 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.476 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.476 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 269.00 MB 2023-10-10T20:23:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 270.00 MB 2023-10-10T20:23:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.477 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.478 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 271.00 MB 2023-10-10T20:23:23.478 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.478 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.478 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 272.00 MB 2023-10-10T20:23:23.478 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 273.00 MB 2023-10-10T20:23:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.479 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 274.00 MB 2023-10-10T20:23:23.480 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.480 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.480 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 275.00 MB 2023-10-10T20:23:23.480 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.481 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.482 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 276.00 MB 2023-10-10T20:23:23.483 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.483 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.483 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 277.00 MB 2023-10-10T20:23:23.483 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.483 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.484 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 278.00 MB 2023-10-10T20:23:23.484 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.484 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.484 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 279.00 MB 2023-10-10T20:23:23.484 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.485 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.485 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 280.00 MB 2023-10-10T20:23:23.485 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.485 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.485 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 281.00 MB 2023-10-10T20:23:23.486 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.486 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.486 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 282.00 MB 2023-10-10T20:23:23.486 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.486 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.487 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 283.00 MB 2023-10-10T20:23:23.487 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.487 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.487 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 284.00 MB 2023-10-10T20:23:23.487 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 285.00 MB 2023-10-10T20:23:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.488 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 286.00 MB 2023-10-10T20:23:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 287.00 MB 2023-10-10T20:23:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.489 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.490 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 288.00 MB 2023-10-10T20:23:23.490 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.490 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.490 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 289.00 MB 2023-10-10T20:23:23.490 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.491 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.491 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 290.00 MB 2023-10-10T20:23:23.491 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.491 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.491 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 291.00 MB 2023-10-10T20:23:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 292.00 MB 2023-10-10T20:23:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.492 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.493 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 293.00 MB 2023-10-10T20:23:23.493 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.493 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.493 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 294.00 MB 2023-10-10T20:23:23.493 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.494 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.494 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 295.00 MB 2023-10-10T20:23:23.494 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.494 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.494 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 296.00 MB 2023-10-10T20:23:23.495 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.495 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.495 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 297.00 MB 2023-10-10T20:23:23.495 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.495 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.496 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 298.00 MB 2023-10-10T20:23:23.496 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.496 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.496 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 299.00 MB 2023-10-10T20:23:23.497 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.497 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.497 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 300.00 MB 2023-10-10T20:23:23.497 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.497 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 301.00 MB 2023-10-10T20:23:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 302.00 MB 2023-10-10T20:23:23.498 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.499 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.499 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 303.00 MB 2023-10-10T20:23:23.499 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.499 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.499 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 304.00 MB 2023-10-10T20:23:23.500 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.500 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.500 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 305.00 MB 2023-10-10T20:23:23.500 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.500 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.501 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 306.00 MB 2023-10-10T20:23:23.501 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.501 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.501 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 307.00 MB 2023-10-10T20:23:23.501 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.502 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.502 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 308.00 MB 2023-10-10T20:23:23.502 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.502 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.502 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 309.00 MB 2023-10-10T20:23:23.503 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.503 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.503 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 310.00 MB 2023-10-10T20:23:23.503 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.503 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.504 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 311.00 MB 2023-10-10T20:23:23.504 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.504 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.504 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 312.00 MB 2023-10-10T20:23:23.504 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.505 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.505 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 313.00 MB 2023-10-10T20:23:23.505 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.505 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.506 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 314.00 MB 2023-10-10T20:23:23.506 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.506 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.506 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 315.00 MB 2023-10-10T20:23:23.507 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.507 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.507 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 316.00 MB 2023-10-10T20:23:23.507 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.508 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.508 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 317.00 MB 2023-10-10T20:23:23.508 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.508 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.509 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 318.00 MB 2023-10-10T20:23:23.509 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.509 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.509 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 319.00 MB 2023-10-10T20:23:23.509 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.510 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.510 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 320.00 MB 2023-10-10T20:23:23.510 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.510 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.510 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 321.00 MB 2023-10-10T20:23:23.511 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.511 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.658 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 322.00 MB 2023-10-10T20:23:23.658 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.658 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.658 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 323.00 MB 2023-10-10T20:23:23.659 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.659 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.659 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 324.00 MB 2023-10-10T20:23:23.659 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.659 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.660 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 325.00 MB 2023-10-10T20:23:23.660 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.660 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.660 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 326.00 MB 2023-10-10T20:23:23.660 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 327.00 MB 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 328.00 MB 2023-10-10T20:23:23.661 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 329.00 MB 2023-10-10T20:23:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 330.00 MB 2023-10-10T20:23:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 331.00 MB 2023-10-10T20:23:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.663 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.664 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 332.00 MB 2023-10-10T20:23:23.664 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.664 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.664 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 333.00 MB 2023-10-10T20:23:23.664 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 334.00 MB 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 335.00 MB 2023-10-10T20:23:23.665 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.666 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.666 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 336.00 MB 2023-10-10T20:23:23.666 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.666 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.666 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 337.00 MB 2023-10-10T20:23:23.667 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.667 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.667 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 338.00 MB 2023-10-10T20:23:23.667 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.667 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.668 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 339.00 MB 2023-10-10T20:23:23.668 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.668 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.668 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 340.00 MB 2023-10-10T20:23:23.668 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.669 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.669 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 341.00 MB 2023-10-10T20:23:23.669 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.669 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.669 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 342.00 MB 2023-10-10T20:23:23.670 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.670 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.670 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 343.00 MB 2023-10-10T20:23:23.670 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.670 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.671 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 344.00 MB 2023-10-10T20:23:23.671 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.671 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.671 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 345.00 MB 2023-10-10T20:23:23.671 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.672 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.672 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 346.00 MB 2023-10-10T20:23:23.672 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.672 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.672 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 347.00 MB 2023-10-10T20:23:23.673 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.673 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.673 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 348.00 MB 2023-10-10T20:23:23.673 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.673 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.674 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 349.00 MB 2023-10-10T20:23:23.674 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.674 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.674 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 350.00 MB 2023-10-10T20:23:23.674 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.675 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.675 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 351.00 MB 2023-10-10T20:23:23.675 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.675 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.675 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 352.00 MB 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 353.00 MB 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.676 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 354.00 MB 2023-10-10T20:23:23.677 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.677 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.677 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 355.00 MB 2023-10-10T20:23:23.677 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.677 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.678 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 356.00 MB 2023-10-10T20:23:23.678 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.678 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.678 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 357.00 MB 2023-10-10T20:23:23.678 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 358.00 MB 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 359.00 MB 2023-10-10T20:23:23.679 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.680 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.680 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 360.00 MB 2023-10-10T20:23:23.680 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.680 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.680 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 361.00 MB 2023-10-10T20:23:23.681 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.681 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.681 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 362.00 MB 2023-10-10T20:23:23.681 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.681 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.682 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 363.00 MB 2023-10-10T20:23:23.682 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.682 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.683 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 364.00 MB 2023-10-10T20:23:23.683 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.683 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.683 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 365.00 MB 2023-10-10T20:23:23.683 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.684 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.684 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 366.00 MB 2023-10-10T20:23:23.684 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.684 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.684 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 367.00 MB 2023-10-10T20:23:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 368.00 MB 2023-10-10T20:23:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.685 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.686 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 369.00 MB 2023-10-10T20:23:23.686 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.686 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.686 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 370.00 MB 2023-10-10T20:23:23.686 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.687 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.687 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 371.00 MB 2023-10-10T20:23:23.687 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.687 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.687 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 372.00 MB 2023-10-10T20:23:23.688 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.688 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.688 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 373.00 MB 2023-10-10T20:23:23.688 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.688 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.689 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 374.00 MB 2023-10-10T20:23:23.689 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.689 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.689 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 375.00 MB 2023-10-10T20:23:23.690 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.690 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.690 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 376.00 MB 2023-10-10T20:23:23.690 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.691 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.691 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 377.00 MB 2023-10-10T20:23:23.691 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.691 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.691 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 378.00 MB 2023-10-10T20:23:23.692 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.692 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.692 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 379.00 MB 2023-10-10T20:23:23.692 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.692 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.693 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 380.00 MB 2023-10-10T20:23:23.693 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.693 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.693 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 381.00 MB 2023-10-10T20:23:23.693 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.694 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.694 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 382.00 MB 2023-10-10T20:23:23.694 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.694 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.694 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 383.00 MB 2023-10-10T20:23:23.695 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.695 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.695 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 384.00 MB 2023-10-10T20:23:23.695 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.695 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 385.00 MB 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 386.00 MB 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.696 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 387.00 MB 2023-10-10T20:23:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 388.00 MB 2023-10-10T20:23:23.697 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.698 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.698 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 389.00 MB 2023-10-10T20:23:23.698 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.698 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.698 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 390.00 MB 2023-10-10T20:23:23.699 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.699 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.699 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 391.00 MB 2023-10-10T20:23:23.699 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.699 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.700 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 392.00 MB 2023-10-10T20:23:23.700 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.700 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.700 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 393.00 MB 2023-10-10T20:23:23.700 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 394.00 MB 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 395.00 MB 2023-10-10T20:23:23.701 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.702 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.702 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 396.00 MB 2023-10-10T20:23:23.702 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.702 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.702 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 397.00 MB 2023-10-10T20:23:23.703 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.703 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.703 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 398.00 MB 2023-10-10T20:23:23.703 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.703 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.704 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 399.00 MB 2023-10-10T20:23:23.704 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.704 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.704 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 400.00 MB 2023-10-10T20:23:23.704 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.705 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.705 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 401.00 MB 2023-10-10T20:23:23.705 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:23.706 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:23.706 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 402.00 MB 2023-10-10T20:23:25.675 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:25.673+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-10-10T20:23:35.508 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:35.506+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-10T20:23:52.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:23:52.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:23:53.901 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.902 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.902 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 403.00 MB 2023-10-10T20:23:53.902 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.903 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.903 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 404.00 MB 2023-10-10T20:23:53.903 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.903 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 405.00 MB 2023-10-10T20:23:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 406.00 MB 2023-10-10T20:23:53.904 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.905 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.905 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 407.00 MB 2023-10-10T20:23:53.905 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.905 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.906 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 408.00 MB 2023-10-10T20:23:53.906 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.906 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.906 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 409.00 MB 2023-10-10T20:23:53.906 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.907 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.907 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 410.00 MB 2023-10-10T20:23:53.907 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.907 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 411.00 MB 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 412.00 MB 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.908 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 413.00 MB 2023-10-10T20:23:53.909 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.909 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.909 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 414.00 MB 2023-10-10T20:23:53.910 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.910 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.910 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 415.00 MB 2023-10-10T20:23:53.910 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.910 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.911 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 416.00 MB 2023-10-10T20:23:53.911 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.911 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.912 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 417.00 MB 2023-10-10T20:23:53.912 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.912 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.912 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 418.00 MB 2023-10-10T20:23:53.913 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.913 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.913 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 419.00 MB 2023-10-10T20:23:53.913 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.913 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.914 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 420.00 MB 2023-10-10T20:23:53.914 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.914 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.914 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 421.00 MB 2023-10-10T20:23:53.915 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.915 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.915 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 422.00 MB 2023-10-10T20:23:53.915 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.915 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.916 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 423.00 MB 2023-10-10T20:23:53.916 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.916 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.916 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 424.00 MB 2023-10-10T20:23:53.916 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.917 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.918 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 425.00 MB 2023-10-10T20:23:53.918 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.918 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.918 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 426.00 MB 2023-10-10T20:23:53.919 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.919 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.919 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 427.00 MB 2023-10-10T20:23:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 428.00 MB 2023-10-10T20:23:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.920 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.921 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 429.00 MB 2023-10-10T20:23:53.921 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.921 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.921 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 430.00 MB 2023-10-10T20:23:53.921 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 431.00 MB 2023-10-10T20:23:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.922 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 432.00 MB 2023-10-10T20:23:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 433.00 MB 2023-10-10T20:23:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.923 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.924 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 434.00 MB 2023-10-10T20:23:53.924 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.924 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.924 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 435.00 MB 2023-10-10T20:23:53.924 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.925 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.925 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 436.00 MB 2023-10-10T20:23:53.925 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.925 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 437.00 MB 2023-10-10T20:23:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 438.00 MB 2023-10-10T20:23:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 439.00 MB 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 440.00 MB 2023-10-10T20:23:53.927 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.928 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.928 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 441.00 MB 2023-10-10T20:23:53.928 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.928 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.928 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 442.00 MB 2023-10-10T20:23:53.929 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.929 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.929 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 443.00 MB 2023-10-10T20:23:53.929 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.930 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.930 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 444.00 MB 2023-10-10T20:23:53.930 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.930 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 445.00 MB 2023-10-10T20:23:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 446.00 MB 2023-10-10T20:23:53.931 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 447.00 MB 2023-10-10T20:23:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 448.00 MB 2023-10-10T20:23:53.933 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.933 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.933 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 449.00 MB 2023-10-10T20:23:53.933 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 450.00 MB 2023-10-10T20:23:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.934 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 451.00 MB 2023-10-10T20:23:53.935 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.935 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.935 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 452.00 MB 2023-10-10T20:23:53.935 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.936 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.936 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 453.00 MB 2023-10-10T20:23:53.936 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.936 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.937 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 454.00 MB 2023-10-10T20:23:53.937 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.937 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.937 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 455.00 MB 2023-10-10T20:23:53.937 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.938 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.938 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 456.00 MB 2023-10-10T20:23:53.938 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.938 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.938 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 457.00 MB 2023-10-10T20:23:53.939 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.939 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.939 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 458.00 MB 2023-10-10T20:23:53.939 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.939 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.940 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 459.00 MB 2023-10-10T20:23:53.940 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.940 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.940 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 460.00 MB 2023-10-10T20:23:53.940 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.941 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.941 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 461.00 MB 2023-10-10T20:23:53.941 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.942 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.943 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 462.00 MB 2023-10-10T20:23:53.943 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.943 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.943 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 463.00 MB 2023-10-10T20:23:53.943 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 464.00 MB 2023-10-10T20:23:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.944 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 465.00 MB 2023-10-10T20:23:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 466.00 MB 2023-10-10T20:23:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.945 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.946 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 467.00 MB 2023-10-10T20:23:53.946 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.946 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.946 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 468.00 MB 2023-10-10T20:23:53.946 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.947 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.947 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 469.00 MB 2023-10-10T20:23:53.947 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.948 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.948 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 470.00 MB 2023-10-10T20:23:53.948 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.948 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.949 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 471.00 MB 2023-10-10T20:23:53.949 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.949 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.949 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 472.00 MB 2023-10-10T20:23:53.950 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.950 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.950 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 473.00 MB 2023-10-10T20:23:53.951 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.951 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.951 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 474.00 MB 2023-10-10T20:23:53.952 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.952 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.952 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 475.00 MB 2023-10-10T20:23:53.953 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.953 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.953 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 476.00 MB 2023-10-10T20:23:53.954 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.954 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.955 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 477.00 MB 2023-10-10T20:23:53.955 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.955 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.956 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 478.00 MB 2023-10-10T20:23:53.956 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.957 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.957 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 479.00 MB 2023-10-10T20:23:53.957 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:23:53.957 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:23:53.958 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:23:53.958 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 480.00 MB 2023-10-10T20:23:53.958 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:23:53.958 INFO:teuthology.orchestra.run.smithi148.stdout:Reached fullness after 480.00 MB 2023-10-10T20:23:53.958 INFO:teuthology.orchestra.run.smithi148.stdout:calling close 2023-10-10T20:23:53.959 INFO:teuthology.orchestra.run.smithi148.stdout:close() did not raise error 2023-10-10T20:23:54.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:23:54.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:23:54.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:23:54.117 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:23:54.118 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:23:54.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:23:54.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:23:54.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:23:54.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:54.160 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:23:54.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:23:54.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:23:54.218 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:23:54.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:23:54.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:23:54.266 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:23:54.282 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.396+0000 7f48b687a700 1 Processor -- start 2023-10-10T20:23:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.396+0000 7f48b687a700 1 -- start start 2023-10-10T20:23:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b687a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 0x7f48b015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b687a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48b015a620 0x7f48b015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b687a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b015cbb0 con 0x7f48b015d580 2023-10-10T20:23:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b687a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b015bfa0 con 0x7f48b015a620 2023-10-10T20:23:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b687a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b015c280 con 0x7f48b0157870 2023-10-10T20:23:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48af7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48b015a620 0x7f48b015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b4e17700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f48b015d580 0x7f48b0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:49996/0 (socket says 172.21.15.148:49996) 2023-10-10T20:23:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48b4e17700 1 -- 172.21.15.148:0/3931956048 learned_addr learned my addr 172.21.15.148:0/3931956048 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.397+0000 7f48affff700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 0x7f48b015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48aeffd700 1 -- 172.21.15.148:0/3931956048 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598556233 0 0) 0x7f48b015cbb0 con 0x7f48b015d580 2023-10-10T20:23:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48aeffd700 1 -- 172.21.15.148:0/3931956048 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4898003570 con 0x7f48b015d580 2023-10-10T20:23:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48af7fe700 1 -- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 msgr2=0x7f48b015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48af7fe700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 0x7f48b015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48af7fe700 1 -- 172.21.15.148:0/3931956048 >> v1:172.21.15.161:6789/0 conn(0x7f48b015d580 legacy=0x7f48b0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48af7fe700 1 -- 172.21.15.148:0/3931956048 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b015c560 con 0x7f48b015a620 2023-10-10T20:23:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48affff700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 0x7f48b015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48af7fe700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b015d040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f48a40098e0 tx=0x7f48a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a59a9d4e011302 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48aeffd700 1 -- 172.21.15.148:0/3931956048 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011630 con 0x7f48b015a620 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48aeffd700 1 -- 172.21.15.148:0/3931956048 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4011790 con 0x7f48b015a620 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.398+0000 7f48aeffd700 1 -- 172.21.15.148:0/3931956048 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011b30 con 0x7f48b015a620 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 -- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 msgr2=0x7f48b015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b015d040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f48a40098e0 tx=0x7f48a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 -- 172.21.15.148:0/3931956048 shutdown_connections 2023-10-10T20:23:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b0157870 0x7f48b015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 --2- 172.21.15.148:0/3931956048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b015d040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 -- 172.21.15.148:0/3931956048 >> 172.21.15.148:0/3931956048 conn(0x7f48b0068ce0 msgr2=0x7f48b00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 -- 172.21.15.148:0/3931956048 shutdown_connections 2023-10-10T20:23:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.399+0000 7f48b687a700 1 -- 172.21.15.148:0/3931956048 wait complete. 2023-10-10T20:23:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.400+0000 7f48b687a700 1 Processor -- start 2023-10-10T20:23:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.400+0000 7f48b687a700 1 -- start start 2023-10-10T20:23:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b00d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b015d580 0x7f48b00b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b1740 con 0x7f48b0157870 2023-10-10T20:23:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01511c0 con 0x7f48b015a620 2023-10-10T20:23:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b687a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b0b30 con 0x7f48b015d580 2023-10-10T20:23:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43352/0 (socket says 172.21.15.148:43352) 2023-10-10T20:23:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 -- 172.21.15.148:0/3040975098 learned_addr learned my addr 172.21.15.148:0/3040975098 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48af7fe700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b00d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48b4e17700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b015d580 0x7f48b00b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 -- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b015d580 msgr2=0x7f48b00b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b015d580 0x7f48b00b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 -- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 msgr2=0x7f48b00d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b00d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0009420 con 0x7f48b0157870 2023-10-10T20:23:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f48affff700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f48a000ec80 tx=0x7f48a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000c850 con 0x7f48b0157870 2023-10-10T20:23:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.401+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000c9b0 con 0x7f48b0157870 2023-10-10T20:23:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.402+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a4009420 con 0x7f48b0157870 2023-10-10T20:23:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.402+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000cd30 con 0x7f48b0157870 2023-10-10T20:23:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.402+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b00b10f0 con 0x7f48b0157870 2023-10-10T20:23:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.402+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b004c3c0 con 0x7f48b0157870 2023-10-10T20:23:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.403+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48a0013680 con 0x7f48b0157870 2023-10-10T20:23:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.403+0000 7f4897fff700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 0x7f489c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.403+0000 7f48af7fe700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 0x7f489c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.403+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48a0084710 con 0x7f48b0157870 2023-10-10T20:23:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.404+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f489c06ea50 con 0x7f48b0157870 2023-10-10T20:23:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.404+0000 7f48af7fe700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 0x7f489c0693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48a40091e0 tx=0x7f48a4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.408+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48a0051980 con 0x7f48b0157870 2023-10-10T20:23:54.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.632+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b015c840 con 0x7f48b0157870 2023-10-10T20:23:54.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.633+0000 7f4897fff700 1 -- 172.21.15.148:0/3040975098 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f48a0017090 con 0x7f48b0157870 2023-10-10T20:23:54.635 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:54.636 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:54.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 msgr2=0x7f489c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 0x7f489c0693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48a40091e0 tx=0x7f48a4017000 comp rx=0 tx=0).stop 2023-10-10T20:23:54.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 msgr2=0x7f48b0150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f48a000ec80 tx=0x7f48a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 shutdown_connections 2023-10-10T20:23:54.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f489c066f30 0x7f489c0693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48b015d580 0x7f48b00b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0157870 0x7f48b0150970 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 --2- 172.21.15.148:0/3040975098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b015a620 0x7f48b00d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 >> 172.21.15.148:0/3040975098 conn(0x7f48b0068ce0 msgr2=0x7f48b015e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:54.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 shutdown_connections 2023-10-10T20:23:54.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.635+0000 7f48b687a700 1 -- 172.21.15.148:0/3040975098 wait complete. 2023-10-10T20:23:54.640 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:54.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:54.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.810+0000 7fc4c12af700 1 Processor -- start 2023-10-10T20:23:54.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.810+0000 7fc4c12af700 1 -- start start 2023-10-10T20:23:54.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c12af700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc4bc0cdab0 0x7fc4bc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c12af700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc0cb7c0 con 0x7fc4bc15dda0 2023-10-10T20:23:54.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c12af700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc15e290 con 0x7fc4bc0cd6e0 2023-10-10T20:23:54.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c12af700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc15e850 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c0aae700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc4bc15dda0 0x7fc4bc15cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60972/0 (socket says 172.21.15.148:60972) 2023-10-10T20:23:54.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4c0aae700 1 -- 172.21.15.148:0/920188428 learned_addr learned my addr 172.21.15.148:0/920188428 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:54.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4bb7fe700 1 --2- 172.21.15.148:0/920188428 >> v2:172.21.15.161:3300/0 conn(0x7fc4bc0cdab0 0x7fc4bc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.811+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461929644 0 0) 0x7fc4bc0cb7c0 con 0x7fc4bc15dda0 2023-10-10T20:23:54.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4003570 con 0x7fc4bc15dda0 2023-10-10T20:23:54.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361695386 0 0) 0x7fc4bc15e290 con 0x7fc4bc0cd6e0 2023-10-10T20:23:54.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4bc0cb7c0 con 0x7fc4bc0cd6e0 2023-10-10T20:23:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/920188428 >> v1:172.21.15.161:6789/0 conn(0x7fc4bc0cd6e0 legacy=0x7fc4bc0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/920188428 >> v1:172.21.15.148:6789/0 conn(0x7fc4bc15dda0 legacy=0x7fc4bc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/920188428 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc15e570 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4bb7fe700 1 --2- 172.21.15.148:0/920188428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc15d860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098e0 tx=0x7fc4b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5709d26531eadf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0011630 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0011790 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4baffd700 1 -- 172.21.15.148:0/920188428 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0011b30 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4c12af700 1 -- 172.21.15.148:0/920188428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc0cdab0 msgr2=0x7fc4bc15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.812+0000 7fc4c12af700 1 --2- 172.21.15.148:0/920188428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc15d860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098e0 tx=0x7fc4b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.813+0000 7fc4c12af700 1 -- 172.21.15.148:0/920188428 shutdown_connections 2023-10-10T20:23:54.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.813+0000 7fc4c12af700 1 --2- 172.21.15.148:0/920188428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc15d860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.813+0000 7fc4c12af700 1 -- 172.21.15.148:0/920188428 >> 172.21.15.148:0/920188428 conn(0x7fc4bc067660 msgr2=0x7fc4bc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:54.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.813+0000 7fc4c12af700 1 -- 172.21.15.148:0/920188428 shutdown_connections 2023-10-10T20:23:54.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.813+0000 7fc4c12af700 1 -- 172.21.15.148:0/920188428 wait complete. 2023-10-10T20:23:54.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 Processor -- start 2023-10-10T20:23:54.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 -- start start 2023-10-10T20:23:54.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc4bc0cd6e0 0x7fc4bc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 0x7fc4bc15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0cc360 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0c7db0 con 0x7fc4bc15dda0 2023-10-10T20:23:54.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.814+0000 7fc4c12af700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc16ac10 con 0x7fc4bc0cd6e0 2023-10-10T20:23:54.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43354/0 (socket says 172.21.15.148:43354) 2023-10-10T20:23:54.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/87202063 learned_addr learned my addr 172.21.15.148:0/87202063 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:54.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4c0aae700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 0x7fc4bc15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bbfff700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc4bc0cd6e0 0x7fc4bc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc4bc0cd6e0 msgr2=0x7fc4bc0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc4bc0cd6e0 0x7fc4bc0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 msgr2=0x7fc4bc15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:54.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 0x7fc4bc15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:54.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0009420 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4c0aae700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 0x7fc4bc15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:54.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4bb7fe700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00093f0 tx=0x7fc4b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001f070 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0002f70 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0011900 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc16aef0 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.815+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc16b4b0 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.816+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc0c7f40 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4b0023070 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4a3fff700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 0x7fc4a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:54.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4b0085670 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4bbfff700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 0x7fc4a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:54.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7fc4a806ea50 con 0x7fc4bc0cdab0 2023-10-10T20:23:54.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.817+0000 7fc4bbfff700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 0x7fc4a80693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:54.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:54.821+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4b0015020 con 0x7fc4bc0cdab0 2023-10-10T20:23:55.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.004+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4bc04bb80 con 0x7fc4bc0cdab0 2023-10-10T20:23:55.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.005+0000 7fc4a3fff700 1 -- 172.21.15.148:0/87202063 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15444 (secure 0 0 0) 0x7fc4b00528e0 con 0x7fc4bc0cdab0 2023-10-10T20:23:55.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:55.008 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":84,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:26.678281+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 msgr2=0x7fc4a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 0x7fc4a80693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac007d50 comp rx=0 tx=0).stop 2023-10-10T20:23:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 msgr2=0x7fc4bc155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00093f0 tx=0x7fc4b0017000 comp rx=0 tx=0).stop 2023-10-10T20:23:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 shutdown_connections 2023-10-10T20:23:55.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc4a8066f30 0x7fc4a80693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4bc15dda0 0x7fc4bc15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc4bc0cd6e0 0x7fc4bc0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 --2- 172.21.15.148:0/87202063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4bc0cdab0 0x7fc4bc155db0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 >> 172.21.15.148:0/87202063 conn(0x7fc4bc067660 msgr2=0x7fc4bc0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 shutdown_connections 2023-10-10T20:23:55.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.007+0000 7fc4c12af700 1 -- 172.21.15.148:0/87202063 wait complete. 2023-10-10T20:23:55.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:23:10.830500+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:23:11.960772+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:23:55.023 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:23:55.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.138+0000 7fbc99a13700 1 Processor -- start 2023-10-10T20:23:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.138+0000 7fbc99a13700 1 -- start start 2023-10-10T20:23:55.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99a13700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc940cce30 0x7fbc9415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99a13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9406aa00 0x7fbc940b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99a13700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc940b4d70 con 0x7fbc940b6580 2023-10-10T20:23:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99a13700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc940cd8d0 con 0x7fbc940cce30 2023-10-10T20:23:55.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99a13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940b4440 con 0x7fbc9406aa00 2023-10-10T20:23:55.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc98a11700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc940cce30 0x7fbc9415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc99212700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9406aa00 0x7fbc940b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc98a11700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc940cce30 0x7fbc9415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:36142/0 (socket says 172.21.15.148:36142) 2023-10-10T20:23:55.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc98a11700 1 -- 172.21.15.148:0/1384088020 learned_addr learned my addr 172.21.15.148:0/1384088020 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:55.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc93fff700 1 --1- 172.21.15.148:0/1384088020 >> v1:172.21.15.161:6789/0 conn(0x7fbc940b6580 0x7fbc9406a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:50010/0 (socket says 172.21.15.148:50010) 2023-10-10T20:23:55.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.139+0000 7fbc98a11700 1 -- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9406aa00 msgr2=0x7fbc940b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:55.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9406aa00 0x7fbc940b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc98a11700 1 -- 172.21.15.148:0/1384088020 >> v1:172.21.15.161:6789/0 conn(0x7fbc940b6580 legacy=0x7fbc9406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc98a11700 1 -- 172.21.15.148:0/1384088020 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940b4720 con 0x7fbc940cce30 2023-10-10T20:23:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc937fe700 1 -- 172.21.15.148:0/1384088020 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475831284 0 0) 0x7fbc940b4d70 con 0x7fbc940b6580 2023-10-10T20:23:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940cce30 0x7fbc9415aca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc880099b0 tx=0x7fbc88009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6832c96c1bfa596 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc937fe700 1 -- 172.21.15.148:0/1384088020 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88009ea0 con 0x7fbc940cce30 2023-10-10T20:23:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc937fe700 1 -- 172.21.15.148:0/1384088020 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800b860 con 0x7fbc940cce30 2023-10-10T20:23:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc937fe700 1 -- 172.21.15.148:0/1384088020 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800bc00 con 0x7fbc940cce30 2023-10-10T20:23:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 -- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940cce30 msgr2=0x7fbc9415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940cce30 0x7fbc9415aca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc880099b0 tx=0x7fbc88009250 comp rx=0 tx=0).stop 2023-10-10T20:23:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 -- 172.21.15.148:0/1384088020 shutdown_connections 2023-10-10T20:23:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940cce30 0x7fbc9415aca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1384088020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9406aa00 0x7fbc940b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 -- 172.21.15.148:0/1384088020 >> 172.21.15.148:0/1384088020 conn(0x7fbc94068cb0 msgr2=0x7fbc940690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 -- 172.21.15.148:0/1384088020 shutdown_connections 2023-10-10T20:23:55.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.140+0000 7fbc99a13700 1 -- 172.21.15.148:0/1384088020 wait complete. 2023-10-10T20:23:55.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.141+0000 7fbc99a13700 1 Processor -- start 2023-10-10T20:23:55.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 -- start start 2023-10-10T20:23:55.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 0x7fbc94149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc940cce30 0x7fbc94155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9414a0f0 con 0x7fbc940b6580 2023-10-10T20:23:55.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc94154890 con 0x7fbc9406aa00 2023-10-10T20:23:55.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc99a13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc94154b50 con 0x7fbc940cce30 2023-10-10T20:23:55.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43356/0 (socket says 172.21.15.148:43356) 2023-10-10T20:23:55.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.142+0000 7fbc93fff700 1 -- 172.21.15.148:0/1021163705 learned_addr learned my addr 172.21.15.148:0/1021163705 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:55.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 0x7fbc94149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc99212700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc940cce30 0x7fbc94155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 -- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc940cce30 msgr2=0x7fbc94155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc940cce30 0x7fbc94155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 -- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 msgr2=0x7fbc94153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880094f0 con 0x7fbc9406aa00 2023-10-10T20:23:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc93fff700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc98a11700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 0x7fbc94149bb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc94089bc0 tx=0x7fbc88017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88000d10 con 0x7fbc9406aa00 2023-10-10T20:23:55.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800bbe0 con 0x7fbc9406aa00 2023-10-10T20:23:55.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc94154e30 con 0x7fbc9406aa00 2023-10-10T20:23:55.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88011630 con 0x7fbc9406aa00 2023-10-10T20:23:55.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.143+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc941553f0 con 0x7fbc9406aa00 2023-10-10T20:23:55.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.144+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc94089550 con 0x7fbc9406aa00 2023-10-10T20:23:55.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc88011a60 con 0x7fbc9406aa00 2023-10-10T20:23:55.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc90ff9700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 0x7fbc780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc88085510 con 0x7fbc9406aa00 2023-10-10T20:23:55.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc93fff700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 0x7fbc780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7fbc7806ea50 con 0x7fbc9406aa00 2023-10-10T20:23:55.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.145+0000 7fbc93fff700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 0x7fbc780693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc940d1760 tx=0x7fbc80007e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.149+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc88028060 con 0x7fbc9406aa00 2023-10-10T20:23:55.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.333+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc94154c90 con 0x7fbc9406aa00 2023-10-10T20:23:55.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.333+0000 7fbc90ff9700 1 -- 172.21.15.148:0/1021163705 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc94154c90 con 0x7fbc9406aa00 2023-10-10T20:23:55.336 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:23:55.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 msgr2=0x7fbc780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 0x7fbc780693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc940d1760 tx=0x7fbc80007e10 comp rx=0 tx=0).stop 2023-10-10T20:23:55.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 msgr2=0x7fbc94149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 0x7fbc94149bb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc94089bc0 tx=0x7fbc88017000 comp rx=0 tx=0).stop 2023-10-10T20:23:55.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 shutdown_connections 2023-10-10T20:23:55.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc78066f30 0x7fbc780693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc940cce30 0x7fbc94155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9406aa00 0x7fbc94149bb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 --2- 172.21.15.148:0/1021163705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc940b6580 0x7fbc94153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.335+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 >> 172.21.15.148:0/1021163705 conn(0x7fbc94068cb0 msgr2=0x7fbc940af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.336+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 shutdown_connections 2023-10-10T20:23:55.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.336+0000 7fbc99a13700 1 -- 172.21.15.148:0/1021163705 wait complete. 2023-10-10T20:23:55.350 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:23:55.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.466+0000 7f59673ef700 1 Processor -- start 2023-10-10T20:23:55.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.466+0000 7f59673ef700 1 -- start start 2023-10-10T20:23:55.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f59673ef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5960157cc0 0x7f59600b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f59673ef700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59600afb50 con 0x7f5960069f80 2023-10-10T20:23:55.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f59673ef700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59600afe30 con 0x7f59600ca9e0 2023-10-10T20:23:55.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f59673ef700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600b0110 con 0x7f5960157cc0 2023-10-10T20:23:55.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596518b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5960157cc0 0x7f59600b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596598c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5960069f80 0x7f59600c90f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60986/0 (socket says 172.21.15.148:60986) 2023-10-10T20:23:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596598c700 1 -- 172.21.15.148:0/1225683611 learned_addr learned my addr 172.21.15.148:0/1225683611 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596518b700 1 -- 172.21.15.148:0/1225683611 >> v1:172.21.15.161:6789/0 conn(0x7f59600ca9e0 legacy=0x7f5960069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596518b700 1 -- 172.21.15.148:0/1225683611 >> v1:172.21.15.148:6789/0 conn(0x7f5960069f80 legacy=0x7f59600c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596518b700 1 -- 172.21.15.148:0/1225683611 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600b03f0 con 0x7f5960157cc0 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f5957fff700 1 -- 172.21.15.148:0/1225683611 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464070832 0 0) 0x7f59600afb50 con 0x7f5960069f80 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.467+0000 7f596518b700 1 --2- 172.21.15.148:0/1225683611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5960157cc0 0x7f59600b0bf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f595c0098e0 tx=0x7f595c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193e0b5a8bbef334 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f5957fff700 1 -- 172.21.15.148:0/1225683611 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c009e80 con 0x7f5960157cc0 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f5957fff700 1 -- 172.21.15.148:0/1225683611 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c00b860 con 0x7f5960157cc0 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f5957fff700 1 -- 172.21.15.148:0/1225683611 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c00bbe0 con 0x7f5960157cc0 2023-10-10T20:23:55.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 -- 172.21.15.148:0/1225683611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5960157cc0 msgr2=0x7f59600b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 --2- 172.21.15.148:0/1225683611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5960157cc0 0x7f59600b0bf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f595c0098e0 tx=0x7f595c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 -- 172.21.15.148:0/1225683611 shutdown_connections 2023-10-10T20:23:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 --2- 172.21.15.148:0/1225683611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5960157cc0 0x7f59600b0bf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 -- 172.21.15.148:0/1225683611 >> 172.21.15.148:0/1225683611 conn(0x7f59600cb5c0 msgr2=0x7f59600cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 -- 172.21.15.148:0/1225683611 shutdown_connections 2023-10-10T20:23:55.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.468+0000 7f59673ef700 1 -- 172.21.15.148:0/1225683611 wait complete. 2023-10-10T20:23:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.469+0000 7f59673ef700 1 Processor -- start 2023-10-10T20:23:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.469+0000 7f59673ef700 1 -- start start 2023-10-10T20:23:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5960069f80 0x7f59601688b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5960157cc0 0x7f59601584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596015b5b0 con 0x7f59600ca9e0 2023-10-10T20:23:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5960168df0 con 0x7f5960157cc0 2023-10-10T20:23:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f59673ef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c5b60 con 0x7f5960069f80 2023-10-10T20:23:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f596498a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f596498a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43370/0 (socket says 172.21.15.148:43370) 2023-10-10T20:23:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.470+0000 7f596498a700 1 -- 172.21.15.148:0/1383991382 learned_addr learned my addr 172.21.15.148:0/1383991382 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 -- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5960069f80 msgr2=0x7f59601688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596598c700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5960157cc0 0x7f59601584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5960069f80 0x7f59601688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 -- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5960157cc0 msgr2=0x7f59601584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5960157cc0 0x7f59601584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009420 con 0x7f59600ca9e0 2023-10-10T20:23:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f596498a700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f595000c590 tx=0x7f595000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950011590 con 0x7f59600ca9e0 2023-10-10T20:23:55.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500116f0 con 0x7f59600ca9e0 2023-10-10T20:23:55.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59600c5e40 con 0x7f59600ca9e0 2023-10-10T20:23:55.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950011ab0 con 0x7f59600ca9e0 2023-10-10T20:23:55.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.471+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59600c6400 con 0x7f59600ca9e0 2023-10-10T20:23:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.472+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596004c3c0 con 0x7f59600ca9e0 2023-10-10T20:23:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f595001b3b0 con 0x7f59600ca9e0 2023-10-10T20:23:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f59557fa700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 0x7f59440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5950083ae0 con 0x7f59600ca9e0 2023-10-10T20:23:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f596518b700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 0x7f59440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f594406ea50 con 0x7f59600ca9e0 2023-10-10T20:23:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.473+0000 7f596518b700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 0x7f59440693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f595c0091e0 tx=0x7f595c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.477+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5950017070 con 0x7f59600ca9e0 2023-10-10T20:23:55.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.660+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59600b06d0 con 0x7f59600ca9e0 2023-10-10T20:23:55.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.660+0000 7f59557fa700 1 -- 172.21.15.148:0/1383991382 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5950050d50 con 0x7f59600ca9e0 2023-10-10T20:23:55.663 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:23:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 msgr2=0x7f59440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 0x7f59440693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f595c0091e0 tx=0x7f595c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 msgr2=0x7f596014ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f595000c590 tx=0x7f595000c950 comp rx=0 tx=0).stop 2023-10-10T20:23:55.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 shutdown_connections 2023-10-10T20:23:55.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5944066f30 0x7f59440693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5960157cc0 0x7f59601584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5960069f80 0x7f59601688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 --2- 172.21.15.148:0/1383991382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59600ca9e0 0x7f596014ceb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 >> 172.21.15.148:0/1383991382 conn(0x7f59600cb5c0 msgr2=0x7f59600cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.662+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 shutdown_connections 2023-10-10T20:23:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.663+0000 7f59673ef700 1 -- 172.21.15.148:0/1383991382 wait complete. 2023-10-10T20:23:55.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:23:55.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.794+0000 7f8c37525700 1 Processor -- start 2023-10-10T20:23:55.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.794+0000 7f8c37525700 1 -- start start 2023-10-10T20:23:55.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c37525700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c300adc80 0x7f8c300b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c37525700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c300cf0f0 0x7f8c300b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c37525700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c300b0e40 con 0x7f8c300b0460 2023-10-10T20:23:55.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c37525700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300b1850 con 0x7f8c300adc80 2023-10-10T20:23:55.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c37525700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300b20f0 con 0x7f8c300cf0f0 2023-10-10T20:23:55.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c300adc80 0x7f8c300b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c300adc80 0x7f8c300b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43380/0 (socket says 172.21.15.148:43380) 2023-10-10T20:23:55.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 -- 172.21.15.148:0/1739076626 learned_addr learned my addr 172.21.15.148:0/1739076626 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:55.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36d24700 1 --2- 172.21.15.148:0/1739076626 >> v2:172.21.15.161:3300/0 conn(0x7f8c300cf0f0 0x7f8c300b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 -- 172.21.15.148:0/1739076626 >> v2:172.21.15.161:3300/0 conn(0x7f8c300cf0f0 msgr2=0x7f8c300b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:55.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 --2- 172.21.15.148:0/1739076626 >> v2:172.21.15.161:3300/0 conn(0x7f8c300cf0f0 0x7f8c300b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 -- 172.21.15.148:0/1739076626 >> v1:172.21.15.161:6789/0 conn(0x7f8c300b0460 legacy=0x7f8c300ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 -- 172.21.15.148:0/1739076626 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300cf500 con 0x7f8c300adc80 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c36523700 1 --2- 172.21.15.148:0/1739076626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300adc80 0x7f8c300b2580 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006ed30 tx=0x7f8c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867fa1e8b0c11c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.795+0000 7f8c35521700 1 -- 172.21.15.148:0/1739076626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c009e70 con 0x7f8c300adc80 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c35521700 1 -- 172.21.15.148:0/1739076626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c00b860 con 0x7f8c300adc80 2023-10-10T20:23:55.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c35521700 1 -- 172.21.15.148:0/1739076626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c00bc00 con 0x7f8c300adc80 2023-10-10T20:23:55.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 -- 172.21.15.148:0/1739076626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300adc80 msgr2=0x7f8c300b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 --2- 172.21.15.148:0/1739076626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300adc80 0x7f8c300b2580 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006ed30 tx=0x7f8c2c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:55.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 -- 172.21.15.148:0/1739076626 shutdown_connections 2023-10-10T20:23:55.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 --2- 172.21.15.148:0/1739076626 >> v2:172.21.15.161:3300/0 conn(0x7f8c300cf0f0 0x7f8c300b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 --2- 172.21.15.148:0/1739076626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300adc80 0x7f8c300b2580 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 -- 172.21.15.148:0/1739076626 >> 172.21.15.148:0/1739076626 conn(0x7f8c3006c700 msgr2=0x7f8c30069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 -- 172.21.15.148:0/1739076626 shutdown_connections 2023-10-10T20:23:55.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.796+0000 7f8c37525700 1 -- 172.21.15.148:0/1739076626 wait complete. 2023-10-10T20:23:55.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.797+0000 7f8c37525700 1 Processor -- start 2023-10-10T20:23:55.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.797+0000 7f8c37525700 1 -- start start 2023-10-10T20:23:55.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 0x7f8c30144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c300cf0f0 0x7f8c300abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300c39d0 con 0x7f8c300b0460 2023-10-10T20:23:55.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300a99c0 con 0x7f8c300cf0f0 2023-10-10T20:23:55.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c37525700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300a8e20 con 0x7f8c300adc80 2023-10-10T20:23:55.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43394/0 (socket says 172.21.15.148:43394) 2023-10-10T20:23:55.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 -- 172.21.15.148:0/2097020932 learned_addr learned my addr 172.21.15.148:0/2097020932 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:55.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c36d24700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c300cf0f0 0x7f8c300abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c36523700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 0x7f8c30144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 -- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 msgr2=0x7f8c30144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 0x7f8c30144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 -- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c300cf0f0 msgr2=0x7f8c300abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c300cf0f0 0x7f8c300abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c35d22700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0094f0 con 0x7f8c300b0460 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.798+0000 7f8c36523700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 0x7f8c30144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c35d22700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c5c0 tx=0x7f8c2000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20011590 con 0x7f8c300b0460 2023-10-10T20:23:55.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20011710 con 0x7f8c300b0460 2023-10-10T20:23:55.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c300a9090 con 0x7f8c300b0460 2023-10-10T20:23:55.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20011ad0 con 0x7f8c300b0460 2023-10-10T20:23:55.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.799+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c300a9420 con 0x7f8c300b0460 2023-10-10T20:23:55.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.800+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c300a9b70 con 0x7f8c300b0460 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.800+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c2001b3b0 con 0x7f8c300b0460 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.800+0000 7f8c267fc700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 0x7f8c140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.801+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c20083980 con 0x7f8c300b0460 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.801+0000 7f8c36523700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 0x7f8c140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.801+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f8c1406ea50 con 0x7f8c300b0460 2023-10-10T20:23:55.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.801+0000 7f8c36523700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 0x7f8c140693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006ed30 tx=0x7f8c2c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:55.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.805+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c20017070 con 0x7f8c300b0460 2023-10-10T20:23:55.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.987+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c300b0e40 con 0x7f8c300b0460 2023-10-10T20:23:55.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.987+0000 7f8c267fc700 1 -- 172.21.15.148:0/2097020932 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c300b0e40 con 0x7f8c300b0460 2023-10-10T20:23:55.990 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:23:55.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 msgr2=0x7f8c140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 0x7f8c140693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006ed30 tx=0x7f8c2c017000 comp rx=0 tx=0).stop 2023-10-10T20:23:55.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 msgr2=0x7f8c300aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:55.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c5c0 tx=0x7f8c2000c980 comp rx=0 tx=0).stop 2023-10-10T20:23:55.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 shutdown_connections 2023-10-10T20:23:55.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c14066f30 0x7f8c140693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c300cf0f0 0x7f8c300abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c300adc80 0x7f8c30144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 --2- 172.21.15.148:0/2097020932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c300b0460 0x7f8c300aba60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:55.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 >> 172.21.15.148:0/2097020932 conn(0x7f8c3006c700 msgr2=0x7f8c300cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:55.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 shutdown_connections 2023-10-10T20:23:55.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:55.989+0000 7f8c37525700 1 -- 172.21.15.148:0/2097020932 wait complete. 2023-10-10T20:23:56.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:56.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.117+0000 7ffad63f4700 1 Processor -- start 2023-10-10T20:23:56.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.117+0000 7ffad63f4700 1 -- start start 2023-10-10T20:23:56.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffad0157870 0x7ffad015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad015a620 0x7ffad015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffad015d580 0x7ffad0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad015cbb0 con 0x7ffad015d580 2023-10-10T20:23:56.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad015bfa0 con 0x7ffad0157870 2023-10-10T20:23:56.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad63f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015c280 con 0x7ffad015a620 2023-10-10T20:23:56.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffad015d580 0x7ffad0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffad015d580 0x7ffad0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43406/0 (socket says 172.21.15.148:43406) 2023-10-10T20:23:56.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 -- 172.21.15.148:0/2969461184 learned_addr learned my addr 172.21.15.148:0/2969461184 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffacf7fe700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad015a620 0x7ffad015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffacffff700 1 --2- 172.21.15.148:0/2969461184 >> v2:172.21.15.161:3300/0 conn(0x7ffad0157870 0x7ffad015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 -- 172.21.15.148:0/2969461184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad015a620 msgr2=0x7ffad015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad015a620 0x7ffad015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 -- 172.21.15.148:0/2969461184 >> v2:172.21.15.161:3300/0 conn(0x7ffad0157870 msgr2=0x7ffad015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:56.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 --2- 172.21.15.148:0/2969461184 >> v2:172.21.15.161:3300/0 conn(0x7ffad0157870 0x7ffad015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 -- 172.21.15.148:0/2969461184 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad015c560 con 0x7ffad015d580 2023-10-10T20:23:56.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffad4991700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015d580 0x7ffad0158110 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffad0069d80 tx=0x7ffac8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88627507ce95fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffaceffd700 1 -- 172.21.15.148:0/2969461184 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800e070 con 0x7ffad015d580 2023-10-10T20:23:56.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffaceffd700 1 -- 172.21.15.148:0/2969461184 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac800b970 con 0x7ffad015d580 2023-10-10T20:23:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.118+0000 7ffaceffd700 1 -- 172.21.15.148:0/2969461184 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80092e0 con 0x7ffad015d580 2023-10-10T20:23:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 -- 172.21.15.148:0/2969461184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015d580 msgr2=0x7ffad0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015d580 0x7ffad0158110 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffad0069d80 tx=0x7ffac8009840 comp rx=0 tx=0).stop 2023-10-10T20:23:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 -- 172.21.15.148:0/2969461184 shutdown_connections 2023-10-10T20:23:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015d580 0x7ffad0158110 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2969461184 >> v2:172.21.15.161:3300/0 conn(0x7ffad0157870 0x7ffad015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2969461184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad015a620 0x7ffad015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 -- 172.21.15.148:0/2969461184 >> 172.21.15.148:0/2969461184 conn(0x7ffad0068ce0 msgr2=0x7ffad00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:56.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 -- 172.21.15.148:0/2969461184 shutdown_connections 2023-10-10T20:23:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.119+0000 7ffad63f4700 1 -- 172.21.15.148:0/2969461184 wait complete. 2023-10-10T20:23:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.120+0000 7ffad63f4700 1 Processor -- start 2023-10-10T20:23:56.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.120+0000 7ffad63f4700 1 -- start start 2023-10-10T20:23:56.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad0157870 0x7ffad00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad015d580 0x7ffad0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00d15d0 con 0x7ffad015a620 2023-10-10T20:23:56.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00b0da0 con 0x7ffad015d580 2023-10-10T20:23:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad63f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00b1080 con 0x7ffad0157870 2023-10-10T20:23:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43416/0 (socket says 172.21.15.148:43416) 2023-10-10T20:23:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 -- 172.21.15.148:0/2376123966 learned_addr learned my addr 172.21.15.148:0/2376123966 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffad4991700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad015d580 0x7ffad0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacffff700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad0157870 0x7ffad00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 -- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad0157870 msgr2=0x7ffad00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad0157870 0x7ffad00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 -- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad015d580 msgr2=0x7ffad0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad015d580 0x7ffad0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8009ae0 con 0x7ffad015a620 2023-10-10T20:23:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.121+0000 7ffacf7fe700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffac400ec50 tx=0x7ffac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.122+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400c9d0 con 0x7ffad015a620 2023-10-10T20:23:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.122+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400cb30 con 0x7ffad015a620 2023-10-10T20:23:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.122+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00b1360 con 0x7ffad015a620 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.122+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4013470 con 0x7ffad015a620 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.122+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00b1920 con 0x7ffad015a620 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.123+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad014aa90 con 0x7ffad015a620 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.123+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffac4013860 con 0x7ffad015a620 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.123+0000 7ffab7fff700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 0x7ffabc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.124+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffac40848b0 con 0x7ffad015a620 2023-10-10T20:23:56.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.124+0000 7ffacffff700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 0x7ffabc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.124+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7ffabc06ea80 con 0x7ffad015a620 2023-10-10T20:23:56.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.124+0000 7ffacffff700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 0x7ffabc069410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009d40 tx=0x7ffac0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.128+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffac4017060 con 0x7ffad015a620 2023-10-10T20:23:56.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.353+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffad004c3c0 con 0x7ffad015a620 2023-10-10T20:23:56.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.353+0000 7ffab7fff700 1 -- 172.21.15.148:0/2376123966 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ffac4017090 con 0x7ffad015a620 2023-10-10T20:23:56.356 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:56.356 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.355+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 msgr2=0x7ffabc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.355+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 0x7ffabc069410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009d40 tx=0x7ffac0008040 comp rx=0 tx=0).stop 2023-10-10T20:23:56.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.355+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 msgr2=0x7ffad014a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.355+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffac400ec50 tx=0x7ffac40091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.355+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 shutdown_connections 2023-10-10T20:23:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffabc066f60 0x7ffabc069410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad015d580 0x7ffad0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffad0157870 0x7ffad00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 --2- 172.21.15.148:0/2376123966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad015a620 0x7ffad014a1a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 >> 172.21.15.148:0/2376123966 conn(0x7ffad0068ce0 msgr2=0x7ffad015b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 shutdown_connections 2023-10-10T20:23:56.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.356+0000 7ffad63f4700 1 -- 172.21.15.148:0/2376123966 wait complete. 2023-10-10T20:23:56.361 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:56.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:56.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.484+0000 7f6141705700 1 Processor -- start 2023-10-10T20:23:56.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f6141705700 1 -- start start 2023-10-10T20:23:56.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f6141705700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f613c157c40 0x7f613c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f6141705700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c15cbb0 con 0x7f613c15a620 2023-10-10T20:23:56.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f6141705700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c15bfa0 con 0x7f613c157870 2023-10-10T20:23:56.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f6141705700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c15bdb0 con 0x7f613c157c40 2023-10-10T20:23:56.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f613b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f613c157c40 0x7f613c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f613b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f613c157c40 0x7f613c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43432/0 (socket says 172.21.15.148:43432) 2023-10-10T20:23:56.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.485+0000 7f61327fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f613c15a620 0x7f613c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32768/0 (socket says 172.21.15.148:32768) 2023-10-10T20:23:56.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613b7fe700 1 -- 172.21.15.148:0/996639890 learned_addr learned my addr 172.21.15.148:0/996639890 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:56.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613b7fe700 1 -- 172.21.15.148:0/996639890 >> v1:172.21.15.161:6789/0 conn(0x7f613c157870 legacy=0x7f613c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:56.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613b7fe700 1 -- 172.21.15.148:0/996639890 >> v1:172.21.15.148:6789/0 conn(0x7f613c15a620 legacy=0x7f613c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613a7fc700 1 -- 172.21.15.148:0/996639890 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827295610 0 0) 0x7f613c15cbb0 con 0x7f613c15a620 2023-10-10T20:23:56.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613b7fe700 1 -- 172.21.15.148:0/996639890 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c15c280 con 0x7f613c157c40 2023-10-10T20:23:56.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613b7fe700 1 --2- 172.21.15.148:0/996639890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c158110 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6134009f20 tx=0x7f6134009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8471c47a833dd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613a7fc700 1 -- 172.21.15.148:0/996639890 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400e070 con 0x7f613c157c40 2023-10-10T20:23:56.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613a7fc700 1 -- 172.21.15.148:0/996639890 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613400ba40 con 0x7f613c157c40 2023-10-10T20:23:56.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.486+0000 7f613a7fc700 1 -- 172.21.15.148:0/996639890 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340092e0 con 0x7f613c157c40 2023-10-10T20:23:56.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 -- 172.21.15.148:0/996639890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 msgr2=0x7f613c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 --2- 172.21.15.148:0/996639890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c158110 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6134009f20 tx=0x7f6134009840 comp rx=0 tx=0).stop 2023-10-10T20:23:56.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 -- 172.21.15.148:0/996639890 shutdown_connections 2023-10-10T20:23:56.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 --2- 172.21.15.148:0/996639890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c158110 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 -- 172.21.15.148:0/996639890 >> 172.21.15.148:0/996639890 conn(0x7f613c068ce0 msgr2=0x7f613c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:56.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 -- 172.21.15.148:0/996639890 shutdown_connections 2023-10-10T20:23:56.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.487+0000 7f6141705700 1 -- 172.21.15.148:0/996639890 wait complete. 2023-10-10T20:23:56.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.488+0000 7f6141705700 1 Processor -- start 2023-10-10T20:23:56.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.488+0000 7f6141705700 1 -- start start 2023-10-10T20:23:56.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f613c157870 0x7f613c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c15a620 0x7f613c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c14e830 con 0x7f613c157c40 2023-10-10T20:23:56.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b16b0 con 0x7f613c15a620 2023-10-10T20:23:56.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b1990 con 0x7f613c157870 2023-10-10T20:23:56.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43438/0 (socket says 172.21.15.148:43438) 2023-10-10T20:23:56.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 -- 172.21.15.148:0/897317159 learned_addr learned my addr 172.21.15.148:0/897317159 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:56.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f613affd700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f613c157870 0x7f613c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f613b7fe700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c15a620 0x7f613c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 -- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f613c157870 msgr2=0x7f613c0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f613c157870 0x7f613c0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 -- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c15a620 msgr2=0x7f613c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c15a620 0x7f613c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134009ae0 con 0x7f613c157c40 2023-10-10T20:23:56.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61327fc700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f612400c510 tx=0x7f612400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124005a70 con 0x7f613c157c40 2023-10-10T20:23:56.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124005bf0 con 0x7f613c157c40 2023-10-10T20:23:56.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240070d0 con 0x7f613c157c40 2023-10-10T20:23:56.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0b1c70 con 0x7f613c157c40 2023-10-10T20:23:56.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.489+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c13ec20 con 0x7f613c157c40 2023-10-10T20:23:56.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.490+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c14e600 con 0x7f613c157c40 2023-10-10T20:23:56.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.491+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6124007480 con 0x7f613c157c40 2023-10-10T20:23:56.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.491+0000 7f61337fe700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.491+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61240838c0 con 0x7f613c157c40 2023-10-10T20:23:56.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.491+0000 7f613affd700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.491+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f611c06ea50 con 0x7f613c157c40 2023-10-10T20:23:56.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.492+0000 7f613affd700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 0x7f611c0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f612c003a30 tx=0x7f612c007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.495+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6124017020 con 0x7f613c157c40 2023-10-10T20:23:56.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.680+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f613c04c3c0 con 0x7f613c157c40 2023-10-10T20:23:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.681+0000 7f61337fe700 1 -- 172.21.15.148:0/897317159 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15444 (secure 0 0 0) 0x7f613c14e600 con 0x7f613c157c40 2023-10-10T20:23:56.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:56.684 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":84,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:26.678281+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:56.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 msgr2=0x7f611c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 0x7f611c0693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f612c003a30 tx=0x7f612c007ea0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 msgr2=0x7f613c13e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f612400c510 tx=0x7f612400c820 comp rx=0 tx=0).stop 2023-10-10T20:23:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 shutdown_connections 2023-10-10T20:23:56.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f611c066f30 0x7f611c0693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f613c15a620 0x7f613c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f613c157870 0x7f613c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 --2- 172.21.15.148:0/897317159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f613c157c40 0x7f613c13e6e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 >> 172.21.15.148:0/897317159 conn(0x7f613c068ce0 msgr2=0x7f613c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:56.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 shutdown_connections 2023-10-10T20:23:56.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.683+0000 7f6141705700 1 -- 172.21.15.148:0/897317159 wait complete. 2023-10-10T20:23:56.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:23:56.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:56.729 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:56.727+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-10T20:23:56.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.813+0000 7fef03068700 1 Processor -- start 2023-10-10T20:23:56.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.813+0000 7fef03068700 1 -- start start 2023-10-10T20:23:56.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef03068700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feefc15a620 0x7feefc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef03068700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15cbb0 con 0x7feefc15d580 2023-10-10T20:23:56.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef03068700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15bfa0 con 0x7feefc157870 2023-10-10T20:23:56.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef03068700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc15bdb0 con 0x7feefc15a620 2023-10-10T20:23:56.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feefc15a620 0x7feefc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef01605700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feefc15d580 0x7feefc158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32774/0 (socket says 172.21.15.148:32774) 2023-10-10T20:23:56.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7fef01605700 1 -- 172.21.15.148:0/4269763237 learned_addr learned my addr 172.21.15.148:0/4269763237 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:56.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefb7fe700 1 -- 172.21.15.148:0/4269763237 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861723224 0 0) 0x7feefc15cbb0 con 0x7feefc15d580 2023-10-10T20:23:56.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefb7fe700 1 -- 172.21.15.148:0/4269763237 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003570 con 0x7feefc15d580 2023-10-10T20:23:56.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefbfff700 1 -- 172.21.15.148:0/4269763237 >> v1:172.21.15.161:6789/0 conn(0x7feefc157870 legacy=0x7feefc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefbfff700 1 -- 172.21.15.148:0/4269763237 >> v1:172.21.15.148:6789/0 conn(0x7feefc15d580 legacy=0x7feefc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefbfff700 1 -- 172.21.15.148:0/4269763237 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc15c280 con 0x7feefc15a620 2023-10-10T20:23:56.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.814+0000 7feefbfff700 1 --2- 172.21.15.148:0/4269763237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc15d040 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feee80098e0 tx=0x7feee80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928382e3a601257 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7feefb7fe700 1 -- 172.21.15.148:0/4269763237 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800b640 con 0x7feefc15a620 2023-10-10T20:23:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7feefb7fe700 1 -- 172.21.15.148:0/4269763237 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee800b7a0 con 0x7feefc15a620 2023-10-10T20:23:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7feefb7fe700 1 -- 172.21.15.148:0/4269763237 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800bb40 con 0x7feefc15a620 2023-10-10T20:23:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 -- 172.21.15.148:0/4269763237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 msgr2=0x7feefc15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 --2- 172.21.15.148:0/4269763237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc15d040 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feee80098e0 tx=0x7feee80091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 -- 172.21.15.148:0/4269763237 shutdown_connections 2023-10-10T20:23:56.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 --2- 172.21.15.148:0/4269763237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc15d040 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 -- 172.21.15.148:0/4269763237 >> 172.21.15.148:0/4269763237 conn(0x7feefc068ce0 msgr2=0x7feefc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:56.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 -- 172.21.15.148:0/4269763237 shutdown_connections 2023-10-10T20:23:56.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.815+0000 7fef03068700 1 -- 172.21.15.148:0/4269763237 wait complete. 2023-10-10T20:23:56.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.816+0000 7fef03068700 1 Processor -- start 2023-10-10T20:23:56.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.816+0000 7fef03068700 1 -- start start 2023-10-10T20:23:56.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc157870 0x7feefc151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feefc15d580 0x7feefc152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc16b530 con 0x7feefc15a620 2023-10-10T20:23:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1516c0 con 0x7feefc157870 2023-10-10T20:23:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef03068700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b2f50 con 0x7feefc15d580 2023-10-10T20:23:56.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43452/0 (socket says 172.21.15.148:43452) 2023-10-10T20:23:56.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 -- 172.21.15.148:0/2315490914 learned_addr learned my addr 172.21.15.148:0/2315490914 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:56.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef01605700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feefc15d580 0x7feefc152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7fef00e04700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc157870 0x7feefc151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 -- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feefc15d580 msgr2=0x7feefc152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feefc15d580 0x7feefc152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 -- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc157870 msgr2=0x7feefc151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc157870 0x7feefc151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.817+0000 7feefbfff700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009420 con 0x7feefc15a620 2023-10-10T20:23:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7feefbfff700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feee8013000 tx=0x7feee8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800bb20 con 0x7feefc15a620 2023-10-10T20:23:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8011420 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc150b00 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee80117c0 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.818+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc15dba0 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.819+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc15dd90 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.819+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feee8011b70 con 0x7feefc15a620 2023-10-10T20:23:56.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.819+0000 7feef8ff9700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 0x7feee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:56.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.820+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feee8085c60 con 0x7feefc15a620 2023-10-10T20:23:56.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.820+0000 7fef00e04700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 0x7feee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:56.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.820+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7feee006ea50 con 0x7feefc15a620 2023-10-10T20:23:56.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.820+0000 7fef00e04700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 0x7feee00693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feef0003860 tx=0x7feef0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:56.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:56.824+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feee8052ed0 con 0x7feefc15a620 2023-10-10T20:23:57.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.047+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feefc04c3c0 con 0x7feefc15a620 2023-10-10T20:23:57.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.048+0000 7feef8ff9700 1 -- 172.21.15.148:0/2315490914 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7feee8029030 con 0x7feefc15a620 2023-10-10T20:23:57.050 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:57.050 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.049+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 msgr2=0x7feee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.049+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 0x7feee00693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feef0003860 tx=0x7feef0009040 comp rx=0 tx=0).stop 2023-10-10T20:23:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.049+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 msgr2=0x7feefc0b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.049+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feee8013000 tx=0x7feee8017000 comp rx=0 tx=0).stop 2023-10-10T20:23:57.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 shutdown_connections 2023-10-10T20:23:57.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feee0066f30 0x7feee00693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feefc15d580 0x7feefc152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc157870 0x7feefc151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 --2- 172.21.15.148:0/2315490914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc15a620 0x7feefc0b2a10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 >> 172.21.15.148:0/2315490914 conn(0x7feefc068ce0 msgr2=0x7feefc0b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 shutdown_connections 2023-10-10T20:23:57.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.050+0000 7fef03068700 1 -- 172.21.15.148:0/2315490914 wait complete. 2023-10-10T20:23:57.054 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:57.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:23:57.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.215+0000 7f0227f1e700 1 Processor -- start 2023-10-10T20:23:57.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.215+0000 7f0227f1e700 1 -- start start 2023-10-10T20:23:57.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f0227f1e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f022015a080 0x7f02200acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f0227f1e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022015e650 con 0x7f0220070e10 2023-10-10T20:23:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f0227f1e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022015dd20 con 0x7f0220159cb0 2023-10-10T20:23:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f0227f1e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200ad270 con 0x7f022015a080 2023-10-10T20:23:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f02254b9700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0220070e10 0x7f02200ac510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32780/0 (socket says 172.21.15.148:32780) 2023-10-10T20:23:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.216+0000 7f02254b9700 1 -- 172.21.15.148:0/364854274 learned_addr learned my addr 172.21.15.148:0/364854274 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 --2- 172.21.15.148:0/364854274 >> v2:172.21.15.161:3300/0 conn(0x7f022015a080 0x7f02200acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331447925 0 0) 0x7f022015e650 con 0x7f0220070e10 2023-10-10T20:23:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0208003570 con 0x7f0220070e10 2023-10-10T20:23:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323943558 0 0) 0x7f022015dd20 con 0x7f0220159cb0 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f022015e650 con 0x7f0220159cb0 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259049219 0 0) 0x7f0208003570 con 0x7f0220070e10 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 >> v2:172.21.15.161:3300/0 conn(0x7f022015a080 msgr2=0x7f02200acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 --2- 172.21.15.148:0/364854274 >> v2:172.21.15.161:3300/0 conn(0x7f022015a080 0x7f02200acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 >> v1:172.21.15.161:6789/0 conn(0x7f0220159cb0 legacy=0x7f0220070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015e000 con 0x7f0220070e10 2023-10-10T20:23:57.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f02140015f0 con 0x7f0220070e10 2023-10-10T20:23:57.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 >> v1:172.21.15.148:6789/0 conn(0x7f0220070e10 legacy=0x7f02200ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0208003bf0 0x7f0208003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0208004810 0x7f0208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f020800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200ad270 con 0x7f02080051c0 2023-10-10T20:23:57.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020800c160 con 0x7f0208003bf0 2023-10-10T20:23:57.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020800c320 con 0x7f0208004810 2023-10-10T20:23:57.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f020800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02254b9700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0208004810 0x7f0208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0225cba700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0208003bf0 0x7f0208003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 -- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0208004810 msgr2=0x7f0208004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:57.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0208004810 0x7f0208004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 -- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0208003bf0 msgr2=0x7f0208003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:57.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0208003bf0 0x7f0208003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f02264bb700 1 -- 172.21.15.148:0/364854274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208003570 con 0x7f02080051c0 2023-10-10T20:23:57.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.217+0000 7f0225cba700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0208003bf0 0x7f0208003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:23:57.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f02264bb700 1 -- 172.21.15.148:0/364854274 reap_dead start 2023-10-10T20:23:57.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f02264bb700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f020800ba60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0218006f00 tx=0x7f02180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7f7dabd02cd9c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800e710 con 0x7f02080051c0 2023-10-10T20:23:57.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800e870 con 0x7f02080051c0 2023-10-10T20:23:57.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0224cb8700 1 -- 172.21.15.148:0/364854274 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800ec10 con 0x7f02080051c0 2023-10-10T20:23:57.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 -- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 msgr2=0x7f020800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f020800ba60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0218006f00 tx=0x7f02180091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 -- 172.21.15.148:0/364854274 shutdown_connections 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 --2- 172.21.15.148:0/364854274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f020800ba60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 -- 172.21.15.148:0/364854274 >> 172.21.15.148:0/364854274 conn(0x7f022006ad40 msgr2=0x7f02200cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 -- 172.21.15.148:0/364854274 shutdown_connections 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.218+0000 7f0227f1e700 1 -- 172.21.15.148:0/364854274 wait complete. 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.219+0000 7f0227f1e700 1 Processor -- start 2023-10-10T20:23:57.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.219+0000 7f0227f1e700 1 -- start start 2023-10-10T20:23:57.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f022015c940 0x7f022015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02200b2320 0x7f02200b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b2cd0 con 0x7f02080051c0 2023-10-10T20:23:57.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200aed50 con 0x7f02200b2320 2023-10-10T20:23:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0227f1e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200ae440 con 0x7f022015c940 2023-10-10T20:23:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0225cba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0225cba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43476/0 (socket says 172.21.15.148:43476) 2023-10-10T20:23:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f0225cba700 1 -- 172.21.15.148:0/938741879 learned_addr learned my addr 172.21.15.148:0/938741879 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f02254b9700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f022015c940 0x7f022015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.220+0000 7f02264bb700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02200b2320 0x7f02200b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 -- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f022015c940 msgr2=0x7f022015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f022015c940 0x7f022015cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 -- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02200b2320 msgr2=0x7f02200b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02200b2320 0x7f02200b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022014db60 con 0x7f02080051c0 2023-10-10T20:23:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0225cba700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f021000eb00 tx=0x7f0210009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210006ba0 con 0x7f02080051c0 2023-10-10T20:23:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210010040 con 0x7f02080051c0 2023-10-10T20:23:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200ae700 con 0x7f02080051c0 2023-10-10T20:23:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100032e0 con 0x7f02080051c0 2023-10-10T20:23:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.221+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0220141c70 con 0x7f02080051c0 2023-10-10T20:23:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.222+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022004c3c0 con 0x7f02080051c0 2023-10-10T20:23:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.222+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02100016e0 con 0x7f02080051c0 2023-10-10T20:23:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.222+0000 7f021dffb700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 0x7f020c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.223+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0210084000 con 0x7f02080051c0 2023-10-10T20:23:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.223+0000 7f02254b9700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 0x7f020c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.223+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7f020c06ea50 con 0x7f02080051c0 2023-10-10T20:23:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.223+0000 7f02254b9700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 0x7f020c0693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0214001380 tx=0x7f0214009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.227+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0210051270 con 0x7f02080051c0 2023-10-10T20:23:57.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.453+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f022006f470 con 0x7f02080051c0 2023-10-10T20:23:57.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.453+0000 7f021dffb700 1 -- 172.21.15.148:0/938741879 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f0210054890 con 0x7f02080051c0 2023-10-10T20:23:57.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:57.456 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:23:12.996722+0000","modified":"2023-10-10T20:23:16.652575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6060":{"gid":6060,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/934736302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":934736302},{"type":"v1","addr":"172.21.15.148:6837","nonce":934736302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-10T20:23:57.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 msgr2=0x7f020c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 0x7f020c0693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0214001380 tx=0x7f0214009040 comp rx=0 tx=0).stop 2023-10-10T20:23:57.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 msgr2=0x7f022016d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f021000eb00 tx=0x7f0210009040 comp rx=0 tx=0).stop 2023-10-10T20:23:57.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 shutdown_connections 2023-10-10T20:23:57.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f020c066f30 0x7f020c0693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02200b2320 0x7f02200b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02080051c0 0x7f022016d0e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 --2- 172.21.15.148:0/938741879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f022015c940 0x7f022015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.455+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 >> 172.21.15.148:0/938741879 conn(0x7f022006ad40 msgr2=0x7f02200cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.456+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 shutdown_connections 2023-10-10T20:23:57.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.456+0000 7f0227f1e700 1 -- 172.21.15.148:0/938741879 wait complete. 2023-10-10T20:23:57.460 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2023-10-10T20:23:57.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:23:57.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.582+0000 7fa56a32c700 1 Processor -- start 2023-10-10T20:23:57.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.582+0000 7fa56a32c700 1 -- start start 2023-10-10T20:23:57.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa56a32c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5640cdab0 0x7fa56415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa56a32c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa56415dda0 0x7fa56415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa56a32c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640cb7c0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa56a32c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56415e850 con 0x7fa56415dda0 2023-10-10T20:23:57.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa56a32c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5640cc360 con 0x7fa5640cdab0 2023-10-10T20:23:57.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa56415dda0 0x7fa56415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa56415dda0 0x7fa56415cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43486/0 (socket says 172.21.15.148:43486) 2023-10-10T20:23:57.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 -- 172.21.15.148:0/1256914261 learned_addr learned my addr 172.21.15.148:0/1256914261 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:57.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa568b29700 1 --2- 172.21.15.148:0/1256914261 >> v2:172.21.15.161:3300/0 conn(0x7fa5640cdab0 0x7fa56415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 -- 172.21.15.148:0/1256914261 >> v2:172.21.15.161:3300/0 conn(0x7fa5640cdab0 msgr2=0x7fa56415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:57.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 --2- 172.21.15.148:0/1256914261 >> v2:172.21.15.161:3300/0 conn(0x7fa5640cdab0 0x7fa56415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.583+0000 7fa569b2b700 1 -- 172.21.15.148:0/1256914261 >> v1:172.21.15.161:6789/0 conn(0x7fa5640cd6e0 legacy=0x7fa5640cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa569b2b700 1 -- 172.21.15.148:0/1256914261 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56415e290 con 0x7fa56415dda0 2023-10-10T20:23:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa569b2b700 1 --2- 172.21.15.148:0/1256914261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56415dda0 0x7fa56415cbd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa5640b6c70 tx=0x7fa55c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a547b5e97a3d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa55bfff700 1 -- 172.21.15.148:0/1256914261 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009e70 con 0x7fa56415dda0 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa55bfff700 1 -- 172.21.15.148:0/1256914261 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c00b860 con 0x7fa56415dda0 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa55bfff700 1 -- 172.21.15.148:0/1256914261 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00bc00 con 0x7fa56415dda0 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 -- 172.21.15.148:0/1256914261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56415dda0 msgr2=0x7fa56415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1256914261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56415dda0 0x7fa56415cbd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa5640b6c70 tx=0x7fa55c009250 comp rx=0 tx=0).stop 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 -- 172.21.15.148:0/1256914261 shutdown_connections 2023-10-10T20:23:57.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1256914261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56415dda0 0x7fa56415cbd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1256914261 >> v2:172.21.15.161:3300/0 conn(0x7fa5640cdab0 0x7fa56415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 -- 172.21.15.148:0/1256914261 >> 172.21.15.148:0/1256914261 conn(0x7fa564067660 msgr2=0x7fa564067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 -- 172.21.15.148:0/1256914261 shutdown_connections 2023-10-10T20:23:57.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.584+0000 7fa56a32c700 1 -- 172.21.15.148:0/1256914261 wait complete. 2023-10-10T20:23:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.585+0000 7fa56a32c700 1 Processor -- start 2023-10-10T20:23:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 -- start start 2023-10-10T20:23:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5640cdab0 0x7fa5640ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56415dda0 0x7fa56416a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c8410 con 0x7fa5640cd6e0 2023-10-10T20:23:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c8de0 con 0x7fa56415dda0 2023-10-10T20:23:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56a32c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa564158930 con 0x7fa5640cdab0 2023-10-10T20:23:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.586+0000 7fa56932a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43490/0 (socket says 172.21.15.148:43490) 2023-10-10T20:23:57.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 -- 172.21.15.148:0/1591267679 learned_addr learned my addr 172.21.15.148:0/1591267679 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:57.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa568b29700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5640cdab0 0x7fa5640ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa569b2b700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56415dda0 0x7fa56416a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 -- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5640cdab0 msgr2=0x7fa5640ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5640cdab0 0x7fa5640ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 -- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56415dda0 msgr2=0x7fa56416a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56415dda0 0x7fa56416a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0094f0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56932a700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb10 tx=0x7fa5600017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000ce20 con 0x7fa5640cd6e0 2023-10-10T20:23:57.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56000a7e0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000aba0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56416b3b0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.587+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5640c7dc0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.588+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa564089280 con 0x7fa5640cd6e0 2023-10-10T20:23:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa560003030 con 0x7fa5640cd6e0 2023-10-10T20:23:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa5597fa700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 0x7fa548069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa568b29700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 0x7fa548069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa560084510 con 0x7fa5640cd6e0 2023-10-10T20:23:57.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7fa54806ea80 con 0x7fa5640cd6e0 2023-10-10T20:23:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.589+0000 7fa568b29700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 0x7fa548069410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa56415c840 tx=0x7fa5540091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.594+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa560051780 con 0x7fa5640cd6e0 2023-10-10T20:23:57.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.738+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(86..86 src has 1..86) v4 ==== 288+0+0 (secure 0 0 0) 0x7fa560055b40 con 0x7fa5640cd6e0 2023-10-10T20:23:57.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.738+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=87}) v3 -- 0x7fa54806f3d0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.784+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5640c86f0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.785+0000 7fa5597fa700 1 -- 172.21.15.148:0/1591267679 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+15430 (secure 0 0 0) 0x7fa560054da0 con 0x7fa5640cd6e0 2023-10-10T20:23:57.787 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:23:57.788 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":86,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:23:57.734988+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:23:10.830500+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-10T20:23:11.960772+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:23:57.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 msgr2=0x7fa548069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 0x7fa548069410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa56415c840 tx=0x7fa5540091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 msgr2=0x7fa5640c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb10 tx=0x7fa5600017b0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 shutdown_connections 2023-10-10T20:23:57.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa548066f60 0x7fa548069410 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56415dda0 0x7fa56416a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5640cd6e0 0x7fa5640c88a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 --2- 172.21.15.148:0/1591267679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5640cdab0 0x7fa5640ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 >> 172.21.15.148:0/1591267679 conn(0x7fa564067660 msgr2=0x7fa5641567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 shutdown_connections 2023-10-10T20:23:57.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.787+0000 7fa56a32c700 1 -- 172.21.15.148:0/1591267679 wait complete. 2023-10-10T20:23:57.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:23:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.915+0000 7f709c031700 1 Processor -- start 2023-10-10T20:23:57.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.915+0000 7f709c031700 1 -- start start 2023-10-10T20:23:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709c031700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7094157870 0x7f709415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709c031700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f7094158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709c031700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709415cbb0 con 0x7f709415a620 2023-10-10T20:23:57.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709c031700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709415bfa0 con 0x7f7094157870 2023-10-10T20:23:57.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709c031700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709415c280 con 0x7f7094157c40 2023-10-10T20:23:57.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f7099dcd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7094157870 0x7f709415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f7099dcd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7094157870 0x7f709415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43496/0 (socket says 172.21.15.148:43496) 2023-10-10T20:23:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f7099dcd700 1 -- 172.21.15.148:0/2767569400 learned_addr learned my addr 172.21.15.148:0/2767569400 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f70995cc700 1 --1- 172.21.15.148:0/2767569400 >> v1:172.21.15.148:6789/0 conn(0x7f709415a620 0x7f709415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32792/0 (socket says 172.21.15.148:32792) 2023-10-10T20:23:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.916+0000 7f709a5ce700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f7094158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7099dcd700 1 -- 172.21.15.148:0/2767569400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 msgr2=0x7f7094158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:57.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7099dcd700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f7094158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7099dcd700 1 -- 172.21.15.148:0/2767569400 >> v1:172.21.15.148:6789/0 conn(0x7f709415a620 legacy=0x7f709415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7099dcd700 1 -- 172.21.15.148:0/2767569400 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709415c560 con 0x7f7094157870 2023-10-10T20:23:57.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7098dcb700 1 -- 172.21.15.148:0/2767569400 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877248108 0 0) 0x7f709415cbb0 con 0x7f709415a620 2023-10-10T20:23:57.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7099dcd700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7094157870 0x7f709415b6a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7090009f20 tx=0x7f7090009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2809a47683ad36 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7098dcb700 1 -- 172.21.15.148:0/2767569400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000e070 con 0x7f7094157870 2023-10-10T20:23:57.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7098dcb700 1 -- 172.21.15.148:0/2767569400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000ba80 con 0x7f7094157870 2023-10-10T20:23:57.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f7098dcb700 1 -- 172.21.15.148:0/2767569400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900092e0 con 0x7f7094157870 2023-10-10T20:23:57.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 -- 172.21.15.148:0/2767569400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7094157870 msgr2=0x7f709415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7094157870 0x7f709415b6a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7090009f20 tx=0x7f7090009840 comp rx=0 tx=0).stop 2023-10-10T20:23:57.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 -- 172.21.15.148:0/2767569400 shutdown_connections 2023-10-10T20:23:57.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f7094158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 --2- 172.21.15.148:0/2767569400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7094157870 0x7f709415b6a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 -- 172.21.15.148:0/2767569400 >> 172.21.15.148:0/2767569400 conn(0x7f7094068ce0 msgr2=0x7f70940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:57.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 -- 172.21.15.148:0/2767569400 shutdown_connections 2023-10-10T20:23:57.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.917+0000 7f709c031700 1 -- 172.21.15.148:0/2767569400 wait complete. 2023-10-10T20:23:57.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.918+0000 7f709c031700 1 Processor -- start 2023-10-10T20:23:57.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 -- start start 2023-10-10T20:23:57.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094157870 0x7f709413eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f709414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709416d5a0 con 0x7f709415a620 2023-10-10T20:23:57.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709414eda0 con 0x7f7094157870 2023-10-10T20:23:57.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709c031700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709413f980 con 0x7f7094157c40 2023-10-10T20:23:57.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709a5ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709a5ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43508/0 (socket says 172.21.15.148:43508) 2023-10-10T20:23:57.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f709a5ce700 1 -- 172.21.15.148:0/1396901312 learned_addr learned my addr 172.21.15.148:0/1396901312 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:57.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f7099dcd700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094157870 0x7f709413eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.919+0000 7f70995cc700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f709414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 -- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 msgr2=0x7f709414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f709414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 -- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094157870 msgr2=0x7f709413eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:57.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094157870 0x7f709413eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:57.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7090009ae0 con 0x7f709415a620 2023-10-10T20:23:57.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709a5ce700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d890 tx=0x7f708c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c0125f0 con 0x7f709415a620 2023-10-10T20:23:57.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c012750 con 0x7f709415a620 2023-10-10T20:23:57.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70940b2540 con 0x7f709415a620 2023-10-10T20:23:57.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c012b10 con 0x7f709415a620 2023-10-10T20:23:57.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.920+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70940b2b00 con 0x7f709415a620 2023-10-10T20:23:57.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.921+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709414ef50 con 0x7f709415a620 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.921+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f708c013480 con 0x7f709415a620 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.922+0000 7f7089ffb700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 0x7f70780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.922+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f708c084860 con 0x7f709415a620 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.922+0000 7f7099dcd700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 0x7f70780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.922+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=87}) v3 -- 0x7f707806ea50 con 0x7f709415a620 2023-10-10T20:23:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.922+0000 7f7099dcd700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 0x7f70780693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7090009880 tx=0x7f7090018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:57.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:57.926+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f708c055560 con 0x7f709415a620 2023-10-10T20:23:58.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.150+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f709404c430 con 0x7f709415a620 2023-10-10T20:23:58.153 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:23:58.151+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:23:58.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.154+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(87..87 src has 1..87) v4 ==== 366+0+0 (secure 0 0 0) 0x7f708c055790 con 0x7f709415a620 2023-10-10T20:23:58.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.154+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f707806f3a0 con 0x7f709415a620 2023-10-10T20:23:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.157+0000 7f7089ffb700 1 -- 172.21.15.148:0/1396901312 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f709404c430 con 0x7f709415a620 2023-10-10T20:23:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 msgr2=0x7f70780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 0x7f70780693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7090009880 tx=0x7f7090018000 comp rx=0 tx=0).stop 2023-10-10T20:23:58.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 msgr2=0x7f70941510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d890 tx=0x7f708c008040 comp rx=0 tx=0).stop 2023-10-10T20:23:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 shutdown_connections 2023-10-10T20:23:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7078066f30 0x7f70780693e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f709415a620 0x7f70941510f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094157870 0x7f709413eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 --2- 172.21.15.148:0/1396901312 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7094157c40 0x7f709414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 >> 172.21.15.148:0/1396901312 conn(0x7f7094068ce0 msgr2=0x7f709415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 shutdown_connections 2023-10-10T20:23:58.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.159+0000 7f709c031700 1 -- 172.21.15.148:0/1396901312 wait complete. 2023-10-10T20:23:58.165 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:23:58.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:23:58.191 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:23:58.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.289+0000 7fbb5c293700 1 Processor -- start 2023-10-10T20:23:58.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.289+0000 7fbb5c293700 1 -- start start 2023-10-10T20:23:58.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5c293700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 0x7fbb5415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5c293700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 0x7fbb540cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5c293700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb54157010 con 0x7fbb540cb670 2023-10-10T20:23:58.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5c293700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540cbdf0 con 0x7fbb54156c40 2023-10-10T20:23:58.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5c293700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540ccb40 con 0x7fbb540cc160 2023-10-10T20:23:58.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5982e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb540cb670 0x7fbb540c98a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32800/0 (socket says 172.21.15.148:32800) 2023-10-10T20:23:58.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5982e700 1 -- 172.21.15.148:0/1376640119 learned_addr learned my addr 172.21.15.148:0/1376640119 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:23:58.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5a830700 1 --2- 172.21.15.148:0/1376640119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 0x7fbb540cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.290+0000 7fbb5a02f700 1 --2- 172.21.15.148:0/1376640119 >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 0x7fbb5415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5902d700 1 -- 172.21.15.148:0/1376640119 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627592764 0 0) 0x7fbb54157010 con 0x7fbb540cb670 2023-10-10T20:23:58.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5902d700 1 -- 172.21.15.148:0/1376640119 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40003570 con 0x7fbb540cb670 2023-10-10T20:23:58.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a830700 1 -- 172.21.15.148:0/1376640119 >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 msgr2=0x7fbb5415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:58.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a830700 1 --2- 172.21.15.148:0/1376640119 >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 0x7fbb5415b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a830700 1 -- 172.21.15.148:0/1376640119 >> v1:172.21.15.148:6789/0 conn(0x7fbb540cb670 legacy=0x7fbb540c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a830700 1 -- 172.21.15.148:0/1376640119 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540c9410 con 0x7fbb540cc160 2023-10-10T20:23:58.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a02f700 1 --2- 172.21.15.148:0/1376640119 >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 0x7fbb5415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:23:58.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5a830700 1 --2- 172.21.15.148:0/1376640119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 0x7fbb540cc600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009e60 tx=0x7fbb4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf8f928b17b8571 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:58.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5902d700 1 -- 172.21.15.148:0/1376640119 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00e070 con 0x7fbb540cc160 2023-10-10T20:23:58.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5902d700 1 -- 172.21.15.148:0/1376640119 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c00bab0 con 0x7fbb540cc160 2023-10-10T20:23:58.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.291+0000 7fbb5902d700 1 -- 172.21.15.148:0/1376640119 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00be70 con 0x7fbb540cc160 2023-10-10T20:23:58.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 -- 172.21.15.148:0/1376640119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 msgr2=0x7fbb540cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1376640119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 0x7fbb540cc600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009e60 tx=0x7fbb4c009780 comp rx=0 tx=0).stop 2023-10-10T20:23:58.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 -- 172.21.15.148:0/1376640119 shutdown_connections 2023-10-10T20:23:58.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1376640119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cc160 0x7fbb540cc600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1376640119 >> v2:172.21.15.161:3300/0 conn(0x7fbb54156c40 0x7fbb5415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 -- 172.21.15.148:0/1376640119 >> 172.21.15.148:0/1376640119 conn(0x7fbb540ce570 msgr2=0x7fbb540ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:58.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 -- 172.21.15.148:0/1376640119 shutdown_connections 2023-10-10T20:23:58.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.292+0000 7fbb5c293700 1 -- 172.21.15.148:0/1376640119 wait complete. 2023-10-10T20:23:58.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.293+0000 7fbb5c293700 1 Processor -- start 2023-10-10T20:23:58.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.293+0000 7fbb5c293700 1 -- start start 2023-10-10T20:23:58.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cb670 0x7fbb540b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb54156c40 0x7fbb540b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5416a310 con 0x7fbb540cc160 2023-10-10T20:23:58.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5416b130 con 0x7fbb54156c40 2023-10-10T20:23:58.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5416ade0 con 0x7fbb540cb670 2023-10-10T20:23:58.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43528/0 (socket says 172.21.15.148:43528) 2023-10-10T20:23:58.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 -- 172.21.15.148:0/1853613369 learned_addr learned my addr 172.21.15.148:0/1853613369 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:58.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5a02f700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cb670 0x7fbb540b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5a830700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb54156c40 0x7fbb540b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 -- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cb670 msgr2=0x7fbb540b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cb670 0x7fbb540b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 -- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb54156c40 msgr2=0x7fbb540b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb54156c40 0x7fbb540b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009420 con 0x7fbb540cc160 2023-10-10T20:23:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5982e700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c7c0 tx=0x7fbb44007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:58.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44005c00 con 0x7fbb540cc160 2023-10-10T20:23:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44005d60 con 0x7fbb540cc160 2023-10-10T20:23:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c009a20 con 0x7fbb540cc160 2023-10-10T20:23:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44007320 con 0x7fbb540cc160 2023-10-10T20:23:58.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.294+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5416a820 con 0x7fbb540cc160 2023-10-10T20:23:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.295+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5416b2a0 con 0x7fbb540cc160 2023-10-10T20:23:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.296+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb4401b3b0 con 0x7fbb540cc160 2023-10-10T20:23:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.296+0000 7fbb4a7fc700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:58.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.296+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbb44083a60 con 0x7fbb540cc160 2023-10-10T20:23:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.296+0000 7fbb5a02f700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.296+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7fbb3806eaf0 con 0x7fbb540cc160 2023-10-10T20:23:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.297+0000 7fbb5a02f700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 0x7fbb380693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbb54160e00 tx=0x7fbb500091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:58.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.301+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb44017070 con 0x7fbb540cc160 2023-10-10T20:23:58.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:58.526+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb5404c3c0 con 0x7fbb540cc160 2023-10-10T20:23:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.157+0000 7fbb4a7fc700 1 -- 172.21.15.148:0/1853613369 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb44050c80 con 0x7fbb540cc160 2023-10-10T20:23:59.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 msgr2=0x7fbb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 0x7fbb380693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbb54160e00 tx=0x7fbb500091a0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 msgr2=0x7fbb54169dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c7c0 tx=0x7fbb44007dd0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 shutdown_connections 2023-10-10T20:23:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb54156c40 0x7fbb540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbb540cb670 0x7fbb540b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 --2- 172.21.15.148:0/1853613369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb540cc160 0x7fbb54169dd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 >> 172.21.15.148:0/1853613369 conn(0x7fbb540ce570 msgr2=0x7fbb54152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:59.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 shutdown_connections 2023-10-10T20:23:59.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.160+0000 7fbb5c293700 1 -- 172.21.15.148:0/1853613369 wait complete. 2023-10-10T20:23:59.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:23:59.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.332+0000 7f542e95e700 1 Processor -- start 2023-10-10T20:23:59.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.332+0000 7f542e95e700 1 -- start start 2023-10-10T20:23:59.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f542e95e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b36c0 0x7f5428158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f542e95e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5428152520 0x7f54280ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f542e95e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5428159350 con 0x7f54280b1950 2023-10-10T20:23:59.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f542e95e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280b1110 con 0x7f54280b36c0 2023-10-10T20:23:59.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f542e95e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280ae4e0 con 0x7f5428152520 2023-10-10T20:23:59.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f5427fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b36c0 0x7f5428158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f5427fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b36c0 0x7f5428158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43540/0 (socket says 172.21.15.148:43540) 2023-10-10T20:23:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.333+0000 7f5427fff700 1 -- 172.21.15.148:0/2211934352 learned_addr learned my addr 172.21.15.148:0/2211934352 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f542cefb700 1 --2- 172.21.15.148:0/2211934352 >> v2:172.21.15.161:3300/0 conn(0x7f5428152520 0x7f54280ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5427fff700 1 -- 172.21.15.148:0/2211934352 >> v2:172.21.15.161:3300/0 conn(0x7f5428152520 msgr2=0x7f54280ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:23:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5427fff700 1 --2- 172.21.15.148:0/2211934352 >> v2:172.21.15.161:3300/0 conn(0x7f5428152520 0x7f54280ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5427fff700 1 -- 172.21.15.148:0/2211934352 >> v1:172.21.15.161:6789/0 conn(0x7f54280b1950 legacy=0x7f5428151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5427fff700 1 -- 172.21.15.148:0/2211934352 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280b1ea0 con 0x7f54280b36c0 2023-10-10T20:23:59.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5427fff700 1 --2- 172.21.15.148:0/2211934352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b36c0 0x7f5428158e10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5428155f10 tx=0x7f5418009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ada4c2b7816b175 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:59.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5426ffd700 1 -- 172.21.15.148:0/2211934352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418009e70 con 0x7f54280b36c0 2023-10-10T20:23:59.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5426ffd700 1 -- 172.21.15.148:0/2211934352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800b860 con 0x7f54280b36c0 2023-10-10T20:23:59.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.334+0000 7f5426ffd700 1 -- 172.21.15.148:0/2211934352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800bc00 con 0x7f54280b36c0 2023-10-10T20:23:59.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 -- 172.21.15.148:0/2211934352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b36c0 msgr2=0x7f5428158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 --2- 172.21.15.148:0/2211934352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b36c0 0x7f5428158e10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5428155f10 tx=0x7f5418009250 comp rx=0 tx=0).stop 2023-10-10T20:23:59.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 -- 172.21.15.148:0/2211934352 shutdown_connections 2023-10-10T20:23:59.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 --2- 172.21.15.148:0/2211934352 >> v2:172.21.15.161:3300/0 conn(0x7f5428152520 0x7f54280ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 --2- 172.21.15.148:0/2211934352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b36c0 0x7f5428158e10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 -- 172.21.15.148:0/2211934352 >> 172.21.15.148:0/2211934352 conn(0x7f5428069870 msgr2=0x7f5428069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:23:59.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 -- 172.21.15.148:0/2211934352 shutdown_connections 2023-10-10T20:23:59.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.335+0000 7f542e95e700 1 -- 172.21.15.148:0/2211934352 wait complete. 2023-10-10T20:23:59.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.336+0000 7f542e95e700 1 Processor -- start 2023-10-10T20:23:59.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.336+0000 7f542e95e700 1 -- start start 2023-10-10T20:23:59.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280b36c0 0x7f5428157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428152520 0x7f5428157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a6460 con 0x7f54280b1950 2023-10-10T20:23:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a6740 con 0x7f54280b36c0 2023-10-10T20:23:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280a6a20 con 0x7f5428152520 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43542/0 (socket says 172.21.15.148:43542) 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 -- 172.21.15.148:0/967239686 learned_addr learned my addr 172.21.15.148:0/967239686 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f54277fe700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280b36c0 0x7f5428157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542cefb700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428152520 0x7f5428157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 -- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428152520 msgr2=0x7f5428157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428152520 0x7f5428157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 -- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280b36c0 msgr2=0x7f5428157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:23:59.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280b36c0 0x7f5428157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:23:59.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180094f0 con 0x7f54280b1950 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f5427fff700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f54281594e0 tx=0x7f5418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011420 con 0x7f54280b1950 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418011580 con 0x7f54280b1950 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54280a6d00 con 0x7f54280b1950 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418011940 con 0x7f54280b1950 2023-10-10T20:23:59.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.337+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428169920 con 0x7f54280b1950 2023-10-10T20:23:59.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.338+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54280a7220 con 0x7f54280b1950 2023-10-10T20:23:59.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.339+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f541802c3b0 con 0x7f54280b1950 2023-10-10T20:23:59.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.339+0000 7f540ffff700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:23:59.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.339+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5418085c30 con 0x7f54280b1950 2023-10-10T20:23:59.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.339+0000 7f54277fe700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:23:59.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.339+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f541406eaf0 con 0x7f54280b1950 2023-10-10T20:23:59.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.340+0000 7f54277fe700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5428151dc0 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:23:59.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.343+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54280a7220 con 0x7f54280b1950 2023-10-10T20:23:59.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:23:59.525+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f542806b900 con 0x7f54280b1950 2023-10-10T20:24:00.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.168+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(88..88 src has 1..88) v4 ==== 296+0+0 (secure 0 0 0) 0x7f5418057210 con 0x7f54280b1950 2023-10-10T20:24:00.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.168+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f541406cf20 con 0x7f54280b1950 2023-10-10T20:24:00.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.170+0000 7f540ffff700 1 -- 172.21.15.148:0/967239686 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5418015030 con 0x7f54280b1950 2023-10-10T20:24:00.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.176+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 msgr2=0x7f54140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.176+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5428151dc0 tx=0x7f541c009040 comp rx=0 tx=0).stop 2023-10-10T20:24:00.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.176+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 msgr2=0x7f5428151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.176+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f54281594e0 tx=0x7f5418017000 comp rx=0 tx=0).stop 2023-10-10T20:24:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 shutdown_connections 2023-10-10T20:24:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428152520 0x7f5428157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b1950 0x7f5428151880 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 --2- 172.21.15.148:0/967239686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280b36c0 0x7f5428157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 >> 172.21.15.148:0/967239686 conn(0x7f5428069870 msgr2=0x7f542806d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:00.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 shutdown_connections 2023-10-10T20:24:00.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.177+0000 7f542e95e700 1 -- 172.21.15.148:0/967239686 wait complete. 2023-10-10T20:24:00.182 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:24:00.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:24:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.308+0000 7ff6b43ce700 1 Processor -- start 2023-10-10T20:24:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.308+0000 7ff6b43ce700 1 -- start start 2023-10-10T20:24:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b43ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff6ac164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b43ce700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0ceb40 con 0x7ff6ac157b00 2023-10-10T20:24:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b43ce700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0cf3e0 con 0x7ff6ac06acf0 2023-10-10T20:24:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b43ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15b200 con 0x7ff6ac15add0 2023-10-10T20:24:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b216a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff6ac157b00 0x7ff6ac06a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32810/0 (socket says 172.21.15.148:32810) 2023-10-10T20:24:00.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.309+0000 7ff6b216a700 1 -- 172.21.15.148:0/807856841 learned_addr learned my addr 172.21.15.148:0/807856841 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:00.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b296b700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff6ac164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918831716 0 0) 0x7ff6ac0ceb40 con 0x7ff6ac157b00 2023-10-10T20:24:00.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff698003570 con 0x7ff6ac157b00 2023-10-10T20:24:00.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116507074 0 0) 0x7ff698003570 con 0x7ff6ac157b00 2023-10-10T20:24:00.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 msgr2=0x7ff6ac164fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:00.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff6ac164fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 >> v1:172.21.15.161:6789/0 conn(0x7ff6ac06acf0 legacy=0x7ff6ac15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac15b7f0 con 0x7ff6ac157b00 2023-10-10T20:24:00.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7ff6a80017f0 con 0x7ff6ac157b00 2023-10-10T20:24:00.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 >> v1:172.21.15.148:6789/0 conn(0x7ff6ac157b00 legacy=0x7ff6ac06a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff698003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff698004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff698004d50 0x7ff6980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15b200 con 0x7ff698003fd0 2023-10-10T20:24:00.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800bb40 con 0x7ff698004d50 2023-10-10T20:24:00.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800bd00 con 0x7ff6ac15add0 2023-10-10T20:24:00.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff698004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b216a700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff698003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b296b700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff698004d50 0x7ff6980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 -- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 msgr2=0x7ff698003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:00.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac15add0 0x7ff698003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 -- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff698004d50 msgr2=0x7ff6980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:00.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff698004d50 0x7ff6980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.310+0000 7ff6b1969700 1 -- 172.21.15.148:0/807856841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698003570 con 0x7ff698003fd0 2023-10-10T20:24:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b1969700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff698004810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0019a0 tx=0x7ff69c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c42fd79b327750 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:00.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00c5d0 con 0x7ff698003fd0 2023-10-10T20:24:00.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00c750 con 0x7ff698003fd0 2023-10-10T20:24:00.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b1168700 1 -- 172.21.15.148:0/807856841 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00cb10 con 0x7ff698003fd0 2023-10-10T20:24:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 -- 172.21.15.148:0/807856841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 msgr2=0x7ff698004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/807856841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff698004810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0019a0 tx=0x7ff69c007d10 comp rx=0 tx=0).stop 2023-10-10T20:24:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b296b700 1 -- 172.21.15.148:0/807856841 reap_dead start 2023-10-10T20:24:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 -- 172.21.15.148:0/807856841 shutdown_connections 2023-10-10T20:24:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 -- 172.21.15.148:0/807856841 >> 172.21.15.148:0/807856841 conn(0x7ff6ac0d1a10 msgr2=0x7ff6ac068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 -- 172.21.15.148:0/807856841 shutdown_connections 2023-10-10T20:24:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.311+0000 7ff6b43ce700 1 -- 172.21.15.148:0/807856841 wait complete. 2023-10-10T20:24:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.312+0000 7ff6b43ce700 1 Processor -- start 2023-10-10T20:24:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.312+0000 7ff6b43ce700 1 -- start start 2023-10-10T20:24:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac0c7350 0x7ff6ac0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac0b4d20 0x7ff6ac0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0c8020 con 0x7ff698003fd0 2023-10-10T20:24:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0c7d00 con 0x7ff6ac0b4d20 2023-10-10T20:24:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b43ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac159b30 con 0x7ff6ac0c7350 2023-10-10T20:24:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43554/0 (socket says 172.21.15.148:43554) 2023-10-10T20:24:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 -- 172.21.15.148:0/3777046745 learned_addr learned my addr 172.21.15.148:0/3777046745 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b296b700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac0b4d20 0x7ff6ac0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b1969700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac0c7350 0x7ff6ac0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 -- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac0c7350 msgr2=0x7ff6ac0c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac0c7350 0x7ff6ac0c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 -- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac0b4d20 msgr2=0x7ff6ac0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac0b4d20 0x7ff6ac0b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.313+0000 7ff6b216a700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac159fc0 con 0x7ff698003fd0 2023-10-10T20:24:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6b216a700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007ac0 tx=0x7ff6a80021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ae60 con 0x7ff698003fd0 2023-10-10T20:24:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a80133f0 con 0x7ff698003fd0 2023-10-10T20:24:00.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac158f20 con 0x7ff698003fd0 2023-10-10T20:24:00.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8013790 con 0x7ff698003fd0 2023-10-10T20:24:00.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.314+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1597c0 con 0x7ff698003fd0 2023-10-10T20:24:00.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.315+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac04c3c0 con 0x7ff698003fd0 2023-10-10T20:24:00.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.315+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6a8013b00 con 0x7ff698003fd0 2023-10-10T20:24:00.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.315+0000 7ff6a27fc700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 0x7ff690069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:00.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.315+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5353+0+0 (secure 0 0 0) 0x7ff6a8085240 con 0x7ff698003fd0 2023-10-10T20:24:00.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.316+0000 7ff6b1969700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 0x7ff690069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.316+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7ff69006dd20 con 0x7ff698003fd0 2023-10-10T20:24:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.316+0000 7ff6b1969700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 0x7ff690069410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0c7f70 tx=0x7ff69c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.320+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6a8017020 con 0x7ff698003fd0 2023-10-10T20:24:00.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:00.503+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6ac0c5d10 con 0x7ff698003fd0 2023-10-10T20:24:01.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.167+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(89..89 src has 1..89) v4 ==== 288+0+0 (secure 0 0 0) 0x7ff6a8085080 con 0x7ff698003fd0 2023-10-10T20:24:01.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.167+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7ff69006cab0 con 0x7ff698003fd0 2023-10-10T20:24:01.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.169+0000 7ff6a27fc700 1 -- 172.21.15.148:0/3777046745 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff6a80560a0 con 0x7ff698003fd0 2023-10-10T20:24:01.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 msgr2=0x7ff690069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 0x7ff690069410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0c7f70 tx=0x7ff69c015000 comp rx=0 tx=0).stop 2023-10-10T20:24:01.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 msgr2=0x7ff6ac1448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007ac0 tx=0x7ff6a80021f0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 shutdown_connections 2023-10-10T20:24:01.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff690066f60 0x7ff690069410 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0c7f70 tx=0x7ff69c015000 comp rx=0 tx=0).stop 2023-10-10T20:24:01.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6ac0b4d20 0x7ff6ac0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698003fd0 0x7ff6ac1448d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 --2- 172.21.15.148:0/3777046745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6ac0c7350 0x7ff6ac0c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.176+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 >> 172.21.15.148:0/3777046745 conn(0x7ff6ac0d1a10 msgr2=0x7ff6ac0695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.177+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 shutdown_connections 2023-10-10T20:24:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.178+0000 7ff6b43ce700 1 -- 172.21.15.148:0/3777046745 wait complete. 2023-10-10T20:24:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:24:01.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:24:01.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.310+0000 7f0e026ba700 1 Processor -- start 2023-10-10T20:24:01.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.310+0000 7f0e026ba700 1 -- start start 2023-10-10T20:24:01.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e026ba700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 0x7f0dfc15db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e026ba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0dfc0c9390 0x7f0dfc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e026ba700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0cae70 con 0x7f0dfc0cbf10 2023-10-10T20:24:01.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e026ba700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0cb150 con 0x7f0dfc0c9390 2023-10-10T20:24:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e026ba700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0cb430 con 0x7f0dfc0678b0 2023-10-10T20:24:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e00c57700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0dfc0c9390 0x7f0dfc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e00c57700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0dfc0c9390 0x7f0dfc15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43558/0 (socket says 172.21.15.148:43558) 2023-10-10T20:24:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.311+0000 7f0e00c57700 1 -- 172.21.15.148:0/203155333 learned_addr learned my addr 172.21.15.148:0/203155333 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0e00c57700 1 -- 172.21.15.148:0/203155333 >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 msgr2=0x7f0dfc15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:01.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/203155333 >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 0x7f0dfc15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0e00c57700 1 --2- 172.21.15.148:0/203155333 >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 0x7f0dfc15db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0e00c57700 1 -- 172.21.15.148:0/203155333 >> v1:172.21.15.161:6789/0 conn(0x7f0dfc0cbf10 legacy=0x7f0dfc15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:01.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0e00c57700 1 -- 172.21.15.148:0/203155333 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0cb710 con 0x7f0dfc0c9390 2023-10-10T20:24:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/203155333 >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 0x7f0dfc15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0e00c57700 1 --2- 172.21.15.148:0/203155333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc15a370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc0c5b10 tx=0x7f0df4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d050b09781b3f876 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0dfaffd700 1 -- 172.21.15.148:0/203155333 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df400e070 con 0x7f0dfc0c9390 2023-10-10T20:24:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0dfaffd700 1 -- 172.21.15.148:0/203155333 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df400ba10 con 0x7f0dfc0c9390 2023-10-10T20:24:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.312+0000 7f0dfaffd700 1 -- 172.21.15.148:0/203155333 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40092e0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 -- 172.21.15.148:0/203155333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 msgr2=0x7f0dfc15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 --2- 172.21.15.148:0/203155333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc15a370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc0c5b10 tx=0x7f0df4009840 comp rx=0 tx=0).stop 2023-10-10T20:24:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 -- 172.21.15.148:0/203155333 shutdown_connections 2023-10-10T20:24:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 --2- 172.21.15.148:0/203155333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc15a370 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 --2- 172.21.15.148:0/203155333 >> v2:172.21.15.161:3300/0 conn(0x7f0dfc0678b0 0x7f0dfc15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 -- 172.21.15.148:0/203155333 >> 172.21.15.148:0/203155333 conn(0x7f0dfc069870 msgr2=0x7f0dfc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 -- 172.21.15.148:0/203155333 shutdown_connections 2023-10-10T20:24:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.313+0000 7f0e026ba700 1 -- 172.21.15.148:0/203155333 wait complete. 2023-10-10T20:24:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.314+0000 7f0e026ba700 1 Processor -- start 2023-10-10T20:24:01.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.314+0000 7f0e026ba700 1 -- start start 2023-10-10T20:24:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 0x7f0dfc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dfc0cbf10 0x7f0dfc16dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0b3d40 con 0x7f0dfc0c9390 2023-10-10T20:24:01.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc15cea0 con 0x7f0dfc0cbf10 2023-10-10T20:24:01.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc16a3e0 con 0x7f0dfc0678b0 2023-10-10T20:24:01.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43568/0 (socket says 172.21.15.148:43568) 2023-10-10T20:24:01.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 -- 172.21.15.148:0/4003173905 learned_addr learned my addr 172.21.15.148:0/4003173905 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:01.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e00c57700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dfc0cbf10 0x7f0dfc16dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 0x7f0dfc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 -- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 msgr2=0x7f0dfc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 0x7f0dfc15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 -- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dfc0cbf10 msgr2=0x7f0dfc16dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dfc0cbf10 0x7f0dfc16dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4009ae0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 0x7f0dfc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0dfb7fe700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c5d0 tx=0x7f0df0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0011590 con 0x7f0dfc0c9390 2023-10-10T20:24:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0011710 con 0x7f0dfc0c9390 2023-10-10T20:24:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc0b2eb0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0011ad0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.315+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc0b36f0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.316+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc04c3c0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.317+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0df001c460 con 0x7f0dfc0c9390 2023-10-10T20:24:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.317+0000 7f0de3fff700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.317+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0df0083f90 con 0x7f0dfc0c9390 2023-10-10T20:24:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.317+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.317+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f0de806c6b0 con 0x7f0dfc0c9390 2023-10-10T20:24:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.318+0000 7f0dfbfff700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0dec006fd0 tx=0x7f0dec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.321+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0df0051500 con 0x7f0dfc0c9390 2023-10-10T20:24:01.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.506+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0dfc157980 con 0x7f0dfc0c9390 2023-10-10T20:24:01.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.510+0000 7f0de3fff700 1 -- 172.21.15.148:0/4003173905 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0df0054b20 con 0x7f0dfc0c9390 2023-10-10T20:24:01.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 msgr2=0x7f0de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 0x7f0de80693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0dec006fd0 tx=0x7f0dec008040 comp rx=0 tx=0).stop 2023-10-10T20:24:01.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 msgr2=0x7f0dfc0b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c5d0 tx=0x7f0df0007f60 comp rx=0 tx=0).stop 2023-10-10T20:24:01.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 shutdown_connections 2023-10-10T20:24:01.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0de8066f30 0x7f0de80693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dfc0cbf10 0x7f0dfc16dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0dfc0678b0 0x7f0dfc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 --2- 172.21.15.148:0/4003173905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0dfc0c9390 0x7f0dfc0b2970 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 >> 172.21.15.148:0/4003173905 conn(0x7f0dfc069870 msgr2=0x7f0dfc158670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:01.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 shutdown_connections 2023-10-10T20:24:01.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.512+0000 7f0e026ba700 1 -- 172.21.15.148:0/4003173905 wait complete. 2023-10-10T20:24:01.527 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-10T20:24:01.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 Processor -- start 2023-10-10T20:24:01.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 -- start start 2023-10-10T20:24:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 0x7f59b40b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b40c5d80 con 0x7f59b40cfe20 2023-10-10T20:24:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b40c6060 con 0x7f59b40d1c30 2023-10-10T20:24:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.640+0000 7f59ba04a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4068470 con 0x7f59b415a2b0 2023-10-10T20:24:01.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59b40cfe20 0x7f59b40c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:32812/0 (socket says 172.21.15.148:32812) 2023-10-10T20:24:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b3fff700 1 -- 172.21.15.148:0/4180825890 learned_addr learned my addr 172.21.15.148:0/4180825890 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b37fe700 1 --2- 172.21.15.148:0/4180825890 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 0x7f59b40b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498756879 0 0) 0x7f59b40c5d80 con 0x7f59b40cfe20 2023-10-10T20:24:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f599c003570 con 0x7f59b40cfe20 2023-10-10T20:24:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091547713 0 0) 0x7f59b40c6060 con 0x7f59b40d1c30 2023-10-10T20:24:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b40c5d80 con 0x7f59b40d1c30 2023-10-10T20:24:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b37fe700 1 -- 172.21.15.148:0/4180825890 >> v1:172.21.15.161:6789/0 conn(0x7f59b40d1c30 legacy=0x7f59b40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b37fe700 1 -- 172.21.15.148:0/4180825890 >> v1:172.21.15.148:6789/0 conn(0x7f59b40cfe20 legacy=0x7f59b40c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b37fe700 1 -- 172.21.15.148:0/4180825890 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40c6340 con 0x7f59b415a2b0 2023-10-10T20:24:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b37fe700 1 --2- 172.21.15.148:0/4180825890 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 0x7f59b40b3da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ee0 tx=0x7f59a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e9ac7790ce63de server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400e070 con 0x7f59b415a2b0 2023-10-10T20:24:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400bb80 con 0x7f59b415a2b0 2023-10-10T20:24:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.641+0000 7f59b27fc700 1 -- 172.21.15.148:0/4180825890 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4012430 con 0x7f59b415a2b0 2023-10-10T20:24:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 -- 172.21.15.148:0/4180825890 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 msgr2=0x7f59b40b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 --2- 172.21.15.148:0/4180825890 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 0x7f59b40b3da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ee0 tx=0x7f59a4009780 comp rx=0 tx=0).stop 2023-10-10T20:24:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 -- 172.21.15.148:0/4180825890 shutdown_connections 2023-10-10T20:24:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 --2- 172.21.15.148:0/4180825890 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b415a2b0 0x7f59b40b3da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 -- 172.21.15.148:0/4180825890 >> 172.21.15.148:0/4180825890 conn(0x7f59b4067e40 msgr2=0x7f59b40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 -- 172.21.15.148:0/4180825890 shutdown_connections 2023-10-10T20:24:01.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.642+0000 7f59ba04a700 1 -- 172.21.15.148:0/4180825890 wait complete. 2023-10-10T20:24:01.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.643+0000 7f59ba04a700 1 Processor -- start 2023-10-10T20:24:01.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.643+0000 7f59ba04a700 1 -- start start 2023-10-10T20:24:01.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b40cfe20 0x7f59b4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 0x7f59b415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4160bc0 con 0x7f59b40d1c30 2023-10-10T20:24:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4159d90 con 0x7f59b415a2b0 2023-10-10T20:24:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59ba04a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415f560 con 0x7f59b40cfe20 2023-10-10T20:24:01.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43578/0 (socket says 172.21.15.148:43578) 2023-10-10T20:24:01.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 -- 172.21.15.148:0/264928264 learned_addr learned my addr 172.21.15.148:0/264928264 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:01.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b37fe700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b40cfe20 0x7f59b4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b3fff700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 0x7f59b415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 -- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b40cfe20 msgr2=0x7f59b4159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b40cfe20 0x7f59b4159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 -- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 msgr2=0x7f59b415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:01.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 0x7f59b415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:01.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.644+0000 7f59b2ffd700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4009a20 con 0x7f59b40d1c30 2023-10-10T20:24:01.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f59b3fff700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 0x7f59b415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:01.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f59b2ffd700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c5c0 tx=0x7f59a8007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010510 con 0x7f59b40d1c30 2023-10-10T20:24:01.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a8010690 con 0x7f59b40d1c30 2023-10-10T20:24:01.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b415f840 con 0x7f59b40d1c30 2023-10-10T20:24:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010a50 con 0x7f59b40d1c30 2023-10-10T20:24:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.645+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4160520 con 0x7f59b40d1c30 2023-10-10T20:24:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.646+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b4160d70 con 0x7f59b40d1c30 2023-10-10T20:24:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.646+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59a801b3b0 con 0x7f59b40d1c30 2023-10-10T20:24:01.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.647+0000 7f599b7fe700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 0x7f59a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:01.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.647+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f59a8083a30 con 0x7f59b40d1c30 2023-10-10T20:24:01.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.647+0000 7f59b37fe700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 0x7f59a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:01.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.647+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f59a006c6b0 con 0x7f59b40d1c30 2023-10-10T20:24:01.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.647+0000 7f59b37fe700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 0x7f59a00693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f59a40099f0 tx=0x7f59a4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:01.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.651+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59a8017020 con 0x7f59b40d1c30 2023-10-10T20:24:01.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:01.909+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f59b4160f50 con 0x7f59b40d1c30 2023-10-10T20:24:02.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.171+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(90..90 src has 1..90) v4 ==== 658+0+0 (secure 0 0 0) 0x7f59a8055360 con 0x7f59b40d1c30 2023-10-10T20:24:02.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.171+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f59a006d130 con 0x7f59b40d1c30 2023-10-10T20:24:02.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.173+0000 7f599b7fe700 1 -- 172.21.15.148:0/264928264 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v237) v1 ==== 133+0+0 (secure 0 0 0) 0x7f59b4160f50 con 0x7f59b40d1c30 2023-10-10T20:24:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 msgr2=0x7f59a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 0x7f59a00693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f59a40099f0 tx=0x7f59a4019000 comp rx=0 tx=0).stop 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 msgr2=0x7f59b415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c5c0 tx=0x7f59a8007de0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 shutdown_connections 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f59a0066f30 0x7f59a00693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59b415a2b0 0x7f59b415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59b40cfe20 0x7f59b4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 --2- 172.21.15.148:0/264928264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b40d1c30 0x7f59b415b310 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.175+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 >> 172.21.15.148:0/264928264 conn(0x7f59b4067e40 msgr2=0x7f59b406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:02.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.176+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 shutdown_connections 2023-10-10T20:24:02.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.176+0000 7f59ba04a700 1 -- 172.21.15.148:0/264928264 wait complete. 2023-10-10T20:24:02.191 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-10T20:24:02.192 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-10T20:24:02.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-10T20:24:02.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 Processor -- start 2023-10-10T20:24:02.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 -- start start 2023-10-10T20:24:02.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f91c4157870 0x7f91c415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415a620 0x7f91c415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c415cbb0 con 0x7f91c415d580 2023-10-10T20:24:02.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c415bfa0 con 0x7f91c4157870 2023-10-10T20:24:02.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.306+0000 7f91c94f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c415c280 con 0x7f91c415a620 2023-10-10T20:24:02.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f91c4157870 0x7f91c415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f91c4157870 0x7f91c415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43594/0 (socket says 172.21.15.148:43594) 2023-10-10T20:24:02.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 -- 172.21.15.148:0/469819516 learned_addr learned my addr 172.21.15.148:0/469819516 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:02.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91ba7fc700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415a620 0x7f91c415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 -- 172.21.15.148:0/469819516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415a620 msgr2=0x7f91c415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415a620 0x7f91c415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 -- 172.21.15.148:0/469819516 >> v1:172.21.15.161:6789/0 conn(0x7f91c415d580 legacy=0x7f91c4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 -- 172.21.15.148:0/469819516 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c415c560 con 0x7f91c4157870 2023-10-10T20:24:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c27fc700 1 -- 172.21.15.148:0/469819516 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534981323 0 0) 0x7f91c415cbb0 con 0x7f91c415d580 2023-10-10T20:24:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4157870 0x7f91c415b6a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099b0 tx=0x7f91b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5164f4b33067aed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c27fc700 1 -- 172.21.15.148:0/469819516 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4009ea0 con 0x7f91c4157870 2023-10-10T20:24:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c27fc700 1 -- 172.21.15.148:0/469819516 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b400b860 con 0x7f91c4157870 2023-10-10T20:24:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.307+0000 7f91c27fc700 1 -- 172.21.15.148:0/469819516 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400bc00 con 0x7f91c4157870 2023-10-10T20:24:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 -- 172.21.15.148:0/469819516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4157870 msgr2=0x7f91c415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4157870 0x7f91c415b6a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099b0 tx=0x7f91b4009250 comp rx=0 tx=0).stop 2023-10-10T20:24:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 -- 172.21.15.148:0/469819516 shutdown_connections 2023-10-10T20:24:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c4157870 0x7f91c415b6a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 --2- 172.21.15.148:0/469819516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415a620 0x7f91c415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 -- 172.21.15.148:0/469819516 >> 172.21.15.148:0/469819516 conn(0x7f91c4068ce0 msgr2=0x7f91c40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 -- 172.21.15.148:0/469819516 shutdown_connections 2023-10-10T20:24:02.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.308+0000 7f91c94f4700 1 -- 172.21.15.148:0/469819516 wait complete. 2023-10-10T20:24:02.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.309+0000 7f91c94f4700 1 Processor -- start 2023-10-10T20:24:02.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.309+0000 7f91c94f4700 1 -- start start 2023-10-10T20:24:02.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 0x7f91c40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415d580 0x7f91c414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4152d40 con 0x7f91c415a620 2023-10-10T20:24:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4151eb0 con 0x7f91c4157870 2023-10-10T20:24:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c41531d0 con 0x7f91c415d580 2023-10-10T20:24:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43602/0 (socket says 172.21.15.148:43602) 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 -- 172.21.15.148:0/2538798256 learned_addr learned my addr 172.21.15.148:0/2538798256 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 0x7f91c40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c37fe700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415d580 0x7f91c414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 -- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415d580 msgr2=0x7f91c414c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415d580 0x7f91c414c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 -- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 msgr2=0x7f91c40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 0x7f91c40b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 -- 172.21.15.148:0/2538798256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b40094f0 con 0x7f91c415a620 2023-10-10T20:24:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 0x7f91c40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91ba7fc700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ec80 tx=0x7f91ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00c9d0 con 0x7f91c415a620 2023-10-10T20:24:02.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac00cb30 con 0x7f91c415a620 2023-10-10T20:24:02.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.310+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c4152130 con 0x7f91c415a620 2023-10-10T20:24:02.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.311+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac013470 con 0x7f91c415a620 2023-10-10T20:24:02.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.311+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c41526f0 con 0x7f91c415a620 2023-10-10T20:24:02.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.311+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c404c3c0 con 0x7f91c415a620 2023-10-10T20:24:02.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.312+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91ac013860 con 0x7f91c415a620 2023-10-10T20:24:02.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.312+0000 7f91bb7fe700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:02.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.312+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f91ac084560 con 0x7f91c415a620 2023-10-10T20:24:02.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.312+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:02.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.313+0000 7f91c2ffd700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 0x7f91a40693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099b0 tx=0x7f91b4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.317+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91ac051ad0 con 0x7f91c415a620 2023-10-10T20:24:02.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:02.576+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f91c415c840 con 0x7f91c415a620 2023-10-10T20:24:03.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.174+0000 7f91bb7fe700 1 -- 172.21.15.148:0/2538798256 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v238) v1 ==== 140+0+0 (secure 0 0 0) 0x7f91ac0550f0 con 0x7f91c415a620 2023-10-10T20:24:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 msgr2=0x7f91a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 0x7f91a40693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f91b40099b0 tx=0x7f91b4017000 comp rx=0 tx=0).stop 2023-10-10T20:24:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 msgr2=0x7f91c4151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00ec80 tx=0x7f91ac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 shutdown_connections 2023-10-10T20:24:03.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91c415d580 0x7f91c414c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91c4157870 0x7f91c40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 --2- 172.21.15.148:0/2538798256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91c415a620 0x7f91c4151970 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 >> 172.21.15.148:0/2538798256 conn(0x7f91c4068ce0 msgr2=0x7f91c415aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:03.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 shutdown_connections 2023-10-10T20:24:03.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.177+0000 7f91c94f4700 1 -- 172.21.15.148:0/2538798256 wait complete. 2023-10-10T20:24:03.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-10T20:24:03.321 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_objectstore":"memstore"} 2023-10-10T20:24:03.321 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-10T20:24:03.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:24:03.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.439+0000 7f12475eb700 1 Processor -- start 2023-10-10T20:24:03.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.439+0000 7f12475eb700 1 -- start start 2023-10-10T20:24:03.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f12475eb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1240159ec0 0x7f12400af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f12475eb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12400b0790 con 0x7f124006acb0 2023-10-10T20:24:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f12475eb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12400ce1f0 con 0x7f124015a290 2023-10-10T20:24:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f12475eb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12400b0b20 con 0x7f1240159ec0 2023-10-10T20:24:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f1245b88700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f124006acb0 0x7f12400cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35668/0 (socket says 172.21.15.148:35668) 2023-10-10T20:24:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.440+0000 7f1245b88700 1 -- 172.21.15.148:0/2358981647 learned_addr learned my addr 172.21.15.148:0/2358981647 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:03.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 --2- 172.21.15.148:0/2358981647 >> v2:172.21.15.161:3300/0 conn(0x7f1240159ec0 0x7f12400af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330610136 0 0) 0x7f12400b0790 con 0x7f124006acb0 2023-10-10T20:24:03.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f122c003570 con 0x7f124006acb0 2023-10-10T20:24:03.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707381023 0 0) 0x7f122c003570 con 0x7f124006acb0 2023-10-10T20:24:03.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 >> v2:172.21.15.161:3300/0 conn(0x7f1240159ec0 msgr2=0x7f12400af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:03.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 --2- 172.21.15.148:0/2358981647 >> v2:172.21.15.161:3300/0 conn(0x7f1240159ec0 0x7f12400af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 >> v1:172.21.15.161:6789/0 conn(0x7f124015a290 legacy=0x7f124006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400b04b0 con 0x7f124006acb0 2023-10-10T20:24:03.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f12380017f0 con 0x7f124006acb0 2023-10-10T20:24:03.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 >> v1:172.21.15.148:6789/0 conn(0x7f124006acb0 legacy=0x7f12400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 0x7f122c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c004810 0x7f122c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c0051c0 0x7f122c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12400b0b20 con 0x7f122c003a60 2023-10-10T20:24:03.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c00c160 con 0x7f122c004810 2023-10-10T20:24:03.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c00c320 con 0x7f122c0051c0 2023-10-10T20:24:03.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 0x7f122c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1244b86700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c004810 0x7f122c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245b88700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c0051c0 0x7f122c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 -- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c0051c0 msgr2=0x7f122c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:03.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c0051c0 0x7f122c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 -- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c004810 msgr2=0x7f122c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:03.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c004810 0x7f122c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245387700 1 -- 172.21.15.148:0/2358981647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c003570 con 0x7f122c003a60 2023-10-10T20:24:03.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245b88700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c0051c0 0x7f122c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:03.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.441+0000 7f1245b88700 1 -- 172.21.15.148:0/2358981647 reap_dead start 2023-10-10T20:24:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f1245387700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 0x7f122c003e70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f123c009c00 tx=0x7f123c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c938c4983ac69a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00e710 con 0x7f122c003a60 2023-10-10T20:24:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00e870 con 0x7f122c003a60 2023-10-10T20:24:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f1237fff700 1 -- 172.21.15.148:0/2358981647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00ec30 con 0x7f122c003a60 2023-10-10T20:24:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 -- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 msgr2=0x7f122c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 0x7f122c003e70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f123c009c00 tx=0x7f123c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 -- 172.21.15.148:0/2358981647 shutdown_connections 2023-10-10T20:24:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 --2- 172.21.15.148:0/2358981647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f122c003a60 0x7f122c003e70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 -- 172.21.15.148:0/2358981647 >> 172.21.15.148:0/2358981647 conn(0x7f12400d1880 msgr2=0x7f12400cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:03.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 -- 172.21.15.148:0/2358981647 shutdown_connections 2023-10-10T20:24:03.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.442+0000 7f12475eb700 1 -- 172.21.15.148:0/2358981647 wait complete. 2023-10-10T20:24:03.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.443+0000 7f12475eb700 1 Processor -- start 2023-10-10T20:24:03.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.443+0000 7f12475eb700 1 -- start start 2023-10-10T20:24:03.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c003a60 0x7f124016b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12400cfdb0 0x7f124014d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124016b710 con 0x7f12400cf940 2023-10-10T20:24:03.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124014da40 con 0x7f12400cfdb0 2023-10-10T20:24:03.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f12475eb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124016e160 con 0x7f122c003a60 2023-10-10T20:24:03.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f1244b86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f1244b86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49122/0 (socket says 172.21.15.148:49122) 2023-10-10T20:24:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f1244b86700 1 -- 172.21.15.148:0/310027667 learned_addr learned my addr 172.21.15.148:0/310027667 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.444+0000 7f1245387700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c003a60 0x7f124016b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1245b88700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12400cfdb0 0x7f124014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 -- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c003a60 msgr2=0x7f124016b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c003a60 0x7f124016b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 -- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12400cfdb0 msgr2=0x7f124014d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12400cfdb0 0x7f124014d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 -- 172.21.15.148:0/310027667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124016e440 con 0x7f12400cf940 2023-10-10T20:24:03.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f1244b86700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f123000e5f0 tx=0x7f1230009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300105f0 con 0x7f12400cf940 2023-10-10T20:24:03.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1230010750 con 0x7f12400cf940 2023-10-10T20:24:03.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230010af0 con 0x7f12400cf940 2023-10-10T20:24:03.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124016e720 con 0x7f12400cf940 2023-10-10T20:24:03.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.445+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12400cab60 con 0x7f12400cf940 2023-10-10T20:24:03.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.446+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124016ebc0 con 0x7f12400cf940 2023-10-10T20:24:03.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.446+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1230014480 con 0x7f12400cf940 2023-10-10T20:24:03.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.447+0000 7f12357fa700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.447+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1230083fb0 con 0x7f12400cf940 2023-10-10T20:24:03.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.447+0000 7f1245387700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.447+0000 7f1245387700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 0x7f12240693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f123c0091e0 tx=0x7f123c01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.451+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f123004d5a0 con 0x7f12400cf940 2023-10-10T20:24:03.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.634+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f12400b01d0 con 0x7f12400cf940 2023-10-10T20:24:03.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.638+0000 7f12357fa700 1 -- 172.21.15.148:0/310027667 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12300190c0 con 0x7f12400cf940 2023-10-10T20:24:03.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 msgr2=0x7f12240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 0x7f12240693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f123c0091e0 tx=0x7f123c01a000 comp rx=0 tx=0).stop 2023-10-10T20:24:03.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 msgr2=0x7f124014cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f123000e5f0 tx=0x7f1230009040 comp rx=0 tx=0).stop 2023-10-10T20:24:03.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 shutdown_connections 2023-10-10T20:24:03.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1224066f30 0x7f12240693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12400cfdb0 0x7f124014d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f122c003a60 0x7f124016b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 --2- 172.21.15.148:0/310027667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12400cf940 0x7f124014cfc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 >> 172.21.15.148:0/310027667 conn(0x7f12400d1880 msgr2=0x7f1240155810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:03.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 shutdown_connections 2023-10-10T20:24:03.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.640+0000 7f12475eb700 1 -- 172.21.15.148:0/310027667 wait complete. 2023-10-10T20:24:03.653 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:24:03.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:03.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:03.739 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:24:03.740 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:24:03.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:03.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.854+0000 7f15e1446700 1 Processor -- start 2023-10-10T20:24:03.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.854+0000 7f15e1446700 1 -- start start 2023-10-10T20:24:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e1446700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e1446700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e1446700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15dc0cb7c0 con 0x7f15dc15dda0 2023-10-10T20:24:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e1446700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15dc15e850 con 0x7f15dc0cdab0 2023-10-10T20:24:03.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e1446700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0cd030 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15dbfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15db7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15dbfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:52624/0 (socket says 172.21.15.148:52624) 2023-10-10T20:24:03.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15dbfff700 1 -- 172.21.15.148:0/1919849541 learned_addr learned my addr 172.21.15.148:0/1919849541 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:03.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15e0c45700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f15dc15dda0 0x7f15dc15cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:40742/0 (socket says 172.21.15.148:40742) 2023-10-10T20:24:03.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.855+0000 7f15db7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:45384/0 (socket says 172.21.15.148:45384) 2023-10-10T20:24:03.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15dbfff700 1 -- 172.21.15.148:0/1919849541 >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 msgr2=0x7f15dc15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:03.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15dbfff700 1 --2- 172.21.15.148:0/1919849541 >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15dbfff700 1 -- 172.21.15.148:0/1919849541 >> v1:172.21.15.161:6789/0 conn(0x7f15dc15dda0 legacy=0x7f15dc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15dbfff700 1 -- 172.21.15.148:0/1919849541 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc15e290 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15db7fe700 1 --2- 172.21.15.148:0/1919849541 >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15dbfff700 1 --2- 172.21.15.148:0/1919849541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0099b0 tx=0x7f15cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33ece59b3c31d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15daffd700 1 -- 172.21.15.148:0/1919849541 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc009ea0 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15daffd700 1 -- 172.21.15.148:0/1919849541 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00b860 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.856+0000 7f15daffd700 1 -- 172.21.15.148:0/1919849541 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00bc00 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 -- 172.21.15.148:0/1919849541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 msgr2=0x7f15dc0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 --2- 172.21.15.148:0/1919849541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0099b0 tx=0x7f15cc009250 comp rx=0 tx=0).stop 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 -- 172.21.15.148:0/1919849541 shutdown_connections 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 --2- 172.21.15.148:0/1919849541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc0cbc50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 --2- 172.21.15.148:0/1919849541 >> v2:172.21.15.161:3300/0 conn(0x7f15dc0cdab0 0x7f15dc15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 -- 172.21.15.148:0/1919849541 >> 172.21.15.148:0/1919849541 conn(0x7f15dc067660 msgr2=0x7f15dc067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 -- 172.21.15.148:0/1919849541 shutdown_connections 2023-10-10T20:24:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.857+0000 7f15e1446700 1 -- 172.21.15.148:0/1919849541 wait complete. 2023-10-10T20:24:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.858+0000 7f15e1446700 1 Processor -- start 2023-10-10T20:24:03.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.858+0000 7f15e1446700 1 -- start start 2023-10-10T20:24:03.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15dc0cdab0 0x7f15dc1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc15bea0 con 0x7f15dc15dda0 2023-10-10T20:24:03.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc16a710 con 0x7f15dc0cdab0 2023-10-10T20:24:03.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc16a9f0 con 0x7f15dc0cd6e0 2023-10-10T20:24:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49130/0 (socket says 172.21.15.148:49130) 2023-10-10T20:24:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 -- 172.21.15.148:0/3182744738 learned_addr learned my addr 172.21.15.148:0/3182744738 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15dbfff700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15db7fe700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15dc0cdab0 0x7f15dc1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 -- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 msgr2=0x7f15dc155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 -- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15dc0cdab0 msgr2=0x7f15dc1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:03.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15dc0cdab0 0x7f15dc1562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 -- 172.21.15.148:0/3182744738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009420 con 0x7f15dc15dda0 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e0c45700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f15d400cc70 tx=0x7f15d4001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4005df0 con 0x7f15dc15dda0 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d400a5b0 con 0x7f15dc15dda0 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc0094f0 con 0x7f15dc15dda0 2023-10-10T20:24:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.859+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400a930 con 0x7f15dc15dda0 2023-10-10T20:24:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.860+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc16afb0 con 0x7f15dc15dda0 2023-10-10T20:24:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.860+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15dc089280 con 0x7f15dc15dda0 2023-10-10T20:24:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.861+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15d400ace0 con 0x7f15dc15dda0 2023-10-10T20:24:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.861+0000 7f15c3fff700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 0x7f15c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.861+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f15d40841c0 con 0x7f15dc15dda0 2023-10-10T20:24:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.861+0000 7f15dbfff700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 0x7f15c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.862+0000 7f15dbfff700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 0x7f15c80693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0099b0 tx=0x7f15cc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:03.866+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15d40517b0 con 0x7f15dc15dda0 2023-10-10T20:24:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.089+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15dc16b1a0 con 0x7f15dc15dda0 2023-10-10T20:24:04.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.090+0000 7f15c3fff700 1 -- 172.21.15.148:0/3182744738 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f15d4054dd0 con 0x7f15dc15dda0 2023-10-10T20:24:04.092 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:04.092 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/399246323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":399246323},{"type":"v1","addr":"172.21.15.161:6835","nonce":399246323}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2997411711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2997411711},{"type":"v1","addr":"172.21.15.148:6836","nonce":2997411711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2713095078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2713095078},{"type":"v1","addr":"172.21.15.161:6834","nonce":2713095078}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4193106564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4193106564},{"type":"v1","addr":"172.21.15.148:6837","nonce":4193106564}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2023-10-10T20:24:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 msgr2=0x7f15c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 0x7f15c80693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0099b0 tx=0x7f15cc017000 comp rx=0 tx=0).stop 2023-10-10T20:24:04.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 msgr2=0x7f15dc14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f15d400cc70 tx=0x7f15d4001a70 comp rx=0 tx=0).stop 2023-10-10T20:24:04.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 shutdown_connections 2023-10-10T20:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f15c8066f30 0x7f15c80693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15dc15dda0 0x7f15dc14f850 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f15dc0cd6e0 0x7f15dc155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 --2- 172.21.15.148:0/3182744738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15dc0cdab0 0x7f15dc1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 >> 172.21.15.148:0/3182744738 conn(0x7f15dc067660 msgr2=0x7f15dc0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 shutdown_connections 2023-10-10T20:24:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.092+0000 7f15e1446700 1 -- 172.21.15.148:0/3182744738 wait complete. 2023-10-10T20:24:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 42 2023-10-10T20:24:04.106 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:24:04.106 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:24:04.106 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:24:04.106 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:04.107 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:24:04.107 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:24:04.107 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:24:04.108 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:04.108 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:24:04.108 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:24:04.109 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:24:04.109 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:04.109 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:24:04.109 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:24:04.110 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:24:04.110 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:04.114 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:24:04.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:24:04.116 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:24:04.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:24:04.119 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:24:04.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:24:04.120 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:24:04.120 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:24:04.158 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:24:04.159 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:04.160 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:24:04.160 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:04.162 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:24:04.162 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:04.163 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:24:04.163 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:04.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:24:04.232 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:24:04.240 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:24:04.243 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:24:04.243 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:24:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.293+0000 7fa562c9d700 1 Processor -- start 2023-10-10T20:24:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.293+0000 7fa562c9d700 1 -- start start 2023-10-10T20:24:04.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa562c9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa55c15a170 0x7fa55c068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa562c9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa562c9d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c15ea70 con 0x7fa55c159da0 2023-10-10T20:24:04.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa562c9d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c0b34b0 con 0x7fa55c15a170 2023-10-10T20:24:04.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa562c9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0694e0 con 0x7fa55c069040 2023-10-10T20:24:04.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa561c9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa55c159da0 0x7fa55c0b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35676/0 (socket says 172.21.15.148:35676) 2023-10-10T20:24:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa561c9b700 1 -- 172.21.15.148:0/4157934990 learned_addr learned my addr 172.21.15.148:0/4157934990 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 --2- 172.21.15.148:0/4157934990 >> v2:172.21.15.148:3300/0 conn(0x7fa55c15a170 0x7fa55c068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56249c700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 -- 172.21.15.148:0/4157934990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 msgr2=0x7fa55c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 -- 172.21.15.148:0/4157934990 >> v1:172.21.15.148:6789/0 conn(0x7fa55c159da0 legacy=0x7fa55c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 -- 172.21.15.148:0/4157934990 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0b3990 con 0x7fa55c15a170 2023-10-10T20:24:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56249c700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.294+0000 7fa56149a700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c15a170 0x7fa55c068b00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0098e0 tx=0x7fa54c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58348cc7633c293e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa560c99700 1 -- 172.21.15.148:0/4157934990 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c009e80 con 0x7fa55c15a170 2023-10-10T20:24:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa560c99700 1 -- 172.21.15.148:0/4157934990 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c00b860 con 0x7fa55c15a170 2023-10-10T20:24:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa560c99700 1 -- 172.21.15.148:0/4157934990 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00bbe0 con 0x7fa55c15a170 2023-10-10T20:24:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 -- 172.21.15.148:0/4157934990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c15a170 msgr2=0x7fa55c068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c15a170 0x7fa55c068b00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0098e0 tx=0x7fa54c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 -- 172.21.15.148:0/4157934990 shutdown_connections 2023-10-10T20:24:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 --2- 172.21.15.148:0/4157934990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c15a170 0x7fa55c068b00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 -- 172.21.15.148:0/4157934990 >> 172.21.15.148:0/4157934990 conn(0x7fa55c0d0bc0 msgr2=0x7fa55c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 -- 172.21.15.148:0/4157934990 shutdown_connections 2023-10-10T20:24:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.295+0000 7fa562c9d700 1 -- 172.21.15.148:0/4157934990 wait complete. 2023-10-10T20:24:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.296+0000 7fa562c9d700 1 Processor -- start 2023-10-10T20:24:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.296+0000 7fa562c9d700 1 -- start start 2023-10-10T20:24:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 0x7fa55c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15aac0 con 0x7fa55c159da0 2023-10-10T20:24:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15ad60 con 0x7fa55c15a170 2023-10-10T20:24:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa562c9d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c15b040 con 0x7fa55c069040 2023-10-10T20:24:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49184/0 (socket says 172.21.15.148:49184) 2023-10-10T20:24:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 -- 172.21.15.148:0/2927865776 learned_addr learned my addr 172.21.15.148:0/2927865776 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56249c700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 0x7fa55c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa561c9b700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 -- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 msgr2=0x7fa55c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c15cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 -- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 msgr2=0x7fa55c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 0x7fa55c0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 -- 172.21.15.148:0/2927865776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c009420 con 0x7fa55c159da0 2023-10-10T20:24:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56249c700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 0x7fa55c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa561c9b700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.297+0000 7fa56149a700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b3c0 tx=0x7fa54c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.298+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c011420 con 0x7fa55c159da0 2023-10-10T20:24:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.298+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c0115a0 con 0x7fa55c159da0 2023-10-10T20:24:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.298+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c15b320 con 0x7fa55c159da0 2023-10-10T20:24:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.298+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c011960 con 0x7fa55c159da0 2023-10-10T20:24:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.298+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c0c5230 con 0x7fa55c159da0 2023-10-10T20:24:04.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.299+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c0c54e0 con 0x7fa55c159da0 2023-10-10T20:24:04.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.299+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa54c0253f0 con 0x7fa55c159da0 2023-10-10T20:24:04.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.300+0000 7fa551ffb700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 0x7fa5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:04.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.300+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa54c085b70 con 0x7fa55c159da0 2023-10-10T20:24:04.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.300+0000 7fa561c9b700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 0x7fa5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:04.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.300+0000 7fa561c9b700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 0x7fa5400693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa558001820 tx=0x7fa558009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:04.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.304+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa55c0c54e0 con 0x7fa55c159da0 2023-10-10T20:24:04.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:04.499+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa55c0c54e0 con 0x7fa55c159da0 2023-10-10T20:24:05.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.198+0000 7fa551ffb700 1 -- 172.21.15.148:0/2927865776 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa55c0c54e0 con 0x7fa55c159da0 2023-10-10T20:24:05.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 msgr2=0x7fa5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 0x7fa5400693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa558001820 tx=0x7fa558009040 comp rx=0 tx=0).stop 2023-10-10T20:24:05.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 msgr2=0x7fa55c15a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b3c0 tx=0x7fa54c017000 comp rx=0 tx=0).stop 2023-10-10T20:24:05.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 shutdown_connections 2023-10-10T20:24:05.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa540066f30 0x7fa5400693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55c15a170 0x7fa55c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa55c069040 0x7fa55c15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 --2- 172.21.15.148:0/2927865776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c159da0 0x7fa55c15a580 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 >> 172.21.15.148:0/2927865776 conn(0x7fa55c0d0bc0 msgr2=0x7fa55c158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:05.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 shutdown_connections 2023-10-10T20:24:05.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.200+0000 7fa562c9d700 1 -- 172.21.15.148:0/2927865776 wait complete. 2023-10-10T20:24:05.204 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:24:05.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:24:05.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.328+0000 7f8b4af4b700 1 Processor -- start 2023-10-10T20:24:05.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.328+0000 7f8b4af4b700 1 -- start start 2023-10-10T20:24:05.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b4af4b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 0x7f8b4415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b4af4b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4415cbb0 con 0x7f8b4415a620 2023-10-10T20:24:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b4af4b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4415bfa0 con 0x7f8b44157c40 2023-10-10T20:24:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b4af4b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4415c280 con 0x7f8b44157870 2023-10-10T20:24:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b43fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b4415a620 0x7f8b4415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35690/0 (socket says 172.21.15.148:35690) 2023-10-10T20:24:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b43fff700 1 -- 172.21.15.148:0/880166145 learned_addr learned my addr 172.21.15.148:0/880166145 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:05.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.329+0000 7f8b48ce7700 1 --2- 172.21.15.148:0/880166145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 0x7f8b4415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103601771 0 0) 0x7f8b4415cbb0 con 0x7f8b4415a620 2023-10-10T20:24:05.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b34003570 con 0x7f8b4415a620 2023-10-10T20:24:05.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640398106 0 0) 0x7f8b4415bfa0 con 0x7f8b44157c40 2023-10-10T20:24:05.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b4415cbb0 con 0x7f8b44157c40 2023-10-10T20:24:05.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b48ce7700 1 -- 172.21.15.148:0/880166145 >> v1:172.21.15.161:6789/0 conn(0x7f8b44157c40 legacy=0x7f8b44158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b48ce7700 1 -- 172.21.15.148:0/880166145 >> v1:172.21.15.148:6789/0 conn(0x7f8b4415a620 legacy=0x7f8b4415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b48ce7700 1 -- 172.21.15.148:0/880166145 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4415c560 con 0x7f8b44157870 2023-10-10T20:24:05.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b48ce7700 1 --2- 172.21.15.148:0/880166145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 0x7f8b4415b6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009910 tx=0x7f8b380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee291363dcef9b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38011630 con 0x7f8b44157870 2023-10-10T20:24:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38011790 con 0x7f8b44157870 2023-10-10T20:24:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.330+0000 7f8b437fe700 1 -- 172.21.15.148:0/880166145 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38011b30 con 0x7f8b44157870 2023-10-10T20:24:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 -- 172.21.15.148:0/880166145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 msgr2=0x7f8b4415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/880166145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 0x7f8b4415b6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009910 tx=0x7f8b380091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 -- 172.21.15.148:0/880166145 shutdown_connections 2023-10-10T20:24:05.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/880166145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b44157870 0x7f8b4415b6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 -- 172.21.15.148:0/880166145 >> 172.21.15.148:0/880166145 conn(0x7f8b44068ce0 msgr2=0x7f8b440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:05.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 -- 172.21.15.148:0/880166145 shutdown_connections 2023-10-10T20:24:05.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.331+0000 7f8b4af4b700 1 -- 172.21.15.148:0/880166145 wait complete. 2023-10-10T20:24:05.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.332+0000 7f8b4af4b700 1 Processor -- start 2023-10-10T20:24:05.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.332+0000 7f8b4af4b700 1 -- start start 2023-10-10T20:24:05.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 0x7f8b44150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 0x7f8b440b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4414ffc0 con 0x7f8b44157870 2023-10-10T20:24:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440b2fb0 con 0x7f8b44157c40 2023-10-10T20:24:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4414f3b0 con 0x7f8b4415a620 2023-10-10T20:24:05.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49190/0 (socket says 172.21.15.148:49190) 2023-10-10T20:24:05.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 -- 172.21.15.148:0/3845798554 learned_addr learned my addr 172.21.15.148:0/3845798554 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:05.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b43fff700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 0x7f8b44150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b494e8700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 0x7f8b440b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 -- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 msgr2=0x7f8b440b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 0x7f8b440b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 -- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 msgr2=0x7f8b44150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 0x7f8b44150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 -- 172.21.15.148:0/3845798554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009420 con 0x7f8b44157870 2023-10-10T20:24:05.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b43fff700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 0x7f8b44150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:05.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b494e8700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 0x7f8b440b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:05.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b48ce7700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009fd0 tx=0x7f8b38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3801f070 con 0x7f8b44157870 2023-10-10T20:24:05.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38000bc0 con 0x7f8b44157870 2023-10-10T20:24:05.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4414f690 con 0x7f8b44157870 2023-10-10T20:24:05.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38025420 con 0x7f8b44157870 2023-10-10T20:24:05.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.333+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4414fc50 con 0x7f8b44157870 2023-10-10T20:24:05.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.334+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b440d1cb0 con 0x7f8b44157870 2023-10-10T20:24:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.335+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b38023070 con 0x7f8b44157870 2023-10-10T20:24:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.335+0000 7f8b40ff9700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 0x7f8b280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.335+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8b38085550 con 0x7f8b44157870 2023-10-10T20:24:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.335+0000 7f8b43fff700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 0x7f8b280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.336+0000 7f8b43fff700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 0x7f8b280693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8b440b3800 tx=0x7f8b30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.340+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b38052b10 con 0x7f8b44157870 2023-10-10T20:24:05.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.598+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8b441502a0 con 0x7f8b44157870 2023-10-10T20:24:05.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.599+0000 7f8b40ff9700 1 -- 172.21.15.148:0/3845798554 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8b38015090 con 0x7f8b44157870 2023-10-10T20:24:05.600 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:05.600 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:05.601 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:24:05.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.601 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:24:05.601 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:24:05.601 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.602 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.602 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:05.602 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:05.602 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:24:05.603 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.604 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.604 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:05.604 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:05.604 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.604 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.605 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:24:05.605 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:24:05.605 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.606 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.606 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:05.606 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:05.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.606 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.607 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:24:05.607 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:24:05.608 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.608 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.608 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:05.608 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:05.608 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.609 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:24:05.609 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:24:05.610 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.610 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.610 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.610 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.611 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.612 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.612 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.613 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:24:05.613 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:24:05.613 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.613 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.613 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:24:05.614 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:24:05.615 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.615 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.615 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.615 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.616 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:24:05.616 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:24:05.616 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.617 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.617 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.617 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:24:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:24:05.618 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.619 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.619 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.619 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:24:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:24:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.621 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:24:05.622 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:24:05.622 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.622 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:05.622 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:05.622 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.623 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.623 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:24:05.623 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:24:05.624 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.624 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.624 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:24:05.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:24:05.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.625 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.627 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:24:05.627 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:24:05.627 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.627 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.628 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:05.628 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:05.628 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:05.628 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:05.629 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.631 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:24:05.631 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:24:05.631 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.631 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:05.631 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.632 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:24:05.633 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:24:05.633 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.633 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:05.633 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:24:05.633 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:05.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:05.634 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:24:05.634 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:24:05.635 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:05.635 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:05.635 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:24:05.635 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:05.635 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:05.636 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:05.636 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 msgr2=0x7f8b280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 0x7f8b280693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8b440b3800 tx=0x7f8b30009040 comp rx=0 tx=0).stop 2023-10-10T20:24:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 msgr2=0x7f8b44152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009fd0 tx=0x7f8b38017000 comp rx=0 tx=0).stop 2023-10-10T20:24:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 shutdown_connections 2023-10-10T20:24:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8b28066f30 0x7f8b280693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4415a620 0x7f8b440b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44157870 0x7f8b44152440 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 --2- 172.21.15.148:0/3845798554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b44157c40 0x7f8b44150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 >> 172.21.15.148:0/3845798554 conn(0x7f8b44068ce0 msgr2=0x7f8b440b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 shutdown_connections 2023-10-10T20:24:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.601+0000 7f8b4af4b700 1 -- 172.21.15.148:0/3845798554 wait complete. 2023-10-10T20:24:05.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.755+0000 7f13f1fd3700 1 Processor -- start 2023-10-10T20:24:05.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.755+0000 7f13f1fd3700 1 -- start start 2023-10-10T20:24:05.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13f1fd3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13ec15a620 0x7f13ec15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13f1fd3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec15cbb0 con 0x7f13ec15d580 2023-10-10T20:24:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13f1fd3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec15bfa0 con 0x7f13ec157870 2023-10-10T20:24:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13f1fd3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec15bdb0 con 0x7f13ec15a620 2023-10-10T20:24:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13eaffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13ec15a620 0x7f13ec15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13ebfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f13ec15d580 0x7f13ec158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35700/0 (socket says 172.21.15.148:35700) 2023-10-10T20:24:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13ebfff700 1 -- 172.21.15.148:0/2210915297 learned_addr learned my addr 172.21.15.148:0/2210915297 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13eaffd700 1 -- 172.21.15.148:0/2210915297 >> v1:172.21.15.161:6789/0 conn(0x7f13ec157870 legacy=0x7f13ec15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13eaffd700 1 -- 172.21.15.148:0/2210915297 >> v1:172.21.15.148:6789/0 conn(0x7f13ec15d580 legacy=0x7f13ec158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13eaffd700 1 -- 172.21.15.148:0/2210915297 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec15c280 con 0x7f13ec15a620 2023-10-10T20:24:05.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13ea7fc700 1 -- 172.21.15.148:0/2210915297 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975255477 0 0) 0x7f13ec15cbb0 con 0x7f13ec15d580 2023-10-10T20:24:05.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.756+0000 7f13eaffd700 1 --2- 172.21.15.148:0/2210915297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec15a620 0x7f13ec15d040 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f13e00098e0 tx=0x7f13e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f1dce883f8a739 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13ea7fc700 1 -- 172.21.15.148:0/2210915297 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0009e80 con 0x7f13ec15a620 2023-10-10T20:24:05.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13ea7fc700 1 -- 172.21.15.148:0/2210915297 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e000b860 con 0x7f13ec15a620 2023-10-10T20:24:05.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13ea7fc700 1 -- 172.21.15.148:0/2210915297 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000bbe0 con 0x7f13ec15a620 2023-10-10T20:24:05.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 -- 172.21.15.148:0/2210915297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec15a620 msgr2=0x7f13ec15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/2210915297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec15a620 0x7f13ec15d040 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f13e00098e0 tx=0x7f13e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 -- 172.21.15.148:0/2210915297 shutdown_connections 2023-10-10T20:24:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/2210915297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec15a620 0x7f13ec15d040 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 -- 172.21.15.148:0/2210915297 >> 172.21.15.148:0/2210915297 conn(0x7f13ec068ce0 msgr2=0x7f13ec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 -- 172.21.15.148:0/2210915297 shutdown_connections 2023-10-10T20:24:05.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.757+0000 7f13f1fd3700 1 -- 172.21.15.148:0/2210915297 wait complete. 2023-10-10T20:24:05.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.758+0000 7f13f1fd3700 1 Processor -- start 2023-10-10T20:24:05.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.758+0000 7f13f1fd3700 1 -- start start 2023-10-10T20:24:05.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13ec15a620 0x7f13ec16b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ec15d580 0x7f13ec0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec16b060 con 0x7f13ec157870 2023-10-10T20:24:05.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0d08d0 con 0x7f13ec15d580 2023-10-10T20:24:05.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13f1fd3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0d0bb0 con 0x7f13ec15a620 2023-10-10T20:24:05.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49212/0 (socket says 172.21.15.148:49212) 2023-10-10T20:24:05.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13eb7fe700 1 -- 172.21.15.148:0/3821574625 learned_addr learned my addr 172.21.15.148:0/3821574625 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:05.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13ebfff700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ec15d580 0x7f13ec0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.759+0000 7f13eaffd700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13ec15a620 0x7f13ec16b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 -- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13ec15a620 msgr2=0x7f13ec16b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13ec15a620 0x7f13ec16b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 -- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ec15d580 msgr2=0x7f13ec0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ec15d580 0x7f13ec0b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 -- 172.21.15.148:0/3821574625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0009420 con 0x7f13ec157870 2023-10-10T20:24:05.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13eb7fe700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c590 tx=0x7f13dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc011590 con 0x7f13ec157870 2023-10-10T20:24:05.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc011710 con 0x7f13ec157870 2023-10-10T20:24:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec16acf0 con 0x7f13ec157870 2023-10-10T20:24:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc011ad0 con 0x7f13ec157870 2023-10-10T20:24:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.760+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0d1290 con 0x7f13ec157870 2023-10-10T20:24:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.761+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec16b340 con 0x7f13ec157870 2023-10-10T20:24:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.761+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13dc01b3b0 con 0x7f13ec157870 2023-10-10T20:24:05.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.762+0000 7f13d37fe700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 0x7f13d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:05.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.762+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f13dc0835f0 con 0x7f13ec157870 2023-10-10T20:24:05.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.762+0000 7f13eaffd700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 0x7f13d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:05.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.762+0000 7f13eaffd700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 0x7f13d80693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13ec15aa20 tx=0x7f13e0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:05.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.766+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13dc050bb0 con 0x7f13ec157870 2023-10-10T20:24:05.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.949+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13ec16ab30 con 0x7f13ec157870 2023-10-10T20:24:05.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.950+0000 7f13d37fe700 1 -- 172.21.15.148:0/3821574625 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+12183 (secure 0 0 0) 0x7f13dc017020 con 0x7f13ec157870 2023-10-10T20:24:05.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:05.951 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":94,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:05.211445+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:05.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 msgr2=0x7f13d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 0x7f13d80693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13ec15aa20 tx=0x7f13e0017000 comp rx=0 tx=0).stop 2023-10-10T20:24:05.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 msgr2=0x7f13ec16e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:05.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c590 tx=0x7f13dc00c8a0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 shutdown_connections 2023-10-10T20:24:05.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f13d8066f30 0x7f13d80693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13ec15d580 0x7f13ec0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13ec157870 0x7f13ec16e970 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 --2- 172.21.15.148:0/3821574625 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f13ec15a620 0x7f13ec16b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:05.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 >> 172.21.15.148:0/3821574625 conn(0x7f13ec068ce0 msgr2=0x7f13ec14c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:05.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 shutdown_connections 2023-10-10T20:24:05.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:05.952+0000 7f13f1fd3700 1 -- 172.21.15.148:0/3821574625 wait complete. 2023-10-10T20:24:05.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:24:05.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:24:06.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.083+0000 7fe6d48de700 1 Processor -- start 2023-10-10T20:24:06.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.083+0000 7fe6d48de700 1 -- start start 2023-10-10T20:24:06.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6d48de700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6d00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6d48de700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6d015a290 0x7fe6d00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6d48de700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00b0790 con 0x7fe6d006a5a0 2023-10-10T20:24:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6d48de700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d00ce1f0 con 0x7fe6d015a290 2023-10-10T20:24:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6d48de700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b0b20 con 0x7fe6d0159ec0 2023-10-10T20:24:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6ced9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe6d006a5a0 0x7fe6d015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35714/0 (socket says 172.21.15.148:35714) 2023-10-10T20:24:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6ced9d700 1 -- 172.21.15.148:0/2102562968 learned_addr learned my addr 172.21.15.148:0/2102562968 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6d00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6c5d9b700 1 --2- 172.21.15.148:0/2102562968 >> v2:172.21.15.161:3300/0 conn(0x7fe6d015a290 0x7fe6d00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576796284 0 0) 0x7fe6d00b0790 con 0x7fe6d006a5a0 2023-10-10T20:24:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.084+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6bc003570 con 0x7fe6d006a5a0 2023-10-10T20:24:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601568420 0 0) 0x7fe6bc003570 con 0x7fe6d006a5a0 2023-10-10T20:24:06.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 msgr2=0x7fe6d00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6d00af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 >> v2:172.21.15.161:3300/0 conn(0x7fe6d015a290 msgr2=0x7fe6d00afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 --2- 172.21.15.148:0/2102562968 >> v2:172.21.15.161:3300/0 conn(0x7fe6d015a290 0x7fe6d00afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00b04b0 con 0x7fe6d006a5a0 2023-10-10T20:24:06.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fe6c80017f0 con 0x7fe6d006a5a0 2023-10-10T20:24:06.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 >> v1:172.21.15.148:6789/0 conn(0x7fe6d006a5a0 legacy=0x7fe6d015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc004d50 0x7fe6bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00ce1f0 con 0x7fe6bc004d50 2023-10-10T20:24:06.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b0b20 con 0x7fe6bc003a60 2023-10-10T20:24:06.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc00ba60 con 0x7fe6d0159ec0 2023-10-10T20:24:06.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc004d50 0x7fe6bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6c5d9b700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 -- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 msgr2=0x7fe6bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d0159ec0 0x7fe6bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 -- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc003a60 msgr2=0x7fe6bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 -- 172.21.15.148:0/2102562968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc003570 con 0x7fe6bc004d50 2023-10-10T20:24:06.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc004d50 0x7fe6bc0051c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800a970 tx=0x7fe6c8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db79f574d9e9866 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8012440 con 0x7fe6bc004d50 2023-10-10T20:24:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80125c0 con 0x7fe6bc004d50 2023-10-10T20:24:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.085+0000 7fe6cdd9b700 1 -- 172.21.15.148:0/2102562968 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8012980 con 0x7fe6bc004d50 2023-10-10T20:24:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 -- 172.21.15.148:0/2102562968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc004d50 msgr2=0x7fe6bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 --2- 172.21.15.148:0/2102562968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc004d50 0x7fe6bc0051c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800a970 tx=0x7fe6c8002210 comp rx=0 tx=0).stop 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6ced9d700 1 -- 172.21.15.148:0/2102562968 reap_dead start 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 -- 172.21.15.148:0/2102562968 shutdown_connections 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 -- 172.21.15.148:0/2102562968 >> 172.21.15.148:0/2102562968 conn(0x7fe6d00d1880 msgr2=0x7fe6d00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 -- 172.21.15.148:0/2102562968 shutdown_connections 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.086+0000 7fe6d48de700 1 -- 172.21.15.148:0/2102562968 wait complete. 2023-10-10T20:24:06.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.087+0000 7fe6d48de700 1 Processor -- start 2023-10-10T20:24:06.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.087+0000 7fe6d48de700 1 -- start start 2023-10-10T20:24:06.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 0x7fe6d01477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 0x7fe6d00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00c90b0 con 0x7fe6d00c6b20 2023-10-10T20:24:06.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00c74e0 con 0x7fe6bc004d50 2023-10-10T20:24:06.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6d48de700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0157200 con 0x7fe6d00c8700 2023-10-10T20:24:06.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49240/0 (socket says 172.21.15.148:49240) 2023-10-10T20:24:06.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 -- 172.21.15.148:0/4198293164 learned_addr learned my addr 172.21.15.148:0/4198293164 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 0x7fe6d01477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6c5d9b700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 0x7fe6d00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 -- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 msgr2=0x7fe6d00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 0x7fe6d00c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 -- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 msgr2=0x7fe6d01477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 0x7fe6d01477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 -- 172.21.15.148:0/4198293164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d0156310 con 0x7fe6d00c6b20 2023-10-10T20:24:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6c5d9b700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 0x7fe6d00c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 0x7fe6d01477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.088+0000 7fe6ced9d700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc004730 tx=0x7fe6c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8001360 con 0x7fe6d00c6b20 2023-10-10T20:24:06.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c8012440 con 0x7fe6d00c6b20 2023-10-10T20:24:06.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d01565f0 con 0x7fe6d00c6b20 2023-10-10T20:24:06.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80017b0 con 0x7fe6d00c6b20 2023-10-10T20:24:06.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d0156bb0 con 0x7fe6d00c6b20 2023-10-10T20:24:06.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.089+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d0157390 con 0x7fe6d00c6b20 2023-10-10T20:24:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.090+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6c8019070 con 0x7fe6d00c6b20 2023-10-10T20:24:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.090+0000 7fe6c6ffd700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 0x7fe6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.090+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 0x7fe6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.090+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe6c8085e60 con 0x7fe6d00c6b20 2023-10-10T20:24:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.091+0000 7fe6ce59c700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 0x7fe6b00693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00093f0 tx=0x7fe6c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.095+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6c8053420 con 0x7fe6d00c6b20 2023-10-10T20:24:06.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.282+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6d004c3c0 con 0x7fe6d00c6b20 2023-10-10T20:24:06.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.282+0000 7fe6c6ffd700 1 -- 172.21.15.148:0/4198293164 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6c8012940 con 0x7fe6d00c6b20 2023-10-10T20:24:06.287 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:24:06.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 msgr2=0x7fe6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 0x7fe6b00693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00093f0 tx=0x7fe6c0008040 comp rx=0 tx=0).stop 2023-10-10T20:24:06.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 msgr2=0x7fe6d00c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc004730 tx=0x7fe6c8015000 comp rx=0 tx=0).stop 2023-10-10T20:24:06.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 shutdown_connections 2023-10-10T20:24:06.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe6b0066f30 0x7fe6b00693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6d00c6b20 0x7fe6d00c6fa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6bc004d50 0x7fe6d01477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 --2- 172.21.15.148:0/4198293164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6d00c8700 0x7fe6d00c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 >> 172.21.15.148:0/4198293164 conn(0x7fe6d00d1880 msgr2=0x7fe6d0155970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:06.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 shutdown_connections 2023-10-10T20:24:06.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.285+0000 7fe6d48de700 1 -- 172.21.15.148:0/4198293164 wait complete. 2023-10-10T20:24:06.298 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:24:06.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.412+0000 7f1c3de9b700 1 Processor -- start 2023-10-10T20:24:06.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.412+0000 7f1c3de9b700 1 -- start start 2023-10-10T20:24:06.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c3de9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c38157870 0x7f1c3815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c3de9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c3de9b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3815cbb0 con 0x7f1c3815d580 2023-10-10T20:24:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c3de9b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3815bfa0 con 0x7f1c38157870 2023-10-10T20:24:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c3de9b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3815c280 con 0x7f1c3815a620 2023-10-10T20:24:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c38157870 0x7f1c3815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c38157870 0x7f1c3815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49244/0 (socket says 172.21.15.148:49244) 2023-10-10T20:24:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 -- 172.21.15.148:0/4021390556 learned_addr learned my addr 172.21.15.148:0/4021390556 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c36ffd700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 -- 172.21.15.148:0/4021390556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 msgr2=0x7f1c3815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 -- 172.21.15.148:0/4021390556 >> v1:172.21.15.161:6789/0 conn(0x7f1c3815d580 legacy=0x7f1c38158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.413+0000 7f1c377fe700 1 -- 172.21.15.148:0/4021390556 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3815c560 con 0x7f1c38157870 2023-10-10T20:24:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c36ffd700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c377fe700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c38157870 0x7f1c3815b6a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009f20 tx=0x7f1c28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18d79ecb84c072d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c367fc700 1 -- 172.21.15.148:0/4021390556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800e070 con 0x7f1c38157870 2023-10-10T20:24:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c367fc700 1 -- 172.21.15.148:0/4021390556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2800ba40 con 0x7f1c38157870 2023-10-10T20:24:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c367fc700 1 -- 172.21.15.148:0/4021390556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280092e0 con 0x7f1c38157870 2023-10-10T20:24:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4021390556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c38157870 msgr2=0x7f1c3815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c38157870 0x7f1c3815b6a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009f20 tx=0x7f1c28009840 comp rx=0 tx=0).stop 2023-10-10T20:24:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4021390556 shutdown_connections 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c38157870 0x7f1c3815b6a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4021390556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4021390556 >> 172.21.15.148:0/4021390556 conn(0x7f1c38068ce0 msgr2=0x7f1c380690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4021390556 shutdown_connections 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.414+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4021390556 wait complete. 2023-10-10T20:24:06.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 Processor -- start 2023-10-10T20:24:06.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 -- start start 2023-10-10T20:24:06.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c38157870 0x7f1c380b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3816b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b1800 con 0x7f1c3815d580 2023-10-10T20:24:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3816b540 con 0x7f1c38157870 2023-10-10T20:24:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c3de9b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381526d0 con 0x7f1c3815a620 2023-10-10T20:24:06.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c37fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49250/0 (socket says 172.21.15.148:49250) 2023-10-10T20:24:06.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.416+0000 7f1c37fff700 1 -- 172.21.15.148:0/4175649319 learned_addr learned my addr 172.21.15.148:0/4175649319 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:06.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c377fe700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c38157870 0x7f1c380b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c36ffd700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3816b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 -- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 msgr2=0x7f1c3816b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3816b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 -- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c38157870 msgr2=0x7f1c380b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c38157870 0x7f1c380b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 -- 172.21.15.148:0/4175649319 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28009ae0 con 0x7f1c3815d580 2023-10-10T20:24:06.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c37fff700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b5e0 tx=0x7f1c30009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000f510 con 0x7f1c3815d580 2023-10-10T20:24:06.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3000f670 con 0x7f1c3815d580 2023-10-10T20:24:06.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c380b0bf0 con 0x7f1c3815d580 2023-10-10T20:24:06.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000fa30 con 0x7f1c3815d580 2023-10-10T20:24:06.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.417+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c380b11b0 con 0x7f1c3815d580 2023-10-10T20:24:06.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.418+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c380b19b0 con 0x7f1c3815d580 2023-10-10T20:24:06.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.418+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c3001a400 con 0x7f1c3815d580 2023-10-10T20:24:06.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.419+0000 7f1c1f7fe700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.419+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1c30083b80 con 0x7f1c3815d580 2023-10-10T20:24:06.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.419+0000 7f1c377fe700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.419+0000 7f1c377fe700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 0x7f1c240693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009880 tx=0x7f1c28017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.423+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c380b19b0 con 0x7f1c3815d580 2023-10-10T20:24:06.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.611+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1c380b19b0 con 0x7f1c3815d580 2023-10-10T20:24:06.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.611+0000 7f1c1f7fe700 1 -- 172.21.15.148:0/4175649319 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1c380b19b0 con 0x7f1c3815d580 2023-10-10T20:24:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:06.613 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:24:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:06.613 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:24:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:24:06.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:06.614 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:24:06.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 msgr2=0x7f1c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 0x7f1c240693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009880 tx=0x7f1c28017000 comp rx=0 tx=0).stop 2023-10-10T20:24:06.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 msgr2=0x7f1c38152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000b5e0 tx=0x7f1c30009060 comp rx=0 tx=0).stop 2023-10-10T20:24:06.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 shutdown_connections 2023-10-10T20:24:06.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3815d580 0x7f1c38152190 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c38157870 0x7f1c380b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 --2- 172.21.15.148:0/4175649319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3815a620 0x7f1c3816b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 >> 172.21.15.148:0/4175649319 conn(0x7f1c38068ce0 msgr2=0x7f1c381568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 shutdown_connections 2023-10-10T20:24:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.613+0000 7f1c3de9b700 1 -- 172.21.15.148:0/4175649319 wait complete. 2023-10-10T20:24:06.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:24:06.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:24:06.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.743+0000 7fd8c2e3d700 1 Processor -- start 2023-10-10T20:24:06.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.743+0000 7fd8c2e3d700 1 -- start start 2023-10-10T20:24:06.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c2e3d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8bc0c91c0 0x7fd8bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c2e3d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc06dd90 con 0x7fd8bc06d930 2023-10-10T20:24:06.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c2e3d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc15e940 con 0x7fd8bc0b10a0 2023-10-10T20:24:06.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c2e3d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc0c9660 con 0x7fd8bc0c91c0 2023-10-10T20:24:06.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c13da700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd8bc06d930 0x7fd8bc067940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35728/0 (socket says 172.21.15.148:35728) 2023-10-10T20:24:06.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8c13da700 1 -- 172.21.15.148:0/2374476988 learned_addr learned my addr 172.21.15.148:0/2374476988 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:06.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8bbfff700 1 --2- 172.21.15.148:0/2374476988 >> v2:172.21.15.161:3300/0 conn(0x7fd8bc0c91c0 0x7fd8bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327396580 0 0) 0x7fd8b4001490 con 0x7fd8bc06d930 2023-10-10T20:24:06.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.744+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ac003570 con 0x7fd8bc06d930 2023-10-10T20:24:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758215639 0 0) 0x7fd8ac003570 con 0x7fd8bc06d930 2023-10-10T20:24:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 >> v2:172.21.15.161:3300/0 conn(0x7fd8bc0c91c0 msgr2=0x7fd8bc06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 --2- 172.21.15.148:0/2374476988 >> v2:172.21.15.161:3300/0 conn(0x7fd8bc0c91c0 0x7fd8bc06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 >> v1:172.21.15.161:6789/0 conn(0x7fd8bc0b10a0 legacy=0x7fd8bc0cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc15a250 con 0x7fd8bc06d930 2023-10-10T20:24:06.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fd8b4001980 con 0x7fd8bc06d930 2023-10-10T20:24:06.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 >> v1:172.21.15.148:6789/0 conn(0x7fd8bc06d930 legacy=0x7fd8bc067940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8ac003a60 0x7fd8ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8ac004810 0x7fd8ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac00c130 con 0x7fd8ac0051c0 2023-10-10T20:24:06.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac00c2a0 con 0x7fd8ac003a60 2023-10-10T20:24:06.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac00c460 con 0x7fd8ac004810 2023-10-10T20:24:06.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c0bd9700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8ac003a60 0x7fd8ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8bbfff700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8ac004810 0x7fd8ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 -- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8ac004810 msgr2=0x7fd8ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8ac004810 0x7fd8ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 -- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8ac003a60 msgr2=0x7fd8ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:06.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8ac003a60 0x7fd8ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 -- 172.21.15.148:0/2374476988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac003570 con 0x7fd8ac0051c0 2023-10-10T20:24:06.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 -- 172.21.15.148:0/2374476988 reap_dead start 2023-10-10T20:24:06.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.745+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8ac00ba60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400aab0 tx=0x7fd8bc0cd470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43e90c11e2c6775 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400a440 con 0x7fd8ac0051c0 2023-10-10T20:24:06.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400c5d0 con 0x7fd8ac0051c0 2023-10-10T20:24:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8bb7fe700 1 -- 172.21.15.148:0/2374476988 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400c9b0 con 0x7fd8ac0051c0 2023-10-10T20:24:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 msgr2=0x7fd8ac00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8ac00ba60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400aab0 tx=0x7fd8bc0cd470 comp rx=0 tx=0).stop 2023-10-10T20:24:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2374476988 shutdown_connections 2023-10-10T20:24:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2374476988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8ac00ba60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2374476988 >> 172.21.15.148:0/2374476988 conn(0x7fd8bc069870 msgr2=0x7fd8bc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2374476988 shutdown_connections 2023-10-10T20:24:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.746+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2374476988 wait complete. 2023-10-10T20:24:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.747+0000 7fd8c2e3d700 1 Processor -- start 2023-10-10T20:24:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.747+0000 7fd8c2e3d700 1 -- start start 2023-10-10T20:24:06.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8bc1515c0 0x7fd8bc14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 0x7fd8bc1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc15d110 con 0x7fd8ac0051c0 2023-10-10T20:24:06.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b2610 con 0x7fd8bc155e60 2023-10-10T20:24:06.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c2e3d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b28f0 con 0x7fd8bc1515c0 2023-10-10T20:24:06.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c0bd9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c0bd9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49272/0 (socket says 172.21.15.148:49272) 2023-10-10T20:24:06.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c0bd9700 1 -- 172.21.15.148:0/2113032727 learned_addr learned my addr 172.21.15.148:0/2113032727 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:06.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8bbfff700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8bc1515c0 0x7fd8bc14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.748+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 0x7fd8bc1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 -- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8bc1515c0 msgr2=0x7fd8bc14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8bc1515c0 0x7fd8bc14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 -- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 msgr2=0x7fd8bc1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:06.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 0x7fd8bc1562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:06.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 -- 172.21.15.148:0/2113032727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc15cda0 con 0x7fd8ac0051c0 2023-10-10T20:24:06.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c13da700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 0x7fd8bc1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:06.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c0bd9700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0002950 tx=0x7fd8b0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0006820 con 0x7fd8ac0051c0 2023-10-10T20:24:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0006980 con 0x7fd8ac0051c0 2023-10-10T20:24:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0006d40 con 0x7fd8ac0051c0 2023-10-10T20:24:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc14dc40 con 0x7fd8ac0051c0 2023-10-10T20:24:06.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.749+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc156810 con 0x7fd8ac0051c0 2023-10-10T20:24:06.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.750+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc155cb0 con 0x7fd8ac0051c0 2023-10-10T20:24:06.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.750+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8b0007240 con 0x7fd8ac0051c0 2023-10-10T20:24:06.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.751+0000 7fd8b8ff9700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 0x7fd8a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:06.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.751+0000 7fd8bbfff700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 0x7fd8a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:06.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.751+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd8b0084290 con 0x7fd8ac0051c0 2023-10-10T20:24:06.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.751+0000 7fd8bbfff700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 0x7fd8a00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc156e90 tx=0x7fd8a80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:06.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.755+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8b0051850 con 0x7fd8ac0051c0 2023-10-10T20:24:06.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:06.935+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd8bc04c3c0 con 0x7fd8ac0051c0 2023-10-10T20:24:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.222+0000 7fd8b8ff9700 1 -- 172.21.15.148:0/2113032727 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd8b0054e70 con 0x7fd8ac0051c0 2023-10-10T20:24:07.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.225+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 msgr2=0x7fd8a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.225+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 0x7fd8a00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc156e90 tx=0x7fd8a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.225+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 msgr2=0x7fd8bc14e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.225+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0002950 tx=0x7fd8b0007d30 comp rx=0 tx=0).stop 2023-10-10T20:24:07.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 shutdown_connections 2023-10-10T20:24:07.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd8a0066f30 0x7fd8a00693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8bc155e60 0x7fd8bc1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8ac0051c0 0x7fd8bc14e780 secure :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0002950 tx=0x7fd8b0007d30 comp rx=0 tx=0).stop 2023-10-10T20:24:07.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 --2- 172.21.15.148:0/2113032727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8bc1515c0 0x7fd8bc14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.226+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 >> 172.21.15.148:0/2113032727 conn(0x7fd8bc069870 msgr2=0x7fd8bc159490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:07.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.227+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 shutdown_connections 2023-10-10T20:24:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.227+0000 7fd8c2e3d700 1 -- 172.21.15.148:0/2113032727 wait complete. 2023-10-10T20:24:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:24:07.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:24:07.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 Processor -- start 2023-10-10T20:24:07.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 -- start start 2023-10-10T20:24:07.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec15e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ec0c9390 0x7f06ec15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ec0cae70 con 0x7f06ec0678b0 2023-10-10T20:24:07.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec0cb150 con 0x7f06ec0c9390 2023-10-10T20:24:07.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f2c33700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec0cb430 con 0x7f06ec0cbf10 2023-10-10T20:24:07.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.362+0000 7f06f11d0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ec0c9390 0x7f06ec15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ec0c9390 0x7f06ec15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49278/0 (socket says 172.21.15.148:49278) 2023-10-10T20:24:07.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f09cf700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06ec0678b0 0x7f06ec15db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35738/0 (socket says 172.21.15.148:35738) 2023-10-10T20:24:07.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 -- 172.21.15.148:0/2417638919 learned_addr learned my addr 172.21.15.148:0/2417638919 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:07.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 -- 172.21.15.148:0/2417638919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 msgr2=0x7f06ec15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:07.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06e3fff700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06e37fe700 1 -- 172.21.15.148:0/2417638919 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728580625 0 0) 0x7f06ec0cae70 con 0x7f06ec0678b0 2023-10-10T20:24:07.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 -- 172.21.15.148:0/2417638919 >> v1:172.21.15.148:6789/0 conn(0x7f06ec0678b0 legacy=0x7f06ec15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 -- 172.21.15.148:0/2417638919 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0cb710 con 0x7f06ec0c9390 2023-10-10T20:24:07.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06f11d0700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0c9390 0x7f06ec15a370 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f06e40099b0 tx=0x7f06e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4782e8e12903c3fd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:07.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06e37fe700 1 -- 172.21.15.148:0/2417638919 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e400b860 con 0x7f06ec0c9390 2023-10-10T20:24:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06e37fe700 1 -- 172.21.15.148:0/2417638919 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e400b9c0 con 0x7f06ec0c9390 2023-10-10T20:24:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.363+0000 7f06e37fe700 1 -- 172.21.15.148:0/2417638919 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e400bd60 con 0x7f06ec0c9390 2023-10-10T20:24:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 -- 172.21.15.148:0/2417638919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0c9390 msgr2=0x7f06ec15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0c9390 0x7f06ec15a370 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f06e40099b0 tx=0x7f06e4009250 comp rx=0 tx=0).stop 2023-10-10T20:24:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 -- 172.21.15.148:0/2417638919 shutdown_connections 2023-10-10T20:24:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0c9390 0x7f06ec15a370 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2417638919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 -- 172.21.15.148:0/2417638919 >> 172.21.15.148:0/2417638919 conn(0x7f06ec069870 msgr2=0x7f06ec069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 -- 172.21.15.148:0/2417638919 shutdown_connections 2023-10-10T20:24:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.364+0000 7f06f2c33700 1 -- 172.21.15.148:0/2417638919 wait complete. 2023-10-10T20:24:07.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.365+0000 7f06f2c33700 1 Processor -- start 2023-10-10T20:24:07.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.365+0000 7f06f2c33700 1 -- start start 2023-10-10T20:24:07.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ec0c9390 0x7f06ec16a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec16a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec14ddf0 con 0x7f06ec0678b0 2023-10-10T20:24:07.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec0cc730 con 0x7f06ec0c9390 2023-10-10T20:24:07.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec16ab80 con 0x7f06ec0cbf10 2023-10-10T20:24:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49282/0 (socket says 172.21.15.148:49282) 2023-10-10T20:24:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 -- 172.21.15.148:0/2261320874 learned_addr learned my addr 172.21.15.148:0/2261320874 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f11d0700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec16a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06e3fff700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ec0c9390 0x7f06ec16a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 -- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 msgr2=0x7f06ec16a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec16a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 -- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ec0c9390 msgr2=0x7f06ec16a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ec0c9390 0x7f06ec16a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 -- 172.21.15.148:0/2261320874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40094f0 con 0x7f06ec0678b0 2023-10-10T20:24:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f09cf700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f06e800cdd0 tx=0x7f06e8001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:07.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8005df0 con 0x7f06ec0678b0 2023-10-10T20:24:07.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e800a960 con 0x7f06ec0678b0 2023-10-10T20:24:07.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800ad20 con 0x7f06ec0678b0 2023-10-10T20:24:07.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.366+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ec16ae60 con 0x7f06ec0678b0 2023-10-10T20:24:07.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.367+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec0ccc60 con 0x7f06ec0678b0 2023-10-10T20:24:07.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.367+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ec04c3c0 con 0x7f06ec0678b0 2023-10-10T20:24:07.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.368+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06e8007300 con 0x7f06ec0678b0 2023-10-10T20:24:07.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.368+0000 7f06e0ff9700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 0x7f06d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:07.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.368+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f06e80845b0 con 0x7f06ec0678b0 2023-10-10T20:24:07.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.368+0000 7f06e3fff700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 0x7f06d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:07.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.369+0000 7f06e3fff700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 0x7f06d80693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:07.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.373+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06e8051a80 con 0x7f06ec0678b0 2023-10-10T20:24:07.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:07.556+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06ec068da0 con 0x7f06ec0678b0 2023-10-10T20:24:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.225+0000 7f06e0ff9700 1 -- 172.21.15.148:0/2261320874 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v96) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06e80550a0 con 0x7f06ec0678b0 2023-10-10T20:24:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 msgr2=0x7f06d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 0x7f06d80693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006fd0 tx=0x7f06dc008040 comp rx=0 tx=0).stop 2023-10-10T20:24:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 msgr2=0x7f06ec14d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f06e800cdd0 tx=0x7f06e8001bc0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 shutdown_connections 2023-10-10T20:24:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f06d8066f30 0x7f06d80693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06ec0cbf10 0x7f06ec16a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0678b0 0x7f06ec14d8b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 --2- 172.21.15.148:0/2261320874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06ec0c9390 0x7f06ec16a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.233+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 >> 172.21.15.148:0/2261320874 conn(0x7f06ec069870 msgr2=0x7f06ec1505f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.235+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 shutdown_connections 2023-10-10T20:24:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.235+0000 7f06f2c33700 1 -- 172.21.15.148:0/2261320874 wait complete. 2023-10-10T20:24:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:24:08.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:24:08.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.368+0000 7fbe3f1bb700 1 Processor -- start 2023-10-10T20:24:08.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.368+0000 7fbe3f1bb700 1 -- start start 2023-10-10T20:24:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3f1bb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbe3815a250 0x7fbe3815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3f1bb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380b6b10 con 0x7fbe380b4b60 2023-10-10T20:24:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3f1bb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380b0e70 con 0x7fbe38159e80 2023-10-10T20:24:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3f1bb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380b2310 con 0x7fbe3815a250 2023-10-10T20:24:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d9b8700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbe380b4b60 0x7fbe380ab4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35750/0 (socket says 172.21.15.148:35750) 2023-10-10T20:24:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/1229449154 learned_addr learned my addr 172.21.15.148:0/1229449154 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:08.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3e9ba700 1 --2- 172.21.15.148:0/1229449154 >> v2:172.21.15.161:3300/0 conn(0x7fbe3815a250 0x7fbe3815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342251981 0 0) 0x7fbe380b6b10 con 0x7fbe380b4b60 2023-10-10T20:24:08.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe20003570 con 0x7fbe380b4b60 2023-10-10T20:24:08.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404840498 0 0) 0x7fbe380b0e70 con 0x7fbe38159e80 2023-10-10T20:24:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe380b6b10 con 0x7fbe38159e80 2023-10-10T20:24:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486810242 0 0) 0x7fbe20003570 con 0x7fbe380b4b60 2023-10-10T20:24:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 >> v2:172.21.15.161:3300/0 conn(0x7fbe3815a250 msgr2=0x7fbe3815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 --2- 172.21.15.148:0/1229449154 >> v2:172.21.15.161:3300/0 conn(0x7fbe3815a250 0x7fbe3815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 >> v1:172.21.15.161:6789/0 conn(0x7fbe38159e80 legacy=0x7fbe380aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380b5000 con 0x7fbe380b4b60 2023-10-10T20:24:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fbe2c001470 con 0x7fbe380b4b60 2023-10-10T20:24:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 >> v1:172.21.15.148:6789/0 conn(0x7fbe380b4b60 legacy=0x7fbe380ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20003bf0 0x7fbe20003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 0x7fbe20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe200051c0 0x7fbe2000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b2310 con 0x7fbe20004810 2023-10-10T20:24:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2000c0b0 con 0x7fbe20003bf0 2023-10-10T20:24:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.369+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2000c270 con 0x7fbe200051c0 2023-10-10T20:24:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 0x7fbe20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3e1b9700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20003bf0 0x7fbe20003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3e9ba700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe200051c0 0x7fbe2000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe200051c0 msgr2=0x7fbe2000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe200051c0 0x7fbe2000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20003bf0 msgr2=0x7fbe20003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:08.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe20003bf0 0x7fbe20003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/1229449154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20003570 con 0x7fbe20004810 2023-10-10T20:24:08.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3e9ba700 1 -- 172.21.15.148:0/1229449154 reap_dead start 2023-10-10T20:24:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 0x7fbe20004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c2d0 tx=0x7fbe2c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489676ea7a59b057 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00cb80 con 0x7fbe20004810 2023-10-10T20:24:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00cce0 con 0x7fbe20004810 2023-10-10T20:24:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.370+0000 7fbe3d1b7700 1 -- 172.21.15.148:0/1229449154 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c005b70 con 0x7fbe20004810 2023-10-10T20:24:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 msgr2=0x7fbe20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 0x7fbe20004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c2d0 tx=0x7fbe2c001e10 comp rx=0 tx=0).stop 2023-10-10T20:24:08.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/1229449154 shutdown_connections 2023-10-10T20:24:08.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/1229449154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20004810 0x7fbe20004c80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/1229449154 >> 172.21.15.148:0/1229449154 conn(0x7fbe38099f50 msgr2=0x7fbe38067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:08.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/1229449154 shutdown_connections 2023-10-10T20:24:08.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.371+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/1229449154 wait complete. 2023-10-10T20:24:08.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.372+0000 7fbe3f1bb700 1 Processor -- start 2023-10-10T20:24:08.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.372+0000 7fbe3f1bb700 1 -- start start 2023-10-10T20:24:08.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.372+0000 7fbe3f1bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe20004810 0x7fbe38157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe38157a50 0x7fbe38157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe381583f0 con 0x7fbe3809a8c0 2023-10-10T20:24:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3809b940 con 0x7fbe38157a50 2023-10-10T20:24:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3809ad30 con 0x7fbe20004810 2023-10-10T20:24:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49300/0 (socket says 172.21.15.148:49300) 2023-10-10T20:24:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/2952737838 learned_addr learned my addr 172.21.15.148:0/2952737838 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:08.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3e1b9700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe20004810 0x7fbe38157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3e9ba700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe38157a50 0x7fbe38157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe20004810 msgr2=0x7fbe38157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe20004810 0x7fbe38157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe38157a50 msgr2=0x7fbe38157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:08.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe38157a50 0x7fbe38157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:08.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 -- 172.21.15.148:0/2952737838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3809b010 con 0x7fbe3809a8c0 2023-10-10T20:24:08.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3d9b8700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c011000 tx=0x7fbe2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:08.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c001b30 con 0x7fbe3809a8c0 2023-10-10T20:24:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c004b10 con 0x7fbe3809a8c0 2023-10-10T20:24:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3809b2f0 con 0x7fbe3809a8c0 2023-10-10T20:24:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0070d0 con 0x7fbe3809a8c0 2023-10-10T20:24:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.373+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38157650 con 0x7fbe3809a8c0 2023-10-10T20:24:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.374+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3816a380 con 0x7fbe3809a8c0 2023-10-10T20:24:08.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.375+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe2c0074c0 con 0x7fbe3809a8c0 2023-10-10T20:24:08.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.375+0000 7fbe367fc700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:08.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.375+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fbe2c085840 con 0x7fbe3809a8c0 2023-10-10T20:24:08.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.375+0000 7fbe3e1b9700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:08.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.376+0000 7fbe3e1b9700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 0x7fbe240693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000c00 tx=0x7fbe30007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:08.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.380+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe3816a380 con 0x7fbe3809a8c0 2023-10-10T20:24:08.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:08.604+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbe3816a380 con 0x7fbe3809a8c0 2023-10-10T20:24:09.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.257+0000 7fbe367fc700 1 -- 172.21.15.148:0/2952737838 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbe3816a380 con 0x7fbe3809a8c0 2023-10-10T20:24:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 msgr2=0x7fbe240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 0x7fbe240693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000c00 tx=0x7fbe30007d30 comp rx=0 tx=0).stop 2023-10-10T20:24:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 msgr2=0x7fbe380695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c011000 tx=0x7fbe2c015000 comp rx=0 tx=0).stop 2023-10-10T20:24:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 shutdown_connections 2023-10-10T20:24:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe38157a50 0x7fbe38157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe20004810 0x7fbe38157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 --2- 172.21.15.148:0/2952737838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe3809a8c0 0x7fbe380695a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 >> 172.21.15.148:0/2952737838 conn(0x7fbe38099f50 msgr2=0x7fbe3815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:09.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 shutdown_connections 2023-10-10T20:24:09.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.259+0000 7fbe3f1bb700 1 -- 172.21.15.148:0/2952737838 wait complete. 2023-10-10T20:24:09.263 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 12 and data pool 13 2023-10-10T20:24:09.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:09.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.388+0000 7f870d105700 1 Processor -- start 2023-10-10T20:24:09.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.388+0000 7f870d105700 1 -- start start 2023-10-10T20:24:09.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870d105700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8708143700 0x7f8708068890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870d105700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f87080cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870d105700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080677f0 con 0x7f8708070950 2023-10-10T20:24:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870d105700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8708067ad0 con 0x7f8708143700 2023-10-10T20:24:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870d105700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8708067db0 con 0x7f8708070d20 2023-10-10T20:24:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f8706d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8708070950 0x7f8708142ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35752/0 (socket says 172.21.15.148:35752) 2023-10-10T20:24:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f8706d9d700 1 -- 172.21.15.148:0/2961955572 learned_addr learned my addr 172.21.15.148:0/2961955572 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037824988 0 0) 0x7f87080677f0 con 0x7f8708070950 2023-10-10T20:24:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870759e700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f87080cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4003570 con 0x7f8708070950 2023-10-10T20:24:09.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.389+0000 7f86fe59c700 1 --2- 172.21.15.148:0/2961955572 >> v2:172.21.15.161:3300/0 conn(0x7f8708143700 0x7f8708068890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544313502 0 0) 0x7f86f4003570 con 0x7f8708070950 2023-10-10T20:24:09.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 msgr2=0x7f87080cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f87080cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 >> v2:172.21.15.161:3300/0 conn(0x7f8708143700 msgr2=0x7f8708068890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 --2- 172.21.15.148:0/2961955572 >> v2:172.21.15.161:3300/0 conn(0x7f8708143700 0x7f8708068890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8708068090 con 0x7f8708070950 2023-10-10T20:24:09.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f86f8001470 con 0x7f8708070950 2023-10-10T20:24:09.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f86fe59c700 1 --2- 172.21.15.148:0/2961955572 >> v2:172.21.15.161:3300/0 conn(0x7f8708143700 0x7f8708068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:09.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 >> v1:172.21.15.148:6789/0 conn(0x7f8708070950 legacy=0x7f8708142ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f4003a60 0x7f86f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f86f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4004d50 0x7f86f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708067ad0 con 0x7f86f4003a60 2023-10-10T20:24:09.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708067db0 con 0x7f86f4004d50 2023-10-10T20:24:09.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f400bae0 con 0x7f8708070d20 2023-10-10T20:24:09.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f4003a60 0x7f86f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f86fe59c700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f86f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870759e700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4004d50 0x7f86f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 -- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 msgr2=0x7f86f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8708070d20 0x7f86f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 -- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4004d50 msgr2=0x7f86f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4004d50 0x7f86f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 -- 172.21.15.148:0/2961955572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4003570 con 0x7f86f4003a60 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f8706d9d700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f4003a60 0x7f86f4003e70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c3c0 tx=0x7f86f8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4019924f9d78d3d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f800cc90 con 0x7f86f4003a60 2023-10-10T20:24:09.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f800ce10 con 0x7f86f4003a60 2023-10-10T20:24:09.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.390+0000 7f870659c700 1 -- 172.21.15.148:0/2961955572 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8005b70 con 0x7f86f4003a60 2023-10-10T20:24:09.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 -- 172.21.15.148:0/2961955572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f4003a60 msgr2=0x7f86f4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 --2- 172.21.15.148:0/2961955572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86f4003a60 0x7f86f4003e70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c3c0 tx=0x7f86f8001e90 comp rx=0 tx=0).stop 2023-10-10T20:24:09.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870759e700 1 -- 172.21.15.148:0/2961955572 reap_dead start 2023-10-10T20:24:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 -- 172.21.15.148:0/2961955572 shutdown_connections 2023-10-10T20:24:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 -- 172.21.15.148:0/2961955572 >> 172.21.15.148:0/2961955572 conn(0x7f870806ad20 msgr2=0x7f87080cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 -- 172.21.15.148:0/2961955572 shutdown_connections 2023-10-10T20:24:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.391+0000 7f870d105700 1 -- 172.21.15.148:0/2961955572 wait complete. 2023-10-10T20:24:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.392+0000 7f870d105700 1 Processor -- start 2023-10-10T20:24:09.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.392+0000 7f870d105700 1 -- start start 2023-10-10T20:24:09.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 0x7f8708139580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f870815a4a0 0x7f87080c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870815ae80 con 0x7f870815a910 2023-10-10T20:24:09.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870815b160 con 0x7f86f4003a60 2023-10-10T20:24:09.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870d105700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870815b440 con 0x7f870815a4a0 2023-10-10T20:24:09.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49316/0 (socket says 172.21.15.148:49316) 2023-10-10T20:24:09.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 -- 172.21.15.148:0/4288042513 learned_addr learned my addr 172.21.15.148:0/4288042513 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:09.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f8706d9d700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 0x7f8708139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f86fe59c700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f870815a4a0 0x7f87080c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 -- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f870815a4a0 msgr2=0x7f87080c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f870815a4a0 0x7f87080c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 -- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 msgr2=0x7f8708139580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 0x7f8708139580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 -- 172.21.15.148:0/4288042513 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870815b720 con 0x7f870815a910 2023-10-10T20:24:09.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f8706d9d700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 0x7f8708139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f870759e700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f870806d8e0 tx=0x7f870000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.393+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000e040 con 0x7f870815a910 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.394+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f040 con 0x7f870815a910 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.394+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000fe20 con 0x7f870815a910 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.394+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8708139ac0 con 0x7f870815a910 2023-10-10T20:24:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.394+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870815c580 con 0x7f870815a910 2023-10-10T20:24:09.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.395+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870815ba50 con 0x7f870815a910 2023-10-10T20:24:09.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.395+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8700015640 con 0x7f870815a910 2023-10-10T20:24:09.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.395+0000 7f86ff7fe700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 0x7f86e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.395+0000 7f8706d9d700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 0x7f86e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.395+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8700084d20 con 0x7f870815a910 2023-10-10T20:24:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.396+0000 7f8706d9d700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 0x7f86e80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c1a0 tx=0x7f86f8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.400+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8700051f90 con 0x7f870815a910 2023-10-10T20:24:09.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.581+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f870804c3c0 con 0x7f870815a910 2023-10-10T20:24:09.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.582+0000 7f86ff7fe700 1 -- 172.21.15.148:0/4288042513 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f87000555b0 con 0x7f870815a910 2023-10-10T20:24:09.583 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:09.584 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:08.605228+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:09.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 msgr2=0x7f86e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 0x7f86e80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c1a0 tx=0x7f86f8015000 comp rx=0 tx=0).stop 2023-10-10T20:24:09.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 msgr2=0x7f87080c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f870806d8e0 tx=0x7f870000b040 comp rx=0 tx=0).stop 2023-10-10T20:24:09.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 shutdown_connections 2023-10-10T20:24:09.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f86e8066f30 0x7f86e80693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f870815a910 0x7f87080c99d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86f4003a60 0x7f8708139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 --2- 172.21.15.148:0/4288042513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f870815a4a0 0x7f87080c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 >> 172.21.15.148:0/4288042513 conn(0x7f870806ad20 msgr2=0x7f870806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:09.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 shutdown_connections 2023-10-10T20:24:09.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.584+0000 7f870d105700 1 -- 172.21.15.148:0/4288042513 wait complete. 2023-10-10T20:24:09.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.711+0000 7f063b04a700 1 Processor -- start 2023-10-10T20:24:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.711+0000 7f063b04a700 1 -- start start 2023-10-10T20:24:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f063415a620 0x7f063415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f063415d580 0x7f0634158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063415cbb0 con 0x7f063415a620 2023-10-10T20:24:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063415bfa0 con 0x7f063415d580 2023-10-10T20:24:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f063b04a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063415c280 con 0x7f0634157870 2023-10-10T20:24:09.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f062bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f063415a620 0x7f063415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f062bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f063415a620 0x7f063415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49322/0 (socket says 172.21.15.148:49322) 2023-10-10T20:24:09.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f062bfff700 1 -- 172.21.15.148:0/4292297474 learned_addr learned my addr 172.21.15.148:0/4292297474 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:09.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.712+0000 7f0638de6700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f06395e7700 1 --2- 172.21.15.148:0/4292297474 >> v2:172.21.15.161:3300/0 conn(0x7f063415d580 0x7f0634158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 -- 172.21.15.148:0/4292297474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 msgr2=0x7f063415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 -- 172.21.15.148:0/4292297474 >> v2:172.21.15.161:3300/0 conn(0x7f063415d580 msgr2=0x7f0634158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:09.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 --2- 172.21.15.148:0/4292297474 >> v2:172.21.15.161:3300/0 conn(0x7f063415d580 0x7f0634158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 -- 172.21.15.148:0/4292297474 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063415c560 con 0x7f063415a620 2023-10-10T20:24:09.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062bfff700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f063415d040 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f06340b03b0 tx=0x7f0624009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3b961d870bd1d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062b7fe700 1 -- 172.21.15.148:0/4292297474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062400b640 con 0x7f063415a620 2023-10-10T20:24:09.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062b7fe700 1 -- 172.21.15.148:0/4292297474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062400b7a0 con 0x7f063415a620 2023-10-10T20:24:09.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.713+0000 7f062b7fe700 1 -- 172.21.15.148:0/4292297474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062400bb60 con 0x7f063415a620 2023-10-10T20:24:09.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 -- 172.21.15.148:0/4292297474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 msgr2=0x7f063415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f063415d040 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f06340b03b0 tx=0x7f0624009250 comp rx=0 tx=0).stop 2023-10-10T20:24:09.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 -- 172.21.15.148:0/4292297474 shutdown_connections 2023-10-10T20:24:09.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 --2- 172.21.15.148:0/4292297474 >> v2:172.21.15.161:3300/0 conn(0x7f063415d580 0x7f0634158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 --2- 172.21.15.148:0/4292297474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f063415d040 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 -- 172.21.15.148:0/4292297474 >> 172.21.15.148:0/4292297474 conn(0x7f0634068ce0 msgr2=0x7f06340690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:09.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 -- 172.21.15.148:0/4292297474 shutdown_connections 2023-10-10T20:24:09.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.714+0000 7f063b04a700 1 -- 172.21.15.148:0/4292297474 wait complete. 2023-10-10T20:24:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.715+0000 7f063b04a700 1 Processor -- start 2023-10-10T20:24:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.715+0000 7f063b04a700 1 -- start start 2023-10-10T20:24:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f063415d580 0x7f0634153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063415e650 con 0x7f063415a620 2023-10-10T20:24:09.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063415dab0 con 0x7f063415d580 2023-10-10T20:24:09.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063415dd20 con 0x7f0634157870 2023-10-10T20:24:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49330/0 (socket says 172.21.15.148:49330) 2023-10-10T20:24:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 -- 172.21.15.148:0/1618890606 learned_addr learned my addr 172.21.15.148:0/1618890606 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f0638de6700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f06395e7700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f063415d580 0x7f0634153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 -- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 msgr2=0x7f063414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063414a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 -- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f063415d580 msgr2=0x7f0634153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f063415d580 0x7f0634153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 -- 172.21.15.148:0/1618890606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06240094f0 con 0x7f063415a620 2023-10-10T20:24:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f062bfff700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624000d10 con 0x7f063415a620 2023-10-10T20:24:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062400bb20 con 0x7f063415a620 2023-10-10T20:24:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624011670 con 0x7f063415a620 2023-10-10T20:24:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.716+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063415e000 con 0x7f063415a620 2023-10-10T20:24:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.717+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063414ae70 con 0x7f063415a620 2023-10-10T20:24:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.718+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0634070b10 con 0x7f063415a620 2023-10-10T20:24:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.718+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06240119e0 con 0x7f063415a620 2023-10-10T20:24:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.718+0000 7f0628ff9700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 0x7f06200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.718+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0624085db0 con 0x7f063415a620 2023-10-10T20:24:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.718+0000 7f0638de6700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 0x7f06200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.719+0000 7f0638de6700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 0x7f06200693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06300093f0 tx=0x7f0630008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:09.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.723+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0624053020 con 0x7f063415a620 2023-10-10T20:24:09.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.906+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f063404c3c0 con 0x7f063415a620 2023-10-10T20:24:09.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.907+0000 7f0628ff9700 1 -- 172.21.15.148:0/1618890606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f0624056640 con 0x7f063415a620 2023-10-10T20:24:09.908 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:09.908 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:08.605228+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:09.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 msgr2=0x7f06200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 0x7f06200693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f06300093f0 tx=0x7f0630008040 comp rx=0 tx=0).stop 2023-10-10T20:24:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 msgr2=0x7f06340b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0624006fd0 tx=0x7f0624017000 comp rx=0 tx=0).stop 2023-10-10T20:24:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 shutdown_connections 2023-10-10T20:24:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0620066f30 0x7f06200693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f063415d580 0x7f0634153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0634157870 0x7f063414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 --2- 172.21.15.148:0/1618890606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f063415a620 0x7f06340b1290 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 >> 172.21.15.148:0/1618890606 conn(0x7f0634068ce0 msgr2=0x7f0634156990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 shutdown_connections 2023-10-10T20:24:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:09.909+0000 7f063b04a700 1 -- 172.21.15.148:0/1618890606 wait complete. 2023-10-10T20:24:09.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:24:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 Processor -- start 2023-10-10T20:24:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 -- start start 2023-10-10T20:24:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff7040cce30 0x7ff70415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040b4d70 con 0x7ff7040b6580 2023-10-10T20:24:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040cd8d0 con 0x7ff70406aa00 2023-10-10T20:24:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.040+0000 7ff7096fb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7040b4440 con 0x7ff7040cce30 2023-10-10T20:24:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff7037fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff7040b6580 0x7ff70406a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35768/0 (socket says 172.21.15.148:35768) 2023-10-10T20:24:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff7037fe700 1 -- 172.21.15.148:0/4136745796 learned_addr learned my addr 172.21.15.148:0/4136745796 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff703fff700 1 --2- 172.21.15.148:0/4136745796 >> v2:172.21.15.161:3300/0 conn(0x7ff7040cce30 0x7ff70415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323682514 0 0) 0x7ff7040b4d70 con 0x7ff7040b6580 2023-10-10T20:24:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ec003570 con 0x7ff7040b6580 2023-10-10T20:24:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138928501 0 0) 0x7ff7040cd8d0 con 0x7ff70406aa00 2023-10-10T20:24:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7040b4d70 con 0x7ff70406aa00 2023-10-10T20:24:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff703fff700 1 -- 172.21.15.148:0/4136745796 >> v1:172.21.15.161:6789/0 conn(0x7ff70406aa00 legacy=0x7ff7040b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff703fff700 1 -- 172.21.15.148:0/4136745796 >> v1:172.21.15.148:6789/0 conn(0x7ff7040b6580 legacy=0x7ff70406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff703fff700 1 -- 172.21.15.148:0/4136745796 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040b4720 con 0x7ff7040cce30 2023-10-10T20:24:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff703fff700 1 --2- 172.21.15.148:0/4136745796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7040cce30 0x7ff70415aca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40098e0 tx=0x7ff6f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac64a94957eece9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011630 con 0x7ff7040cce30 2023-10-10T20:24:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f4011790 con 0x7ff7040cce30 2023-10-10T20:24:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.041+0000 7ff702ffd700 1 -- 172.21.15.148:0/4136745796 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011b10 con 0x7ff7040cce30 2023-10-10T20:24:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 -- 172.21.15.148:0/4136745796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7040cce30 msgr2=0x7ff70415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 --2- 172.21.15.148:0/4136745796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7040cce30 0x7ff70415aca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40098e0 tx=0x7ff6f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:10.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 -- 172.21.15.148:0/4136745796 shutdown_connections 2023-10-10T20:24:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 --2- 172.21.15.148:0/4136745796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7040cce30 0x7ff70415aca0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 -- 172.21.15.148:0/4136745796 >> 172.21.15.148:0/4136745796 conn(0x7ff704068cb0 msgr2=0x7ff7040690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 -- 172.21.15.148:0/4136745796 shutdown_connections 2023-10-10T20:24:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.042+0000 7ff7096fb700 1 -- 172.21.15.148:0/4136745796 wait complete. 2023-10-10T20:24:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.043+0000 7ff7096fb700 1 Processor -- start 2023-10-10T20:24:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.043+0000 7ff7096fb700 1 -- start start 2023-10-10T20:24:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff70406aa00 0x7ff7040cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff7040b6580 0x7ff704158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704153540 con 0x7ff7040cce30 2023-10-10T20:24:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704153820 con 0x7ff70406aa00 2023-10-10T20:24:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7096fb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff704153b00 con 0x7ff7040b6580 2023-10-10T20:24:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49334/0 (socket says 172.21.15.148:49334) 2023-10-10T20:24:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 -- 172.21.15.148:0/3632359847 learned_addr learned my addr 172.21.15.148:0/3632359847 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff703fff700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff70406aa00 0x7ff7040cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:10.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff7037fe700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff7040b6580 0x7ff704158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:10.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 -- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff7040b6580 msgr2=0x7ff704158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:10.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff7040b6580 0x7ff704158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:10.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 -- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff70406aa00 msgr2=0x7ff7040cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:10.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff70406aa00 0x7ff7040cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:10.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 -- 172.21.15.148:0/3632359847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4009420 con 0x7ff7040cce30 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.044+0000 7ff708efa700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00c910 tx=0x7ff6fc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.045+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00a7a0 con 0x7ff7040cce30 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.045+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc00a920 con 0x7ff7040cce30 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.045+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00ace0 con 0x7ff7040cce30 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.045+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff704153de0 con 0x7ff7040cce30 2023-10-10T20:24:10.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.045+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7041529c0 con 0x7ff7040cce30 2023-10-10T20:24:10.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.046+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff704064990 con 0x7ff7040cce30 2023-10-10T20:24:10.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.046+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6fc012630 con 0x7ff7040cce30 2023-10-10T20:24:10.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.046+0000 7ff6ebfff700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 0x7ff6f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:10.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.046+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6fc0849b0 con 0x7ff7040cce30 2023-10-10T20:24:10.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.046+0000 7ff703fff700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 0x7ff6f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:10.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.047+0000 7ff703fff700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 0x7ff6f00693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b3460 tx=0x7ff6f4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:10.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.050+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6fc051cd0 con 0x7ff7040cce30 2023-10-10T20:24:10.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:10.278+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff7040b4a00 con 0x7ff7040cce30 2023-10-10T20:24:11.199 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-10-10T20:24:11.198+0000 7f919ff55700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:24:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.265+0000 7ff6ebfff700 1 -- 172.21.15.148:0/3632359847 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6fc0552f0 con 0x7ff7040cce30 2023-10-10T20:24:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 msgr2=0x7ff6f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 0x7ff6f00693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b3460 tx=0x7ff6f4017000 comp rx=0 tx=0).stop 2023-10-10T20:24:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 msgr2=0x7ff7041585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00c910 tx=0x7ff6fc007f90 comp rx=0 tx=0).stop 2023-10-10T20:24:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 shutdown_connections 2023-10-10T20:24:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff6f0066f30 0x7ff6f00693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7040cce30 0x7ff7041585d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff70406aa00 0x7ff7040cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 --2- 172.21.15.148:0/3632359847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff7040b6580 0x7ff704158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.267+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 >> 172.21.15.148:0/3632359847 conn(0x7ff704068cb0 msgr2=0x7ff7040cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:11.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.268+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 shutdown_connections 2023-10-10T20:24:11.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.268+0000 7ff7096fb700 1 -- 172.21.15.148:0/3632359847 wait complete. 2023-10-10T20:24:11.283 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:24:11.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.404+0000 7f542dfe6700 1 Processor -- start 2023-10-10T20:24:11.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.404+0000 7f542dfe6700 1 -- start start 2023-10-10T20:24:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b4860 0x7f54280b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 0x7f5428159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428159fd0 0x7f54280cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280b14f0 con 0x7f54280b4860 2023-10-10T20:24:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280b17d0 con 0x7f54280b4c30 2023-10-10T20:24:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f542dfe6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54280b1ab0 con 0x7f5428159fd0 2023-10-10T20:24:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f54277fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b4860 0x7f54280b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f54277fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54280b4860 0x7f54280b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49344/0 (socket says 172.21.15.148:49344) 2023-10-10T20:24:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f54277fe700 1 -- 172.21.15.148:0/2159901830 learned_addr learned my addr 172.21.15.148:0/2159901830 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f5427fff700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428159fd0 0x7f54280cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.405+0000 7f5426ffd700 1 --2- 172.21.15.148:0/2159901830 >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 0x7f5428159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 -- 172.21.15.148:0/2159901830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428159fd0 msgr2=0x7f54280cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:11.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428159fd0 0x7f54280cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 -- 172.21.15.148:0/2159901830 >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 msgr2=0x7f5428159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:11.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 --2- 172.21.15.148:0/2159901830 >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 0x7f5428159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 -- 172.21.15.148:0/2159901830 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280b1d90 con 0x7f54280b4860 2023-10-10T20:24:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f5426ffd700 1 --2- 172.21.15.148:0/2159901830 >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 0x7f5428159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54277fe700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f54280b6700 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f54280899c0 tx=0x7f5418009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a06aac04f57e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54267fc700 1 -- 172.21.15.148:0/2159901830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800e070 con 0x7f54280b4860 2023-10-10T20:24:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54267fc700 1 -- 172.21.15.148:0/2159901830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800ba80 con 0x7f54280b4860 2023-10-10T20:24:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f54267fc700 1 -- 172.21.15.148:0/2159901830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180092e0 con 0x7f54280b4860 2023-10-10T20:24:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 -- 172.21.15.148:0/2159901830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 msgr2=0x7f54280b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f54280b6700 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f54280899c0 tx=0x7f5418009840 comp rx=0 tx=0).stop 2023-10-10T20:24:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 -- 172.21.15.148:0/2159901830 shutdown_connections 2023-10-10T20:24:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5428159fd0 0x7f54280cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 --2- 172.21.15.148:0/2159901830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f54280b6700 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 --2- 172.21.15.148:0/2159901830 >> v2:172.21.15.161:3300/0 conn(0x7f54280b4c30 0x7f5428159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.406+0000 7f542dfe6700 1 -- 172.21.15.148:0/2159901830 >> 172.21.15.148:0/2159901830 conn(0x7f542809a0f0 msgr2=0x7f5428067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:11.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.407+0000 7f542dfe6700 1 -- 172.21.15.148:0/2159901830 shutdown_connections 2023-10-10T20:24:11.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.407+0000 7f542dfe6700 1 -- 172.21.15.148:0/2159901830 wait complete. 2023-10-10T20:24:11.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 Processor -- start 2023-10-10T20:24:11.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 -- start start 2023-10-10T20:24:11.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54280b4c30 0x7f54280ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428159fd0 0x7f5428156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542809d1c0 con 0x7f54280b4860 2023-10-10T20:24:11.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542809c5b0 con 0x7f5428159fd0 2023-10-10T20:24:11.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.408+0000 7f542dfe6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542809c890 con 0x7f54280b4c30 2023-10-10T20:24:11.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49346/0 (socket says 172.21.15.148:49346) 2023-10-10T20:24:11.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 -- 172.21.15.148:0/3555333642 learned_addr learned my addr 172.21.15.148:0/3555333642 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:11.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f5426ffd700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54280b4c30 0x7f54280ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f5427fff700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428159fd0 0x7f5428156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 -- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54280b4c30 msgr2=0x7f54280ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54280b4c30 0x7f54280ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 -- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428159fd0 msgr2=0x7f5428156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:11.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428159fd0 0x7f5428156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:11.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 -- 172.21.15.148:0/3555333642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009ae0 con 0x7f54280b4860 2023-10-10T20:24:11.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f54277fe700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5428068cd0 tx=0x7f5418018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:11.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418008980 con 0x7f54280b4860 2023-10-10T20:24:11.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418008b20 con 0x7f54280b4860 2023-10-10T20:24:11.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542809cb70 con 0x7f54280b4860 2023-10-10T20:24:11.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541801d420 con 0x7f54280b4860 2023-10-10T20:24:11.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.409+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54280ad560 con 0x7f54280b4860 2023-10-10T20:24:11.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.410+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542804c3c0 con 0x7f54280b4860 2023-10-10T20:24:11.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.411+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5418003040 con 0x7f54280b4860 2023-10-10T20:24:11.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.411+0000 7f540f7fe700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:11.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.411+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5418085df0 con 0x7f54280b4860 2023-10-10T20:24:11.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.411+0000 7f5426ffd700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:11.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.411+0000 7f5426ffd700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f542809e880 tx=0x7f541c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:11.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.415+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5418053060 con 0x7f54280b4860 2023-10-10T20:24:11.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:11.640+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f54280b2070 con 0x7f54280b4860 2023-10-10T20:24:12.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.269+0000 7f540f7fe700 1 -- 172.21.15.148:0/3555333642 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5418009940 con 0x7f54280b4860 2023-10-10T20:24:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 msgr2=0x7f54140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f542809e880 tx=0x7f541c008040 comp rx=0 tx=0).stop 2023-10-10T20:24:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 msgr2=0x7f542809d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5428068cd0 tx=0x7f5418018000 comp rx=0 tx=0).stop 2023-10-10T20:24:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 shutdown_connections 2023-10-10T20:24:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428159fd0 0x7f5428156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54280b4860 0x7f542809d650 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 --2- 172.21.15.148:0/3555333642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54280b4c30 0x7f54280ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 >> 172.21.15.148:0/3555333642 conn(0x7f542809a0f0 msgr2=0x7f542815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 shutdown_connections 2023-10-10T20:24:12.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.271+0000 7f542dfe6700 1 -- 172.21.15.148:0/3555333642 wait complete. 2023-10-10T20:24:12.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 Processor -- start 2023-10-10T20:24:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 -- start start 2023-10-10T20:24:12.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43d4156570 0x7f43d406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d4157c30 con 0x7f43d415a330 2023-10-10T20:24:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d4157490 con 0x7f43d4156570 2023-10-10T20:24:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43da9f2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41584f0 con 0x7f43d4157050 2023-10-10T20:24:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.400+0000 7f43d3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43d4156570 0x7f43d406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43d4156570 0x7f43d406d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49350/0 (socket says 172.21.15.148:49350) 2023-10-10T20:24:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 -- 172.21.15.148:0/155779227 learned_addr learned my addr 172.21.15.148:0/155779227 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d8f8f700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 -- 172.21.15.148:0/155779227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 msgr2=0x7f43d406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d406d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 -- 172.21.15.148:0/155779227 >> v1:172.21.15.161:6789/0 conn(0x7f43d415a330 legacy=0x7f43d4156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 -- 172.21.15.148:0/155779227 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d4157670 con 0x7f43d4156570 2023-10-10T20:24:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d3fff700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d406d380 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f43c40099b0 tx=0x7f43c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d678ae0d3ff6a4e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d2ffd700 1 -- 172.21.15.148:0/155779227 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400b860 con 0x7f43d4156570 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d2ffd700 1 -- 172.21.15.148:0/155779227 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400b9c0 con 0x7f43d4156570 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.401+0000 7f43d2ffd700 1 -- 172.21.15.148:0/155779227 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400bd80 con 0x7f43d4156570 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 -- 172.21.15.148:0/155779227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 msgr2=0x7f43d406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d406d380 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f43c40099b0 tx=0x7f43c4009250 comp rx=0 tx=0).stop 2023-10-10T20:24:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 -- 172.21.15.148:0/155779227 shutdown_connections 2023-10-10T20:24:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d406d380 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 --2- 172.21.15.148:0/155779227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d406d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 -- 172.21.15.148:0/155779227 >> 172.21.15.148:0/155779227 conn(0x7f43d4067660 msgr2=0x7f43d4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 -- 172.21.15.148:0/155779227 shutdown_connections 2023-10-10T20:24:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.402+0000 7f43da9f2700 1 -- 172.21.15.148:0/155779227 wait complete. 2023-10-10T20:24:12.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.403+0000 7f43da9f2700 1 Processor -- start 2023-10-10T20:24:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.403+0000 7f43da9f2700 1 -- start start 2023-10-10T20:24:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d415a330 0x7f43d414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40b1e20 con 0x7f43d4156570 2023-10-10T20:24:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40b2100 con 0x7f43d415a330 2023-10-10T20:24:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40b23e0 con 0x7f43d4157050 2023-10-10T20:24:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:49360/0 (socket says 172.21.15.148:49360) 2023-10-10T20:24:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 -- 172.21.15.148:0/2594547491 learned_addr learned my addr 172.21.15.148:0/2594547491 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d8f8f700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d415a330 0x7f43d414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d37fe700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 -- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 msgr2=0x7f43d414c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d414c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 -- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d415a330 msgr2=0x7f43d414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d415a330 0x7f43d414cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 -- 172.21.15.148:0/2594547491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40094f0 con 0x7f43d4156570 2023-10-10T20:24:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43d3fff700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f43d406ba00 tx=0x7f43c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c401f070 con 0x7f43d4156570 2023-10-10T20:24:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4006ab0 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d40c3920 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400b860 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.404+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d4149210 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.405+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d404c3c0 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.406+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43c401a630 con 0x7f43d4156570 2023-10-10T20:24:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.406+0000 7f43bbfff700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 0x7f43c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.406+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43c4085810 con 0x7f43d4156570 2023-10-10T20:24:12.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.406+0000 7f43d37fe700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 0x7f43c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.407+0000 7f43d37fe700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 0x7f43c00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43d40c2d90 tx=0x7f43c8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.411+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43c4052b00 con 0x7f43d4156570 2023-10-10T20:24:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.633+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43d4157950 con 0x7f43d4156570 2023-10-10T20:24:12.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.634+0000 7f43bbfff700 1 -- 172.21.15.148:0/2594547491 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f43c4015090 con 0x7f43d4156570 2023-10-10T20:24:12.635 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:12.635 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:12.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 msgr2=0x7f43c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 0x7f43c00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43d40c2d90 tx=0x7f43c8007ec0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 msgr2=0x7f43d4148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f43d406ba00 tx=0x7f43c4017000 comp rx=0 tx=0).stop 2023-10-10T20:24:12.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 shutdown_connections 2023-10-10T20:24:12.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43c0066f30 0x7f43c00693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d415a330 0x7f43d414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4156570 0x7f43d4148cd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 --2- 172.21.15.148:0/2594547491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43d4157050 0x7f43d414c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 >> 172.21.15.148:0/2594547491 conn(0x7f43d4067660 msgr2=0x7f43d4159800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:12.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 shutdown_connections 2023-10-10T20:24:12.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.636+0000 7f43da9f2700 1 -- 172.21.15.148:0/2594547491 wait complete. 2023-10-10T20:24:12.641 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:12.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.763+0000 7f7712c9f700 1 Processor -- start 2023-10-10T20:24:12.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.763+0000 7f7712c9f700 1 -- start start 2023-10-10T20:24:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f770c0cd6e0 0x7f770c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f770c15dda0 0x7f770c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c15e850 con 0x7f770c15dda0 2023-10-10T20:24:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0cd030 con 0x7f770c0cd6e0 2023-10-10T20:24:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7712c9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0cb7c0 con 0x7f770c0cdab0 2023-10-10T20:24:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f771249e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f770c15dda0 0x7f770c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f771249e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f770c15dda0 0x7f770c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42832/0 (socket says 172.21.15.148:42832) 2023-10-10T20:24:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f771249e700 1 -- 172.21.15.148:0/965571215 learned_addr learned my addr 172.21.15.148:0/965571215 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f771149c700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.764+0000 7f7711c9d700 1 --2- 172.21.15.148:0/965571215 >> v2:172.21.15.161:3300/0 conn(0x7f770c0cd6e0 0x7f770c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 -- 172.21.15.148:0/965571215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 msgr2=0x7f770c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 -- 172.21.15.148:0/965571215 >> v2:172.21.15.161:3300/0 conn(0x7f770c0cd6e0 msgr2=0x7f770c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 --2- 172.21.15.148:0/965571215 >> v2:172.21.15.161:3300/0 conn(0x7f770c0cd6e0 0x7f770c0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 -- 172.21.15.148:0/965571215 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c15e290 con 0x7f770c15dda0 2023-10-10T20:24:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f771249e700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c15dda0 0x7f770c15cbd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f770c06b730 tx=0x7f7704009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35382ba9a0951562 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7710c9b700 1 -- 172.21.15.148:0/965571215 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770400e070 con 0x7f770c15dda0 2023-10-10T20:24:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7710c9b700 1 -- 172.21.15.148:0/965571215 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770400b970 con 0x7f770c15dda0 2023-10-10T20:24:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7710c9b700 1 -- 172.21.15.148:0/965571215 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040092e0 con 0x7f770c15dda0 2023-10-10T20:24:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 -- 172.21.15.148:0/965571215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c15dda0 msgr2=0x7f770c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c15dda0 0x7f770c15cbd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f770c06b730 tx=0x7f7704009840 comp rx=0 tx=0).stop 2023-10-10T20:24:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 -- 172.21.15.148:0/965571215 shutdown_connections 2023-10-10T20:24:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c15dda0 0x7f770c15cbd0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 --2- 172.21.15.148:0/965571215 >> v2:172.21.15.161:3300/0 conn(0x7f770c0cd6e0 0x7f770c0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 --2- 172.21.15.148:0/965571215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.765+0000 7f7712c9f700 1 -- 172.21.15.148:0/965571215 >> 172.21.15.148:0/965571215 conn(0x7f770c067660 msgr2=0x7f770c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.766+0000 7f7712c9f700 1 -- 172.21.15.148:0/965571215 shutdown_connections 2023-10-10T20:24:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.766+0000 7f7712c9f700 1 -- 172.21.15.148:0/965571215 wait complete. 2023-10-10T20:24:12.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 Processor -- start 2023-10-10T20:24:12.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 -- start start 2023-10-10T20:24:12.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c14c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 0x7f770c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0b5850 con 0x7f770c0cd6e0 2023-10-10T20:24:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0d0fd0 con 0x7f770c15dda0 2023-10-10T20:24:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7712c9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0b4e80 con 0x7f770c0cdab0 2023-10-10T20:24:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7711c9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7711c9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42842/0 (socket says 172.21.15.148:42842) 2023-10-10T20:24:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.767+0000 7f7711c9d700 1 -- 172.21.15.148:0/1552818477 learned_addr learned my addr 172.21.15.148:0/1552818477 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f771149c700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c14c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 -- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 msgr2=0x7f770c14c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f771249e700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 0x7f770c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c14c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 -- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 msgr2=0x7f770c0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 0x7f770c0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 -- 172.21.15.148:0/1552818477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704009ae0 con 0x7f770c0cd6e0 2023-10-10T20:24:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f771149c700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c14c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f771249e700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 0x7f770c0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:12.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7711c9d700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f770800e930 tx=0x7f77080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800eed0 con 0x7f770c0cd6e0 2023-10-10T20:24:12.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800c850 con 0x7f770c0cd6e0 2023-10-10T20:24:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c14ce30 con 0x7f770c0cd6e0 2023-10-10T20:24:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800cbf0 con 0x7f770c0cd6e0 2023-10-10T20:24:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.768+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c0b4830 con 0x7f770c0cd6e0 2023-10-10T20:24:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.769+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c089280 con 0x7f770c0cd6e0 2023-10-10T20:24:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.770+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7708013470 con 0x7f770c0cd6e0 2023-10-10T20:24:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.770+0000 7f7701ffb700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 0x7f76f0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.770+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7708084720 con 0x7f770c0cd6e0 2023-10-10T20:24:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.770+0000 7f771149c700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 0x7f76f0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.770+0000 7f771149c700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 0x7f76f0069410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006f00 tx=0x7f76fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.774+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7708051990 con 0x7f770c0cd6e0 2023-10-10T20:24:12.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.957+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f770c15e570 con 0x7f770c0cd6e0 2023-10-10T20:24:12.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.958+0000 7f7701ffb700 1 -- 172.21.15.148:0/1552818477 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f7708054fb0 con 0x7f770c0cd6e0 2023-10-10T20:24:12.959 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:12.960 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":98,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:10.239239+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:12.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 msgr2=0x7f76f0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 0x7f76f0069410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006f00 tx=0x7f76fc008040 comp rx=0 tx=0).stop 2023-10-10T20:24:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 msgr2=0x7f770c0b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f770800e930 tx=0x7f77080091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 shutdown_connections 2023-10-10T20:24:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f76f0066f60 0x7f76f0069410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f770c15dda0 0x7f770c0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c0cd6e0 0x7f770c0b5310 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 --2- 172.21.15.148:0/1552818477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f770c0cdab0 0x7f770c14c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:12.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 >> 172.21.15.148:0/1552818477 conn(0x7f770c067660 msgr2=0x7f770c160070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:12.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 shutdown_connections 2023-10-10T20:24:12.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:12.960+0000 7f7712c9f700 1 -- 172.21.15.148:0/1552818477 wait complete. 2023-10-10T20:24:12.974 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:24:12.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:24:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.088+0000 7f46cef0c700 1 Processor -- start 2023-10-10T20:24:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.088+0000 7f46cef0c700 1 -- start start 2023-10-10T20:24:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cef0c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46c80cd6e0 0x7f46c80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cef0c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f46c80cdab0 0x7f46c815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cef0c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80cb7c0 con 0x7f46c815dda0 2023-10-10T20:24:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cef0c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c815e850 con 0x7f46c80cd6e0 2023-10-10T20:24:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cef0c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80cd030 con 0x7f46c80cdab0 2023-10-10T20:24:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cdf0a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46c80cd6e0 0x7f46c80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46ce70b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46c815dda0 0x7f46c815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41092/0 (socket says 172.21.15.148:41092) 2023-10-10T20:24:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46ce70b700 1 -- 172.21.15.148:0/674771487 learned_addr learned my addr 172.21.15.148:0/674771487 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cd709700 1 --2- 172.21.15.148:0/674771487 >> v2:172.21.15.161:3300/0 conn(0x7f46c80cdab0 0x7f46c815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cdf0a700 1 -- 172.21.15.148:0/674771487 >> v2:172.21.15.161:3300/0 conn(0x7f46c80cdab0 msgr2=0x7f46c815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cdf0a700 1 --2- 172.21.15.148:0/674771487 >> v2:172.21.15.161:3300/0 conn(0x7f46c80cdab0 0x7f46c815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.089+0000 7f46cdf0a700 1 -- 172.21.15.148:0/674771487 >> v1:172.21.15.148:6789/0 conn(0x7f46c815dda0 legacy=0x7f46c815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46ccf08700 1 -- 172.21.15.148:0/674771487 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678182389 0 0) 0x7f46c80cb7c0 con 0x7f46c815dda0 2023-10-10T20:24:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cdf0a700 1 -- 172.21.15.148:0/674771487 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c815e290 con 0x7f46c80cd6e0 2023-10-10T20:24:13.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cdf0a700 1 --2- 172.21.15.148:0/674771487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cd6e0 0x7f46c80cbc50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b6c70 tx=0x7f46c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a301e159176000 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46ccf08700 1 -- 172.21.15.148:0/674771487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c400e070 con 0x7f46c80cd6e0 2023-10-10T20:24:13.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46ccf08700 1 -- 172.21.15.148:0/674771487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c400bc30 con 0x7f46c80cd6e0 2023-10-10T20:24:13.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46ccf08700 1 -- 172.21.15.148:0/674771487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4012430 con 0x7f46c80cd6e0 2023-10-10T20:24:13.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 -- 172.21.15.148:0/674771487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cd6e0 msgr2=0x7f46c80cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 --2- 172.21.15.148:0/674771487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cd6e0 0x7f46c80cbc50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b6c70 tx=0x7f46c4009780 comp rx=0 tx=0).stop 2023-10-10T20:24:13.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 -- 172.21.15.148:0/674771487 shutdown_connections 2023-10-10T20:24:13.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 --2- 172.21.15.148:0/674771487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cd6e0 0x7f46c80cbc50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 --2- 172.21.15.148:0/674771487 >> v2:172.21.15.161:3300/0 conn(0x7f46c80cdab0 0x7f46c815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 -- 172.21.15.148:0/674771487 >> 172.21.15.148:0/674771487 conn(0x7f46c8067660 msgr2=0x7f46c8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 -- 172.21.15.148:0/674771487 shutdown_connections 2023-10-10T20:24:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.090+0000 7f46cef0c700 1 -- 172.21.15.148:0/674771487 wait complete. 2023-10-10T20:24:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 Processor -- start 2023-10-10T20:24:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 -- start start 2023-10-10T20:24:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46c80cd6e0 0x7f46c8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 0x7f46c8159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b0130 con 0x7f46c80cdab0 2023-10-10T20:24:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b5430 con 0x7f46c80cd6e0 2023-10-10T20:24:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cef0c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c815c0b0 con 0x7f46c815dda0 2023-10-10T20:24:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cd709700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cd709700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42860/0 (socket says 172.21.15.148:42860) 2023-10-10T20:24:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.092+0000 7f46cd709700 1 -- 172.21.15.148:0/2859258593 learned_addr learned my addr 172.21.15.148:0/2859258593 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cdf0a700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46c80cd6e0 0x7f46c8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46ce70b700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 0x7f46c8159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 -- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 msgr2=0x7f46c8159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 0x7f46c8159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 -- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46c80cd6e0 msgr2=0x7f46c8155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46c80cd6e0 0x7f46c8155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 -- 172.21.15.148:0/2859258593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4009a20 con 0x7f46c80cdab0 2023-10-10T20:24:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46ce70b700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 0x7f46c8159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cd709700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d890 tx=0x7f46b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800b600 con 0x7f46c80cdab0 2023-10-10T20:24:13.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800b760 con 0x7f46c80cdab0 2023-10-10T20:24:13.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c815c390 con 0x7f46c80cdab0 2023-10-10T20:24:13.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800bae0 con 0x7f46c80cdab0 2023-10-10T20:24:13.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.093+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80b50c0 con 0x7f46c80cdab0 2023-10-10T20:24:13.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.094+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c80b55e0 con 0x7f46c80cdab0 2023-10-10T20:24:13.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.095+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46b801e460 con 0x7f46c80cdab0 2023-10-10T20:24:13.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.095+0000 7f46bdffb700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.095+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46b8084c10 con 0x7f46c80cdab0 2023-10-10T20:24:13.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.095+0000 7f46cdf0a700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.095+0000 7f46cdf0a700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 0x7f46ac0693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b6c70 tx=0x7f46c4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.099+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46c80b55e0 con 0x7f46c80cdab0 2023-10-10T20:24:13.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.278+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46c80b55a0 con 0x7f46ac066f30 2023-10-10T20:24:13.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.285+0000 7f46bdffb700 1 -- 172.21.15.148:0/2859258593 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276212 (secure 0 0 0) 0x7f46c80b55a0 con 0x7f46ac066f30 2023-10-10T20:24:13.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:13.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 msgr2=0x7f46ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 0x7f46ac0693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f46c80b6c70 tx=0x7f46c4019000 comp rx=0 tx=0).stop 2023-10-10T20:24:13.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 msgr2=0x7f46c81586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d890 tx=0x7f46b8008040 comp rx=0 tx=0).stop 2023-10-10T20:24:13.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 shutdown_connections 2023-10-10T20:24:13.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f46ac066f30 0x7f46ac0693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46c815dda0 0x7f46c8159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46c80cd6e0 0x7f46c8155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 --2- 172.21.15.148:0/2859258593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c80cdab0 0x7f46c81586e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.288+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 >> 172.21.15.148:0/2859258593 conn(0x7f46c8067660 msgr2=0x7f46c80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:13.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.289+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 shutdown_connections 2023-10-10T20:24:13.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.289+0000 7f46cef0c700 1 -- 172.21.15.148:0/2859258593 wait complete. 2023-10-10T20:24:13.295 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:24:13.302 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2023-10-10T20:24:11.386934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5076,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557423,"statfs":{"total":1600000000,"available":1594801484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.165437"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226623+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:38:54.651029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226854+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:50:41.115858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224553+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:38.475507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224483+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:50.759210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225388+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:33.642243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225511+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:43.812205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224106+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:36.468461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226603+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:43:39.451232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225941+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:18.398874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226342+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:40.404588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226212+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:50.364205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224837+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:49.945114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226144+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:57:43.470009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226075+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:43.660583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224780+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:01.033588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224165+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:59:49.257153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225003+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:43.812205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224155+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:51:36.235162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224002+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:59:05.521336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225679+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:35:37.285175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.223289+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:47.736869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.223265+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:04:39.988548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225760+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:05.446901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226251+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:03:56.054413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226819+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:32:23.649088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224606+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:50:41.115858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224680+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:33.642243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224141+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:01.006466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224949+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:35.756985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.225127+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:01.033588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.224826+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:58:53.123207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-10T20:24:08.221142+0000","last_change":"2023-10-10T20:24:08.221142+0000","last_active":"2023-10-10T20:24:08.221142+0000","last_peered":"2023-10-10T20:24:08.221142+0000","last_clean":"2023-10-10T20:24:08.221142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:08.221142+0000","last_undegraded":"2023-10-10T20:24:08.221142+0000","last_fullsized":"2023-10-10T20:24:08.221142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-10T20:24:07.217834+0000","last_change":"2023-10-10T20:24:07.226072+0000","last_active":"2023-10-10T20:24:07.217834+0000","last_peered":"2023-10-10T20:24:07.217834+0000","last_clean":"2023-10-10T20:24:07.217834+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-10T20:24:07.217834+0000","last_undegraded":"2023-10-10T20:24:07.217834+0000","last_fullsized":"2023-10-10T20:24:07.217834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:50.759210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":231,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-10-10T20:24:05.650141+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:24:05.650141+0000","last_peered":"2023-10-10T20:24:05.650141+0000","last_clean":"2023-10-10T20:24:05.650141+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:24:05.650141+0000","last_undegraded":"2023-10-10T20:24:05.650141+0000","last_fullsized":"2023-10-10T20:24:05.650141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1065,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908608,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.47099999999999997},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70499999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.70999999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70199999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.62}]},{"osd":5,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67300000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":489,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194823,"statfs":{"total":200000000,"available":199498976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76700000000000002,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67600000000000005,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.60799999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.626,"5min":1.865,"15min":1.865},"last":0.62},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.749,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69399999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.68100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":15.862,"15min":15.862},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44500000000000001,"15min":0.43099999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42499999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68200000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":494,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194817,"statfs":{"total":200000000,"available":199493171,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.50700000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.37,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54800000000000004,"5min":1.647,"15min":1.647},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56899999999999995,"5min":1.524,"15min":1.524},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.58799999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42499999999999999,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61199999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.38},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.317,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63,"5min":1.673,"15min":1.673},"last":0.33600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.633,"15min":1.633},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":1.506,"15min":1.506},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":1.696,"15min":1.696},"last":0.441},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.627,"15min":0.68300000000000005},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":1.585,"15min":1.585},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80500000000000005,"5min":1.478,"15min":1.478},"last":0.505},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.722,"15min":1.722},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1064,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194247,"statfs":{"total":200000000,"available":198909619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39500000000000002,"15min":0.46300000000000002},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47899999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.432,"15min":0.48199999999999998},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53300000000000003,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42399999999999999,"15min":0.48999999999999999},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.38500000000000001,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.57399999999999995,"5min":1.256,"15min":1.256},"last":0.377}]},{"osd":3,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36599999999999999,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47899999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41299999999999998,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.57899999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.44}]},{"osd":4,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2030000000000001,"5min":1.204,"15min":1.204},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.67700000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.78100000000000003,"15min":0.98399999999999999},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.82799999999999996,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":1.347,"15min":1.347},"last":0.622}]},{"osd":6,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.432,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.745,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":488,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194824,"statfs":{"total":200000000,"available":199499987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.371},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.47299999999999998,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.497,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.51900000000000002,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48699999999999999,"5min":11.815,"15min":11.815},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.58399999999999996,"15min":0.46899999999999997},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58399999999999996,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.316}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74299999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72399999999999998,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.54300000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":6.399,"15min":6.399},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.47899999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.66900000000000004,"15min":0.60299999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.60899999999999999},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66500000000000004,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76900000000000002,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":494,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194817,"statfs":{"total":200000000,"available":199493171,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40200000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48699999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.505,"5min":0.751,"15min":0.751},"last":0.317}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53100000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53700000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":489,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194823,"statfs":{"total":200000000,"available":199498976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.35899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.82899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.48699999999999999},"min":{"1min":0.436,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.435,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.72999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51700000000000002,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84599999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.627}]},{"osd":7,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76600000000000001,"5min":0.875,"15min":0.875},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.54500000000000004},"min":{"1min":0.36499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":489,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194823,"statfs":{"total":200000000,"available":199498976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.505,"15min":0.497},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68799999999999994,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68799999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.875,"15min":0.875},"last":0.376}]},{"osd":6,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60199999999999998}]}]}],"pool_statfs":[]}} 2023-10-10T20:24:13.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:24:13.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.456+0000 7f40a18a2700 1 Processor -- start 2023-10-10T20:24:13.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.456+0000 7f40a18a2700 1 -- start start 2023-10-10T20:24:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f40a18a2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f409c157870 0x7f409c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f40a18a2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f409c157c40 0x7f409c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f40a18a2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c15cbb0 con 0x7f409c15a620 2023-10-10T20:24:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f40a18a2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15bfa0 con 0x7f409c157c40 2023-10-10T20:24:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f40a18a2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15c280 con 0x7f409c157870 2023-10-10T20:24:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f409c157c40 0x7f409c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f409c157c40 0x7f409c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42872/0 (socket says 172.21.15.148:42872) 2023-10-10T20:24:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 -- 172.21.15.148:0/247292782 learned_addr learned my addr 172.21.15.148:0/247292782 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409affd700 1 --2- 172.21.15.148:0/247292782 >> v2:172.21.15.161:3300/0 conn(0x7f409c157870 0x7f409c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 -- 172.21.15.148:0/247292782 >> v2:172.21.15.161:3300/0 conn(0x7f409c157870 msgr2=0x7f409c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 --2- 172.21.15.148:0/247292782 >> v2:172.21.15.161:3300/0 conn(0x7f409c157870 0x7f409c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 -- 172.21.15.148:0/247292782 >> v1:172.21.15.161:6789/0 conn(0x7f409c15a620 legacy=0x7f409c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 -- 172.21.15.148:0/247292782 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c15c560 con 0x7f409c157c40 2023-10-10T20:24:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f409b7fe700 1 --2- 172.21.15.148:0/247292782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157c40 0x7f409c158110 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f409c069d80 tx=0x7f408c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b10ee14bcba50f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f4099ffb700 1 -- 172.21.15.148:0/247292782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c009e70 con 0x7f409c157c40 2023-10-10T20:24:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f4099ffb700 1 -- 172.21.15.148:0/247292782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00b860 con 0x7f409c157c40 2023-10-10T20:24:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.457+0000 7f4099ffb700 1 -- 172.21.15.148:0/247292782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00bc00 con 0x7f409c157c40 2023-10-10T20:24:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 -- 172.21.15.148:0/247292782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157c40 msgr2=0x7f409c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 --2- 172.21.15.148:0/247292782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157c40 0x7f409c158110 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f409c069d80 tx=0x7f408c009250 comp rx=0 tx=0).stop 2023-10-10T20:24:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 -- 172.21.15.148:0/247292782 shutdown_connections 2023-10-10T20:24:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 --2- 172.21.15.148:0/247292782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157c40 0x7f409c158110 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 --2- 172.21.15.148:0/247292782 >> v2:172.21.15.161:3300/0 conn(0x7f409c157870 0x7f409c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 -- 172.21.15.148:0/247292782 >> 172.21.15.148:0/247292782 conn(0x7f409c068ce0 msgr2=0x7f409c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 -- 172.21.15.148:0/247292782 shutdown_connections 2023-10-10T20:24:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.458+0000 7f40a18a2700 1 -- 172.21.15.148:0/247292782 wait complete. 2023-10-10T20:24:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.459+0000 7f40a18a2700 1 Processor -- start 2023-10-10T20:24:13.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.459+0000 7f40a18a2700 1 -- start start 2023-10-10T20:24:13.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 0x7f409c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b1400 con 0x7f409c157870 2023-10-10T20:24:13.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b07f0 con 0x7f409c15a620 2023-10-10T20:24:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f40a18a2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b0ad0 con 0x7f409c157c40 2023-10-10T20:24:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42878/0 (socket says 172.21.15.148:42878) 2023-10-10T20:24:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 -- 172.21.15.148:0/1521289174 learned_addr learned my addr 172.21.15.148:0/1521289174 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:13.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409b7fe700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409a7fc700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 0x7f409c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 -- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 msgr2=0x7f409c14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 0x7f409c14f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 -- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 msgr2=0x7f409c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 -- 172.21.15.148:0/1521289174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0094f0 con 0x7f409c157870 2023-10-10T20:24:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409a7fc700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 0x7f409c14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409b7fe700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f409affd700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f409400ec80 tx=0x7f40940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400c9d0 con 0x7f409c157870 2023-10-10T20:24:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.460+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400cb30 con 0x7f409c157870 2023-10-10T20:24:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.461+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094013470 con 0x7f409c157870 2023-10-10T20:24:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.461+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c0b0db0 con 0x7f409c157870 2023-10-10T20:24:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.461+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c14ff20 con 0x7f409c157870 2023-10-10T20:24:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.461+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c04c3c0 con 0x7f409c157870 2023-10-10T20:24:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.462+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4094013820 con 0x7f409c157870 2023-10-10T20:24:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.462+0000 7f4082ffd700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 0x7f40880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.462+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40940848a0 con 0x7f409c157870 2023-10-10T20:24:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.462+0000 7f409a7fc700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 0x7f40880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.463+0000 7f409a7fc700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 0x7f40880693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f409c15b460 tx=0x7f4090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.467+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4094051b10 con 0x7f409c157870 2023-10-10T20:24:13.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.648+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f409c0b1540 con 0x7f4088066f30 2023-10-10T20:24:13.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.655+0000 7f4082ffd700 1 -- 172.21.15.148:0/1521289174 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285074 (secure 0 0 0) 0x7f409000e9c0 con 0x7f4088066f30 2023-10-10T20:24:13.661 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:13.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 msgr2=0x7f40880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 0x7f40880693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f409c15b460 tx=0x7f4090009040 comp rx=0 tx=0).stop 2023-10-10T20:24:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 msgr2=0x7f409c13f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f409400ec80 tx=0x7f40940091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 shutdown_connections 2023-10-10T20:24:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4088066f30 0x7f40880693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c13f920 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 --2- 172.21.15.148:0/1521289174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c157c40 0x7f409c14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 >> 172.21.15.148:0/1521289174 conn(0x7f409c068ce0 msgr2=0x7f409c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 shutdown_connections 2023-10-10T20:24:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.659+0000 7f40a18a2700 1 -- 172.21.15.148:0/1521289174 wait complete. 2023-10-10T20:24:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:24:13.672 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2023-10-10T20:24:13.387213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557233,"statfs":{"total":1600000000,"available":1594607530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000968"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.256694+0000","last_change":"2023-10-10T20:24:09.256626+0000","last_active":"2023-10-10T20:24:09.256694+0000","last_peered":"2023-10-10T20:24:09.256694+0000","last_clean":"2023-10-10T20:24:09.256694+0000","last_became_active":"2023-10-10T20:24:09.256349+0000","last_became_peered":"2023-10-10T20:24:09.256349+0000","last_unstale":"2023-10-10T20:24:09.256694+0000","last_undegraded":"2023-10-10T20:24:09.256694+0000","last_fullsized":"2023-10-10T20:24:09.256694+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:07:17.310358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254760+0000","last_change":"2023-10-10T20:24:09.254708+0000","last_active":"2023-10-10T20:24:09.254760+0000","last_peered":"2023-10-10T20:24:09.254760+0000","last_clean":"2023-10-10T20:24:09.254760+0000","last_became_active":"2023-10-10T20:24:09.252170+0000","last_became_peered":"2023-10-10T20:24:09.252170+0000","last_unstale":"2023-10-10T20:24:09.254760+0000","last_undegraded":"2023-10-10T20:24:09.254760+0000","last_fullsized":"2023-10-10T20:24:09.254760+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:14:34.645551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254331+0000","last_change":"2023-10-10T20:24:09.254282+0000","last_active":"2023-10-10T20:24:09.254331+0000","last_peered":"2023-10-10T20:24:09.254331+0000","last_clean":"2023-10-10T20:24:09.254331+0000","last_became_active":"2023-10-10T20:24:09.254041+0000","last_became_peered":"2023-10-10T20:24:09.254041+0000","last_unstale":"2023-10-10T20:24:09.254331+0000","last_undegraded":"2023-10-10T20:24:09.254331+0000","last_fullsized":"2023-10-10T20:24:09.254331+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:34.163988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249352+0000","last_change":"2023-10-10T20:24:09.248851+0000","last_active":"2023-10-10T20:24:09.249352+0000","last_peered":"2023-10-10T20:24:09.249352+0000","last_clean":"2023-10-10T20:24:09.249352+0000","last_became_active":"2023-10-10T20:24:09.248512+0000","last_became_peered":"2023-10-10T20:24:09.248512+0000","last_unstale":"2023-10-10T20:24:09.249352+0000","last_undegraded":"2023-10-10T20:24:09.249352+0000","last_fullsized":"2023-10-10T20:24:09.249352+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:08:36.726790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.248967+0000","last_change":"2023-10-10T20:24:09.248892+0000","last_active":"2023-10-10T20:24:09.248967+0000","last_peered":"2023-10-10T20:24:09.248967+0000","last_clean":"2023-10-10T20:24:09.248967+0000","last_became_active":"2023-10-10T20:24:09.248658+0000","last_became_peered":"2023-10-10T20:24:09.248658+0000","last_unstale":"2023-10-10T20:24:09.248967+0000","last_undegraded":"2023-10-10T20:24:09.248967+0000","last_fullsized":"2023-10-10T20:24:09.248967+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:06.450209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.246376+0000","last_change":"2023-10-10T20:24:09.246217+0000","last_active":"2023-10-10T20:24:09.246376+0000","last_peered":"2023-10-10T20:24:09.246376+0000","last_clean":"2023-10-10T20:24:09.246376+0000","last_became_active":"2023-10-10T20:24:09.245992+0000","last_became_peered":"2023-10-10T20:24:09.245992+0000","last_unstale":"2023-10-10T20:24:09.246376+0000","last_undegraded":"2023-10-10T20:24:09.246376+0000","last_fullsized":"2023-10-10T20:24:09.246376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:51.762518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.253502+0000","last_change":"2023-10-10T20:24:09.253451+0000","last_active":"2023-10-10T20:24:09.253502+0000","last_peered":"2023-10-10T20:24:09.253502+0000","last_clean":"2023-10-10T20:24:09.253502+0000","last_became_active":"2023-10-10T20:24:09.253213+0000","last_became_peered":"2023-10-10T20:24:09.253213+0000","last_unstale":"2023-10-10T20:24:09.253502+0000","last_undegraded":"2023-10-10T20:24:09.253502+0000","last_fullsized":"2023-10-10T20:24:09.253502+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:23.917184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250690+0000","last_change":"2023-10-10T20:24:09.250622+0000","last_active":"2023-10-10T20:24:09.250690+0000","last_peered":"2023-10-10T20:24:09.250690+0000","last_clean":"2023-10-10T20:24:09.250690+0000","last_became_active":"2023-10-10T20:24:09.250339+0000","last_became_peered":"2023-10-10T20:24:09.250339+0000","last_unstale":"2023-10-10T20:24:09.250690+0000","last_undegraded":"2023-10-10T20:24:09.250690+0000","last_fullsized":"2023-10-10T20:24:09.250690+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:09:30.020170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.243123+0000","last_change":"2023-10-10T20:24:09.243050+0000","last_active":"2023-10-10T20:24:09.243123+0000","last_peered":"2023-10-10T20:24:09.243123+0000","last_clean":"2023-10-10T20:24:09.243123+0000","last_became_active":"2023-10-10T20:24:09.242874+0000","last_became_peered":"2023-10-10T20:24:09.242874+0000","last_unstale":"2023-10-10T20:24:09.243123+0000","last_undegraded":"2023-10-10T20:24:09.243123+0000","last_fullsized":"2023-10-10T20:24:09.243123+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:07.984464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249602+0000","last_change":"2023-10-10T20:24:09.249480+0000","last_active":"2023-10-10T20:24:09.249602+0000","last_peered":"2023-10-10T20:24:09.249602+0000","last_clean":"2023-10-10T20:24:09.249602+0000","last_became_active":"2023-10-10T20:24:09.249249+0000","last_became_peered":"2023-10-10T20:24:09.249249+0000","last_unstale":"2023-10-10T20:24:09.249602+0000","last_undegraded":"2023-10-10T20:24:09.249602+0000","last_fullsized":"2023-10-10T20:24:09.249602+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:40.348233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.255555+0000","last_change":"2023-10-10T20:24:09.255490+0000","last_active":"2023-10-10T20:24:09.255555+0000","last_peered":"2023-10-10T20:24:09.255555+0000","last_clean":"2023-10-10T20:24:09.255555+0000","last_became_active":"2023-10-10T20:24:09.255299+0000","last_became_peered":"2023-10-10T20:24:09.255299+0000","last_unstale":"2023-10-10T20:24:09.255555+0000","last_undegraded":"2023-10-10T20:24:09.255555+0000","last_fullsized":"2023-10-10T20:24:09.255555+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:19.402182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.252812+0000","last_change":"2023-10-10T20:24:09.252592+0000","last_active":"2023-10-10T20:24:09.252812+0000","last_peered":"2023-10-10T20:24:09.252812+0000","last_clean":"2023-10-10T20:24:09.252812+0000","last_became_active":"2023-10-10T20:24:09.252298+0000","last_became_peered":"2023-10-10T20:24:09.252298+0000","last_unstale":"2023-10-10T20:24:09.252812+0000","last_undegraded":"2023-10-10T20:24:09.252812+0000","last_fullsized":"2023-10-10T20:24:09.252812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:06.827615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.252627+0000","last_change":"2023-10-10T20:24:09.252522+0000","last_active":"2023-10-10T20:24:09.252627+0000","last_peered":"2023-10-10T20:24:09.252627+0000","last_clean":"2023-10-10T20:24:09.252627+0000","last_became_active":"2023-10-10T20:24:09.252281+0000","last_became_peered":"2023-10-10T20:24:09.252281+0000","last_unstale":"2023-10-10T20:24:09.252627+0000","last_undegraded":"2023-10-10T20:24:09.252627+0000","last_fullsized":"2023-10-10T20:24:09.252627+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:07:17.310358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254298+0000","last_change":"2023-10-10T20:24:09.254253+0000","last_active":"2023-10-10T20:24:09.254298+0000","last_peered":"2023-10-10T20:24:09.254298+0000","last_clean":"2023-10-10T20:24:09.254298+0000","last_became_active":"2023-10-10T20:24:09.253801+0000","last_became_peered":"2023-10-10T20:24:09.253801+0000","last_unstale":"2023-10-10T20:24:09.254298+0000","last_undegraded":"2023-10-10T20:24:09.254298+0000","last_fullsized":"2023-10-10T20:24:09.254298+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:42:38.578279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.252154+0000","last_change":"2023-10-10T20:24:09.252083+0000","last_active":"2023-10-10T20:24:09.252154+0000","last_peered":"2023-10-10T20:24:09.252154+0000","last_clean":"2023-10-10T20:24:09.252154+0000","last_became_active":"2023-10-10T20:24:09.251834+0000","last_became_peered":"2023-10-10T20:24:09.251834+0000","last_unstale":"2023-10-10T20:24:09.252154+0000","last_undegraded":"2023-10-10T20:24:09.252154+0000","last_fullsized":"2023-10-10T20:24:09.252154+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:27:42.036201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.255398+0000","last_change":"2023-10-10T20:24:09.255331+0000","last_active":"2023-10-10T20:24:09.255398+0000","last_peered":"2023-10-10T20:24:09.255398+0000","last_clean":"2023-10-10T20:24:09.255398+0000","last_became_active":"2023-10-10T20:24:09.250714+0000","last_became_peered":"2023-10-10T20:24:09.250714+0000","last_unstale":"2023-10-10T20:24:09.255398+0000","last_undegraded":"2023-10-10T20:24:09.255398+0000","last_fullsized":"2023-10-10T20:24:09.255398+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:06.827615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247959+0000","last_change":"2023-10-10T20:24:09.246683+0000","last_active":"2023-10-10T20:24:09.247959+0000","last_peered":"2023-10-10T20:24:09.247959+0000","last_clean":"2023-10-10T20:24:09.247959+0000","last_became_active":"2023-10-10T20:24:09.246317+0000","last_became_peered":"2023-10-10T20:24:09.246317+0000","last_unstale":"2023-10-10T20:24:09.247959+0000","last_undegraded":"2023-10-10T20:24:09.247959+0000","last_fullsized":"2023-10-10T20:24:09.247959+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:58.758537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254026+0000","last_change":"2023-10-10T20:24:09.253969+0000","last_active":"2023-10-10T20:24:09.254026+0000","last_peered":"2023-10-10T20:24:09.254026+0000","last_clean":"2023-10-10T20:24:09.254026+0000","last_became_active":"2023-10-10T20:24:09.253619+0000","last_became_peered":"2023-10-10T20:24:09.253619+0000","last_unstale":"2023-10-10T20:24:09.254026+0000","last_undegraded":"2023-10-10T20:24:09.254026+0000","last_fullsized":"2023-10-10T20:24:09.254026+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:09:02.124671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250722+0000","last_change":"2023-10-10T20:24:09.250082+0000","last_active":"2023-10-10T20:24:09.250722+0000","last_peered":"2023-10-10T20:24:09.250722+0000","last_clean":"2023-10-10T20:24:09.250722+0000","last_became_active":"2023-10-10T20:24:09.249855+0000","last_became_peered":"2023-10-10T20:24:09.249855+0000","last_unstale":"2023-10-10T20:24:09.250722+0000","last_undegraded":"2023-10-10T20:24:09.250722+0000","last_fullsized":"2023-10-10T20:24:09.250722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:40.348233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.253341+0000","last_change":"2023-10-10T20:24:09.253281+0000","last_active":"2023-10-10T20:24:09.253341+0000","last_peered":"2023-10-10T20:24:09.253341+0000","last_clean":"2023-10-10T20:24:09.253341+0000","last_became_active":"2023-10-10T20:24:09.253025+0000","last_became_peered":"2023-10-10T20:24:09.253025+0000","last_unstale":"2023-10-10T20:24:09.253341+0000","last_undegraded":"2023-10-10T20:24:09.253341+0000","last_fullsized":"2023-10-10T20:24:09.253341+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:19:57.973885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.253254+0000","last_change":"2023-10-10T20:24:09.253179+0000","last_active":"2023-10-10T20:24:09.253254+0000","last_peered":"2023-10-10T20:24:09.253254+0000","last_clean":"2023-10-10T20:24:09.253254+0000","last_became_active":"2023-10-10T20:24:09.252952+0000","last_became_peered":"2023-10-10T20:24:09.252952+0000","last_unstale":"2023-10-10T20:24:09.253254+0000","last_undegraded":"2023-10-10T20:24:09.253254+0000","last_fullsized":"2023-10-10T20:24:09.253254+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:38.934988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.248065+0000","last_change":"2023-10-10T20:24:09.247819+0000","last_active":"2023-10-10T20:24:09.248065+0000","last_peered":"2023-10-10T20:24:09.248065+0000","last_clean":"2023-10-10T20:24:09.248065+0000","last_became_active":"2023-10-10T20:24:09.247598+0000","last_became_peered":"2023-10-10T20:24:09.247598+0000","last_unstale":"2023-10-10T20:24:09.248065+0000","last_undegraded":"2023-10-10T20:24:09.248065+0000","last_fullsized":"2023-10-10T20:24:09.248065+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:23:36.760293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254377+0000","last_change":"2023-10-10T20:24:09.254311+0000","last_active":"2023-10-10T20:24:09.254377+0000","last_peered":"2023-10-10T20:24:09.254377+0000","last_clean":"2023-10-10T20:24:09.254377+0000","last_became_active":"2023-10-10T20:24:09.253423+0000","last_became_peered":"2023-10-10T20:24:09.253423+0000","last_unstale":"2023-10-10T20:24:09.254377+0000","last_undegraded":"2023-10-10T20:24:09.254377+0000","last_fullsized":"2023-10-10T20:24:09.254377+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:44.663891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249334+0000","last_change":"2023-10-10T20:24:09.248721+0000","last_active":"2023-10-10T20:24:09.249334+0000","last_peered":"2023-10-10T20:24:09.249334+0000","last_clean":"2023-10-10T20:24:09.249334+0000","last_became_active":"2023-10-10T20:24:09.248222+0000","last_became_peered":"2023-10-10T20:24:09.248222+0000","last_unstale":"2023-10-10T20:24:09.249334+0000","last_undegraded":"2023-10-10T20:24:09.249334+0000","last_fullsized":"2023-10-10T20:24:09.249334+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:05:21.009854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.255569+0000","last_change":"2023-10-10T20:24:09.255487+0000","last_active":"2023-10-10T20:24:09.255569+0000","last_peered":"2023-10-10T20:24:09.255569+0000","last_clean":"2023-10-10T20:24:09.255569+0000","last_became_active":"2023-10-10T20:24:09.255245+0000","last_became_peered":"2023-10-10T20:24:09.255245+0000","last_unstale":"2023-10-10T20:24:09.255569+0000","last_undegraded":"2023-10-10T20:24:09.255569+0000","last_fullsized":"2023-10-10T20:24:09.255569+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:02.036896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.256223+0000","last_change":"2023-10-10T20:24:09.256074+0000","last_active":"2023-10-10T20:24:09.256223+0000","last_peered":"2023-10-10T20:24:09.256223+0000","last_clean":"2023-10-10T20:24:09.256223+0000","last_became_active":"2023-10-10T20:24:09.255831+0000","last_became_peered":"2023-10-10T20:24:09.255831+0000","last_unstale":"2023-10-10T20:24:09.256223+0000","last_undegraded":"2023-10-10T20:24:09.256223+0000","last_fullsized":"2023-10-10T20:24:09.256223+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:54.602985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.256255+0000","last_change":"2023-10-10T20:24:09.256118+0000","last_active":"2023-10-10T20:24:09.256255+0000","last_peered":"2023-10-10T20:24:09.256255+0000","last_clean":"2023-10-10T20:24:09.256255+0000","last_became_active":"2023-10-10T20:24:09.255833+0000","last_became_peered":"2023-10-10T20:24:09.255833+0000","last_unstale":"2023-10-10T20:24:09.256255+0000","last_undegraded":"2023-10-10T20:24:09.256255+0000","last_fullsized":"2023-10-10T20:24:09.256255+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:35:38.288483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.257287+0000","last_change":"2023-10-10T20:24:09.257233+0000","last_active":"2023-10-10T20:24:09.257287+0000","last_peered":"2023-10-10T20:24:09.257287+0000","last_clean":"2023-10-10T20:24:09.257287+0000","last_became_active":"2023-10-10T20:24:09.257005+0000","last_became_peered":"2023-10-10T20:24:09.257005+0000","last_unstale":"2023-10-10T20:24:09.257287+0000","last_undegraded":"2023-10-10T20:24:09.257287+0000","last_fullsized":"2023-10-10T20:24:09.257287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:27.002335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250856+0000","last_change":"2023-10-10T20:24:09.250711+0000","last_active":"2023-10-10T20:24:09.250856+0000","last_peered":"2023-10-10T20:24:09.250856+0000","last_clean":"2023-10-10T20:24:09.250856+0000","last_became_active":"2023-10-10T20:24:09.249923+0000","last_became_peered":"2023-10-10T20:24:09.249923+0000","last_unstale":"2023-10-10T20:24:09.250856+0000","last_undegraded":"2023-10-10T20:24:09.250856+0000","last_fullsized":"2023-10-10T20:24:09.250856+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:44.815512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249334+0000","last_change":"2023-10-10T20:24:09.248612+0000","last_active":"2023-10-10T20:24:09.249334+0000","last_peered":"2023-10-10T20:24:09.249334+0000","last_clean":"2023-10-10T20:24:09.249334+0000","last_became_active":"2023-10-10T20:24:09.248396+0000","last_became_peered":"2023-10-10T20:24:09.248396+0000","last_unstale":"2023-10-10T20:24:09.249334+0000","last_undegraded":"2023-10-10T20:24:09.249334+0000","last_fullsized":"2023-10-10T20:24:09.249334+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:58:54.126515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244561+0000","last_change":"2023-10-10T20:24:09.244492+0000","last_active":"2023-10-10T20:24:09.244561+0000","last_peered":"2023-10-10T20:24:09.244561+0000","last_clean":"2023-10-10T20:24:09.244561+0000","last_became_active":"2023-10-10T20:24:09.244258+0000","last_became_peered":"2023-10-10T20:24:09.244258+0000","last_unstale":"2023-10-10T20:24:09.244561+0000","last_undegraded":"2023-10-10T20:24:09.244561+0000","last_fullsized":"2023-10-10T20:24:09.244561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:09:30.020170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.257253+0000","last_change":"2023-10-10T20:24:09.257135+0000","last_active":"2023-10-10T20:24:09.257253+0000","last_peered":"2023-10-10T20:24:09.257253+0000","last_clean":"2023-10-10T20:24:09.257253+0000","last_became_active":"2023-10-10T20:24:09.256744+0000","last_became_peered":"2023-10-10T20:24:09.256744+0000","last_unstale":"2023-10-10T20:24:09.257253+0000","last_undegraded":"2023-10-10T20:24:09.257253+0000","last_fullsized":"2023-10-10T20:24:09.257253+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:09:06.450209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249529+0000","last_change":"2023-10-10T20:24:09.249440+0000","last_active":"2023-10-10T20:24:09.249529+0000","last_peered":"2023-10-10T20:24:09.249529+0000","last_clean":"2023-10-10T20:24:09.249529+0000","last_became_active":"2023-10-10T20:24:09.249217+0000","last_became_peered":"2023-10-10T20:24:09.249217+0000","last_unstale":"2023-10-10T20:24:09.249529+0000","last_undegraded":"2023-10-10T20:24:09.249529+0000","last_fullsized":"2023-10-10T20:24:09.249529+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:40.348233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"97'4","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.265430+0000","last_change":"2023-10-10T20:24:08.237223+0000","last_active":"2023-10-10T20:24:09.265430+0000","last_peered":"2023-10-10T20:24:09.265430+0000","last_clean":"2023-10-10T20:24:09.265430+0000","last_became_active":"2023-10-10T20:24:08.235703+0000","last_became_peered":"2023-10-10T20:24:08.235703+0000","last_unstale":"2023-10-10T20:24:09.265430+0000","last_undegraded":"2023-10-10T20:24:09.265430+0000","last_fullsized":"2023-10-10T20:24:09.265430+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:36.468461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254747+0000","last_change":"2023-10-10T20:24:09.254459+0000","last_active":"2023-10-10T20:24:09.254747+0000","last_peered":"2023-10-10T20:24:09.254747+0000","last_clean":"2023-10-10T20:24:09.254747+0000","last_became_active":"2023-10-10T20:24:09.253927+0000","last_became_peered":"2023-10-10T20:24:09.253927+0000","last_unstale":"2023-10-10T20:24:09.254747+0000","last_undegraded":"2023-10-10T20:24:09.254747+0000","last_fullsized":"2023-10-10T20:24:09.254747+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:54.602985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.264774+0000","last_change":"2023-10-10T20:24:08.238258+0000","last_active":"2023-10-10T20:24:09.264774+0000","last_peered":"2023-10-10T20:24:09.264774+0000","last_clean":"2023-10-10T20:24:09.264774+0000","last_became_active":"2023-10-10T20:24:08.238007+0000","last_became_peered":"2023-10-10T20:24:08.238007+0000","last_unstale":"2023-10-10T20:24:09.264774+0000","last_undegraded":"2023-10-10T20:24:09.264774+0000","last_fullsized":"2023-10-10T20:24:09.264774+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:49:40.404588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.251658+0000","last_change":"2023-10-10T20:24:09.251026+0000","last_active":"2023-10-10T20:24:09.251658+0000","last_peered":"2023-10-10T20:24:09.251658+0000","last_clean":"2023-10-10T20:24:09.251658+0000","last_became_active":"2023-10-10T20:24:09.250773+0000","last_became_peered":"2023-10-10T20:24:09.250773+0000","last_unstale":"2023-10-10T20:24:09.251658+0000","last_undegraded":"2023-10-10T20:24:09.251658+0000","last_fullsized":"2023-10-10T20:24:09.251658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:43.421965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.264493+0000","last_change":"2023-10-10T20:24:08.231832+0000","last_active":"2023-10-10T20:24:09.264493+0000","last_peered":"2023-10-10T20:24:09.264493+0000","last_clean":"2023-10-10T20:24:09.264493+0000","last_became_active":"2023-10-10T20:24:08.231139+0000","last_became_peered":"2023-10-10T20:24:08.231139+0000","last_unstale":"2023-10-10T20:24:09.264493+0000","last_undegraded":"2023-10-10T20:24:09.264493+0000","last_fullsized":"2023-10-10T20:24:09.264493+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:18:19.614291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254754+0000","last_change":"2023-10-10T20:24:09.254697+0000","last_active":"2023-10-10T20:24:09.254754+0000","last_peered":"2023-10-10T20:24:09.254754+0000","last_clean":"2023-10-10T20:24:09.254754+0000","last_became_active":"2023-10-10T20:24:09.254248+0000","last_became_peered":"2023-10-10T20:24:09.254248+0000","last_unstale":"2023-10-10T20:24:09.254754+0000","last_undegraded":"2023-10-10T20:24:09.254754+0000","last_fullsized":"2023-10-10T20:24:09.254754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:11.088321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.264248+0000","last_change":"2023-10-10T20:24:08.233237+0000","last_active":"2023-10-10T20:24:09.264248+0000","last_peered":"2023-10-10T20:24:09.264248+0000","last_clean":"2023-10-10T20:24:09.264248+0000","last_became_active":"2023-10-10T20:24:08.233022+0000","last_became_peered":"2023-10-10T20:24:08.233022+0000","last_unstale":"2023-10-10T20:24:09.264248+0000","last_undegraded":"2023-10-10T20:24:09.264248+0000","last_fullsized":"2023-10-10T20:24:09.264248+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:09:29.016862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254498+0000","last_change":"2023-10-10T20:24:09.252410+0000","last_active":"2023-10-10T20:24:09.254498+0000","last_peered":"2023-10-10T20:24:09.254498+0000","last_clean":"2023-10-10T20:24:09.254498+0000","last_became_active":"2023-10-10T20:24:09.251199+0000","last_became_peered":"2023-10-10T20:24:09.251199+0000","last_unstale":"2023-10-10T20:24:09.254498+0000","last_undegraded":"2023-10-10T20:24:09.254498+0000","last_fullsized":"2023-10-10T20:24:09.254498+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:43:59.022933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.264367+0000","last_change":"2023-10-10T20:24:08.240539+0000","last_active":"2023-10-10T20:24:09.264367+0000","last_peered":"2023-10-10T20:24:09.264367+0000","last_clean":"2023-10-10T20:24:09.264367+0000","last_became_active":"2023-10-10T20:24:08.240313+0000","last_became_peered":"2023-10-10T20:24:08.240313+0000","last_unstale":"2023-10-10T20:24:09.264367+0000","last_undegraded":"2023-10-10T20:24:09.264367+0000","last_fullsized":"2023-10-10T20:24:09.264367+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:18.398874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247939+0000","last_change":"2023-10-10T20:24:09.246544+0000","last_active":"2023-10-10T20:24:09.247939+0000","last_peered":"2023-10-10T20:24:09.247939+0000","last_clean":"2023-10-10T20:24:09.247939+0000","last_became_active":"2023-10-10T20:24:09.245973+0000","last_became_peered":"2023-10-10T20:24:09.245973+0000","last_unstale":"2023-10-10T20:24:09.247939+0000","last_undegraded":"2023-10-10T20:24:09.247939+0000","last_fullsized":"2023-10-10T20:24:09.247939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:05.322899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.269795+0000","last_change":"2023-10-10T20:24:08.237581+0000","last_active":"2023-10-10T20:24:09.269795+0000","last_peered":"2023-10-10T20:24:09.269795+0000","last_clean":"2023-10-10T20:24:09.269795+0000","last_became_active":"2023-10-10T20:24:08.237164+0000","last_became_peered":"2023-10-10T20:24:08.237164+0000","last_unstale":"2023-10-10T20:24:09.269795+0000","last_undegraded":"2023-10-10T20:24:09.269795+0000","last_fullsized":"2023-10-10T20:24:09.269795+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:44:01.033588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254025+0000","last_change":"2023-10-10T20:24:09.253714+0000","last_active":"2023-10-10T20:24:09.254025+0000","last_peered":"2023-10-10T20:24:09.254025+0000","last_clean":"2023-10-10T20:24:09.254025+0000","last_became_active":"2023-10-10T20:24:09.252700+0000","last_became_peered":"2023-10-10T20:24:09.252700+0000","last_unstale":"2023-10-10T20:24:09.254025+0000","last_undegraded":"2023-10-10T20:24:09.254025+0000","last_fullsized":"2023-10-10T20:24:09.254025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:06.827615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.240507+0000","last_change":"2023-10-10T20:24:08.230957+0000","last_active":"2023-10-10T20:24:09.240507+0000","last_peered":"2023-10-10T20:24:09.240507+0000","last_clean":"2023-10-10T20:24:09.240507+0000","last_became_active":"2023-10-10T20:24:08.230488+0000","last_became_peered":"2023-10-10T20:24:08.230488+0000","last_unstale":"2023-10-10T20:24:09.240507+0000","last_undegraded":"2023-10-10T20:24:09.240507+0000","last_fullsized":"2023-10-10T20:24:09.240507+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:03:13.517800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250316+0000","last_change":"2023-10-10T20:24:09.250245+0000","last_active":"2023-10-10T20:24:09.250316+0000","last_peered":"2023-10-10T20:24:09.250316+0000","last_clean":"2023-10-10T20:24:09.250316+0000","last_became_active":"2023-10-10T20:24:09.250023+0000","last_became_peered":"2023-10-10T20:24:09.250023+0000","last_unstale":"2023-10-10T20:24:09.250316+0000","last_undegraded":"2023-10-10T20:24:09.250316+0000","last_fullsized":"2023-10-10T20:24:09.250316+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:58:49.293329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247433+0000","last_change":"2023-10-10T20:24:08.239537+0000","last_active":"2023-10-10T20:24:09.247433+0000","last_peered":"2023-10-10T20:24:09.247433+0000","last_clean":"2023-10-10T20:24:09.247433+0000","last_became_active":"2023-10-10T20:24:08.239335+0000","last_became_peered":"2023-10-10T20:24:08.239335+0000","last_unstale":"2023-10-10T20:24:09.247433+0000","last_undegraded":"2023-10-10T20:24:09.247433+0000","last_fullsized":"2023-10-10T20:24:09.247433+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:41.339873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250333+0000","last_change":"2023-10-10T20:24:09.250252+0000","last_active":"2023-10-10T20:24:09.250333+0000","last_peered":"2023-10-10T20:24:09.250333+0000","last_clean":"2023-10-10T20:24:09.250333+0000","last_became_active":"2023-10-10T20:24:09.249479+0000","last_became_peered":"2023-10-10T20:24:09.249479+0000","last_unstale":"2023-10-10T20:24:09.250333+0000","last_undegraded":"2023-10-10T20:24:09.250333+0000","last_fullsized":"2023-10-10T20:24:09.250333+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:27.002335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247720+0000","last_change":"2023-10-10T20:24:08.235172+0000","last_active":"2023-10-10T20:24:09.247720+0000","last_peered":"2023-10-10T20:24:09.247720+0000","last_clean":"2023-10-10T20:24:09.247720+0000","last_became_active":"2023-10-10T20:24:08.234933+0000","last_became_peered":"2023-10-10T20:24:08.234933+0000","last_unstale":"2023-10-10T20:24:09.247720+0000","last_undegraded":"2023-10-10T20:24:09.247720+0000","last_fullsized":"2023-10-10T20:24:09.247720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:51:36.235162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250579+0000","last_change":"2023-10-10T20:24:09.250050+0000","last_active":"2023-10-10T20:24:09.250579+0000","last_peered":"2023-10-10T20:24:09.250579+0000","last_clean":"2023-10-10T20:24:09.250579+0000","last_became_active":"2023-10-10T20:24:09.249781+0000","last_became_peered":"2023-10-10T20:24:09.249781+0000","last_unstale":"2023-10-10T20:24:09.250579+0000","last_undegraded":"2023-10-10T20:24:09.250579+0000","last_fullsized":"2023-10-10T20:24:09.250579+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:59:50.260461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247761+0000","last_change":"2023-10-10T20:24:08.240365+0000","last_active":"2023-10-10T20:24:09.247761+0000","last_peered":"2023-10-10T20:24:09.247761+0000","last_clean":"2023-10-10T20:24:09.247761+0000","last_became_active":"2023-10-10T20:24:08.240142+0000","last_became_peered":"2023-10-10T20:24:08.240142+0000","last_unstale":"2023-10-10T20:24:09.247761+0000","last_undegraded":"2023-10-10T20:24:09.247761+0000","last_fullsized":"2023-10-10T20:24:09.247761+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:59:05.521336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.251503+0000","last_change":"2023-10-10T20:24:09.251257+0000","last_active":"2023-10-10T20:24:09.251503+0000","last_peered":"2023-10-10T20:24:09.251503+0000","last_clean":"2023-10-10T20:24:09.251503+0000","last_became_active":"2023-10-10T20:24:09.251019+0000","last_became_peered":"2023-10-10T20:24:09.251019+0000","last_unstale":"2023-10-10T20:24:09.251503+0000","last_undegraded":"2023-10-10T20:24:09.251503+0000","last_fullsized":"2023-10-10T20:24:09.251503+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:11.088321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"97'3","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.264269+0000","last_change":"2023-10-10T20:24:08.237407+0000","last_active":"2023-10-10T20:24:09.264269+0000","last_peered":"2023-10-10T20:24:09.264269+0000","last_clean":"2023-10-10T20:24:09.264269+0000","last_became_active":"2023-10-10T20:24:08.236457+0000","last_became_peered":"2023-10-10T20:24:08.236457+0000","last_unstale":"2023-10-10T20:24:09.264269+0000","last_undegraded":"2023-10-10T20:24:09.264269+0000","last_fullsized":"2023-10-10T20:24:09.264269+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:01.006466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.250608+0000","last_change":"2023-10-10T20:24:09.250008+0000","last_active":"2023-10-10T20:24:09.250608+0000","last_peered":"2023-10-10T20:24:09.250608+0000","last_clean":"2023-10-10T20:24:09.250608+0000","last_became_active":"2023-10-10T20:24:09.249773+0000","last_became_peered":"2023-10-10T20:24:09.249773+0000","last_unstale":"2023-10-10T20:24:09.250608+0000","last_undegraded":"2023-10-10T20:24:09.250608+0000","last_fullsized":"2023-10-10T20:24:09.250608+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:18:20.617599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"97'6","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.266308+0000","last_change":"2023-10-10T20:24:08.236362+0000","last_active":"2023-10-10T20:24:09.266308+0000","last_peered":"2023-10-10T20:24:09.266308+0000","last_clean":"2023-10-10T20:24:09.266308+0000","last_became_active":"2023-10-10T20:24:08.235887+0000","last_became_peered":"2023-10-10T20:24:08.235887+0000","last_unstale":"2023-10-10T20:24:09.266308+0000","last_undegraded":"2023-10-10T20:24:09.266308+0000","last_fullsized":"2023-10-10T20:24:09.266308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:50.364205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244831+0000","last_change":"2023-10-10T20:24:09.244594+0000","last_active":"2023-10-10T20:24:09.244831+0000","last_peered":"2023-10-10T20:24:09.244831+0000","last_clean":"2023-10-10T20:24:09.244831+0000","last_became_active":"2023-10-10T20:24:09.244231+0000","last_became_peered":"2023-10-10T20:24:09.244231+0000","last_unstale":"2023-10-10T20:24:09.244831+0000","last_undegraded":"2023-10-10T20:24:09.244831+0000","last_fullsized":"2023-10-10T20:24:09.244831+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:34.163988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.267484+0000","last_change":"2023-10-10T20:24:08.230764+0000","last_active":"2023-10-10T20:24:09.267484+0000","last_peered":"2023-10-10T20:24:09.267484+0000","last_clean":"2023-10-10T20:24:09.267484+0000","last_became_active":"2023-10-10T20:24:08.230546+0000","last_became_peered":"2023-10-10T20:24:08.230546+0000","last_unstale":"2023-10-10T20:24:09.267484+0000","last_undegraded":"2023-10-10T20:24:09.267484+0000","last_fullsized":"2023-10-10T20:24:09.267484+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:07:33.538386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254063+0000","last_change":"2023-10-10T20:24:09.253902+0000","last_active":"2023-10-10T20:24:09.254063+0000","last_peered":"2023-10-10T20:24:09.254063+0000","last_clean":"2023-10-10T20:24:09.254063+0000","last_became_active":"2023-10-10T20:24:09.253033+0000","last_became_peered":"2023-10-10T20:24:09.253033+0000","last_unstale":"2023-10-10T20:24:09.254063+0000","last_undegraded":"2023-10-10T20:24:09.254063+0000","last_fullsized":"2023-10-10T20:24:09.254063+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:44.663891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244654+0000","last_change":"2023-10-10T20:24:08.228816+0000","last_active":"2023-10-10T20:24:09.244654+0000","last_peered":"2023-10-10T20:24:09.244654+0000","last_clean":"2023-10-10T20:24:09.244654+0000","last_became_active":"2023-10-10T20:24:08.228622+0000","last_became_peered":"2023-10-10T20:24:08.228622+0000","last_unstale":"2023-10-10T20:24:09.244654+0000","last_undegraded":"2023-10-10T20:24:09.244654+0000","last_fullsized":"2023-10-10T20:24:09.244654+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:39.344925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254516+0000","last_change":"2023-10-10T20:24:09.252685+0000","last_active":"2023-10-10T20:24:09.254516+0000","last_peered":"2023-10-10T20:24:09.254516+0000","last_clean":"2023-10-10T20:24:09.254516+0000","last_became_active":"2023-10-10T20:24:09.251501+0000","last_became_peered":"2023-10-10T20:24:09.251501+0000","last_unstale":"2023-10-10T20:24:09.254516+0000","last_undegraded":"2023-10-10T20:24:09.254516+0000","last_fullsized":"2023-10-10T20:24:09.254516+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:15:23.917184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.241914+0000","last_change":"2023-10-10T20:24:08.238410+0000","last_active":"2023-10-10T20:24:09.241914+0000","last_peered":"2023-10-10T20:24:09.241914+0000","last_clean":"2023-10-10T20:24:09.241914+0000","last_became_active":"2023-10-10T20:24:08.238183+0000","last_became_peered":"2023-10-10T20:24:08.238183+0000","last_unstale":"2023-10-10T20:24:09.241914+0000","last_undegraded":"2023-10-10T20:24:09.241914+0000","last_fullsized":"2023-10-10T20:24:09.241914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:05:18.398874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254694+0000","last_change":"2023-10-10T20:24:09.254563+0000","last_active":"2023-10-10T20:24:09.254694+0000","last_peered":"2023-10-10T20:24:09.254694+0000","last_clean":"2023-10-10T20:24:09.254694+0000","last_became_active":"2023-10-10T20:24:09.251976+0000","last_became_peered":"2023-10-10T20:24:09.251976+0000","last_unstale":"2023-10-10T20:24:09.254694+0000","last_undegraded":"2023-10-10T20:24:09.254694+0000","last_fullsized":"2023-10-10T20:24:09.254694+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:50:42.119166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"97'8","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.265611+0000","last_change":"2023-10-10T20:24:08.236244+0000","last_active":"2023-10-10T20:24:09.265611+0000","last_peered":"2023-10-10T20:24:09.265611+0000","last_clean":"2023-10-10T20:24:09.265611+0000","last_became_active":"2023-10-10T20:24:08.236054+0000","last_became_peered":"2023-10-10T20:24:08.236054+0000","last_unstale":"2023-10-10T20:24:09.265611+0000","last_undegraded":"2023-10-10T20:24:09.265611+0000","last_fullsized":"2023-10-10T20:24:09.265611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:44.116011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254557+0000","last_change":"2023-10-10T20:24:09.254498+0000","last_active":"2023-10-10T20:24:09.254557+0000","last_peered":"2023-10-10T20:24:09.254557+0000","last_clean":"2023-10-10T20:24:09.254557+0000","last_became_active":"2023-10-10T20:24:09.253398+0000","last_became_peered":"2023-10-10T20:24:09.253398+0000","last_unstale":"2023-10-10T20:24:09.254557+0000","last_undegraded":"2023-10-10T20:24:09.254557+0000","last_fullsized":"2023-10-10T20:24:09.254557+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:43.421965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.262450+0000","last_change":"2023-10-10T20:24:08.235371+0000","last_active":"2023-10-10T20:24:09.262450+0000","last_peered":"2023-10-10T20:24:09.262450+0000","last_clean":"2023-10-10T20:24:09.262450+0000","last_became_active":"2023-10-10T20:24:08.234812+0000","last_became_peered":"2023-10-10T20:24:08.234812+0000","last_unstale":"2023-10-10T20:24:09.262450+0000","last_undegraded":"2023-10-10T20:24:09.262450+0000","last_fullsized":"2023-10-10T20:24:09.262450+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:57:56.835843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.253376+0000","last_change":"2023-10-10T20:24:09.253311+0000","last_active":"2023-10-10T20:24:09.253376+0000","last_peered":"2023-10-10T20:24:09.253376+0000","last_clean":"2023-10-10T20:24:09.253376+0000","last_became_active":"2023-10-10T20:24:09.253104+0000","last_became_peered":"2023-10-10T20:24:09.253104+0000","last_unstale":"2023-10-10T20:24:09.253376+0000","last_undegraded":"2023-10-10T20:24:09.253376+0000","last_fullsized":"2023-10-10T20:24:09.253376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:53.815580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.263402+0000","last_change":"2023-10-10T20:24:08.232168+0000","last_active":"2023-10-10T20:24:09.263402+0000","last_peered":"2023-10-10T20:24:09.263402+0000","last_clean":"2023-10-10T20:24:09.263402+0000","last_became_active":"2023-10-10T20:24:08.231595+0000","last_became_peered":"2023-10-10T20:24:08.231595+0000","last_unstale":"2023-10-10T20:24:09.263402+0000","last_undegraded":"2023-10-10T20:24:09.263402+0000","last_fullsized":"2023-10-10T20:24:09.263402+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:00:44.712904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.248668+0000","last_change":"2023-10-10T20:24:09.248518+0000","last_active":"2023-10-10T20:24:09.248668+0000","last_peered":"2023-10-10T20:24:09.248668+0000","last_clean":"2023-10-10T20:24:09.248668+0000","last_became_active":"2023-10-10T20:24:09.247811+0000","last_became_peered":"2023-10-10T20:24:09.247811+0000","last_unstale":"2023-10-10T20:24:09.248668+0000","last_undegraded":"2023-10-10T20:24:09.248668+0000","last_fullsized":"2023-10-10T20:24:09.248668+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:44.663891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.240026+0000","last_change":"2023-10-10T20:24:08.229868+0000","last_active":"2023-10-10T20:24:09.240026+0000","last_peered":"2023-10-10T20:24:09.240026+0000","last_clean":"2023-10-10T20:24:09.240026+0000","last_became_active":"2023-10-10T20:24:08.229647+0000","last_became_peered":"2023-10-10T20:24:08.229647+0000","last_unstale":"2023-10-10T20:24:09.240026+0000","last_undegraded":"2023-10-10T20:24:09.240026+0000","last_fullsized":"2023-10-10T20:24:09.240026+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:06.981156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254381+0000","last_change":"2023-10-10T20:24:09.254346+0000","last_active":"2023-10-10T20:24:09.254381+0000","last_peered":"2023-10-10T20:24:09.254381+0000","last_clean":"2023-10-10T20:24:09.254381+0000","last_became_active":"2023-10-10T20:24:09.254028+0000","last_became_peered":"2023-10-10T20:24:09.254028+0000","last_unstale":"2023-10-10T20:24:09.254381+0000","last_undegraded":"2023-10-10T20:24:09.254381+0000","last_fullsized":"2023-10-10T20:24:09.254381+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:02:10.461486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.269622+0000","last_change":"2023-10-10T20:24:08.233812+0000","last_active":"2023-10-10T20:24:09.269622+0000","last_peered":"2023-10-10T20:24:09.269622+0000","last_clean":"2023-10-10T20:24:09.269622+0000","last_became_active":"2023-10-10T20:24:08.233144+0000","last_became_peered":"2023-10-10T20:24:08.233144+0000","last_unstale":"2023-10-10T20:24:09.269622+0000","last_undegraded":"2023-10-10T20:24:09.269622+0000","last_fullsized":"2023-10-10T20:24:09.269622+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:05.824307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.246337+0000","last_change":"2023-10-10T20:24:09.245599+0000","last_active":"2023-10-10T20:24:09.246337+0000","last_peered":"2023-10-10T20:24:09.246337+0000","last_clean":"2023-10-10T20:24:09.246337+0000","last_became_active":"2023-10-10T20:24:09.245332+0000","last_became_peered":"2023-10-10T20:24:09.245332+0000","last_unstale":"2023-10-10T20:24:09.246337+0000","last_undegraded":"2023-10-10T20:24:09.246337+0000","last_fullsized":"2023-10-10T20:24:09.246337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:18:20.617599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.245100+0000","last_change":"2023-10-10T20:24:08.231443+0000","last_active":"2023-10-10T20:24:09.245100+0000","last_peered":"2023-10-10T20:24:09.245100+0000","last_clean":"2023-10-10T20:24:09.245100+0000","last_became_active":"2023-10-10T20:24:08.231227+0000","last_became_peered":"2023-10-10T20:24:08.231227+0000","last_unstale":"2023-10-10T20:24:09.245100+0000","last_undegraded":"2023-10-10T20:24:09.245100+0000","last_fullsized":"2023-10-10T20:24:09.245100+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:15.273053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.252343+0000","last_change":"2023-10-10T20:24:09.250022+0000","last_active":"2023-10-10T20:24:09.252343+0000","last_peered":"2023-10-10T20:24:09.252343+0000","last_clean":"2023-10-10T20:24:09.252343+0000","last_became_active":"2023-10-10T20:24:09.249746+0000","last_became_peered":"2023-10-10T20:24:09.249746+0000","last_unstale":"2023-10-10T20:24:09.252343+0000","last_undegraded":"2023-10-10T20:24:09.252343+0000","last_fullsized":"2023-10-10T20:24:09.252343+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:46.478794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244091+0000","last_change":"2023-10-10T20:24:08.235444+0000","last_active":"2023-10-10T20:24:09.244091+0000","last_peered":"2023-10-10T20:24:09.244091+0000","last_clean":"2023-10-10T20:24:09.244091+0000","last_became_active":"2023-10-10T20:24:08.235109+0000","last_became_peered":"2023-10-10T20:24:08.235109+0000","last_unstale":"2023-10-10T20:24:09.244091+0000","last_undegraded":"2023-10-10T20:24:09.244091+0000","last_fullsized":"2023-10-10T20:24:09.244091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:17:27.135564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.252520+0000","last_change":"2023-10-10T20:24:09.252055+0000","last_active":"2023-10-10T20:24:09.252520+0000","last_peered":"2023-10-10T20:24:09.252520+0000","last_clean":"2023-10-10T20:24:09.252520+0000","last_became_active":"2023-10-10T20:24:09.251385+0000","last_became_peered":"2023-10-10T20:24:09.251385+0000","last_unstale":"2023-10-10T20:24:09.252520+0000","last_undegraded":"2023-10-10T20:24:09.252520+0000","last_fullsized":"2023-10-10T20:24:09.252520+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:02.009774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244728+0000","last_change":"2023-10-10T20:24:08.234637+0000","last_active":"2023-10-10T20:24:09.244728+0000","last_peered":"2023-10-10T20:24:09.244728+0000","last_clean":"2023-10-10T20:24:09.244728+0000","last_became_active":"2023-10-10T20:24:08.234394+0000","last_became_peered":"2023-10-10T20:24:08.234394+0000","last_unstale":"2023-10-10T20:24:09.244728+0000","last_undegraded":"2023-10-10T20:24:09.244728+0000","last_fullsized":"2023-10-10T20:24:09.244728+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:10.085013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244018+0000","last_change":"2023-10-10T20:24:09.243971+0000","last_active":"2023-10-10T20:24:09.244018+0000","last_peered":"2023-10-10T20:24:09.244018+0000","last_clean":"2023-10-10T20:24:09.244018+0000","last_became_active":"2023-10-10T20:24:09.243715+0000","last_became_peered":"2023-10-10T20:24:09.243715+0000","last_unstale":"2023-10-10T20:24:09.244018+0000","last_undegraded":"2023-10-10T20:24:09.244018+0000","last_fullsized":"2023-10-10T20:24:09.244018+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:45.119319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.265374+0000","last_change":"2023-10-10T20:24:08.237231+0000","last_active":"2023-10-10T20:24:09.265374+0000","last_peered":"2023-10-10T20:24:09.265374+0000","last_clean":"2023-10-10T20:24:09.265374+0000","last_became_active":"2023-10-10T20:24:08.237008+0000","last_became_peered":"2023-10-10T20:24:08.237008+0000","last_unstale":"2023-10-10T20:24:09.265374+0000","last_undegraded":"2023-10-10T20:24:09.265374+0000","last_fullsized":"2023-10-10T20:24:09.265374+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:44:46.850452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254045+0000","last_change":"2023-10-10T20:24:09.253737+0000","last_active":"2023-10-10T20:24:09.254045+0000","last_peered":"2023-10-10T20:24:09.254045+0000","last_clean":"2023-10-10T20:24:09.254045+0000","last_became_active":"2023-10-10T20:24:09.252936+0000","last_became_peered":"2023-10-10T20:24:09.252936+0000","last_unstale":"2023-10-10T20:24:09.254045+0000","last_undegraded":"2023-10-10T20:24:09.254045+0000","last_fullsized":"2023-10-10T20:24:09.254045+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:35:40.348233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.248009+0000","last_change":"2023-10-10T20:24:08.230584+0000","last_active":"2023-10-10T20:24:09.248009+0000","last_peered":"2023-10-10T20:24:09.248009+0000","last_clean":"2023-10-10T20:24:09.248009+0000","last_became_active":"2023-10-10T20:24:08.229988+0000","last_became_peered":"2023-10-10T20:24:08.229988+0000","last_unstale":"2023-10-10T20:24:09.248009+0000","last_undegraded":"2023-10-10T20:24:09.248009+0000","last_fullsized":"2023-10-10T20:24:09.248009+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:16:37.931680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.253916+0000","last_change":"2023-10-10T20:24:09.253843+0000","last_active":"2023-10-10T20:24:09.253916+0000","last_peered":"2023-10-10T20:24:09.253916+0000","last_clean":"2023-10-10T20:24:09.253916+0000","last_became_active":"2023-10-10T20:24:09.253520+0000","last_became_peered":"2023-10-10T20:24:09.253520+0000","last_unstale":"2023-10-10T20:24:09.253916+0000","last_undegraded":"2023-10-10T20:24:09.253916+0000","last_fullsized":"2023-10-10T20:24:09.253916+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:17:11.088321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247340+0000","last_change":"2023-10-10T20:24:08.235557+0000","last_active":"2023-10-10T20:24:09.247340+0000","last_peered":"2023-10-10T20:24:09.247340+0000","last_clean":"2023-10-10T20:24:09.247340+0000","last_became_active":"2023-10-10T20:24:08.235139+0000","last_became_peered":"2023-10-10T20:24:08.235139+0000","last_unstale":"2023-10-10T20:24:09.247340+0000","last_undegraded":"2023-10-10T20:24:09.247340+0000","last_fullsized":"2023-10-10T20:24:09.247340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:49.945114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.243947+0000","last_change":"2023-10-10T20:24:09.243874+0000","last_active":"2023-10-10T20:24:09.243947+0000","last_peered":"2023-10-10T20:24:09.243947+0000","last_clean":"2023-10-10T20:24:09.243947+0000","last_became_active":"2023-10-10T20:24:09.243615+0000","last_became_peered":"2023-10-10T20:24:09.243615+0000","last_unstale":"2023-10-10T20:24:09.243947+0000","last_undegraded":"2023-10-10T20:24:09.243947+0000","last_fullsized":"2023-10-10T20:24:09.243947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:03:14.521108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.247861+0000","last_change":"2023-10-10T20:24:08.236639+0000","last_active":"2023-10-10T20:24:09.247861+0000","last_peered":"2023-10-10T20:24:09.247861+0000","last_clean":"2023-10-10T20:24:09.247861+0000","last_became_active":"2023-10-10T20:24:08.236231+0000","last_became_peered":"2023-10-10T20:24:08.236231+0000","last_unstale":"2023-10-10T20:24:09.247861+0000","last_undegraded":"2023-10-10T20:24:09.247861+0000","last_fullsized":"2023-10-10T20:24:09.247861+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:15:39.200804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249543+0000","last_change":"2023-10-10T20:24:09.249448+0000","last_active":"2023-10-10T20:24:09.249543+0000","last_peered":"2023-10-10T20:24:09.249543+0000","last_clean":"2023-10-10T20:24:09.249543+0000","last_became_active":"2023-10-10T20:24:09.249208+0000","last_became_peered":"2023-10-10T20:24:09.249208+0000","last_unstale":"2023-10-10T20:24:09.249543+0000","last_undegraded":"2023-10-10T20:24:09.249543+0000","last_fullsized":"2023-10-10T20:24:09.249543+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:39.478815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.240576+0000","last_change":"2023-10-10T20:24:08.237039+0000","last_active":"2023-10-10T20:24:09.240576+0000","last_peered":"2023-10-10T20:24:09.240576+0000","last_clean":"2023-10-10T20:24:09.240576+0000","last_became_active":"2023-10-10T20:24:08.236829+0000","last_became_peered":"2023-10-10T20:24:08.236829+0000","last_unstale":"2023-10-10T20:24:09.240576+0000","last_undegraded":"2023-10-10T20:24:09.240576+0000","last_fullsized":"2023-10-10T20:24:09.240576+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:23:04.319591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.244516+0000","last_change":"2023-10-10T20:24:09.244456+0000","last_active":"2023-10-10T20:24:09.244516+0000","last_peered":"2023-10-10T20:24:09.244516+0000","last_clean":"2023-10-10T20:24:09.244516+0000","last_became_active":"2023-10-10T20:24:09.243948+0000","last_became_peered":"2023-10-10T20:24:09.243948+0000","last_unstale":"2023-10-10T20:24:09.244516+0000","last_undegraded":"2023-10-10T20:24:09.244516+0000","last_fullsized":"2023-10-10T20:24:09.244516+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:31:43.421965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.243384+0000","last_change":"2023-10-10T20:24:08.238440+0000","last_active":"2023-10-10T20:24:09.243384+0000","last_peered":"2023-10-10T20:24:09.243384+0000","last_clean":"2023-10-10T20:24:09.243384+0000","last_became_active":"2023-10-10T20:24:08.238017+0000","last_became_peered":"2023-10-10T20:24:08.238017+0000","last_unstale":"2023-10-10T20:24:09.243384+0000","last_undegraded":"2023-10-10T20:24:09.243384+0000","last_fullsized":"2023-10-10T20:24:09.243384+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:49.945114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.255676+0000","last_change":"2023-10-10T20:24:09.255603+0000","last_active":"2023-10-10T20:24:09.255676+0000","last_peered":"2023-10-10T20:24:09.255676+0000","last_clean":"2023-10-10T20:24:09.255676+0000","last_became_active":"2023-10-10T20:24:09.255353+0000","last_became_peered":"2023-10-10T20:24:09.255353+0000","last_unstale":"2023-10-10T20:24:09.255676+0000","last_undegraded":"2023-10-10T20:24:09.255676+0000","last_fullsized":"2023-10-10T20:24:09.255676+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:15:53.815580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.251234+0000","last_change":"2023-10-10T20:24:08.237610+0000","last_active":"2023-10-10T20:24:09.251234+0000","last_peered":"2023-10-10T20:24:09.251234+0000","last_clean":"2023-10-10T20:24:09.251234+0000","last_became_active":"2023-10-10T20:24:08.236989+0000","last_became_peered":"2023-10-10T20:24:08.236989+0000","last_unstale":"2023-10-10T20:24:09.251234+0000","last_undegraded":"2023-10-10T20:24:09.251234+0000","last_fullsized":"2023-10-10T20:24:09.251234+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:58:26.884427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254234+0000","last_change":"2023-10-10T20:24:09.254164+0000","last_active":"2023-10-10T20:24:09.254234+0000","last_peered":"2023-10-10T20:24:09.254234+0000","last_clean":"2023-10-10T20:24:09.254234+0000","last_became_active":"2023-10-10T20:24:09.253127+0000","last_became_peered":"2023-10-10T20:24:09.253127+0000","last_unstale":"2023-10-10T20:24:09.254234+0000","last_undegraded":"2023-10-10T20:24:09.254234+0000","last_fullsized":"2023-10-10T20:24:09.254234+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:34.163988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.243240+0000","last_change":"2023-10-10T20:24:08.241008+0000","last_active":"2023-10-10T20:24:09.243240+0000","last_peered":"2023-10-10T20:24:09.243240+0000","last_clean":"2023-10-10T20:24:09.243240+0000","last_became_active":"2023-10-10T20:24:08.240770+0000","last_became_peered":"2023-10-10T20:24:08.240770+0000","last_unstale":"2023-10-10T20:24:09.243240+0000","last_undegraded":"2023-10-10T20:24:09.243240+0000","last_fullsized":"2023-10-10T20:24:09.243240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:50.364205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.254532+0000","last_change":"2023-10-10T20:24:09.254421+0000","last_active":"2023-10-10T20:24:09.254532+0000","last_peered":"2023-10-10T20:24:09.254532+0000","last_clean":"2023-10-10T20:24:09.254532+0000","last_became_active":"2023-10-10T20:24:09.251387+0000","last_became_peered":"2023-10-10T20:24:09.251387+0000","last_unstale":"2023-10-10T20:24:09.254532+0000","last_undegraded":"2023-10-10T20:24:09.254532+0000","last_fullsized":"2023-10-10T20:24:09.254532+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:08.221142+0000","last_clean_scrub_stamp":"2023-10-10T20:24:08.221142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:22:50.948422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.263979+0000","last_change":"2023-10-10T20:24:08.239270+0000","last_active":"2023-10-10T20:24:09.263979+0000","last_peered":"2023-10-10T20:24:09.263979+0000","last_clean":"2023-10-10T20:24:09.263979+0000","last_became_active":"2023-10-10T20:24:08.238994+0000","last_became_peered":"2023-10-10T20:24:08.238994+0000","last_unstale":"2023-10-10T20:24:09.263979+0000","last_undegraded":"2023-10-10T20:24:09.263979+0000","last_fullsized":"2023-10-10T20:24:09.263979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:07.217834+0000","last_clean_scrub_stamp":"2023-10-10T20:24:07.217834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:38.475507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-10-10T20:24:09.249282+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:24:09.249282+0000","last_peered":"2023-10-10T20:24:09.249282+0000","last_clean":"2023-10-10T20:24:09.249282+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:24:09.249282+0000","last_undegraded":"2023-10-10T20:24:09.249282+0000","last_fullsized":"2023-10-10T20:24:09.249282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1090,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194222,"statfs":{"total":200000000,"available":198883419,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.47099999999999997},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70499999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.70999999999999996,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50600000000000001,"15min":0.48799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69999999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.62},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70199999999999996,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57199999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.65000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.318,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67300000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:23:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76700000000000002,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67600000000000005,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.626,"5min":1.865,"15min":1.865},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.749,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.504},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68500000000000005,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73299999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.38100000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69399999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.68100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":15.862,"15min":15.862},"last":0.497}]},{"osd":4,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.44500000000000001,"15min":0.43099999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.63300000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68200000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.38900000000000001},"min":{"1min":0.23400000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.50700000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.37,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54800000000000004,"5min":1.647,"15min":1.647},"last":0.38}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56899999999999995,"5min":1.524,"15min":1.524},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39800000000000002,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.58799999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.42499999999999999,"15min":0.46200000000000002},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61199999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.40500000000000003},"min":{"1min":0.317,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63,"5min":1.673,"15min":1.673},"last":0.40000000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":1.633,"15min":1.633},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":1.506,"15min":1.506},"last":0.79800000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":1.696,"15min":1.696},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.627,"15min":0.68300000000000005},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77700000000000002,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":1.585,"15min":1.585},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80500000000000005,"5min":1.478,"15min":1.478},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":1.722,"15min":1.722},"last":0.83599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1089,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198884466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.39500000000000002,"15min":0.46300000000000002},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.47899999999999998,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.432,"15min":0.48199999999999998},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53300000000000003,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.34999999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.42399999999999999,"15min":0.48999999999999999},"min":{"1min":0.24199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.38500000000000001,"15min":0.40500000000000003},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.57399999999999995,"5min":1.256,"15min":1.256},"last":0.31}]},{"osd":3,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36599999999999999,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47899999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41299999999999998,"15min":0.434},"min":{"1min":0.28299999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.57899999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.36399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2030000000000001,"5min":1.204,"15min":1.204},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.78100000000000003,"15min":0.98399999999999999},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.82799999999999996,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.503},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.80200000000000005,"5min":1.347,"15min":1.347},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.432,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73499999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73199999999999998,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.45600000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.42199999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.436},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.745,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199474314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.38200000000000001,"15min":0.39600000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54100000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.47299999999999998,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.50800000000000001,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.376}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.42599999999999999,"15min":0.39800000000000002},"min":{"1min":0.22600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.52500000000000002,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.497,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.56200000000000006,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.51900000000000002,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.48699999999999999,"5min":11.815,"15min":11.815},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.58399999999999996,"15min":0.46899999999999997},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.58399999999999996,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.311}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.74299999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72399999999999998,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":6.399,"15min":6.399},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.54200000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.66900000000000004,"15min":0.60299999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.64600000000000002,"15min":0.60899999999999999},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66500000000000004,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.44400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76900000000000002,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":515,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194797,"statfs":{"total":200000000,"available":199472321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.29899999999999999},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40200000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57099999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48699999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.505,"5min":0.751,"15min":0.751},"last":0.376}]},{"osd":2,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.36199999999999999},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53100000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53700000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72899999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74399999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.755}]},{"osd":7,"last update":"Tue Oct 10 20:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.72799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.48199999999999998},"min":{"1min":0.35899999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.82899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.48699999999999999},"min":{"1min":0.436,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.499}]},{"osd":1,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.435,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76900000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.498},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.72999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51700000000000002,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.52200000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84599999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.434}]},{"osd":7,"last update":"Tue Oct 10 20:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.50900000000000001},"min":{"1min":0.32500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76600000000000001,"5min":0.875,"15min":0.875},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.54500000000000004},"min":{"1min":0.36499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":515,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194797,"statfs":{"total":200000000,"available":199472299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.505,"15min":0.497},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68799999999999994,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.436}]},{"osd":1,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.65800000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68799999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41499999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69699999999999995,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.45900000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.498}]},{"osd":4,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.875,"15min":0.875},"last":0.34999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56999999999999995}]}]}],"pool_statfs":[]}} 2023-10-10T20:24:13.678 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:24:13.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:24:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.833+0000 7fa9e7d77700 1 Processor -- start 2023-10-10T20:24:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.833+0000 7fa9e7d77700 1 -- start start 2023-10-10T20:24:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e7d77700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e0159ec0 0x7fa9e00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e7d77700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa9e006acb0 0x7fa9e00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e7d77700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e00b0790 con 0x7fa9e015a290 2023-10-10T20:24:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e7d77700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00ce1f0 con 0x7fa9e0159ec0 2023-10-10T20:24:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e7d77700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00b0b20 con 0x7fa9e006acb0 2023-10-10T20:24:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e5b13700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e0159ec0 0x7fa9e00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e5b13700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e0159ec0 0x7fa9e00af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42890/0 (socket says 172.21.15.148:42890) 2023-10-10T20:24:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e5b13700 1 -- 172.21.15.148:0/3160514892 learned_addr learned my addr 172.21.15.148:0/3160514892 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.834+0000 7fa9e6314700 1 --2- 172.21.15.148:0/3160514892 >> v2:172.21.15.161:3300/0 conn(0x7fa9e006acb0 0x7fa9e00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e5b13700 1 -- 172.21.15.148:0/3160514892 >> v2:172.21.15.161:3300/0 conn(0x7fa9e006acb0 msgr2=0x7fa9e00afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e5b13700 1 --2- 172.21.15.148:0/3160514892 >> v2:172.21.15.161:3300/0 conn(0x7fa9e006acb0 0x7fa9e00afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e5b13700 1 -- 172.21.15.148:0/3160514892 >> v1:172.21.15.161:6789/0 conn(0x7fa9e015a290 legacy=0x7fa9e006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e5b13700 1 -- 172.21.15.148:0/3160514892 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00b04b0 con 0x7fa9e0159ec0 2023-10-10T20:24:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e5b13700 1 --2- 172.21.15.148:0/3160514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e00af940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00711f0 tx=0x7fa9dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea875cad117b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e4b11700 1 -- 172.21.15.148:0/3160514892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc009e70 con 0x7fa9e0159ec0 2023-10-10T20:24:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e4b11700 1 -- 172.21.15.148:0/3160514892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc00b860 con 0x7fa9e0159ec0 2023-10-10T20:24:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e4b11700 1 -- 172.21.15.148:0/3160514892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00bc00 con 0x7fa9e0159ec0 2023-10-10T20:24:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 -- 172.21.15.148:0/3160514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 msgr2=0x7fa9e00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/3160514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e00af940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00711f0 tx=0x7fa9dc009250 comp rx=0 tx=0).stop 2023-10-10T20:24:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 -- 172.21.15.148:0/3160514892 shutdown_connections 2023-10-10T20:24:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/3160514892 >> v2:172.21.15.161:3300/0 conn(0x7fa9e006acb0 0x7fa9e00afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/3160514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e00af940 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.835+0000 7fa9e7d77700 1 -- 172.21.15.148:0/3160514892 >> 172.21.15.148:0/3160514892 conn(0x7fa9e00d1880 msgr2=0x7fa9e00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.836+0000 7fa9e7d77700 1 -- 172.21.15.148:0/3160514892 shutdown_connections 2023-10-10T20:24:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.836+0000 7fa9e7d77700 1 -- 172.21.15.148:0/3160514892 wait complete. 2023-10-10T20:24:13.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 Processor -- start 2023-10-10T20:24:13.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 -- start start 2023-10-10T20:24:13.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9e006acb0 0x7fa9e016e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e015a290 0x7fa9e015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015e530 con 0x7fa9e0159ec0 2023-10-10T20:24:13.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e016eb20 con 0x7fa9e015a290 2023-10-10T20:24:13.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e7d77700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01571c0 con 0x7fa9e006acb0 2023-10-10T20:24:13.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e5312700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e5312700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42904/0 (socket says 172.21.15.148:42904) 2023-10-10T20:24:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.837+0000 7fa9e5312700 1 -- 172.21.15.148:0/4161352604 learned_addr learned my addr 172.21.15.148:0/4161352604 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e6314700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e015a290 0x7fa9e015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5b13700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9e006acb0 0x7fa9e016e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 -- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9e006acb0 msgr2=0x7fa9e016e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9e006acb0 0x7fa9e016e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 -- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e015a290 msgr2=0x7fa9e015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:13.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e015a290 0x7fa9e015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:13.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 -- 172.21.15.148:0/4161352604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0094f0 con 0x7fa9e0159ec0 2023-10-10T20:24:13.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e5312700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000c590 tx=0x7fa9d0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0011590 con 0x7fa9e0159ec0 2023-10-10T20:24:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0011730 con 0x7fa9e0159ec0 2023-10-10T20:24:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e015a8b0 con 0x7fa9e0159ec0 2023-10-10T20:24:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0011af0 con 0x7fa9e0159ec0 2023-10-10T20:24:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.838+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e015df70 con 0x7fa9e0159ec0 2023-10-10T20:24:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.839+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e015aa40 con 0x7fa9e0159ec0 2023-10-10T20:24:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.839+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9d0012420 con 0x7fa9e0159ec0 2023-10-10T20:24:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.840+0000 7fa9d5ffb700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 0x7fa9c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.840+0000 7fa9e5b13700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 0x7fa9c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.840+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa9d0083850 con 0x7fa9e0159ec0 2023-10-10T20:24:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.840+0000 7fa9e5b13700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 0x7fa9c40693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00711f0 tx=0x7fa9dc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:13.844+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9d004b5a0 con 0x7fa9e0159ec0 2023-10-10T20:24:14.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.106+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa9e00ceab0 con 0x7fa9e0159ec0 2023-10-10T20:24:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.110+0000 7fa9d5ffb700 1 -- 172.21.15.148:0/4161352604 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa9d0017070 con 0x7fa9e0159ec0 2023-10-10T20:24:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 msgr2=0x7fa9c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 0x7fa9c40693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00711f0 tx=0x7fa9dc017000 comp rx=0 tx=0).stop 2023-10-10T20:24:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 msgr2=0x7fa9e0156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000c590 tx=0x7fa9d0007f30 comp rx=0 tx=0).stop 2023-10-10T20:24:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 shutdown_connections 2023-10-10T20:24:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa9c4066f30 0x7fa9c40693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e015a290 0x7fa9e015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9e006acb0 0x7fa9e016e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 --2- 172.21.15.148:0/4161352604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e0159ec0 0x7fa9e0156c80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 >> 172.21.15.148:0/4161352604 conn(0x7fa9e00d1880 msgr2=0x7fa9e006a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:14.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 shutdown_connections 2023-10-10T20:24:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.112+0000 7fa9e7d77700 1 -- 172.21.15.148:0/4161352604 wait complete. 2023-10-10T20:24:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:24:14.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:14.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.239+0000 7fba7acd0700 1 Processor -- start 2023-10-10T20:24:14.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.239+0000 7fba7acd0700 1 -- start start 2023-10-10T20:24:14.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba7acd0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fba7415a620 0x7fba7415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba7acd0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7415cbb0 con 0x7fba74157870 2023-10-10T20:24:14.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba7acd0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7415bfa0 con 0x7fba7415d580 2023-10-10T20:24:14.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba7acd0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7415bdb0 con 0x7fba7415a620 2023-10-10T20:24:14.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba78a6c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fba74157870 0x7fba7415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41108/0 (socket says 172.21.15.148:41108) 2023-10-10T20:24:14.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.240+0000 7fba78a6c700 1 -- 172.21.15.148:0/1663977781 learned_addr learned my addr 172.21.15.148:0/1663977781 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:14.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> v2:172.21.15.161:3300/0 conn(0x7fba7415a620 0x7fba7415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715587761 0 0) 0x7fba7415cbb0 con 0x7fba74157870 2023-10-10T20:24:14.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba64003570 con 0x7fba74157870 2023-10-10T20:24:14.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244702739 0 0) 0x7fba7415bfa0 con 0x7fba7415d580 2023-10-10T20:24:14.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7415cbb0 con 0x7fba7415d580 2023-10-10T20:24:14.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3719934477 0 0) 0x7fba64003570 con 0x7fba74157870 2023-10-10T20:24:14.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 >> v2:172.21.15.161:3300/0 conn(0x7fba7415a620 msgr2=0x7fba7415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:14.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 --2- 172.21.15.148:0/1663977781 >> v2:172.21.15.161:3300/0 conn(0x7fba7415a620 0x7fba7415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 >> v1:172.21.15.161:6789/0 conn(0x7fba7415d580 legacy=0x7fba74158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7415c280 con 0x7fba74157870 2023-10-10T20:24:14.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fba68001470 con 0x7fba74157870 2023-10-10T20:24:14.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 >> v1:172.21.15.148:6789/0 conn(0x7fba74157870 legacy=0x7fba7415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba64003bf0 0x7fba64003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba640051c0 0x7fba6400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400c130 con 0x7fba64004810 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400c2a0 con 0x7fba640051c0 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6400c460 con 0x7fba64003bf0 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> v2:172.21.15.161:3300/0 conn(0x7fba7415a620 0x7fba7415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:14.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba7926d700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba640051c0 0x7fba6400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba78a6c700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba64003bf0 0x7fba64003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 -- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba64003bf0 msgr2=0x7fba64003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba64003bf0 0x7fba64003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 -- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba640051c0 msgr2=0x7fba6400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:14.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba640051c0 0x7fba6400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba73fff700 1 -- 172.21.15.148:0/1663977781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64003570 con 0x7fba64004810 2023-10-10T20:24:14.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.241+0000 7fba7926d700 1 -- 172.21.15.148:0/1663977781 reap_dead start 2023-10-10T20:24:14.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba73fff700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba64004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fba60006f00 tx=0x7fba600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa7e837605e4602 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:14.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60009740 con 0x7fba64004810 2023-10-10T20:24:14.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000e8f0 con 0x7fba64004810 2023-10-10T20:24:14.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba737fe700 1 -- 172.21.15.148:0/1663977781 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000ec90 con 0x7fba64004810 2023-10-10T20:24:14.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 -- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 msgr2=0x7fba64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba64004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fba60006f00 tx=0x7fba600091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 -- 172.21.15.148:0/1663977781 shutdown_connections 2023-10-10T20:24:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 --2- 172.21.15.148:0/1663977781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba64004c80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 -- 172.21.15.148:0/1663977781 >> 172.21.15.148:0/1663977781 conn(0x7fba74068ce0 msgr2=0x7fba740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 -- 172.21.15.148:0/1663977781 shutdown_connections 2023-10-10T20:24:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.242+0000 7fba7acd0700 1 -- 172.21.15.148:0/1663977781 wait complete. 2023-10-10T20:24:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.243+0000 7fba7acd0700 1 Processor -- start 2023-10-10T20:24:14.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 -- start start 2023-10-10T20:24:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 0x7fba740d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba7416db20 0x7fba7416df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74158760 con 0x7fba64004810 2023-10-10T20:24:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740d0c30 con 0x7fba740d0280 2023-10-10T20:24:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba7acd0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba74158bf0 con 0x7fba7416db20 2023-10-10T20:24:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba78a6c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba78a6c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42914/0 (socket says 172.21.15.148:42914) 2023-10-10T20:24:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.244+0000 7fba78a6c700 1 -- 172.21.15.148:0/367067556 learned_addr learned my addr 172.21.15.148:0/367067556 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 -- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba7416db20 msgr2=0x7fba7416df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba7416db20 0x7fba7416df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 -- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 msgr2=0x7fba740d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba73fff700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 0x7fba740d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 0x7fba740d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 -- 172.21.15.148:0/367067556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74157e30 con 0x7fba64004810 2023-10-10T20:24:14.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba73fff700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 0x7fba740d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:14.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba78a6c700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba680073c0 tx=0x7fba68001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68018030 con 0x7fba64004810 2023-10-10T20:24:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba680010c0 con 0x7fba64004810 2023-10-10T20:24:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba74158110 con 0x7fba64004810 2023-10-10T20:24:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800fb00 con 0x7fba64004810 2023-10-10T20:24:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.245+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba740b2460 con 0x7fba64004810 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.246+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7404c3c0 con 0x7fba64004810 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.246+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba6801b070 con 0x7fba64004810 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.247+0000 7fba70ff9700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.247+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba68019070 con 0x7fba64004810 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.247+0000 7fba73fff700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.247+0000 7fba73fff700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 0x7fba580693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fba7414fca0 tx=0x7fba60019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.251+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba68051730 con 0x7fba64004810 2023-10-10T20:24:14.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.475+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba7415c560 con 0x7fba64004810 2023-10-10T20:24:14.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.475+0000 7fba70ff9700 1 -- 172.21.15.148:0/367067556 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fba68054d50 con 0x7fba64004810 2023-10-10T20:24:14.478 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:14.479 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:14.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.477+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 msgr2=0x7fba580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.477+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 0x7fba580693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fba7414fca0 tx=0x7fba60019000 comp rx=0 tx=0).stop 2023-10-10T20:24:14.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 msgr2=0x7fba740b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:14.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba680073c0 tx=0x7fba68001df0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 shutdown_connections 2023-10-10T20:24:14.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fba58066f30 0x7fba580693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba7416db20 0x7fba7416df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba64004810 0x7fba740b1f20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 --2- 172.21.15.148:0/367067556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba740d0280 0x7fba740d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:14.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 >> 172.21.15.148:0/367067556 conn(0x7fba74068ce0 msgr2=0x7fba741565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:14.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 shutdown_connections 2023-10-10T20:24:14.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:14.478+0000 7fba7acd0700 1 -- 172.21.15.148:0/367067556 wait complete. 2023-10-10T20:24:14.483 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:14.893 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:24:08.605542+0000', 'modified': '2023-10-10T20:24:12.264440+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'b', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/3811497457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 3811497457}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3811497457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:24:14.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:24:14.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:24:15.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.011+0000 7f594089b700 1 Processor -- start 2023-10-10T20:24:15.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.011+0000 7f594089b700 1 -- start start 2023-10-10T20:24:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f593c157870 0x7f593c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f593c15a620 0x7f593c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c15d580 0x7f593c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593c15cbb0 con 0x7f593c157870 2023-10-10T20:24:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593c15bfa0 con 0x7f593c15a620 2023-10-10T20:24:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f594089b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c15c280 con 0x7f593c15d580 2023-10-10T20:24:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f593c157870 0x7f593c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f593c157870 0x7f593c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42926/0 (socket says 172.21.15.148:42926) 2023-10-10T20:24:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 -- 172.21.15.148:0/1914795331 learned_addr learned my addr 172.21.15.148:0/1914795331 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593ad9d700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c15d580 0x7f593c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f5931d9b700 1 --2- 172.21.15.148:0/1914795331 >> v2:172.21.15.161:3300/0 conn(0x7f593c15a620 0x7f593c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 -- 172.21.15.148:0/1914795331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c15d580 msgr2=0x7f593c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c15d580 0x7f593c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 -- 172.21.15.148:0/1914795331 >> v2:172.21.15.161:3300/0 conn(0x7f593c15a620 msgr2=0x7f593c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 --2- 172.21.15.148:0/1914795331 >> v2:172.21.15.161:3300/0 conn(0x7f593c15a620 0x7f593c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 -- 172.21.15.148:0/1914795331 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c15c560 con 0x7f593c157870 2023-10-10T20:24:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.012+0000 7f593a59c700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c157870 0x7f593c15b6a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f593c15cb20 tx=0x7f592c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db8e9f8d72ebf2d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f5939d9b700 1 -- 172.21.15.148:0/1914795331 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c00b640 con 0x7f593c157870 2023-10-10T20:24:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f5939d9b700 1 -- 172.21.15.148:0/1914795331 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c00b7a0 con 0x7f593c157870 2023-10-10T20:24:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f5939d9b700 1 -- 172.21.15.148:0/1914795331 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c00bb60 con 0x7f593c157870 2023-10-10T20:24:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 -- 172.21.15.148:0/1914795331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c157870 msgr2=0x7f593c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c157870 0x7f593c15b6a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f593c15cb20 tx=0x7f592c009250 comp rx=0 tx=0).stop 2023-10-10T20:24:15.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 -- 172.21.15.148:0/1914795331 shutdown_connections 2023-10-10T20:24:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c15d580 0x7f593c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 --2- 172.21.15.148:0/1914795331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c157870 0x7f593c15b6a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 --2- 172.21.15.148:0/1914795331 >> v2:172.21.15.161:3300/0 conn(0x7f593c15a620 0x7f593c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 -- 172.21.15.148:0/1914795331 >> 172.21.15.148:0/1914795331 conn(0x7f593c068ce0 msgr2=0x7f593c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 -- 172.21.15.148:0/1914795331 shutdown_connections 2023-10-10T20:24:15.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.013+0000 7f594089b700 1 -- 172.21.15.148:0/1914795331 wait complete. 2023-10-10T20:24:15.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.014+0000 7f594089b700 1 Processor -- start 2023-10-10T20:24:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.014+0000 7f594089b700 1 -- start start 2023-10-10T20:24:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c157870 0x7f593c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c15d580 0x7f593c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0b21d0 con 0x7f593c15a620 2023-10-10T20:24:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0b18b0 con 0x7f593c15d580 2023-10-10T20:24:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f594089b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c0b0ca0 con 0x7f593c157870 2023-10-10T20:24:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f5931d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f5931d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42936/0 (socket says 172.21.15.148:42936) 2023-10-10T20:24:15.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f5931d9b700 1 -- 172.21.15.148:0/3663911512 learned_addr learned my addr 172.21.15.148:0/3663911512 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f593a59c700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c157870 0x7f593c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.015+0000 7f593ad9d700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c15d580 0x7f593c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 -- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c157870 msgr2=0x7f593c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c157870 0x7f593c0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 -- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c15d580 msgr2=0x7f593c154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c15d580 0x7f593c154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 -- 172.21.15.148:0/3663911512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0094f0 con 0x7f593c15a620 2023-10-10T20:24:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f592400ec50 tx=0x7f59240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5932ffd700 1 -- 172.21.15.148:0/3663911512 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592400c850 con 0x7f593c15a620 2023-10-10T20:24:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5932ffd700 1 -- 172.21.15.148:0/3663911512 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592400c9b0 con 0x7f593c15a620 2023-10-10T20:24:15.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c0b0f80 con 0x7f593c15a620 2023-10-10T20:24:15.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f5932ffd700 1 -- 172.21.15.148:0/3663911512 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592400cd70 con 0x7f593c15a620 2023-10-10T20:24:15.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.016+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c0b1540 con 0x7f593c15a620 2023-10-10T20:24:15.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.017+0000 7f5932ffd700 1 -- 172.21.15.148:0/3663911512 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5924013680 con 0x7f593c15a620 2023-10-10T20:24:15.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.018+0000 7f5932ffd700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 0x7f591c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.018+0000 7f5932ffd700 1 -- 172.21.15.148:0/3663911512 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5924084760 con 0x7f593c15a620 2023-10-10T20:24:15.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.018+0000 7f593a59c700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 0x7f591c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.018+0000 7f593a59c700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 0x7f591c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f593c15cb20 tx=0x7f592c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 Processor -- start 2023-10-10T20:24:15.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 -- start start 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4ead00 0x56049f4c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 0x56049f49ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f3f7a70 con 0x56049f4a0a50 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f3f5270 con 0x56049f4ead00 2023-10-10T20:24:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.020+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56049f3f1060 con 0x56049f4c9f60 2023-10-10T20:24:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42938/0 (socket says 172.21.15.148:42938) 2023-10-10T20:24:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 -- 172.21.15.148:0/4239573541 learned_addr learned my addr 172.21.15.148:0/4239573541 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f5931d9b700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4ead00 0x56049f4c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593ad9d700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 0x56049f49ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 -- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 msgr2=0x56049f49ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 0x56049f49ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 -- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4ead00 msgr2=0x56049f4c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4ead00 0x56049f4c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 -- 172.21.15.148:0/4239573541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049f28b270 con 0x56049f4a0a50 2023-10-10T20:24:15.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593ad9d700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 0x56049f49ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:15.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f593a59c700 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f592c000d00 tx=0x7f592c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db73d8f5b789a1d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f591b7fe700 1 -- 172.21.15.148:0/4239573541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c02d070 con 0x56049f4a0a50 2023-10-10T20:24:15.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f591b7fe700 1 -- 172.21.15.148:0/4239573541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c01a8b0 con 0x56049f4a0a50 2023-10-10T20:24:15.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.021+0000 7f591b7fe700 1 -- 172.21.15.148:0/4239573541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c021070 con 0x56049f4a0a50 2023-10-10T20:24:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 -- 172.21.15.148:0/4239573541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 msgr2=0x56049f4a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f592c000d00 tx=0x7f592c021000 comp rx=0 tx=0).stop 2023-10-10T20:24:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 -- 172.21.15.148:0/4239573541 shutdown_connections 2023-10-10T20:24:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4c9f60 0x56049f49ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f4a0e20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 --2- 172.21.15.148:0/4239573541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4ead00 0x56049f4c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 -- 172.21.15.148:0/4239573541 >> 172.21.15.148:0/4239573541 conn(0x56049f4a2370 msgr2=0x56049f4a2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 -- 172.21.15.148:0/4239573541 shutdown_connections 2023-10-10T20:24:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.022+0000 7f5949b94b80 1 -- 172.21.15.148:0/4239573541 wait complete. 2023-10-10T20:24:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.023+0000 7f5949b94b80 1 -- start start 2023-10-10T20:24:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.023+0000 7f5949b94b80 1 Processor -- start 2023-10-10T20:24:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4a0a50 0x56049f4df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f39c890 con 0x56049f4c9f60 2023-10-10T20:24:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f3b7ea0 con 0x56049f4a0a50 2023-10-10T20:24:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56049f273e30 con 0x56049f4ead00 2023-10-10T20:24:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 10 client.0 _ll_register_callbacks cb 0x56049f464110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:24:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42942/0 (socket says 172.21.15.148:42942) 2023-10-10T20:24:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 -- 172.21.15.148:0/3534749991 learned_addr learned my addr 172.21.15.148:0/3534749991 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f593ad9d700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f593a59c700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4a0a50 0x56049f4df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 -- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 msgr2=0x56049f49bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 -- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4a0a50 msgr2=0x56049f4df5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4a0a50 0x56049f4df5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 -- 172.21.15.148:0/3534749991 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c15c560 con 0x56049f4c9f60 2023-10-10T20:24:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5931d9b700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f592400cfd0 tx=0x7f5924015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc12bf1b75391e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5918ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:24:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924055460 con 0x56049f4c9f60 2023-10-10T20:24:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5924055690 con 0x56049f4c9f60 2023-10-10T20:24:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56049f495b70 con 0x56049f4c9f60 2023-10-10T20:24:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59240588e0 con 0x56049f4c9f60 2023-10-10T20:24:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 10 client.6513 fetch_fsmap learned FSMap version 49 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.024+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x56049f22dfe0 con 0x56049f4c9f60 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56049f495b70 con 0x56049f4c9f60 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f592405c1a0 con 0x56049f4c9f60 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 10 client.6513 fetch_fsmap finished waiting for FSMap version 49 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 10 client.6513 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:24:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 20 client.6513 populate_metadata read hostname 'smithi148' 2023-10-10T20:24:15.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4cef50 0x56049f4cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 4 client.6513 mds_command: new command op to 6381 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:24:15.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 --> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56049f472aa0 con 0x56049f4cef50 2023-10-10T20:24:15.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f593a59c700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4cef50 0x56049f4cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f5918ff9700 10 client.6513 ms_handle_connect on v2:172.21.15.161:6833/3811497457 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.025+0000 7f593a59c700 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4cef50 0x56049f4cf320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f56e61526e8441 server_cookie=f6704180adfbc136 in_seq=0 out_seq=0 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5918ff9700 1 -- 172.21.15.148:0/3534749991 <== mds.0 v2:172.21.15.161:6833/3811497457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56049f472aa0 con 0x56049f4cef50 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5918ff9700 10 client.6513 handle_command_reply: tid=0 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 client.6513 shutdown 2023-10-10T20:24:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 msgr2=0x56049f4e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f592400cfd0 tx=0x7f5924015040 comp rx=0 tx=0).stop 2023-10-10T20:24:15.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 shutdown_connections 2023-10-10T20:24:15.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4cef50 0x56049f4cf320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5918ff9700 0 client.6513 ms_handle_reset on v2:172.21.15.161:6833/3811497457 2023-10-10T20:24:15.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4a0a50 0x56049f4df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 --2- 172.21.15.148:0/3534749991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4c9f60 0x56049f4e19e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 >> 172.21.15.148:0/3534749991 conn(0x56049f4a2370 msgr2=0x56049f49d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 shutdown_connections 2023-10-10T20:24:15.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.026+0000 7f5949b94b80 1 -- 172.21.15.148:0/3534749991 wait complete. 2023-10-10T20:24:15.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.027+0000 7f5949b94b80 20 client.6513 trim_cache size 0 max 16384 2023-10-10T20:24:15.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.043+0000 7f5949b94b80 1 Processor -- start 2023-10-10T20:24:15.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.043+0000 7f5949b94b80 1 -- start start 2023-10-10T20:24:15.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f4a0370 con 0x56049f4a0a50 2023-10-10T20:24:15.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f4a30a0 con 0x56049f4c9f60 2023-10-10T20:24:15.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56049f479410 con 0x56049f4ead00 2023-10-10T20:24:15.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42958/0 (socket says 172.21.15.148:42958) 2023-10-10T20:24:15.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 -- 172.21.15.148:0/479452601 learned_addr learned my addr 172.21.15.148:0/479452601 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 -- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 msgr2=0x56049f49cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 -- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 msgr2=0x56049f49c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.044+0000 7f593a59c700 1 -- 172.21.15.148:0/479452601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049f3bf780 con 0x56049f4a0a50 2023-10-10T20:24:15.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f593a59c700 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f592c006e80 tx=0x7f592c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81991722cd41a92 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f591bfff700 1 -- 172.21.15.148:0/479452601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c02d040 con 0x56049f4a0a50 2023-10-10T20:24:15.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f591bfff700 1 -- 172.21.15.148:0/479452601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c039040 con 0x56049f4a0a50 2023-10-10T20:24:15.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f591bfff700 1 -- 172.21.15.148:0/479452601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c037040 con 0x56049f4a0a50 2023-10-10T20:24:15.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 -- 172.21.15.148:0/479452601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 msgr2=0x56049f49df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f592c006e80 tx=0x7f592c014000 comp rx=0 tx=0).stop 2023-10-10T20:24:15.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 -- 172.21.15.148:0/479452601 shutdown_connections 2023-10-10T20:24:15.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f49df20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 --2- 172.21.15.148:0/479452601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 -- 172.21.15.148:0/479452601 >> 172.21.15.148:0/479452601 conn(0x56049f4a2370 msgr2=0x56049f4a3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 -- 172.21.15.148:0/479452601 shutdown_connections 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.045+0000 7f5949b94b80 1 -- 172.21.15.148:0/479452601 wait complete. 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.046+0000 7f5949b94b80 1 -- start start 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.046+0000 7f5949b94b80 1 Processor -- start 2023-10-10T20:24:15.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f4ea360 con 0x56049f4a0a50 2023-10-10T20:24:15.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049f4c88b0 con 0x56049f4c9f60 2023-10-10T20:24:15.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56049f54cee0 con 0x56049f4ead00 2023-10-10T20:24:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 10 client.0 _ll_register_callbacks cb 0x56049f464110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:24:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42970/0 (socket says 172.21.15.148:42970) 2023-10-10T20:24:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 -- 172.21.15.148:0/1672621241 learned_addr learned my addr 172.21.15.148:0/1672621241 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593ad9d700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5931d9b700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 -- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 msgr2=0x56049f49cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 -- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 msgr2=0x56049f49c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 -- 172.21.15.148:0/1672621241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56049f28b270 con 0x56049f4a0a50 2023-10-10T20:24:15.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f593a59c700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f592c0093a0 tx=0x7f592c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6ab88091467be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5918ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:24:15.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c02d040 con 0x56049f4a0a50 2023-10-10T20:24:15.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f592c04ab00 con 0x56049f4a0a50 2023-10-10T20:24:15.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.047+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f592c01a8b0 con 0x56049f4a0a50 2023-10-10T20:24:15.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592c037040 con 0x56049f4a0a50 2023-10-10T20:24:15.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 10 client.6519 fetch_fsmap learned FSMap version 49 2023-10-10T20:24:15.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x56049f3a6470 con 0x56049f4a0a50 2023-10-10T20:24:15.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c01a8b0 con 0x56049f4a0a50 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f592c03e970 con 0x56049f4a0a50 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 10 client.6519 fetch_fsmap finished waiting for FSMap version 49 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 10 client.6519 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 20 client.6519 populate_metadata read hostname 'smithi148' 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4e6760 0x56049f4e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 4 client.6519 mds_command: new command op to 6381 tid=0[{"prefix": "status"}] 2023-10-10T20:24:15.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 --> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56049f393e00 con 0x56049f4e6760 2023-10-10T20:24:15.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5931d9b700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4e6760 0x56049f4e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5918ff9700 10 client.6519 ms_handle_connect on v2:172.21.15.161:6833/3811497457 2023-10-10T20:24:15.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.048+0000 7f5931d9b700 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4e6760 0x56049f4e6b30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ed9bcd256b4113 server_cookie=ebe246ddf9afda44 in_seq=0 out_seq=0 2023-10-10T20:24:15.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5918ff9700 1 -- 172.21.15.148:0/1672621241 <== mds.0 v2:172.21.15.161:6833/3811497457 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56049f393e00 con 0x56049f4e6760 2023-10-10T20:24:15.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5918ff9700 10 client.6519 handle_command_reply: tid=0 2023-10-10T20:24:15.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 client.6519 shutdown 2023-10-10T20:24:15.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 msgr2=0x56049f54c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f592c0093a0 tx=0x7f592c03d000 comp rx=0 tx=0).stop 2023-10-10T20:24:15.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 shutdown_connections 2023-10-10T20:24:15.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:6833/3811497457,v1:172.21.15.161:6835/3811497457] conn(0x56049f4e6760 0x56049f4e6b30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56049f4ead00 0x56049f49cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5918ff9700 0 client.6519 ms_handle_reset on v2:172.21.15.161:6833/3811497457 2023-10-10T20:24:15.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56049f4a0a50 0x56049f54c9a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 --2- 172.21.15.148:0/1672621241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56049f4c9f60 0x56049f49c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 >> 172.21.15.148:0/1672621241 conn(0x56049f4a2370 msgr2=0x56049f4a3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 shutdown_connections 2023-10-10T20:24:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 1 -- 172.21.15.148:0/1672621241 wait complete. 2023-10-10T20:24:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f5949b94b80 20 client.6519 trim_cache size 0 max 16384 2023-10-10T20:24:15.074 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:15.074 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:15.074 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:24:15.074 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 6381, 2023-10-10T20:24:15.074 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:24:15.075 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:24:15.075 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:24:15.075 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.7922359390000002, 2023-10-10T20:24:15.075 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 49, 2023-10-10T20:24:15.075 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 98, 2023-10-10T20:24:15.076 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 98, 2023-10-10T20:24:15.076 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.805043254999999 2023-10-10T20:24:15.076 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:15.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 msgr2=0x7f591c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 0x7f591c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f593c15cb20 tx=0x7f592c017000 comp rx=0 tx=0).stop 2023-10-10T20:24:15.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 msgr2=0x7f593c0b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f592400ec50 tx=0x7f59240091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 shutdown_connections 2023-10-10T20:24:15.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f591c066f30 0x7f591c0693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f593c15d580 0x7f593c154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f593c157870 0x7f593c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 --2- 172.21.15.148:0/3663911512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c15a620 0x7f593c0b2570 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.049+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 >> 172.21.15.148:0/3663911512 conn(0x7f593c068ce0 msgr2=0x7f593c1582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.050+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 shutdown_connections 2023-10-10T20:24:15.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.050+0000 7f594089b700 1 -- 172.21.15.148:0/3663911512 wait complete. 2023-10-10T20:24:15.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:24:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.193+0000 7f1a3041f700 1 Processor -- start 2023-10-10T20:24:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.193+0000 7f1a3041f700 1 -- start start 2023-10-10T20:24:15.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a3041f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a28157870 0x7f1a2815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a3041f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1a28157c40 0x7f1a28158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a3041f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2815cbb0 con 0x7f1a2815a620 2023-10-10T20:24:15.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a3041f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2815bfa0 con 0x7f1a28157870 2023-10-10T20:24:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a3041f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2815c280 con 0x7f1a28157c40 2023-10-10T20:24:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a2e1bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a28157870 0x7f1a2815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a2e1bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a28157870 0x7f1a2815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42978/0 (socket says 172.21.15.148:42978) 2023-10-10T20:24:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.194+0000 7f1a2e1bb700 1 -- 172.21.15.148:0/3423674086 learned_addr learned my addr 172.21.15.148:0/3423674086 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2e1bb700 1 -- 172.21.15.148:0/3423674086 >> v2:172.21.15.161:3300/0 conn(0x7f1a28157c40 msgr2=0x7f1a28158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2e1bb700 1 --2- 172.21.15.148:0/3423674086 >> v2:172.21.15.161:3300/0 conn(0x7f1a28157c40 0x7f1a28158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2e1bb700 1 -- 172.21.15.148:0/3423674086 >> v1:172.21.15.161:6789/0 conn(0x7f1a2815a620 legacy=0x7f1a2815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2e1bb700 1 -- 172.21.15.148:0/3423674086 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2815c560 con 0x7f1a28157870 2023-10-10T20:24:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2e1bb700 1 --2- 172.21.15.148:0/3423674086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157870 0x7f1a2815b6a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806b4e0 tx=0x7f1a24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9662b055f90120c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2d1b9700 1 -- 172.21.15.148:0/3423674086 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400e070 con 0x7f1a28157870 2023-10-10T20:24:15.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2d1b9700 1 -- 172.21.15.148:0/3423674086 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2400ba10 con 0x7f1a28157870 2023-10-10T20:24:15.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a2d1b9700 1 -- 172.21.15.148:0/3423674086 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240092e0 con 0x7f1a28157870 2023-10-10T20:24:15.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a3041f700 1 -- 172.21.15.148:0/3423674086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157870 msgr2=0x7f1a2815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.195+0000 7f1a3041f700 1 --2- 172.21.15.148:0/3423674086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157870 0x7f1a2815b6a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a2806b4e0 tx=0x7f1a24009840 comp rx=0 tx=0).stop 2023-10-10T20:24:15.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 -- 172.21.15.148:0/3423674086 shutdown_connections 2023-10-10T20:24:15.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 --2- 172.21.15.148:0/3423674086 >> v2:172.21.15.161:3300/0 conn(0x7f1a28157c40 0x7f1a28158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 --2- 172.21.15.148:0/3423674086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157870 0x7f1a2815b6a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 -- 172.21.15.148:0/3423674086 >> 172.21.15.148:0/3423674086 conn(0x7f1a28068ce0 msgr2=0x7f1a280690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 -- 172.21.15.148:0/3423674086 shutdown_connections 2023-10-10T20:24:15.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.196+0000 7f1a3041f700 1 -- 172.21.15.148:0/3423674086 wait complete. 2023-10-10T20:24:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 Processor -- start 2023-10-10T20:24:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 -- start start 2023-10-10T20:24:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28157870 0x7f1a280c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 0x7f1a280c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2815dac0 con 0x7f1a28157c40 2023-10-10T20:24:15.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280c6920 con 0x7f1a28157870 2023-10-10T20:24:15.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a3041f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280c6c00 con 0x7f1a2815a620 2023-10-10T20:24:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.197+0000 7f1a2d9ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42986/0 (socket says 172.21.15.148:42986) 2023-10-10T20:24:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 -- 172.21.15.148:0/2305490477 learned_addr learned my addr 172.21.15.148:0/2305490477 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2e9bc700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 0x7f1a280c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2e1bb700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28157870 0x7f1a280c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 -- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 msgr2=0x7f1a280c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 0x7f1a280c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 -- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28157870 msgr2=0x7f1a280c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28157870 0x7f1a280c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 -- 172.21.15.148:0/2305490477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009ae0 con 0x7f1a28157c40 2023-10-10T20:24:15.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2e9bc700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 0x7f1a280c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:15.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a2d9ba700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800b5e0 tx=0x7f1a180090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800f510 con 0x7f1a28157c40 2023-10-10T20:24:15.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1800f670 con 0x7f1a28157c40 2023-10-10T20:24:15.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a280c6ee0 con 0x7f1a28157c40 2023-10-10T20:24:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800fa30 con 0x7f1a28157c40 2023-10-10T20:24:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.198+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a280c9620 con 0x7f1a28157c40 2023-10-10T20:24:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.199+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2815dc30 con 0x7f1a28157c40 2023-10-10T20:24:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.199+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a1801a400 con 0x7f1a28157c40 2023-10-10T20:24:15.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.200+0000 7f1a1e7fc700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 0x7f1a0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.200+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a18084040 con 0x7f1a28157c40 2023-10-10T20:24:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.200+0000 7f1a2e1bb700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 0x7f1a0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:15.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.200+0000 7f1a2e1bb700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 0x7f1a0c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009880 tx=0x7f1a24017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.204+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a18051330 con 0x7f1a28157c40 2023-10-10T20:24:15.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.430+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1a2804c3c0 con 0x7f1a28157c40 2023-10-10T20:24:15.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.430+0000 7f1a1e7fc700 1 -- 172.21.15.148:0/2305490477 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a18054950 con 0x7f1a28157c40 2023-10-10T20:24:15.433 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:24:15.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.432+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 msgr2=0x7f1a0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.432+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 0x7f1a0c0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009880 tx=0x7f1a24017000 comp rx=0 tx=0).stop 2023-10-10T20:24:15.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.432+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 msgr2=0x7f1a280c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:15.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.432+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800b5e0 tx=0x7f1a180090f0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 shutdown_connections 2023-10-10T20:24:15.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1a0c066f30 0x7f1a0c0693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a2815a620 0x7f1a280c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a28157870 0x7f1a280c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 --2- 172.21.15.148:0/2305490477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a28157c40 0x7f1a280c7b40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:15.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 >> 172.21.15.148:0/2305490477 conn(0x7f1a28068ce0 msgr2=0x7f1a280690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:15.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 shutdown_connections 2023-10-10T20:24:15.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:15.433+0000 7f1a3041f700 1 -- 172.21.15.148:0/2305490477 wait complete. 2023-10-10T20:24:15.447 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:24:15.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:24:15.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:24:15.448 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:24:15.448 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:24:15.448 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:24:15.449 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:24:15.449 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:24:15.449 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:24:15.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.450 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:24:15.466 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:24:15.466 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:24:15.466 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:24:15.466 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.467 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:24:15.467 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.467 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:15.467 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:15.468 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:24:15.468 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:15.468 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:15.468 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:24:15.468 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:15.469 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:15.469 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:24:15.469 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:15.469 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:24:15.469 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42256sec preferred_lft 42256sec 2023-10-10T20:24:15.470 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:24:15.470 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.470 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:24:15.470 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:15.470 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:24:15.471 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.471 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:24:15.471 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.471 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:24:15.472 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:24:15.472 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:24:15.474 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.475 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:24:15.475 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:24:15.475 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:24:15.475 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:15.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.476 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:24:15.529 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:24:15.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.531 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:24:15.586 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:24:15.587 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:24:15.587 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:24:15.587 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:24:15.587 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:24:15.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.588 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:24:15.589 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:24:15.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:24:15.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:24:15.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:24:15.592 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:24:15.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.687 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:24:15.701 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:24:15.702 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:24:15.702 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:24:15.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:15.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:24:15.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:24:15.767 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:24:15.822 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:24:15.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:24:15.823 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:24:15.876 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:24:15.876 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:24:15.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:15.957 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:15.957 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:15.976 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:15.976 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:24:15.977 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:24:15.998 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:15.998 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:15.999 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:24:15.999 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:24:15.999 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:24:15.999 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:24:15.999 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:24:16.000 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:24:16.000 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:24:16.000 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:24:16.000 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:24:16.001 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:24:16.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:24:16.067 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:24:16.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.221+0000 7f1c3bfff700 1 Processor -- start 2023-10-10T20:24:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.221+0000 7f1c3bfff700 1 -- start start 2023-10-10T20:24:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3bfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1c3c143520 0x7f1c3c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c3c06a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3bfff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3c0d0870 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3bfff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cdfb0 con 0x7f1c3c143520 2023-10-10T20:24:16.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cfc60 con 0x7f1c3c1438f0 2023-10-10T20:24:16.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3b7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1c3c0ccf00 0x7f1c3c15da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41116/0 (socket says 172.21.15.148:41116) 2023-10-10T20:24:16.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/732523925 learned_addr learned my addr 172.21.15.148:0/732523925 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c327fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c3c06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3affd700 1 --2- 172.21.15.148:0/732523925 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c143520 0x7f1c3c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627532174 0 0) 0x7f1c3c0d0870 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.222+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c28003570 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049598025 0 0) 0x7f1c28003570 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 msgr2=0x7f1c3c06a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:16.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c3c06a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c143520 msgr2=0x7f1c3c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:16.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 --2- 172.21.15.148:0/732523925 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c143520 0x7f1c3c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0cff40 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f1c340017f0 con 0x7f1c3c0ccf00 2023-10-10T20:24:16.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 >> v1:172.21.15.148:6789/0 conn(0x7f1c3c0ccf00 legacy=0x7f1c3c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c28003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c28003fe0 0x7f1c28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c28004d50 0x7f1c280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cdfb0 con 0x7f1c28003fe0 2023-10-10T20:24:16.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cfc60 con 0x7f1c28004d50 2023-10-10T20:24:16.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2800ba90 con 0x7f1c3c1438f0 2023-10-10T20:24:16.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c28003fe0 0x7f1c28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3affd700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c28003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c28004d50 0x7f1c280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 -- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 msgr2=0x7f1c28003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c1438f0 0x7f1c28003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 -- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c28004d50 msgr2=0x7f1c280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c28004d50 0x7f1c280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 -- 172.21.15.148:0/732523925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28003570 con 0x7f1c28003fe0 2023-10-10T20:24:16.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c28004d50 0x7f1c280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:16.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c327fc700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c28003fe0 0x7f1c28004810 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009c00 tx=0x7f1c240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ce35209ccc5c88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:16.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400e6b0 con 0x7f1c28003fe0 2023-10-10T20:24:16.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2400e810 con 0x7f1c28003fe0 2023-10-10T20:24:16.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.223+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/732523925 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400ebd0 con 0x7f1c28003fe0 2023-10-10T20:24:16.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 -- 172.21.15.148:0/732523925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c28003fe0 msgr2=0x7f1c28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/732523925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c28003fe0 0x7f1c28004810 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009c00 tx=0x7f1c240091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/732523925 reap_dead start 2023-10-10T20:24:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 -- 172.21.15.148:0/732523925 shutdown_connections 2023-10-10T20:24:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 -- 172.21.15.148:0/732523925 >> 172.21.15.148:0/732523925 conn(0x7f1c3c069750 msgr2=0x7f1c3c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 -- 172.21.15.148:0/732523925 shutdown_connections 2023-10-10T20:24:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.224+0000 7f1c3bfff700 1 -- 172.21.15.148:0/732523925 wait complete. 2023-10-10T20:24:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.225+0000 7f1c3bfff700 1 Processor -- start 2023-10-10T20:24:16.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.225+0000 7f1c3bfff700 1 -- start start 2023-10-10T20:24:16.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c28003fe0 0x7f1c3c0c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c0c8ca0 0x7f1c3c16dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0ca0f0 con 0x7f1c3c159850 2023-10-10T20:24:16.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cb540 con 0x7f1c3c0c8ca0 2023-10-10T20:24:16.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0c9520 con 0x7f1c28003fe0 2023-10-10T20:24:16.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42998/0 (socket says 172.21.15.148:42998) 2023-10-10T20:24:16.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 -- 172.21.15.148:0/1737619501 learned_addr learned my addr 172.21.15.148:0/1737619501 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:16.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3affd700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c28003fe0 0x7f1c3c0c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 -- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c28003fe0 msgr2=0x7f1c3c0c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c28003fe0 0x7f1c3c0c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c0c8ca0 0x7f1c3c16dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 -- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c0c8ca0 msgr2=0x7f1c3c16dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c0c8ca0 0x7f1c3c16dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 -- 172.21.15.148:0/1737619501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0caef0 con 0x7f1c3c159850 2023-10-10T20:24:16.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c327fc700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1c240091e0 tx=0x7f1c2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:16.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24022070 con 0x7f1c3c159850 2023-10-10T20:24:16.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2400e980 con 0x7f1c3c159850 2023-10-10T20:24:16.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.226+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c0cb1d0 con 0x7f1c3c159850 2023-10-10T20:24:16.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.227+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24027470 con 0x7f1c3c159850 2023-10-10T20:24:16.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.227+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c159cc0 con 0x7f1c3c159850 2023-10-10T20:24:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.227+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c089030 con 0x7f1c3c159850 2023-10-10T20:24:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.228+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c2401c450 con 0x7f1c3c159850 2023-10-10T20:24:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.228+0000 7f1c337fe700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 0x7f1c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.228+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c24086140 con 0x7f1c3c159850 2023-10-10T20:24:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.228+0000 7f1c3affd700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 0x7f1c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:16.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.229+0000 7f1c3affd700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 0x7f1c1c0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0093f0 tx=0x7f1c2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:16.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.233+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c240533b0 con 0x7f1c3c159850 2023-10-10T20:24:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.460+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c3c0cb680 con 0x7f1c3c159850 2023-10-10T20:24:16.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.460+0000 7f1c337fe700 1 -- 172.21.15.148:0/1737619501 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c240569d0 con 0x7f1c3c159850 2023-10-10T20:24:16.463 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:24:16.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 msgr2=0x7f1c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 0x7f1c1c0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0093f0 tx=0x7f1c2c008040 comp rx=0 tx=0).stop 2023-10-10T20:24:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 msgr2=0x7f1c3c15b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1c240091e0 tx=0x7f1c2401a000 comp rx=0 tx=0).stop 2023-10-10T20:24:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 shutdown_connections 2023-10-10T20:24:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c1c066f30 0x7f1c1c0693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c0c8ca0 0x7f1c3c16dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c28003fe0 0x7f1c3c0c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/1737619501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159850 0x7f1c3c15b350 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 >> 172.21.15.148:0/1737619501 conn(0x7f1c3c069750 msgr2=0x7f1c3c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:16.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 shutdown_connections 2023-10-10T20:24:16.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:16.462+0000 7f1c3bfff700 1 -- 172.21.15.148:0/1737619501 wait complete. 2023-10-10T20:24:16.476 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:24:16.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:24:16.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:24:16.477 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:24:16.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:24:16.478 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:24:16.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:24:16.478 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:24:16.479 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:24:16.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.480 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:24:16.498 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:24:16.498 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:24:16.499 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:24:16.499 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.499 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:24:16.499 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.499 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:16.500 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:16.500 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:24:16.500 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:16.500 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:16.500 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:24:16.501 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42251sec preferred_lft 42251sec 2023-10-10T20:24:16.502 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:24:16.502 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.502 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:24:16.502 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:24:16.502 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:24:16.503 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.503 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:24:16.503 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.503 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:24:16.503 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:24:16.504 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:24:16.504 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.504 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:24:16.504 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:24:16.505 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:24:16.505 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:24:16.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.506 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:24:16.567 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:24:16.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.569 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:24:16.627 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:24:16.627 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:24:16.628 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:24:16.628 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:24:16.628 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:24:16.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.629 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:24:16.629 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:24:16.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:24:16.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:24:16.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:24:16.631 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:24:16.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.734 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:24:16.749 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:24:16.749 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:24:16.749 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:24:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:16.750 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:24:16.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:24:16.815 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:24:16.870 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:24:16.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:24:16.871 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:24:16.926 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:24:16.926 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:24:16.927 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:17.006 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:17.007 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:17.025 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:17.026 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:17.026 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:24:17.026 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:24:17.027 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:24:17.027 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:24:17.027 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:24:17.027 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:24:17.046 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:24:17.047 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:24:17.047 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:24:17.047 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:24:17.047 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:24:17.048 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:24:17.049 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:24:17.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:17.050 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:24:17.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.228+0000 7fa7cdec2700 1 Processor -- start 2023-10-10T20:24:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.228+0000 7fa7cdec2700 1 -- start start 2023-10-10T20:24:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7c8157870 0x7fa7c815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c815a620 0x7fa7c815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7c815d580 0x7fa7c8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c815cbb0 con 0x7fa7c8157870 2023-10-10T20:24:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c815bfa0 con 0x7fa7c815d580 2023-10-10T20:24:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7cdec2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c815c280 con 0x7fa7c815a620 2023-10-10T20:24:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7c8157870 0x7fa7c815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7c8157870 0x7fa7c815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43014/0 (socket says 172.21.15.148:43014) 2023-10-10T20:24:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 -- 172.21.15.148:0/2337932995 learned_addr learned my addr 172.21.15.148:0/2337932995 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c7fff700 1 --2- 172.21.15.148:0/2337932995 >> v2:172.21.15.161:3300/0 conn(0x7fa7c815d580 0x7fa7c8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c6ffd700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c815a620 0x7fa7c815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 -- 172.21.15.148:0/2337932995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c815a620 msgr2=0x7fa7c815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:17.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c815a620 0x7fa7c815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 -- 172.21.15.148:0/2337932995 >> v2:172.21.15.161:3300/0 conn(0x7fa7c815d580 msgr2=0x7fa7c8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:17.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/2337932995 >> v2:172.21.15.161:3300/0 conn(0x7fa7c815d580 0x7fa7c8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.229+0000 7fa7c77fe700 1 -- 172.21.15.148:0/2337932995 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c815c560 con 0x7fa7c8157870 2023-10-10T20:24:17.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c8157870 0x7fa7c815b6a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80b03b0 tx=0x7fa7b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461bd5b482687230 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:17.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7c67fc700 1 -- 172.21.15.148:0/2337932995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800e070 con 0x7fa7c8157870 2023-10-10T20:24:17.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7c67fc700 1 -- 172.21.15.148:0/2337932995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b800b970 con 0x7fa7c8157870 2023-10-10T20:24:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7c67fc700 1 -- 172.21.15.148:0/2337932995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80092e0 con 0x7fa7c8157870 2023-10-10T20:24:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 -- 172.21.15.148:0/2337932995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c8157870 msgr2=0x7fa7c815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c8157870 0x7fa7c815b6a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80b03b0 tx=0x7fa7b8009840 comp rx=0 tx=0).stop 2023-10-10T20:24:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 -- 172.21.15.148:0/2337932995 shutdown_connections 2023-10-10T20:24:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/2337932995 >> v2:172.21.15.161:3300/0 conn(0x7fa7c815d580 0x7fa7c8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c8157870 0x7fa7c815b6a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/2337932995 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c815a620 0x7fa7c815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 -- 172.21.15.148:0/2337932995 >> 172.21.15.148:0/2337932995 conn(0x7fa7c8068ce0 msgr2=0x7fa7c80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 -- 172.21.15.148:0/2337932995 shutdown_connections 2023-10-10T20:24:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.230+0000 7fa7cdec2700 1 -- 172.21.15.148:0/2337932995 wait complete. 2023-10-10T20:24:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.231+0000 7fa7cdec2700 1 Processor -- start 2023-10-10T20:24:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 -- start start 2023-10-10T20:24:17.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 0x7fa7c814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7c815a620 0x7fa7c81480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c8148b50 con 0x7fa7c815d580 2023-10-10T20:24:17.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c816a9c0 con 0x7fa7c815a620 2023-10-10T20:24:17.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7cdec2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c816ac60 con 0x7fa7c8157870 2023-10-10T20:24:17.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7c7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7c7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43018/0 (socket says 172.21.15.148:43018) 2023-10-10T20:24:17.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7c7fff700 1 -- 172.21.15.148:0/1386663403 learned_addr learned my addr 172.21.15.148:0/1386663403 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:17.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7c6ffd700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7c815a620 0x7fa7c81480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.232+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 0x7fa7c814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 -- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 msgr2=0x7fa7c814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:17.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 0x7fa7c814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 -- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7c815a620 msgr2=0x7fa7c81480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:17.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7c815a620 0x7fa7c81480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 -- 172.21.15.148:0/1386663403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8009ae0 con 0x7fa7c815d580 2023-10-10T20:24:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 0x7fa7c814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7c7fff700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e900 tx=0x7fa7c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0013680 con 0x7fa7c815d580 2023-10-10T20:24:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0013800 con 0x7fa7c815d580 2023-10-10T20:24:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c816af40 con 0x7fa7c815d580 2023-10-10T20:24:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0013bc0 con 0x7fa7c815d580 2023-10-10T20:24:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.233+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c816b580 con 0x7fa7c815d580 2023-10-10T20:24:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.234+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c816b870 con 0x7fa7c815d580 2023-10-10T20:24:17.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.234+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7c00123d0 con 0x7fa7c815d580 2023-10-10T20:24:17.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.235+0000 7fa7af7fe700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:17.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.235+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7c00845e0 con 0x7fa7c815d580 2023-10-10T20:24:17.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.235+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.235+0000 7fa7c77fe700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009880 tx=0x7fa7b8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.239+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7c0051850 con 0x7fa7c815d580 2023-10-10T20:24:17.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.467+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7c804c3c0 con 0x7fa7c815d580 2023-10-10T20:24:17.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.468+0000 7fa7af7fe700 1 -- 172.21.15.148:0/1386663403 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fa7c0054e70 con 0x7fa7c815d580 2023-10-10T20:24:17.471 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:17.471 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:17.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 msgr2=0x7fa7b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009880 tx=0x7fa7b8018000 comp rx=0 tx=0).stop 2023-10-10T20:24:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 msgr2=0x7fa7c8148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e900 tx=0x7fa7c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 shutdown_connections 2023-10-10T20:24:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c815d580 0x7fa7c8148610 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7c8157870 0x7fa7c814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 --2- 172.21.15.148:0/1386663403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7c815a620 0x7fa7c81480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 >> 172.21.15.148:0/1386663403 conn(0x7fa7c8068ce0 msgr2=0x7fa7c81594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:17.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 shutdown_connections 2023-10-10T20:24:17.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:17.470+0000 7fa7cdec2700 1 -- 172.21.15.148:0/1386663403 wait complete. 2023-10-10T20:24:17.475 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:17.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:24:17.619 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3505853769999998,"mdsmap_epoch":49,"osdmap_epoch":98,"osdmap_epoch_barrier":98,"uptime":13.363392715} 2023-10-10T20:24:17.619 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 6381, "mdsmap_epoch": 49, "osdmap_epoch": 98, "osdmap_epoch_barrier": 98, "rank_uptime": 8.350585377, "state": "up:active", "uptime": 13.363392715, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:24:17.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:24:17.751 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:24:17.751 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:24:17.752 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-10T20:24:17.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:17.753 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-10T20:24:19.093 INFO:teuthology.orchestra.run.smithi148.stderr:265+0 records in 2023-10-10T20:24:19.093 INFO:teuthology.orchestra.run.smithi148.stderr:265+0 records out 2023-10-10T20:24:19.093 INFO:teuthology.orchestra.run.smithi148.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.32246 s, 210 MB/s 2023-10-10T20:24:19.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:19.094 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-10T20:24:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:292+0 records in 2023-10-10T20:24:20.596 INFO:teuthology.orchestra.run.smithi148.stderr:292+0 records out 2023-10-10T20:24:20.596 INFO:teuthology.orchestra.run.smithi148.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.48405 s, 206 MB/s 2023-10-10T20:24:20.596 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-10T20:24:20.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.709+0000 7f62c59b8700 1 Processor -- start 2023-10-10T20:24:20.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.709+0000 7f62c59b8700 1 -- start start 2023-10-10T20:24:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62c0068b80 0x7f62c0068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62c00cbcc0 0x7f62c00cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c0069530 con 0x7f62c00cbcc0 2023-10-10T20:24:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c00b0970 con 0x7f62c0068b80 2023-10-10T20:24:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c59b8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c00b0f30 con 0x7f62c015a010 2023-10-10T20:24:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c51b7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62c00cbcc0 0x7f62c00cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c51b7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62c00cbcc0 0x7f62c00cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43030/0 (socket says 172.21.15.148:43030) 2023-10-10T20:24:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c51b7700 1 -- 172.21.15.148:0/2562651515 learned_addr learned my addr 172.21.15.148:0/2562651515 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62bffff700 1 --2- 172.21.15.148:0/2562651515 >> v2:172.21.15.161:3300/0 conn(0x7f62c0068b80 0x7f62c0068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.710+0000 7f62c49b6700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 -- 172.21.15.148:0/2562651515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 msgr2=0x7f62c00b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 -- 172.21.15.148:0/2562651515 >> v2:172.21.15.161:3300/0 conn(0x7f62c0068b80 msgr2=0x7f62c0068ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 --2- 172.21.15.148:0/2562651515 >> v2:172.21.15.161:3300/0 conn(0x7f62c0068b80 0x7f62c0068ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 -- 172.21.15.148:0/2562651515 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c015b1f0 con 0x7f62c00cbcc0 2023-10-10T20:24:20.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c51b7700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c00cbcc0 0x7f62c00cc140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f62c00689b0 tx=0x7f62b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b52ceff85de5b19 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:20.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2562651515 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b800b640 con 0x7f62c00cbcc0 2023-10-10T20:24:20.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2562651515 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b800b7a0 con 0x7f62c00cbcc0 2023-10-10T20:24:20.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62bf7fe700 1 -- 172.21.15.148:0/2562651515 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b800bb60 con 0x7f62c00cbcc0 2023-10-10T20:24:20.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 -- 172.21.15.148:0/2562651515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c00cbcc0 msgr2=0x7f62c00cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:20.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c00cbcc0 0x7f62c00cc140 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f62c00689b0 tx=0x7f62b8009250 comp rx=0 tx=0).stop 2023-10-10T20:24:20.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 -- 172.21.15.148:0/2562651515 shutdown_connections 2023-10-10T20:24:20.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c00cbcc0 0x7f62c00cc140 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 --2- 172.21.15.148:0/2562651515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 --2- 172.21.15.148:0/2562651515 >> v2:172.21.15.161:3300/0 conn(0x7f62c0068b80 0x7f62c0068ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 -- 172.21.15.148:0/2562651515 >> 172.21.15.148:0/2562651515 conn(0x7f62c00cd640 msgr2=0x7f62c00cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 -- 172.21.15.148:0/2562651515 shutdown_connections 2023-10-10T20:24:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.711+0000 7f62c59b8700 1 -- 172.21.15.148:0/2562651515 wait complete. 2023-10-10T20:24:20.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.712+0000 7f62c59b8700 1 Processor -- start 2023-10-10T20:24:20.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 -- start start 2023-10-10T20:24:20.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c00cbcc0 0x7f62c016ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c00a8690 con 0x7f62c0068b80 2023-10-10T20:24:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c016b1b0 con 0x7f62c00cbcc0 2023-10-10T20:24:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.713+0000 7f62c59b8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c00b0610 con 0x7f62c015a010 2023-10-10T20:24:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43032/0 (socket says 172.21.15.148:43032) 2023-10-10T20:24:20.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 -- 172.21.15.148:0/3720255014 learned_addr learned my addr 172.21.15.148:0/3720255014 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:20.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62bffff700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c00cbcc0 0x7f62c016ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c51b7700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 -- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 msgr2=0x7f62c00b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:20.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 -- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c00cbcc0 msgr2=0x7f62c016ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:20.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c00cbcc0 0x7f62c016ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:20.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 -- 172.21.15.148:0/3720255014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80094f0 con 0x7f62c0068b80 2023-10-10T20:24:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c49b6700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009f20 tx=0x7f62b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b000cb70 con 0x7f62c0068b80 2023-10-10T20:24:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b000f040 con 0x7f62c0068b80 2023-10-10T20:24:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c00a7a80 con 0x7f62c0068b80 2023-10-10T20:24:20.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0014480 con 0x7f62c0068b80 2023-10-10T20:24:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.714+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c00a8040 con 0x7f62c0068b80 2023-10-10T20:24:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.715+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c00897e0 con 0x7f62c0068b80 2023-10-10T20:24:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.716+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62b0014830 con 0x7f62c0068b80 2023-10-10T20:24:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.716+0000 7f62bcff9700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 0x7f62ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:20.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.716+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62b0084b90 con 0x7f62c0068b80 2023-10-10T20:24:20.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.716+0000 7f62bffff700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 0x7f62ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:20.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.716+0000 7f62bffff700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 0x7f62ac0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f62c00b6a20 tx=0x7f62b40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:20.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.720+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62b0051e00 con 0x7f62c0068b80 2023-10-10T20:24:20.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.902+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f62c015b4d0 con 0x7f62c0068b80 2023-10-10T20:24:20.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.903+0000 7f62bcff9700 1 -- 172.21.15.148:0/3720255014 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v98) v1 ==== 81+0+32311 (secure 0 0 0) 0x7f62b0018090 con 0x7f62c0068b80 2023-10-10T20:24:20.905 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:20.906 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:20.906 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 98, 2023-10-10T20:24:20.906 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:20.906 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:20.907 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:10.239239+0000", 2023-10-10T20:24:20.907 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:20.907 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:20.907 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:20.907 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:20.908 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:20.908 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:20.908 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:20.908 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:20.908 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:20.909 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:20.909 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:20.909 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:20.909 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:20.909 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:20.910 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:20.911 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:20.911 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.911 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:20.911 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:20.911 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:20.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:20.912 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:20.912 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:20.912 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:20.912 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:20.913 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:20.913 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:20.913 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:20.913 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:20.914 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:20.914 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:20.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:20.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:20.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:20.915 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:20.916 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:20.916 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:20.916 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:20.916 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:20.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.917 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "90", 2023-10-10T20:24:20.917 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:20.917 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:20.917 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:20.917 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:20.918 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:20.918 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:20.918 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:20.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:20.918 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:20.919 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:20.919 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:20.919 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:20.919 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:20.919 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:20.920 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:20.920 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:20.920 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:20.920 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:20.920 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:20.921 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:20.921 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:20.921 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:20.921 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:20.922 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:20.922 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:20.922 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:20.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.922 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:20.924 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:20.926 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:20.927 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:20.928 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:20.928 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:20.928 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:20.928 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:20.928 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:20.929 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:20.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:20.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:20.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:20.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:20.930 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:20.930 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:20.930 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:20.930 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:20.931 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:20.931 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:20.931 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:20.931 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:20.931 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:20.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.932 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "97", 2023-10-10T20:24:20.932 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:20.932 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:20.932 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:20.933 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:20.934 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:20.935 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:20.936 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.937 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:20.938 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:20.939 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:20.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:20.942 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:20.943 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:20.944 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:20.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:20.946 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:20.947 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "98", 2023-10-10T20:24:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:20.948 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:20.949 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:20.950 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:20.951 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:20.952 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.953 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:20.954 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:20.955 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:20.956 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.957 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:20.958 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:20.959 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:20.960 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.962 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.963 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.964 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.966 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.967 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.985 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.986 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.987 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:20.988 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:20.989 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:20.990 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:20.991 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:20.992 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.993 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:20.994 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:20.995 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:20.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:20.997 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:20.998 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:20.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:21.000 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.002 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:21.004 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.005 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:21.006 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:21.007 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:21.008 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:21.009 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:21.010 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:21.011 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:21.013 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.014 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:21.015 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:21.016 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.018 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.019 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:21.020 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.061 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:21.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.061 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:21.062 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.062 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.062 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:21.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.063 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.063 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:21.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:21.065 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:21.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.066 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.066 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:21.068 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:21.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.069 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.069 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:21.069 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.070 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.071 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:21.071 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:21.072 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:21.072 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:21.072 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:21.072 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:21.072 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:21.073 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:21.073 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:21.073 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:21.073 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:21.074 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:21.074 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:21.074 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.076 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:21.076 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.076 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.076 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:21.078 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.078 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:21.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:21.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.082 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:21.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:21.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:21.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:21.083 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.083 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.084 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:21.085 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:21.085 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:21.086 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:21.086 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:21.086 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:21.086 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:21.086 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:21.087 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.088 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.088 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.088 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.088 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.088 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.089 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.090 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.090 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.090 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.090 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:21.090 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.091 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:21.091 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.091 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.092 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.092 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.092 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.092 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:21.092 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.093 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:21.093 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.094 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.094 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.094 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.096 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.097 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:21.097 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.097 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:21.098 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.098 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.098 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.098 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.098 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:21.099 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.100 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.100 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.100 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.100 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.100 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:21.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:21.101 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:21.102 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:21.102 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:21.102 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:21.102 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:21.103 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:21.103 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:21.103 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:21.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.103 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:21.104 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:21.105 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:21.105 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:21.105 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:21.105 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:21.105 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:21.106 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:21.107 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:21.107 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:21.107 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:21.107 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:21.107 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:21.108 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:21.108 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:21.108 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:21.108 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:21.109 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:21.109 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 msgr2=0x7f62ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 0x7f62ac0693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f62c00b6a20 tx=0x7f62b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 msgr2=0x7f62c00a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:21.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009f20 tx=0x7f62b000b040 comp rx=0 tx=0).stop 2023-10-10T20:24:21.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 shutdown_connections 2023-10-10T20:24:21.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62ac066f30 0x7f62ac0693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:21.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62c015a010 0x7f62c00b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:21.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0068b80 0x7f62c00a74a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:21.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 --2- 172.21.15.148:0/3720255014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c00cbcc0 0x7f62c016ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:21.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 >> 172.21.15.148:0/3720255014 conn(0x7f62c00cd640 msgr2=0x7f62c00b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:21.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 shutdown_connections 2023-10-10T20:24:21.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:20.905+0000 7f62c59b8700 1 -- 172.21.15.148:0/3720255014 wait complete. 2023-10-10T20:24:21.114 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:24:22.522 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:24:22.519+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-10-10T20:24:23.022 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:24:23.025 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:24:26.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:26.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.238+0000 7f30e9dfe700 1 Processor -- start 2023-10-10T20:24:26.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.238+0000 7f30e9dfe700 1 -- start start 2023-10-10T20:24:26.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.239+0000 7f30e9dfe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 0x7f30e415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.239+0000 7f30e9dfe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.239+0000 7f30e9dfe700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e415cbb0 con 0x7f30e415a620 2023-10-10T20:24:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.239+0000 7f30e9dfe700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e415bfa0 con 0x7f30e4157c40 2023-10-10T20:24:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.239+0000 7f30e9dfe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e415c280 con 0x7f30e4157870 2023-10-10T20:24:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e3fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e3fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:49454/0 (socket says 172.21.15.148:49454) 2023-10-10T20:24:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 0x7f30e415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e2ffd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f30e415a620 0x7f30e415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:45454/0 (socket says 172.21.15.148:45454) 2023-10-10T20:24:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e3fff700 1 -- 172.21.15.148:0/2939864442 learned_addr learned my addr 172.21.15.148:0/2939864442 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 -- 172.21.15.148:0/2939864442 >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 msgr2=0x7f30e4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 --2- 172.21.15.148:0/2939864442 >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 -- 172.21.15.148:0/2939864442 >> v1:172.21.15.161:6789/0 conn(0x7f30e415a620 legacy=0x7f30e415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 -- 172.21.15.148:0/2939864442 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e415c560 con 0x7f30e4157870 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e3fff700 1 --2- 172.21.15.148:0/2939864442 >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e37fe700 1 --2- 172.21.15.148:0/2939864442 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 0x7f30e415b6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009e60 tx=0x7f30d4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d21827af1c2ac server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e27fc700 1 -- 172.21.15.148:0/2939864442 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400e070 con 0x7f30e4157870 2023-10-10T20:24:26.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e27fc700 1 -- 172.21.15.148:0/2939864442 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400bb60 con 0x7f30e4157870 2023-10-10T20:24:26.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.240+0000 7f30e27fc700 1 -- 172.21.15.148:0/2939864442 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4012430 con 0x7f30e4157870 2023-10-10T20:24:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 -- 172.21.15.148:0/2939864442 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 msgr2=0x7f30e415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/2939864442 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 0x7f30e415b6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009e60 tx=0x7f30d4009780 comp rx=0 tx=0).stop 2023-10-10T20:24:26.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 -- 172.21.15.148:0/2939864442 shutdown_connections 2023-10-10T20:24:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/2939864442 >> v2:172.21.15.161:3300/0 conn(0x7f30e4157c40 0x7f30e4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/2939864442 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e4157870 0x7f30e415b6a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 -- 172.21.15.148:0/2939864442 >> 172.21.15.148:0/2939864442 conn(0x7f30e4068ce0 msgr2=0x7f30e40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 -- 172.21.15.148:0/2939864442 shutdown_connections 2023-10-10T20:24:26.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.241+0000 7f30e9dfe700 1 -- 172.21.15.148:0/2939864442 wait complete. 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.242+0000 7f30e9dfe700 1 Processor -- start 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.242+0000 7f30e9dfe700 1 -- start start 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30e4157c40 0x7f30e41492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e415a620 0x7f30e4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4149d50 con 0x7f30e4157870 2023-10-10T20:24:26.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e414a030 con 0x7f30e4157c40 2023-10-10T20:24:26.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e9dfe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e414a310 con 0x7f30e415a620 2023-10-10T20:24:26.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40090/0 (socket says 172.21.15.148:40090) 2023-10-10T20:24:26.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e37fe700 1 -- 172.21.15.148:0/3009135116 learned_addr learned my addr 172.21.15.148:0/3009135116 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:26.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e2ffd700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30e4157c40 0x7f30e41492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.243+0000 7f30e3fff700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e415a620 0x7f30e4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 -- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e415a620 msgr2=0x7f30e4148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e415a620 0x7f30e4148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 -- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30e4157c40 msgr2=0x7f30e41492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30e4157c40 0x7f30e41492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0094f0 con 0x7f30e4157870 2023-10-10T20:24:26.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e37fe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f30d40097c0 tx=0x7f30d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:26.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4012da0 con 0x7f30e4157870 2023-10-10T20:24:26.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d4027470 con 0x7f30e4157870 2023-10-10T20:24:26.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d4009a20 con 0x7f30e4157870 2023-10-10T20:24:26.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4027830 con 0x7f30e4157870 2023-10-10T20:24:26.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.244+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e414a8d0 con 0x7f30e4157870 2023-10-10T20:24:26.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.245+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e414aaf0 con 0x7f30e4157870 2023-10-10T20:24:26.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.245+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30d4024070 con 0x7f30e4157870 2023-10-10T20:24:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.246+0000 7f30cb7fe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.246+0000 7f30e2ffd700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.246+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f30d4086c40 con 0x7f30e4157870 2023-10-10T20:24:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.246+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f30d006ea50 con 0x7f30e4157870 2023-10-10T20:24:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.246+0000 7f30e2ffd700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 0x7f30d00693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007fd0 tx=0x7f30d8007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:26.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.250+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30d402c060 con 0x7f30e4157870 2023-10-10T20:24:26.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.436+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f30e404c3c0 con 0x7f30e4157870 2023-10-10T20:24:26.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.437+0000 7f30cb7fe700 1 -- 172.21.15.148:0/3009135116 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+32479 (secure 0 0 0) 0x7f30d4053eb0 con 0x7f30e4157870 2023-10-10T20:24:26.439 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:26.440 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:26.440 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 101, 2023-10-10T20:24:26.440 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:26.440 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:26.440 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:24.532180+0000", 2023-10-10T20:24:26.441 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:26.441 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:26.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:26.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:26.441 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:26.442 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:26.442 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:26.442 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:26.442 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:26.442 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:26.443 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:26.444 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:26.444 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:26.444 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:26.444 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:26.444 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:26.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.445 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:26.445 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:26.445 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:26.445 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:26.446 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:26.447 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:26.447 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:26.447 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:26.447 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:26.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:26.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:26.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:26.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:26.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:26.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:26.449 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:26.449 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:26.449 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:26.449 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:26.449 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:26.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:26.451 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:26.452 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:26.452 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:26.452 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:26.452 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:26.452 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:26.453 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:26.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:26.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:26.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:26.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:26.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:26.455 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:26.456 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:26.456 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:26.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.456 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:26.456 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:26.457 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:26.457 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:26.457 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:26.457 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:26.457 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:26.458 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:26.458 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:26.458 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:26.458 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:26.458 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:26.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:26.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:26.460 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:26.461 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:26.462 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:26.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:26.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:26.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:26.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:26.463 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:26.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:26.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:26.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:26.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:26.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:26.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:26.465 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:26.465 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:26.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:26.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:26.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:26.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:26.466 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:26.466 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:26.467 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:26.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:26.470 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:26.470 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:26.470 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:26.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:26.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.471 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:26.471 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:26.471 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:26.471 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:26.471 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:26.472 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:26.473 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:26.473 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:26.473 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:26.473 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:26.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.474 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:26.474 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:26.474 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:26.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:26.476 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:26.476 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:26.476 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:26.476 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:26.476 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:26.477 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:26.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:26.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:26.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:26.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:26.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:26.479 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:26.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:26.481 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:26.481 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:26.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:26.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:26.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:26.482 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:26.483 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:26.483 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:26.483 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:26.483 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:26.483 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:26.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:26.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:26.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:26.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:26.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:26.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:26.486 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:26.486 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:26.486 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:26.486 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:26.486 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:26.487 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:26.488 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:26.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.489 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:26.489 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.490 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.491 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.491 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.491 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.491 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.491 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.492 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.492 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.492 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.492 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.495 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:26.495 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.496 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.497 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.497 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.497 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:26.498 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.499 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:26.499 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.499 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.500 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.500 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:26.501 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.501 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.502 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.503 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:26.504 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.504 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:26.505 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:26.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.505 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.505 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:26.506 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:26.506 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:26.506 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:26.506 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.506 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.507 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:26.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.579 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.579 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.579 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.580 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:26.580 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:24:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:26.641 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 msgr2=0x7f30d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 0x7f30d00693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007fd0 tx=0x7f30d8007d70 comp rx=0 tx=0).stop 2023-10-10T20:24:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 msgr2=0x7f30e414f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f30d40097c0 tx=0x7f30d4019000 comp rx=0 tx=0).stop 2023-10-10T20:24:26.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 shutdown_connections 2023-10-10T20:24:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f30d0066f30 0x7f30d00693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30e415a620 0x7f30e4148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30e4157870 0x7f30e414f610 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 --2- 172.21.15.148:0/3009135116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30e4157c40 0x7f30e41492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:26.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 >> 172.21.15.148:0/3009135116 conn(0x7f30e4068ce0 msgr2=0x7f30e40b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 shutdown_connections 2023-10-10T20:24:26.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:26.439+0000 7f30e9dfe700 1 -- 172.21.15.148:0/3009135116 wait complete. 2023-10-10T20:24:26.645 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-10T20:24:26.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:26.646 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-10T20:24:26.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:24:26.895 INFO:teuthology.orchestra.run.smithi148.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-10T20:24:26.896 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:24:26.896 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:24:26.896 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.227051 s, 231 MB/s 2023-10-10T20:24:26.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:27.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.009+0000 7fe226a4b700 1 Processor -- start 2023-10-10T20:24:27.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.009+0000 7fe226a4b700 1 -- start start 2023-10-10T20:24:27.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe226a4b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe220157870 0x7fe22015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe226a4b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe22015a620 0x7fe22015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe226a4b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22015cbb0 con 0x7fe22015d580 2023-10-10T20:24:27.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe226a4b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22015bfa0 con 0x7fe220157870 2023-10-10T20:24:27.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe226a4b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22015c280 con 0x7fe22015a620 2023-10-10T20:24:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe21ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe220157870 0x7fe22015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe21ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe220157870 0x7fe22015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40100/0 (socket says 172.21.15.148:40100) 2023-10-10T20:24:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe21ffff700 1 -- 172.21.15.148:0/151810249 learned_addr learned my addr 172.21.15.148:0/151810249 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.010+0000 7fe21f7fe700 1 --2- 172.21.15.148:0/151810249 >> v2:172.21.15.161:3300/0 conn(0x7fe22015a620 0x7fe22015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21ffff700 1 -- 172.21.15.148:0/151810249 >> v2:172.21.15.161:3300/0 conn(0x7fe22015a620 msgr2=0x7fe22015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21ffff700 1 --2- 172.21.15.148:0/151810249 >> v2:172.21.15.161:3300/0 conn(0x7fe22015a620 0x7fe22015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21ffff700 1 -- 172.21.15.148:0/151810249 >> v1:172.21.15.148:6789/0 conn(0x7fe22015d580 legacy=0x7fe220158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21ffff700 1 -- 172.21.15.148:0/151810249 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22015c560 con 0x7fe220157870 2023-10-10T20:24:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21effd700 1 -- 172.21.15.148:0/151810249 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513299614 0 0) 0x7fe22015cbb0 con 0x7fe22015d580 2023-10-10T20:24:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21ffff700 1 --2- 172.21.15.148:0/151810249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe22015b6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe22015cb20 tx=0x7fe210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab16d4e823b2e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21effd700 1 -- 172.21.15.148:0/151810249 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210009e70 con 0x7fe220157870 2023-10-10T20:24:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21effd700 1 -- 172.21.15.148:0/151810249 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21000b860 con 0x7fe220157870 2023-10-10T20:24:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.011+0000 7fe21effd700 1 -- 172.21.15.148:0/151810249 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21000bc00 con 0x7fe220157870 2023-10-10T20:24:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 -- 172.21.15.148:0/151810249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 msgr2=0x7fe22015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 --2- 172.21.15.148:0/151810249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe22015b6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe22015cb20 tx=0x7fe210009250 comp rx=0 tx=0).stop 2023-10-10T20:24:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 -- 172.21.15.148:0/151810249 shutdown_connections 2023-10-10T20:24:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 --2- 172.21.15.148:0/151810249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe22015b6a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 --2- 172.21.15.148:0/151810249 >> v2:172.21.15.161:3300/0 conn(0x7fe22015a620 0x7fe22015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 -- 172.21.15.148:0/151810249 >> 172.21.15.148:0/151810249 conn(0x7fe220068ce0 msgr2=0x7fe2200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:27.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 -- 172.21.15.148:0/151810249 shutdown_connections 2023-10-10T20:24:27.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.012+0000 7fe226a4b700 1 -- 172.21.15.148:0/151810249 wait complete. 2023-10-10T20:24:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.013+0000 7fe226a4b700 1 Processor -- start 2023-10-10T20:24:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.013+0000 7fe226a4b700 1 -- start start 2023-10-10T20:24:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 0x7fe2200b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22015d580 0x7fe22014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220155ea0 con 0x7fe220157870 2023-10-10T20:24:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200cf6d0 con 0x7fe22015d580 2023-10-10T20:24:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe220154180 con 0x7fe22015a620 2023-10-10T20:24:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40108/0 (socket says 172.21.15.148:40108) 2023-10-10T20:24:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 -- 172.21.15.148:0/2760126452 learned_addr learned my addr 172.21.15.148:0/2760126452 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21f7fe700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 0x7fe2200b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe224fe8700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22015d580 0x7fe22014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 -- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 msgr2=0x7fe2200b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 0x7fe2200b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 -- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22015d580 msgr2=0x7fe22014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22015d580 0x7fe22014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100094f0 con 0x7fe220157870 2023-10-10T20:24:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21f7fe700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 0x7fe2200b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe21ffff700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe21000b3c0 tx=0x7fe210017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210000d10 con 0x7fe220157870 2023-10-10T20:24:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21000bbe0 con 0x7fe220157870 2023-10-10T20:24:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22016b8b0 con 0x7fe220157870 2023-10-10T20:24:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210011670 con 0x7fe220157870 2023-10-10T20:24:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.014+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2200cf940 con 0x7fe220157870 2023-10-10T20:24:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.015+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22004c3c0 con 0x7fe220157870 2023-10-10T20:24:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe210011a60 con 0x7fe220157870 2023-10-10T20:24:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe207fff700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 0x7fe20c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2100858b0 con 0x7fe220157870 2023-10-10T20:24:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe21f7fe700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 0x7fe20c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7fe20c06ea50 con 0x7fe220157870 2023-10-10T20:24:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.016+0000 7fe21f7fe700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 0x7fe20c0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cf450 tx=0x7fe214008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.020+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe210052ba0 con 0x7fe220157870 2023-10-10T20:24:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.206+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe22016b8b0 con 0x7fe220157870 2023-10-10T20:24:27.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.206+0000 7fe207fff700 1 -- 172.21.15.148:0/2760126452 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+32479 (secure 0 0 0) 0x7fe22016b8b0 con 0x7fe220157870 2023-10-10T20:24:27.209 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:27.209 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:27.209 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 101, 2023-10-10T20:24:27.210 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:27.210 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:27.210 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:24.532180+0000", 2023-10-10T20:24:27.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:27.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:27.212 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:27.212 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:27.212 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:27.212 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:27.212 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:27.213 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:27.213 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:27.213 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:27.213 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:27.213 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:27.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:27.215 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:27.216 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:27.216 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:27.216 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:27.216 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:27.217 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:27.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:27.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:27.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:27.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:27.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:27.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:27.219 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:27.219 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:27.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:27.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:27.220 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:27.220 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:27.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:27.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:27.221 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:27.222 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:27.222 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:27.222 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:27.222 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:27.222 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:27.223 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:27.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:27.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:27.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:27.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:27.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:27.225 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:27.225 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:27.225 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:27.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.225 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:27.226 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:27.226 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:27.226 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:27.226 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:27.226 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:27.227 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:27.228 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:27.228 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:27.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.228 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:27.228 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:27.229 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:27.230 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:27.230 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:27.230 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:27.230 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:27.230 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:27.232 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:27.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:27.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:27.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:27.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:27.233 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:27.234 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:27.234 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:27.234 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:27.234 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:27.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:27.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:27.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:27.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:27.235 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:27.236 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:27.237 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:27.237 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:27.237 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:27.237 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:27.237 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:27.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:27.238 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:27.238 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:27.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:27.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:27.239 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:27.239 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:27.239 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:27.239 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:27.239 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:27.240 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:27.241 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:27.241 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:27.241 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:27.241 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:27.241 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:27.242 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:27.242 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:27.242 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:27.242 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:27.242 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:27.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.244 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.244 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:27.244 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:27.245 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:27.245 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:27.245 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:27.245 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:27.245 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:27.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:27.247 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:27.247 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:27.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:27.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:27.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:27.248 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:27.248 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:27.248 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:27.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:27.248 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:27.249 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:27.249 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:27.249 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:27.249 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:27.249 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:27.251 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:27.252 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:27.252 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:27.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:27.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:27.252 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:27.253 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:27.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:27.253 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:27.253 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:27.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:27.254 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:27.254 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:27.254 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:27.254 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:27.254 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:27.255 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:27.256 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:27.256 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:27.256 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:27.256 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:27.256 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:27.257 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:27.258 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:27.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.258 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:27.258 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:27.258 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:27.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.259 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:27.259 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:27.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.260 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:27.260 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.260 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.260 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.261 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.261 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.261 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.261 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.261 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.262 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.262 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.262 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.262 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.262 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.264 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.264 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:27.264 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.264 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.265 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.265 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.266 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:27.266 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.268 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.268 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.268 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.268 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.269 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:27.270 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.270 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.270 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.270 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.271 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.271 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.271 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:27.271 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:27.272 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:27.273 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.275 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.276 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.277 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.278 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.278 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:27.278 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.280 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.280 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.282 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.283 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:27.284 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.284 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.285 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.286 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.286 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:27.286 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.287 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.287 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:27.287 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:27.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.288 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.288 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:27.288 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:27.288 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:27.289 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:27.289 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.289 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.289 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:27.289 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.290 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.291 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.291 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.291 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.292 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.292 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.292 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.292 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.292 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:27.293 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.294 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.294 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:27.294 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.295 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.297 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.297 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:27.297 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.297 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.297 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.298 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.298 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.298 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.299 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:27.300 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.300 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.300 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.302 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.302 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:27.302 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:27.302 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:27.303 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:27.304 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:27.304 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.304 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.304 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:27.304 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.305 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.306 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.307 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.307 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.307 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.307 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.307 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:27.308 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.309 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:27.309 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.311 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.311 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:27.311 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.321 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.322 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.323 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.325 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.328 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.329 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:27.330 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.332 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:27.332 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.332 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.333 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.333 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.333 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.334 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:27.334 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.335 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.336 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:27.337 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.337 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.337 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:27.338 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.338 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.338 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.338 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.338 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.339 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.339 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.339 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.339 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.340 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.340 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.340 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.341 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.342 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:27.342 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.343 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.343 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.343 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.343 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:27.344 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.344 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.345 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.346 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.346 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:27.346 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.347 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.347 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.347 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:27.347 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.349 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:27.349 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.350 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:27.351 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:27.351 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:27.351 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:27.351 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.351 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.352 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.352 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.352 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.353 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.354 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.354 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.354 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.354 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.354 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.355 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.355 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:27.356 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.356 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.357 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.358 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:27.358 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.359 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:27.359 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.360 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.360 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.360 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.363 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.363 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.363 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:27.364 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:27.365 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.365 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.365 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:27.365 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:27.366 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:27.366 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:27.366 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.366 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.366 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:27.367 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:27.368 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:27.368 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:27.368 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:27.368 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:27.368 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:27.369 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:27.369 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:27.369 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:27.369 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:27.369 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:27.370 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.371 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:27.371 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.372 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.373 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:27.373 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.374 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.374 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:27.374 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.374 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.374 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.375 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:27.375 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.375 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.375 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.376 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:27.376 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.376 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.378 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:27.378 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:27.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:27.378 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:27.379 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:27.380 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:27.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.380 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.380 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:27.381 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:27.381 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:27.381 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:27.381 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:27.381 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:24:27.382 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:27.382 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:27.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:27.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.382 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:27.383 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:27.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.383 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:27.383 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.383 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.384 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.384 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.384 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.384 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:27.384 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.385 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.387 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.387 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:27.387 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.387 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.387 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.389 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:27.389 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.389 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.389 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.389 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.390 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.390 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:27.390 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.391 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.392 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:27.392 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.392 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:27.393 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.393 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.393 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.393 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.393 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:27.394 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.397 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.397 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.398 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.398 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.398 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:27.398 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.399 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:27.399 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:27.399 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:27.399 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:27.399 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:27.400 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:27.400 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:27.400 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:27.400 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:27.400 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:27.401 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:27.402 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:27.404 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:27.405 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:27.406 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:27.406 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:27.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.208+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 msgr2=0x7fe20c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.208+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 0x7fe20c0693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cf450 tx=0x7fe214008040 comp rx=0 tx=0).stop 2023-10-10T20:24:27.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.208+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 msgr2=0x7fe220153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.208+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe21000b3c0 tx=0x7fe210017000 comp rx=0 tx=0).stop 2023-10-10T20:24:27.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 shutdown_connections 2023-10-10T20:24:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe20c066f30 0x7fe20c0693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe22015d580 0x7fe22014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe220157870 0x7fe220153c40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 --2- 172.21.15.148:0/2760126452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe22015a620 0x7fe2200b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 >> 172.21.15.148:0/2760126452 conn(0x7fe220068ce0 msgr2=0x7fe220158220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:27.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 shutdown_connections 2023-10-10T20:24:27.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.209+0000 7fe226a4b700 1 -- 172.21.15.148:0/2760126452 wait complete. 2023-10-10T20:24:27.411 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:27.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.524+0000 7f863e473700 1 Processor -- start 2023-10-10T20:24:27.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.524+0000 7f863e473700 1 -- start start 2023-10-10T20:24:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863e473700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8638157870 0x7f863815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863e473700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863815cbb0 con 0x7f8638157c40 2023-10-10T20:24:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863e473700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863815bfa0 con 0x7f863815a620 2023-10-10T20:24:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863e473700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863815c280 con 0x7f8638157870 2023-10-10T20:24:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863ca10700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8638157c40 0x7f8638158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41136/0 (socket says 172.21.15.148:41136) 2023-10-10T20:24:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.525+0000 7f863ca10700 1 -- 172.21.15.148:0/1234294849 learned_addr learned my addr 172.21.15.148:0/1234294849 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8637fff700 1 --2- 172.21.15.148:0/1234294849 >> v2:172.21.15.161:3300/0 conn(0x7f8638157870 0x7f863815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680128387 0 0) 0x7f863815cbb0 con 0x7f8638157c40 2023-10-10T20:24:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8620003570 con 0x7f8638157c40 2023-10-10T20:24:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169684132 0 0) 0x7f8620003570 con 0x7f8638157c40 2023-10-10T20:24:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 >> v2:172.21.15.161:3300/0 conn(0x7f8638157870 msgr2=0x7f863815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 --2- 172.21.15.148:0/1234294849 >> v2:172.21.15.161:3300/0 conn(0x7f8638157870 0x7f863815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 >> v1:172.21.15.161:6789/0 conn(0x7f863815a620 legacy=0x7f863815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863815c560 con 0x7f8638157c40 2023-10-10T20:24:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f86300017f0 con 0x7f8638157c40 2023-10-10T20:24:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 >> v1:172.21.15.148:6789/0 conn(0x7f8638157c40 legacy=0x7f8638158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620003a60 0x7f8620003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 0x7f8620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86200051c0 0x7f862000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863815c280 con 0x7f8620004810 2023-10-10T20:24:27.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862000c160 con 0x7f8620003a60 2023-10-10T20:24:27.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862000c320 con 0x7f86200051c0 2023-10-10T20:24:27.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 0x7f8620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f8637fff700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620003a60 0x7f8620003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f863ca10700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86200051c0 0x7f862000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 -- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86200051c0 msgr2=0x7f862000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86200051c0 0x7f862000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 -- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620003a60 msgr2=0x7f8620003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620003a60 0x7f8620003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f86377fe700 1 -- 172.21.15.148:0/1234294849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620003570 con 0x7f8620004810 2023-10-10T20:24:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.526+0000 7f863ca10700 1 -- 172.21.15.148:0/1234294849 reap_dead start 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f86377fe700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 0x7f8620004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f862c001880 tx=0x7f862c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2fac6ad5060b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c012420 con 0x7f8620004810 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c012580 con 0x7f8620004810 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f8636ffd700 1 -- 172.21.15.148:0/1234294849 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c012920 con 0x7f8620004810 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 -- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 msgr2=0x7f8620004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 0x7f8620004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f862c001880 tx=0x7f862c007d30 comp rx=0 tx=0).stop 2023-10-10T20:24:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 -- 172.21.15.148:0/1234294849 shutdown_connections 2023-10-10T20:24:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 --2- 172.21.15.148:0/1234294849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8620004810 0x7f8620004c80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 -- 172.21.15.148:0/1234294849 >> 172.21.15.148:0/1234294849 conn(0x7f8638068ce0 msgr2=0x7f86380690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 -- 172.21.15.148:0/1234294849 shutdown_connections 2023-10-10T20:24:27.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.527+0000 7f863e473700 1 -- 172.21.15.148:0/1234294849 wait complete. 2023-10-10T20:24:27.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.528+0000 7f863e473700 1 Processor -- start 2023-10-10T20:24:27.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 -- start start 2023-10-10T20:24:27.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620004810 0x7f8638156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86380a9050 0x7f86380a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86380a9a00 con 0x7f8638160c40 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86380a7e40 con 0x7f8620004810 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863e473700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86380a8120 con 0x7f86380a9050 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f86377fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f86377fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40130/0 (socket says 172.21.15.148:40130) 2023-10-10T20:24:27.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f86377fe700 1 -- 172.21.15.148:0/1563872872 learned_addr learned my addr 172.21.15.148:0/1563872872 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:27.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f8637fff700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620004810 0x7f8638156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.529+0000 7f863ca10700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86380a9050 0x7f86380a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 -- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86380a9050 msgr2=0x7f86380a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86380a9050 0x7f86380a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 -- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620004810 msgr2=0x7f8638156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620004810 0x7f8638156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380a8400 con 0x7f8638160c40 2023-10-10T20:24:27.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f86377fe700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f862c001e70 tx=0x7f862c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c012420 con 0x7f8638160c40 2023-10-10T20:24:27.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c001200 con 0x7f8638160c40 2023-10-10T20:24:27.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86380a86e0 con 0x7f8638160c40 2023-10-10T20:24:27.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0126f0 con 0x7f8638160c40 2023-10-10T20:24:27.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.530+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863816da30 con 0x7f8638160c40 2023-10-10T20:24:27.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.531+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863815d480 con 0x7f8638160c40 2023-10-10T20:24:27.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.531+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f862c019070 con 0x7f8638160c40 2023-10-10T20:24:27.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.532+0000 7f861ffff700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 0x7f86240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:27.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.532+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f862c085aa0 con 0x7f8638160c40 2023-10-10T20:24:27.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.532+0000 7f8637fff700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 0x7f86240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:27.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.532+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f862406ea50 con 0x7f8638160c40 2023-10-10T20:24:27.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.532+0000 7f8637fff700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 0x7f86240693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f863804d1b0 tx=0x7f86280091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:27.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.536+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f862c017060 con 0x7f8638160c40 2023-10-10T20:24:27.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.763+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863804c3c0 con 0x7f8638160c40 2023-10-10T20:24:27.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.764+0000 7f861ffff700 1 -- 172.21.15.148:0/1563872872 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f863815d480 con 0x7f8638160c40 2023-10-10T20:24:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:27.767 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:27.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 msgr2=0x7f86240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 0x7f86240693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f863804d1b0 tx=0x7f86280091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 msgr2=0x7f86380a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f862c001e70 tx=0x7f862c015000 comp rx=0 tx=0).stop 2023-10-10T20:24:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 shutdown_connections 2023-10-10T20:24:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8624066f30 0x7f86240693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86380a9050 0x7f86380a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8620004810 0x7f8638156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 --2- 172.21.15.148:0/1563872872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8638160c40 0x7f86380a7610 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 >> 172.21.15.148:0/1563872872 conn(0x7f8638068ce0 msgr2=0x7f86380b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:27.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 shutdown_connections 2023-10-10T20:24:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:27.766+0000 7f863e473700 1 -- 172.21.15.148:0/1563872872 wait complete. 2023-10-10T20:24:27.771 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:27.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:24:27.924 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.656509972999999,"mdsmap_epoch":49,"osdmap_epoch":101,"osdmap_epoch_barrier":101,"uptime":23.669318796999999} 2023-10-10T20:24:27.924 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 6381, "mdsmap_epoch": 49, "osdmap_epoch": 101, "osdmap_epoch_barrier": 101, "rank_uptime": 18.656509973, "state": "up:active", "uptime": 23.669318797, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:24:27.925 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:24:27.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:27.925 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-10T20:24:27.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:24:27.941 INFO:teuthology.orchestra.run.smithi148.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-10T20:24:27.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:27.942 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-10T20:24:27.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:27.999 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-10T20:24:28.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:28.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.230+0000 7f8c1c206700 1 Processor -- start 2023-10-10T20:24:28.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.230+0000 7f8c1c206700 1 -- start start 2023-10-10T20:24:28.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.230+0000 7f8c1c206700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415a620 0x7f8c1415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.230+0000 7f8c1c206700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c1415d580 0x7f8c14158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1c206700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1415cbb0 con 0x7f8c14157870 2023-10-10T20:24:28.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1c206700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1415bdb0 con 0x7f8c1415d580 2023-10-10T20:24:28.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1c206700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1415bfa0 con 0x7f8c1415a620 2023-10-10T20:24:28.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c1415d580 0x7f8c14158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:28.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c1415d580 0x7f8c14158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40136/0 (socket says 172.21.15.148:40136) 2023-10-10T20:24:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 -- 172.21.15.148:0/268144538 learned_addr learned my addr 172.21.15.148:0/268144538 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 -- 172.21.15.148:0/268144538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415a620 msgr2=0x7f8c1415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 --2- 172.21.15.148:0/268144538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415a620 0x7f8c1415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 -- 172.21.15.148:0/268144538 >> v1:172.21.15.161:6789/0 conn(0x7f8c14157870 legacy=0x7f8c1415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 -- 172.21.15.148:0/268144538 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1415c280 con 0x7f8c1415d580 2023-10-10T20:24:28.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c1a7a3700 1 --2- 172.21.15.148:0/268144538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415d580 0x7f8c14158110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0099b0 tx=0x7f8c0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dbe6d6c6b1c6a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:28.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c18fa0700 1 -- 172.21.15.148:0/268144538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00b820 con 0x7f8c1415d580 2023-10-10T20:24:28.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c18fa0700 1 -- 172.21.15.148:0/268144538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00b980 con 0x7f8c1415d580 2023-10-10T20:24:28.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.231+0000 7f8c18fa0700 1 -- 172.21.15.148:0/268144538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00bd20 con 0x7f8c1415d580 2023-10-10T20:24:28.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 -- 172.21.15.148:0/268144538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415d580 msgr2=0x7f8c14158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:28.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 --2- 172.21.15.148:0/268144538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415d580 0x7f8c14158110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0099b0 tx=0x7f8c0c009250 comp rx=0 tx=0).stop 2023-10-10T20:24:28.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 -- 172.21.15.148:0/268144538 shutdown_connections 2023-10-10T20:24:28.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 --2- 172.21.15.148:0/268144538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c1415d580 0x7f8c14158110 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 --2- 172.21.15.148:0/268144538 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415a620 0x7f8c1415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 -- 172.21.15.148:0/268144538 >> 172.21.15.148:0/268144538 conn(0x7f8c14068ce0 msgr2=0x7f8c140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:28.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 -- 172.21.15.148:0/268144538 shutdown_connections 2023-10-10T20:24:28.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.232+0000 7f8c1c206700 1 -- 172.21.15.148:0/268144538 wait complete. 2023-10-10T20:24:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.233+0000 7f8c1c206700 1 Processor -- start 2023-10-10T20:24:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.233+0000 7f8c1c206700 1 -- start start 2023-10-10T20:24:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1415a620 0x7f8c140d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415d580 0x7f8c14159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14159380 con 0x7f8c14157870 2023-10-10T20:24:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140d0f60 con 0x7f8c1415a620 2023-10-10T20:24:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140d1240 con 0x7f8c1415d580 2023-10-10T20:24:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:28.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40140/0 (socket says 172.21.15.148:40140) 2023-10-10T20:24:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 -- 172.21.15.148:0/3165981698 learned_addr learned my addr 172.21.15.148:0/3165981698 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1a7a3700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415d580 0x7f8c14159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c197a1700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1415a620 0x7f8c140d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:28.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 -- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415d580 msgr2=0x7f8c14159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415d580 0x7f8c14159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 -- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1415a620 msgr2=0x7f8c140d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1415a620 0x7f8c140d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0094f0 con 0x7f8c14157870 2023-10-10T20:24:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c19fa2700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000c950 tx=0x7f8c10002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10012630 con 0x7f8c14157870 2023-10-10T20:24:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c10012790 con 0x7f8c14157870 2023-10-10T20:24:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140d1520 con 0x7f8c14157870 2023-10-10T20:24:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.234+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10012b50 con 0x7f8c14157870 2023-10-10T20:24:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.235+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140b2f20 con 0x7f8c14157870 2023-10-10T20:24:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c14159660 con 0x7f8c14157870 2023-10-10T20:24:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c1001b400 con 0x7f8c14157870 2023-10-10T20:24:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c09ffb700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c10084940 con 0x7f8c14157870 2023-10-10T20:24:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c197a1700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:28.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.236+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f8bf806ea50 con 0x7f8c14157870 2023-10-10T20:24:28.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.237+0000 7f8c197a1700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 0x7f8bf80693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b2970 tx=0x7f8c040091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:28.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.241+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c10016060 con 0x7f8c14157870 2023-10-10T20:24:28.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.424+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8c1404c3c0 con 0x7f8c14157870 2023-10-10T20:24:28.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.425+0000 7f8c09ffb700 1 -- 172.21.15.148:0/3165981698 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+32479 (secure 0 0 0) 0x7f8c10051c60 con 0x7f8c14157870 2023-10-10T20:24:28.427 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:28.427 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:28.428 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 101, 2023-10-10T20:24:28.428 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:28.428 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:28.428 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:24.532180+0000", 2023-10-10T20:24:28.428 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:28.429 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:28.429 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:28.429 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:28.429 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:28.429 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:28.430 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:28.431 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:28.432 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:28.432 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:28.432 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:28.432 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:28.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:28.433 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:28.434 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:28.434 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:28.434 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:28.434 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:28.434 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:28.435 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:28.435 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:28.435 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:28.435 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:28.436 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:28.436 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:28.436 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:28.436 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:28.436 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:28.437 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:28.437 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:28.437 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:28.437 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:28.437 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:28.438 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:28.438 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:28.438 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:28.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.438 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:28.439 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:28.441 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:28.441 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:28.441 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:28.441 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:28.441 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:28.442 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:28.442 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:28.442 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:28.442 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:28.442 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:28.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.444 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:28.444 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:28.444 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:28.444 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:28.444 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:28.445 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:28.445 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:28.445 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:28.445 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:28.445 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:28.446 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:28.446 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:28.446 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:28.446 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:28.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.447 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:28.448 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:28.449 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:28.449 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:28.449 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:28.449 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:28.449 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:28.450 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:28.450 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:28.450 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:28.450 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:28.450 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:28.451 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:28.454 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:28.455 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:28.457 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:28.457 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:28.457 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:28.457 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:28.457 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.458 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:28.459 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:28.459 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:28.459 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:28.459 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:28.459 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:28.460 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:28.461 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:28.461 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:28.461 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:28.461 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.461 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:28.462 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:28.462 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:28.462 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.463 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.463 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:28.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:28.464 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:28.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:28.464 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:28.464 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:28.464 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:28.465 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:28.466 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:28.466 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:28.466 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:28.466 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:28.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:28.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:28.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:28.467 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:28.467 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:28.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.469 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:28.469 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:28.469 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:28.469 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:28.469 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:28.470 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:28.470 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:28.470 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:28.470 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:28.470 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:28.471 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:28.472 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:28.472 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:28.472 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:28.472 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:28.472 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:28.473 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:28.473 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:28.473 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:28.473 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:28.473 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:28.474 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:28.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.474 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:28.474 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:28.474 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:28.475 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:28.476 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:28.476 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:28.476 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:28.476 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:28.476 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:28.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.477 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:28.477 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:28.477 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:28.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.478 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:28.479 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:28.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.479 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:28.480 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.480 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.480 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.480 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.480 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.481 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.482 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.482 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.482 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.482 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.485 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.485 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:28.485 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.486 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.486 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:28.486 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.487 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.487 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.487 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.489 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:28.489 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.489 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.490 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.490 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.490 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.491 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:28.491 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.491 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.492 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:28.492 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:28.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.492 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.493 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.494 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:28.494 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.494 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.495 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.496 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.496 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.496 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.496 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.496 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.497 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.498 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:28.498 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.500 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.500 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.501 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.502 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:28.508 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.510 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.511 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.512 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.513 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:28.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:28.516 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.521 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.522 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.523 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.525 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.526 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:28.531 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.534 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:28.538 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.540 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.541 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.542 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:28.543 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:28.548 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.549 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.549 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.550 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.550 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:28.550 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:28.551 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:28.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:28.552 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:28.553 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.553 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.553 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:28.553 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.554 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:28.554 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.554 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.555 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.556 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.556 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.556 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.556 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.556 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:28.558 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.558 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.558 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.559 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.559 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.560 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:28.560 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.560 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:28.563 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.567 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.567 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:28.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:28.567 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.569 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:28.569 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.569 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.569 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.570 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.570 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.570 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.570 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.570 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:28.572 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:28.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:28.576 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.576 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.576 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.578 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.578 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:28.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.579 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.580 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:28.580 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.580 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.581 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:28.581 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:28.581 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.581 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.582 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:28.582 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:28.582 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:28.582 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:28.582 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.584 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:28.584 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:28.584 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:28.584 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:28.584 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:28.585 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:28.586 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:28.586 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:28.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.586 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.588 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:28.589 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:28.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:28.592 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.594 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:28.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:28.594 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.594 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:28.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:28.595 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:28.596 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:28.597 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:28.597 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:28.597 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:24:28.597 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:28.597 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:28.598 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:28.598 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.598 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:28.598 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:28.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.599 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:28.600 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:28.601 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.602 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.603 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.603 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.603 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.603 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:28.603 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.604 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:28.606 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.606 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.606 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.606 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.607 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.607 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:28.607 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.608 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.609 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:28.609 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:28.610 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.610 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.610 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.610 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.610 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.611 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:28.611 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:28.611 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:28.612 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:28.613 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:28.613 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.613 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:28.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:28.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:28.615 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:28.615 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:28.615 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:28.615 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:28.615 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:28.616 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:28.616 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:28.616 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:28.616 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:28.617 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:28.617 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:28.617 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:28.617 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:28.617 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:28.618 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:28.618 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:28.618 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:28.618 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:28.618 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:28.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 msgr2=0x7f8bf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:28.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 0x7f8bf80693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b2970 tx=0x7f8c040091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 msgr2=0x7f8c14155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:28.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000c950 tx=0x7f8c10002760 comp rx=0 tx=0).stop 2023-10-10T20:24:28.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 shutdown_connections 2023-10-10T20:24:28.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1415d580 0x7f8c14159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c14157870 0x7f8c14155710 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 --2- 172.21.15.148:0/3165981698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1415a620 0x7f8c140d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:28.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 >> 172.21.15.148:0/3165981698 conn(0x7f8c14068ce0 msgr2=0x7f8c140690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:28.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 shutdown_connections 2023-10-10T20:24:28.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:28.427+0000 7f8c1c206700 1 -- 172.21.15.148:0/3165981698 wait complete. 2023-10-10T20:24:28.623 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:24:28.623 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:24:28.542+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-10T20:24:33.557 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:24:33.555+0000 7f2ae3be5700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-10T20:24:33.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:33.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.752+0000 7efd26027700 1 Processor -- start 2023-10-10T20:24:33.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.752+0000 7efd26027700 1 -- start start 2023-10-10T20:24:33.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd26027700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd2015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd26027700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2015cbb0 con 0x7efd2015a620 2023-10-10T20:24:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd26027700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2015bfa0 con 0x7efd20157c40 2023-10-10T20:24:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd26027700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2015c280 con 0x7efd20157870 2023-10-10T20:24:33.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd16ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efd2015a620 0x7efd2015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46308/0 (socket says 172.21.15.148:46308) 2023-10-10T20:24:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd16ffd700 1 -- 172.21.15.148:0/2563818977 learned_addr learned my addr 172.21.15.148:0/2563818977 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1f7fe700 1 --2- 172.21.15.148:0/2563818977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd2015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322294849 0 0) 0x7efd2015cbb0 con 0x7efd2015a620 2023-10-10T20:24:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd0c003570 con 0x7efd2015a620 2023-10-10T20:24:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261759231 0 0) 0x7efd2015bfa0 con 0x7efd20157c40 2023-10-10T20:24:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2015cbb0 con 0x7efd20157c40 2023-10-10T20:24:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1f7fe700 1 -- 172.21.15.148:0/2563818977 >> v1:172.21.15.161:6789/0 conn(0x7efd20157c40 legacy=0x7efd20158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1f7fe700 1 -- 172.21.15.148:0/2563818977 >> v1:172.21.15.148:6789/0 conn(0x7efd2015a620 legacy=0x7efd2015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1f7fe700 1 -- 172.21.15.148:0/2563818977 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2015c560 con 0x7efd20157870 2023-10-10T20:24:33.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.753+0000 7efd1f7fe700 1 --2- 172.21.15.148:0/2563818977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd2015b6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd10009910 tx=0x7efd100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b230a896e858c49e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10011630 con 0x7efd20157870 2023-10-10T20:24:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd10011790 con 0x7efd20157870 2023-10-10T20:24:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd1effd700 1 -- 172.21.15.148:0/2563818977 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10011b30 con 0x7efd20157870 2023-10-10T20:24:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 -- 172.21.15.148:0/2563818977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 msgr2=0x7efd2015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:33.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 --2- 172.21.15.148:0/2563818977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd2015b6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd10009910 tx=0x7efd100091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 -- 172.21.15.148:0/2563818977 shutdown_connections 2023-10-10T20:24:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 --2- 172.21.15.148:0/2563818977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd2015b6a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 -- 172.21.15.148:0/2563818977 >> 172.21.15.148:0/2563818977 conn(0x7efd20068ce0 msgr2=0x7efd200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 -- 172.21.15.148:0/2563818977 shutdown_connections 2023-10-10T20:24:33.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.754+0000 7efd26027700 1 -- 172.21.15.148:0/2563818977 wait complete. 2023-10-10T20:24:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.755+0000 7efd26027700 1 Processor -- start 2023-10-10T20:24:33.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.755+0000 7efd26027700 1 -- start start 2023-10-10T20:24:33.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd20152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd20157c40 0x7efd20150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2014ffc0 con 0x7efd2015a620 2023-10-10T20:24:33.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200b2fb0 con 0x7efd20157c40 2023-10-10T20:24:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd26027700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2014f3b0 con 0x7efd20157870 2023-10-10T20:24:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd1ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd1ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58392/0 (socket says 172.21.15.148:58392) 2023-10-10T20:24:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.756+0000 7efd1ffff700 1 -- 172.21.15.148:0/1930608242 learned_addr learned my addr 172.21.15.148:0/1930608242 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:33.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd16ffd700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd20157c40 0x7efd20150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 -- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 msgr2=0x7efd20152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1f7fe700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd20152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd20152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 -- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd20157c40 msgr2=0x7efd20150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:33.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd20157c40 0x7efd20150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd10009420 con 0x7efd2015a620 2023-10-10T20:24:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd1ffff700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c940 tx=0x7efd180027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800a7a0 con 0x7efd2015a620 2023-10-10T20:24:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1800a920 con 0x7efd2015a620 2023-10-10T20:24:33.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2014f690 con 0x7efd2015a620 2023-10-10T20:24:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800ace0 con 0x7efd2015a620 2023-10-10T20:24:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.757+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2014fc50 con 0x7efd2015a620 2023-10-10T20:24:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.758+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd20150170 con 0x7efd2015a620 2023-10-10T20:24:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.758+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd18012630 con 0x7efd2015a620 2023-10-10T20:24:33.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.759+0000 7efd17fff700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.759+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd18084950 con 0x7efd2015a620 2023-10-10T20:24:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.759+0000 7efd1f7fe700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.759+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=104}) v3 -- 0x7efd0006ea50 con 0x7efd2015a620 2023-10-10T20:24:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.759+0000 7efd1f7fe700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 0x7efd000693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd10009860 tx=0x7efd10017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:33.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.763+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd18016020 con 0x7efd2015a620 2023-10-10T20:24:33.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.948+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efd200d1cb0 con 0x7efd2015a620 2023-10-10T20:24:33.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.949+0000 7efd17fff700 1 -- 172.21.15.148:0/1930608242 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+32463 (secure 0 0 0) 0x7efd20150170 con 0x7efd2015a620 2023-10-10T20:24:33.952 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:33.952 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:33.952 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 103, 2023-10-10T20:24:33.952 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:33.953 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:33.953 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:33.556829+0000", 2023-10-10T20:24:33.953 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:33.953 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:33.954 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:33.955 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:33.955 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:33.955 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:33.956 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:33.956 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:33.956 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:33.957 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:33.957 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:33.958 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:33.958 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:33.959 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:33.959 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:33.959 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:33.960 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:33.960 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:33.961 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:33.961 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:33.962 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:33.962 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:33.962 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:33.963 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:33.963 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:33.963 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:33.964 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:33.964 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:33.964 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:33.965 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:33.965 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:33.965 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:33.965 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:33.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:33.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:33.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:33.967 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:33.968 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:33.968 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:33.968 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:33.968 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:33.969 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:33.970 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:33.970 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:33.970 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:33.971 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:33.972 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:33.973 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:33.975 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:33.976 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:33.976 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:33.976 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:33.976 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:33.976 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:33.977 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:33.977 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:33.977 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:33.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.977 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:33.978 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:33.979 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:33.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:33.979 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:33.979 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:33.979 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:33.980 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:33.980 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:33.980 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:33.980 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:33.980 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:33.981 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:33.981 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:33.981 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:33.981 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:33.981 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:33.982 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:33.983 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:33.983 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:33.983 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:33.983 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:33.983 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:33.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.984 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:33.984 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:33.984 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:33.984 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:33.985 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:33.985 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:33.985 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:33.985 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:33.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:33.986 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:33.986 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:33.986 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:33.986 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:33.986 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:33.987 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:33.987 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:33.987 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:33.987 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:33.987 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:33.989 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:33.990 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:33.991 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:33.991 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:33.991 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:33.991 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:33.991 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:33.992 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:33.993 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:33.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:33.993 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:33.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:33.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:33.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:33.994 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:33.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 3, 2023-10-10T20:24:33.994 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full", 2023-10-10T20:24:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:33.995 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:33.996 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:33.996 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:33.996 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:33.996 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:33.996 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:33.997 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:33.998 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:33.998 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:33.998 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:33.998 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:33.998 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:33.999 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:33.999 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:33.999 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "99", 2023-10-10T20:24:33.999 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:33.999 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:34.000 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:34.000 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:34.000 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:34.000 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:34.000 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:34.001 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:34.002 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:34.002 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:34.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:34.002 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:34.003 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:34.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:34.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:34.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:34.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:34.004 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:34.004 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:34.004 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:34.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:34.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:34.005 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:34.006 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:34.006 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:34.006 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:34.006 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:34.006 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.008 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:34.009 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:34.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.009 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:34.009 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.009 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.010 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.010 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.010 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.010 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.010 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.011 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.012 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.012 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.012 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:34.012 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.012 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.014 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.014 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:34.015 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.015 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.015 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.016 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.016 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.017 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.017 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.018 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:34.018 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.020 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:34.020 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.020 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.021 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.022 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.023 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.023 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:34.023 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.024 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:34.025 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.027 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.027 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.028 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:34.029 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.030 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.030 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:34.031 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:34.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.032 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:34.032 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:34.033 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:34.033 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:34.033 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.034 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.034 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:34.035 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.035 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.036 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:34.036 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.037 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.037 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.038 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.038 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.038 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.040 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.040 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.040 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.041 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.041 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.041 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.041 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.042 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.042 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:34.042 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.043 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.043 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:34.043 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.045 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:34.045 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.046 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.048 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:34.048 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:34.049 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.049 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.050 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:34.051 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:34.052 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:34.053 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:34.053 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:34.053 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.053 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.053 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:34.054 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.055 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.055 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.055 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.055 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.055 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.056 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.056 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.056 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.056 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.056 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.057 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.057 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:34.058 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.058 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.058 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:34.060 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.061 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:34.061 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.061 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.062 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.062 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:34.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:34.064 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.065 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.065 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:34.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.067 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:34.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:34.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.068 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.068 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:34.068 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:34.069 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:34.069 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:34.069 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.069 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.069 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:34.070 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.070 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.071 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:34.071 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.071 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.071 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.071 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.072 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.072 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.072 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.072 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.072 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.073 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.073 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.073 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.075 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.077 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.081 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:34.082 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:34.083 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:34.084 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.085 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.086 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.090 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.096 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:34.097 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:34.098 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:34.099 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:24:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.100 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:34.101 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.102 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.103 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.104 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.105 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.106 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.108 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.111 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.113 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.115 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.117 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.118 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.119 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:34.120 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.121 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.123 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.128 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:34.130 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "full", 2023-10-10T20:24:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.131 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:34.132 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:34.133 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:34.134 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:34.135 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:34.136 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:34.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.139 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.142 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:34.144 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull", 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:34.146 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.147 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.148 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.149 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.150 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.152 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:34.153 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.154 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:34.155 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.156 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:34.157 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.158 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.159 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.160 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.160 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.160 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:34.161 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:34.161 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:34.161 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:34.162 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:34.162 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:34.162 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:34.162 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:34.162 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:34.163 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:34.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.163 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:34.163 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:34.163 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:34.165 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:34.165 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:34.165 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:34.165 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:34.166 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:34.166 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:34.166 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:34.166 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:34.166 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:34.167 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:34.167 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:34.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:34.167 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:34.168 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:34.169 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:34.169 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:34.169 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:34.169 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:34.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:34.170 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 msgr2=0x7efd000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:34.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 0x7efd000693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efd10009860 tx=0x7efd10017000 comp rx=0 tx=0).stop 2023-10-10T20:24:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 msgr2=0x7efd200b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c940 tx=0x7efd180027a0 comp rx=0 tx=0).stop 2023-10-10T20:24:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 shutdown_connections 2023-10-10T20:24:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efd00066f30 0x7efd000693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:34.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd2015a620 0x7efd200b15e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd20157870 0x7efd20152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 --2- 172.21.15.148:0/1930608242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd20157c40 0x7efd20150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 >> 172.21.15.148:0/1930608242 conn(0x7efd20068ce0 msgr2=0x7efd200b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 shutdown_connections 2023-10-10T20:24:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:33.951+0000 7efd26027700 1 -- 172.21.15.148:0/1930608242 wait complete. 2023-10-10T20:24:34.174 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:24:39.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:39.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.303+0000 7fd79ffff700 1 Processor -- start 2023-10-10T20:24:39.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.303+0000 7fd79ffff700 1 -- start start 2023-10-10T20:24:39.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a00cdab0 0x7fd7a015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79ffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd7a015dda0 0x7fd7a015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79ffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00cb7c0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79ffff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a015e850 con 0x7fd7a00cdab0 2023-10-10T20:24:39.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79ffff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a00cc360 con 0x7fd7a015dda0 2023-10-10T20:24:39.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd7967fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a00cdab0 0x7fd7a015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd7967fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a00cdab0 0x7fd7a015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58402/0 (socket says 172.21.15.148:58402) 2023-10-10T20:24:39.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd7967fc700 1 -- 172.21.15.148:0/2080073074 learned_addr learned my addr 172.21.15.148:0/2080073074 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:39.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.304+0000 7fd79f7fe700 1 --2- 172.21.15.148:0/2080073074 >> v2:172.21.15.161:3300/0 conn(0x7fd7a015dda0 0x7fd7a015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd7967fc700 1 -- 172.21.15.148:0/2080073074 >> v2:172.21.15.161:3300/0 conn(0x7fd7a015dda0 msgr2=0x7fd7a015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:39.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd7967fc700 1 --2- 172.21.15.148:0/2080073074 >> v2:172.21.15.161:3300/0 conn(0x7fd7a015dda0 0x7fd7a015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd7967fc700 1 -- 172.21.15.148:0/2080073074 >> v1:172.21.15.161:6789/0 conn(0x7fd7a00cd6e0 legacy=0x7fd7a00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd7967fc700 1 -- 172.21.15.148:0/2080073074 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a015e290 con 0x7fd7a00cdab0 2023-10-10T20:24:39.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd7967fc700 1 --2- 172.21.15.148:0/2080073074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cdab0 0x7fd7a015d860 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00cb730 tx=0x7fd788009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ea83817716f8b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79e7fc700 1 -- 172.21.15.148:0/2080073074 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788009e70 con 0x7fd7a00cdab0 2023-10-10T20:24:39.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79e7fc700 1 -- 172.21.15.148:0/2080073074 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800b860 con 0x7fd7a00cdab0 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79e7fc700 1 -- 172.21.15.148:0/2080073074 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800bc00 con 0x7fd7a00cdab0 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 -- 172.21.15.148:0/2080073074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cdab0 msgr2=0x7fd7a015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 --2- 172.21.15.148:0/2080073074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cdab0 0x7fd7a015d860 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00cb730 tx=0x7fd788009250 comp rx=0 tx=0).stop 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 -- 172.21.15.148:0/2080073074 shutdown_connections 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 --2- 172.21.15.148:0/2080073074 >> v2:172.21.15.161:3300/0 conn(0x7fd7a015dda0 0x7fd7a015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 --2- 172.21.15.148:0/2080073074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cdab0 0x7fd7a015d860 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 -- 172.21.15.148:0/2080073074 >> 172.21.15.148:0/2080073074 conn(0x7fd7a0067660 msgr2=0x7fd7a0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:39.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 -- 172.21.15.148:0/2080073074 shutdown_connections 2023-10-10T20:24:39.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.305+0000 7fd79ffff700 1 -- 172.21.15.148:0/2080073074 wait complete. 2023-10-10T20:24:39.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 Processor -- start 2023-10-10T20:24:39.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 -- start start 2023-10-10T20:24:39.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a00cdab0 0x7fd7a00c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 0x7fd7a00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0159f10 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0159300 con 0x7fd7a00cdab0 2023-10-10T20:24:39.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.307+0000 7fd79ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a01595e0 con 0x7fd7a015dda0 2023-10-10T20:24:39.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58418/0 (socket says 172.21.15.148:58418) 2023-10-10T20:24:39.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 -- 172.21.15.148:0/3774787812 learned_addr learned my addr 172.21.15.148:0/3774787812 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:39.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd7967fc700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a00cdab0 0x7fd7a00c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79f7fe700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 0x7fd7a00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 -- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 msgr2=0x7fd7a00b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 0x7fd7a00b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 -- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a00cdab0 msgr2=0x7fd7a00c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a00cdab0 0x7fd7a00c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 -- 172.21.15.148:0/3774787812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880094f0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79f7fe700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 0x7fd7a00b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79effd700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c590 tx=0x7fd79000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790011590 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd790011710 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a01598c0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790011ad0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.308+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a0156220 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.309+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0089280 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.310+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd79001b3b0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.310+0000 7fd7977fe700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.310+0000 7fd7967fc700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.310+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd790083a70 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.310+0000 7fd7967fc700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 0x7fd7800693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00cb730 tx=0x7fd788017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.314+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd790050ce0 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.497+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd7a004bb80 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.498+0000 7fd7977fe700 1 -- 172.21.15.148:0/3774787812 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+32315 (secure 0 0 0) 0x7fd790017070 con 0x7fd7a00cd6e0 2023-10-10T20:24:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:39.501 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 104, 2023-10-10T20:24:39.501 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:34.609051+0000", 2023-10-10T20:24:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:39.502 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:39.503 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:39.504 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:39.506 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:39.507 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:39.508 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:39.509 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:39.510 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:39.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:39.512 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:39.513 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:39.514 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:39.515 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:39.516 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:39.518 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:39.519 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:39.521 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:39.522 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:39.523 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:39.524 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:39.525 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:39.526 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:39.527 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:39.528 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:39.529 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:39.530 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:39.531 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:39.532 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:39.533 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:39.534 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:39.535 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.536 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:39.537 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:39.538 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:39.539 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:39.540 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:39.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:39.543 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:39.544 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:39.545 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:39.546 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:39.547 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:39.548 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:39.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:39.550 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:39.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.552 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.553 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:39.565 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.567 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.568 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.571 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.573 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.576 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:39.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:39.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:39.579 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.580 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.581 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.583 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.588 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.591 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:39.593 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:39.594 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.596 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.597 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.598 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:39.601 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:39.602 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.603 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.604 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.605 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.607 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:39.608 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.609 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.610 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.611 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.618 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.619 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.620 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:39.622 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:39.623 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.625 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.626 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.629 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.630 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.631 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.632 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.633 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.634 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.635 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.636 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.637 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.638 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:39.639 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.640 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.646 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:39.648 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:39.652 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:39.654 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:39.655 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.658 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.663 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:39.664 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:39.665 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:39.667 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.668 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.669 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:39.670 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:39.671 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.672 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.674 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.676 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.678 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.679 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.680 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:39.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:39.683 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:39.684 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:39.685 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:39.686 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:39.687 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:39.688 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:39.688 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:39.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 msgr2=0x7fd7800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 0x7fd7800693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00cb730 tx=0x7fd788017000 comp rx=0 tx=0).stop 2023-10-10T20:24:39.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 msgr2=0x7fd7a0155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd79000c590 tx=0x7fd79000c8a0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 shutdown_connections 2023-10-10T20:24:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd780066f30 0x7fd7800693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7a015dda0 0x7fd7a00b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a00cd6e0 0x7fd7a0155ce0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 --2- 172.21.15.148:0/3774787812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7a00cdab0 0x7fd7a00c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.500+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 >> 172.21.15.148:0/3774787812 conn(0x7fd7a0067660 msgr2=0x7fd7a00c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.501+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 shutdown_connections 2023-10-10T20:24:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.501+0000 7fd79ffff700 1 -- 172.21.15.148:0/3774787812 wait complete. 2023-10-10T20:24:39.693 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:24:39.693 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:24:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 Processor -- start 2023-10-10T20:24:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 -- start start 2023-10-10T20:24:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb4015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4006c010 con 0x7fdb40158e50 2023-10-10T20:24:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4006bd10 con 0x7fdb401523e0 2023-10-10T20:24:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.806+0000 7fdb44c0e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4015c060 con 0x7fdb40152010 2023-10-10T20:24:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3ed9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdb40158e50 0x7fdb400cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46320/0 (socket says 172.21.15.148:46320) 2023-10-10T20:24:39.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/428804751 learned_addr learned my addr 172.21.15.148:0/428804751 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:39.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172573163 0 0) 0x7fdb4006c010 con 0x7fdb40158e50 2023-10-10T20:24:39.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c003570 con 0x7fdb40158e50 2023-10-10T20:24:39.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3e59c700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb4015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297221171 0 0) 0x7fdb2c003570 con 0x7fdb40158e50 2023-10-10T20:24:39.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 msgr2=0x7fdb4015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:39.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb4015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 >> v1:172.21.15.161:6789/0 conn(0x7fdb401523e0 legacy=0x7fdb4006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4015c340 con 0x7fdb40158e50 2023-10-10T20:24:39.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fdb380017f0 con 0x7fdb40158e50 2023-10-10T20:24:39.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 >> v1:172.21.15.148:6789/0 conn(0x7fdb40158e50 legacy=0x7fdb400cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb2c003a90 0x7fdb2c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb2c004d50 0x7fdb2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4015c060 con 0x7fdb2c004d50 2023-10-10T20:24:39.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00bb70 con 0x7fdb2c003a90 2023-10-10T20:24:39.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c00bd30 con 0x7fdb40152010 2023-10-10T20:24:39.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb2c004d50 0x7fdb2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3e59c700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb2c003a90 0x7fdb2c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb35d9b700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.807+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 msgr2=0x7fdb2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:39.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb40152010 0x7fdb2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb2c003a90 msgr2=0x7fdb2c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:39.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb2c003a90 0x7fdb2c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/428804751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c003570 con 0x7fdb2c004d50 2023-10-10T20:24:39.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb2c004d50 0x7fdb2c0051c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800a900 tx=0x7fdb4006bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79e13057773b0c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380070d0 con 0x7fdb2c004d50 2023-10-10T20:24:39.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38007230 con 0x7fdb2c004d50 2023-10-10T20:24:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3dd9b700 1 -- 172.21.15.148:0/428804751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380075f0 con 0x7fdb2c004d50 2023-10-10T20:24:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 -- 172.21.15.148:0/428804751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb2c004d50 msgr2=0x7fdb2c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/428804751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb2c004d50 0x7fdb2c0051c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800a900 tx=0x7fdb4006bf80 comp rx=0 tx=0).stop 2023-10-10T20:24:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/428804751 reap_dead start 2023-10-10T20:24:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 -- 172.21.15.148:0/428804751 shutdown_connections 2023-10-10T20:24:39.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 -- 172.21.15.148:0/428804751 >> 172.21.15.148:0/428804751 conn(0x7fdb40069870 msgr2=0x7fdb40069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:39.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 -- 172.21.15.148:0/428804751 shutdown_connections 2023-10-10T20:24:39.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.808+0000 7fdb44c0e700 1 -- 172.21.15.148:0/428804751 wait complete. 2023-10-10T20:24:39.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 Processor -- start 2023-10-10T20:24:39.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 -- start start 2023-10-10T20:24:39.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb400b35e0 0x7fdb400685a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 0x7fdb401570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4016b210 con 0x7fdb40068ae0 2023-10-10T20:24:39.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4016a640 con 0x7fdb400b39b0 2023-10-10T20:24:39.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb44c0e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4016a8e0 con 0x7fdb400b35e0 2023-10-10T20:24:39.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb3ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb3ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58430/0 (socket says 172.21.15.148:58430) 2023-10-10T20:24:39.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/606687062 learned_addr learned my addr 172.21.15.148:0/606687062 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:39.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.810+0000 7fdb3e59c700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb400b35e0 0x7fdb400685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb35d9b700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 0x7fdb401570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb400b35e0 msgr2=0x7fdb400685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb400b35e0 0x7fdb400685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 msgr2=0x7fdb401570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 0x7fdb401570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 -- 172.21.15.148:0/606687062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4016abc0 con 0x7fdb40068ae0 2023-10-10T20:24:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb35d9b700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 0x7fdb401570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb3ed9d700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdb38012000 tx=0x7fdb38015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380070d0 con 0x7fdb40068ae0 2023-10-10T20:24:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38001a90 con 0x7fdb40068ae0 2023-10-10T20:24:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4016aea0 con 0x7fdb40068ae0 2023-10-10T20:24:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007960 con 0x7fdb40068ae0 2023-10-10T20:24:39.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.811+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb400b18e0 con 0x7fdb40068ae0 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.812+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4016b380 con 0x7fdb40068ae0 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.813+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb38029460 con 0x7fdb40068ae0 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.813+0000 7fdb36ffd700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 0x7fdb200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.813+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdb38086a70 con 0x7fdb40068ae0 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.813+0000 7fdb3e59c700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 0x7fdb200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:39.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.813+0000 7fdb3e59c700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 0x7fdb200693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b0c30 tx=0x7fdb300091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:39.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:39.817+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb38053ce0 con 0x7fdb40068ae0 2023-10-10T20:24:40.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.001+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdb4004c3c0 con 0x7fdb40068ae0 2023-10-10T20:24:40.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.002+0000 7fdb36ffd700 1 -- 172.21.15.148:0/606687062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+32315 (secure 0 0 0) 0x7fdb38016090 con 0x7fdb40068ae0 2023-10-10T20:24:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:40.004 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 104, 2023-10-10T20:24:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:24:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:24:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:24:34.609051+0000", 2023-10-10T20:24:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:24:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:24:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:24:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:24:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:24:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:24:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:24:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:24:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:24:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:24:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:24:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:24:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:24:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:24:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 13, 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:24:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:24:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:24:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:24:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:24:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:24:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:24:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:24:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:24:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:24:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:24:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:24:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 12, 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:06.952553+0000", 2023-10-10T20:24:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:24:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:24:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:24:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:24:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:24:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:24:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:24:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:24:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:24:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 13, 2023-10-10T20:24:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:24:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:24:07.573984+0000", 2023-10-10T20:24:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:24:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:24:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:24:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:24:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:24:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:24:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:24:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:24:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:24:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:24:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:24:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:24:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:24:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:24:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:24:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:24:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:24:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:24:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:24:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:24:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:24:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:24:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:24:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:24:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:24:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:24:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:24:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:24:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:24:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:24:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:24:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:24:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:24:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:24:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:24:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:24:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:24:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:24:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:24:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:24:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:24:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:24:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:24:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:24:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:24:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:24:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:24:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:24:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:24:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:24:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:24:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:24:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:24:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:24:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:24:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:24:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:24:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:24:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:24:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:24:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:24:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:24:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:24:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:24:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:24:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:24:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:24:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:24:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:24:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:24:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:24:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:24:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:24:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:24:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:24:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:24:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:24:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:24:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:24:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:24:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:24:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:24:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:24:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:24:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:24:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.120 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.120 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.120 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:24:40.120 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:24:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:24:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:24:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:24:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:24:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:24:40.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.129 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.129 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:24:40.129 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:24:40.129 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:24:40.130 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:24:40.130 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.130 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.130 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.131 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:40.131 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.131 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.132 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.133 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.133 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.133 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.133 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.133 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.134 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.134 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.134 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:24:40.134 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.135 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.136 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.136 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.136 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.136 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.136 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.137 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:24:40.138 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.138 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.138 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.139 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.139 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:24:40.139 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.140 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.141 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.141 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.141 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.142 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.143 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:24:40.143 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:24:40.143 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.143 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.144 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.145 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.145 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.145 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.146 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.147 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.147 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.147 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.147 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.147 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:24:40.148 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.149 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.149 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:24:40.149 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.150 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.151 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.151 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:24:40.151 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.151 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.152 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.152 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:24:40.152 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.152 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.152 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.153 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:24:40.154 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.154 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.154 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.154 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:24:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.155 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:24:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.157 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.157 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:24:40.157 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:24:40.158 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.158 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.158 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:24:40.158 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.159 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:40.160 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:24:40.160 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:24:40.161 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:24:40.161 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:24:40.161 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:24:40.161 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:24:40.161 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:24:40.162 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:24:40.162 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 96, 2023-10-10T20:24:40.162 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:24:40.162 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:24:40.162 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.164 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.164 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:24:40.164 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.164 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.165 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.166 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:24:40.166 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.166 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.167 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:24:40.167 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.167 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:24:40.168 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.169 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.169 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.169 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.169 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:24:40.169 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:24:40.170 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.171 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:24:40.171 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:24:40.171 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.171 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.172 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:24:40.172 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:24:40.172 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:24:40.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.172 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:24:40.173 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:24:40.174 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:24:40.174 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:24:40.174 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:40.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.174 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.175 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.176 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.176 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.176 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:24:40.176 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.177 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.178 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.178 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:24:40.178 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:24:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.179 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.180 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:24:40.181 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.181 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.181 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.181 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.181 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.182 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:24:40.182 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.182 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.183 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:24:40.184 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.184 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:24:40.184 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:24:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:24:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.187 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.187 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.187 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.187 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:24:40.187 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:24:40.188 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:24:40.189 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:24:40.189 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:24:40.189 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:24:40.189 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:24:40.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.190 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:24:40.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:24:40.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:24:40.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:24:40.190 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:24:40.191 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:40.192 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:24:40.193 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:24:40.193 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:24:40.193 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:24:40.193 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:24:40.193 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:24:40.194 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:40.195 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:40.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.003+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 msgr2=0x7fdb200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.003+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 0x7fdb200693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b0c30 tx=0x7fdb300091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.003+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 msgr2=0x7fdb40157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.003+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdb38012000 tx=0x7fdb38015000 comp rx=0 tx=0).stop 2023-10-10T20:24:40.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 shutdown_connections 2023-10-10T20:24:40.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdb20066f30 0x7fdb200693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40068ae0 0x7fdb40157620 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb400b35e0 0x7fdb400685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 --2- 172.21.15.148:0/606687062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb400b39b0 0x7fdb401570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 >> 172.21.15.148:0/606687062 conn(0x7fdb40069870 msgr2=0x7fdb40158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:40.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 shutdown_connections 2023-10-10T20:24:40.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.004+0000 7fdb44c0e700 1 -- 172.21.15.148:0/606687062 wait complete. 2023-10-10T20:24:40.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:40.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.315+0000 7fd0ad1c5700 1 Processor -- start 2023-10-10T20:24:40.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.315+0000 7fd0ad1c5700 1 -- start start 2023-10-10T20:24:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0ad1c5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd0a81523e0 0x7fd0a806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0ad1c5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0a8158e50 0x7fd0a80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0ad1c5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a806c010 con 0x7fd0a8152010 2023-10-10T20:24:40.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0ad1c5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a806bd10 con 0x7fd0a8158e50 2023-10-10T20:24:40.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0ad1c5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a815c060 con 0x7fd0a81523e0 2023-10-10T20:24:40.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0a759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0a8158e50 0x7fd0a80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0a6d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd0a8152010 0x7fd0a815b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46330/0 (socket says 172.21.15.148:46330) 2023-10-10T20:24:40.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0a6d9d700 1 -- 172.21.15.148:0/620056643 learned_addr learned my addr 172.21.15.148:0/620056643 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:40.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd09e59c700 1 --2- 172.21.15.148:0/620056643 >> v2:172.21.15.161:3300/0 conn(0x7fd0a81523e0 0x7fd0a806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.316+0000 7fd0a759e700 1 -- 172.21.15.148:0/620056643 >> v2:172.21.15.161:3300/0 conn(0x7fd0a81523e0 msgr2=0x7fd0a806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a759e700 1 --2- 172.21.15.148:0/620056643 >> v2:172.21.15.161:3300/0 conn(0x7fd0a81523e0 0x7fd0a806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a759e700 1 -- 172.21.15.148:0/620056643 >> v1:172.21.15.148:6789/0 conn(0x7fd0a8152010 legacy=0x7fd0a815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a759e700 1 -- 172.21.15.148:0/620056643 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a815c340 con 0x7fd0a8158e50 2023-10-10T20:24:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a659c700 1 -- 172.21.15.148:0/620056643 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187928681 0 0) 0x7fd0a806c010 con 0x7fd0a8152010 2023-10-10T20:24:40.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a759e700 1 --2- 172.21.15.148:0/620056643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a8158e50 0x7fd0a80cdf30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80ceae0 tx=0x7fd0a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a22a6584925c8bb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:40.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a659c700 1 -- 172.21.15.148:0/620056643 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a000e070 con 0x7fd0a8158e50 2023-10-10T20:24:40.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a659c700 1 -- 172.21.15.148:0/620056643 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a000bb60 con 0x7fd0a8158e50 2023-10-10T20:24:40.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0a659c700 1 -- 172.21.15.148:0/620056643 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0012430 con 0x7fd0a8158e50 2023-10-10T20:24:40.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/620056643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a8158e50 msgr2=0x7fd0a80cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/620056643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a8158e50 0x7fd0a80cdf30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80ceae0 tx=0x7fd0a0009780 comp rx=0 tx=0).stop 2023-10-10T20:24:40.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/620056643 shutdown_connections 2023-10-10T20:24:40.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/620056643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a8158e50 0x7fd0a80cdf30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/620056643 >> v2:172.21.15.161:3300/0 conn(0x7fd0a81523e0 0x7fd0a806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/620056643 >> 172.21.15.148:0/620056643 conn(0x7fd0a8069870 msgr2=0x7fd0a8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:40.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/620056643 shutdown_connections 2023-10-10T20:24:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.317+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/620056643 wait complete. 2023-10-10T20:24:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 Processor -- start 2023-10-10T20:24:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 -- start start 2023-10-10T20:24:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 0x7fd0a814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a8158e50 0x7fd0a80c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8159fd0 con 0x7fd0a81523e0 2023-10-10T20:24:40.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81593c0 con 0x7fd0a8152010 2023-10-10T20:24:40.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd0ad1c5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81596a0 con 0x7fd0a8158e50 2023-10-10T20:24:40.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.319+0000 7fd09e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58436/0 (socket says 172.21.15.148:58436) 2023-10-10T20:24:40.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 -- 172.21.15.148:0/4241400 learned_addr learned my addr 172.21.15.148:0/4241400 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:40.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd0a6d9d700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 0x7fd0a814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd0a759e700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a8158e50 0x7fd0a80c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 -- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a8158e50 msgr2=0x7fd0a80c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a8158e50 0x7fd0a80c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 -- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 msgr2=0x7fd0a814b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 0x7fd0a814b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 -- 172.21.15.148:0/4241400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0009a20 con 0x7fd0a81523e0 2023-10-10T20:24:40.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd0a6d9d700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 0x7fd0a814b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:40.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09e59c700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd09000ec80 tx=0x7fd0900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:40.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000c9d0 con 0x7fd0a81523e0 2023-10-10T20:24:40.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09000cb30 con 0x7fd0a81523e0 2023-10-10T20:24:40.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a8159980 con 0x7fd0a81523e0 2023-10-10T20:24:40.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090013470 con 0x7fd0a81523e0 2023-10-10T20:24:40.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.320+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a814b590 con 0x7fd0a81523e0 2023-10-10T20:24:40.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.321+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a815a140 con 0x7fd0a81523e0 2023-10-10T20:24:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.321+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd090013820 con 0x7fd0a81523e0 2023-10-10T20:24:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.322+0000 7fd09f7fe700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 0x7fd0880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.322+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0900848a0 con 0x7fd0a81523e0 2023-10-10T20:24:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.322+0000 7fd0a6d9d700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 0x7fd0880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:40.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.322+0000 7fd0a6d9d700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 0x7fd0880693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80c7e70 tx=0x7fd098009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:40.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.326+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd090051b10 con 0x7fd0a81523e0 2023-10-10T20:24:40.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.550+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0a804c3c0 con 0x7fd0a81523e0 2023-10-10T20:24:40.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.550+0000 7fd09f7fe700 1 -- 172.21.15.148:0/4241400 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fd090055130 con 0x7fd0a81523e0 2023-10-10T20:24:40.553 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:40.553 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 msgr2=0x7fd0880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 0x7fd0880693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80c7e70 tx=0x7fd098009040 comp rx=0 tx=0).stop 2023-10-10T20:24:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 msgr2=0x7fd0a815a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd09000ec80 tx=0x7fd0900091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 shutdown_connections 2023-10-10T20:24:40.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd088066f30 0x7fd0880693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a8158e50 0x7fd0a80c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a8152010 0x7fd0a814b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 --2- 172.21.15.148:0/4241400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a81523e0 0x7fd0a815a460 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:40.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 >> 172.21.15.148:0/4241400 conn(0x7fd0a8069870 msgr2=0x7fd0a814c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:40.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 shutdown_connections 2023-10-10T20:24:40.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:40.552+0000 7fd0ad1c5700 1 -- 172.21.15.148:0/4241400 wait complete. 2023-10-10T20:24:40.557 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:40.567 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:24:40.715 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":6381,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.447076413000001,"mdsmap_epoch":49,"osdmap_epoch":104,"osdmap_epoch_barrier":101,"uptime":36.459885364999998} 2023-10-10T20:24:40.715 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 6381, "mdsmap_epoch": 49, "osdmap_epoch": 104, "osdmap_epoch_barrier": 101, "rank_uptime": 31.447076413, "state": "up:active", "uptime": 36.459885365, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:24:40.716 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:24:40.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:40.717 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-10T20:24:40.998 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:24:40.998 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:24:40.999 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.264397 s, 198 MB/s 2023-10-10T20:24:40.999 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:24:41.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:41.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:41.078 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:24:41.079 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:24:41.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:41.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:24:41.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:24:41.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:41.145 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:24:41.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:24:41.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:24:41.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:24:41.200 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:24:41.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:24:41.200 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:24:41.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:41.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.367+0000 7fb5e2adc700 1 Processor -- start 2023-10-10T20:24:41.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.367+0000 7fb5e2adc700 1 -- start start 2023-10-10T20:24:41.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e2adc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5dc1568d0 0x7fb5dc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e2adc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5dc06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e2adc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc067ba0 con 0x7fb5dc156ca0 2023-10-10T20:24:41.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e2adc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc15aa90 con 0x7fb5dc1568d0 2023-10-10T20:24:41.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e2adc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc067870 con 0x7fb5dc0ad9b0 2023-10-10T20:24:41.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e1079700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb5dc156ca0 0x7fb5dc0c8090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46342/0 (socket says 172.21.15.148:46342) 2023-10-10T20:24:41.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.368+0000 7fb5e1079700 1 -- 172.21.15.148:0/2941445543 learned_addr learned my addr 172.21.15.148:0/2941445543 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:41.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947163341 0 0) 0x7fb5dc067ba0 con 0x7fb5dc156ca0 2023-10-10T20:24:41.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5dc06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5e0878700 1 --2- 172.21.15.148:0/2941445543 >> v2:172.21.15.161:3300/0 conn(0x7fb5dc1568d0 0x7fb5dc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c4003570 con 0x7fb5dc156ca0 2023-10-10T20:24:41.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235518648 0 0) 0x7fb5c4003570 con 0x7fb5dc156ca0 2023-10-10T20:24:41.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 msgr2=0x7fb5dc06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:41.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5dc06a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 >> v2:172.21.15.161:3300/0 conn(0x7fb5dc1568d0 msgr2=0x7fb5dc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:41.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 --2- 172.21.15.148:0/2941445543 >> v2:172.21.15.161:3300/0 conn(0x7fb5dc1568d0 0x7fb5dc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0c8ca0 con 0x7fb5dc156ca0 2023-10-10T20:24:41.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fb5d40017f0 con 0x7fb5dc156ca0 2023-10-10T20:24:41.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 >> v1:172.21.15.148:6789/0 conn(0x7fb5dc156ca0 legacy=0x7fb5dc0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5c4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c4003fe0 0x7fb5c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4004d50 0x7fb5c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc15aa90 con 0x7fb5c4003fe0 2023-10-10T20:24:41.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc067870 con 0x7fb5c4004d50 2023-10-10T20:24:41.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c400ba90 con 0x7fb5dc0ad9b0 2023-10-10T20:24:41.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c4003fe0 0x7fb5c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5e0878700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5c4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5e1079700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4004d50 0x7fb5c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 -- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 msgr2=0x7fb5c4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:41.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ad9b0 0x7fb5c4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 -- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4004d50 msgr2=0x7fb5c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:41.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4004d50 0x7fb5c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 -- 172.21.15.148:0/2941445543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4003570 con 0x7fb5c4003fe0 2023-10-10T20:24:41.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.369+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c4003fe0 0x7fb5c4004810 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0ade00 tx=0x7fb5cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba677384d22db72 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00e6b0 con 0x7fb5c4003fe0 2023-10-10T20:24:41.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00e810 con 0x7fb5c4003fe0 2023-10-10T20:24:41.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5d37fe700 1 -- 172.21.15.148:0/2941445543 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00ebd0 con 0x7fb5c4003fe0 2023-10-10T20:24:41.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 -- 172.21.15.148:0/2941445543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c4003fe0 msgr2=0x7fb5c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/2941445543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5c4003fe0 0x7fb5c4004810 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0ade00 tx=0x7fb5cc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e1079700 1 -- 172.21.15.148:0/2941445543 reap_dead start 2023-10-10T20:24:41.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 -- 172.21.15.148:0/2941445543 shutdown_connections 2023-10-10T20:24:41.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 -- 172.21.15.148:0/2941445543 >> 172.21.15.148:0/2941445543 conn(0x7fb5dc0cda50 msgr2=0x7fb5dc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:41.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 -- 172.21.15.148:0/2941445543 shutdown_connections 2023-10-10T20:24:41.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.370+0000 7fb5e2adc700 1 -- 172.21.15.148:0/2941445543 wait complete. 2023-10-10T20:24:41.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.371+0000 7fb5e2adc700 1 Processor -- start 2023-10-10T20:24:41.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.371+0000 7fb5e2adc700 1 -- start start 2023-10-10T20:24:41.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 0x7fb5dc0c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ade30 0x7fb5dc0ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0ae810 con 0x7fb5dc0b0940 2023-10-10T20:24:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0b2560 con 0x7fb5c4003fe0 2023-10-10T20:24:41.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e2adc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0b1c70 con 0x7fb5dc0ade30 2023-10-10T20:24:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5d3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5d3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58460/0 (socket says 172.21.15.148:58460) 2023-10-10T20:24:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5d3fff700 1 -- 172.21.15.148:0/4087906133 learned_addr learned my addr 172.21.15.148:0/4087906133 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e1079700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ade30 0x7fb5dc0ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.372+0000 7fb5e0878700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 0x7fb5dc0c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 -- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ade30 msgr2=0x7fb5dc0ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ade30 0x7fb5dc0ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 -- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 msgr2=0x7fb5dc0c14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 0x7fb5dc0c14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 -- 172.21.15.148:0/4087906133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0b0e70 con 0x7fb5dc0b0940 2023-10-10T20:24:41.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5e0878700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 0x7fb5dc0c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:41.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d3fff700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004730 tx=0x7fb5cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc023070 con 0x7fb5dc0b0940 2023-10-10T20:24:41.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00e980 con 0x7fb5dc0b0940 2023-10-10T20:24:41.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc01e030 con 0x7fb5dc0b0940 2023-10-10T20:24:41.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc0b1f10 con 0x7fb5dc0b0940 2023-10-10T20:24:41.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.373+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc152340 con 0x7fb5dc0b0940 2023-10-10T20:24:41.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.374+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc0b26f0 con 0x7fb5dc0b0940 2023-10-10T20:24:41.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.374+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5cc013470 con 0x7fb5dc0b0940 2023-10-10T20:24:41.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.375+0000 7fb5d0ff9700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.375+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb5cc085b30 con 0x7fb5dc0b0940 2023-10-10T20:24:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.375+0000 7fb5e0878700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.375+0000 7fb5e0878700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 0x7fb5c80693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006f00 tx=0x7fb5d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.379+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5cc052da0 con 0x7fb5dc0b0940 2023-10-10T20:24:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.603+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5dc04c3c0 con 0x7fb5dc0b0940 2023-10-10T20:24:41.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.604+0000 7fb5d0ff9700 1 -- 172.21.15.148:0/4087906133 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fb5cc017090 con 0x7fb5dc0b0940 2023-10-10T20:24:41.607 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:41.607 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 msgr2=0x7fb5c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 0x7fb5c80693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006f00 tx=0x7fb5d8008040 comp rx=0 tx=0).stop 2023-10-10T20:24:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 msgr2=0x7fb5dc151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004730 tx=0x7fb5cc01a000 comp rx=0 tx=0).stop 2023-10-10T20:24:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 shutdown_connections 2023-10-10T20:24:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5dc0ade30 0x7fb5dc0ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5c4003fe0 0x7fb5dc0c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 --2- 172.21.15.148:0/4087906133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0b0940 0x7fb5dc151e00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 >> 172.21.15.148:0/4087906133 conn(0x7fb5dc0cda50 msgr2=0x7fb5dc0cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 shutdown_connections 2023-10-10T20:24:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.606+0000 7fb5e2adc700 1 -- 172.21.15.148:0/4087906133 wait complete. 2023-10-10T20:24:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:41.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:41.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.734+0000 7f2985404700 1 Processor -- start 2023-10-10T20:24:41.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.734+0000 7f2985404700 1 -- start start 2023-10-10T20:24:41.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f2985404700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 0x7f298006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f2985404700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f298006ef80 0x7f29800aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f2985404700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800aab30 con 0x7f29800b2700 2023-10-10T20:24:41.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f2985404700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29800b31a0 con 0x7f298006ef80 2023-10-10T20:24:41.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f2985404700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298006f3b0 con 0x7f29800b2ad0 2023-10-10T20:24:41.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f298006ef80 0x7f29800aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297effd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f29800b2700 0x7f29800af290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46350/0 (socket says 172.21.15.148:46350) 2023-10-10T20:24:41.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f298006ef80 0x7f29800aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58474/0 (socket says 172.21.15.148:58474) 2023-10-10T20:24:41.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297effd700 1 -- 172.21.15.148:0/1056369720 learned_addr learned my addr 172.21.15.148:0/1056369720 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:41.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f29767fc700 1 --2- 172.21.15.148:0/1056369720 >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 0x7f298006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 -- 172.21.15.148:0/1056369720 >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 msgr2=0x7f298006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:41.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 --2- 172.21.15.148:0/1056369720 >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 0x7f298006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 -- 172.21.15.148:0/1056369720 >> v1:172.21.15.148:6789/0 conn(0x7f29800b2700 legacy=0x7f29800af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297e7fc700 1 -- 172.21.15.148:0/1056369720 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709438344 0 0) 0x7f29800aab30 con 0x7f29800b2700 2023-10-10T20:24:41.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f297f7fe700 1 -- 172.21.15.148:0/1056369720 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800a9f20 con 0x7f298006ef80 2023-10-10T20:24:41.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.735+0000 7f29767fc700 1 --2- 172.21.15.148:0/1056369720 >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 0x7f298006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:41.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f297f7fe700 1 --2- 172.21.15.148:0/1056369720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f29800aafc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f29800ce380 tx=0x7f2978009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae7fea5fe61ff04 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f297e7fc700 1 -- 172.21.15.148:0/1056369720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800e070 con 0x7f298006ef80 2023-10-10T20:24:41.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f297e7fc700 1 -- 172.21.15.148:0/1056369720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800bc30 con 0x7f298006ef80 2023-10-10T20:24:41.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f297e7fc700 1 -- 172.21.15.148:0/1056369720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978012430 con 0x7f298006ef80 2023-10-10T20:24:41.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 -- 172.21.15.148:0/1056369720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 msgr2=0x7f29800aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 --2- 172.21.15.148:0/1056369720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f29800aafc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f29800ce380 tx=0x7f2978009780 comp rx=0 tx=0).stop 2023-10-10T20:24:41.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 -- 172.21.15.148:0/1056369720 shutdown_connections 2023-10-10T20:24:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 --2- 172.21.15.148:0/1056369720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f29800aafc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 --2- 172.21.15.148:0/1056369720 >> v2:172.21.15.161:3300/0 conn(0x7f29800b2ad0 0x7f298006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 -- 172.21.15.148:0/1056369720 >> 172.21.15.148:0/1056369720 conn(0x7f29800cd270 msgr2=0x7f29800cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 -- 172.21.15.148:0/1056369720 shutdown_connections 2023-10-10T20:24:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.736+0000 7f2985404700 1 -- 172.21.15.148:0/1056369720 wait complete. 2023-10-10T20:24:41.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 Processor -- start 2023-10-10T20:24:41.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 -- start start 2023-10-10T20:24:41.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29800b2700 0x7f298014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29800b2ad0 0x7f29801572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980159d50 con 0x7f298006ef80 2023-10-10T20:24:41.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298014c360 con 0x7f29800b2ad0 2023-10-10T20:24:41.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f2985404700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298014c640 con 0x7f29800b2700 2023-10-10T20:24:41.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f297effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f297effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58490/0 (socket says 172.21.15.148:58490) 2023-10-10T20:24:41.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.738+0000 7f297effd700 1 -- 172.21.15.148:0/2274620112 learned_addr learned my addr 172.21.15.148:0/2274620112 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:41.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 -- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29800b2700 msgr2=0x7f298014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29800b2700 0x7f298014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 -- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29800b2ad0 msgr2=0x7f29801572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29800b2ad0 0x7f29801572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 -- 172.21.15.148:0/2274620112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978009a20 con 0x7f298006ef80 2023-10-10T20:24:41.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f297effd700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f297000c7e0 tx=0x7f2970007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970005b30 con 0x7f298006ef80 2023-10-10T20:24:41.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970005c90 con 0x7f298006ef80 2023-10-10T20:24:41.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298014c920 con 0x7f298006ef80 2023-10-10T20:24:41.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970007300 con 0x7f298006ef80 2023-10-10T20:24:41.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.739+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298014cee0 con 0x7f298006ef80 2023-10-10T20:24:41.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.740+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298014d100 con 0x7f298006ef80 2023-10-10T20:24:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.740+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29700076b0 con 0x7f298006ef80 2023-10-10T20:24:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.741+0000 7f29777fe700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 0x7f29600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.741+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2970083fd0 con 0x7f298006ef80 2023-10-10T20:24:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.741+0000 7f29767fc700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 0x7f29600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.741+0000 7f29767fc700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 0x7f29600693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2968006fd0 tx=0x7f2968008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:41.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.745+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29700512c0 con 0x7f298006ef80 2023-10-10T20:24:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.929+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29800aa200 con 0x7f298006ef80 2023-10-10T20:24:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.930+0000 7f29777fe700 1 -- 172.21.15.148:0/2274620112 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+15405 (secure 0 0 0) 0x7f29700548e0 con 0x7f298006ef80 2023-10-10T20:24:41.932 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:41.933 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":104,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:34.609051+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 msgr2=0x7f29600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 0x7f29600693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2968006fd0 tx=0x7f2968008040 comp rx=0 tx=0).stop 2023-10-10T20:24:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 msgr2=0x7f2980159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f297000c7e0 tx=0x7f2970007df0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 shutdown_connections 2023-10-10T20:24:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2960066f30 0x7f29600693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29800b2ad0 0x7f29801572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298006ef80 0x7f2980159810 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 --2- 172.21.15.148:0/2274620112 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29800b2700 0x7f298014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 >> 172.21.15.148:0/2274620112 conn(0x7f29800cd270 msgr2=0x7f2980069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 shutdown_connections 2023-10-10T20:24:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:41.932+0000 7f2985404700 1 -- 172.21.15.148:0/2274620112 wait complete. 2023-10-10T20:24:41.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:24:06.952553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:24:07.573984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:24:41.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:24:42.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.062+0000 7ffbf2327700 1 Processor -- start 2023-10-10T20:24:42.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.062+0000 7ffbf2327700 1 -- start start 2023-10-10T20:24:42.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf2327700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 0x7ffbec0aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf2327700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbec0b03e0 0x7ffbec0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf2327700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec06a2b0 con 0x7ffbec159bb0 2023-10-10T20:24:42.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf2327700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec0b0d20 con 0x7ffbec0b03e0 2023-10-10T20:24:42.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf2327700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec069040 con 0x7ffbec0b17b0 2023-10-10T20:24:42.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf1b26700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbec0b03e0 0x7ffbec0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.063+0000 7ffbf1325700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffbec159bb0 0x7ffbec15e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46358/0 (socket says 172.21.15.148:46358) 2023-10-10T20:24:42.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbec0b03e0 0x7ffbec0b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58498/0 (socket says 172.21.15.148:58498) 2023-10-10T20:24:42.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1325700 1 -- 172.21.15.148:0/2971460306 learned_addr learned my addr 172.21.15.148:0/2971460306 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:42.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/2971460306 >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 0x7ffbec0aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 -- 172.21.15.148:0/2971460306 >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 msgr2=0x7ffbec0aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:42.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 --2- 172.21.15.148:0/2971460306 >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 0x7ffbec0aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 -- 172.21.15.148:0/2971460306 >> v1:172.21.15.148:6789/0 conn(0x7ffbec159bb0 legacy=0x7ffbec15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 -- 172.21.15.148:0/2971460306 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec069320 con 0x7ffbec0b03e0 2023-10-10T20:24:42.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbe3fff700 1 -- 172.21.15.148:0/2971460306 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715256559 0 0) 0x7ffbec06a2b0 con 0x7ffbec159bb0 2023-10-10T20:24:42.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/2971460306 >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 0x7ffbec0aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:42.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbf1b26700 1 --2- 172.21.15.148:0/2971460306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0b07e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffbec06ec90 tx=0x7ffbe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4ce73d3c90c804 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbe3fff700 1 -- 172.21.15.148:0/2971460306 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400b6b0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbe3fff700 1 -- 172.21.15.148:0/2971460306 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe400b810 con 0x7ffbec0b03e0 2023-10-10T20:24:42.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.064+0000 7ffbe3fff700 1 -- 172.21.15.148:0/2971460306 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400bbb0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 -- 172.21.15.148:0/2971460306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 msgr2=0x7ffbec0b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 --2- 172.21.15.148:0/2971460306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0b07e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffbec06ec90 tx=0x7ffbe40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 -- 172.21.15.148:0/2971460306 shutdown_connections 2023-10-10T20:24:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 --2- 172.21.15.148:0/2971460306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0b07e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 --2- 172.21.15.148:0/2971460306 >> v2:172.21.15.161:3300/0 conn(0x7ffbec0b17b0 0x7ffbec0aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 -- 172.21.15.148:0/2971460306 >> 172.21.15.148:0/2971460306 conn(0x7ffbec06c700 msgr2=0x7ffbec069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 -- 172.21.15.148:0/2971460306 shutdown_connections 2023-10-10T20:24:42.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.065+0000 7ffbf2327700 1 -- 172.21.15.148:0/2971460306 wait complete. 2023-10-10T20:24:42.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.066+0000 7ffbf2327700 1 Processor -- start 2023-10-10T20:24:42.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.066+0000 7ffbf2327700 1 -- start start 2023-10-10T20:24:42.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 0x7ffbec153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbec159bb0 0x7ffbec1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec159f80 con 0x7ffbec0b03e0 2023-10-10T20:24:42.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec153570 con 0x7ffbec0b17b0 2023-10-10T20:24:42.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0acef0 con 0x7ffbec159bb0 2023-10-10T20:24:42.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58512/0 (socket says 172.21.15.148:58512) 2023-10-10T20:24:42.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 -- 172.21.15.148:0/1322087077 learned_addr learned my addr 172.21.15.148:0/1322087077 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:42.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 0x7ffbec153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1b26700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbec159bb0 0x7ffbec1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 -- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbec159bb0 msgr2=0x7ffbec1494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbec159bb0 0x7ffbec1494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 -- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 msgr2=0x7ffbec153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 0x7ffbec153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 -- 172.21.15.148:0/1322087077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4009420 con 0x7ffbec0b03e0 2023-10-10T20:24:42.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 0x7ffbec153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:42.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf1325700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbe800cca0 tx=0x7ffbe8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe800ab30 con 0x7ffbec0b03e0 2023-10-10T20:24:42.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe800ac90 con 0x7ffbec0b03e0 2023-10-10T20:24:42.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec0ad1d0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8007300 con 0x7ffbec0b03e0 2023-10-10T20:24:42.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.067+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec15a4b0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.068+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec15a6a0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.069+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbe80076f0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.069+0000 7ffbe17fa700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 0x7ffbd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.069+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffbe80849a0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.069+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 0x7ffbd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.070+0000 7ffbf0b24700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 0x7ffbd00693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc006f00 tx=0x7ffbdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.073+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbe8051c90 con 0x7ffbec0b03e0 2023-10-10T20:24:42.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.253+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbec069600 con 0x7ffbec0b03e0 2023-10-10T20:24:42.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.253+0000 7ffbe17fa700 1 -- 172.21.15.148:0/1322087077 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbe80552b0 con 0x7ffbec0b03e0 2023-10-10T20:24:42.256 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:24:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.255+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 msgr2=0x7ffbd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.255+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 0x7ffbd00693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc006f00 tx=0x7ffbdc008040 comp rx=0 tx=0).stop 2023-10-10T20:24:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.255+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 msgr2=0x7ffbec0ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.255+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffbe800cca0 tx=0x7ffbe8001a30 comp rx=0 tx=0).stop 2023-10-10T20:24:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 shutdown_connections 2023-10-10T20:24:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffbd0066f30 0x7ffbd00693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbec159bb0 0x7ffbec1494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbec0b03e0 0x7ffbec0ac860 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 --2- 172.21.15.148:0/1322087077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbec0b17b0 0x7ffbec153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 >> 172.21.15.148:0/1322087077 conn(0x7ffbec06c700 msgr2=0x7ffbec149e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 shutdown_connections 2023-10-10T20:24:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.256+0000 7ffbf2327700 1 -- 172.21.15.148:0/1322087077 wait complete. 2023-10-10T20:24:42.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:24:42.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.384+0000 7f32ccffd700 1 Processor -- start 2023-10-10T20:24:42.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.384+0000 7f32ccffd700 1 -- start start 2023-10-10T20:24:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32ccffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32c80d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32ccffd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c81513f0 con 0x7f32c80cd300 2023-10-10T20:24:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32ccffd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c80d1700 con 0x7f32c80d12a0 2023-10-10T20:24:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32ccffd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815e7a0 con 0x7f32c815afb0 2023-10-10T20:24:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32c659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32c80cd300 0x7f32c80b6760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46366/0 (socket says 172.21.15.148:46366) 2023-10-10T20:24:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32c659c700 1 -- 172.21.15.148:0/3761117797 learned_addr learned my addr 172.21.15.148:0/3761117797 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327651635 0 0) 0x7f32c81513f0 con 0x7f32c80cd300 2023-10-10T20:24:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b4003570 con 0x7f32c80cd300 2023-10-10T20:24:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.385+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32c80d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258833763 0 0) 0x7f32b4003570 con 0x7f32c80cd300 2023-10-10T20:24:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 msgr2=0x7f32c80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32c80d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 >> v1:172.21.15.161:6789/0 conn(0x7f32c80d12a0 legacy=0x7f32c80cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815b400 con 0x7f32c80cd300 2023-10-10T20:24:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f32b8001470 con 0x7f32c80cd300 2023-10-10T20:24:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 >> v1:172.21.15.148:6789/0 conn(0x7f32c80cd300 legacy=0x7f32c80b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32b4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32b4004010 0x7f32b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b4004d50 0x7f32b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815e7a0 con 0x7f32b4004010 2023-10-10T20:24:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b400bbf0 con 0x7f32b4004d50 2023-10-10T20:24:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b400bdb0 con 0x7f32c815afb0 2023-10-10T20:24:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32b4004010 0x7f32b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c659c700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32b4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c6d9d700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b4004d50 0x7f32b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 -- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 msgr2=0x7f32b4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:42.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c815afb0 0x7f32b4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 -- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b4004d50 msgr2=0x7f32b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:42.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b4004d50 0x7f32b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 -- 172.21.15.148:0/3761117797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4003570 con 0x7f32b4004010 2023-10-10T20:24:42.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32b4004010 0x7f32b4004810 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006f00 tx=0x7f32b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c73cb243a6e2df4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000e710 con 0x7f32b4004010 2023-10-10T20:24:42.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b000e870 con 0x7f32b4004010 2023-10-10T20:24:42.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.386+0000 7f32c5d9b700 1 -- 172.21.15.148:0/3761117797 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000ec30 con 0x7f32b4004010 2023-10-10T20:24:42.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 -- 172.21.15.148:0/3761117797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32b4004010 msgr2=0x7f32b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3761117797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32b4004010 0x7f32b4004810 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006f00 tx=0x7f32b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32c6d9d700 1 -- 172.21.15.148:0/3761117797 reap_dead start 2023-10-10T20:24:42.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 -- 172.21.15.148:0/3761117797 shutdown_connections 2023-10-10T20:24:42.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 -- 172.21.15.148:0/3761117797 >> 172.21.15.148:0/3761117797 conn(0x7f32c8068d90 msgr2=0x7f32c8069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 -- 172.21.15.148:0/3761117797 shutdown_connections 2023-10-10T20:24:42.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.387+0000 7f32ccffd700 1 -- 172.21.15.148:0/3761117797 wait complete. 2023-10-10T20:24:42.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.388+0000 7f32ccffd700 1 Processor -- start 2023-10-10T20:24:42.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.388+0000 7f32ccffd700 1 -- start start 2023-10-10T20:24:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8143c60 0x7f32c816d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c816df00 0x7f32c8150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8152230 con 0x7f32c8143890 2023-10-10T20:24:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81531e0 con 0x7f32c8143c60 2023-10-10T20:24:42.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32ccffd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c816e4f0 con 0x7f32c816df00 2023-10-10T20:24:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58522/0 (socket says 172.21.15.148:58522) 2023-10-10T20:24:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 -- 172.21.15.148:0/3754649050 learned_addr learned my addr 172.21.15.148:0/3754649050 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 -- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c816df00 msgr2=0x7f32c8150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c816df00 0x7f32c8150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 -- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8143c60 msgr2=0x7f32c816d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8143c60 0x7f32c816d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.389+0000 7f32c659c700 1 -- 172.21.15.148:0/3754649050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c81524a0 con 0x7f32c8143890 2023-10-10T20:24:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32c659c700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f32b8004890 tx=0x7f32c8154ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8018030 con 0x7f32c8143890 2023-10-10T20:24:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b800f8d0 con 0x7f32c8143890 2023-10-10T20:24:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800fc70 con 0x7f32c8143890 2023-10-10T20:24:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c8152780 con 0x7f32c8143890 2023-10-10T20:24:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.390+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8150880 con 0x7f32c8143890 2023-10-10T20:24:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.391+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c81529c0 con 0x7f32c8143890 2023-10-10T20:24:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.391+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b801b070 con 0x7f32c8143890 2023-10-10T20:24:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.391+0000 7f32beffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.391+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32b8019070 con 0x7f32c8143890 2023-10-10T20:24:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.391+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.392+0000 7f32bdd9b700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32c804d2d0 tx=0x7f32b001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.396+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c81529c0 con 0x7f32c8143890 2023-10-10T20:24:42.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.575+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32c81529c0 con 0x7f32c8143890 2023-10-10T20:24:42.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.576+0000 7f32beffd700 1 -- 172.21.15.148:0/3754649050 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32b8055300 con 0x7f32c8143890 2023-10-10T20:24:42.578 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:24:42.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 msgr2=0x7f32a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f32c804d2d0 tx=0x7f32b001a000 comp rx=0 tx=0).stop 2023-10-10T20:24:42.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 msgr2=0x7f32c8152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f32b8004890 tx=0x7f32c8154ac0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 shutdown_connections 2023-10-10T20:24:42.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32c816df00 0x7f32c8150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c8143890 0x7f32c8152ca0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 --2- 172.21.15.148:0/3754649050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8143c60 0x7f32c816d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 >> 172.21.15.148:0/3754649050 conn(0x7f32c8068d90 msgr2=0x7f32c814fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 shutdown_connections 2023-10-10T20:24:42.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.578+0000 7f32ccffd700 1 -- 172.21.15.148:0/3754649050 wait complete. 2023-10-10T20:24:42.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:24:42.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.708+0000 7fd68b179700 1 Processor -- start 2023-10-10T20:24:42.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.708+0000 7fd68b179700 1 -- start start 2023-10-10T20:24:42.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd68b179700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 0x7fd6840af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd68b179700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd68415a290 0x7fd6840afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd68b179700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6840b0790 con 0x7fd68406a5a0 2023-10-10T20:24:42.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd68b179700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840ce1f0 con 0x7fd68415a290 2023-10-10T20:24:42.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd68b179700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840b0b20 con 0x7fd684159ec0 2023-10-10T20:24:42.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd689716700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd68406a5a0 0x7fd68415db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54196/0 (socket says 172.21.15.148:54196) 2023-10-10T20:24:42.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd689716700 1 -- 172.21.15.148:0/1151690856 learned_addr learned my addr 172.21.15.148:0/1151690856 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:42.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd688f15700 1 --2- 172.21.15.148:0/1151690856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 0x7fd6840af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd683fff700 1 --2- 172.21.15.148:0/1151690856 >> v2:172.21.15.161:3300/0 conn(0x7fd68415a290 0x7fd6840afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd6837fe700 1 -- 172.21.15.148:0/1151690856 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426878422 0 0) 0x7fd6840b0790 con 0x7fd68406a5a0 2023-10-10T20:24:42.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd6837fe700 1 -- 172.21.15.148:0/1151690856 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd670003570 con 0x7fd68406a5a0 2023-10-10T20:24:42.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd688f15700 1 -- 172.21.15.148:0/1151690856 >> v2:172.21.15.161:3300/0 conn(0x7fd68415a290 msgr2=0x7fd6840afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:42.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd688f15700 1 --2- 172.21.15.148:0/1151690856 >> v2:172.21.15.161:3300/0 conn(0x7fd68415a290 0x7fd6840afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd688f15700 1 -- 172.21.15.148:0/1151690856 >> v1:172.21.15.148:6789/0 conn(0x7fd68406a5a0 legacy=0x7fd68415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.709+0000 7fd688f15700 1 -- 172.21.15.148:0/1151690856 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840b04b0 con 0x7fd684159ec0 2023-10-10T20:24:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd688f15700 1 --2- 172.21.15.148:0/1151690856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 0x7fd6840af940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009e60 tx=0x7fd67c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682ce7fe17b3bc86 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd6837fe700 1 -- 172.21.15.148:0/1151690856 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00e070 con 0x7fd684159ec0 2023-10-10T20:24:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd6837fe700 1 -- 172.21.15.148:0/1151690856 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c00bab0 con 0x7fd684159ec0 2023-10-10T20:24:42.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd6837fe700 1 -- 172.21.15.148:0/1151690856 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00be70 con 0x7fd684159ec0 2023-10-10T20:24:42.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 -- 172.21.15.148:0/1151690856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 msgr2=0x7fd6840af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 --2- 172.21.15.148:0/1151690856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 0x7fd6840af940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009e60 tx=0x7fd67c009780 comp rx=0 tx=0).stop 2023-10-10T20:24:42.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 -- 172.21.15.148:0/1151690856 shutdown_connections 2023-10-10T20:24:42.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 --2- 172.21.15.148:0/1151690856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd684159ec0 0x7fd6840af940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 --2- 172.21.15.148:0/1151690856 >> v2:172.21.15.161:3300/0 conn(0x7fd68415a290 0x7fd6840afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 -- 172.21.15.148:0/1151690856 >> 172.21.15.148:0/1151690856 conn(0x7fd6840d1880 msgr2=0x7fd6840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 -- 172.21.15.148:0/1151690856 shutdown_connections 2023-10-10T20:24:42.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.710+0000 7fd68b179700 1 -- 172.21.15.148:0/1151690856 wait complete. 2023-10-10T20:24:42.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 Processor -- start 2023-10-10T20:24:42.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 -- start start 2023-10-10T20:24:42.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 0x7fd68416e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 0x7fd6840b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68415b750 con 0x7fd684159ec0 2023-10-10T20:24:42.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68415b4d0 con 0x7fd68406a5a0 2023-10-10T20:24:42.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.712+0000 7fd68b179700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840ca060 con 0x7fd68415a290 2023-10-10T20:24:42.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35178/0 (socket says 172.21.15.148:35178) 2023-10-10T20:24:42.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 -- 172.21.15.148:0/3829654155 learned_addr learned my addr 172.21.15.148:0/3829654155 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:42.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd689716700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 0x7fd6840b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd688f15700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 0x7fd68416e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 -- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 msgr2=0x7fd6840b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 0x7fd6840b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 -- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 msgr2=0x7fd68416e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 0x7fd68416e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 -- 172.21.15.148:0/3829654155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c009a20 con 0x7fd684159ec0 2023-10-10T20:24:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd688f15700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 0x7fd68416e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:42.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd689716700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 0x7fd6840b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:42.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd683fff700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd67400ecb0 tx=0x7fd6740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400c9d0 con 0x7fd684159ec0 2023-10-10T20:24:42.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67400cb30 con 0x7fd684159ec0 2023-10-10T20:24:42.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840ca340 con 0x7fd684159ec0 2023-10-10T20:24:42.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400ced0 con 0x7fd684159ec0 2023-10-10T20:24:42.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.713+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6840ca900 con 0x7fd684159ec0 2023-10-10T20:24:42.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.714+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68404c3c0 con 0x7fd684159ec0 2023-10-10T20:24:42.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.715+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd674013680 con 0x7fd684159ec0 2023-10-10T20:24:42.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.715+0000 7fd680ff9700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 0x7fd6640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:42.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.715+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6740848f0 con 0x7fd684159ec0 2023-10-10T20:24:42.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.715+0000 7fd688f15700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 0x7fd6640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:42.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.715+0000 7fd688f15700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 0x7fd6640693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0097c0 tx=0x7fd67c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:42.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.720+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd674051b60 con 0x7fd684159ec0 2023-10-10T20:24:42.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.902+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6840ceab0 con 0x7fd684159ec0 2023-10-10T20:24:42.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.903+0000 7fd680ff9700 1 -- 172.21.15.148:0/3829654155 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd674055180 con 0x7fd684159ec0 2023-10-10T20:24:42.905 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:24:42.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 msgr2=0x7fd6640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 0x7fd6640693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0097c0 tx=0x7fd67c019000 comp rx=0 tx=0).stop 2023-10-10T20:24:42.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 msgr2=0x7fd68415af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:42.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd67400ecb0 tx=0x7fd6740091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 shutdown_connections 2023-10-10T20:24:42.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd664066f30 0x7fd6640693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd68415a290 0x7fd6840b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd68406a5a0 0x7fd68416e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 --2- 172.21.15.148:0/3829654155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd684159ec0 0x7fd68415af90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:42.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 >> 172.21.15.148:0/3829654155 conn(0x7fd6840d1880 msgr2=0x7fd6840b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:42.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 shutdown_connections 2023-10-10T20:24:42.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:42.905+0000 7fd68b179700 1 -- 172.21.15.148:0/3829654155 wait complete. 2023-10-10T20:24:42.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:43.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 Processor -- start 2023-10-10T20:24:43.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 -- start start 2023-10-10T20:24:43.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 0x7f16980cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169815db80 con 0x7f1698159e10 2023-10-10T20:24:43.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169815e800 con 0x7f16980b3430 2023-10-10T20:24:43.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.031+0000 7f169d63d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169815de60 con 0x7f169815a1e0 2023-10-10T20:24:43.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f1696ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1698159e10 0x7f16980b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54204/0 (socket says 172.21.15.148:54204) 2023-10-10T20:24:43.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f1696ffd700 1 -- 172.21.15.148:0/3386881959 learned_addr learned my addr 172.21.15.148:0/3386881959 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:43.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 --2- 172.21.15.148:0/3386881959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16977fe700 1 --2- 172.21.15.148:0/3386881959 >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 0x7f16980cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16967fc700 1 -- 172.21.15.148:0/3386881959 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615273917 0 0) 0x7f169815db80 con 0x7f1698159e10 2023-10-10T20:24:43.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16967fc700 1 -- 172.21.15.148:0/3386881959 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1684003570 con 0x7f1698159e10 2023-10-10T20:24:43.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 -- 172.21.15.148:0/3386881959 >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 msgr2=0x7f16980cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:43.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 --2- 172.21.15.148:0/3386881959 >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 0x7f16980cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 -- 172.21.15.148:0/3386881959 >> v1:172.21.15.148:6789/0 conn(0x7f1698159e10 legacy=0x7f16980b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 -- 172.21.15.148:0/3386881959 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169815e140 con 0x7f169815a1e0 2023-10-10T20:24:43.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16977fe700 1 --2- 172.21.15.148:0/3386881959 >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 0x7f16980cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:43.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f168e7fc700 1 --2- 172.21.15.148:0/3386881959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980cd740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16800098e0 tx=0x7f16800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b615a10e33adc1a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16967fc700 1 -- 172.21.15.148:0/3386881959 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680011630 con 0x7f169815a1e0 2023-10-10T20:24:43.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16967fc700 1 -- 172.21.15.148:0/3386881959 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680011790 con 0x7f169815a1e0 2023-10-10T20:24:43.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.032+0000 7f16967fc700 1 -- 172.21.15.148:0/3386881959 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680011b30 con 0x7f169815a1e0 2023-10-10T20:24:43.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 -- 172.21.15.148:0/3386881959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 msgr2=0x7f16980cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 --2- 172.21.15.148:0/3386881959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980cd740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16800098e0 tx=0x7f16800091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 -- 172.21.15.148:0/3386881959 shutdown_connections 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 --2- 172.21.15.148:0/3386881959 >> v2:172.21.15.161:3300/0 conn(0x7f16980b3430 0x7f16980cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 --2- 172.21.15.148:0/3386881959 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980cd740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 -- 172.21.15.148:0/3386881959 >> 172.21.15.148:0/3386881959 conn(0x7f1698067660 msgr2=0x7f1698067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 -- 172.21.15.148:0/3386881959 shutdown_connections 2023-10-10T20:24:43.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.033+0000 7f169d63d700 1 -- 172.21.15.148:0/3386881959 wait complete. 2023-10-10T20:24:43.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.034+0000 7f169d63d700 1 Processor -- start 2023-10-10T20:24:43.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.034+0000 7f169d63d700 1 -- start start 2023-10-10T20:24:43.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 0x7f16980c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169815ca00 con 0x7f16980b3430 2023-10-10T20:24:43.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169815bdf0 con 0x7f1698159e10 2023-10-10T20:24:43.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f169d63d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169815c0d0 con 0x7f169815a1e0 2023-10-10T20:24:43.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35190/0 (socket says 172.21.15.148:35190) 2023-10-10T20:24:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 -- 172.21.15.148:0/1813161307 learned_addr learned my addr 172.21.15.148:0/1813161307 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f16977fe700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f168e7fc700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 0x7f16980c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 -- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 msgr2=0x7f16980c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 -- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 msgr2=0x7f16980c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 0x7f16980c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 -- 172.21.15.148:0/1813161307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009a20 con 0x7f16980b3430 2023-10-10T20:24:43.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f16977fe700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:43.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f168e7fc700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 0x7f16980c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:43.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.035+0000 7f1696ffd700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f168800c7c0 tx=0x7f1688007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.036+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688005b30 con 0x7f16980b3430 2023-10-10T20:24:43.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.036+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688005c90 con 0x7f16980b3430 2023-10-10T20:24:43.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.036+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680009420 con 0x7f16980b3430 2023-10-10T20:24:43.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.036+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688007340 con 0x7f16980b3430 2023-10-10T20:24:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.036+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169815ba70 con 0x7f16980b3430 2023-10-10T20:24:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.037+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169815c4c0 con 0x7f16980b3430 2023-10-10T20:24:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.037+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f168801b400 con 0x7f16980b3430 2023-10-10T20:24:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.037+0000 7f168f7fe700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 0x7f16780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.037+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1688084040 con 0x7f16980b3430 2023-10-10T20:24:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.037+0000 7f168e7fc700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 0x7f16780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.038+0000 7f168e7fc700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 0x7f16780693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f16980ceb20 tx=0x7f1680017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.042+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1688051330 con 0x7f16980b3430 2023-10-10T20:24:43.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.264+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f169815c290 con 0x7f16980b3430 2023-10-10T20:24:43.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.265+0000 7f168f7fe700 1 -- 172.21.15.148:0/1813161307 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f1688054950 con 0x7f16980b3430 2023-10-10T20:24:43.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:43.268 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:43.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 msgr2=0x7f16780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 0x7f16780693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f16980ceb20 tx=0x7f1680017000 comp rx=0 tx=0).stop 2023-10-10T20:24:43.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 msgr2=0x7f16980c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f168800c7c0 tx=0x7f1688007e00 comp rx=0 tx=0).stop 2023-10-10T20:24:43.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 shutdown_connections 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1678066f30 0x7f16780693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f169815a1e0 0x7f16980c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16980b3430 0x7f16980c4760 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 --2- 172.21.15.148:0/1813161307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1698159e10 0x7f16980c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 >> 172.21.15.148:0/1813161307 conn(0x7f1698067660 msgr2=0x7f1698159180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 shutdown_connections 2023-10-10T20:24:43.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.267+0000 7f169d63d700 1 -- 172.21.15.148:0/1813161307 wait complete. 2023-10-10T20:24:43.272 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:43.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:43.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.393+0000 7f50ff32c700 1 Processor -- start 2023-10-10T20:24:43.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.393+0000 7f50ff32c700 1 -- start start 2023-10-10T20:24:43.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50ff32c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f50f8159ef0 0x7f50f80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50ff32c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50f80cbca0 0x7f50f806be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50ff32c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f80c91d0 con 0x7f50f815a2c0 2023-10-10T20:24:43.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50ff32c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f815dc60 con 0x7f50f80cbca0 2023-10-10T20:24:43.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50ff32c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f815de70 con 0x7f50f8159ef0 2023-10-10T20:24:43.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50fd8c9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50f80cbca0 0x7f50f806be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50fd8c9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50f80cbca0 0x7f50f806be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35192/0 (socket says 172.21.15.148:35192) 2023-10-10T20:24:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.394+0000 7f50fd8c9700 1 -- 172.21.15.148:0/2974532022 learned_addr learned my addr 172.21.15.148:0/2974532022 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd0c8700 1 --2- 172.21.15.148:0/2974532022 >> v2:172.21.15.161:3300/0 conn(0x7f50f8159ef0 0x7f50f80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd8c9700 1 -- 172.21.15.148:0/2974532022 >> v2:172.21.15.161:3300/0 conn(0x7f50f8159ef0 msgr2=0x7f50f80ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd8c9700 1 --2- 172.21.15.148:0/2974532022 >> v2:172.21.15.161:3300/0 conn(0x7f50f8159ef0 0x7f50f80ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd8c9700 1 -- 172.21.15.148:0/2974532022 >> v1:172.21.15.161:6789/0 conn(0x7f50f815a2c0 legacy=0x7f50f80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd8c9700 1 -- 172.21.15.148:0/2974532022 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f815e150 con 0x7f50f80cbca0 2023-10-10T20:24:43.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50fd8c9700 1 --2- 172.21.15.148:0/2974532022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f80cbca0 0x7f50f806be20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f50f806a8c0 tx=0x7f50f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9145128459df88 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50effff700 1 -- 172.21.15.148:0/2974532022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000e070 con 0x7f50f80cbca0 2023-10-10T20:24:43.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50effff700 1 -- 172.21.15.148:0/2974532022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000ba20 con 0x7f50f80cbca0 2023-10-10T20:24:43.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.395+0000 7f50effff700 1 -- 172.21.15.148:0/2974532022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00092e0 con 0x7f50f80cbca0 2023-10-10T20:24:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 -- 172.21.15.148:0/2974532022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f80cbca0 msgr2=0x7f50f806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 --2- 172.21.15.148:0/2974532022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f80cbca0 0x7f50f806be20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f50f806a8c0 tx=0x7f50f0009840 comp rx=0 tx=0).stop 2023-10-10T20:24:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 -- 172.21.15.148:0/2974532022 shutdown_connections 2023-10-10T20:24:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 --2- 172.21.15.148:0/2974532022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f80cbca0 0x7f50f806be20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 --2- 172.21.15.148:0/2974532022 >> v2:172.21.15.161:3300/0 conn(0x7f50f8159ef0 0x7f50f80ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 -- 172.21.15.148:0/2974532022 >> 172.21.15.148:0/2974532022 conn(0x7f50f8069870 msgr2=0x7f50f8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 -- 172.21.15.148:0/2974532022 shutdown_connections 2023-10-10T20:24:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.396+0000 7f50ff32c700 1 -- 172.21.15.148:0/2974532022 wait complete. 2023-10-10T20:24:43.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.397+0000 7f50ff32c700 1 Processor -- start 2023-10-10T20:24:43.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.397+0000 7f50ff32c700 1 -- start start 2023-10-10T20:24:43.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50f80cbca0 0x7f50f8153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50f8159ef0 0x7f50f80c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8155f70 con 0x7f50f815a2c0 2023-10-10T20:24:43.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8155360 con 0x7f50f8159ef0 2023-10-10T20:24:43.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8155640 con 0x7f50f80cbca0 2023-10-10T20:24:43.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35194/0 (socket says 172.21.15.148:35194) 2023-10-10T20:24:43.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 -- 172.21.15.148:0/1231617119 learned_addr learned my addr 172.21.15.148:0/1231617119 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:43.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fc8c7700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50f8159ef0 0x7f50f80c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd0c8700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50f80cbca0 0x7f50f8153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 -- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50f80cbca0 msgr2=0x7f50f8153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50f80cbca0 0x7f50f8153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 -- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50f8159ef0 msgr2=0x7f50f80c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50f8159ef0 0x7f50f80c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 -- 172.21.15.148:0/1231617119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009ae0 con 0x7f50f815a2c0 2023-10-10T20:24:43.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50fd8c9700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009880 tx=0x7f50f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0008a80 con 0x7f50f815a2c0 2023-10-10T20:24:43.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0008be0 con 0x7f50f815a2c0 2023-10-10T20:24:43.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f8155920 con 0x7f50f815a2c0 2023-10-10T20:24:43.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0025460 con 0x7f50f815a2c0 2023-10-10T20:24:43.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.398+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8154fb0 con 0x7f50f815a2c0 2023-10-10T20:24:43.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.399+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f804c3c0 con 0x7f50f815a2c0 2023-10-10T20:24:43.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.400+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50f0003040 con 0x7f50f815a2c0 2023-10-10T20:24:43.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.400+0000 7f50ed7fa700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.400+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50f0085a10 con 0x7f50f815a2c0 2023-10-10T20:24:43.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.400+0000 7f50fd0c8700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.401+0000 7f50fd0c8700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 0x7f50dc0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d10 tx=0x7f50f40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.405+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50f0052c80 con 0x7f50f815a2c0 2023-10-10T20:24:43.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.588+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f81560b0 con 0x7f50f815a2c0 2023-10-10T20:24:43.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.589+0000 7f50ed7fa700 1 -- 172.21.15.148:0/1231617119 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+15405 (secure 0 0 0) 0x7f50f00562a0 con 0x7f50f815a2c0 2023-10-10T20:24:43.591 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:43.591 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":104,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:34.609051+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:43.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 msgr2=0x7f50dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 0x7f50dc0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d10 tx=0x7f50f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 msgr2=0x7f50f815b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009880 tx=0x7f50f0018000 comp rx=0 tx=0).stop 2023-10-10T20:24:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 shutdown_connections 2023-10-10T20:24:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50dc066f30 0x7f50dc0693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50f815a2c0 0x7f50f815b060 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50f80cbca0 0x7f50f8153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 --2- 172.21.15.148:0/1231617119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50f8159ef0 0x7f50f80c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 >> 172.21.15.148:0/1231617119 conn(0x7f50f8069870 msgr2=0x7f50f806bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 shutdown_connections 2023-10-10T20:24:43.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.592+0000 7f50ff32c700 1 -- 172.21.15.148:0/1231617119 wait complete. 2023-10-10T20:24:43.605 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:24:43.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.718+0000 7fa659704700 1 Processor -- start 2023-10-10T20:24:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.718+0000 7fa659704700 1 -- start start 2023-10-10T20:24:43.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa659704700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6540b0890 0x7fa6540cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa659704700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540cbd40 0x7fa654067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa659704700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6540cb370 con 0x7fa65406c630 2023-10-10T20:24:43.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa659704700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6540b0c60 con 0x7fa6540b0890 2023-10-10T20:24:43.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa659704700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65406a1e0 con 0x7fa6540cbd40 2023-10-10T20:24:43.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa652ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6540b0890 0x7fa6540cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.719+0000 7fa6537fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa65406c630 0x7fa65415dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54214/0 (socket says 172.21.15.148:54214) 2023-10-10T20:24:43.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa6537fe700 1 -- 172.21.15.148:0/303517493 learned_addr learned my addr 172.21.15.148:0/303517493 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa64a7fc700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540cbd40 0x7fa654067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa652ffd700 1 -- 172.21.15.148:0/303517493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540cbd40 msgr2=0x7fa654067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa652ffd700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540cbd40 0x7fa654067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa652ffd700 1 -- 172.21.15.148:0/303517493 >> v1:172.21.15.148:6789/0 conn(0x7fa65406c630 legacy=0x7fa65415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa652ffd700 1 -- 172.21.15.148:0/303517493 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65406cb80 con 0x7fa6540b0890 2023-10-10T20:24:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa6527fc700 1 -- 172.21.15.148:0/303517493 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354459473 0 0) 0x7fa6540cb370 con 0x7fa65406c630 2023-10-10T20:24:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa652ffd700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6540b0890 0x7fa6540cb800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa6440098e0 tx=0x7fa6440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c911e6c84b2554 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa6527fc700 1 -- 172.21.15.148:0/303517493 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400b640 con 0x7fa6540b0890 2023-10-10T20:24:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa6527fc700 1 -- 172.21.15.148:0/303517493 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400b7a0 con 0x7fa6540b0890 2023-10-10T20:24:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.720+0000 7fa6527fc700 1 -- 172.21.15.148:0/303517493 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400bb40 con 0x7fa6540b0890 2023-10-10T20:24:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 -- 172.21.15.148:0/303517493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6540b0890 msgr2=0x7fa6540cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6540b0890 0x7fa6540cb800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa6440098e0 tx=0x7fa6440091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 -- 172.21.15.148:0/303517493 shutdown_connections 2023-10-10T20:24:43.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6540b0890 0x7fa6540cb800 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 --2- 172.21.15.148:0/303517493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540cbd40 0x7fa654067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 -- 172.21.15.148:0/303517493 >> 172.21.15.148:0/303517493 conn(0x7fa65406fe50 msgr2=0x7fa654069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 -- 172.21.15.148:0/303517493 shutdown_connections 2023-10-10T20:24:43.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.721+0000 7fa659704700 1 -- 172.21.15.148:0/303517493 wait complete. 2023-10-10T20:24:43.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.722+0000 7fa659704700 1 Processor -- start 2023-10-10T20:24:43.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.722+0000 7fa659704700 1 -- start start 2023-10-10T20:24:43.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540b0890 0x7fa6540b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 0x7fa6540b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c7420 con 0x7fa65406c630 2023-10-10T20:24:43.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c7700 con 0x7fa6540cbd40 2023-10-10T20:24:43.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c79e0 con 0x7fa6540b0890 2023-10-10T20:24:43.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35212/0 (socket says 172.21.15.148:35212) 2023-10-10T20:24:43.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 -- 172.21.15.148:0/3246177594 learned_addr learned my addr 172.21.15.148:0/3246177594 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:43.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa64a7fc700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540b0890 0x7fa6540b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa6537fe700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 0x7fa6540b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 -- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540b0890 msgr2=0x7fa6540b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540b0890 0x7fa6540b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 -- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 msgr2=0x7fa6540b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 0x7fa6540b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 -- 172.21.15.148:0/3246177594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009420 con 0x7fa65406c630 2023-10-10T20:24:43.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa6537fe700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 0x7fa6540b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:43.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa652ffd700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa644009f80 tx=0x7fa644017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400bb20 con 0x7fa65406c630 2023-10-10T20:24:43.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644011420 con 0x7fa65406c630 2023-10-10T20:24:43.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6540c7cc0 con 0x7fa65406c630 2023-10-10T20:24:43.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644011800 con 0x7fa65406c630 2023-10-10T20:24:43.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.723+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65414e340 con 0x7fa65406c630 2023-10-10T20:24:43.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.724+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65414e530 con 0x7fa65406c630 2023-10-10T20:24:43.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.725+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa644011c30 con 0x7fa65406c630 2023-10-10T20:24:43.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.725+0000 7fa64b7fe700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:43.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.725+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa644086080 con 0x7fa65406c630 2023-10-10T20:24:43.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.725+0000 7fa64a7fc700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:43.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.726+0000 7fa64a7fc700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 0x7fa6340693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa65416d9c0 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:43.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.730+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6440532f0 con 0x7fa65406c630 2023-10-10T20:24:43.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.952+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa65404c3c0 con 0x7fa65406c630 2023-10-10T20:24:43.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.953+0000 7fa64b7fe700 1 -- 172.21.15.148:0/3246177594 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fa644056910 con 0x7fa65406c630 2023-10-10T20:24:43.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:43.957 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 msgr2=0x7fa6340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 0x7fa6340693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa65416d9c0 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2023-10-10T20:24:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 msgr2=0x7fa65414de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:43.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa644009f80 tx=0x7fa644017000 comp rx=0 tx=0).stop 2023-10-10T20:24:43.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 shutdown_connections 2023-10-10T20:24:43.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa634066f30 0x7fa6340693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6540cbd40 0x7fa6540b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa65406c630 0x7fa65414de00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 --2- 172.21.15.148:0/3246177594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6540b0890 0x7fa6540b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:43.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 >> 172.21.15.148:0/3246177594 conn(0x7fa65406fe50 msgr2=0x7fa65415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:43.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 shutdown_connections 2023-10-10T20:24:43.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:43.956+0000 7fa659704700 1 -- 172.21.15.148:0/3246177594 wait complete. 2023-10-10T20:24:43.962 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:43.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:44.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.084+0000 7f80101c0700 1 Processor -- start 2023-10-10T20:24:44.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.084+0000 7f80101c0700 1 -- start start 2023-10-10T20:24:44.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f80101c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f8008161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f80101c0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8008067ba0 con 0x7f800806a530 2023-10-10T20:24:44.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f80101c0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8008067870 con 0x7f80081568d0 2023-10-10T20:24:44.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f80101c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800815aa90 con 0x7f80080c87a0 2023-10-10T20:24:44.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f800d75b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f800806a530 0x7f80080c8090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54228/0 (socket says 172.21.15.148:54228) 2023-10-10T20:24:44.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f800d75b700 1 -- 172.21.15.148:0/731874550 learned_addr learned my addr 172.21.15.148:0/731874550 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:44.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.085+0000 7f800e75d700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f8008161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657316393 0 0) 0x7f8008067ba0 con 0x7f800806a530 2023-10-10T20:24:44.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff4003570 con 0x7f800806a530 2023-10-10T20:24:44.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673889557 0 0) 0x7f8008067870 con 0x7f80081568d0 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8008067ba0 con 0x7f80081568d0 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442272212 0 0) 0x7f7ff4003570 con 0x7f800806a530 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 msgr2=0x7f8008161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f8008161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 >> v1:172.21.15.161:6789/0 conn(0x7f80081568d0 legacy=0x7f8008069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080c8ca0 con 0x7f800806a530 2023-10-10T20:24:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f7ff8001470 con 0x7f800806a530 2023-10-10T20:24:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800e75d700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f8008161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 >> v1:172.21.15.148:6789/0 conn(0x7f800806a530 legacy=0x7f80080c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f7ff4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ff4004160 0x7f7ff4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ff4004d50 0x7f7ff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800815aa90 con 0x7f7ff4004160 2023-10-10T20:24:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff400bba0 con 0x7f7ff4004d50 2023-10-10T20:24:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff400bd60 con 0x7f80080c87a0 2023-10-10T20:24:44.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ff4004160 0x7f7ff4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800df5c700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f7ff4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800e75d700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ff4004d50 0x7f7ff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 -- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 msgr2=0x7f7ff4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80080c87a0 0x7f7ff4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 -- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ff4004d50 msgr2=0x7f7ff40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ff4004d50 0x7f7ff40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 -- 172.21.15.148:0/731874550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009420 con 0x7f7ff4004160 2023-10-10T20:24:44.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.086+0000 7f800d75b700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ff4004160 0x7f7ff4004810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c2d0 tx=0x7f7ff8001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ee1bc5850dd48d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800cbc0 con 0x7f7ff4004160 2023-10-10T20:24:44.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800cd20 con 0x7f7ff4004160 2023-10-10T20:24:44.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f800cf5a700 1 -- 172.21.15.148:0/731874550 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8005b70 con 0x7f7ff4004160 2023-10-10T20:24:44.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 -- 172.21.15.148:0/731874550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ff4004160 msgr2=0x7f7ff4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 --2- 172.21.15.148:0/731874550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ff4004160 0x7f7ff4004810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c2d0 tx=0x7f7ff8001e10 comp rx=0 tx=0).stop 2023-10-10T20:24:44.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f800e75d700 1 -- 172.21.15.148:0/731874550 reap_dead start 2023-10-10T20:24:44.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 -- 172.21.15.148:0/731874550 shutdown_connections 2023-10-10T20:24:44.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 -- 172.21.15.148:0/731874550 >> 172.21.15.148:0/731874550 conn(0x7f80080cda50 msgr2=0x7f80080cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:44.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 -- 172.21.15.148:0/731874550 shutdown_connections 2023-10-10T20:24:44.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.087+0000 7f80101c0700 1 -- 172.21.15.148:0/731874550 wait complete. 2023-10-10T20:24:44.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.088+0000 7f80101c0700 1 Processor -- start 2023-10-10T20:24:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.088+0000 7f80101c0700 1 -- start start 2023-10-10T20:24:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 0x7f80080aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 0x7f80080c1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008159b80 con 0x7f80080ac030 2023-10-10T20:24:44.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080aaf90 con 0x7f8008155fe0 2023-10-10T20:24:44.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f80101c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80080ab270 con 0x7f7ff4004160 2023-10-10T20:24:44.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35226/0 (socket says 172.21.15.148:35226) 2023-10-10T20:24:44.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 -- 172.21.15.148:0/2016854569 learned_addr learned my addr 172.21.15.148:0/2016854569 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:44.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800df5c700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 0x7f80080aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800e75d700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 0x7f80080c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 -- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 msgr2=0x7f80080aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 0x7f80080aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 -- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 msgr2=0x7f80080c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 0x7f80080c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.089+0000 7f800d75b700 1 -- 172.21.15.148:0/2016854569 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080ab550 con 0x7f80080ac030 2023-10-10T20:24:44.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f800df5c700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 0x7f80080aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:44.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f800e75d700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 0x7f80080c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:44.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f800d75b700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400bb70 tx=0x7f7ff8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80070d0 con 0x7f80080ac030 2023-10-10T20:24:44.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff8007270 con 0x7f80080ac030 2023-10-10T20:24:44.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80080ab830 con 0x7f80080ac030 2023-10-10T20:24:44.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8007630 con 0x7f80080ac030 2023-10-10T20:24:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.090+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008156450 con 0x7f80080ac030 2023-10-10T20:24:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.091+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80080abd50 con 0x7f80080ac030 2023-10-10T20:24:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.091+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ff8019030 con 0x7f80080ac030 2023-10-10T20:24:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.092+0000 7f7ffdffb700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 0x7f7fec069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.092+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ff8085c50 con 0x7f80080ac030 2023-10-10T20:24:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.092+0000 7f800df5c700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 0x7f7fec069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.092+0000 7f800df5c700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 0x7f7fec069410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f80 tx=0x7f80000017a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.096+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80080abd50 con 0x7f80080ac030 2023-10-10T20:24:44.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.320+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80080abd50 con 0x7f80080ac030 2023-10-10T20:24:44.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.321+0000 7f7ffdffb700 1 -- 172.21.15.148:0/2016854569 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f80080abd50 con 0x7f80080ac030 2023-10-10T20:24:44.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:44.322 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:08.605542+0000","modified":"2023-10-10T20:24:12.264440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"b","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/3811497457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3811497457},{"type":"v1","addr":"172.21.15.161:6835","nonce":3811497457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-10T20:24:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 msgr2=0x7f7fec069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 0x7f7fec069410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f80 tx=0x7f80000017a0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 msgr2=0x7f8008159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400bb70 tx=0x7f7ff8015000 comp rx=0 tx=0).stop 2023-10-10T20:24:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 shutdown_connections 2023-10-10T20:24:44.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7fec066f60 0x7f7fec069410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8008155fe0 0x7f80080c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ff4004160 0x7f80080aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 --2- 172.21.15.148:0/2016854569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080ac030 0x7f8008159640 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 >> 172.21.15.148:0/2016854569 conn(0x7f80080cda50 msgr2=0x7f80080ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:44.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 shutdown_connections 2023-10-10T20:24:44.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.323+0000 7f80101c0700 1 -- 172.21.15.148:0/2016854569 wait complete. 2023-10-10T20:24:44.327 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2023-10-10T20:24:44.336 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.450+0000 7f95432d1700 1 Processor -- start 2023-10-10T20:24:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 -- start start 2023-10-10T20:24:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 0x7f953c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f953c063cf0 0x7f953c099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953c064170 con 0x7f953c0a7b20 2023-10-10T20:24:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953c0990a0 con 0x7f953c063cf0 2023-10-10T20:24:44.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f95432d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0984b0 con 0x7f953c06f1c0 2023-10-10T20:24:44.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f954186e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f953c063cf0 0x7f953c099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.451+0000 7f954186e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f953c063cf0 0x7f953c099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35234/0 (socket says 172.21.15.148:35234) 2023-10-10T20:24:44.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 -- 172.21.15.148:0/401528719 learned_addr learned my addr 172.21.15.148:0/401528719 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:44.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954086c700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 0x7f953c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 -- 172.21.15.148:0/401528719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 msgr2=0x7f953c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 0x7f953c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 -- 172.21.15.148:0/401528719 >> v1:172.21.15.161:6789/0 conn(0x7f953c0a7b20 legacy=0x7f953c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 -- 172.21.15.148:0/401528719 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c098770 con 0x7f953c063cf0 2023-10-10T20:24:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954086c700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 0x7f953c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f954186e700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c063cf0 0x7f953c099530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9534009f20 tx=0x7f9534009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ae3ed9e6a1807 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f9533fff700 1 -- 172.21.15.148:0/401528719 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953400e070 con 0x7f953c063cf0 2023-10-10T20:24:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f9533fff700 1 -- 172.21.15.148:0/401528719 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953400ba80 con 0x7f953c063cf0 2023-10-10T20:24:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.452+0000 7f9533fff700 1 -- 172.21.15.148:0/401528719 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95340092e0 con 0x7f953c063cf0 2023-10-10T20:24:44.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 -- 172.21.15.148:0/401528719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c063cf0 msgr2=0x7f953c099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c063cf0 0x7f953c099530 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9534009f20 tx=0x7f9534009840 comp rx=0 tx=0).stop 2023-10-10T20:24:44.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 -- 172.21.15.148:0/401528719 shutdown_connections 2023-10-10T20:24:44.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c063cf0 0x7f953c099530 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 --2- 172.21.15.148:0/401528719 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c06f1c0 0x7f953c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 -- 172.21.15.148:0/401528719 >> 172.21.15.148:0/401528719 conn(0x7f953c0cd1e0 msgr2=0x7f953c0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:44.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 -- 172.21.15.148:0/401528719 shutdown_connections 2023-10-10T20:24:44.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.453+0000 7f95432d1700 1 -- 172.21.15.148:0/401528719 wait complete. 2023-10-10T20:24:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.454+0000 7f95432d1700 1 Processor -- start 2023-10-10T20:24:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.454+0000 7f95432d1700 1 -- start start 2023-10-10T20:24:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 0x7f953c144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953c06f1c0 0x7f953c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c16aed0 con 0x7f953c0a7b20 2023-10-10T20:24:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c160830 con 0x7f953c06f1c0 2023-10-10T20:24:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c160270 con 0x7f953c063cf0 2023-10-10T20:24:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35236/0 (socket says 172.21.15.148:35236) 2023-10-10T20:24:44.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 -- 172.21.15.148:0/1720814530 learned_addr learned my addr 172.21.15.148:0/1720814530 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:44.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954106d700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 0x7f953c144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954086c700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953c06f1c0 0x7f953c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 -- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 msgr2=0x7f953c144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 0x7f953c144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 -- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953c06f1c0 msgr2=0x7f953c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953c06f1c0 0x7f953c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 -- 172.21.15.148:0/1720814530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009ae0 con 0x7f953c0a7b20 2023-10-10T20:24:44.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954106d700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 0x7f953c144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:44.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f954186e700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f953c06c610 tx=0x7f9534018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95340092e0 con 0x7f953c0a7b20 2023-10-10T20:24:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534008dd0 con 0x7f953c0a7b20 2023-10-10T20:24:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c160550 con 0x7f953c0a7b20 2023-10-10T20:24:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534026690 con 0x7f953c0a7b20 2023-10-10T20:24:44.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.455+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c145260 con 0x7f953c0a7b20 2023-10-10T20:24:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.456+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c16b040 con 0x7f953c0a7b20 2023-10-10T20:24:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.457+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9534003040 con 0x7f953c0a7b20 2023-10-10T20:24:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.457+0000 7f95317fa700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 0x7f9520069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.457+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95340864f0 con 0x7f953c0a7b20 2023-10-10T20:24:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.457+0000 7f954106d700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 0x7f9520069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.458+0000 7f954106d700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 0x7f9520069410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9538007fd0 tx=0x7f9538009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.462+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9534053810 con 0x7f953c0a7b20 2023-10-10T20:24:44.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.647+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f953c145480 con 0x7f953c0a7b20 2023-10-10T20:24:44.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.647+0000 7f95317fa700 1 -- 172.21.15.148:0/1720814530 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+15405 (secure 0 0 0) 0x7f9534056e30 con 0x7f953c0a7b20 2023-10-10T20:24:44.649 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:44.649 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":104,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:34.609051+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:06.952553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:07.573984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:44.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 msgr2=0x7f9520069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 0x7f9520069410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9538007fd0 tx=0x7f9538009040 comp rx=0 tx=0).stop 2023-10-10T20:24:44.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 msgr2=0x7f953c16cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f953c06c610 tx=0x7f9534018000 comp rx=0 tx=0).stop 2023-10-10T20:24:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 shutdown_connections 2023-10-10T20:24:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9520066f60 0x7f9520069410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f953c0a7b20 0x7f953c16cf90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f953c063cf0 0x7f953c144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 --2- 172.21.15.148:0/1720814530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f953c06f1c0 0x7f953c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 >> 172.21.15.148:0/1720814530 conn(0x7f953c0cd1e0 msgr2=0x7f953c06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:44.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 shutdown_connections 2023-10-10T20:24:44.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.650+0000 7f95432d1700 1 -- 172.21.15.148:0/1720814530 wait complete. 2023-10-10T20:24:44.663 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:24:44.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.777+0000 7fb3a8c15700 1 Processor -- start 2023-10-10T20:24:44.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.777+0000 7fb3a8c15700 1 -- start start 2023-10-10T20:24:44.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a8c15700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb3a4069210 0x7fb3a40af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a8c15700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a415ea70 con 0x7fb3a415a170 2023-10-10T20:24:44.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a8c15700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a40b3990 con 0x7fb3a4159da0 2023-10-10T20:24:44.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a8c15700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a40b34b0 con 0x7fb3a4069210 2023-10-10T20:24:44.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb3a415a170 0x7fb3a4068b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54240/0 (socket says 172.21.15.148:54240) 2023-10-10T20:24:44.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/3503457548 learned_addr learned my addr 172.21.15.148:0/3503457548 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:44.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781270486 0 0) 0x7fb3a415ea70 con 0x7fb3a415a170 2023-10-10T20:24:44.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb38c003570 con 0x7fb3a415a170 2023-10-10T20:24:44.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a3fff700 1 --2- 172.21.15.148:0/3503457548 >> v2:172.21.15.161:3300/0 conn(0x7fb3a4069210 0x7fb3a40af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744860910 0 0) 0x7fb38c003570 con 0x7fb3a415a170 2023-10-10T20:24:44.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 >> v2:172.21.15.161:3300/0 conn(0x7fb3a4069210 msgr2=0x7fb3a40af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 --2- 172.21.15.148:0/3503457548 >> v2:172.21.15.161:3300/0 conn(0x7fb3a4069210 0x7fb3a40af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 >> v1:172.21.15.161:6789/0 conn(0x7fb3a4159da0 legacy=0x7fb3a40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40b0b70 con 0x7fb3a415a170 2023-10-10T20:24:44.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fb398001470 con 0x7fb3a415a170 2023-10-10T20:24:44.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 >> v1:172.21.15.148:6789/0 conn(0x7fb3a415a170 legacy=0x7fb3a4068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c003a60 0x7fb38c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 0x7fb38c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb38c0051c0 0x7fb38c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40b34b0 con 0x7fb38c004810 2023-10-10T20:24:44.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.778+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c00c1e0 con 0x7fb38c003a60 2023-10-10T20:24:44.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c00c3a0 con 0x7fb38c0051c0 2023-10-10T20:24:44.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 0x7fb38c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a37fe700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c003a60 0x7fb38c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a3fff700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb38c0051c0 0x7fb38c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb38c0051c0 msgr2=0x7fb38c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb38c0051c0 0x7fb38c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c003a60 msgr2=0x7fb38c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:44.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c003a60 0x7fb38c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a3fff700 1 -- 172.21.15.148:0/3503457548 reap_dead start 2023-10-10T20:24:44.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/3503457548 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c003570 con 0x7fb38c004810 2023-10-10T20:24:44.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a37fe700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c003a60 0x7fb38c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:44.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 0x7fb38c004c80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb39800c390 tx=0x7fb398001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3962cf062680e98b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800cbe0 con 0x7fb38c004810 2023-10-10T20:24:44.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39800cd40 con 0x7fb38c004810 2023-10-10T20:24:44.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.779+0000 7fb3a27fc700 1 -- 172.21.15.148:0/3503457548 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398005b70 con 0x7fb38c004810 2023-10-10T20:24:44.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 -- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 msgr2=0x7fb38c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 0x7fb38c004c80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb39800c390 tx=0x7fb398001e70 comp rx=0 tx=0).stop 2023-10-10T20:24:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 -- 172.21.15.148:0/3503457548 shutdown_connections 2023-10-10T20:24:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/3503457548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c004810 0x7fb38c004c80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 -- 172.21.15.148:0/3503457548 >> 172.21.15.148:0/3503457548 conn(0x7fb3a40d0bc0 msgr2=0x7fb3a40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 -- 172.21.15.148:0/3503457548 shutdown_connections 2023-10-10T20:24:44.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.780+0000 7fb3a8c15700 1 -- 172.21.15.148:0/3503457548 wait complete. 2023-10-10T20:24:44.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.781+0000 7fb3a8c15700 1 Processor -- start 2023-10-10T20:24:44.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.781+0000 7fb3a8c15700 1 -- start start 2023-10-10T20:24:44.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c004810 0x7fb3a40b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3a40b4940 0x7fb3a40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40b52c0 con 0x7fb3a40c4620 2023-10-10T20:24:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a416abe0 con 0x7fb38c004810 2023-10-10T20:24:44.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40c4a90 con 0x7fb3a40b4940 2023-10-10T20:24:44.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35264/0 (socket says 172.21.15.148:35264) 2023-10-10T20:24:44.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/2236609683 learned_addr learned my addr 172.21.15.148:0/2236609683 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:44.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a3fff700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3a40b4940 0x7fb3a40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a37fe700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c004810 0x7fb3a40b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3a40b4940 msgr2=0x7fb3a40b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3a40b4940 0x7fb3a40b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c004810 msgr2=0x7fb3a40b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:44.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c004810 0x7fb3a40b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 -- 172.21.15.148:0/2236609683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a416ae80 con 0x7fb3a40c4620 2023-10-10T20:24:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a2ffd700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00c1b0 tx=0x7fb398015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398022070 con 0x7fb3a40c4620 2023-10-10T20:24:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398000fe0 con 0x7fb3a40c4620 2023-10-10T20:24:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a416b160 con 0x7fb3a40c4620 2023-10-10T20:24:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800cbe0 con 0x7fb3a40c4620 2023-10-10T20:24:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.782+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a415d250 con 0x7fb3a40c4620 2023-10-10T20:24:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.783+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a4089480 con 0x7fb3a40c4620 2023-10-10T20:24:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.784+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3980075d0 con 0x7fb3a40c4620 2023-10-10T20:24:44.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.784+0000 7fb38b7fe700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.784+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3980855f0 con 0x7fb3a40c4620 2023-10-10T20:24:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.784+0000 7fb3a37fe700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.785+0000 7fb3a37fe700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 0x7fb3900693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb394007fd0 tx=0x7fb394009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:44.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:44.789+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb398052860 con 0x7fb3a40c4620 2023-10-10T20:24:45.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.013+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb3a415def0 con 0x7fb3a40c4620 2023-10-10T20:24:45.015 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:24:45.014+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:24:45.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.018+0000 7fb38b7fe700 1 -- 172.21.15.148:0/2236609683 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb398017090 con 0x7fb3a40c4620 2023-10-10T20:24:45.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.020+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 msgr2=0x7fb3900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.020+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 0x7fb3900693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb394007fd0 tx=0x7fb394009040 comp rx=0 tx=0).stop 2023-10-10T20:24:45.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.020+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 msgr2=0x7fb3a4154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.020+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00c1b0 tx=0x7fb398015000 comp rx=0 tx=0).stop 2023-10-10T20:24:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.020+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 shutdown_connections 2023-10-10T20:24:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3a40b4940 0x7fb3a40b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb38c004810 0x7fb3a40b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 --2- 172.21.15.148:0/2236609683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3a40c4620 0x7fb3a4154f70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 >> 172.21.15.148:0/2236609683 conn(0x7fb3a40d0bc0 msgr2=0x7fb3a4159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 shutdown_connections 2023-10-10T20:24:45.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.021+0000 7fb3a8c15700 1 -- 172.21.15.148:0/2236609683 wait complete. 2023-10-10T20:24:45.027 INFO:tasks.ceph.mds.b.smithi161.stderr: -9142> 2023-10-10T20:24:11.198+0000 7f919ff55700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:24:45.027 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:24:45.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:24:45.054 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:24:45.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.149+0000 7f0beda09700 1 Processor -- start 2023-10-10T20:24:45.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.149+0000 7f0beda09700 1 -- start start 2023-10-10T20:24:45.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0beda09700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0be806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0beda09700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80b0790 con 0x7f0be806aae0 2023-10-10T20:24:45.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0beda09700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80ce1f0 con 0x7f0be8159ec0 2023-10-10T20:24:45.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0beda09700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be806af60 con 0x7f0be815a290 2023-10-10T20:24:45.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0be77fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0be806aae0 0x7f0be80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54242/0 (socket says 172.21.15.148:54242) 2023-10-10T20:24:45.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0be77fe700 1 -- 172.21.15.148:0/2063603679 learned_addr learned my addr 172.21.15.148:0/2063603679 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:45.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0be806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493994802 0 0) 0x7f0be80b0790 con 0x7f0be806aae0 2023-10-10T20:24:45.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.150+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd4003570 con 0x7f0be806aae0 2023-10-10T20:24:45.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766364890 0 0) 0x7f0bd4003570 con 0x7f0be806aae0 2023-10-10T20:24:45.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 msgr2=0x7f0be806a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:45.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0be806a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 >> v1:172.21.15.161:6789/0 conn(0x7f0be8159ec0 legacy=0x7f0be80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80b0b20 con 0x7f0be806aae0 2023-10-10T20:24:45.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f0be00017f0 con 0x7f0be806aae0 2023-10-10T20:24:45.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 >> v1:172.21.15.148:6789/0 conn(0x7f0be806aae0 legacy=0x7f0be80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0bd4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bd4003fd0 0x7f0bd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd4004d50 0x7f0bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be806af60 con 0x7f0bd4003fd0 2023-10-10T20:24:45.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd400bba0 con 0x7f0bd4004d50 2023-10-10T20:24:45.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd400bd60 con 0x7f0be815a290 2023-10-10T20:24:45.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bd4003fd0 0x7f0bd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be6ffd700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0bd4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be77fe700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd4004d50 0x7f0bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 -- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 msgr2=0x7f0bd4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0be815a290 0x7f0bd4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 -- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd4004d50 msgr2=0x7f0bd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd4004d50 0x7f0bd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 -- 172.21.15.148:0/2063603679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4003570 con 0x7f0bd4003fd0 2023-10-10T20:24:45.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be77fe700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bd4004d50 0x7f0bd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:45.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bd4003fd0 0x7f0bd4004810 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c00 tx=0x7f0bd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d726dfa115dea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:45.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000e710 con 0x7f0bd4003fd0 2023-10-10T20:24:45.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd000e870 con 0x7f0bd4003fd0 2023-10-10T20:24:45.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.151+0000 7f0be67fc700 1 -- 172.21.15.148:0/2063603679 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000ec30 con 0x7f0bd4003fd0 2023-10-10T20:24:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 -- 172.21.15.148:0/2063603679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bd4003fd0 msgr2=0x7f0bd4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 --2- 172.21.15.148:0/2063603679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0bd4003fd0 0x7f0bd4004810 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009c00 tx=0x7f0bd00091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0be77fe700 1 -- 172.21.15.148:0/2063603679 reap_dead start 2023-10-10T20:24:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 -- 172.21.15.148:0/2063603679 shutdown_connections 2023-10-10T20:24:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 -- 172.21.15.148:0/2063603679 >> 172.21.15.148:0/2063603679 conn(0x7f0be80d1880 msgr2=0x7f0be80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 -- 172.21.15.148:0/2063603679 shutdown_connections 2023-10-10T20:24:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.152+0000 7f0beda09700 1 -- 172.21.15.148:0/2063603679 wait complete. 2023-10-10T20:24:45.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.153+0000 7f0beda09700 1 Processor -- start 2023-10-10T20:24:45.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.153+0000 7f0beda09700 1 -- start start 2023-10-10T20:24:45.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 0x7f0be8158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 0x7f0be8169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be815b860 con 0x7f0be816a170 2023-10-10T20:24:45.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be816ab20 con 0x7f0be81587d0 2023-10-10T20:24:45.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0beda09700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8158fc0 con 0x7f0bd4003fd0 2023-10-10T20:24:45.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35280/0 (socket says 172.21.15.148:35280) 2023-10-10T20:24:45.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 -- 172.21.15.148:0/3096655068 learned_addr learned my addr 172.21.15.148:0/3096655068 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:45.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be6ffd700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 0x7f0be8158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 0x7f0be8169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 -- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 msgr2=0x7f0be8158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 0x7f0be8158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 -- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 msgr2=0x7f0be8169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:45.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 0x7f0be8169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:45.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be77fe700 1 -- 172.21.15.148:0/3096655068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be81592a0 con 0x7f0be816a170 2023-10-10T20:24:45.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0be6ffd700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 0x7f0be8158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:45.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.154+0000 7f0bde7fc700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 0x7f0be8169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:45.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0be77fe700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0be8071eb0 tx=0x7f0be0001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:45.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000ebc0 con 0x7f0be816a170 2023-10-10T20:24:45.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be000ed20 con 0x7f0be816a170 2023-10-10T20:24:45.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0013670 con 0x7f0be816a170 2023-10-10T20:24:45.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8159580 con 0x7f0be816a170 2023-10-10T20:24:45.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.155+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be815bcf0 con 0x7f0be816a170 2023-10-10T20:24:45.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.156+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be8071b90 con 0x7f0be816a170 2023-10-10T20:24:45.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.156+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0be00139e0 con 0x7f0be816a170 2023-10-10T20:24:45.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.157+0000 7f0bdf7fe700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 0x7f0bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:45.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.157+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f0be0084ae0 con 0x7f0be816a170 2023-10-10T20:24:45.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.157+0000 7f0be6ffd700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 0x7f0bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:45.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.157+0000 7f0be6ffd700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 0x7f0bc80693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:45.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.161+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0be8071b90 con 0x7f0be816a170 2023-10-10T20:24:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:45.385+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0be8071b90 con 0x7f0be816a170 2023-10-10T20:24:46.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.019+0000 7f0bdf7fe700 1 -- 172.21.15.148:0/3096655068 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0be8071b90 con 0x7f0be816a170 2023-10-10T20:24:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 msgr2=0x7f0bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 0x7f0bc80693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd8009040 comp rx=0 tx=0).stop 2023-10-10T20:24:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 msgr2=0x7f0be816a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0be8071eb0 tx=0x7f0be0001300 comp rx=0 tx=0).stop 2023-10-10T20:24:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 shutdown_connections 2023-10-10T20:24:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0bc8066f30 0x7f0bc80693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be816a170 0x7f0be816a5e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0bd4003fd0 0x7f0be8158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 --2- 172.21.15.148:0/3096655068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0be81587d0 0x7f0be8169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 >> 172.21.15.148:0/3096655068 conn(0x7f0be80d1880 msgr2=0x7f0be80b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:46.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 shutdown_connections 2023-10-10T20:24:46.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.022+0000 7f0beda09700 1 -- 172.21.15.148:0/3096655068 wait complete. 2023-10-10T20:24:46.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:24:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.153+0000 7ff66caa1700 1 Processor -- start 2023-10-10T20:24:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.153+0000 7ff66caa1700 1 -- start start 2023-10-10T20:24:46.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff66caa1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff66806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff66caa1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66806acb0 con 0x7ff6680b0950 2023-10-10T20:24:46.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff66caa1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680d05b0 con 0x7ff668159d10 2023-10-10T20:24:46.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff66caa1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680cf510 con 0x7ff66806a300 2023-10-10T20:24:46.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff667fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff6680b0950 0x7ff6680b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54254/0 (socket says 172.21.15.148:54254) 2023-10-10T20:24:46.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff667fff700 1 -- 172.21.15.148:0/2035914683 learned_addr learned my addr 172.21.15.148:0/2035914683 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:46.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff65effd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff66806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413965794 0 0) 0x7ff66806acb0 con 0x7ff6680b0950 2023-10-10T20:24:46.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff654003570 con 0x7ff6680b0950 2023-10-10T20:24:46.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100598553 0 0) 0x7ff6680d05b0 con 0x7ff668159d10 2023-10-10T20:24:46.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66806acb0 con 0x7ff668159d10 2023-10-10T20:24:46.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557810934 0 0) 0x7ff654003570 con 0x7ff6680b0950 2023-10-10T20:24:46.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 msgr2=0x7ff66806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:46.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff66806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 >> v1:172.21.15.161:6789/0 conn(0x7ff668159d10 legacy=0x7ff6680cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680cf7f0 con 0x7ff6680b0950 2023-10-10T20:24:46.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.154+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7ff6600017f0 con 0x7ff6680b0950 2023-10-10T20:24:46.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 >> v1:172.21.15.148:6789/0 conn(0x7ff6680b0950 legacy=0x7ff6680b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff654003c20 0x7ff654004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff6540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680cf510 con 0x7ff654004810 2023-10-10T20:24:46.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400bb40 con 0x7ff654003c20 2023-10-10T20:24:46.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400bd00 con 0x7ff66806a300 2023-10-10T20:24:46.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff6677fe700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff654003c20 0x7ff654004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff667fff700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff6540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 -- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 msgr2=0x7ff6540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:46.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff6540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 -- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff654003c20 msgr2=0x7ff654004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:46.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff654003c20 0x7ff654004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 -- 172.21.15.148:0/2035914683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654003570 con 0x7ff654004810 2023-10-10T20:24:46.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff667fff700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66806a300 0x7ff6540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:46.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff6677fe700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff654003c20 0x7ff654004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:46.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff65effd700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff654004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff650009c00 tx=0x7ff6500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52dc0e6a8f26cc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:46.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000e710 con 0x7ff654004810 2023-10-10T20:24:46.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000e870 con 0x7ff654004810 2023-10-10T20:24:46.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.155+0000 7ff666ffd700 1 -- 172.21.15.148:0/2035914683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000ec30 con 0x7ff654004810 2023-10-10T20:24:46.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 -- 172.21.15.148:0/2035914683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 msgr2=0x7ff654004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 --2- 172.21.15.148:0/2035914683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff654004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff650009c00 tx=0x7ff6500091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff667fff700 1 -- 172.21.15.148:0/2035914683 reap_dead start 2023-10-10T20:24:46.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 -- 172.21.15.148:0/2035914683 shutdown_connections 2023-10-10T20:24:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 -- 172.21.15.148:0/2035914683 >> 172.21.15.148:0/2035914683 conn(0x7ff668069640 msgr2=0x7ff6680d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 -- 172.21.15.148:0/2035914683 shutdown_connections 2023-10-10T20:24:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.156+0000 7ff66caa1700 1 -- 172.21.15.148:0/2035914683 wait complete. 2023-10-10T20:24:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.157+0000 7ff66caa1700 1 Processor -- start 2023-10-10T20:24:46.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.157+0000 7ff66caa1700 1 -- start start 2023-10-10T20:24:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6680b0d70 0x7ff6680b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66815a0e0 0x7ff66815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668159250 con 0x7ff654004810 2023-10-10T20:24:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680b1720 con 0x7ff6680b0d70 2023-10-10T20:24:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff66caa1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680c7cb0 con 0x7ff66815a0e0 2023-10-10T20:24:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35292/0 (socket says 172.21.15.148:35292) 2023-10-10T20:24:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 -- 172.21.15.148:0/3170076724 learned_addr learned my addr 172.21.15.148:0/3170076724 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff65effd700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6680b0d70 0x7ff6680b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff667fff700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66815a0e0 0x7ff66815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 -- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66815a0e0 msgr2=0x7ff66815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66815a0e0 0x7ff66815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 -- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6680b0d70 msgr2=0x7ff6680b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:46.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6680b0d70 0x7ff6680b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 -- 172.21.15.148:0/3170076724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680c8270 con 0x7ff654004810 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.158+0000 7ff6677fe700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e810 tx=0x7ff658009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.159+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658006820 con 0x7ff654004810 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.159+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658006980 con 0x7ff654004810 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.159+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66815ad60 con 0x7ff654004810 2023-10-10T20:24:46.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.159+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658006d00 con 0x7ff654004810 2023-10-10T20:24:46.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.159+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6680c7f00 con 0x7ff654004810 2023-10-10T20:24:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.160+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680cfad0 con 0x7ff654004810 2023-10-10T20:24:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.160+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff658007230 con 0x7ff654004810 2023-10-10T20:24:46.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.160+0000 7ff65ffff700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:46.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.160+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff658084960 con 0x7ff654004810 2023-10-10T20:24:46.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.160+0000 7ff65effd700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:46.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.161+0000 7ff65effd700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 0x7ff6480693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6500091e0 tx=0x7ff65001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:46.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.165+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff658055610 con 0x7ff654004810 2023-10-10T20:24:46.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:46.346+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff66804bb80 con 0x7ff654004810 2023-10-10T20:24:47.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.023+0000 7ff65ffff700 1 -- 172.21.15.148:0/3170076724 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff6580170d0 con 0x7ff654004810 2023-10-10T20:24:47.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 msgr2=0x7ff6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 0x7ff6480693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff6500091e0 tx=0x7ff65001a000 comp rx=0 tx=0).stop 2023-10-10T20:24:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 msgr2=0x7ff6681539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e810 tx=0x7ff658009040 comp rx=0 tx=0).stop 2023-10-10T20:24:47.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 shutdown_connections 2023-10-10T20:24:47.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff648066f30 0x7ff6480693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff66815a0e0 0x7ff66815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff654004810 0x7ff6681539f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 --2- 172.21.15.148:0/3170076724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6680b0d70 0x7ff6680b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 >> 172.21.15.148:0/3170076724 conn(0x7ff668069640 msgr2=0x7ff6680b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 shutdown_connections 2023-10-10T20:24:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.025+0000 7ff66caa1700 1 -- 172.21.15.148:0/3170076724 wait complete. 2023-10-10T20:24:47.029 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:24:47.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:24:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.156+0000 7f48fb155700 1 Processor -- start 2023-10-10T20:24:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.156+0000 7f48fb155700 1 -- start start 2023-10-10T20:24:47.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fb155700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f415d240 0x7f48f40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fb155700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48f40cc940 0x7f48f40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fb155700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40b3f40 con 0x7f48f40c59e0 2023-10-10T20:24:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fb155700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40cbb80 con 0x7f48f415d240 2023-10-10T20:24:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fb155700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40cbe60 con 0x7f48f40cc940 2023-10-10T20:24:47.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f415d240 0x7f48f40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f415d240 0x7f48f40b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35298/0 (socket says 172.21.15.148:35298) 2023-10-10T20:24:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 -- 172.21.15.148:0/613679165 learned_addr learned my addr 172.21.15.148:0/613679165 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 -- 172.21.15.148:0/613679165 >> v2:172.21.15.161:3300/0 conn(0x7f48f40cc940 msgr2=0x7f48f40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48f9952700 1 --2- 172.21.15.148:0/613679165 >> v2:172.21.15.161:3300/0 conn(0x7f48f40cc940 0x7f48f40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 --2- 172.21.15.148:0/613679165 >> v2:172.21.15.161:3300/0 conn(0x7f48f40cc940 0x7f48f40ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 -- 172.21.15.148:0/613679165 >> v1:172.21.15.161:6789/0 conn(0x7f48f40c59e0 legacy=0x7f48f40c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 -- 172.21.15.148:0/613679165 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40cc140 con 0x7f48f415d240 2023-10-10T20:24:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48fa153700 1 --2- 172.21.15.148:0/613679165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f415d240 0x7f48f40b3a00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f48f406b8c0 tx=0x7f48ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10399df8ac7e25a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.157+0000 7f48f9151700 1 -- 172.21.15.148:0/613679165 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00e070 con 0x7f48f415d240 2023-10-10T20:24:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48f9151700 1 -- 172.21.15.148:0/613679165 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00ba90 con 0x7f48f415d240 2023-10-10T20:24:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48f9151700 1 -- 172.21.15.148:0/613679165 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0092e0 con 0x7f48f415d240 2023-10-10T20:24:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 -- 172.21.15.148:0/613679165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f415d240 msgr2=0x7f48f40b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 --2- 172.21.15.148:0/613679165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f415d240 0x7f48f40b3a00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f48f406b8c0 tx=0x7f48ec009840 comp rx=0 tx=0).stop 2023-10-10T20:24:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 -- 172.21.15.148:0/613679165 shutdown_connections 2023-10-10T20:24:47.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 --2- 172.21.15.148:0/613679165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f415d240 0x7f48f40b3a00 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 --2- 172.21.15.148:0/613679165 >> v2:172.21.15.161:3300/0 conn(0x7f48f40cc940 0x7f48f40ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 -- 172.21.15.148:0/613679165 >> 172.21.15.148:0/613679165 conn(0x7f48f4067e40 msgr2=0x7f48f40d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 -- 172.21.15.148:0/613679165 shutdown_connections 2023-10-10T20:24:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.158+0000 7f48fb155700 1 -- 172.21.15.148:0/613679165 wait complete. 2023-10-10T20:24:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.159+0000 7f48fb155700 1 Processor -- start 2023-10-10T20:24:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.159+0000 7f48fb155700 1 -- start start 2023-10-10T20:24:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 0x7f48f40b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 0x7f48f40cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f409ac20 con 0x7f48f40cc940 2023-10-10T20:24:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40b3580 con 0x7f48f40c59e0 2023-10-10T20:24:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fb155700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c8770 con 0x7f48f415d240 2023-10-10T20:24:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35312/0 (socket says 172.21.15.148:35312) 2023-10-10T20:24:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 -- 172.21.15.148:0/3994586148 learned_addr learned my addr 172.21.15.148:0/3994586148 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fa153700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 0x7f48f40b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 -- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 msgr2=0x7f48f40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fa954700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 0x7f48f40cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 0x7f48f40cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 -- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 msgr2=0x7f48f40b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 0x7f48f40b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48f9952700 1 -- 172.21.15.148:0/3994586148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009ae0 con 0x7f48f40cc940 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fa153700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 0x7f48f40b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.160+0000 7f48fa954700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 0x7f48f40cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48f9952700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c880 tx=0x7f48e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011600 con 0x7f48f40cc940 2023-10-10T20:24:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4011780 con 0x7f48f40cc940 2023-10-10T20:24:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011b40 con 0x7f48f40cc940 2023-10-10T20:24:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40c8a50 con 0x7f48f40cc940 2023-10-10T20:24:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.161+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f409a8b0 con 0x7f48f40cc940 2023-10-10T20:24:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.162+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f406ac60 con 0x7f48f40cc940 2023-10-10T20:24:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.162+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48e4018360 con 0x7f48f40cc940 2023-10-10T20:24:47.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.162+0000 7f48ea7fc700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:47.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.162+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f48e4084150 con 0x7f48f40cc940 2023-10-10T20:24:47.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.163+0000 7f48fa153700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:47.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.163+0000 7f48fa153700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009fd0 tx=0x7f48ec017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.167+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48e4054fb0 con 0x7f48f40cc940 2023-10-10T20:24:47.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:47.354+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48f409ad60 con 0x7f48f40cc940 2023-10-10T20:24:48.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.029+0000 7f48ea7fc700 1 -- 172.21.15.148:0/3994586148 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48e405a0d0 con 0x7f48f40cc940 2023-10-10T20:24:48.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 msgr2=0x7f48d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 0x7f48d80693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009fd0 tx=0x7f48ec017000 comp rx=0 tx=0).stop 2023-10-10T20:24:48.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 msgr2=0x7f48f40c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c880 tx=0x7f48e4008040 comp rx=0 tx=0).stop 2023-10-10T20:24:48.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 shutdown_connections 2023-10-10T20:24:48.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f48d8066f30 0x7f48d80693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48f415d240 0x7f48f40cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f40c59e0 0x7f48f40b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 --2- 172.21.15.148:0/3994586148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f40cc940 0x7f48f40c7cf0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 >> 172.21.15.148:0/3994586148 conn(0x7f48f4067e40 msgr2=0x7f48f415be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:48.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 shutdown_connections 2023-10-10T20:24:48.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.031+0000 7f48fb155700 1 -- 172.21.15.148:0/3994586148 wait complete. 2023-10-10T20:24:48.035 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:24:48.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:24:48.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.161+0000 7f196fe64700 1 Processor -- start 2023-10-10T20:24:48.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196fe64700 1 -- start start 2023-10-10T20:24:48.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196fe64700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f196815a290 0x7f196806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196fe64700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680b0790 con 0x7f196806aae0 2023-10-10T20:24:48.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196fe64700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196806af20 con 0x7f1968159ec0 2023-10-10T20:24:48.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196fe64700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19680ce1f0 con 0x7f196815a290 2023-10-10T20:24:48.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.162+0000 7f196d3ff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f196815a290 0x7f196806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f196815a290 0x7f196806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35320/0 (socket says 172.21.15.148:35320) 2023-10-10T20:24:48.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 -- 172.21.15.148:0/1300696567 learned_addr learned my addr 172.21.15.148:0/1300696567 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:48.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 -- 172.21.15.148:0/1300696567 >> v1:172.21.15.161:6789/0 conn(0x7f1968159ec0 legacy=0x7f19680af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 -- 172.21.15.148:0/1300696567 >> v1:172.21.15.148:6789/0 conn(0x7f196806aae0 legacy=0x7f19680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 -- 172.21.15.148:0/1300696567 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680b0b20 con 0x7f196815a290 2023-10-10T20:24:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196cbfe700 1 -- 172.21.15.148:0/1300696567 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55513173 0 0) 0x7f19680b0790 con 0x7f196806aae0 2023-10-10T20:24:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196d3ff700 1 --2- 172.21.15.148:0/1300696567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196815a290 0x7f196806a5a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f19580099b0 tx=0x7f1958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94040620d850d78 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196cbfe700 1 -- 172.21.15.148:0/1300696567 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800b820 con 0x7f196815a290 2023-10-10T20:24:48.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196cbfe700 1 -- 172.21.15.148:0/1300696567 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195800b980 con 0x7f196815a290 2023-10-10T20:24:48.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.163+0000 7f196cbfe700 1 -- 172.21.15.148:0/1300696567 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800bd20 con 0x7f196815a290 2023-10-10T20:24:48.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 -- 172.21.15.148:0/1300696567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196815a290 msgr2=0x7f196806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 --2- 172.21.15.148:0/1300696567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196815a290 0x7f196806a5a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f19580099b0 tx=0x7f1958009250 comp rx=0 tx=0).stop 2023-10-10T20:24:48.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 -- 172.21.15.148:0/1300696567 shutdown_connections 2023-10-10T20:24:48.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 --2- 172.21.15.148:0/1300696567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196815a290 0x7f196806a5a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 -- 172.21.15.148:0/1300696567 >> 172.21.15.148:0/1300696567 conn(0x7f19680d1880 msgr2=0x7f19680cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:48.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 -- 172.21.15.148:0/1300696567 shutdown_connections 2023-10-10T20:24:48.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.164+0000 7f196fe64700 1 -- 172.21.15.148:0/1300696567 wait complete. 2023-10-10T20:24:48.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.165+0000 7f196fe64700 1 Processor -- start 2023-10-10T20:24:48.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.165+0000 7f196fe64700 1 -- start start 2023-10-10T20:24:48.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 0x7f19681543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 0x7f19680b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19681555e0 con 0x7f196806aae0 2023-10-10T20:24:48.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196814ee10 con 0x7f1968159ec0 2023-10-10T20:24:48.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19681549d0 con 0x7f196815a290 2023-10-10T20:24:48.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35332/0 (socket says 172.21.15.148:35332) 2023-10-10T20:24:48.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 -- 172.21.15.148:0/685461011 learned_addr learned my addr 172.21.15.148:0/685461011 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:48.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196d3ff700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 0x7f19681543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196e401700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 0x7f19680b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 -- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 msgr2=0x7f19680b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 0x7f19680b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 -- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 msgr2=0x7f19681543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 0x7f19681543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 -- 172.21.15.148:0/685461011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580094f0 con 0x7f196806aae0 2023-10-10T20:24:48.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196d3ff700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 0x7f19681543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:48.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196e401700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 0x7f19680b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:48.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196dc00700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f196400c5d0 tx=0x7f1964007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964011590 con 0x7f196806aae0 2023-10-10T20:24:48.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964011710 con 0x7f196806aae0 2023-10-10T20:24:48.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1968154cb0 con 0x7f196806aae0 2023-10-10T20:24:48.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.166+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964011ad0 con 0x7f196806aae0 2023-10-10T20:24:48.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.167+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1968155270 con 0x7f196806aae0 2023-10-10T20:24:48.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.167+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968071b90 con 0x7f196806aae0 2023-10-10T20:24:48.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.168+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f196401c460 con 0x7f196806aae0 2023-10-10T20:24:48.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.168+0000 7f195dffb700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 0x7f194c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.168+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1964084030 con 0x7f196806aae0 2023-10-10T20:24:48.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.168+0000 7f196d3ff700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 0x7f194c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.168+0000 7f196d3ff700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 0x7f194c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19580099b0 tx=0x7f1958016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.172+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1968071b90 con 0x7f196806aae0 2023-10-10T20:24:48.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.355+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1968071b90 con 0x7f196806aae0 2023-10-10T20:24:48.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.361+0000 7f195dffb700 1 -- 172.21.15.148:0/685461011 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1968071b90 con 0x7f196806aae0 2023-10-10T20:24:48.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 msgr2=0x7f194c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 0x7f194c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19580099b0 tx=0x7f1958016000 comp rx=0 tx=0).stop 2023-10-10T20:24:48.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 msgr2=0x7f1968153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f196400c5d0 tx=0x7f1964007f60 comp rx=0 tx=0).stop 2023-10-10T20:24:48.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 shutdown_connections 2023-10-10T20:24:48.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f194c066f30 0x7f194c0693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f196815a290 0x7f19680b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f196806aae0 0x7f1968153ea0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 --2- 172.21.15.148:0/685461011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1968159ec0 0x7f19681543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 >> 172.21.15.148:0/685461011 conn(0x7f19680d1880 msgr2=0x7f1968158490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:48.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 shutdown_connections 2023-10-10T20:24:48.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.363+0000 7f196fe64700 1 -- 172.21.15.148:0/685461011 wait complete. 2023-10-10T20:24:48.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-10T20:24:48.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.494+0000 7f40f57e8700 1 Processor -- start 2023-10-10T20:24:48.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.494+0000 7f40f57e8700 1 -- start start 2023-10-10T20:24:48.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f57e8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40f015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f57e8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f00cb7c0 con 0x7f40f015dda0 2023-10-10T20:24:48.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f57e8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f015e290 con 0x7f40f00cd6e0 2023-10-10T20:24:48.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f57e8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f015e850 con 0x7f40f00cdab0 2023-10-10T20:24:48.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f4fe7700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f40f015dda0 0x7f40f015cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54274/0 (socket says 172.21.15.148:54274) 2023-10-10T20:24:48.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.495+0000 7f40f4fe7700 1 -- 172.21.15.148:0/956604978 learned_addr learned my addr 172.21.15.148:0/956604978 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:48.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40ef7fe700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40f015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117424375 0 0) 0x7f40f00cb7c0 con 0x7f40f015dda0 2023-10-10T20:24:48.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d8003570 con 0x7f40f015dda0 2023-10-10T20:24:48.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496681399 0 0) 0x7f40f015e290 con 0x7f40f00cd6e0 2023-10-10T20:24:48.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f00cb7c0 con 0x7f40f00cd6e0 2023-10-10T20:24:48.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071969134 0 0) 0x7f40d8003570 con 0x7f40f015dda0 2023-10-10T20:24:48.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 msgr2=0x7f40f015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:48.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40f015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 >> v1:172.21.15.161:6789/0 conn(0x7f40f00cd6e0 legacy=0x7f40f00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f015e570 con 0x7f40f015dda0 2023-10-10T20:24:48.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f40e80017f0 con 0x7f40f015dda0 2023-10-10T20:24:48.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 >> v1:172.21.15.148:6789/0 conn(0x7f40f015dda0 legacy=0x7f40f015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40d8003c20 0x7f40d8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8004d50 0x7f40d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f015e850 con 0x7f40d8003c20 2023-10-10T20:24:48.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d800bb70 con 0x7f40d8004d50 2023-10-10T20:24:48.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d800bd30 con 0x7f40f00cdab0 2023-10-10T20:24:48.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40d8003c20 0x7f40d8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40f4fe7700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8004d50 0x7f40d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40ef7fe700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 -- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 msgr2=0x7f40d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:48.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00cdab0 0x7f40d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 -- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8004d50 msgr2=0x7f40d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:48.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8004d50 0x7f40d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.496+0000 7f40effff700 1 -- 172.21.15.148:0/956604978 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8003570 con 0x7f40d8003c20 2023-10-10T20:24:48.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40effff700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40d8003c20 0x7f40d8004010 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f40e0001a10 tx=0x7f40e0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1ba1a181b5c21d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000c5d0 con 0x7f40d8003c20 2023-10-10T20:24:48.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e000c730 con 0x7f40d8003c20 2023-10-10T20:24:48.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40eeffd700 1 -- 172.21.15.148:0/956604978 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cad0 con 0x7f40d8003c20 2023-10-10T20:24:48.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 -- 172.21.15.148:0/956604978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40d8003c20 msgr2=0x7f40d8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 --2- 172.21.15.148:0/956604978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40d8003c20 0x7f40d8004010 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f40e0001a10 tx=0x7f40e0007d30 comp rx=0 tx=0).stop 2023-10-10T20:24:48.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f4fe7700 1 -- 172.21.15.148:0/956604978 reap_dead start 2023-10-10T20:24:48.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 -- 172.21.15.148:0/956604978 shutdown_connections 2023-10-10T20:24:48.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 -- 172.21.15.148:0/956604978 >> 172.21.15.148:0/956604978 conn(0x7f40f0067660 msgr2=0x7f40f0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:48.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 -- 172.21.15.148:0/956604978 shutdown_connections 2023-10-10T20:24:48.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.497+0000 7f40f57e8700 1 -- 172.21.15.148:0/956604978 wait complete. 2023-10-10T20:24:48.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.498+0000 7f40f57e8700 1 Processor -- start 2023-10-10T20:24:48.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 -- start start 2023-10-10T20:24:48.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 0x7f40f00b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 0x7f40f00b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00cc360 con 0x7f40f014c8f0 2023-10-10T20:24:48.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f016a710 con 0x7f40d8003c20 2023-10-10T20:24:48.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f57e8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00c7f50 con 0x7f40f00b2080 2023-10-10T20:24:48.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40ef7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40ef7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35348/0 (socket says 172.21.15.148:35348) 2023-10-10T20:24:48.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40ef7fe700 1 -- 172.21.15.148:0/223020533 learned_addr learned my addr 172.21.15.148:0/223020533 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:48.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40effff700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 0x7f40f00b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.499+0000 7f40f4fe7700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 0x7f40f00b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 -- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 msgr2=0x7f40f00b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 0x7f40f00b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 -- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 msgr2=0x7f40f00b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:48.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 0x7f40f00b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:48.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 -- 172.21.15.148:0/223020533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00b2a30 con 0x7f40f014c8f0 2023-10-10T20:24:48.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40f4fe7700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 0x7f40f00b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:48.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40effff700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 0x7f40f00b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:48.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40ef7fe700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40e4011d70 tx=0x7f40e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009e20 con 0x7f40f014c8f0 2023-10-10T20:24:48.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e400fcd0 con 0x7f40f014c8f0 2023-10-10T20:24:48.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f014cd60 con 0x7f40f014c8f0 2023-10-10T20:24:48.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4014650 con 0x7f40f014c8f0 2023-10-10T20:24:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.500+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f016abf0 con 0x7f40f014c8f0 2023-10-10T20:24:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.501+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f00c80c0 con 0x7f40f014c8f0 2023-10-10T20:24:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.501+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40e4014a80 con 0x7f40f014c8f0 2023-10-10T20:24:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.502+0000 7f40d7fff700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 0x7f40dc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:48.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.502+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f40e4018030 con 0x7f40f014c8f0 2023-10-10T20:24:48.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.502+0000 7f40effff700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 0x7f40dc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:48.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.502+0000 7f40effff700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 0x7f40dc069410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f40e0000c00 tx=0x7f40e0015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:48.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.506+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40f00c80c0 con 0x7f40f014c8f0 2023-10-10T20:24:48.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:48.763+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f40f014cd60 con 0x7f40f014c8f0 2023-10-10T20:24:49.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.031+0000 7f40d7fff700 1 -- 172.21.15.148:0/223020533 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v280) v1 ==== 137+0+0 (secure 0 0 0) 0x7f40f014cd60 con 0x7f40f014c8f0 2023-10-10T20:24:49.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 msgr2=0x7f40dc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:49.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 0x7f40dc069410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f40e0000c00 tx=0x7f40e0015000 comp rx=0 tx=0).stop 2023-10-10T20:24:49.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 msgr2=0x7f40f016a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:49.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40e4011d70 tx=0x7f40e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 shutdown_connections 2023-10-10T20:24:49.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f40dc066f60 0x7f40dc069410 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40f00b2080 0x7f40f00b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8003c20 0x7f40f00b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 --2- 172.21.15.148:0/223020533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f014c8f0 0x7f40f016a1d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 >> 172.21.15.148:0/223020533 conn(0x7f40f0067660 msgr2=0x7f40f0156520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:49.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 shutdown_connections 2023-10-10T20:24:49.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.033+0000 7f40f57e8700 1 -- 172.21.15.148:0/223020533 wait complete. 2023-10-10T20:24:49.047 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-10T20:24:49.047 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-10T20:24:49.048 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-10-10T20:24:49.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-10T20:24:49.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.161+0000 7f0d3c07d700 1 Processor -- start 2023-10-10T20:24:49.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 -- start start 2023-10-10T20:24:49.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d340b0e50 0x7f0d340c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d340b1220 0x7f0d3406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0d3406dad0 0x7f0d340cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3415de60 con 0x7f0d340b0e50 2023-10-10T20:24:49.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3415e140 con 0x7f0d3406dad0 2023-10-10T20:24:49.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d3c07d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3415e420 con 0x7f0d340b1220 2023-10-10T20:24:49.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.162+0000 7f0d39e19700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d340b0e50 0x7f0d340c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0d340b0e50 0x7f0d340c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35354/0 (socket says 172.21.15.148:35354) 2023-10-10T20:24:49.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 -- 172.21.15.148:0/204558957 learned_addr learned my addr 172.21.15.148:0/204558957 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:49.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39618700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d340b1220 0x7f0d3406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d3a61a700 1 --2- 172.21.15.148:0/204558957 >> v2:172.21.15.161:3300/0 conn(0x7f0d3406dad0 0x7f0d340cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 -- 172.21.15.148:0/204558957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d340b1220 msgr2=0x7f0d3406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:49.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d340b1220 0x7f0d3406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 -- 172.21.15.148:0/204558957 >> v2:172.21.15.161:3300/0 conn(0x7f0d3406dad0 msgr2=0x7f0d340cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:49.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 --2- 172.21.15.148:0/204558957 >> v2:172.21.15.161:3300/0 conn(0x7f0d3406dad0 0x7f0d340cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 -- 172.21.15.148:0/204558957 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3415e700 con 0x7f0d340b0e50 2023-10-10T20:24:49.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d39e19700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340c9180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d240099b0 tx=0x7f0d24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d83b0d0abc26a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:49.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d38e17700 1 -- 172.21.15.148:0/204558957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400b640 con 0x7f0d340b0e50 2023-10-10T20:24:49.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d38e17700 1 -- 172.21.15.148:0/204558957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2400b7a0 con 0x7f0d340b0e50 2023-10-10T20:24:49.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.163+0000 7f0d38e17700 1 -- 172.21.15.148:0/204558957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400bb60 con 0x7f0d340b0e50 2023-10-10T20:24:49.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 -- 172.21.15.148:0/204558957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 msgr2=0x7f0d340c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:49.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340c9180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d240099b0 tx=0x7f0d24009250 comp rx=0 tx=0).stop 2023-10-10T20:24:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 -- 172.21.15.148:0/204558957 shutdown_connections 2023-10-10T20:24:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/204558957 >> v2:172.21.15.161:3300/0 conn(0x7f0d3406dad0 0x7f0d340cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340c9180 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/204558957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d340b1220 0x7f0d3406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 -- 172.21.15.148:0/204558957 >> 172.21.15.148:0/204558957 conn(0x7f0d34069870 msgr2=0x7f0d34069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:49.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 -- 172.21.15.148:0/204558957 shutdown_connections 2023-10-10T20:24:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.164+0000 7f0d3c07d700 1 -- 172.21.15.148:0/204558957 wait complete. 2023-10-10T20:24:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.165+0000 7f0d3c07d700 1 Processor -- start 2023-10-10T20:24:49.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.165+0000 7f0d3c07d700 1 -- start start 2023-10-10T20:24:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d3406dad0 0x7f0d34141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d340b1220 0x7f0d340b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3415b580 con 0x7f0d340b0e50 2023-10-10T20:24:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340b4160 con 0x7f0d340b1220 2023-10-10T20:24:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34155140 con 0x7f0d3406dad0 2023-10-10T20:24:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35366/0 (socket says 172.21.15.148:35366) 2023-10-10T20:24:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 -- 172.21.15.148:0/2267338430 learned_addr learned my addr 172.21.15.148:0/2267338430 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3a61a700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d340b1220 0x7f0d340b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39e19700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d3406dad0 0x7f0d34141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 -- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d3406dad0 msgr2=0x7f0d34141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d3406dad0 0x7f0d34141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 -- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d340b1220 msgr2=0x7f0d340b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d340b1220 0x7f0d340b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 -- 172.21.15.148:0/2267338430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d240094f0 con 0x7f0d340b0e50 2023-10-10T20:24:49.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d39618700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e930 tx=0x7f0d280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:49.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28013680 con 0x7f0d340b0e50 2023-10-10T20:24:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d280137e0 con 0x7f0d340b0e50 2023-10-10T20:24:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3415a970 con 0x7f0d340b0e50 2023-10-10T20:24:49.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.166+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28013b80 con 0x7f0d340b0e50 2023-10-10T20:24:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.167+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3415af30 con 0x7f0d340b0e50 2023-10-10T20:24:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.167+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3404c3c0 con 0x7f0d340b0e50 2023-10-10T20:24:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.168+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d2800c3e0 con 0x7f0d340b0e50 2023-10-10T20:24:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.168+0000 7f0d31ffb700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 0x7f0d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.168+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0d280842e0 con 0x7f0d340b0e50 2023-10-10T20:24:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.168+0000 7f0d39e19700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 0x7f0d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.169+0000 7f0d39e19700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 0x7f0d200693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d240099b0 tx=0x7f0d24017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:49.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.172+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d28051850 con 0x7f0d340b0e50 2023-10-10T20:24:49.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:49.431+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f0d340ce7d0 con 0x7f0d340b0e50 2023-10-10T20:24:50.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.037+0000 7f0d31ffb700 1 -- 172.21.15.148:0/2267338430 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v281) v1 ==== 131+0+0 (secure 0 0 0) 0x7f0d28054e70 con 0x7f0d340b0e50 2023-10-10T20:24:50.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 msgr2=0x7f0d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 0x7f0d200693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d240099b0 tx=0x7f0d24017000 comp rx=0 tx=0).stop 2023-10-10T20:24:50.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 msgr2=0x7f0d340b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e930 tx=0x7f0d280091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 shutdown_connections 2023-10-10T20:24:50.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0d20066f30 0x7f0d200693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d340b1220 0x7f0d340b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d3406dad0 0x7f0d34141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 --2- 172.21.15.148:0/2267338430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d340b0e50 0x7f0d340b36e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 >> 172.21.15.148:0/2267338430 conn(0x7f0d34069870 msgr2=0x7f0d340c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:50.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 shutdown_connections 2023-10-10T20:24:50.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.040+0000 7f0d3c07d700 1 -- 172.21.15.148:0/2267338430 wait complete. 2023-10-10T20:24:50.054 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:24:50.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.173+0000 7f26a53d6700 1 Processor -- start 2023-10-10T20:24:50.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.173+0000 7f26a53d6700 1 -- start start 2023-10-10T20:24:50.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 0x7f26a015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 0x7f26a006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26a006a830 0x7f26a015d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a00b4d70 con 0x7f26a006a830 2023-10-10T20:24:50.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a006ac70 con 0x7f26a00cce30 2023-10-10T20:24:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a53d6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a00cd8d0 con 0x7f26a00b6580 2023-10-10T20:24:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26a006a830 0x7f26a015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26a006a830 0x7f26a015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35382/0 (socket says 172.21.15.148:35382) 2023-10-10T20:24:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 -- 172.21.15.148:0/3178332334 learned_addr learned my addr 172.21.15.148:0/3178332334 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:50.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f269ffff700 1 --2- 172.21.15.148:0/3178332334 >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 0x7f26a015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f269f7fe700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 0x7f26a006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 -- 172.21.15.148:0/3178332334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 msgr2=0x7f26a006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 0x7f26a006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 -- 172.21.15.148:0/3178332334 >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 msgr2=0x7f26a015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:50.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/3178332334 >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 0x7f26a015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f26a4bd5700 1 -- 172.21.15.148:0/3178332334 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00b4440 con 0x7f26a006a830 2023-10-10T20:24:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f269ffff700 1 --2- 172.21.15.148:0/3178332334 >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 0x7f26a015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.174+0000 7f269f7fe700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 0x7f26a006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a006a830 0x7f26a015d880 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f26a00b3770 tx=0x7f2698009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d346adf03d7c38a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f269effd700 1 -- 172.21.15.148:0/3178332334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269800e070 con 0x7f26a006a830 2023-10-10T20:24:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f269effd700 1 -- 172.21.15.148:0/3178332334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269800ba50 con 0x7f26a006a830 2023-10-10T20:24:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f269effd700 1 -- 172.21.15.148:0/3178332334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980092e0 con 0x7f26a006a830 2023-10-10T20:24:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 -- 172.21.15.148:0/3178332334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a006a830 msgr2=0x7f26a015d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a006a830 0x7f26a015d880 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f26a00b3770 tx=0x7f2698009840 comp rx=0 tx=0).stop 2023-10-10T20:24:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 -- 172.21.15.148:0/3178332334 shutdown_connections 2023-10-10T20:24:50.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a006a830 0x7f26a015d880 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 --2- 172.21.15.148:0/3178332334 >> v2:172.21.15.161:3300/0 conn(0x7f26a00cce30 0x7f26a015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 --2- 172.21.15.148:0/3178332334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a00b6580 0x7f26a006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 -- 172.21.15.148:0/3178332334 >> 172.21.15.148:0/3178332334 conn(0x7f26a0068cb0 msgr2=0x7f26a00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:50.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 -- 172.21.15.148:0/3178332334 shutdown_connections 2023-10-10T20:24:50.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.175+0000 7f26a53d6700 1 -- 172.21.15.148:0/3178332334 wait complete. 2023-10-10T20:24:50.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.176+0000 7f26a53d6700 1 Processor -- start 2023-10-10T20:24:50.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 -- start start 2023-10-10T20:24:50.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a006a830 0x7f26a00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 0x7f26a014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0155270 con 0x7f26a00cce30 2023-10-10T20:24:50.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a00b1f10 con 0x7f26a00b6580 2023-10-10T20:24:50.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a53d6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0155530 con 0x7f26a006a830 2023-10-10T20:24:50.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.177+0000 7f26a4bd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35388/0 (socket says 172.21.15.148:35388) 2023-10-10T20:24:50.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 -- 172.21.15.148:0/2068678344 learned_addr learned my addr 172.21.15.148:0/2068678344 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:50.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f269ffff700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a006a830 0x7f26a00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f269f7fe700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 0x7f26a014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 -- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a006a830 msgr2=0x7f26a00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a006a830 0x7f26a00b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 -- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 msgr2=0x7f26a014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 0x7f26a014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 -- 172.21.15.148:0/2068678344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698009ae0 con 0x7f26a00cce30 2023-10-10T20:24:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f269f7fe700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 0x7f26a014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a4bd5700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f26a006b300 tx=0x7f2698018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698008dd0 con 0x7f26a00cce30 2023-10-10T20:24:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698026460 con 0x7f26a00cce30 2023-10-10T20:24:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a01500b0 con 0x7f26a00cce30 2023-10-10T20:24:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698026820 con 0x7f26a00cce30 2023-10-10T20:24:50.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.178+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a00b1950 con 0x7f26a00cce30 2023-10-10T20:24:50.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.179+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a0089550 con 0x7f26a00cce30 2023-10-10T20:24:50.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.179+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2698003040 con 0x7f26a00cce30 2023-10-10T20:24:50.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.180+0000 7f2687fff700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 0x7f268c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.180+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2698086290 con 0x7f26a00cce30 2023-10-10T20:24:50.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.180+0000 7f269ffff700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 0x7f268c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.180+0000 7f269ffff700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 0x7f268c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2690006fd0 tx=0x7f2690008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.184+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26980538b0 con 0x7f26a00cce30 2023-10-10T20:24:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.366+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f26a004bb80 con 0x7f26a00cce30 2023-10-10T20:24:50.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.369+0000 7f2687fff700 1 -- 172.21.15.148:0/2068678344 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2698056ed0 con 0x7f26a00cce30 2023-10-10T20:24:50.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 msgr2=0x7f268c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 0x7f268c0693e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2690006fd0 tx=0x7f2690008040 comp rx=0 tx=0).stop 2023-10-10T20:24:50.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 msgr2=0x7f26a00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f26a006b300 tx=0x7f2698018000 comp rx=0 tx=0).stop 2023-10-10T20:24:50.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 shutdown_connections 2023-10-10T20:24:50.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f268c066f30 0x7f268c0693e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a00cce30 0x7f26a00b0e40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f26a006a830 0x7f26a00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 --2- 172.21.15.148:0/2068678344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26a00b6580 0x7f26a014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 >> 172.21.15.148:0/2068678344 conn(0x7f26a0068cb0 msgr2=0x7f26a01503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:50.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.371+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 shutdown_connections 2023-10-10T20:24:50.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.372+0000 7f26a53d6700 1 -- 172.21.15.148:0/2068678344 wait complete. 2023-10-10T20:24:50.384 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:24:50.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:50.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:24:50.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:24:50.466 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:24:50.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:50.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.582+0000 7f9e548f3700 1 Processor -- start 2023-10-10T20:24:50.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.582+0000 7f9e548f3700 1 -- start start 2023-10-10T20:24:50.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.582+0000 7f9e548f3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e50157c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.582+0000 7f9e548f3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5015bd50 con 0x7f9e501578b0 2023-10-10T20:24:50.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e548f3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5006a150 con 0x7f9e50069870 2023-10-10T20:24:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e548f3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5006a710 con 0x7f9e5006fe80 2023-10-10T20:24:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4f7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9e501578b0 0x7f9e5015c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54288/0 (socket says 172.21.15.148:54288) 2023-10-10T20:24:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4f7fe700 1 -- 172.21.15.148:0/1202347040 learned_addr learned my addr 172.21.15.148:0/1202347040 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4effd700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e50157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982935713 0 0) 0x7f9e5015bd50 con 0x7f9e501578b0 2023-10-10T20:24:50.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e38003570 con 0x7f9e501578b0 2023-10-10T20:24:50.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3934797859 0 0) 0x7f9e38003570 con 0x7f9e501578b0 2023-10-10T20:24:50.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 msgr2=0x7f9e50157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:50.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e50157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 >> v1:172.21.15.161:6789/0 conn(0x7f9e50069870 legacy=0x7f9e500ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5006a430 con 0x7f9e501578b0 2023-10-10T20:24:50.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9e44001470 con 0x7f9e501578b0 2023-10-10T20:24:50.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 >> v1:172.21.15.148:6789/0 conn(0x7f9e501578b0 legacy=0x7f9e5015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e38003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e38004010 0x7f9e38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e38004d50 0x7f9e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5006a710 con 0x7f9e38004010 2023-10-10T20:24:50.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800bbf0 con 0x7f9e38004d50 2023-10-10T20:24:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800bdb0 con 0x7f9e5006fe80 2023-10-10T20:24:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4effd700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e38004010 0x7f9e38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.583+0000 7f9e4f7fe700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e38003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4ffff700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e38004d50 0x7f9e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 -- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 msgr2=0x7f9e38003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e5006fe80 0x7f9e38003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 -- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e38004d50 msgr2=0x7f9e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e38004d50 0x7f9e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 -- 172.21.15.148:0/1202347040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e38003570 con 0x7f9e38004010 2023-10-10T20:24:50.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4effd700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e38004010 0x7f9e38004810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006f00 tx=0x7f9e400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b803513863882dc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000e710 con 0x7f9e38004010 2023-10-10T20:24:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000e870 con 0x7f9e38004010 2023-10-10T20:24:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e4e7fc700 1 -- 172.21.15.148:0/1202347040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000ec30 con 0x7f9e38004010 2023-10-10T20:24:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e548f3700 1 -- 172.21.15.148:0/1202347040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e38004010 msgr2=0x7f9e38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.584+0000 7f9e548f3700 1 --2- 172.21.15.148:0/1202347040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e38004010 0x7f9e38004810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006f00 tx=0x7f9e400091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.585+0000 7f9e4ffff700 1 -- 172.21.15.148:0/1202347040 reap_dead start 2023-10-10T20:24:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.585+0000 7f9e548f3700 1 -- 172.21.15.148:0/1202347040 shutdown_connections 2023-10-10T20:24:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.585+0000 7f9e548f3700 1 -- 172.21.15.148:0/1202347040 >> 172.21.15.148:0/1202347040 conn(0x7f9e5006bc70 msgr2=0x7f9e5006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.585+0000 7f9e548f3700 1 -- 172.21.15.148:0/1202347040 shutdown_connections 2023-10-10T20:24:50.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.585+0000 7f9e548f3700 1 -- 172.21.15.148:0/1202347040 wait complete. 2023-10-10T20:24:50.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.586+0000 7f9e548f3700 1 Processor -- start 2023-10-10T20:24:50.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.586+0000 7f9e548f3700 1 -- start start 2023-10-10T20:24:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e38004010 0x7f9e501690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 0x7f9e50157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50169620 con 0x7f9e500c4670 2023-10-10T20:24:50.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50159000 con 0x7f9e5015a0a0 2023-10-10T20:24:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501592e0 con 0x7f9e38004010 2023-10-10T20:24:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35414/0 (socket says 172.21.15.148:35414) 2023-10-10T20:24:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 -- 172.21.15.148:0/2754423819 learned_addr learned my addr 172.21.15.148:0/2754423819 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:50.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4f7fe700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e38004010 0x7f9e501690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4ffff700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 0x7f9e50157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 -- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e38004010 msgr2=0x7f9e501690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e38004010 0x7f9e501690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 -- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 msgr2=0x7f9e50157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 0x7f9e50157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 -- 172.21.15.148:0/2754423819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e501595c0 con 0x7f9e500c4670 2023-10-10T20:24:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4ffff700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 0x7f9e50157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e4effd700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9e380057d0 tx=0x7f9e4001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40022070 con 0x7f9e500c4670 2023-10-10T20:24:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000e9e0 con 0x7f9e500c4670 2023-10-10T20:24:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e501598a0 con 0x7f9e500c4670 2023-10-10T20:24:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4001c450 con 0x7f9e500c4670 2023-10-10T20:24:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.587+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e500c4260 con 0x7f9e500c4670 2023-10-10T20:24:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.588+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5006e120 con 0x7f9e500c4670 2023-10-10T20:24:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.589+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e4001c860 con 0x7f9e500c4670 2023-10-10T20:24:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.589+0000 7f9e377fe700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 0x7f9e3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.589+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9e40085aa0 con 0x7f9e500c4670 2023-10-10T20:24:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.589+0000 7f9e4f7fe700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 0x7f9e3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.589+0000 7f9e4f7fe700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 0x7f9e3c0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001120 tx=0x7f9e44001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.594+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e40053010 con 0x7f9e500c4670 2023-10-10T20:24:50.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.821+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e5004bb80 con 0x7f9e500c4670 2023-10-10T20:24:50.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.822+0000 7f9e377fe700 1 -- 172.21.15.148:0/2754423819 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f9e40017df0 con 0x7f9e500c4670 2023-10-10T20:24:50.824 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:50.824 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/490861444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":490861444},{"type":"v1","addr":"172.21.15.148:6837","nonce":490861444}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3133958934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3133958934},{"type":"v1","addr":"172.21.15.161:6834","nonce":3133958934}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6271,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1781860630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1781860630},{"type":"v1","addr":"172.21.15.161:6835","nonce":1781860630}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2344732915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2344732915},{"type":"v1","addr":"172.21.15.148:6835","nonce":2344732915}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[]} 2023-10-10T20:24:50.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 msgr2=0x7f9e3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 0x7f9e3c0693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001120 tx=0x7f9e44001e50 comp rx=0 tx=0).stop 2023-10-10T20:24:50.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 msgr2=0x7f9e500c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:50.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9e380057d0 tx=0x7f9e4001a000 comp rx=0 tx=0).stop 2023-10-10T20:24:50.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 shutdown_connections 2023-10-10T20:24:50.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9e3c066f30 0x7f9e3c0693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e5015a0a0 0x7f9e50157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e38004010 0x7f9e501690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 --2- 172.21.15.148:0/2754423819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e500c4670 0x7f9e500c3d20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 >> 172.21.15.148:0/2754423819 conn(0x7f9e5006bc70 msgr2=0x7f9e500b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:50.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 shutdown_connections 2023-10-10T20:24:50.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:50.824+0000 7f9e548f3700 1 -- 172.21.15.148:0/2754423819 wait complete. 2023-10-10T20:24:50.828 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 51 2023-10-10T20:24:50.837 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:24:50.837 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:24:50.838 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:24:50.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:50.839 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:24:50.839 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:24:50.839 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:24:50.839 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:50.840 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:24:50.840 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:24:50.840 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:24:50.841 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:50.841 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:24:50.841 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:24:50.842 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:24:50.842 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:24:50.846 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:24:50.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:24:50.848 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:24:50.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:24:50.853 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:24:50.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:24:50.854 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:24:50.854 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:24:50.890 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:24:50.890 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:50.891 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:24:50.892 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:50.895 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:24:50.895 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:50.896 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:24:50.897 DEBUG:teuthology.parallel:result is None 2023-10-10T20:24:50.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:24:50.963 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:24:50.964 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:24:50.975 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:24:50.976 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:24:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.017+0000 7fe859236700 1 Processor -- start 2023-10-10T20:24:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.017+0000 7fe859236700 1 -- start start 2023-10-10T20:24:51.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe859236700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe85415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe859236700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe85415a620 0x7fe85415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe859236700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85415cbb0 con 0x7fe85415d580 2023-10-10T20:24:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe859236700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85415bfa0 con 0x7fe85415a620 2023-10-10T20:24:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe859236700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85415c280 con 0x7fe854157870 2023-10-10T20:24:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe84a59c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe85415a620 0x7fe85415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe852d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe85415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe85359e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe85415d580 0x7fe854158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:51202/0 (socket says 172.21.15.148:51202) 2023-10-10T20:24:51.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe85359e700 1 -- 172.21.15.148:0/2045251503 learned_addr learned my addr 172.21.15.148:0/2045251503 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:51.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.018+0000 7fe84a59c700 1 --2- 172.21.15.148:0/2045251503 >> v2:172.21.15.161:3300/0 conn(0x7fe85415a620 0x7fe85415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:45954/0 (socket says 172.21.15.148:45954) 2023-10-10T20:24:51.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe84a59c700 1 -- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 msgr2=0x7fe85415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:51.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe84a59c700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe85415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe84a59c700 1 -- 172.21.15.148:0/2045251503 >> v1:172.21.15.161:6789/0 conn(0x7fe85415d580 legacy=0x7fe854158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe85259c700 1 -- 172.21.15.148:0/2045251503 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071044541 0 0) 0x7fe85415cbb0 con 0x7fe85415d580 2023-10-10T20:24:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe84a59c700 1 -- 172.21.15.148:0/2045251503 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85415c560 con 0x7fe85415a620 2023-10-10T20:24:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe852d9d700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe85415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:24:51.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe84a59c700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415a620 0x7fe85415d040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0098e0 tx=0x7fe83c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492e3332cb8ef5c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe85259c700 1 -- 172.21.15.148:0/2045251503 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00b640 con 0x7fe85415a620 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe85259c700 1 -- 172.21.15.148:0/2045251503 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c00b7a0 con 0x7fe85415a620 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.019+0000 7fe85259c700 1 -- 172.21.15.148:0/2045251503 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00bb40 con 0x7fe85415a620 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 -- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415a620 msgr2=0x7fe85415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415a620 0x7fe85415d040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0098e0 tx=0x7fe83c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 -- 172.21.15.148:0/2045251503 shutdown_connections 2023-10-10T20:24:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe85415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 --2- 172.21.15.148:0/2045251503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415a620 0x7fe85415d040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 -- 172.21.15.148:0/2045251503 >> 172.21.15.148:0/2045251503 conn(0x7fe854068ce0 msgr2=0x7fe8540690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 -- 172.21.15.148:0/2045251503 shutdown_connections 2023-10-10T20:24:51.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.020+0000 7fe859236700 1 -- 172.21.15.148:0/2045251503 wait complete. 2023-10-10T20:24:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.021+0000 7fe859236700 1 Processor -- start 2023-10-10T20:24:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.021+0000 7fe859236700 1 -- start start 2023-10-10T20:24:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe8540b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415d580 0x7fe8540d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b63b0 con 0x7fe85415a620 2023-10-10T20:24:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b57a0 con 0x7fe85415d580 2023-10-10T20:24:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe859236700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b5a80 con 0x7fe854157870 2023-10-10T20:24:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35472/0 (socket says 172.21.15.148:35472) 2023-10-10T20:24:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 -- 172.21.15.148:0/723758399 learned_addr learned my addr 172.21.15.148:0/723758399 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe852d9d700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe8540b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe85359e700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415d580 0x7fe8540d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 -- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 msgr2=0x7fe8540b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe8540b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 -- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415d580 msgr2=0x7fe8540d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415d580 0x7fe8540d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 -- 172.21.15.148:0/723758399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009420 con 0x7fe85415a620 2023-10-10T20:24:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84a59c700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe85406b530 tx=0x7fe83c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.022+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00bb20 con 0x7fe85415a620 2023-10-10T20:24:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.023+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c011420 con 0x7fe85415a620 2023-10-10T20:24:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.023+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c009420 con 0x7fe85415a620 2023-10-10T20:24:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.023+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0117c0 con 0x7fe85415a620 2023-10-10T20:24:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.023+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8540b6040 con 0x7fe85415a620 2023-10-10T20:24:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.023+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85404c3c0 con 0x7fe85415a620 2023-10-10T20:24:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.024+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe83c011b70 con 0x7fe85415a620 2023-10-10T20:24:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.024+0000 7fe84b7fe700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.024+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe83c085910 con 0x7fe85415a620 2023-10-10T20:24:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.024+0000 7fe852d9d700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.025+0000 7fe852d9d700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 0x7fe8340693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8440093f0 tx=0x7fe844008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.029+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe83c052e80 con 0x7fe85415a620 2023-10-10T20:24:51.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:51.229+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe85415c840 con 0x7fe85415a620 2023-10-10T20:24:52.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.069+0000 7fe84b7fe700 1 -- 172.21.15.148:0/723758399 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe83c0564a0 con 0x7fe85415a620 2023-10-10T20:24:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 msgr2=0x7fe8340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 0x7fe8340693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8440093f0 tx=0x7fe844008040 comp rx=0 tx=0).stop 2023-10-10T20:24:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 msgr2=0x7fe8540b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe85406b530 tx=0x7fe83c017000 comp rx=0 tx=0).stop 2023-10-10T20:24:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 shutdown_connections 2023-10-10T20:24:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85415d580 0x7fe8540d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854157870 0x7fe8540b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 --2- 172.21.15.148:0/723758399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe85415a620 0x7fe8540b4ea0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 >> 172.21.15.148:0/723758399 conn(0x7fe854068ce0 msgr2=0x7fe85414c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 shutdown_connections 2023-10-10T20:24:52.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.071+0000 7fe859236700 1 -- 172.21.15.148:0/723758399 wait complete. 2023-10-10T20:24:52.075 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:24:52.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:24:52.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.199+0000 7fae7cd6a700 1 Processor -- start 2023-10-10T20:24:52.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.199+0000 7fae7cd6a700 1 -- start start 2023-10-10T20:24:52.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae78143520 0x7fae780cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 0x7fae7806a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7806a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780d0870 con 0x7fae78143520 2023-10-10T20:24:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780cdfb0 con 0x7fae781438f0 2023-10-10T20:24:52.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae7cd6a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780cfc60 con 0x7fae780ccf00 2023-10-10T20:24:52.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae78143520 0x7fae780cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae78143520 0x7fae780cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35482/0 (socket says 172.21.15.148:35482) 2023-10-10T20:24:52.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 -- 172.21.15.148:0/37719601 learned_addr learned my addr 172.21.15.148:0/37719601 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:52.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae77fff700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7806a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae76ffd700 1 --2- 172.21.15.148:0/37719601 >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 0x7fae7806a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 -- 172.21.15.148:0/37719601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 msgr2=0x7fae7806a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7806a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 -- 172.21.15.148:0/37719601 >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 msgr2=0x7fae7806a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:52.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 --2- 172.21.15.148:0/37719601 >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 0x7fae7806a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae777fe700 1 -- 172.21.15.148:0/37719601 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780cff40 con 0x7fae78143520 2023-10-10T20:24:52.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae77fff700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7806a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:52.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.200+0000 7fae76ffd700 1 --2- 172.21.15.148:0/37719601 >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 0x7fae7806a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:52.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae777fe700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae78143520 0x7fae780cf580 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fae7815d9c0 tx=0x7fae68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce63258464e46ca server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae767fc700 1 -- 172.21.15.148:0/37719601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800e070 con 0x7fae78143520 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae767fc700 1 -- 172.21.15.148:0/37719601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800ba50 con 0x7fae78143520 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae767fc700 1 -- 172.21.15.148:0/37719601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680092e0 con 0x7fae78143520 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 -- 172.21.15.148:0/37719601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae78143520 msgr2=0x7fae780cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae78143520 0x7fae780cf580 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fae7815d9c0 tx=0x7fae68009840 comp rx=0 tx=0).stop 2023-10-10T20:24:52.208 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 -- 172.21.15.148:0/37719601 shutdown_connections 2023-10-10T20:24:52.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7806a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/37719601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae78143520 0x7fae780cf580 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/37719601 >> v2:172.21.15.161:3300/0 conn(0x7fae781438f0 0x7fae7806a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 -- 172.21.15.148:0/37719601 >> 172.21.15.148:0/37719601 conn(0x7fae78069750 msgr2=0x7fae780b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 -- 172.21.15.148:0/37719601 shutdown_connections 2023-10-10T20:24:52.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.201+0000 7fae7cd6a700 1 -- 172.21.15.148:0/37719601 wait complete. 2023-10-10T20:24:52.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.202+0000 7fae7cd6a700 1 Processor -- start 2023-10-10T20:24:52.210 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.202+0000 7fae7cd6a700 1 -- start start 2023-10-10T20:24:52.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 0x7fae780b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815c210 con 0x7fae781438f0 2023-10-10T20:24:52.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7813bec0 con 0x7fae78143520 2023-10-10T20:24:52.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae7cd6a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7813b2d0 con 0x7fae780ccf00 2023-10-10T20:24:52.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae77fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae77fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35494/0 (socket says 172.21.15.148:35494) 2023-10-10T20:24:52.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae77fff700 1 -- 172.21.15.148:0/1574193927 learned_addr learned my addr 172.21.15.148:0/1574193927 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:52.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae777fe700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.203+0000 7fae76ffd700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 0x7fae780b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 -- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 msgr2=0x7fae7815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 -- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 msgr2=0x7fae780b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 0x7fae780b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 -- 172.21.15.148:0/1574193927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009ae0 con 0x7fae781438f0 2023-10-10T20:24:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae76ffd700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 0x7fae780b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:52.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae77fff700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c880 tx=0x7fae70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70011600 con 0x7fae781438f0 2023-10-10T20:24:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70011760 con 0x7fae781438f0 2023-10-10T20:24:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7813b590 con 0x7fae781438f0 2023-10-10T20:24:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70011b20 con 0x7fae781438f0 2023-10-10T20:24:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.204+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7813bb50 con 0x7fae781438f0 2023-10-10T20:24:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.205+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78089030 con 0x7fae781438f0 2023-10-10T20:24:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.205+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae70021350 con 0x7fae781438f0 2023-10-10T20:24:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.206+0000 7fae5f7fe700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 0x7fae64069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.206+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fae70083f60 con 0x7fae781438f0 2023-10-10T20:24:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.206+0000 7fae777fe700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 0x7fae64069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.206+0000 7fae777fe700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 0x7fae64069410 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fae78070ce0 tx=0x7fae68018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.210+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae70051520 con 0x7fae781438f0 2023-10-10T20:24:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.465+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fae780d0220 con 0x7fae781438f0 2023-10-10T20:24:52.467 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.465+0000 7fae5f7fe700 1 -- 172.21.15.148:0/1574193927 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fae7001e090 con 0x7fae781438f0 2023-10-10T20:24:52.467 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:52.468 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:24:52.468 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:24:52.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.468 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:24:52.469 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:24:52.469 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.469 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.469 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:52.469 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:52.470 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.470 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.470 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:24:52.471 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:24:52.471 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.471 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.471 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:52.472 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:52.472 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.473 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:24:52.473 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:24:52.473 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.473 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.473 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:52.474 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:52.474 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.475 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:24:52.475 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:24:52.475 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.475 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:52.476 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:52.476 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.476 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.477 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.477 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:24:52.477 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:24:52.477 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.478 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.478 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.478 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.478 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.479 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:24:52.479 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:24:52.479 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.479 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.481 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:24:52.481 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:24:52.481 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.481 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.482 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.482 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.482 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.482 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:24:52.483 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:24:52.483 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.484 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.484 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:24:52.484 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:24:52.485 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.485 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.485 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.485 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.486 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.486 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:24:52.486 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:24:52.487 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.487 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.487 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.487 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.487 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.488 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:24:52.488 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:24:52.488 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.489 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.489 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.489 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:24:52.490 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:24:52.491 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.491 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:24:52.492 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:24:52.492 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:24:52.492 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:24:52.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.493 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:24:52.493 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:24:52.494 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:52.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.495 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:24:52.496 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:24:52.496 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:52.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:52.497 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:52.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.497 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:24:52.498 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:24:52.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.499 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:24:52.499 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:24:52.500 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:52.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:24:52.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:24:52.500 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.501 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:24:52.501 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:24:52.502 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:52.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:24:52.502 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:24:52.503 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:24:52.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:24:52.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:24:52.504 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:24:52.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:52.505 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:24:52.505 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:24:52.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 msgr2=0x7fae64069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 0x7fae64069410 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fae78070ce0 tx=0x7fae68018000 comp rx=0 tx=0).stop 2023-10-10T20:24:52.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 msgr2=0x7fae780b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c880 tx=0x7fae70008040 comp rx=0 tx=0).stop 2023-10-10T20:24:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 shutdown_connections 2023-10-10T20:24:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fae64066f60 0x7fae64069410 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae781438f0 0x7fae780b1ad0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae780ccf00 0x7fae7815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 --2- 172.21.15.148:0/1574193927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae78143520 0x7fae780b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 >> 172.21.15.148:0/1574193927 conn(0x7fae78069750 msgr2=0x7fae780b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 shutdown_connections 2023-10-10T20:24:52.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.468+0000 7fae7cd6a700 1 -- 172.21.15.148:0/1574193927 wait complete. 2023-10-10T20:24:52.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.624+0000 7f5c5860a700 1 Processor -- start 2023-10-10T20:24:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.624+0000 7f5c5860a700 1 -- start start 2023-10-10T20:24:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 0x7f5c5006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c500c9af0 0x7f5c5015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c50070700 0x7f5c50070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c50071080 con 0x7f5c50070700 2023-10-10T20:24:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5015e280 con 0x7f5c500c9af0 2023-10-10T20:24:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c5860a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500ca0b0 con 0x7f5c501510f0 2023-10-10T20:24:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c50070700 0x7f5c50070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c50070700 0x7f5c50070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36878/0 (socket says 172.21.15.148:36878) 2023-10-10T20:24:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 -- 172.21.15.148:0/239403231 learned_addr learned my addr 172.21.15.148:0/239403231 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c55ba5700 1 --2- 172.21.15.148:0/239403231 >> v2:172.21.15.161:3300/0 conn(0x7f5c500c9af0 0x7f5c5015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c563a6700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 0x7f5c5006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 -- 172.21.15.148:0/239403231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 msgr2=0x7f5c5006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 0x7f5c5006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 -- 172.21.15.148:0/239403231 >> v2:172.21.15.161:3300/0 conn(0x7f5c500c9af0 msgr2=0x7f5c5015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 --2- 172.21.15.148:0/239403231 >> v2:172.21.15.161:3300/0 conn(0x7f5c500c9af0 0x7f5c5015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 -- 172.21.15.148:0/239403231 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50068a10 con 0x7f5c50070700 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c563a6700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 0x7f5c5006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c56ba7700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c50070700 0x7f5c50070b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5c5006d700 tx=0x7f5c48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be829861e5c9521 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c553a4700 1 -- 172.21.15.148:0/239403231 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800e070 con 0x7f5c50070700 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c553a4700 1 -- 172.21.15.148:0/239403231 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4800ba20 con 0x7f5c50070700 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.625+0000 7f5c553a4700 1 -- 172.21.15.148:0/239403231 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480092e0 con 0x7f5c50070700 2023-10-10T20:24:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 -- 172.21.15.148:0/239403231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c50070700 msgr2=0x7f5c50070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c50070700 0x7f5c50070b40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5c5006d700 tx=0x7f5c48009840 comp rx=0 tx=0).stop 2023-10-10T20:24:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 -- 172.21.15.148:0/239403231 shutdown_connections 2023-10-10T20:24:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c50070700 0x7f5c50070b40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 --2- 172.21.15.148:0/239403231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c501510f0 0x7f5c5006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 --2- 172.21.15.148:0/239403231 >> v2:172.21.15.161:3300/0 conn(0x7f5c500c9af0 0x7f5c5015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 -- 172.21.15.148:0/239403231 >> 172.21.15.148:0/239403231 conn(0x7f5c500ca4f0 msgr2=0x7f5c500ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 -- 172.21.15.148:0/239403231 shutdown_connections 2023-10-10T20:24:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.626+0000 7f5c5860a700 1 -- 172.21.15.148:0/239403231 wait complete. 2023-10-10T20:24:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.627+0000 7f5c5860a700 1 Processor -- start 2023-10-10T20:24:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.627+0000 7f5c5860a700 1 -- start start 2023-10-10T20:24:52.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 0x7f5c5015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 0x7f5c5016d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5016b4e0 con 0x7f5c500c9af0 2023-10-10T20:24:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5016a860 con 0x7f5c501510f0 2023-10-10T20:24:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c5860a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5016ab40 con 0x7f5c50070700 2023-10-10T20:24:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36892/0 (socket says 172.21.15.148:36892) 2023-10-10T20:24:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 -- 172.21.15.148:0/3733790415 learned_addr learned my addr 172.21.15.148:0/3733790415 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c56ba7700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 0x7f5c5016d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c563a6700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 0x7f5c5015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 -- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 msgr2=0x7f5c5015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 0x7f5c5015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 -- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 msgr2=0x7f5c5016d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 0x7f5c5016d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 -- 172.21.15.148:0/3733790415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009ae0 con 0x7f5c500c9af0 2023-10-10T20:24:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c563a6700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 0x7f5c5015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c56ba7700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 0x7f5c5016d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:52.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c55ba5700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ec80 tx=0x7f5c400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.628+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000c9d0 con 0x7f5c500c9af0 2023-10-10T20:24:52.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.629+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4000cb30 con 0x7f5c500c9af0 2023-10-10T20:24:52.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.629+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40013470 con 0x7f5c500c9af0 2023-10-10T20:24:52.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.629+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5016ae20 con 0x7f5c500c9af0 2023-10-10T20:24:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.629+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5015d050 con 0x7f5c500c9af0 2023-10-10T20:24:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.630+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5004c3c0 con 0x7f5c500c9af0 2023-10-10T20:24:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.630+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c40013820 con 0x7f5c500c9af0 2023-10-10T20:24:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.630+0000 7f5c467fc700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 0x7f5c340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.630+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5c400844f0 con 0x7f5c500c9af0 2023-10-10T20:24:52.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.630+0000 7f5c563a6700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 0x7f5c340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.631+0000 7f5c563a6700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 0x7f5c340693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006f00 tx=0x7f5c4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.635+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c40051ab0 con 0x7f5c500c9af0 2023-10-10T20:24:52.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.819+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c5006b620 con 0x7f5c500c9af0 2023-10-10T20:24:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.820+0000 7f5c467fc700 1 -- 172.21.15.148:0/3733790415 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+12185 (secure 0 0 0) 0x7f5c400550d0 con 0x7f5c500c9af0 2023-10-10T20:24:52.821 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:52.821 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":112,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:52.061687+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:52.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 msgr2=0x7f5c340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 0x7f5c340693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006f00 tx=0x7f5c4c008040 comp rx=0 tx=0).stop 2023-10-10T20:24:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 msgr2=0x7f5c5016d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ec80 tx=0x7f5c400091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 shutdown_connections 2023-10-10T20:24:52.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5c34066f30 0x7f5c340693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c501510f0 0x7f5c5016d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c50070700 0x7f5c5015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 --2- 172.21.15.148:0/3733790415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c500c9af0 0x7f5c5016d070 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 >> 172.21.15.148:0/3733790415 conn(0x7f5c500ca4f0 msgr2=0x7f5c500cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 shutdown_connections 2023-10-10T20:24:52.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.822+0000 7f5c5860a700 1 -- 172.21.15.148:0/3733790415 wait complete. 2023-10-10T20:24:52.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:24:52.836 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:24:52.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.952+0000 7ff4f0e34700 1 Processor -- start 2023-10-10T20:24:52.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.952+0000 7ff4f0e34700 1 -- start start 2023-10-10T20:24:52.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4f0e34700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4ec159ec0 0x7ff4ec0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4f0e34700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec15a290 0x7ff4ec0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4f0e34700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0b0790 con 0x7ff4ec06a5a0 2023-10-10T20:24:52.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4f0e34700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0ce1f0 con 0x7ff4ec159ec0 2023-10-10T20:24:52.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4f0e34700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0b0b20 con 0x7ff4ec15a290 2023-10-10T20:24:52.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ea59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4ec159ec0 0x7ff4ec0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ead9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff4ec06a5a0 0x7ff4ec15db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40476/0 (socket says 172.21.15.148:40476) 2023-10-10T20:24:52.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ead9d700 1 -- 172.21.15.148:0/3962893013 learned_addr learned my addr 172.21.15.148:0/3962893013 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:52.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4e3fff700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec15a290 0x7ff4ec0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ea59c700 1 -- 172.21.15.148:0/3962893013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec15a290 msgr2=0x7ff4ec0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:52.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ea59c700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec15a290 0x7ff4ec0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4ea59c700 1 -- 172.21.15.148:0/3962893013 >> v1:172.21.15.148:6789/0 conn(0x7ff4ec06a5a0 legacy=0x7ff4ec15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4ea59c700 1 -- 172.21.15.148:0/3962893013 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0b04b0 con 0x7ff4ec159ec0 2023-10-10T20:24:52.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.953+0000 7ff4e9d9b700 1 -- 172.21.15.148:0/3962893013 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279325504 0 0) 0x7ff4ec0b0790 con 0x7ff4ec06a5a0 2023-10-10T20:24:52.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4ea59c700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec159ec0 0x7ff4ec0af940 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e80a418a5289b2b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4e9d9b700 1 -- 172.21.15.148:0/3962893013 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc009e80 con 0x7ff4ec159ec0 2023-10-10T20:24:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4e9d9b700 1 -- 172.21.15.148:0/3962893013 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00b860 con 0x7ff4ec159ec0 2023-10-10T20:24:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4e9d9b700 1 -- 172.21.15.148:0/3962893013 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00bbe0 con 0x7ff4ec159ec0 2023-10-10T20:24:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 -- 172.21.15.148:0/3962893013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec159ec0 msgr2=0x7ff4ec0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec159ec0 0x7ff4ec0af940 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098e0 tx=0x7ff4dc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 -- 172.21.15.148:0/3962893013 shutdown_connections 2023-10-10T20:24:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec159ec0 0x7ff4ec0af940 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/3962893013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec15a290 0x7ff4ec0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 -- 172.21.15.148:0/3962893013 >> 172.21.15.148:0/3962893013 conn(0x7ff4ec0d1880 msgr2=0x7ff4ec0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 -- 172.21.15.148:0/3962893013 shutdown_connections 2023-10-10T20:24:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.954+0000 7ff4f0e34700 1 -- 172.21.15.148:0/3962893013 wait complete. 2023-10-10T20:24:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.955+0000 7ff4f0e34700 1 Processor -- start 2023-10-10T20:24:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.955+0000 7ff4f0e34700 1 -- start start 2023-10-10T20:24:52.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4ec06a5a0 0x7ff4ec0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec159ec0 0x7ff4ec0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c8870 con 0x7ff4ec15a290 2023-10-10T20:24:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c8ac0 con 0x7ff4ec06a5a0 2023-10-10T20:24:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4f0e34700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c8da0 con 0x7ff4ec159ec0 2023-10-10T20:24:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4ead9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4ead9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36914/0 (socket says 172.21.15.148:36914) 2023-10-10T20:24:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.956+0000 7ff4ead9d700 1 -- 172.21.15.148:0/1814724909 learned_addr learned my addr 172.21.15.148:0/1814724909 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4e3fff700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec159ec0 0x7ff4ec0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ea59c700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4ec06a5a0 0x7ff4ec0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 -- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec159ec0 msgr2=0x7ff4ec0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec159ec0 0x7ff4ec0cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 -- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4ec06a5a0 msgr2=0x7ff4ec0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:52.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4ec06a5a0 0x7ff4ec0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 -- 172.21.15.148:0/1814724909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc009420 con 0x7ff4ec15a290 2023-10-10T20:24:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4ead9d700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400cbf0 tx=0x7ff4e40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400a9c0 con 0x7ff4ec15a290 2023-10-10T20:24:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e400ab20 con 0x7ff4ec15a290 2023-10-10T20:24:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec0c9080 con 0x7ff4ec15a290 2023-10-10T20:24:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40070f0 con 0x7ff4ec15a290 2023-10-10T20:24:52.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.957+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec16a470 con 0x7ff4ec15a290 2023-10-10T20:24:52.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.958+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec04c3c0 con 0x7ff4ec15a290 2023-10-10T20:24:52.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.958+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4e4007520 con 0x7ff4ec15a290 2023-10-10T20:24:52.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.959+0000 7ff4e27fc700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.959+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff4e40845e0 con 0x7ff4ec15a290 2023-10-10T20:24:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.959+0000 7ff4ea59c700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.959+0000 7ff4ea59c700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 0x7ff4cc0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0091e0 tx=0x7ff4dc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:52.963+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4e4051c20 con 0x7ff4ec15a290 2023-10-10T20:24:53.087 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:24:53.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:24:53.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.148+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4ec0ceab0 con 0x7ff4ec15a290 2023-10-10T20:24:53.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.148+0000 7ff4e27fc700 1 -- 172.21.15.148:0/1814724909 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4e4055240 con 0x7ff4ec15a290 2023-10-10T20:24:53.150 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:24:53.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 msgr2=0x7ff4cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 0x7ff4cc0693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0091e0 tx=0x7ff4dc017000 comp rx=0 tx=0).stop 2023-10-10T20:24:53.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 msgr2=0x7ff4ec14e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400cbf0 tx=0x7ff4e40019b0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 shutdown_connections 2023-10-10T20:24:53.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff4cc066f30 0x7ff4cc0693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4ec15a290 0x7ff4ec14e840 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4ec06a5a0 0x7ff4ec0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 --2- 172.21.15.148:0/1814724909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4ec159ec0 0x7ff4ec0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 >> 172.21.15.148:0/1814724909 conn(0x7ff4ec0d1880 msgr2=0x7ff4ec0b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 shutdown_connections 2023-10-10T20:24:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.151+0000 7ff4f0e34700 1 -- 172.21.15.148:0/1814724909 wait complete. 2023-10-10T20:24:53.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:24:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.317+0000 7f3520c84700 1 Processor -- start 2023-10-10T20:24:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.317+0000 7f3520c84700 1 -- start start 2023-10-10T20:24:53.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f3520c84700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f351c157c40 0x7f351c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f3520c84700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c15cbb0 con 0x7f351c157870 2023-10-10T20:24:53.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f3520c84700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c15bfa0 con 0x7f351c15a620 2023-10-10T20:24:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f3520c84700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c15bdb0 con 0x7f351c157c40 2023-10-10T20:24:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f351ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f351c157c40 0x7f351c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f351ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f351c157c40 0x7f351c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36920/0 (socket says 172.21.15.148:36920) 2023-10-10T20:24:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f351a59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f351c157870 0x7f351c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40482/0 (socket says 172.21.15.148:40482) 2023-10-10T20:24:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.318+0000 7f351ad9d700 1 -- 172.21.15.148:0/319492816 learned_addr learned my addr 172.21.15.148:0/319492816 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f351ad9d700 1 -- 172.21.15.148:0/319492816 >> v1:172.21.15.161:6789/0 conn(0x7f351c15a620 legacy=0x7f351c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f351ad9d700 1 -- 172.21.15.148:0/319492816 >> v1:172.21.15.148:6789/0 conn(0x7f351c157870 legacy=0x7f351c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f351ad9d700 1 -- 172.21.15.148:0/319492816 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c15c280 con 0x7f351c157c40 2023-10-10T20:24:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f3519d9b700 1 -- 172.21.15.148:0/319492816 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932430179 0 0) 0x7f351c15cbb0 con 0x7f351c157870 2023-10-10T20:24:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f351ad9d700 1 --2- 172.21.15.148:0/319492816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c158110 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3514009f20 tx=0x7f3514009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199ced51b4cdeba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f3519d9b700 1 -- 172.21.15.148:0/319492816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e070 con 0x7f351c157c40 2023-10-10T20:24:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f3519d9b700 1 -- 172.21.15.148:0/319492816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400ba80 con 0x7f351c157c40 2023-10-10T20:24:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.319+0000 7f3519d9b700 1 -- 172.21.15.148:0/319492816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35140092e0 con 0x7f351c157c40 2023-10-10T20:24:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 -- 172.21.15.148:0/319492816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 msgr2=0x7f351c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 --2- 172.21.15.148:0/319492816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c158110 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3514009f20 tx=0x7f3514009840 comp rx=0 tx=0).stop 2023-10-10T20:24:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 -- 172.21.15.148:0/319492816 shutdown_connections 2023-10-10T20:24:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 --2- 172.21.15.148:0/319492816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c158110 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 -- 172.21.15.148:0/319492816 >> 172.21.15.148:0/319492816 conn(0x7f351c068ce0 msgr2=0x7f351c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 -- 172.21.15.148:0/319492816 shutdown_connections 2023-10-10T20:24:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.320+0000 7f3520c84700 1 -- 172.21.15.148:0/319492816 wait complete. 2023-10-10T20:24:53.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 Processor -- start 2023-10-10T20:24:53.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 -- start start 2023-10-10T20:24:53.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 0x7f351c138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f351c15a620 0x7f351c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0b3530 con 0x7f351c157c40 2023-10-10T20:24:53.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0b3250 con 0x7f351c15a620 2023-10-10T20:24:53.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.321+0000 7f3520c84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c14e840 con 0x7f351c157870 2023-10-10T20:24:53.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36922/0 (socket says 172.21.15.148:36922) 2023-10-10T20:24:53.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 -- 172.21.15.148:0/3043141408 learned_addr learned my addr 172.21.15.148:0/3043141408 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:53.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f351ad9d700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f351c15a620 0x7f351c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f351a59c700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 0x7f351c138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 -- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 msgr2=0x7f351c138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 0x7f351c138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 -- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f351c15a620 msgr2=0x7f351c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f351c15a620 0x7f351c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 -- 172.21.15.148:0/3043141408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009ae0 con 0x7f351c157c40 2023-10-10T20:24:53.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f351a59c700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 0x7f351c138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:53.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3513fff700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f350400c590 tx=0x7f3504007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504011590 con 0x7f351c157c40 2023-10-10T20:24:53.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504011710 con 0x7f351c157c40 2023-10-10T20:24:53.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c14eb20 con 0x7f351c157c40 2023-10-10T20:24:53.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504011ad0 con 0x7f351c157c40 2023-10-10T20:24:53.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.322+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c139380 con 0x7f351c157c40 2023-10-10T20:24:53.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.323+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c04c3c0 con 0x7f351c157c40 2023-10-10T20:24:53.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.324+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f350401a360 con 0x7f351c157c40 2023-10-10T20:24:53.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.324+0000 7f35127fc700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 0x7f34fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.324+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3504083170 con 0x7f351c157c40 2023-10-10T20:24:53.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.324+0000 7f351a59c700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 0x7f34fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.324+0000 7f351a59c700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 0x7f34fc0693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f350c0011e0 tx=0x7f350c007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.328+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3504050730 con 0x7f351c157c40 2023-10-10T20:24:53.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.514+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f351c15c560 con 0x7f351c157c40 2023-10-10T20:24:53.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.514+0000 7f35127fc700 1 -- 172.21.15.148:0/3043141408 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3504053d50 con 0x7f351c157c40 2023-10-10T20:24:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:53.516 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:24:53.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:24:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:24:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:24:53.517 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:24:53.517 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:24:53.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 msgr2=0x7f34fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 0x7f34fc0693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f350c0011e0 tx=0x7f350c007c70 comp rx=0 tx=0).stop 2023-10-10T20:24:53.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 msgr2=0x7f351c150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f350400c590 tx=0x7f3504007f50 comp rx=0 tx=0).stop 2023-10-10T20:24:53.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 shutdown_connections 2023-10-10T20:24:53.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f34fc066f30 0x7f34fc0693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f351c15a620 0x7f351c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f351c157870 0x7f351c138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 --2- 172.21.15.148:0/3043141408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f351c157c40 0x7f351c150f40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 >> 172.21.15.148:0/3043141408 conn(0x7f351c068ce0 msgr2=0x7f351c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:53.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 shutdown_connections 2023-10-10T20:24:53.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.516+0000 7f3520c84700 1 -- 172.21.15.148:0/3043141408 wait complete. 2023-10-10T20:24:53.530 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:24:53.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:24:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.647+0000 7ff55aeec700 1 Processor -- start 2023-10-10T20:24:53.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.647+0000 7ff55aeec700 1 -- start start 2023-10-10T20:24:53.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff55aeec700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5540b36c0 0x7ff554158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff55aeec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff5540ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff55aeec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff554159350 con 0x7ff5540b1950 2023-10-10T20:24:53.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff55aeec700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540b1110 con 0x7ff5540b36c0 2023-10-10T20:24:53.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff55aeec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540ae4e0 con 0x7ff554152520 2023-10-10T20:24:53.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff558c88700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5540b36c0 0x7ff554158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff558c88700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5540b36c0 0x7ff554158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36928/0 (socket says 172.21.15.148:36928) 2023-10-10T20:24:53.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.648+0000 7ff558c88700 1 -- 172.21.15.148:0/1588816947 learned_addr learned my addr 172.21.15.148:0/1588816947 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:53.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff559489700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff5540ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff558c88700 1 -- 172.21.15.148:0/1588816947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 msgr2=0x7ff5540ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:53.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff558c88700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff5540ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff558c88700 1 -- 172.21.15.148:0/1588816947 >> v1:172.21.15.161:6789/0 conn(0x7ff5540b1950 legacy=0x7ff554151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff558c88700 1 -- 172.21.15.148:0/1588816947 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540b1ea0 con 0x7ff5540b36c0 2023-10-10T20:24:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff558c88700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b36c0 0x7ff554158e10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff550009f20 tx=0x7ff550009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29653d2e4592e70e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff54b7fe700 1 -- 172.21.15.148:0/1588816947 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55000e070 con 0x7ff5540b36c0 2023-10-10T20:24:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff54b7fe700 1 -- 172.21.15.148:0/1588816947 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55000ba80 con 0x7ff5540b36c0 2023-10-10T20:24:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.649+0000 7ff54b7fe700 1 -- 172.21.15.148:0/1588816947 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500092e0 con 0x7ff5540b36c0 2023-10-10T20:24:53.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 -- 172.21.15.148:0/1588816947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b36c0 msgr2=0x7ff554158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b36c0 0x7ff554158e10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff550009f20 tx=0x7ff550009840 comp rx=0 tx=0).stop 2023-10-10T20:24:53.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 -- 172.21.15.148:0/1588816947 shutdown_connections 2023-10-10T20:24:53.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b36c0 0x7ff554158e10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1588816947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff5540ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 -- 172.21.15.148:0/1588816947 >> 172.21.15.148:0/1588816947 conn(0x7ff554069870 msgr2=0x7ff554069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:53.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 -- 172.21.15.148:0/1588816947 shutdown_connections 2023-10-10T20:24:53.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.650+0000 7ff55aeec700 1 -- 172.21.15.148:0/1588816947 wait complete. 2023-10-10T20:24:53.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.651+0000 7ff55aeec700 1 Processor -- start 2023-10-10T20:24:53.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.651+0000 7ff55aeec700 1 -- start start 2023-10-10T20:24:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5540b36c0 0x7ff554150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff554153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554154f60 con 0x7ff5540b1950 2023-10-10T20:24:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541510e0 con 0x7ff5540b36c0 2023-10-10T20:24:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff554154350 con 0x7ff554152520 2023-10-10T20:24:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36932/0 (socket says 172.21.15.148:36932) 2023-10-10T20:24:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 -- 172.21.15.148:0/1793590130 learned_addr learned my addr 172.21.15.148:0/1793590130 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff54bfff700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5540b36c0 0x7ff554150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff559489700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff554153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 -- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 msgr2=0x7ff554153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff554153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 -- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5540b36c0 msgr2=0x7ff554150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5540b36c0 0x7ff554150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 -- 172.21.15.148:0/1793590130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550009ae0 con 0x7ff5540b1950 2023-10-10T20:24:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff558c88700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b6c0 tx=0x7ff550018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500088b0 con 0x7ff5540b1950 2023-10-10T20:24:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff550008a30 con 0x7ff5540b1950 2023-10-10T20:24:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554154630 con 0x7ff5540b1950 2023-10-10T20:24:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550008df0 con 0x7ff5540b1950 2023-10-10T20:24:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.652+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff554154bf0 con 0x7ff5540b1950 2023-10-10T20:24:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.653+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55406b900 con 0x7ff5540b1950 2023-10-10T20:24:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.654+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff550003040 con 0x7ff5540b1950 2023-10-10T20:24:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.654+0000 7ff548ff9700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 0x7ff5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.654+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff550085f20 con 0x7ff5540b1950 2023-10-10T20:24:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.654+0000 7ff54bfff700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 0x7ff5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.654+0000 7ff54bfff700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 0x7ff5400693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff5540cf680 tx=0x7ff544000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.659+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5500534e0 con 0x7ff5540b1950 2023-10-10T20:24:53.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:53.841+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff55404c3c0 con 0x7ff5540b1950 2023-10-10T20:24:54.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.076+0000 7ff548ff9700 1 -- 172.21.15.148:0/1793590130 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff550056b00 con 0x7ff5540b1950 2023-10-10T20:24:54.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 msgr2=0x7ff5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 0x7ff5400693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff5540cf680 tx=0x7ff544000bc0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 msgr2=0x7ff5540688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b6c0 tx=0x7ff550018000 comp rx=0 tx=0).stop 2023-10-10T20:24:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 shutdown_connections 2023-10-10T20:24:54.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff540066f30 0x7ff5400693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff554152520 0x7ff554153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5540b1950 0x7ff5540688f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 --2- 172.21.15.148:0/1793590130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5540b36c0 0x7ff554150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 >> 172.21.15.148:0/1793590130 conn(0x7ff554069870 msgr2=0x7ff554159570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:54.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 shutdown_connections 2023-10-10T20:24:54.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.079+0000 7ff55aeec700 1 -- 172.21.15.148:0/1793590130 wait complete. 2023-10-10T20:24:54.083 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:24:54.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:24:54.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.212+0000 7f59949b8700 1 Processor -- start 2023-10-10T20:24:54.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.212+0000 7f59949b8700 1 -- start start 2023-10-10T20:24:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f59949b8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59900d05a0 0x7f5990164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f59949b8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59900b0790 con 0x7f599015a290 2023-10-10T20:24:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f59949b8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59900ce1f0 con 0x7f5990159ec0 2023-10-10T20:24:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f59949b8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59900b0b20 con 0x7f59900d05a0 2023-10-10T20:24:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f5985d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f599015a290 0x7f599006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40492/0 (socket says 172.21.15.148:40492) 2023-10-10T20:24:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f5985d9b700 1 -- 172.21.15.148:0/572511546 learned_addr learned my addr 172.21.15.148:0/572511546 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:54.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.213+0000 7f598ed9d700 1 --2- 172.21.15.148:0/572511546 >> v2:172.21.15.148:3300/0 conn(0x7f59900d05a0 0x7f5990164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:54.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598dd9b700 1 -- 172.21.15.148:0/572511546 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562321793 0 0) 0x7f59900b0790 con 0x7f599015a290 2023-10-10T20:24:54.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598dd9b700 1 -- 172.21.15.148:0/572511546 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f597c003570 con 0x7f599015a290 2023-10-10T20:24:54.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598ed9d700 1 -- 172.21.15.148:0/572511546 >> v1:172.21.15.161:6789/0 conn(0x7f5990159ec0 legacy=0x7f59900af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598ed9d700 1 -- 172.21.15.148:0/572511546 >> v1:172.21.15.148:6789/0 conn(0x7f599015a290 legacy=0x7f599006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598ed9d700 1 -- 172.21.15.148:0/572511546 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59900b04b0 con 0x7f59900d05a0 2023-10-10T20:24:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598ed9d700 1 --2- 172.21.15.148:0/572511546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f5990164ed0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5988009e60 tx=0x7f5988009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79e55a8999cc5be server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598dd9b700 1 -- 172.21.15.148:0/572511546 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800e070 con 0x7f59900d05a0 2023-10-10T20:24:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598dd9b700 1 -- 172.21.15.148:0/572511546 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598800bb20 con 0x7f59900d05a0 2023-10-10T20:24:54.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f598dd9b700 1 -- 172.21.15.148:0/572511546 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800bea0 con 0x7f59900d05a0 2023-10-10T20:24:54.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f59949b8700 1 -- 172.21.15.148:0/572511546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 msgr2=0x7f5990164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f59949b8700 1 --2- 172.21.15.148:0/572511546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f5990164ed0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5988009e60 tx=0x7f5988009780 comp rx=0 tx=0).stop 2023-10-10T20:24:54.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f59949b8700 1 -- 172.21.15.148:0/572511546 shutdown_connections 2023-10-10T20:24:54.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.214+0000 7f59949b8700 1 --2- 172.21.15.148:0/572511546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f5990164ed0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.215+0000 7f59949b8700 1 -- 172.21.15.148:0/572511546 >> 172.21.15.148:0/572511546 conn(0x7f59900d1880 msgr2=0x7f59900cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.215+0000 7f59949b8700 1 -- 172.21.15.148:0/572511546 shutdown_connections 2023-10-10T20:24:54.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.215+0000 7f59949b8700 1 -- 172.21.15.148:0/572511546 wait complete. 2023-10-10T20:24:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 Processor -- start 2023-10-10T20:24:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 -- start start 2023-10-10T20:24:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5990159ec0 0x7f59900cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f599015a290 0x7f5990151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59900b1cc0 con 0x7f59900d05a0 2023-10-10T20:24:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.216+0000 7f59949b8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59900b10b0 con 0x7f5990159ec0 2023-10-10T20:24:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f59949b8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59900b1390 con 0x7f599015a290 2023-10-10T20:24:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36938/0 (socket says 172.21.15.148:36938) 2023-10-10T20:24:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 -- 172.21.15.148:0/1330695481 learned_addr learned my addr 172.21.15.148:0/1330695481 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f5985d9b700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5990159ec0 0x7f59900cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598ed9d700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f599015a290 0x7f5990151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 -- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f599015a290 msgr2=0x7f5990151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f599015a290 0x7f5990151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 -- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5990159ec0 msgr2=0x7f59900cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:54.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5990159ec0 0x7f59900cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:54.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 -- 172.21.15.148:0/1330695481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988009a20 con 0x7f59900d05a0 2023-10-10T20:24:54.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f598e59c700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f598000c5d0 tx=0x7f5980007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011590 con 0x7f59900d05a0 2023-10-10T20:24:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980011710 con 0x7f59900d05a0 2023-10-10T20:24:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59900b1670 con 0x7f59900d05a0 2023-10-10T20:24:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011ad0 con 0x7f59900d05a0 2023-10-10T20:24:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.217+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59900c83a0 con 0x7f59900d05a0 2023-10-10T20:24:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.218+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599004c3c0 con 0x7f59900d05a0 2023-10-10T20:24:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.219+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f598001c460 con 0x7f59900d05a0 2023-10-10T20:24:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.219+0000 7f5986ffd700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 0x7f59700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.219+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f5980084020 con 0x7f59900d05a0 2023-10-10T20:24:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.219+0000 7f5985d9b700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 0x7f59700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.219+0000 7f5985d9b700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 0x7f59700693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f59900c7b50 tx=0x7f5978009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:54.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.224+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5980051470 con 0x7f59900d05a0 2023-10-10T20:24:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:54.406+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f59900ceab0 con 0x7f59900d05a0 2023-10-10T20:24:55.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.080+0000 7f5986ffd700 1 -- 172.21.15.148:0/1330695481 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5980054a90 con 0x7f59900d05a0 2023-10-10T20:24:55.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 msgr2=0x7f59700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 0x7f59700693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f59900c7b50 tx=0x7f5978009040 comp rx=0 tx=0).stop 2023-10-10T20:24:55.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 msgr2=0x7f59900c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f598000c5d0 tx=0x7f5980007f60 comp rx=0 tx=0).stop 2023-10-10T20:24:55.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 shutdown_connections 2023-10-10T20:24:55.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5970066f30 0x7f59700693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f599015a290 0x7f5990151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59900d05a0 0x7f59900c7e60 secure :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f598000c5d0 tx=0x7f5980007f60 comp rx=0 tx=0).stop 2023-10-10T20:24:55.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 --2- 172.21.15.148:0/1330695481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5990159ec0 0x7f59900cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 >> 172.21.15.148:0/1330695481 conn(0x7f59900d1880 msgr2=0x7f59900cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:55.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.091+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 shutdown_connections 2023-10-10T20:24:55.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.092+0000 7f59949b8700 1 -- 172.21.15.148:0/1330695481 wait complete. 2023-10-10T20:24:55.095 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:24:55.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:24:55.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.222+0000 7fed46502700 1 Processor -- start 2023-10-10T20:24:55.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.222+0000 7fed46502700 1 -- start start 2023-10-10T20:24:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed46502700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 0x7fed4006a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed46502700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed400b0790 con 0x7fed4006aae0 2023-10-10T20:24:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed46502700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed400ce1f0 con 0x7fed40159ec0 2023-10-10T20:24:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed46502700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4006af60 con 0x7fed4015a290 2023-10-10T20:24:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed44a9f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fed4006aae0 0x7fed400cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40504/0 (socket says 172.21.15.148:40504) 2023-10-10T20:24:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed44a9f700 1 -- 172.21.15.148:0/89357183 learned_addr learned my addr 172.21.15.148:0/89357183 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.223+0000 7fed3f7fe700 1 --2- 172.21.15.148:0/89357183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 0x7fed4006a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:55.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315858032 0 0) 0x7fed400b0790 con 0x7fed4006aae0 2023-10-10T20:24:55.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed28003570 con 0x7fed4006aae0 2023-10-10T20:24:55.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160667268 0 0) 0x7fed400ce1f0 con 0x7fed40159ec0 2023-10-10T20:24:55.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed400b0790 con 0x7fed40159ec0 2023-10-10T20:24:55.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3f7fe700 1 -- 172.21.15.148:0/89357183 >> v1:172.21.15.161:6789/0 conn(0x7fed40159ec0 legacy=0x7fed400af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3f7fe700 1 -- 172.21.15.148:0/89357183 >> v1:172.21.15.148:6789/0 conn(0x7fed4006aae0 legacy=0x7fed400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964712257 0 0) 0x7fed28003570 con 0x7fed4006aae0 2023-10-10T20:24:55.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3f7fe700 1 -- 172.21.15.148:0/89357183 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400b0b20 con 0x7fed4015a290 2023-10-10T20:24:55.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3f7fe700 1 --2- 172.21.15.148:0/89357183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 0x7fed4006a5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fed34009910 tx=0x7fed340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28b2a04b2be2d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:55.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed380017f0 con 0x7fed4015a290 2023-10-10T20:24:55.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed34011420 con 0x7fed4015a290 2023-10-10T20:24:55.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.224+0000 7fed3effd700 1 -- 172.21.15.148:0/89357183 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340117c0 con 0x7fed4015a290 2023-10-10T20:24:55.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 -- 172.21.15.148:0/89357183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 msgr2=0x7fed4006a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 --2- 172.21.15.148:0/89357183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 0x7fed4006a5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fed34009910 tx=0x7fed340091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 -- 172.21.15.148:0/89357183 shutdown_connections 2023-10-10T20:24:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 --2- 172.21.15.148:0/89357183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4015a290 0x7fed4006a5a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 -- 172.21.15.148:0/89357183 >> 172.21.15.148:0/89357183 conn(0x7fed400d1880 msgr2=0x7fed400cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 -- 172.21.15.148:0/89357183 shutdown_connections 2023-10-10T20:24:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.225+0000 7fed46502700 1 -- 172.21.15.148:0/89357183 wait complete. 2023-10-10T20:24:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.226+0000 7fed46502700 1 Processor -- start 2023-10-10T20:24:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.226+0000 7fed46502700 1 -- start start 2023-10-10T20:24:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4006aae0 0x7fed4015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40159ec0 0x7fed4015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4016b630 con 0x7fed4015a290 2023-10-10T20:24:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed400ca7d0 con 0x7fed40159ec0 2023-10-10T20:24:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed46502700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4015d450 con 0x7fed4006aae0 2023-10-10T20:24:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36944/0 (socket says 172.21.15.148:36944) 2023-10-10T20:24:55.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 -- 172.21.15.148:0/2300174743 learned_addr learned my addr 172.21.15.148:0/2300174743 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:55.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed3f7fe700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40159ec0 0x7fed4015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:55.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed3ffff700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4006aae0 0x7fed4015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:55.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 -- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4006aae0 msgr2=0x7fed4015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4006aae0 0x7fed4015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 -- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40159ec0 msgr2=0x7fed4015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:55.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40159ec0 0x7fed4015abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:55.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 -- 172.21.15.148:0/2300174743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009420 con 0x7fed4015a290 2023-10-10T20:24:55.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed44a9f700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed3800cc80 tx=0x7fed38007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:55.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.227+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800ce30 con 0x7fed4015a290 2023-10-10T20:24:55.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.228+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4015cb80 con 0x7fed4015a290 2023-10-10T20:24:55.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.228+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38004dc0 con 0x7fed4015a290 2023-10-10T20:24:55.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.228+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800ab70 con 0x7fed4015a290 2023-10-10T20:24:55.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.228+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4015ce00 con 0x7fed4015a290 2023-10-10T20:24:55.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.229+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed400b4280 con 0x7fed4015a290 2023-10-10T20:24:55.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.229+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed38007150 con 0x7fed4015a290 2023-10-10T20:24:55.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.229+0000 7fed27fff700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:55.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.229+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fed38083cb0 con 0x7fed4015a290 2023-10-10T20:24:55.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.229+0000 7fed3ffff700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:55.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.230+0000 7fed3ffff700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 0x7fed2c0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fed30003870 tx=0x7fed30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:55.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.234+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed400b4280 con 0x7fed4015a290 2023-10-10T20:24:55.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:55.465+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fed400b4280 con 0x7fed4015a290 2023-10-10T20:24:56.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.081+0000 7fed27fff700 1 -- 172.21.15.148:0/2300174743 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7fed400b4280 con 0x7fed4015a290 2023-10-10T20:24:56.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 msgr2=0x7fed2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 0x7fed2c0693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fed30003870 tx=0x7fed30009040 comp rx=0 tx=0).stop 2023-10-10T20:24:56.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 msgr2=0x7fed4015b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed3800cc80 tx=0x7fed38007d20 comp rx=0 tx=0).stop 2023-10-10T20:24:56.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 shutdown_connections 2023-10-10T20:24:56.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed2c066f30 0x7fed2c0693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed4015a290 0x7fed4015b100 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed4006aae0 0x7fed4015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 --2- 172.21.15.148:0/2300174743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed40159ec0 0x7fed4015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 >> 172.21.15.148:0/2300174743 conn(0x7fed400d1880 msgr2=0x7fed400b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 shutdown_connections 2023-10-10T20:24:56.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.086+0000 7fed46502700 1 -- 172.21.15.148:0/2300174743 wait complete. 2023-10-10T20:24:56.091 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 14 and data pool 15 2023-10-10T20:24:56.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:56.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.215+0000 7f4ef8eb5700 1 Processor -- start 2023-10-10T20:24:56.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.215+0000 7f4ef8eb5700 1 -- start start 2023-10-10T20:24:56.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef8eb5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ef4068de0 0x7f4ef40cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef8eb5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 0x7f4ef4067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef8eb5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef4067fc0 con 0x7f4ef4159f80 2023-10-10T20:24:56.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef8eb5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef415a350 con 0x7f4ef4068de0 2023-10-10T20:24:56.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef8eb5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4069730 con 0x7f4ef4067660 2023-10-10T20:24:56.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ef4068de0 0x7f4ef40cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ef4068de0 0x7f4ef40cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36956/0 (socket says 172.21.15.148:36956) 2023-10-10T20:24:56.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 -- 172.21.15.148:0/3162632206 learned_addr learned my addr 172.21.15.148:0/3162632206 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:56.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef37fe700 1 --1- 172.21.15.148:0/3162632206 >> v1:172.21.15.148:6789/0 conn(0x7f4ef4159f80 0x7f4ef40cc060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40508/0 (socket says 172.21.15.148:40508) 2023-10-10T20:24:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef3fff700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 0x7f4ef4067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 -- 172.21.15.148:0/3162632206 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 msgr2=0x7f4ef4067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 0x7f4ef4067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 -- 172.21.15.148:0/3162632206 >> v1:172.21.15.148:6789/0 conn(0x7f4ef4159f80 legacy=0x7f4ef40cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef27fc700 1 -- 172.21.15.148:0/3162632206 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763304645 0 0) 0x7f4ef4067fc0 con 0x7f4ef4159f80 2023-10-10T20:24:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef2ffd700 1 -- 172.21.15.148:0/3162632206 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40cf920 con 0x7f4ef4068de0 2023-10-10T20:24:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.216+0000 7f4ef3fff700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 0x7f4ef4067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4068de0 0x7f4ef40cf3e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099b0 tx=0x7f4ee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ffe0bc93f9a45 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef27fc700 1 -- 172.21.15.148:0/3162632206 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800b860 con 0x7f4ef4068de0 2023-10-10T20:24:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef27fc700 1 -- 172.21.15.148:0/3162632206 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee800b9c0 con 0x7f4ef4068de0 2023-10-10T20:24:56.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef27fc700 1 -- 172.21.15.148:0/3162632206 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800bd60 con 0x7f4ef4068de0 2023-10-10T20:24:56.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/3162632206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4068de0 msgr2=0x7f4ef40cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4068de0 0x7f4ef40cf3e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099b0 tx=0x7f4ee8009250 comp rx=0 tx=0).stop 2023-10-10T20:24:56.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/3162632206 shutdown_connections 2023-10-10T20:24:56.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4067660 0x7f4ef4067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/3162632206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4068de0 0x7f4ef40cf3e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/3162632206 >> 172.21.15.148:0/3162632206 conn(0x7f4ef406c700 msgr2=0x7f4ef4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/3162632206 shutdown_connections 2023-10-10T20:24:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.217+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/3162632206 wait complete. 2023-10-10T20:24:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.218+0000 7f4ef8eb5700 1 Processor -- start 2023-10-10T20:24:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.218+0000 7f4ef8eb5700 1 -- start start 2023-10-10T20:24:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 0x7f4ef40ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 0x7f4ef415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4154de0 con 0x7f4ef4067660 2023-10-10T20:24:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4154540 con 0x7f4ef4068de0 2023-10-10T20:24:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef8eb5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4154820 con 0x7f4ef4159f80 2023-10-10T20:24:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36960/0 (socket says 172.21.15.148:36960) 2023-10-10T20:24:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.219+0000 7f4ef37fe700 1 -- 172.21.15.148:0/4244710616 learned_addr learned my addr 172.21.15.148:0/4244710616 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef3fff700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 0x7f4ef415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 0x7f4ef40ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 -- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 msgr2=0x7f4ef415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 0x7f4ef415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 -- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 msgr2=0x7f4ef40ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 0x7f4ef40ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 -- 172.21.15.148:0/4244710616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80094f0 con 0x7f4ef4067660 2023-10-10T20:24:56.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef3fff700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 0x7f4ef415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:56.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 0x7f4ef40ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:56.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef37fe700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400cd30 tx=0x7f4ee4001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4005df0 con 0x7f4ef4067660 2023-10-10T20:24:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee400a960 con 0x7f4ef4067660 2023-10-10T20:24:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef4154b00 con 0x7f4ef4067660 2023-10-10T20:24:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400ad20 con 0x7f4ef4067660 2023-10-10T20:24:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.220+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef4151a10 con 0x7f4ef4067660 2023-10-10T20:24:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.221+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef40aa870 con 0x7f4ef4067660 2023-10-10T20:24:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.221+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ee4007300 con 0x7f4ef4067660 2023-10-10T20:24:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.222+0000 7f4edb7fe700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 0x7f4ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.222+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ee4084700 con 0x7f4ef4067660 2023-10-10T20:24:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.222+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 0x7f4ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.222+0000 7f4ef2ffd700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 0x7f4ee00693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099b0 tx=0x7f4ee8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.226+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ef40aa870 con 0x7f4ef4067660 2023-10-10T20:24:56.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.414+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef40aa870 con 0x7f4ef4067660 2023-10-10T20:24:56.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.415+0000 7f4edb7fe700 1 -- 172.21.15.148:0/4244710616 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f4ef40aa870 con 0x7f4ef4067660 2023-10-10T20:24:56.417 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:56.418 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":115,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:55.465797+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:56.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 msgr2=0x7f4ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 0x7f4ee00693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099b0 tx=0x7f4ee8017000 comp rx=0 tx=0).stop 2023-10-10T20:24:56.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 msgr2=0x7f4ef41514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400cd30 tx=0x7f4ee4001af0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 shutdown_connections 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4ee0066f30 0x7f4ee00693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ef4159f80 0x7f4ef415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef4067660 0x7f4ef41514d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 --2- 172.21.15.148:0/4244710616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ef4068de0 0x7f4ef40ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 >> 172.21.15.148:0/4244710616 conn(0x7f4ef406c700 msgr2=0x7f4ef40cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 shutdown_connections 2023-10-10T20:24:56.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.417+0000 7f4ef8eb5700 1 -- 172.21.15.148:0/4244710616 wait complete. 2023-10-10T20:24:56.432 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:56.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.546+0000 7f971da4e700 1 Processor -- start 2023-10-10T20:24:56.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.546+0000 7f971da4e700 1 -- start start 2023-10-10T20:24:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f971da4e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f971815a620 0x7f971815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f971da4e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971815cbb0 con 0x7f971815d580 2023-10-10T20:24:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f971da4e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971815bfa0 con 0x7f9718157870 2023-10-10T20:24:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f971da4e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971815bdb0 con 0x7f971815a620 2023-10-10T20:24:56.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97167fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f971815a620 0x7f971815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97177fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f971815d580 0x7f9718158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40510/0 (socket says 172.21.15.148:40510) 2023-10-10T20:24:56.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97167fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f971815a620 0x7f971815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36962/0 (socket says 172.21.15.148:36962) 2023-10-10T20:24:56.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97177fe700 1 -- 172.21.15.148:0/2672930711 learned_addr learned my addr 172.21.15.148:0/2672930711 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:56.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97167fc700 1 -- 172.21.15.148:0/2672930711 >> v1:172.21.15.161:6789/0 conn(0x7f9718157870 legacy=0x7f971815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:56.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97167fc700 1 -- 172.21.15.148:0/2672930711 >> v1:172.21.15.148:6789/0 conn(0x7f971815d580 legacy=0x7f9718158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f9715ffb700 1 -- 172.21.15.148:0/2672930711 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666563540 0 0) 0x7f971815cbb0 con 0x7f971815d580 2023-10-10T20:24:56.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.547+0000 7f97167fc700 1 -- 172.21.15.148:0/2672930711 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971815c280 con 0x7f971815a620 2023-10-10T20:24:56.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f97167fc700 1 --2- 172.21.15.148:0/2672930711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f971815d040 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098e0 tx=0x7f970c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3aaa1d7a3b3d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f9715ffb700 1 -- 172.21.15.148:0/2672930711 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c009e80 con 0x7f971815a620 2023-10-10T20:24:56.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f9715ffb700 1 -- 172.21.15.148:0/2672930711 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c00b820 con 0x7f971815a620 2023-10-10T20:24:56.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f9715ffb700 1 -- 172.21.15.148:0/2672930711 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c00bba0 con 0x7f971815a620 2023-10-10T20:24:56.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 -- 172.21.15.148:0/2672930711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 msgr2=0x7f971815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 --2- 172.21.15.148:0/2672930711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f971815d040 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098e0 tx=0x7f970c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 -- 172.21.15.148:0/2672930711 shutdown_connections 2023-10-10T20:24:56.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 --2- 172.21.15.148:0/2672930711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f971815d040 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 -- 172.21.15.148:0/2672930711 >> 172.21.15.148:0/2672930711 conn(0x7f9718068ce0 msgr2=0x7f97180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 -- 172.21.15.148:0/2672930711 shutdown_connections 2023-10-10T20:24:56.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.548+0000 7f971da4e700 1 -- 172.21.15.148:0/2672930711 wait complete. 2023-10-10T20:24:56.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 Processor -- start 2023-10-10T20:24:56.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 -- start start 2023-10-10T20:24:56.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 0x7f971814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 0x7f971814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180b1d40 con 0x7f971815a620 2023-10-10T20:24:56.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181556b0 con 0x7f9718157870 2023-10-10T20:24:56.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f971da4e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9718155990 con 0x7f971815d580 2023-10-10T20:24:56.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f97167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f97167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36976/0 (socket says 172.21.15.148:36976) 2023-10-10T20:24:56.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.550+0000 7f97167fc700 1 -- 172.21.15.148:0/1098823862 learned_addr learned my addr 172.21.15.148:0/1098823862 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:56.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97177fe700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 0x7f971814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f9716ffd700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 0x7f971814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 -- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 msgr2=0x7f971814b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 0x7f971814b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 -- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 msgr2=0x7f971814ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 0x7f971814ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 -- 172.21.15.148:0/1098823862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009420 con 0x7f971815a620 2023-10-10T20:24:56.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f9716ffd700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 0x7f971814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:56.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97177fe700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 0x7f971814b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:56.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f97167fc700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f970c0030c0 tx=0x7f970c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c00ba10 con 0x7f971815a620 2023-10-10T20:24:56.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c020070 con 0x7f971815a620 2023-10-10T20:24:56.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9718155c70 con 0x7f971815a620 2023-10-10T20:24:56.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c01b6a0 con 0x7f971815a620 2023-10-10T20:24:56.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.551+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9718155270 con 0x7f971815a620 2023-10-10T20:24:56.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.552+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97180b1ef0 con 0x7f971815a620 2023-10-10T20:24:56.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.552+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f970c022030 con 0x7f971815a620 2023-10-10T20:24:56.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.553+0000 7f96feffd700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 0x7f97040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.553+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f970c0865c0 con 0x7f971815a620 2023-10-10T20:24:56.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.553+0000 7f9716ffd700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 0x7f97040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.553+0000 7f9716ffd700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 0x7f97040693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9708007fd0 tx=0x7f9708009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.557+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97180b1ef0 con 0x7f971815a620 2023-10-10T20:24:56.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.739+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f971804c3c0 con 0x7f971815a620 2023-10-10T20:24:56.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.740+0000 7f96feffd700 1 -- 172.21.15.148:0/1098823862 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+15413 (secure 0 0 0) 0x7f970c084e60 con 0x7f971815a620 2023-10-10T20:24:56.743 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:56.743 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":116,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:56.523284+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 msgr2=0x7f97040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 0x7f97040693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9708007fd0 tx=0x7f9708009040 comp rx=0 tx=0).stop 2023-10-10T20:24:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 msgr2=0x7f9718154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f970c0030c0 tx=0x7f970c016000 comp rx=0 tx=0).stop 2023-10-10T20:24:56.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 shutdown_connections 2023-10-10T20:24:56.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9704066f30 0x7f97040693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f971815d580 0x7f971814b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9718157870 0x7f971814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 --2- 172.21.15.148:0/1098823862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f971815a620 0x7f9718154d30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 >> 172.21.15.148:0/1098823862 conn(0x7f9718068ce0 msgr2=0x7f971814c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 shutdown_connections 2023-10-10T20:24:56.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.742+0000 7f971da4e700 1 -- 172.21.15.148:0/1098823862 wait complete. 2023-10-10T20:24:56.757 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:24:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.873+0000 7f20c934f700 1 Processor -- start 2023-10-10T20:24:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.873+0000 7f20c934f700 1 -- start start 2023-10-10T20:24:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c934f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20c415b9a0 0x7f20c40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c934f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c406d370 0x7f20c406d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c934f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c415c8d0 con 0x7f20c4152300 2023-10-10T20:24:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c934f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40b1400 con 0x7f20c406d370 2023-10-10T20:24:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c934f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c415be20 con 0x7f20c415b9a0 2023-10-10T20:24:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f20c4152300 0x7f20c4158df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40520/0 (socket says 172.21.15.148:40520) 2023-10-10T20:24:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c37fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c406d370 0x7f20c406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c2ffd700 1 -- 172.21.15.148:0/1822401359 learned_addr learned my addr 172.21.15.148:0/1822401359 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c27fc700 1 --2- 172.21.15.148:0/1822401359 >> v2:172.21.15.161:3300/0 conn(0x7f20c415b9a0 0x7f20c40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c37fe700 1 -- 172.21.15.148:0/1822401359 >> v2:172.21.15.161:3300/0 conn(0x7f20c415b9a0 msgr2=0x7f20c40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c37fe700 1 --2- 172.21.15.148:0/1822401359 >> v2:172.21.15.161:3300/0 conn(0x7f20c415b9a0 0x7f20c40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c37fe700 1 -- 172.21.15.148:0/1822401359 >> v1:172.21.15.148:6789/0 conn(0x7f20c4152300 legacy=0x7f20c4158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c1ffb700 1 -- 172.21.15.148:0/1822401359 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751267897 0 0) 0x7f20c415c8d0 con 0x7f20c4152300 2023-10-10T20:24:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.874+0000 7f20c37fe700 1 -- 172.21.15.148:0/1822401359 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40c9370 con 0x7f20c406d370 2023-10-10T20:24:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c37fe700 1 --2- 172.21.15.148:0/1822401359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c406d370 0x7f20c406d790 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f20c406e8d0 tx=0x7f20b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b074f784dfd35d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c1ffb700 1 -- 172.21.15.148:0/1822401359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b400b6b0 con 0x7f20c406d370 2023-10-10T20:24:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c1ffb700 1 -- 172.21.15.148:0/1822401359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b400b810 con 0x7f20c406d370 2023-10-10T20:24:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c1ffb700 1 -- 172.21.15.148:0/1822401359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b400bbb0 con 0x7f20c406d370 2023-10-10T20:24:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 -- 172.21.15.148:0/1822401359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c406d370 msgr2=0x7f20c406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 --2- 172.21.15.148:0/1822401359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c406d370 0x7f20c406d790 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f20c406e8d0 tx=0x7f20b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 -- 172.21.15.148:0/1822401359 shutdown_connections 2023-10-10T20:24:56.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 --2- 172.21.15.148:0/1822401359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c406d370 0x7f20c406d790 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 --2- 172.21.15.148:0/1822401359 >> v2:172.21.15.161:3300/0 conn(0x7f20c415b9a0 0x7f20c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 -- 172.21.15.148:0/1822401359 >> 172.21.15.148:0/1822401359 conn(0x7f20c4069870 msgr2=0x7f20c4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:56.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 -- 172.21.15.148:0/1822401359 shutdown_connections 2023-10-10T20:24:56.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.875+0000 7f20c934f700 1 -- 172.21.15.148:0/1822401359 wait complete. 2023-10-10T20:24:56.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 Processor -- start 2023-10-10T20:24:56.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 -- start start 2023-10-10T20:24:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20c406d370 0x7f20c4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 0x7f20c4168c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41535e0 con 0x7f20c4152300 2023-10-10T20:24:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4153020 con 0x7f20c415b9a0 2023-10-10T20:24:56.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c934f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4153300 con 0x7f20c406d370 2023-10-10T20:24:56.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37000/0 (socket says 172.21.15.148:37000) 2023-10-10T20:24:56.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.877+0000 7f20c27fc700 1 -- 172.21.15.148:0/1634755114 learned_addr learned my addr 172.21.15.148:0/1634755114 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:56.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c2ffd700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20c406d370 0x7f20c4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c37fe700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 0x7f20c4168c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 -- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20c406d370 msgr2=0x7f20c4151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20c406d370 0x7f20c4151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 -- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 msgr2=0x7f20c4168c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:56.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 0x7f20c4168c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 -- 172.21.15.148:0/1634755114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4009420 con 0x7f20c4152300 2023-10-10T20:24:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c37fe700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 0x7f20c4168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c27fc700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20b800d860 tx=0x7f20b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80125f0 con 0x7f20c4152300 2023-10-10T20:24:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8012750 con 0x7f20c4152300 2023-10-10T20:24:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c4155c70 con 0x7f20c4152300 2023-10-10T20:24:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8012b10 con 0x7f20c4152300 2023-10-10T20:24:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.878+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c4151b80 con 0x7f20c4152300 2023-10-10T20:24:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.879+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c404c3c0 con 0x7f20c4152300 2023-10-10T20:24:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.879+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20b801e460 con 0x7f20c4152300 2023-10-10T20:24:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.880+0000 7f20aaffd700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 0x7f20b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.880+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20b8084a60 con 0x7f20c4152300 2023-10-10T20:24:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.880+0000 7f20c2ffd700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 0x7f20b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.880+0000 7f20c2ffd700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 0x7f20b00693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f20bc002250 tx=0x7f20bc001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:56.884+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20b8051cd0 con 0x7f20c4152300 2023-10-10T20:24:57.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:57.113+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f20c40cdf10 con 0x7f20c4152300 2023-10-10T20:24:58.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.093+0000 7f20aaffd700 1 -- 172.21.15.148:0/1634755114 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f20b80552f0 con 0x7f20c4152300 2023-10-10T20:24:58.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.095+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 msgr2=0x7f20b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:58.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.095+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 0x7f20b00693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f20bc002250 tx=0x7f20bc001a20 comp rx=0 tx=0).stop 2023-10-10T20:24:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 msgr2=0x7f20c41526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20b800d860 tx=0x7f20b8008040 comp rx=0 tx=0).stop 2023-10-10T20:24:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 shutdown_connections 2023-10-10T20:24:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f20b0066f30 0x7f20b00693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20c415b9a0 0x7f20c4168c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20c406d370 0x7f20c4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 --2- 172.21.15.148:0/1634755114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4152300 0x7f20c41526d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 >> 172.21.15.148:0/1634755114 conn(0x7f20c4069870 msgr2=0x7f20c40b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:58.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 shutdown_connections 2023-10-10T20:24:58.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.096+0000 7f20c934f700 1 -- 172.21.15.148:0/1634755114 wait complete. 2023-10-10T20:24:58.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:24:58.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.228+0000 7f47dc68a700 1 Processor -- start 2023-10-10T20:24:58.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.228+0000 7f47dc68a700 1 -- start start 2023-10-10T20:24:58.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dc68a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 0x7f47d40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dc68a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d406acb0 0x7f47d40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dc68a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40b0790 con 0x7f47d415a290 2023-10-10T20:24:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dc68a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40ce1f0 con 0x7f47d406acb0 2023-10-10T20:24:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dc68a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40b0b20 con 0x7f47d4159ec0 2023-10-10T20:24:58.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d406acb0 0x7f47d40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d406acb0 0x7f47d40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37006/0 (socket says 172.21.15.148:37006) 2023-10-10T20:24:58.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 -- 172.21.15.148:0/3549188853 learned_addr learned my addr 172.21.15.148:0/3549188853 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:58.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47da426700 1 --2- 172.21.15.148:0/3549188853 >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 0x7f47d40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 -- 172.21.15.148:0/3549188853 >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 msgr2=0x7f47d40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:58.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 --2- 172.21.15.148:0/3549188853 >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 0x7f47d40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 -- 172.21.15.148:0/3549188853 >> v1:172.21.15.161:6789/0 conn(0x7f47d415a290 legacy=0x7f47d406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:58.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 -- 172.21.15.148:0/3549188853 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40b04b0 con 0x7f47d406acb0 2023-10-10T20:24:58.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47da426700 1 --2- 172.21.15.148:0/3549188853 >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 0x7f47d40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:24:58.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.229+0000 7f47dac27700 1 --2- 172.21.15.148:0/3549188853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d406acb0 0x7f47d40afe80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f47d40898e0 tx=0x7f47cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09a78fd78929ed server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:58.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47d9424700 1 -- 172.21.15.148:0/3549188853 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00b820 con 0x7f47d406acb0 2023-10-10T20:24:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47d9424700 1 -- 172.21.15.148:0/3549188853 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00b980 con 0x7f47d406acb0 2023-10-10T20:24:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47d9424700 1 -- 172.21.15.148:0/3549188853 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00bd20 con 0x7f47d406acb0 2023-10-10T20:24:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 -- 172.21.15.148:0/3549188853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d406acb0 msgr2=0x7f47d40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3549188853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d406acb0 0x7f47d40afe80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f47d40898e0 tx=0x7f47cc009250 comp rx=0 tx=0).stop 2023-10-10T20:24:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 -- 172.21.15.148:0/3549188853 shutdown_connections 2023-10-10T20:24:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3549188853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d406acb0 0x7f47d40afe80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3549188853 >> v2:172.21.15.161:3300/0 conn(0x7f47d4159ec0 0x7f47d40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 -- 172.21.15.148:0/3549188853 >> 172.21.15.148:0/3549188853 conn(0x7f47d40d1880 msgr2=0x7f47d40cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 -- 172.21.15.148:0/3549188853 shutdown_connections 2023-10-10T20:24:58.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.230+0000 7f47dc68a700 1 -- 172.21.15.148:0/3549188853 wait complete. 2023-10-10T20:24:58.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.231+0000 7f47dc68a700 1 Processor -- start 2023-10-10T20:24:58.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.231+0000 7f47dc68a700 1 -- start start 2023-10-10T20:24:58.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d406acb0 0x7f47d40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d415a290 0x7f47d41555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b4ab0 con 0x7f47d4159ec0 2023-10-10T20:24:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b3ea0 con 0x7f47d406acb0 2023-10-10T20:24:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dc68a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b4180 con 0x7f47d415a290 2023-10-10T20:24:58.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37020/0 (socket says 172.21.15.148:37020) 2023-10-10T20:24:58.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 -- 172.21.15.148:0/3020298213 learned_addr learned my addr 172.21.15.148:0/3020298213 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:58.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47da426700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d406acb0 0x7f47d40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47dac27700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d415a290 0x7f47d41555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 -- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d415a290 msgr2=0x7f47d41555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:58.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d415a290 0x7f47d41555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 -- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d406acb0 msgr2=0x7f47d40b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:58.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.232+0000 7f47d9c25700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d406acb0 0x7f47d40b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:58.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47d9c25700 1 -- 172.21.15.148:0/3020298213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0094f0 con 0x7f47d4159ec0 2023-10-10T20:24:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47d9c25700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b5e0 tx=0x7f47c4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400f510 con 0x7f47d4159ec0 2023-10-10T20:24:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400f670 con 0x7f47d4159ec0 2023-10-10T20:24:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40b4460 con 0x7f47d4159ec0 2023-10-10T20:24:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400fa30 con 0x7f47d4159ec0 2023-10-10T20:24:58.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.233+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d40b5480 con 0x7f47d4159ec0 2023-10-10T20:24:58.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.234+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d40b4c20 con 0x7f47d4159ec0 2023-10-10T20:24:58.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.234+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47c401a400 con 0x7f47d4159ec0 2023-10-10T20:24:58.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.234+0000 7f47ca7fc700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 0x7f47b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:58.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.235+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47c4083fc0 con 0x7f47d4159ec0 2023-10-10T20:24:58.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.235+0000 7f47da426700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 0x7f47b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:58.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.235+0000 7f47da426700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 0x7f47b80693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d6e0 tx=0x7f47d0007850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:58.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.239+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c4051210 con 0x7f47d4159ec0 2023-10-10T20:24:58.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:58.462+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f47d404c3c0 con 0x7f47d4159ec0 2023-10-10T20:24:59.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.103+0000 7f47ca7fc700 1 -- 172.21.15.148:0/3020298213 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f47c4058e60 con 0x7f47d4159ec0 2023-10-10T20:24:59.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 msgr2=0x7f47b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 0x7f47b80693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d6e0 tx=0x7f47d0007850 comp rx=0 tx=0).stop 2023-10-10T20:24:59.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 msgr2=0x7f47d40c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b5e0 tx=0x7f47c4009120 comp rx=0 tx=0).stop 2023-10-10T20:24:59.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 shutdown_connections 2023-10-10T20:24:59.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47b8066f30 0x7f47b80693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d415a290 0x7f47d41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d406acb0 0x7f47d40b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 --2- 172.21.15.148:0/3020298213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4159ec0 0x7f47d40c9ce0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 >> 172.21.15.148:0/3020298213 conn(0x7f47d40d1880 msgr2=0x7f47d40c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 shutdown_connections 2023-10-10T20:24:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.106+0000 7f47dc68a700 1 -- 172.21.15.148:0/3020298213 wait complete. 2023-10-10T20:24:59.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:24:59.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.236+0000 7fbfe5d7d700 1 Processor -- start 2023-10-10T20:24:59.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.236+0000 7fbfe5d7d700 1 -- start start 2023-10-10T20:24:59.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfe5d7d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfe0157870 0x7fbfe015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfe5d7d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe0157c40 0x7fbfe0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfe5d7d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe015cbb0 con 0x7fbfe015a620 2023-10-10T20:24:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfe5d7d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe015bfa0 con 0x7fbfe0157c40 2023-10-10T20:24:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfe5d7d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe015c280 con 0x7fbfe0157870 2023-10-10T20:24:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfdffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe0157c40 0x7fbfe0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfdffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbfe0157c40 0x7fbfe0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37028/0 (socket says 172.21.15.148:37028) 2023-10-10T20:24:59.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.237+0000 7fbfdffff700 1 -- 172.21.15.148:0/187667039 learned_addr learned my addr 172.21.15.148:0/187667039 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:59.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdf7fe700 1 --2- 172.21.15.148:0/187667039 >> v2:172.21.15.161:3300/0 conn(0x7fbfe0157870 0x7fbfe015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdffff700 1 -- 172.21.15.148:0/187667039 >> v2:172.21.15.161:3300/0 conn(0x7fbfe0157870 msgr2=0x7fbfe015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdffff700 1 --2- 172.21.15.148:0/187667039 >> v2:172.21.15.161:3300/0 conn(0x7fbfe0157870 0x7fbfe015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdffff700 1 -- 172.21.15.148:0/187667039 >> v1:172.21.15.161:6789/0 conn(0x7fbfe015a620 legacy=0x7fbfe015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdffff700 1 -- 172.21.15.148:0/187667039 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe015c560 con 0x7fbfe0157c40 2023-10-10T20:24:59.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfdffff700 1 --2- 172.21.15.148:0/187667039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157c40 0x7fbfe0158110 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0069d80 tx=0x7fbfd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c48841fa8bf3cb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfde7fc700 1 -- 172.21.15.148:0/187667039 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800b820 con 0x7fbfe0157c40 2023-10-10T20:24:59.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfde7fc700 1 -- 172.21.15.148:0/187667039 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd800b980 con 0x7fbfe0157c40 2023-10-10T20:24:59.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfde7fc700 1 -- 172.21.15.148:0/187667039 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800bd20 con 0x7fbfe0157c40 2023-10-10T20:24:59.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/187667039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157c40 msgr2=0x7fbfe0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/187667039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157c40 0x7fbfe0158110 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0069d80 tx=0x7fbfd8009250 comp rx=0 tx=0).stop 2023-10-10T20:24:59.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/187667039 shutdown_connections 2023-10-10T20:24:59.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/187667039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157c40 0x7fbfe0158110 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/187667039 >> v2:172.21.15.161:3300/0 conn(0x7fbfe0157870 0x7fbfe015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/187667039 >> 172.21.15.148:0/187667039 conn(0x7fbfe0068ce0 msgr2=0x7fbfe00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.238+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/187667039 shutdown_connections 2023-10-10T20:24:59.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.239+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/187667039 wait complete. 2023-10-10T20:24:59.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 Processor -- start 2023-10-10T20:24:59.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 -- start start 2023-10-10T20:24:59.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe0157c40 0x7fbfe015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 0x7fbfe0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c5b10 con 0x7fbfe0157870 2023-10-10T20:24:59.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe015e550 con 0x7fbfe0157c40 2023-10-10T20:24:59.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.240+0000 7fbfe5d7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01559d0 con 0x7fbfe015a620 2023-10-10T20:24:59.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37042/0 (socket says 172.21.15.148:37042) 2023-10-10T20:24:59.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 -- 172.21.15.148:0/3535002210 learned_addr learned my addr 172.21.15.148:0/3535002210 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:59.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdeffd700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe0157c40 0x7fbfe015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdffff700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 0x7fbfe0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 -- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 msgr2=0x7fbfe0155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 0x7fbfe0155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 -- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe0157c40 msgr2=0x7fbfe015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe0157c40 0x7fbfe015e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 -- 172.21.15.148:0/3535002210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80094f0 con 0x7fbfe0157870 2023-10-10T20:24:59.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdffff700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 0x7fbfe0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:24:59.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfdf7fe700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e190 tx=0x7fbfd0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000ed80 con 0x7fbfe0157870 2023-10-10T20:24:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0014420 con 0x7fbfe0157870 2023-10-10T20:24:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe015e7b0 con 0x7fbfe0157870 2023-10-10T20:24:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00147e0 con 0x7fbfe0157870 2023-10-10T20:24:59.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.241+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe015ed70 con 0x7fbfe0157870 2023-10-10T20:24:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.242+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe004c3c0 con 0x7fbfe0157870 2023-10-10T20:24:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.243+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfd0014b90 con 0x7fbfe0157870 2023-10-10T20:24:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.243+0000 7fbfc77fe700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 0x7fbfcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.243+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbfd0084ce0 con 0x7fbfe0157870 2023-10-10T20:24:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.243+0000 7fbfdeffd700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 0x7fbfcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.243+0000 7fbfdeffd700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 0x7fbfcc0693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0156e70 tx=0x7fbfd4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.247+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfd0051f50 con 0x7fbfe0157870 2023-10-10T20:24:59.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.470+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfe015c840 con 0x7fbfe0157870 2023-10-10T20:24:59.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.471+0000 7fbfc77fe700 1 -- 172.21.15.148:0/3535002210 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fbfd0009a60 con 0x7fbfe0157870 2023-10-10T20:24:59.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:59.474 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:24:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 msgr2=0x7fbfcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 0x7fbfcc0693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0156e70 tx=0x7fbfd4009060 comp rx=0 tx=0).stop 2023-10-10T20:24:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 msgr2=0x7fbfe015ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e190 tx=0x7fbfd0009920 comp rx=0 tx=0).stop 2023-10-10T20:24:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 shutdown_connections 2023-10-10T20:24:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbfcc066f30 0x7fbfcc0693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbfe015a620 0x7fbfe0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfe0157870 0x7fbfe015ffa0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 --2- 172.21.15.148:0/3535002210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfe0157c40 0x7fbfe015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 >> 172.21.15.148:0/3535002210 conn(0x7fbfe0068ce0 msgr2=0x7fbfe00b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 shutdown_connections 2023-10-10T20:24:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.473+0000 7fbfe5d7d700 1 -- 172.21.15.148:0/3535002210 wait complete. 2023-10-10T20:24:59.478 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:24:59.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:24:59.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.601+0000 7f2731c3a700 1 Processor -- start 2023-10-10T20:24:59.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.601+0000 7f2731c3a700 1 -- start start 2023-10-10T20:24:59.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f2731c3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f272c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f2731c3a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c15cbb0 con 0x7f272c157870 2023-10-10T20:24:59.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f2731c3a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c15bfa0 con 0x7f272c15d580 2023-10-10T20:24:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f2731c3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c15bdb0 con 0x7f272c15a620 2023-10-10T20:24:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272b7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f272c157870 0x7f272c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40524/0 (socket says 172.21.15.148:40524) 2023-10-10T20:24:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272b7fe700 1 -- 172.21.15.148:0/1359353944 learned_addr learned my addr 172.21.15.148:0/1359353944 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:24:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272affd700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f272c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978161957 0 0) 0x7f272c15cbb0 con 0x7f272c157870 2023-10-10T20:24:59.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2718003570 con 0x7f272c157870 2023-10-10T20:24:59.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1632746969 0 0) 0x7f2718003570 con 0x7f272c157870 2023-10-10T20:24:59.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 msgr2=0x7f272c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f272c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 >> v1:172.21.15.161:6789/0 conn(0x7f272c15d580 legacy=0x7f272c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c15c280 con 0x7f272c157870 2023-10-10T20:24:59.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f27240017f0 con 0x7f272c157870 2023-10-10T20:24:59.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 >> v1:172.21.15.148:6789/0 conn(0x7f272c157870 legacy=0x7f272c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2718003a90 0x7f2718003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718004810 0x7f2718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f27180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271800bb10 con 0x7f2718003a90 2023-10-10T20:24:59.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271800bc80 con 0x7f2718004810 2023-10-10T20:24:59.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.602+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271800be40 con 0x7f272c15a620 2023-10-10T20:24:59.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2718003a90 0x7f2718003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272affd700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718004810 0x7f2718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272bfff700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f27180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 -- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 msgr2=0x7f27180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f272c15a620 0x7f27180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 -- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718004810 msgr2=0x7f2718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718004810 0x7f2718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 -- 172.21.15.148:0/1359353944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718003570 con 0x7f2718003a90 2023-10-10T20:24:59.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2718003a90 0x7f2718003e60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f272400a950 tx=0x7f27240021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb7d88234f12773 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240070d0 con 0x7f2718003a90 2023-10-10T20:24:59.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724007250 con 0x7f2718003a90 2023-10-10T20:24:59.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.603+0000 7f272a7fc700 1 -- 172.21.15.148:0/1359353944 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724007610 con 0x7f2718003a90 2023-10-10T20:24:59.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 -- 172.21.15.148:0/1359353944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2718003a90 msgr2=0x7f2718003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1359353944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2718003a90 0x7f2718003e60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f272400a950 tx=0x7f27240021f0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f272bfff700 1 -- 172.21.15.148:0/1359353944 reap_dead start 2023-10-10T20:24:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 -- 172.21.15.148:0/1359353944 shutdown_connections 2023-10-10T20:24:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 -- 172.21.15.148:0/1359353944 >> 172.21.15.148:0/1359353944 conn(0x7f272c068ce0 msgr2=0x7f272c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 -- 172.21.15.148:0/1359353944 shutdown_connections 2023-10-10T20:24:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.604+0000 7f2731c3a700 1 -- 172.21.15.148:0/1359353944 wait complete. 2023-10-10T20:24:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.605+0000 7f2731c3a700 1 Processor -- start 2023-10-10T20:24:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.605+0000 7f2731c3a700 1 -- start start 2023-10-10T20:24:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2718003a90 0x7f272c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f272c16b020 0x7f272c0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0d0d30 con 0x7f272c0d08c0 2023-10-10T20:24:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c154a10 con 0x7f272c16b020 2023-10-10T20:24:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c16ab90 con 0x7f2718003a90 2023-10-10T20:24:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37056/0 (socket says 172.21.15.148:37056) 2023-10-10T20:24:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 -- 172.21.15.148:0/1071872062 learned_addr learned my addr 172.21.15.148:0/1071872062 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2718003a90 0x7f272c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272affd700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f272c16b020 0x7f272c0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 -- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2718003a90 msgr2=0x7f272c1494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2718003a90 0x7f272c1494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 -- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f272c16b020 msgr2=0x7f272c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f272c16b020 0x7f272c0d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 -- 172.21.15.148:0/1071872062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c154730 con 0x7f272c0d08c0 2023-10-10T20:24:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f272bfff700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f272c15d9e0 tx=0x7f271c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c013420 con 0x7f272c0d08c0 2023-10-10T20:24:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c0135a0 con 0x7f272c0d08c0 2023-10-10T20:24:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c16a540 con 0x7f272c0d08c0 2023-10-10T20:24:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c013960 con 0x7f272c0d08c0 2023-10-10T20:24:59.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.606+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c149a10 con 0x7f272c0d08c0 2023-10-10T20:24:59.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.607+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c16a3a0 con 0x7f272c0d08c0 2023-10-10T20:24:59.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.608+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f271c013d10 con 0x7f272c0d08c0 2023-10-10T20:24:59.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.608+0000 7f27237fe700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 0x7f270c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.608+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f271c0843b0 con 0x7f272c0d08c0 2023-10-10T20:24:59.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.608+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 0x7f270c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.609+0000 7f272b7fe700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 0x7f270c0693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2724002250 tx=0x7f2724015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.612+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f272c16a3a0 con 0x7f272c0d08c0 2023-10-10T20:24:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.795+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f272c16a3a0 con 0x7f272c0d08c0 2023-10-10T20:24:59.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.796+0000 7f27237fe700 1 -- 172.21.15.148:0/1071872062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+15413 (secure 0 0 0) 0x7f272c16a3a0 con 0x7f272c0d08c0 2023-10-10T20:24:59.798 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:24:59.798 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":116,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:24:56.523284+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:24:59.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 msgr2=0x7f270c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 0x7f270c0693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2724002250 tx=0x7f2724015000 comp rx=0 tx=0).stop 2023-10-10T20:24:59.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 msgr2=0x7f272c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f272c15d9e0 tx=0x7f271c009040 comp rx=0 tx=0).stop 2023-10-10T20:24:59.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 shutdown_connections 2023-10-10T20:24:59.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f270c066f30 0x7f270c0693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f272c0d08c0 0x7f272c153ab0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2718003a90 0x7f272c1494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 --2- 172.21.15.148:0/1071872062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f272c16b020 0x7f272c0d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 >> 172.21.15.148:0/1071872062 conn(0x7f272c068ce0 msgr2=0x7f272c14c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 shutdown_connections 2023-10-10T20:24:59.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.798+0000 7f2731c3a700 1 -- 172.21.15.148:0/1071872062 wait complete. 2023-10-10T20:24:59.812 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:24:59.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:24:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.925+0000 7f9bba9bb700 1 Processor -- start 2023-10-10T20:24:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.925+0000 7f9bba9bb700 1 -- start start 2023-10-10T20:24:59.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bba9bb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9bb4152010 0x7f9bb415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bba9bb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9bb41523e0 0x7f9bb406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bba9bb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb406c010 con 0x7f9bb4158e50 2023-10-10T20:24:59.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bba9bb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb406bd10 con 0x7f9bb4152010 2023-10-10T20:24:59.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bba9bb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb415c060 con 0x7f9bb41523e0 2023-10-10T20:24:59.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bb3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9bb4152010 0x7f9bb415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bb3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9bb4152010 0x7f9bb415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37066/0 (socket says 172.21.15.148:37066) 2023-10-10T20:24:59.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bb3fff700 1 -- 172.21.15.148:0/3948149486 learned_addr learned my addr 172.21.15.148:0/3948149486 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:59.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.926+0000 7f9bb37fe700 1 --2- 172.21.15.148:0/3948149486 >> v2:172.21.15.161:3300/0 conn(0x7f9bb41523e0 0x7f9bb406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb3fff700 1 -- 172.21.15.148:0/3948149486 >> v2:172.21.15.161:3300/0 conn(0x7f9bb41523e0 msgr2=0x7f9bb406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:24:59.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb3fff700 1 --2- 172.21.15.148:0/3948149486 >> v2:172.21.15.161:3300/0 conn(0x7f9bb41523e0 0x7f9bb406c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb3fff700 1 -- 172.21.15.148:0/3948149486 >> v1:172.21.15.161:6789/0 conn(0x7f9bb4158e50 legacy=0x7f9bb40cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb3fff700 1 -- 172.21.15.148:0/3948149486 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb415c340 con 0x7f9bb4152010 2023-10-10T20:24:59.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb3fff700 1 --2- 172.21.15.148:0/3948149486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb415b810 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40ceae0 tx=0x7f9ba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7dd15ffe850167 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb2ffd700 1 -- 172.21.15.148:0/3948149486 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4009e70 con 0x7f9bb4152010 2023-10-10T20:24:59.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb2ffd700 1 -- 172.21.15.148:0/3948149486 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400b860 con 0x7f9bb4152010 2023-10-10T20:24:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bb2ffd700 1 -- 172.21.15.148:0/3948149486 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400bc00 con 0x7f9bb4152010 2023-10-10T20:24:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 -- 172.21.15.148:0/3948149486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 msgr2=0x7f9bb415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/3948149486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb415b810 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40ceae0 tx=0x7f9ba4009250 comp rx=0 tx=0).stop 2023-10-10T20:24:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 -- 172.21.15.148:0/3948149486 shutdown_connections 2023-10-10T20:24:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/3948149486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb415b810 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/3948149486 >> v2:172.21.15.161:3300/0 conn(0x7f9bb41523e0 0x7f9bb406c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 -- 172.21.15.148:0/3948149486 >> 172.21.15.148:0/3948149486 conn(0x7f9bb4069870 msgr2=0x7f9bb4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:24:59.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.927+0000 7f9bba9bb700 1 -- 172.21.15.148:0/3948149486 shutdown_connections 2023-10-10T20:24:59.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.928+0000 7f9bba9bb700 1 -- 172.21.15.148:0/3948149486 wait complete. 2023-10-10T20:24:59.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 Processor -- start 2023-10-10T20:24:59.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 -- start start 2023-10-10T20:24:59.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb41523e0 0x7f9bb40b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bb4158e50 0x7f9bb415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414ead0 con 0x7f9bb4152010 2023-10-10T20:24:59.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414dec0 con 0x7f9bb41523e0 2023-10-10T20:24:59.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.929+0000 7f9bba9bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414e1a0 con 0x7f9bb4158e50 2023-10-10T20:24:59.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37076/0 (socket says 172.21.15.148:37076) 2023-10-10T20:24:59.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 -- 172.21.15.148:0/1191826194 learned_addr learned my addr 172.21.15.148:0/1191826194 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:24:59.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb8f58700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bb4158e50 0x7f9bb415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb37fe700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb41523e0 0x7f9bb40b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 -- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bb4158e50 msgr2=0x7f9bb415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bb4158e50 0x7f9bb415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 -- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb41523e0 msgr2=0x7f9bb40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:24:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb41523e0 0x7f9bb40b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:24:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 -- 172.21.15.148:0/1191826194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40094f0 con 0x7f9bb4152010 2023-10-10T20:24:59.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bb3fff700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40c59e0 tx=0x7f9ba4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400bbe0 con 0x7f9bb4152010 2023-10-10T20:24:59.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba4011420 con 0x7f9bb4152010 2023-10-10T20:24:59.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb414e480 con 0x7f9bb4152010 2023-10-10T20:24:59.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40117e0 con 0x7f9bb4152010 2023-10-10T20:24:59.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.930+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb414dba0 con 0x7f9bb4152010 2023-10-10T20:24:59.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.931+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb414edb0 con 0x7f9bb4152010 2023-10-10T20:24:59.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.931+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ba4011c10 con 0x7f9bb4152010 2023-10-10T20:24:59.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.932+0000 7f9b9bfff700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:24:59.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.932+0000 7f9bb37fe700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:24:59.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.932+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ba4085cf0 con 0x7f9bb4152010 2023-10-10T20:24:59.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.932+0000 7f9bb37fe700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4157ac0 tx=0x7f9ba80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:24:59.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:24:59.936+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ba408a070 con 0x7f9bb4152010 2023-10-10T20:25:00.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.115+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bb414ec10 con 0x7f9ba0066f30 2023-10-10T20:25:00.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.123+0000 7f9b9bfff700 1 -- 172.21.15.148:0/1191826194 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285566 (secure 0 0 0) 0x7f9ba8011a20 con 0x7f9ba0066f30 2023-10-10T20:25:00.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:00.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 msgr2=0x7f9ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4157ac0 tx=0x7f9ba80091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 msgr2=0x7f9bb414d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40c59e0 tx=0x7f9ba4017000 comp rx=0 tx=0).stop 2023-10-10T20:25:00.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 shutdown_connections 2023-10-10T20:25:00.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bb4158e50 0x7f9bb415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4152010 0x7f9bb414d660 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 --2- 172.21.15.148:0/1191826194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bb41523e0 0x7f9bb40b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 >> 172.21.15.148:0/1191826194 conn(0x7f9bb4069870 msgr2=0x7f9bb40cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:00.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 shutdown_connections 2023-10-10T20:25:00.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.126+0000 7f9bba9bb700 1 -- 172.21.15.148:0/1191826194 wait complete. 2023-10-10T20:25:00.134 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:25:00.140 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-10T20:24:59.396279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556428,"statfs":{"total":1600000000,"available":1593783074,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.321452"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531381+0000","last_change":"2023-10-10T20:24:55.531325+0000","last_active":"2023-10-10T20:24:55.531381+0000","last_peered":"2023-10-10T20:24:55.531381+0000","last_clean":"2023-10-10T20:24:55.531381+0000","last_became_active":"2023-10-10T20:24:55.531116+0000","last_became_peered":"2023-10-10T20:24:55.531116+0000","last_unstale":"2023-10-10T20:24:55.531381+0000","last_undegraded":"2023-10-10T20:24:55.531381+0000","last_fullsized":"2023-10-10T20:24:55.531381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:51.099967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538520+0000","last_change":"2023-10-10T20:24:55.538404+0000","last_active":"2023-10-10T20:24:55.538520+0000","last_peered":"2023-10-10T20:24:55.538520+0000","last_clean":"2023-10-10T20:24:55.538520+0000","last_became_active":"2023-10-10T20:24:55.538047+0000","last_became_peered":"2023-10-10T20:24:55.538047+0000","last_unstale":"2023-10-10T20:24:55.538520+0000","last_undegraded":"2023-10-10T20:24:55.538520+0000","last_fullsized":"2023-10-10T20:24:55.538520+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:47:48.401220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534530+0000","last_change":"2023-10-10T20:24:55.534473+0000","last_active":"2023-10-10T20:24:55.534530+0000","last_peered":"2023-10-10T20:24:55.534530+0000","last_clean":"2023-10-10T20:24:55.534530+0000","last_became_active":"2023-10-10T20:24:55.534254+0000","last_became_peered":"2023-10-10T20:24:55.534254+0000","last_unstale":"2023-10-10T20:24:55.534530+0000","last_undegraded":"2023-10-10T20:24:55.534530+0000","last_fullsized":"2023-10-10T20:24:55.534530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:30.381523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534653+0000","last_change":"2023-10-10T20:24:55.534529+0000","last_active":"2023-10-10T20:24:55.534653+0000","last_peered":"2023-10-10T20:24:55.534653+0000","last_clean":"2023-10-10T20:24:55.534653+0000","last_became_active":"2023-10-10T20:24:55.534250+0000","last_became_peered":"2023-10-10T20:24:55.534250+0000","last_unstale":"2023-10-10T20:24:55.534653+0000","last_undegraded":"2023-10-10T20:24:55.534653+0000","last_fullsized":"2023-10-10T20:24:55.534653+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536242+0000","last_change":"2023-10-10T20:24:55.536074+0000","last_active":"2023-10-10T20:24:55.536242+0000","last_peered":"2023-10-10T20:24:55.536242+0000","last_clean":"2023-10-10T20:24:55.536242+0000","last_became_active":"2023-10-10T20:24:55.535511+0000","last_became_peered":"2023-10-10T20:24:55.535511+0000","last_unstale":"2023-10-10T20:24:55.536242+0000","last_undegraded":"2023-10-10T20:24:55.536242+0000","last_fullsized":"2023-10-10T20:24:55.536242+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:41.882397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534662+0000","last_change":"2023-10-10T20:24:55.534599+0000","last_active":"2023-10-10T20:24:55.534662+0000","last_peered":"2023-10-10T20:24:55.534662+0000","last_clean":"2023-10-10T20:24:55.534662+0000","last_became_active":"2023-10-10T20:24:55.534246+0000","last_became_peered":"2023-10-10T20:24:55.534246+0000","last_unstale":"2023-10-10T20:24:55.534662+0000","last_undegraded":"2023-10-10T20:24:55.534662+0000","last_fullsized":"2023-10-10T20:24:55.534662+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:04.842092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533930+0000","last_change":"2023-10-10T20:24:55.533817+0000","last_active":"2023-10-10T20:24:55.533930+0000","last_peered":"2023-10-10T20:24:55.533930+0000","last_clean":"2023-10-10T20:24:55.533930+0000","last_became_active":"2023-10-10T20:24:55.533449+0000","last_became_peered":"2023-10-10T20:24:55.533449+0000","last_unstale":"2023-10-10T20:24:55.533930+0000","last_undegraded":"2023-10-10T20:24:55.533930+0000","last_fullsized":"2023-10-10T20:24:55.533930+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536991+0000","last_change":"2023-10-10T20:24:55.536640+0000","last_active":"2023-10-10T20:24:55.536991+0000","last_peered":"2023-10-10T20:24:55.536991+0000","last_clean":"2023-10-10T20:24:55.536991+0000","last_became_active":"2023-10-10T20:24:55.535792+0000","last_became_peered":"2023-10-10T20:24:55.535792+0000","last_unstale":"2023-10-10T20:24:55.536991+0000","last_undegraded":"2023-10-10T20:24:55.536991+0000","last_fullsized":"2023-10-10T20:24:55.536991+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:27.340665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535501+0000","last_change":"2023-10-10T20:24:55.535358+0000","last_active":"2023-10-10T20:24:55.535501+0000","last_peered":"2023-10-10T20:24:55.535501+0000","last_clean":"2023-10-10T20:24:55.535501+0000","last_became_active":"2023-10-10T20:24:55.535138+0000","last_became_peered":"2023-10-10T20:24:55.535138+0000","last_unstale":"2023-10-10T20:24:55.535501+0000","last_undegraded":"2023-10-10T20:24:55.535501+0000","last_fullsized":"2023-10-10T20:24:55.535501+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:33.159224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537152+0000","last_change":"2023-10-10T20:24:55.537070+0000","last_active":"2023-10-10T20:24:55.537152+0000","last_peered":"2023-10-10T20:24:55.537152+0000","last_clean":"2023-10-10T20:24:55.537152+0000","last_became_active":"2023-10-10T20:24:55.536651+0000","last_became_peered":"2023-10-10T20:24:55.536651+0000","last_unstale":"2023-10-10T20:24:55.537152+0000","last_undegraded":"2023-10-10T20:24:55.537152+0000","last_fullsized":"2023-10-10T20:24:55.537152+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:44.126722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536749+0000","last_change":"2023-10-10T20:24:55.536533+0000","last_active":"2023-10-10T20:24:55.536749+0000","last_peered":"2023-10-10T20:24:55.536749+0000","last_clean":"2023-10-10T20:24:55.536749+0000","last_became_active":"2023-10-10T20:24:55.536039+0000","last_became_peered":"2023-10-10T20:24:55.536039+0000","last_unstale":"2023-10-10T20:24:55.536749+0000","last_undegraded":"2023-10-10T20:24:55.536749+0000","last_fullsized":"2023-10-10T20:24:55.536749+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533068+0000","last_change":"2023-10-10T20:24:55.532978+0000","last_active":"2023-10-10T20:24:55.533068+0000","last_peered":"2023-10-10T20:24:55.533068+0000","last_clean":"2023-10-10T20:24:55.533068+0000","last_became_active":"2023-10-10T20:24:55.532333+0000","last_became_peered":"2023-10-10T20:24:55.532333+0000","last_unstale":"2023-10-10T20:24:55.533068+0000","last_undegraded":"2023-10-10T20:24:55.533068+0000","last_fullsized":"2023-10-10T20:24:55.533068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:50.178429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531259+0000","last_change":"2023-10-10T20:24:55.530742+0000","last_active":"2023-10-10T20:24:55.531259+0000","last_peered":"2023-10-10T20:24:55.531259+0000","last_clean":"2023-10-10T20:24:55.531259+0000","last_became_active":"2023-10-10T20:24:55.530089+0000","last_became_peered":"2023-10-10T20:24:55.530089+0000","last_unstale":"2023-10-10T20:24:55.531259+0000","last_undegraded":"2023-10-10T20:24:55.531259+0000","last_fullsized":"2023-10-10T20:24:55.531259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:51.022829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533045+0000","last_change":"2023-10-10T20:24:55.532491+0000","last_active":"2023-10-10T20:24:55.533045+0000","last_peered":"2023-10-10T20:24:55.533045+0000","last_clean":"2023-10-10T20:24:55.533045+0000","last_became_active":"2023-10-10T20:24:55.532032+0000","last_became_peered":"2023-10-10T20:24:55.532032+0000","last_unstale":"2023-10-10T20:24:55.533045+0000","last_undegraded":"2023-10-10T20:24:55.533045+0000","last_fullsized":"2023-10-10T20:24:55.533045+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:56.398114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532095+0000","last_change":"2023-10-10T20:24:55.531977+0000","last_active":"2023-10-10T20:24:55.532095+0000","last_peered":"2023-10-10T20:24:55.532095+0000","last_clean":"2023-10-10T20:24:55.532095+0000","last_became_active":"2023-10-10T20:24:55.530049+0000","last_became_peered":"2023-10-10T20:24:55.530049+0000","last_unstale":"2023-10-10T20:24:55.532095+0000","last_undegraded":"2023-10-10T20:24:55.532095+0000","last_fullsized":"2023-10-10T20:24:55.532095+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538143+0000","last_change":"2023-10-10T20:24:55.538067+0000","last_active":"2023-10-10T20:24:55.538143+0000","last_peered":"2023-10-10T20:24:55.538143+0000","last_clean":"2023-10-10T20:24:55.538143+0000","last_became_active":"2023-10-10T20:24:55.537819+0000","last_became_peered":"2023-10-10T20:24:55.537819+0000","last_unstale":"2023-10-10T20:24:55.538143+0000","last_undegraded":"2023-10-10T20:24:55.538143+0000","last_fullsized":"2023-10-10T20:24:55.538143+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:33.159224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536497+0000","last_change":"2023-10-10T20:24:55.536073+0000","last_active":"2023-10-10T20:24:55.536497+0000","last_peered":"2023-10-10T20:24:55.536497+0000","last_clean":"2023-10-10T20:24:55.536497+0000","last_became_active":"2023-10-10T20:24:55.535548+0000","last_became_peered":"2023-10-10T20:24:55.535548+0000","last_unstale":"2023-10-10T20:24:55.536497+0000","last_undegraded":"2023-10-10T20:24:55.536497+0000","last_fullsized":"2023-10-10T20:24:55.536497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:50.938300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531278+0000","last_change":"2023-10-10T20:24:55.530882+0000","last_active":"2023-10-10T20:24:55.531278+0000","last_peered":"2023-10-10T20:24:55.531278+0000","last_clean":"2023-10-10T20:24:55.531278+0000","last_became_active":"2023-10-10T20:24:55.530515+0000","last_became_peered":"2023-10-10T20:24:55.530515+0000","last_unstale":"2023-10-10T20:24:55.531278+0000","last_undegraded":"2023-10-10T20:24:55.531278+0000","last_fullsized":"2023-10-10T20:24:55.531278+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533027+0000","last_change":"2023-10-10T20:24:55.532362+0000","last_active":"2023-10-10T20:24:55.533027+0000","last_peered":"2023-10-10T20:24:55.533027+0000","last_clean":"2023-10-10T20:24:55.533027+0000","last_became_active":"2023-10-10T20:24:55.531738+0000","last_became_peered":"2023-10-10T20:24:55.531738+0000","last_unstale":"2023-10-10T20:24:55.533027+0000","last_undegraded":"2023-10-10T20:24:55.533027+0000","last_fullsized":"2023-10-10T20:24:55.533027+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:07.802603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535483+0000","last_change":"2023-10-10T20:24:55.534769+0000","last_active":"2023-10-10T20:24:55.535483+0000","last_peered":"2023-10-10T20:24:55.535483+0000","last_clean":"2023-10-10T20:24:55.535483+0000","last_became_active":"2023-10-10T20:24:55.534545+0000","last_became_peered":"2023-10-10T20:24:55.534545+0000","last_unstale":"2023-10-10T20:24:55.535483+0000","last_undegraded":"2023-10-10T20:24:55.535483+0000","last_fullsized":"2023-10-10T20:24:55.535483+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:04.842092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528939+0000","last_change":"2023-10-10T20:24:55.528893+0000","last_active":"2023-10-10T20:24:55.528939+0000","last_peered":"2023-10-10T20:24:55.528939+0000","last_clean":"2023-10-10T20:24:55.528939+0000","last_became_active":"2023-10-10T20:24:55.528720+0000","last_became_peered":"2023-10-10T20:24:55.528720+0000","last_unstale":"2023-10-10T20:24:55.528939+0000","last_undegraded":"2023-10-10T20:24:55.528939+0000","last_fullsized":"2023-10-10T20:24:55.528939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:51:04.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538313+0000","last_change":"2023-10-10T20:24:55.538222+0000","last_active":"2023-10-10T20:24:55.538313+0000","last_peered":"2023-10-10T20:24:55.538313+0000","last_clean":"2023-10-10T20:24:55.538313+0000","last_became_active":"2023-10-10T20:24:55.537820+0000","last_became_peered":"2023-10-10T20:24:55.537820+0000","last_unstale":"2023-10-10T20:24:55.538313+0000","last_undegraded":"2023-10-10T20:24:55.538313+0000","last_fullsized":"2023-10-10T20:24:55.538313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:59.952047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537022+0000","last_change":"2023-10-10T20:24:55.536894+0000","last_active":"2023-10-10T20:24:55.537022+0000","last_peered":"2023-10-10T20:24:55.537022+0000","last_clean":"2023-10-10T20:24:55.537022+0000","last_became_active":"2023-10-10T20:24:55.536428+0000","last_became_peered":"2023-10-10T20:24:55.536428+0000","last_unstale":"2023-10-10T20:24:55.537022+0000","last_undegraded":"2023-10-10T20:24:55.537022+0000","last_fullsized":"2023-10-10T20:24:55.537022+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:25.002322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532197+0000","last_change":"2023-10-10T20:24:55.532093+0000","last_active":"2023-10-10T20:24:55.532197+0000","last_peered":"2023-10-10T20:24:55.532197+0000","last_clean":"2023-10-10T20:24:55.532197+0000","last_became_active":"2023-10-10T20:24:55.531866+0000","last_became_peered":"2023-10-10T20:24:55.531866+0000","last_unstale":"2023-10-10T20:24:55.532197+0000","last_undegraded":"2023-10-10T20:24:55.532197+0000","last_fullsized":"2023-10-10T20:24:55.532197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536252+0000","last_change":"2023-10-10T20:24:55.536191+0000","last_active":"2023-10-10T20:24:55.536252+0000","last_peered":"2023-10-10T20:24:55.536252+0000","last_clean":"2023-10-10T20:24:55.536252+0000","last_became_active":"2023-10-10T20:24:55.535744+0000","last_became_peered":"2023-10-10T20:24:55.535744+0000","last_unstale":"2023-10-10T20:24:55.536252+0000","last_undegraded":"2023-10-10T20:24:55.536252+0000","last_fullsized":"2023-10-10T20:24:55.536252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:38:34.511054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536472+0000","last_change":"2023-10-10T20:24:55.535193+0000","last_active":"2023-10-10T20:24:55.536472+0000","last_peered":"2023-10-10T20:24:55.536472+0000","last_clean":"2023-10-10T20:24:55.536472+0000","last_became_active":"2023-10-10T20:24:55.532997+0000","last_became_peered":"2023-10-10T20:24:55.532997+0000","last_unstale":"2023-10-10T20:24:55.536472+0000","last_undegraded":"2023-10-10T20:24:55.536472+0000","last_fullsized":"2023-10-10T20:24:55.536472+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:37:02.780454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536110+0000","last_change":"2023-10-10T20:24:55.536041+0000","last_active":"2023-10-10T20:24:55.536110+0000","last_peered":"2023-10-10T20:24:55.536110+0000","last_clean":"2023-10-10T20:24:55.536110+0000","last_became_active":"2023-10-10T20:24:55.535814+0000","last_became_peered":"2023-10-10T20:24:55.535814+0000","last_unstale":"2023-10-10T20:24:55.536110+0000","last_undegraded":"2023-10-10T20:24:55.536110+0000","last_fullsized":"2023-10-10T20:24:55.536110+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:05.676035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537133+0000","last_change":"2023-10-10T20:24:55.537064+0000","last_active":"2023-10-10T20:24:55.537133+0000","last_peered":"2023-10-10T20:24:55.537133+0000","last_clean":"2023-10-10T20:24:55.537133+0000","last_became_active":"2023-10-10T20:24:55.536696+0000","last_became_peered":"2023-10-10T20:24:55.536696+0000","last_unstale":"2023-10-10T20:24:55.537133+0000","last_undegraded":"2023-10-10T20:24:55.537133+0000","last_fullsized":"2023-10-10T20:24:55.537133+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:37:02.780454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531144+0000","last_change":"2023-10-10T20:24:55.531082+0000","last_active":"2023-10-10T20:24:55.531144+0000","last_peered":"2023-10-10T20:24:55.531144+0000","last_clean":"2023-10-10T20:24:55.531144+0000","last_became_active":"2023-10-10T20:24:55.530868+0000","last_became_peered":"2023-10-10T20:24:55.530868+0000","last_unstale":"2023-10-10T20:24:55.531144+0000","last_undegraded":"2023-10-10T20:24:55.531144+0000","last_fullsized":"2023-10-10T20:24:55.531144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:41.882397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533060+0000","last_change":"2023-10-10T20:24:55.532834+0000","last_active":"2023-10-10T20:24:55.533060+0000","last_peered":"2023-10-10T20:24:55.533060+0000","last_clean":"2023-10-10T20:24:55.533060+0000","last_became_active":"2023-10-10T20:24:55.532541+0000","last_became_peered":"2023-10-10T20:24:55.532541+0000","last_unstale":"2023-10-10T20:24:55.533060+0000","last_undegraded":"2023-10-10T20:24:55.533060+0000","last_fullsized":"2023-10-10T20:24:55.533060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:10:16.210335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534636+0000","last_change":"2023-10-10T20:24:55.533981+0000","last_active":"2023-10-10T20:24:55.534636+0000","last_peered":"2023-10-10T20:24:55.534636+0000","last_clean":"2023-10-10T20:24:55.534636+0000","last_became_active":"2023-10-10T20:24:55.533632+0000","last_became_peered":"2023-10-10T20:24:55.533632+0000","last_unstale":"2023-10-10T20:24:55.534636+0000","last_undegraded":"2023-10-10T20:24:55.534636+0000","last_fullsized":"2023-10-10T20:24:55.534636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:16:40.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538816+0000","last_change":"2023-10-10T20:24:55.538754+0000","last_active":"2023-10-10T20:24:55.538816+0000","last_peered":"2023-10-10T20:24:55.538816+0000","last_clean":"2023-10-10T20:24:55.538816+0000","last_became_active":"2023-10-10T20:24:55.538571+0000","last_became_peered":"2023-10-10T20:24:55.538571+0000","last_unstale":"2023-10-10T20:24:55.538816+0000","last_undegraded":"2023-10-10T20:24:55.538816+0000","last_fullsized":"2023-10-10T20:24:55.538816+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:09.738407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535958+0000","last_change":"2023-10-10T20:24:55.535813+0000","last_active":"2023-10-10T20:24:55.535958+0000","last_peered":"2023-10-10T20:24:55.535958+0000","last_clean":"2023-10-10T20:24:55.535958+0000","last_became_active":"2023-10-10T20:24:55.535575+0000","last_became_peered":"2023-10-10T20:24:55.535575+0000","last_unstale":"2023-10-10T20:24:55.535958+0000","last_undegraded":"2023-10-10T20:24:55.535958+0000","last_fullsized":"2023-10-10T20:24:55.535958+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:59:05.141540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"115'4","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087748+0000","last_change":"2023-10-10T20:24:55.087002+0000","last_active":"2023-10-10T20:24:56.087748+0000","last_peered":"2023-10-10T20:24:56.087748+0000","last_clean":"2023-10-10T20:24:56.087748+0000","last_became_active":"2023-10-10T20:24:55.086447+0000","last_became_peered":"2023-10-10T20:24:55.086447+0000","last_unstale":"2023-10-10T20:24:56.087748+0000","last_undegraded":"2023-10-10T20:24:56.087748+0000","last_fullsized":"2023-10-10T20:24:56.087748+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:32.153586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533633+0000","last_change":"2023-10-10T20:24:55.533575+0000","last_active":"2023-10-10T20:24:55.533633+0000","last_peered":"2023-10-10T20:24:55.533633+0000","last_clean":"2023-10-10T20:24:55.533633+0000","last_became_active":"2023-10-10T20:24:55.533291+0000","last_became_peered":"2023-10-10T20:24:55.533291+0000","last_unstale":"2023-10-10T20:24:55.533633+0000","last_undegraded":"2023-10-10T20:24:55.533633+0000","last_fullsized":"2023-10-10T20:24:55.533633+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088623+0000","last_change":"2023-10-10T20:24:55.091517+0000","last_active":"2023-10-10T20:24:56.088623+0000","last_peered":"2023-10-10T20:24:56.088623+0000","last_clean":"2023-10-10T20:24:56.088623+0000","last_became_active":"2023-10-10T20:24:55.090946+0000","last_became_peered":"2023-10-10T20:24:55.090946+0000","last_unstale":"2023-10-10T20:24:56.088623+0000","last_undegraded":"2023-10-10T20:24:56.088623+0000","last_fullsized":"2023-10-10T20:24:56.088623+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:44:51.532047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534061+0000","last_change":"2023-10-10T20:24:55.533995+0000","last_active":"2023-10-10T20:24:55.534061+0000","last_peered":"2023-10-10T20:24:55.534061+0000","last_clean":"2023-10-10T20:24:55.534061+0000","last_became_active":"2023-10-10T20:24:55.533684+0000","last_became_peered":"2023-10-10T20:24:55.533684+0000","last_unstale":"2023-10-10T20:24:55.534061+0000","last_undegraded":"2023-10-10T20:24:55.534061+0000","last_fullsized":"2023-10-10T20:24:55.534061+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:15.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.086931+0000","last_change":"2023-10-10T20:24:55.091372+0000","last_active":"2023-10-10T20:24:56.086931+0000","last_peered":"2023-10-10T20:24:56.086931+0000","last_clean":"2023-10-10T20:24:56.086931+0000","last_became_active":"2023-10-10T20:24:55.091152+0000","last_became_peered":"2023-10-10T20:24:55.091152+0000","last_unstale":"2023-10-10T20:24:56.086931+0000","last_undegraded":"2023-10-10T20:24:56.086931+0000","last_fullsized":"2023-10-10T20:24:56.086931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:04.614889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530857+0000","last_change":"2023-10-10T20:24:55.529353+0000","last_active":"2023-10-10T20:24:55.530857+0000","last_peered":"2023-10-10T20:24:55.530857+0000","last_clean":"2023-10-10T20:24:55.530857+0000","last_became_active":"2023-10-10T20:24:55.529140+0000","last_became_peered":"2023-10-10T20:24:55.529140+0000","last_unstale":"2023-10-10T20:24:55.530857+0000","last_undegraded":"2023-10-10T20:24:55.530857+0000","last_fullsized":"2023-10-10T20:24:55.530857+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:12.621312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087611+0000","last_change":"2023-10-10T20:24:55.086599+0000","last_active":"2023-10-10T20:24:56.087611+0000","last_peered":"2023-10-10T20:24:56.087611+0000","last_clean":"2023-10-10T20:24:56.087611+0000","last_became_active":"2023-10-10T20:24:55.086389+0000","last_became_peered":"2023-10-10T20:24:55.086389+0000","last_unstale":"2023-10-10T20:24:56.087611+0000","last_undegraded":"2023-10-10T20:24:56.087611+0000","last_fullsized":"2023-10-10T20:24:56.087611+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:03.836454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528929+0000","last_change":"2023-10-10T20:24:55.528854+0000","last_active":"2023-10-10T20:24:55.528929+0000","last_peered":"2023-10-10T20:24:55.528929+0000","last_clean":"2023-10-10T20:24:55.528929+0000","last_became_active":"2023-10-10T20:24:55.527778+0000","last_became_peered":"2023-10-10T20:24:55.527778+0000","last_unstale":"2023-10-10T20:24:55.528929+0000","last_undegraded":"2023-10-10T20:24:55.528929+0000","last_fullsized":"2023-10-10T20:24:55.528929+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:20:11.533307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088405+0000","last_change":"2023-10-10T20:24:55.090694+0000","last_active":"2023-10-10T20:24:56.088405+0000","last_peered":"2023-10-10T20:24:56.088405+0000","last_clean":"2023-10-10T20:24:56.088405+0000","last_became_active":"2023-10-10T20:24:55.090441+0000","last_became_peered":"2023-10-10T20:24:55.090441+0000","last_unstale":"2023-10-10T20:24:56.088405+0000","last_undegraded":"2023-10-10T20:24:56.088405+0000","last_fullsized":"2023-10-10T20:24:56.088405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:06.796964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533911+0000","last_change":"2023-10-10T20:24:55.533676+0000","last_active":"2023-10-10T20:24:55.533911+0000","last_peered":"2023-10-10T20:24:55.533911+0000","last_clean":"2023-10-10T20:24:55.533911+0000","last_became_active":"2023-10-10T20:24:55.533139+0000","last_became_peered":"2023-10-10T20:24:55.533139+0000","last_unstale":"2023-10-10T20:24:55.533911+0000","last_undegraded":"2023-10-10T20:24:55.533911+0000","last_fullsized":"2023-10-10T20:24:55.533911+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088788+0000","last_change":"2023-10-10T20:24:55.086225+0000","last_active":"2023-10-10T20:24:56.088788+0000","last_peered":"2023-10-10T20:24:56.088788+0000","last_clean":"2023-10-10T20:24:56.088788+0000","last_became_active":"2023-10-10T20:24:55.085448+0000","last_became_peered":"2023-10-10T20:24:55.085448+0000","last_unstale":"2023-10-10T20:24:56.088788+0000","last_undegraded":"2023-10-10T20:24:56.088788+0000","last_fullsized":"2023-10-10T20:24:56.088788+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:29.463293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532060+0000","last_change":"2023-10-10T20:24:55.530319+0000","last_active":"2023-10-10T20:24:55.532060+0000","last_peered":"2023-10-10T20:24:55.532060+0000","last_clean":"2023-10-10T20:24:55.532060+0000","last_became_active":"2023-10-10T20:24:55.529699+0000","last_became_peered":"2023-10-10T20:24:55.529699+0000","last_unstale":"2023-10-10T20:24:55.532060+0000","last_undegraded":"2023-10-10T20:24:55.532060+0000","last_fullsized":"2023-10-10T20:24:55.532060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529460+0000","last_change":"2023-10-10T20:24:55.093765+0000","last_active":"2023-10-10T20:24:55.529460+0000","last_peered":"2023-10-10T20:24:55.529460+0000","last_clean":"2023-10-10T20:24:55.529460+0000","last_became_active":"2023-10-10T20:24:55.093553+0000","last_became_peered":"2023-10-10T20:24:55.093553+0000","last_unstale":"2023-10-10T20:24:55.529460+0000","last_undegraded":"2023-10-10T20:24:55.529460+0000","last_fullsized":"2023-10-10T20:24:55.529460+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:29.463293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530874+0000","last_change":"2023-10-10T20:24:55.529812+0000","last_active":"2023-10-10T20:24:55.530874+0000","last_peered":"2023-10-10T20:24:55.530874+0000","last_clean":"2023-10-10T20:24:55.530874+0000","last_became_active":"2023-10-10T20:24:55.529596+0000","last_became_peered":"2023-10-10T20:24:55.529596+0000","last_unstale":"2023-10-10T20:24:55.530874+0000","last_undegraded":"2023-10-10T20:24:55.530874+0000","last_fullsized":"2023-10-10T20:24:55.530874+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:34.457048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528651+0000","last_change":"2023-10-10T20:24:55.090776+0000","last_active":"2023-10-10T20:24:55.528651+0000","last_peered":"2023-10-10T20:24:55.528651+0000","last_clean":"2023-10-10T20:24:55.528651+0000","last_became_active":"2023-10-10T20:24:55.090568+0000","last_became_peered":"2023-10-10T20:24:55.090568+0000","last_unstale":"2023-10-10T20:24:55.528651+0000","last_undegraded":"2023-10-10T20:24:55.528651+0000","last_fullsized":"2023-10-10T20:24:55.528651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:34.452779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536513+0000","last_change":"2023-10-10T20:24:55.536227+0000","last_active":"2023-10-10T20:24:55.536513+0000","last_peered":"2023-10-10T20:24:55.536513+0000","last_clean":"2023-10-10T20:24:55.536513+0000","last_became_active":"2023-10-10T20:24:55.535841+0000","last_became_peered":"2023-10-10T20:24:55.535841+0000","last_unstale":"2023-10-10T20:24:55.536513+0000","last_undegraded":"2023-10-10T20:24:55.536513+0000","last_fullsized":"2023-10-10T20:24:55.536513+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:40.323767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528314+0000","last_change":"2023-10-10T20:24:55.091121+0000","last_active":"2023-10-10T20:24:55.528314+0000","last_peered":"2023-10-10T20:24:55.528314+0000","last_clean":"2023-10-10T20:24:55.528314+0000","last_became_active":"2023-10-10T20:24:55.090888+0000","last_became_peered":"2023-10-10T20:24:55.090888+0000","last_unstale":"2023-10-10T20:24:55.528314+0000","last_undegraded":"2023-10-10T20:24:55.528314+0000","last_fullsized":"2023-10-10T20:24:55.528314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:16.016806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528985+0000","last_change":"2023-10-10T20:24:55.528798+0000","last_active":"2023-10-10T20:24:55.528985+0000","last_peered":"2023-10-10T20:24:55.528985+0000","last_clean":"2023-10-10T20:24:55.528985+0000","last_became_active":"2023-10-10T20:24:55.528574+0000","last_became_peered":"2023-10-10T20:24:55.528574+0000","last_unstale":"2023-10-10T20:24:55.528985+0000","last_undegraded":"2023-10-10T20:24:55.528985+0000","last_fullsized":"2023-10-10T20:24:55.528985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:05.676035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526542+0000","last_change":"2023-10-10T20:24:55.088111+0000","last_active":"2023-10-10T20:24:55.526542+0000","last_peered":"2023-10-10T20:24:55.526542+0000","last_clean":"2023-10-10T20:24:55.526542+0000","last_became_active":"2023-10-10T20:24:55.087889+0000","last_became_peered":"2023-10-10T20:24:55.087889+0000","last_unstale":"2023-10-10T20:24:55.526542+0000","last_undegraded":"2023-10-10T20:24:55.526542+0000","last_fullsized":"2023-10-10T20:24:55.526542+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:19.396498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535153+0000","last_change":"2023-10-10T20:24:55.535094+0000","last_active":"2023-10-10T20:24:55.535153+0000","last_peered":"2023-10-10T20:24:55.535153+0000","last_clean":"2023-10-10T20:24:55.535153+0000","last_became_active":"2023-10-10T20:24:55.534723+0000","last_became_peered":"2023-10-10T20:24:55.534723+0000","last_unstale":"2023-10-10T20:24:55.535153+0000","last_undegraded":"2023-10-10T20:24:55.535153+0000","last_fullsized":"2023-10-10T20:24:55.535153+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"115'3","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088144+0000","last_change":"2023-10-10T20:24:55.086404+0000","last_active":"2023-10-10T20:24:56.088144+0000","last_peered":"2023-10-10T20:24:56.088144+0000","last_clean":"2023-10-10T20:24:56.088144+0000","last_became_active":"2023-10-10T20:24:55.086174+0000","last_became_peered":"2023-10-10T20:24:55.086174+0000","last_unstale":"2023-10-10T20:24:56.088144+0000","last_undegraded":"2023-10-10T20:24:56.088144+0000","last_fullsized":"2023-10-10T20:24:56.088144+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:04.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531389+0000","last_change":"2023-10-10T20:24:55.531324+0000","last_active":"2023-10-10T20:24:55.531389+0000","last_peered":"2023-10-10T20:24:55.531389+0000","last_clean":"2023-10-10T20:24:55.531389+0000","last_became_active":"2023-10-10T20:24:55.530970+0000","last_became_peered":"2023-10-10T20:24:55.530970+0000","last_unstale":"2023-10-10T20:24:55.531389+0000","last_undegraded":"2023-10-10T20:24:55.531389+0000","last_fullsized":"2023-10-10T20:24:55.531389+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"115'6","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.090240+0000","last_change":"2023-10-10T20:24:55.090819+0000","last_active":"2023-10-10T20:24:56.090240+0000","last_peered":"2023-10-10T20:24:56.090240+0000","last_clean":"2023-10-10T20:24:56.090240+0000","last_became_active":"2023-10-10T20:24:55.090580+0000","last_became_peered":"2023-10-10T20:24:55.090580+0000","last_unstale":"2023-10-10T20:24:56.090240+0000","last_undegraded":"2023-10-10T20:24:56.090240+0000","last_fullsized":"2023-10-10T20:24:56.090240+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:42.315942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534610+0000","last_change":"2023-10-10T20:24:55.533847+0000","last_active":"2023-10-10T20:24:55.534610+0000","last_peered":"2023-10-10T20:24:55.534610+0000","last_clean":"2023-10-10T20:24:55.534610+0000","last_became_active":"2023-10-10T20:24:55.533342+0000","last_became_peered":"2023-10-10T20:24:55.533342+0000","last_unstale":"2023-10-10T20:24:55.534610+0000","last_undegraded":"2023-10-10T20:24:55.534610+0000","last_fullsized":"2023-10-10T20:24:55.534610+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:50.178429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088507+0000","last_change":"2023-10-10T20:24:55.092557+0000","last_active":"2023-10-10T20:24:56.088507+0000","last_peered":"2023-10-10T20:24:56.088507+0000","last_clean":"2023-10-10T20:24:56.088507+0000","last_became_active":"2023-10-10T20:24:55.091759+0000","last_became_peered":"2023-10-10T20:24:55.091759+0000","last_unstale":"2023-10-10T20:24:56.088507+0000","last_undegraded":"2023-10-10T20:24:56.088507+0000","last_fullsized":"2023-10-10T20:24:56.088507+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:10.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535989+0000","last_change":"2023-10-10T20:24:55.535805+0000","last_active":"2023-10-10T20:24:55.535989+0000","last_peered":"2023-10-10T20:24:55.535989+0000","last_clean":"2023-10-10T20:24:55.535989+0000","last_became_active":"2023-10-10T20:24:55.534918+0000","last_became_peered":"2023-10-10T20:24:55.534918+0000","last_unstale":"2023-10-10T20:24:55.535989+0000","last_undegraded":"2023-10-10T20:24:55.535989+0000","last_fullsized":"2023-10-10T20:24:55.535989+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:23.855778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.524876+0000","last_change":"2023-10-10T20:24:55.082820+0000","last_active":"2023-10-10T20:24:55.524876+0000","last_peered":"2023-10-10T20:24:55.524876+0000","last_clean":"2023-10-10T20:24:55.524876+0000","last_became_active":"2023-10-10T20:24:55.082600+0000","last_became_peered":"2023-10-10T20:24:55.082600+0000","last_unstale":"2023-10-10T20:24:55.524876+0000","last_undegraded":"2023-10-10T20:24:55.524876+0000","last_fullsized":"2023-10-10T20:24:55.524876+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:28:24.174847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530892+0000","last_change":"2023-10-10T20:24:55.530707+0000","last_active":"2023-10-10T20:24:55.530892+0000","last_peered":"2023-10-10T20:24:55.530892+0000","last_clean":"2023-10-10T20:24:55.530892+0000","last_became_active":"2023-10-10T20:24:55.530344+0000","last_became_peered":"2023-10-10T20:24:55.530344+0000","last_unstale":"2023-10-10T20:24:55.530892+0000","last_undegraded":"2023-10-10T20:24:55.530892+0000","last_fullsized":"2023-10-10T20:24:55.530892+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:06.592628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526581+0000","last_change":"2023-10-10T20:24:55.088915+0000","last_active":"2023-10-10T20:24:55.526581+0000","last_peered":"2023-10-10T20:24:55.526581+0000","last_clean":"2023-10-10T20:24:55.526581+0000","last_became_active":"2023-10-10T20:24:55.088688+0000","last_became_peered":"2023-10-10T20:24:55.088688+0000","last_unstale":"2023-10-10T20:24:55.526581+0000","last_undegraded":"2023-10-10T20:24:55.526581+0000","last_fullsized":"2023-10-10T20:24:55.526581+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:16.290974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528505+0000","last_change":"2023-10-10T20:24:55.528080+0000","last_active":"2023-10-10T20:24:55.528505+0000","last_peered":"2023-10-10T20:24:55.528505+0000","last_clean":"2023-10-10T20:24:55.528505+0000","last_became_active":"2023-10-10T20:24:55.527599+0000","last_became_peered":"2023-10-10T20:24:55.527599+0000","last_unstale":"2023-10-10T20:24:55.528505+0000","last_undegraded":"2023-10-10T20:24:55.528505+0000","last_fullsized":"2023-10-10T20:24:55.528505+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:37.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"115'8","reported_seq":18,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.089335+0000","last_change":"2023-10-10T20:24:55.090355+0000","last_active":"2023-10-10T20:24:56.089335+0000","last_peered":"2023-10-10T20:24:56.089335+0000","last_clean":"2023-10-10T20:24:56.089335+0000","last_became_active":"2023-10-10T20:24:55.087348+0000","last_became_peered":"2023-10-10T20:24:55.087348+0000","last_unstale":"2023-10-10T20:24:56.089335+0000","last_undegraded":"2023-10-10T20:24:56.089335+0000","last_fullsized":"2023-10-10T20:24:56.089335+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:27.934432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534056+0000","last_change":"2023-10-10T20:24:55.533961+0000","last_active":"2023-10-10T20:24:55.534056+0000","last_peered":"2023-10-10T20:24:55.534056+0000","last_clean":"2023-10-10T20:24:55.534056+0000","last_became_active":"2023-10-10T20:24:55.533734+0000","last_became_peered":"2023-10-10T20:24:55.533734+0000","last_unstale":"2023-10-10T20:24:55.534056+0000","last_undegraded":"2023-10-10T20:24:55.534056+0000","last_fullsized":"2023-10-10T20:24:55.534056+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:48:44.887263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088453+0000","last_change":"2023-10-10T20:24:55.092387+0000","last_active":"2023-10-10T20:24:56.088453+0000","last_peered":"2023-10-10T20:24:56.088453+0000","last_clean":"2023-10-10T20:24:56.088453+0000","last_became_active":"2023-10-10T20:24:55.091474+0000","last_became_peered":"2023-10-10T20:24:55.091474+0000","last_unstale":"2023-10-10T20:24:56.088453+0000","last_undegraded":"2023-10-10T20:24:56.088453+0000","last_fullsized":"2023-10-10T20:24:56.088453+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:37.283070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532770+0000","last_change":"2023-10-10T20:24:55.532710+0000","last_active":"2023-10-10T20:24:55.532770+0000","last_peered":"2023-10-10T20:24:55.532770+0000","last_clean":"2023-10-10T20:24:55.532770+0000","last_became_active":"2023-10-10T20:24:55.532485+0000","last_became_peered":"2023-10-10T20:24:55.532485+0000","last_unstale":"2023-10-10T20:24:55.532770+0000","last_undegraded":"2023-10-10T20:24:55.532770+0000","last_fullsized":"2023-10-10T20:24:55.532770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:24.816344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087127+0000","last_change":"2023-10-10T20:24:55.089913+0000","last_active":"2023-10-10T20:24:56.087127+0000","last_peered":"2023-10-10T20:24:56.087127+0000","last_clean":"2023-10-10T20:24:56.087127+0000","last_became_active":"2023-10-10T20:24:55.089687+0000","last_became_peered":"2023-10-10T20:24:55.089687+0000","last_unstale":"2023-10-10T20:24:56.087127+0000","last_undegraded":"2023-10-10T20:24:56.087127+0000","last_fullsized":"2023-10-10T20:24:56.087127+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:19.396498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536764+0000","last_change":"2023-10-10T20:24:55.536692+0000","last_active":"2023-10-10T20:24:55.536764+0000","last_peered":"2023-10-10T20:24:55.536764+0000","last_clean":"2023-10-10T20:24:55.536764+0000","last_became_active":"2023-10-10T20:24:55.536314+0000","last_became_peered":"2023-10-10T20:24:55.536314+0000","last_unstale":"2023-10-10T20:24:55.536764+0000","last_undegraded":"2023-10-10T20:24:55.536764+0000","last_fullsized":"2023-10-10T20:24:55.536764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:16:40.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533063+0000","last_change":"2023-10-10T20:24:55.092154+0000","last_active":"2023-10-10T20:24:55.533063+0000","last_peered":"2023-10-10T20:24:55.533063+0000","last_clean":"2023-10-10T20:24:55.533063+0000","last_became_active":"2023-10-10T20:24:55.091723+0000","last_became_peered":"2023-10-10T20:24:55.091723+0000","last_unstale":"2023-10-10T20:24:55.533063+0000","last_undegraded":"2023-10-10T20:24:55.533063+0000","last_fullsized":"2023-10-10T20:24:55.533063+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:05.586990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532352+0000","last_change":"2023-10-10T20:24:55.532233+0000","last_active":"2023-10-10T20:24:55.532352+0000","last_peered":"2023-10-10T20:24:55.532352+0000","last_clean":"2023-10-10T20:24:55.532352+0000","last_became_active":"2023-10-10T20:24:55.531471+0000","last_became_peered":"2023-10-10T20:24:55.531471+0000","last_unstale":"2023-10-10T20:24:55.532352+0000","last_undegraded":"2023-10-10T20:24:55.532352+0000","last_fullsized":"2023-10-10T20:24:55.532352+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:17.022444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087678+0000","last_change":"2023-10-10T20:24:55.087407+0000","last_active":"2023-10-10T20:24:56.087678+0000","last_peered":"2023-10-10T20:24:56.087678+0000","last_clean":"2023-10-10T20:24:56.087678+0000","last_became_active":"2023-10-10T20:24:55.086741+0000","last_became_peered":"2023-10-10T20:24:55.086741+0000","last_unstale":"2023-10-10T20:24:56.087678+0000","last_undegraded":"2023-10-10T20:24:56.087678+0000","last_fullsized":"2023-10-10T20:24:56.087678+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:04.670397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529357+0000","last_change":"2023-10-10T20:24:55.529315+0000","last_active":"2023-10-10T20:24:55.529357+0000","last_peered":"2023-10-10T20:24:55.529357+0000","last_clean":"2023-10-10T20:24:55.529357+0000","last_became_active":"2023-10-10T20:24:55.529151+0000","last_became_peered":"2023-10-10T20:24:55.529151+0000","last_unstale":"2023-10-10T20:24:55.529357+0000","last_undegraded":"2023-10-10T20:24:55.529357+0000","last_fullsized":"2023-10-10T20:24:55.529357+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:55.027809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.525143+0000","last_change":"2023-10-10T20:24:55.092131+0000","last_active":"2023-10-10T20:24:55.525143+0000","last_peered":"2023-10-10T20:24:55.525143+0000","last_clean":"2023-10-10T20:24:55.525143+0000","last_became_active":"2023-10-10T20:24:55.091919+0000","last_became_peered":"2023-10-10T20:24:55.091919+0000","last_unstale":"2023-10-10T20:24:55.525143+0000","last_undegraded":"2023-10-10T20:24:55.525143+0000","last_fullsized":"2023-10-10T20:24:55.525143+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:50.094329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530643+0000","last_change":"2023-10-10T20:24:55.530496+0000","last_active":"2023-10-10T20:24:55.530643+0000","last_peered":"2023-10-10T20:24:55.530643+0000","last_clean":"2023-10-10T20:24:55.530643+0000","last_became_active":"2023-10-10T20:24:55.530283+0000","last_became_peered":"2023-10-10T20:24:55.530283+0000","last_unstale":"2023-10-10T20:24:55.530643+0000","last_undegraded":"2023-10-10T20:24:55.530643+0000","last_fullsized":"2023-10-10T20:24:55.530643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:28.940070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.525381+0000","last_change":"2023-10-10T20:24:55.090532+0000","last_active":"2023-10-10T20:24:55.525381+0000","last_peered":"2023-10-10T20:24:55.525381+0000","last_clean":"2023-10-10T20:24:55.525381+0000","last_became_active":"2023-10-10T20:24:55.090286+0000","last_became_peered":"2023-10-10T20:24:55.090286+0000","last_unstale":"2023-10-10T20:24:55.525381+0000","last_undegraded":"2023-10-10T20:24:55.525381+0000","last_fullsized":"2023-10-10T20:24:55.525381+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:39.162271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533474+0000","last_change":"2023-10-10T20:24:55.533410+0000","last_active":"2023-10-10T20:24:55.533474+0000","last_peered":"2023-10-10T20:24:55.533474+0000","last_clean":"2023-10-10T20:24:55.533474+0000","last_became_active":"2023-10-10T20:24:55.533181+0000","last_became_peered":"2023-10-10T20:24:55.533181+0000","last_unstale":"2023-10-10T20:24:55.533474+0000","last_undegraded":"2023-10-10T20:24:55.533474+0000","last_fullsized":"2023-10-10T20:24:55.533474+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526811+0000","last_change":"2023-10-10T20:24:55.085375+0000","last_active":"2023-10-10T20:24:55.526811+0000","last_peered":"2023-10-10T20:24:55.526811+0000","last_clean":"2023-10-10T20:24:55.526811+0000","last_became_active":"2023-10-10T20:24:55.085147+0000","last_became_peered":"2023-10-10T20:24:55.085147+0000","last_unstale":"2023-10-10T20:24:55.526811+0000","last_undegraded":"2023-10-10T20:24:55.526811+0000","last_fullsized":"2023-10-10T20:24:55.526811+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:49.172791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537007+0000","last_change":"2023-10-10T20:24:55.536769+0000","last_active":"2023-10-10T20:24:55.537007+0000","last_peered":"2023-10-10T20:24:55.537007+0000","last_clean":"2023-10-10T20:24:55.537007+0000","last_became_active":"2023-10-10T20:24:55.536142+0000","last_became_peered":"2023-10-10T20:24:55.536142+0000","last_unstale":"2023-10-10T20:24:55.537007+0000","last_undegraded":"2023-10-10T20:24:55.537007+0000","last_fullsized":"2023-10-10T20:24:55.537007+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:30.129233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088677+0000","last_change":"2023-10-10T20:24:55.088036+0000","last_active":"2023-10-10T20:24:56.088677+0000","last_peered":"2023-10-10T20:24:56.088677+0000","last_clean":"2023-10-10T20:24:56.088677+0000","last_became_active":"2023-10-10T20:24:55.087449+0000","last_became_peered":"2023-10-10T20:24:55.087449+0000","last_unstale":"2023-10-10T20:24:56.088677+0000","last_undegraded":"2023-10-10T20:24:56.088677+0000","last_fullsized":"2023-10-10T20:24:56.088677+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:08.732769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536739+0000","last_change":"2023-10-10T20:24:55.536679+0000","last_active":"2023-10-10T20:24:55.536739+0000","last_peered":"2023-10-10T20:24:55.536739+0000","last_clean":"2023-10-10T20:24:55.536739+0000","last_became_active":"2023-10-10T20:24:55.536469+0000","last_became_peered":"2023-10-10T20:24:55.536469+0000","last_unstale":"2023-10-10T20:24:55.536739+0000","last_undegraded":"2023-10-10T20:24:55.536739+0000","last_fullsized":"2023-10-10T20:24:55.536739+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:37.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528259+0000","last_change":"2023-10-10T20:24:55.087570+0000","last_active":"2023-10-10T20:24:55.528259+0000","last_peered":"2023-10-10T20:24:55.528259+0000","last_clean":"2023-10-10T20:24:55.528259+0000","last_became_active":"2023-10-10T20:24:55.087197+0000","last_became_peered":"2023-10-10T20:24:55.087197+0000","last_unstale":"2023-10-10T20:24:55.528259+0000","last_undegraded":"2023-10-10T20:24:55.528259+0000","last_fullsized":"2023-10-10T20:24:55.528259+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:12.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536006+0000","last_change":"2023-10-10T20:24:55.535939+0000","last_active":"2023-10-10T20:24:55.536006+0000","last_peered":"2023-10-10T20:24:55.536006+0000","last_clean":"2023-10-10T20:24:55.536006+0000","last_became_active":"2023-10-10T20:24:55.535434+0000","last_became_peered":"2023-10-10T20:24:55.535434+0000","last_unstale":"2023-10-10T20:24:55.536006+0000","last_undegraded":"2023-10-10T20:24:55.536006+0000","last_fullsized":"2023-10-10T20:24:55.536006+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:40.167909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526807+0000","last_change":"2023-10-10T20:24:55.088933+0000","last_active":"2023-10-10T20:24:55.526807+0000","last_peered":"2023-10-10T20:24:55.526807+0000","last_clean":"2023-10-10T20:24:55.526807+0000","last_became_active":"2023-10-10T20:24:55.088558+0000","last_became_peered":"2023-10-10T20:24:55.088558+0000","last_unstale":"2023-10-10T20:24:55.526807+0000","last_undegraded":"2023-10-10T20:24:55.526807+0000","last_fullsized":"2023-10-10T20:24:55.526807+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:33.451410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537272+0000","last_change":"2023-10-10T20:24:55.537209+0000","last_active":"2023-10-10T20:24:55.537272+0000","last_peered":"2023-10-10T20:24:55.537272+0000","last_clean":"2023-10-10T20:24:55.537272+0000","last_became_active":"2023-10-10T20:24:55.536969+0000","last_became_peered":"2023-10-10T20:24:55.536969+0000","last_unstale":"2023-10-10T20:24:55.537272+0000","last_undegraded":"2023-10-10T20:24:55.537272+0000","last_fullsized":"2023-10-10T20:24:55.537272+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531942+0000","last_change":"2023-10-10T20:24:55.087642+0000","last_active":"2023-10-10T20:24:55.531942+0000","last_peered":"2023-10-10T20:24:55.531942+0000","last_clean":"2023-10-10T20:24:55.531942+0000","last_became_active":"2023-10-10T20:24:55.087430+0000","last_became_peered":"2023-10-10T20:24:55.087430+0000","last_unstale":"2023-10-10T20:24:55.531942+0000","last_undegraded":"2023-10-10T20:24:55.531942+0000","last_fullsized":"2023-10-10T20:24:55.531942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:15.303357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528522+0000","last_change":"2023-10-10T20:24:55.528204+0000","last_active":"2023-10-10T20:24:55.528522+0000","last_peered":"2023-10-10T20:24:55.528522+0000","last_clean":"2023-10-10T20:24:55.528522+0000","last_became_active":"2023-10-10T20:24:55.527872+0000","last_became_peered":"2023-10-10T20:24:55.527872+0000","last_unstale":"2023-10-10T20:24:55.528522+0000","last_undegraded":"2023-10-10T20:24:55.528522+0000","last_fullsized":"2023-10-10T20:24:55.528522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:44.441784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526901+0000","last_change":"2023-10-10T20:24:55.089694+0000","last_active":"2023-10-10T20:24:55.526901+0000","last_peered":"2023-10-10T20:24:55.526901+0000","last_clean":"2023-10-10T20:24:55.526901+0000","last_became_active":"2023-10-10T20:24:55.089517+0000","last_became_peered":"2023-10-10T20:24:55.089517+0000","last_unstale":"2023-10-10T20:24:55.526901+0000","last_undegraded":"2023-10-10T20:24:55.526901+0000","last_fullsized":"2023-10-10T20:24:55.526901+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:11.615673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534491+0000","last_change":"2023-10-10T20:24:55.534407+0000","last_active":"2023-10-10T20:24:55.534491+0000","last_peered":"2023-10-10T20:24:55.534491+0000","last_clean":"2023-10-10T20:24:55.534491+0000","last_became_active":"2023-10-10T20:24:55.534055+0000","last_became_peered":"2023-10-10T20:24:55.534055+0000","last_unstale":"2023-10-10T20:24:55.534491+0000","last_undegraded":"2023-10-10T20:24:55.534491+0000","last_fullsized":"2023-10-10T20:24:55.534491+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531640+0000","last_change":"2023-10-10T20:24:55.091836+0000","last_active":"2023-10-10T20:24:55.531640+0000","last_peered":"2023-10-10T20:24:55.531640+0000","last_clean":"2023-10-10T20:24:55.531640+0000","last_became_active":"2023-10-10T20:24:55.091476+0000","last_became_peered":"2023-10-10T20:24:55.091476+0000","last_unstale":"2023-10-10T20:24:55.531640+0000","last_undegraded":"2023-10-10T20:24:55.531640+0000","last_fullsized":"2023-10-10T20:24:55.531640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:12.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528479+0000","last_change":"2023-10-10T20:24:55.527267+0000","last_active":"2023-10-10T20:24:55.528479+0000","last_peered":"2023-10-10T20:24:55.528479+0000","last_clean":"2023-10-10T20:24:55.528479+0000","last_became_active":"2023-10-10T20:24:55.527047+0000","last_became_peered":"2023-10-10T20:24:55.527047+0000","last_unstale":"2023-10-10T20:24:55.528479+0000","last_undegraded":"2023-10-10T20:24:55.528479+0000","last_fullsized":"2023-10-10T20:24:55.528479+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:51:47.021997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526801+0000","last_change":"2023-10-10T20:24:55.090144+0000","last_active":"2023-10-10T20:24:55.526801+0000","last_peered":"2023-10-10T20:24:55.526801+0000","last_clean":"2023-10-10T20:24:55.526801+0000","last_became_active":"2023-10-10T20:24:55.089464+0000","last_became_peered":"2023-10-10T20:24:55.089464+0000","last_unstale":"2023-10-10T20:24:55.526801+0000","last_undegraded":"2023-10-10T20:24:55.526801+0000","last_fullsized":"2023-10-10T20:24:55.526801+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:05.586990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534016+0000","last_change":"2023-10-10T20:24:55.533786+0000","last_active":"2023-10-10T20:24:55.534016+0000","last_peered":"2023-10-10T20:24:55.534016+0000","last_clean":"2023-10-10T20:24:55.534016+0000","last_became_active":"2023-10-10T20:24:55.533558+0000","last_became_peered":"2023-10-10T20:24:55.533558+0000","last_unstale":"2023-10-10T20:24:55.534016+0000","last_undegraded":"2023-10-10T20:24:55.534016+0000","last_fullsized":"2023-10-10T20:24:55.534016+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.524043+0000","last_change":"2023-10-10T20:24:55.088405+0000","last_active":"2023-10-10T20:24:55.524043+0000","last_peered":"2023-10-10T20:24:55.524043+0000","last_clean":"2023-10-10T20:24:55.524043+0000","last_became_active":"2023-10-10T20:24:55.087162+0000","last_became_peered":"2023-10-10T20:24:55.087162+0000","last_unstale":"2023-10-10T20:24:55.524043+0000","last_undegraded":"2023-10-10T20:24:55.524043+0000","last_fullsized":"2023-10-10T20:24:55.524043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:29.556416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529967+0000","last_change":"2023-10-10T20:24:55.529634+0000","last_active":"2023-10-10T20:24:55.529967+0000","last_peered":"2023-10-10T20:24:55.529967+0000","last_clean":"2023-10-10T20:24:55.529967+0000","last_became_active":"2023-10-10T20:24:55.529031+0000","last_became_peered":"2023-10-10T20:24:55.529031+0000","last_unstale":"2023-10-10T20:24:55.529967+0000","last_undegraded":"2023-10-10T20:24:55.529967+0000","last_fullsized":"2023-10-10T20:24:55.529967+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:45.452411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.085983+0000","last_change":"2023-10-10T20:24:55.091660+0000","last_active":"2023-10-10T20:24:56.085983+0000","last_peered":"2023-10-10T20:24:56.085983+0000","last_clean":"2023-10-10T20:24:56.085983+0000","last_became_active":"2023-10-10T20:24:55.091286+0000","last_became_peered":"2023-10-10T20:24:55.091286+0000","last_unstale":"2023-10-10T20:24:56.085983+0000","last_undegraded":"2023-10-10T20:24:56.085983+0000","last_fullsized":"2023-10-10T20:24:56.085983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:16:29.018832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":250,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530238+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:24:55.530238+0000","last_peered":"2023-10-10T20:24:55.530238+0000","last_clean":"2023-10-10T20:24:55.530238+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:24:55.530238+0000","last_undegraded":"2023-10-10T20:24:55.530238+0000","last_fullsized":"2023-10-10T20:24:55.530238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1188,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194124,"statfs":{"total":200000000,"available":198783464,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69299999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.746,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.34300000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":611,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194700,"statfs":{"total":200000000,"available":199373320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.47199999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.36799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67000000000000004,"5min":1.865,"15min":1.865},"last":0.438},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.58799999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69599999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56399999999999995,"15min":0.53500000000000003},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72999999999999998,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.68000000000000005,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":15.862,"15min":15.862},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.747,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.62},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.29699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82699999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":7,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":617,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194694,"statfs":{"total":200000000,"available":199367656,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34599999999999997,"15min":0.38400000000000001},"min":{"1min":0.22800000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45800000000000002,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47299999999999998,"5min":1.647,"15min":1.647},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53400000000000003,"5min":1.524,"15min":1.524},"last":0.316},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.375,"15min":0.42799999999999999},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.32300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":1.673,"15min":1.673},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80100000000000005,"5min":1.633,"15min":1.633},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.506,"15min":1.506},"last":0.77200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":1.696,"15min":1.696},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.67300000000000004},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.46899999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.85599999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":1.585,"15min":1.585},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.622},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74199999999999999,"5min":1.478,"15min":1.478},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83599999999999997,"5min":1.722,"15min":1.722},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1187,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194125,"statfs":{"total":200000000,"available":198784033,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.371,"15min":0.45500000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.3260000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.45900000000000002,"15min":0.49099999999999999},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40699999999999997,"15min":0.48499999999999999},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.647,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.42699999999999999,"15min":0.41899999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":5.891,"5min":5.891,"15min":5.891},"last":0.312}]},{"osd":3,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41599999999999998,"15min":0.435},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.626},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71499999999999997,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.0990000000000002,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":5.6379999999999999,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.63200000000000001,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":5.758,"5min":5.758,"15min":5.758},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0720000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.59099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":610,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199374921,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.45400000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.46600000000000003,"15min":0.435},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56399999999999995,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.42199999999999999,"15min":0.39600000000000002},"min":{"1min":0.223,"5min":0.22,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.496,"15min":0.436},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54600000000000004,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.51500000000000001,"15min":0.44800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57899999999999996,"5min":11.815,"15min":11.815},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.57799999999999996,"15min":0.46700000000000003},"min":{"1min":0.25800000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56699999999999995,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61499999999999999,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.004,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.627},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.71199999999999997,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":6.399,"15min":6.399},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0429999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67600000000000005,"15min":0.60499999999999998},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0189999999999999,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.65400000000000003,"15min":0.61199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98399999999999999,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68000000000000005,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92500000000000004,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67500000000000004,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.67100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199366098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.318,"15min":0.30199999999999999},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.33600000000000002},"min":{"1min":0.25600000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.503,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.318},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.373},"min":{"1min":0.26800000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60499999999999998,"5min":0.751,"15min":0.751},"last":0.371}]},{"osd":2,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.36699999999999999},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.621,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56599999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.498}]},{"osd":5,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82799999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199365996,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53100000000000003,"15min":0.48699999999999999},"min":{"1min":0.22700000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.498},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.318,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59299999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68999999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.498}]},{"osd":6,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49099999999999999,"15min":0.46999999999999997},"min":{"1min":0.30499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.79000000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":617,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194694,"statfs":{"total":200000000,"available":199367586,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.499},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.311,"5min":0.31,"15min":0.31},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52800000000000002,"15min":0.51400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71999999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.5,"15min":0.53800000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66100000000000003,"5min":0.875,"15min":0.875},"last":0.497}]},{"osd":6,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.55900000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69699999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999}]}]}],"pool_statfs":[]}} 2023-10-10T20:25:00.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:25:00.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.293+0000 7f10d5b72700 1 Processor -- start 2023-10-10T20:25:00.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.293+0000 7f10d5b72700 1 -- start start 2023-10-10T20:25:00.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 0x7f10d015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10d01523e0 0x7f10d006c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 0x7f10d00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d006c010 con 0x7f10d01523e0 2023-10-10T20:25:00.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d006bd10 con 0x7f10d0158e50 2023-10-10T20:25:00.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10d5b72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d015c060 con 0x7f10d0152010 2023-10-10T20:25:00.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10ceffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10d01523e0 0x7f10d006c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10ceffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10d01523e0 0x7f10d006c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37088/0 (socket says 172.21.15.148:37088) 2023-10-10T20:25:00.300 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10ceffd700 1 -- 172.21.15.148:0/4073292736 learned_addr learned my addr 172.21.15.148:0/4073292736 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10cf7fe700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 0x7f10d015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.294+0000 7f10cffff700 1 --2- 172.21.15.148:0/4073292736 >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 0x7f10d00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 -- 172.21.15.148:0/4073292736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 msgr2=0x7f10d015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 0x7f10d015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 -- 172.21.15.148:0/4073292736 >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 msgr2=0x7f10d00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:00.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 --2- 172.21.15.148:0/4073292736 >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 0x7f10d00cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 -- 172.21.15.148:0/4073292736 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d015c340 con 0x7f10d01523e0 2023-10-10T20:25:00.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10cf7fe700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 0x7f10d015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:00.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10cffff700 1 --2- 172.21.15.148:0/4073292736 >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 0x7f10d00cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:00.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ceffd700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d01523e0 0x7f10d006c4a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b1660 tx=0x7f10c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adca2d9531c72b86 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ce7fc700 1 -- 172.21.15.148:0/4073292736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c400b860 con 0x7f10d01523e0 2023-10-10T20:25:00.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ce7fc700 1 -- 172.21.15.148:0/4073292736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c400b9c0 con 0x7f10d01523e0 2023-10-10T20:25:00.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10ce7fc700 1 -- 172.21.15.148:0/4073292736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c400bd60 con 0x7f10d01523e0 2023-10-10T20:25:00.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 -- 172.21.15.148:0/4073292736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d01523e0 msgr2=0x7f10d006c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d01523e0 0x7f10d006c4a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b1660 tx=0x7f10c4009250 comp rx=0 tx=0).stop 2023-10-10T20:25:00.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 -- 172.21.15.148:0/4073292736 shutdown_connections 2023-10-10T20:25:00.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 --2- 172.21.15.148:0/4073292736 >> v2:172.21.15.161:3300/0 conn(0x7f10d0158e50 0x7f10d00cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d0152010 0x7f10d015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 --2- 172.21.15.148:0/4073292736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d01523e0 0x7f10d006c4a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.295+0000 7f10d5b72700 1 -- 172.21.15.148:0/4073292736 >> 172.21.15.148:0/4073292736 conn(0x7f10d0069870 msgr2=0x7f10d0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:00.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.296+0000 7f10d5b72700 1 -- 172.21.15.148:0/4073292736 shutdown_connections 2023-10-10T20:25:00.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.296+0000 7f10d5b72700 1 -- 172.21.15.148:0/4073292736 wait complete. 2023-10-10T20:25:00.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 Processor -- start 2023-10-10T20:25:00.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 -- start start 2023-10-10T20:25:00.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d01523e0 0x7f10d00c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d0158e50 0x7f10d00c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00c7e30 con 0x7f10d0152010 2023-10-10T20:25:00.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00b2610 con 0x7f10d0158e50 2023-10-10T20:25:00.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.297+0000 7f10d5b72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00c7220 con 0x7f10d01523e0 2023-10-10T20:25:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37098/0 (socket says 172.21.15.148:37098) 2023-10-10T20:25:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 -- 172.21.15.148:0/1996813547 learned_addr learned my addr 172.21.15.148:0/1996813547 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cffff700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d0158e50 0x7f10d00c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10ceffd700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d01523e0 0x7f10d00c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 -- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d01523e0 msgr2=0x7f10d00c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d01523e0 0x7f10d00c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 -- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d0158e50 msgr2=0x7f10d00c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d0158e50 0x7f10d00c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 -- 172.21.15.148:0/1996813547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40094f0 con 0x7f10d0152010 2023-10-10T20:25:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10cf7fe700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d1d0 tx=0x7f10c0009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000dd10 con 0x7f10d0152010 2023-10-10T20:25:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000f040 con 0x7f10d0152010 2023-10-10T20:25:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d00c7500 con 0x7f10d0152010 2023-10-10T20:25:00.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0015670 con 0x7f10d0152010 2023-10-10T20:25:00.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.298+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00c7ac0 con 0x7f10d0152010 2023-10-10T20:25:00.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.299+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d0143ab0 con 0x7f10d0152010 2023-10-10T20:25:00.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.299+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10c0015a60 con 0x7f10d0152010 2023-10-10T20:25:00.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.300+0000 7f10b77fe700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 0x7f10bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.300+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f10c00848f0 con 0x7f10d0152010 2023-10-10T20:25:00.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.300+0000 7f10ceffd700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 0x7f10bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.300+0000 7f10ceffd700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 0x7f10bc0693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b1660 tx=0x7f10c4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.304+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10c004e5a0 con 0x7f10d0152010 2023-10-10T20:25:00.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.482+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10d00c7f70 con 0x7f10bc066f30 2023-10-10T20:25:00.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.490+0000 7f10b77fe700 1 -- 172.21.15.148:0/1996813547 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285566 (secure 0 0 0) 0x7f10c400bb30 con 0x7f10bc066f30 2023-10-10T20:25:00.495 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:00.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 msgr2=0x7f10bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 0x7f10bc0693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b1660 tx=0x7f10c4017000 comp rx=0 tx=0).stop 2023-10-10T20:25:00.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 msgr2=0x7f10d01434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d1d0 tx=0x7f10c0009a90 comp rx=0 tx=0).stop 2023-10-10T20:25:00.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 shutdown_connections 2023-10-10T20:25:00.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10bc066f30 0x7f10bc0693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d0158e50 0x7f10d00c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d0152010 0x7f10d01434b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 --2- 172.21.15.148:0/1996813547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10d01523e0 0x7f10d00c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 >> 172.21.15.148:0/1996813547 conn(0x7f10d0069870 msgr2=0x7f10d0069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:00.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 shutdown_connections 2023-10-10T20:25:00.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.493+0000 7f10d5b72700 1 -- 172.21.15.148:0/1996813547 wait complete. 2023-10-10T20:25:00.501 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:25:00.507 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-10T20:24:59.396279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6071,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556428,"statfs":{"total":1600000000,"available":1593783074,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.321452"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531381+0000","last_change":"2023-10-10T20:24:55.531325+0000","last_active":"2023-10-10T20:24:55.531381+0000","last_peered":"2023-10-10T20:24:55.531381+0000","last_clean":"2023-10-10T20:24:55.531381+0000","last_became_active":"2023-10-10T20:24:55.531116+0000","last_became_peered":"2023-10-10T20:24:55.531116+0000","last_unstale":"2023-10-10T20:24:55.531381+0000","last_undegraded":"2023-10-10T20:24:55.531381+0000","last_fullsized":"2023-10-10T20:24:55.531381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:51.099967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538520+0000","last_change":"2023-10-10T20:24:55.538404+0000","last_active":"2023-10-10T20:24:55.538520+0000","last_peered":"2023-10-10T20:24:55.538520+0000","last_clean":"2023-10-10T20:24:55.538520+0000","last_became_active":"2023-10-10T20:24:55.538047+0000","last_became_peered":"2023-10-10T20:24:55.538047+0000","last_unstale":"2023-10-10T20:24:55.538520+0000","last_undegraded":"2023-10-10T20:24:55.538520+0000","last_fullsized":"2023-10-10T20:24:55.538520+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:47:48.401220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534530+0000","last_change":"2023-10-10T20:24:55.534473+0000","last_active":"2023-10-10T20:24:55.534530+0000","last_peered":"2023-10-10T20:24:55.534530+0000","last_clean":"2023-10-10T20:24:55.534530+0000","last_became_active":"2023-10-10T20:24:55.534254+0000","last_became_peered":"2023-10-10T20:24:55.534254+0000","last_unstale":"2023-10-10T20:24:55.534530+0000","last_undegraded":"2023-10-10T20:24:55.534530+0000","last_fullsized":"2023-10-10T20:24:55.534530+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:53:30.381523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534653+0000","last_change":"2023-10-10T20:24:55.534529+0000","last_active":"2023-10-10T20:24:55.534653+0000","last_peered":"2023-10-10T20:24:55.534653+0000","last_clean":"2023-10-10T20:24:55.534653+0000","last_became_active":"2023-10-10T20:24:55.534250+0000","last_became_peered":"2023-10-10T20:24:55.534250+0000","last_unstale":"2023-10-10T20:24:55.534653+0000","last_undegraded":"2023-10-10T20:24:55.534653+0000","last_fullsized":"2023-10-10T20:24:55.534653+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536242+0000","last_change":"2023-10-10T20:24:55.536074+0000","last_active":"2023-10-10T20:24:55.536242+0000","last_peered":"2023-10-10T20:24:55.536242+0000","last_clean":"2023-10-10T20:24:55.536242+0000","last_became_active":"2023-10-10T20:24:55.535511+0000","last_became_peered":"2023-10-10T20:24:55.535511+0000","last_unstale":"2023-10-10T20:24:55.536242+0000","last_undegraded":"2023-10-10T20:24:55.536242+0000","last_fullsized":"2023-10-10T20:24:55.536242+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:41.882397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534662+0000","last_change":"2023-10-10T20:24:55.534599+0000","last_active":"2023-10-10T20:24:55.534662+0000","last_peered":"2023-10-10T20:24:55.534662+0000","last_clean":"2023-10-10T20:24:55.534662+0000","last_became_active":"2023-10-10T20:24:55.534246+0000","last_became_peered":"2023-10-10T20:24:55.534246+0000","last_unstale":"2023-10-10T20:24:55.534662+0000","last_undegraded":"2023-10-10T20:24:55.534662+0000","last_fullsized":"2023-10-10T20:24:55.534662+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:04.842092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533930+0000","last_change":"2023-10-10T20:24:55.533817+0000","last_active":"2023-10-10T20:24:55.533930+0000","last_peered":"2023-10-10T20:24:55.533930+0000","last_clean":"2023-10-10T20:24:55.533930+0000","last_became_active":"2023-10-10T20:24:55.533449+0000","last_became_peered":"2023-10-10T20:24:55.533449+0000","last_unstale":"2023-10-10T20:24:55.533930+0000","last_undegraded":"2023-10-10T20:24:55.533930+0000","last_fullsized":"2023-10-10T20:24:55.533930+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536991+0000","last_change":"2023-10-10T20:24:55.536640+0000","last_active":"2023-10-10T20:24:55.536991+0000","last_peered":"2023-10-10T20:24:55.536991+0000","last_clean":"2023-10-10T20:24:55.536991+0000","last_became_active":"2023-10-10T20:24:55.535792+0000","last_became_peered":"2023-10-10T20:24:55.535792+0000","last_unstale":"2023-10-10T20:24:55.536991+0000","last_undegraded":"2023-10-10T20:24:55.536991+0000","last_fullsized":"2023-10-10T20:24:55.536991+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:27.340665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535501+0000","last_change":"2023-10-10T20:24:55.535358+0000","last_active":"2023-10-10T20:24:55.535501+0000","last_peered":"2023-10-10T20:24:55.535501+0000","last_clean":"2023-10-10T20:24:55.535501+0000","last_became_active":"2023-10-10T20:24:55.535138+0000","last_became_peered":"2023-10-10T20:24:55.535138+0000","last_unstale":"2023-10-10T20:24:55.535501+0000","last_undegraded":"2023-10-10T20:24:55.535501+0000","last_fullsized":"2023-10-10T20:24:55.535501+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:33.159224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537152+0000","last_change":"2023-10-10T20:24:55.537070+0000","last_active":"2023-10-10T20:24:55.537152+0000","last_peered":"2023-10-10T20:24:55.537152+0000","last_clean":"2023-10-10T20:24:55.537152+0000","last_became_active":"2023-10-10T20:24:55.536651+0000","last_became_peered":"2023-10-10T20:24:55.536651+0000","last_unstale":"2023-10-10T20:24:55.537152+0000","last_undegraded":"2023-10-10T20:24:55.537152+0000","last_fullsized":"2023-10-10T20:24:55.537152+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:44.126722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536749+0000","last_change":"2023-10-10T20:24:55.536533+0000","last_active":"2023-10-10T20:24:55.536749+0000","last_peered":"2023-10-10T20:24:55.536749+0000","last_clean":"2023-10-10T20:24:55.536749+0000","last_became_active":"2023-10-10T20:24:55.536039+0000","last_became_peered":"2023-10-10T20:24:55.536039+0000","last_unstale":"2023-10-10T20:24:55.536749+0000","last_undegraded":"2023-10-10T20:24:55.536749+0000","last_fullsized":"2023-10-10T20:24:55.536749+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533068+0000","last_change":"2023-10-10T20:24:55.532978+0000","last_active":"2023-10-10T20:24:55.533068+0000","last_peered":"2023-10-10T20:24:55.533068+0000","last_clean":"2023-10-10T20:24:55.533068+0000","last_became_active":"2023-10-10T20:24:55.532333+0000","last_became_peered":"2023-10-10T20:24:55.532333+0000","last_unstale":"2023-10-10T20:24:55.533068+0000","last_undegraded":"2023-10-10T20:24:55.533068+0000","last_fullsized":"2023-10-10T20:24:55.533068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:50.178429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531259+0000","last_change":"2023-10-10T20:24:55.530742+0000","last_active":"2023-10-10T20:24:55.531259+0000","last_peered":"2023-10-10T20:24:55.531259+0000","last_clean":"2023-10-10T20:24:55.531259+0000","last_became_active":"2023-10-10T20:24:55.530089+0000","last_became_peered":"2023-10-10T20:24:55.530089+0000","last_unstale":"2023-10-10T20:24:55.531259+0000","last_undegraded":"2023-10-10T20:24:55.531259+0000","last_fullsized":"2023-10-10T20:24:55.531259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:56:51.022829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533045+0000","last_change":"2023-10-10T20:24:55.532491+0000","last_active":"2023-10-10T20:24:55.533045+0000","last_peered":"2023-10-10T20:24:55.533045+0000","last_clean":"2023-10-10T20:24:55.533045+0000","last_became_active":"2023-10-10T20:24:55.532032+0000","last_became_peered":"2023-10-10T20:24:55.532032+0000","last_unstale":"2023-10-10T20:24:55.533045+0000","last_undegraded":"2023-10-10T20:24:55.533045+0000","last_fullsized":"2023-10-10T20:24:55.533045+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:56.398114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532095+0000","last_change":"2023-10-10T20:24:55.531977+0000","last_active":"2023-10-10T20:24:55.532095+0000","last_peered":"2023-10-10T20:24:55.532095+0000","last_clean":"2023-10-10T20:24:55.532095+0000","last_became_active":"2023-10-10T20:24:55.530049+0000","last_became_peered":"2023-10-10T20:24:55.530049+0000","last_unstale":"2023-10-10T20:24:55.532095+0000","last_undegraded":"2023-10-10T20:24:55.532095+0000","last_fullsized":"2023-10-10T20:24:55.532095+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538143+0000","last_change":"2023-10-10T20:24:55.538067+0000","last_active":"2023-10-10T20:24:55.538143+0000","last_peered":"2023-10-10T20:24:55.538143+0000","last_clean":"2023-10-10T20:24:55.538143+0000","last_became_active":"2023-10-10T20:24:55.537819+0000","last_became_peered":"2023-10-10T20:24:55.537819+0000","last_unstale":"2023-10-10T20:24:55.538143+0000","last_undegraded":"2023-10-10T20:24:55.538143+0000","last_fullsized":"2023-10-10T20:24:55.538143+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:33.159224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536497+0000","last_change":"2023-10-10T20:24:55.536073+0000","last_active":"2023-10-10T20:24:55.536497+0000","last_peered":"2023-10-10T20:24:55.536497+0000","last_clean":"2023-10-10T20:24:55.536497+0000","last_became_active":"2023-10-10T20:24:55.535548+0000","last_became_peered":"2023-10-10T20:24:55.535548+0000","last_unstale":"2023-10-10T20:24:55.536497+0000","last_undegraded":"2023-10-10T20:24:55.536497+0000","last_fullsized":"2023-10-10T20:24:55.536497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:27:50.938300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531278+0000","last_change":"2023-10-10T20:24:55.530882+0000","last_active":"2023-10-10T20:24:55.531278+0000","last_peered":"2023-10-10T20:24:55.531278+0000","last_clean":"2023-10-10T20:24:55.531278+0000","last_became_active":"2023-10-10T20:24:55.530515+0000","last_became_peered":"2023-10-10T20:24:55.530515+0000","last_unstale":"2023-10-10T20:24:55.531278+0000","last_undegraded":"2023-10-10T20:24:55.531278+0000","last_fullsized":"2023-10-10T20:24:55.531278+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533027+0000","last_change":"2023-10-10T20:24:55.532362+0000","last_active":"2023-10-10T20:24:55.533027+0000","last_peered":"2023-10-10T20:24:55.533027+0000","last_clean":"2023-10-10T20:24:55.533027+0000","last_became_active":"2023-10-10T20:24:55.531738+0000","last_became_peered":"2023-10-10T20:24:55.531738+0000","last_unstale":"2023-10-10T20:24:55.533027+0000","last_undegraded":"2023-10-10T20:24:55.533027+0000","last_fullsized":"2023-10-10T20:24:55.533027+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:07.802603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535483+0000","last_change":"2023-10-10T20:24:55.534769+0000","last_active":"2023-10-10T20:24:55.535483+0000","last_peered":"2023-10-10T20:24:55.535483+0000","last_clean":"2023-10-10T20:24:55.535483+0000","last_became_active":"2023-10-10T20:24:55.534545+0000","last_became_peered":"2023-10-10T20:24:55.534545+0000","last_unstale":"2023-10-10T20:24:55.535483+0000","last_undegraded":"2023-10-10T20:24:55.535483+0000","last_fullsized":"2023-10-10T20:24:55.535483+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:04.842092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528939+0000","last_change":"2023-10-10T20:24:55.528893+0000","last_active":"2023-10-10T20:24:55.528939+0000","last_peered":"2023-10-10T20:24:55.528939+0000","last_clean":"2023-10-10T20:24:55.528939+0000","last_became_active":"2023-10-10T20:24:55.528720+0000","last_became_peered":"2023-10-10T20:24:55.528720+0000","last_unstale":"2023-10-10T20:24:55.528939+0000","last_undegraded":"2023-10-10T20:24:55.528939+0000","last_fullsized":"2023-10-10T20:24:55.528939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:51:04.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538313+0000","last_change":"2023-10-10T20:24:55.538222+0000","last_active":"2023-10-10T20:24:55.538313+0000","last_peered":"2023-10-10T20:24:55.538313+0000","last_clean":"2023-10-10T20:24:55.538313+0000","last_became_active":"2023-10-10T20:24:55.537820+0000","last_became_peered":"2023-10-10T20:24:55.537820+0000","last_unstale":"2023-10-10T20:24:55.538313+0000","last_undegraded":"2023-10-10T20:24:55.538313+0000","last_fullsized":"2023-10-10T20:24:55.538313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:52:59.952047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537022+0000","last_change":"2023-10-10T20:24:55.536894+0000","last_active":"2023-10-10T20:24:55.537022+0000","last_peered":"2023-10-10T20:24:55.537022+0000","last_clean":"2023-10-10T20:24:55.537022+0000","last_became_active":"2023-10-10T20:24:55.536428+0000","last_became_peered":"2023-10-10T20:24:55.536428+0000","last_unstale":"2023-10-10T20:24:55.537022+0000","last_undegraded":"2023-10-10T20:24:55.537022+0000","last_fullsized":"2023-10-10T20:24:55.537022+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:25.002322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532197+0000","last_change":"2023-10-10T20:24:55.532093+0000","last_active":"2023-10-10T20:24:55.532197+0000","last_peered":"2023-10-10T20:24:55.532197+0000","last_clean":"2023-10-10T20:24:55.532197+0000","last_became_active":"2023-10-10T20:24:55.531866+0000","last_became_peered":"2023-10-10T20:24:55.531866+0000","last_unstale":"2023-10-10T20:24:55.532197+0000","last_undegraded":"2023-10-10T20:24:55.532197+0000","last_fullsized":"2023-10-10T20:24:55.532197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536252+0000","last_change":"2023-10-10T20:24:55.536191+0000","last_active":"2023-10-10T20:24:55.536252+0000","last_peered":"2023-10-10T20:24:55.536252+0000","last_clean":"2023-10-10T20:24:55.536252+0000","last_became_active":"2023-10-10T20:24:55.535744+0000","last_became_peered":"2023-10-10T20:24:55.535744+0000","last_unstale":"2023-10-10T20:24:55.536252+0000","last_undegraded":"2023-10-10T20:24:55.536252+0000","last_fullsized":"2023-10-10T20:24:55.536252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:38:34.511054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536472+0000","last_change":"2023-10-10T20:24:55.535193+0000","last_active":"2023-10-10T20:24:55.536472+0000","last_peered":"2023-10-10T20:24:55.536472+0000","last_clean":"2023-10-10T20:24:55.536472+0000","last_became_active":"2023-10-10T20:24:55.532997+0000","last_became_peered":"2023-10-10T20:24:55.532997+0000","last_unstale":"2023-10-10T20:24:55.536472+0000","last_undegraded":"2023-10-10T20:24:55.536472+0000","last_fullsized":"2023-10-10T20:24:55.536472+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:37:02.780454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536110+0000","last_change":"2023-10-10T20:24:55.536041+0000","last_active":"2023-10-10T20:24:55.536110+0000","last_peered":"2023-10-10T20:24:55.536110+0000","last_clean":"2023-10-10T20:24:55.536110+0000","last_became_active":"2023-10-10T20:24:55.535814+0000","last_became_peered":"2023-10-10T20:24:55.535814+0000","last_unstale":"2023-10-10T20:24:55.536110+0000","last_undegraded":"2023-10-10T20:24:55.536110+0000","last_fullsized":"2023-10-10T20:24:55.536110+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:05.676035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537133+0000","last_change":"2023-10-10T20:24:55.537064+0000","last_active":"2023-10-10T20:24:55.537133+0000","last_peered":"2023-10-10T20:24:55.537133+0000","last_clean":"2023-10-10T20:24:55.537133+0000","last_became_active":"2023-10-10T20:24:55.536696+0000","last_became_peered":"2023-10-10T20:24:55.536696+0000","last_unstale":"2023-10-10T20:24:55.537133+0000","last_undegraded":"2023-10-10T20:24:55.537133+0000","last_fullsized":"2023-10-10T20:24:55.537133+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:37:02.780454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531144+0000","last_change":"2023-10-10T20:24:55.531082+0000","last_active":"2023-10-10T20:24:55.531144+0000","last_peered":"2023-10-10T20:24:55.531144+0000","last_clean":"2023-10-10T20:24:55.531144+0000","last_became_active":"2023-10-10T20:24:55.530868+0000","last_became_peered":"2023-10-10T20:24:55.530868+0000","last_unstale":"2023-10-10T20:24:55.531144+0000","last_undegraded":"2023-10-10T20:24:55.531144+0000","last_fullsized":"2023-10-10T20:24:55.531144+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:06:41.882397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533060+0000","last_change":"2023-10-10T20:24:55.532834+0000","last_active":"2023-10-10T20:24:55.533060+0000","last_peered":"2023-10-10T20:24:55.533060+0000","last_clean":"2023-10-10T20:24:55.533060+0000","last_became_active":"2023-10-10T20:24:55.532541+0000","last_became_peered":"2023-10-10T20:24:55.532541+0000","last_unstale":"2023-10-10T20:24:55.533060+0000","last_undegraded":"2023-10-10T20:24:55.533060+0000","last_fullsized":"2023-10-10T20:24:55.533060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:10:16.210335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534636+0000","last_change":"2023-10-10T20:24:55.533981+0000","last_active":"2023-10-10T20:24:55.534636+0000","last_peered":"2023-10-10T20:24:55.534636+0000","last_clean":"2023-10-10T20:24:55.534636+0000","last_became_active":"2023-10-10T20:24:55.533632+0000","last_became_peered":"2023-10-10T20:24:55.533632+0000","last_unstale":"2023-10-10T20:24:55.534636+0000","last_undegraded":"2023-10-10T20:24:55.534636+0000","last_fullsized":"2023-10-10T20:24:55.534636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:16:40.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.538816+0000","last_change":"2023-10-10T20:24:55.538754+0000","last_active":"2023-10-10T20:24:55.538816+0000","last_peered":"2023-10-10T20:24:55.538816+0000","last_clean":"2023-10-10T20:24:55.538816+0000","last_became_active":"2023-10-10T20:24:55.538571+0000","last_became_peered":"2023-10-10T20:24:55.538571+0000","last_unstale":"2023-10-10T20:24:55.538816+0000","last_undegraded":"2023-10-10T20:24:55.538816+0000","last_fullsized":"2023-10-10T20:24:55.538816+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:09.738407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535958+0000","last_change":"2023-10-10T20:24:55.535813+0000","last_active":"2023-10-10T20:24:55.535958+0000","last_peered":"2023-10-10T20:24:55.535958+0000","last_clean":"2023-10-10T20:24:55.535958+0000","last_became_active":"2023-10-10T20:24:55.535575+0000","last_became_peered":"2023-10-10T20:24:55.535575+0000","last_unstale":"2023-10-10T20:24:55.535958+0000","last_undegraded":"2023-10-10T20:24:55.535958+0000","last_fullsized":"2023-10-10T20:24:55.535958+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:59:05.141540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"115'4","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087748+0000","last_change":"2023-10-10T20:24:55.087002+0000","last_active":"2023-10-10T20:24:56.087748+0000","last_peered":"2023-10-10T20:24:56.087748+0000","last_clean":"2023-10-10T20:24:56.087748+0000","last_became_active":"2023-10-10T20:24:55.086447+0000","last_became_peered":"2023-10-10T20:24:55.086447+0000","last_unstale":"2023-10-10T20:24:56.087748+0000","last_undegraded":"2023-10-10T20:24:56.087748+0000","last_fullsized":"2023-10-10T20:24:56.087748+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:37:32.153586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533633+0000","last_change":"2023-10-10T20:24:55.533575+0000","last_active":"2023-10-10T20:24:55.533633+0000","last_peered":"2023-10-10T20:24:55.533633+0000","last_clean":"2023-10-10T20:24:55.533633+0000","last_became_active":"2023-10-10T20:24:55.533291+0000","last_became_peered":"2023-10-10T20:24:55.533291+0000","last_unstale":"2023-10-10T20:24:55.533633+0000","last_undegraded":"2023-10-10T20:24:55.533633+0000","last_fullsized":"2023-10-10T20:24:55.533633+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088623+0000","last_change":"2023-10-10T20:24:55.091517+0000","last_active":"2023-10-10T20:24:56.088623+0000","last_peered":"2023-10-10T20:24:56.088623+0000","last_clean":"2023-10-10T20:24:56.088623+0000","last_became_active":"2023-10-10T20:24:55.090946+0000","last_became_peered":"2023-10-10T20:24:55.090946+0000","last_unstale":"2023-10-10T20:24:56.088623+0000","last_undegraded":"2023-10-10T20:24:56.088623+0000","last_fullsized":"2023-10-10T20:24:56.088623+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:44:51.532047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534061+0000","last_change":"2023-10-10T20:24:55.533995+0000","last_active":"2023-10-10T20:24:55.534061+0000","last_peered":"2023-10-10T20:24:55.534061+0000","last_clean":"2023-10-10T20:24:55.534061+0000","last_became_active":"2023-10-10T20:24:55.533684+0000","last_became_peered":"2023-10-10T20:24:55.533684+0000","last_unstale":"2023-10-10T20:24:55.534061+0000","last_undegraded":"2023-10-10T20:24:55.534061+0000","last_fullsized":"2023-10-10T20:24:55.534061+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:09:15.612677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.086931+0000","last_change":"2023-10-10T20:24:55.091372+0000","last_active":"2023-10-10T20:24:56.086931+0000","last_peered":"2023-10-10T20:24:56.086931+0000","last_clean":"2023-10-10T20:24:56.086931+0000","last_became_active":"2023-10-10T20:24:55.091152+0000","last_became_peered":"2023-10-10T20:24:55.091152+0000","last_unstale":"2023-10-10T20:24:56.086931+0000","last_undegraded":"2023-10-10T20:24:56.086931+0000","last_fullsized":"2023-10-10T20:24:56.086931+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:57:04.614889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530857+0000","last_change":"2023-10-10T20:24:55.529353+0000","last_active":"2023-10-10T20:24:55.530857+0000","last_peered":"2023-10-10T20:24:55.530857+0000","last_clean":"2023-10-10T20:24:55.530857+0000","last_became_active":"2023-10-10T20:24:55.529140+0000","last_became_peered":"2023-10-10T20:24:55.529140+0000","last_unstale":"2023-10-10T20:24:55.530857+0000","last_undegraded":"2023-10-10T20:24:55.530857+0000","last_fullsized":"2023-10-10T20:24:55.530857+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:12.621312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087611+0000","last_change":"2023-10-10T20:24:55.086599+0000","last_active":"2023-10-10T20:24:56.087611+0000","last_peered":"2023-10-10T20:24:56.087611+0000","last_clean":"2023-10-10T20:24:56.087611+0000","last_became_active":"2023-10-10T20:24:55.086389+0000","last_became_peered":"2023-10-10T20:24:55.086389+0000","last_unstale":"2023-10-10T20:24:56.087611+0000","last_undegraded":"2023-10-10T20:24:56.087611+0000","last_fullsized":"2023-10-10T20:24:56.087611+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:48:03.836454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528929+0000","last_change":"2023-10-10T20:24:55.528854+0000","last_active":"2023-10-10T20:24:55.528929+0000","last_peered":"2023-10-10T20:24:55.528929+0000","last_clean":"2023-10-10T20:24:55.528929+0000","last_became_active":"2023-10-10T20:24:55.527778+0000","last_became_peered":"2023-10-10T20:24:55.527778+0000","last_unstale":"2023-10-10T20:24:55.528929+0000","last_undegraded":"2023-10-10T20:24:55.528929+0000","last_fullsized":"2023-10-10T20:24:55.528929+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:20:11.533307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088405+0000","last_change":"2023-10-10T20:24:55.090694+0000","last_active":"2023-10-10T20:24:56.088405+0000","last_peered":"2023-10-10T20:24:56.088405+0000","last_clean":"2023-10-10T20:24:56.088405+0000","last_became_active":"2023-10-10T20:24:55.090441+0000","last_became_peered":"2023-10-10T20:24:55.090441+0000","last_unstale":"2023-10-10T20:24:56.088405+0000","last_undegraded":"2023-10-10T20:24:56.088405+0000","last_fullsized":"2023-10-10T20:24:56.088405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:52:06.796964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533911+0000","last_change":"2023-10-10T20:24:55.533676+0000","last_active":"2023-10-10T20:24:55.533911+0000","last_peered":"2023-10-10T20:24:55.533911+0000","last_clean":"2023-10-10T20:24:55.533911+0000","last_became_active":"2023-10-10T20:24:55.533139+0000","last_became_peered":"2023-10-10T20:24:55.533139+0000","last_unstale":"2023-10-10T20:24:55.533911+0000","last_undegraded":"2023-10-10T20:24:55.533911+0000","last_fullsized":"2023-10-10T20:24:55.533911+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088788+0000","last_change":"2023-10-10T20:24:55.086225+0000","last_active":"2023-10-10T20:24:56.088788+0000","last_peered":"2023-10-10T20:24:56.088788+0000","last_clean":"2023-10-10T20:24:56.088788+0000","last_became_active":"2023-10-10T20:24:55.085448+0000","last_became_peered":"2023-10-10T20:24:55.085448+0000","last_unstale":"2023-10-10T20:24:56.088788+0000","last_undegraded":"2023-10-10T20:24:56.088788+0000","last_fullsized":"2023-10-10T20:24:56.088788+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:29.463293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532060+0000","last_change":"2023-10-10T20:24:55.530319+0000","last_active":"2023-10-10T20:24:55.532060+0000","last_peered":"2023-10-10T20:24:55.532060+0000","last_clean":"2023-10-10T20:24:55.532060+0000","last_became_active":"2023-10-10T20:24:55.529699+0000","last_became_peered":"2023-10-10T20:24:55.529699+0000","last_unstale":"2023-10-10T20:24:55.532060+0000","last_undegraded":"2023-10-10T20:24:55.532060+0000","last_fullsized":"2023-10-10T20:24:55.532060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529460+0000","last_change":"2023-10-10T20:24:55.093765+0000","last_active":"2023-10-10T20:24:55.529460+0000","last_peered":"2023-10-10T20:24:55.529460+0000","last_clean":"2023-10-10T20:24:55.529460+0000","last_became_active":"2023-10-10T20:24:55.093553+0000","last_became_peered":"2023-10-10T20:24:55.093553+0000","last_unstale":"2023-10-10T20:24:55.529460+0000","last_undegraded":"2023-10-10T20:24:55.529460+0000","last_fullsized":"2023-10-10T20:24:55.529460+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:03:29.463293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530874+0000","last_change":"2023-10-10T20:24:55.529812+0000","last_active":"2023-10-10T20:24:55.530874+0000","last_peered":"2023-10-10T20:24:55.530874+0000","last_clean":"2023-10-10T20:24:55.530874+0000","last_became_active":"2023-10-10T20:24:55.529596+0000","last_became_peered":"2023-10-10T20:24:55.529596+0000","last_unstale":"2023-10-10T20:24:55.530874+0000","last_undegraded":"2023-10-10T20:24:55.530874+0000","last_fullsized":"2023-10-10T20:24:55.530874+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:34.457048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528651+0000","last_change":"2023-10-10T20:24:55.090776+0000","last_active":"2023-10-10T20:24:55.528651+0000","last_peered":"2023-10-10T20:24:55.528651+0000","last_clean":"2023-10-10T20:24:55.528651+0000","last_became_active":"2023-10-10T20:24:55.090568+0000","last_became_peered":"2023-10-10T20:24:55.090568+0000","last_unstale":"2023-10-10T20:24:55.528651+0000","last_undegraded":"2023-10-10T20:24:55.528651+0000","last_fullsized":"2023-10-10T20:24:55.528651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:20:34.452779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536513+0000","last_change":"2023-10-10T20:24:55.536227+0000","last_active":"2023-10-10T20:24:55.536513+0000","last_peered":"2023-10-10T20:24:55.536513+0000","last_clean":"2023-10-10T20:24:55.536513+0000","last_became_active":"2023-10-10T20:24:55.535841+0000","last_became_peered":"2023-10-10T20:24:55.535841+0000","last_unstale":"2023-10-10T20:24:55.536513+0000","last_undegraded":"2023-10-10T20:24:55.536513+0000","last_fullsized":"2023-10-10T20:24:55.536513+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:40.323767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528314+0000","last_change":"2023-10-10T20:24:55.091121+0000","last_active":"2023-10-10T20:24:55.528314+0000","last_peered":"2023-10-10T20:24:55.528314+0000","last_clean":"2023-10-10T20:24:55.528314+0000","last_became_active":"2023-10-10T20:24:55.090888+0000","last_became_peered":"2023-10-10T20:24:55.090888+0000","last_unstale":"2023-10-10T20:24:55.528314+0000","last_undegraded":"2023-10-10T20:24:55.528314+0000","last_fullsized":"2023-10-10T20:24:55.528314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:16.016806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528985+0000","last_change":"2023-10-10T20:24:55.528798+0000","last_active":"2023-10-10T20:24:55.528985+0000","last_peered":"2023-10-10T20:24:55.528985+0000","last_clean":"2023-10-10T20:24:55.528985+0000","last_became_active":"2023-10-10T20:24:55.528574+0000","last_became_peered":"2023-10-10T20:24:55.528574+0000","last_unstale":"2023-10-10T20:24:55.528985+0000","last_undegraded":"2023-10-10T20:24:55.528985+0000","last_fullsized":"2023-10-10T20:24:55.528985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:05.676035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526542+0000","last_change":"2023-10-10T20:24:55.088111+0000","last_active":"2023-10-10T20:24:55.526542+0000","last_peered":"2023-10-10T20:24:55.526542+0000","last_clean":"2023-10-10T20:24:55.526542+0000","last_became_active":"2023-10-10T20:24:55.087889+0000","last_became_peered":"2023-10-10T20:24:55.087889+0000","last_unstale":"2023-10-10T20:24:55.526542+0000","last_undegraded":"2023-10-10T20:24:55.526542+0000","last_fullsized":"2023-10-10T20:24:55.526542+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:19.396498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535153+0000","last_change":"2023-10-10T20:24:55.535094+0000","last_active":"2023-10-10T20:24:55.535153+0000","last_peered":"2023-10-10T20:24:55.535153+0000","last_clean":"2023-10-10T20:24:55.535153+0000","last_became_active":"2023-10-10T20:24:55.534723+0000","last_became_peered":"2023-10-10T20:24:55.534723+0000","last_unstale":"2023-10-10T20:24:55.535153+0000","last_undegraded":"2023-10-10T20:24:55.535153+0000","last_fullsized":"2023-10-10T20:24:55.535153+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:29.184446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"115'3","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088144+0000","last_change":"2023-10-10T20:24:55.086404+0000","last_active":"2023-10-10T20:24:56.088144+0000","last_peered":"2023-10-10T20:24:56.088144+0000","last_clean":"2023-10-10T20:24:56.088144+0000","last_became_active":"2023-10-10T20:24:55.086174+0000","last_became_peered":"2023-10-10T20:24:55.086174+0000","last_unstale":"2023-10-10T20:24:56.088144+0000","last_undegraded":"2023-10-10T20:24:56.088144+0000","last_fullsized":"2023-10-10T20:24:56.088144+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:37:04.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531389+0000","last_change":"2023-10-10T20:24:55.531324+0000","last_active":"2023-10-10T20:24:55.531389+0000","last_peered":"2023-10-10T20:24:55.531389+0000","last_clean":"2023-10-10T20:24:55.531389+0000","last_became_active":"2023-10-10T20:24:55.530970+0000","last_became_peered":"2023-10-10T20:24:55.530970+0000","last_unstale":"2023-10-10T20:24:55.531389+0000","last_undegraded":"2023-10-10T20:24:55.531389+0000","last_fullsized":"2023-10-10T20:24:55.531389+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"115'6","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.090240+0000","last_change":"2023-10-10T20:24:55.090819+0000","last_active":"2023-10-10T20:24:56.090240+0000","last_peered":"2023-10-10T20:24:56.090240+0000","last_clean":"2023-10-10T20:24:56.090240+0000","last_became_active":"2023-10-10T20:24:55.090580+0000","last_became_peered":"2023-10-10T20:24:55.090580+0000","last_unstale":"2023-10-10T20:24:56.090240+0000","last_undegraded":"2023-10-10T20:24:56.090240+0000","last_fullsized":"2023-10-10T20:24:56.090240+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:57:42.315942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534610+0000","last_change":"2023-10-10T20:24:55.533847+0000","last_active":"2023-10-10T20:24:55.534610+0000","last_peered":"2023-10-10T20:24:55.534610+0000","last_clean":"2023-10-10T20:24:55.534610+0000","last_became_active":"2023-10-10T20:24:55.533342+0000","last_became_peered":"2023-10-10T20:24:55.533342+0000","last_unstale":"2023-10-10T20:24:55.534610+0000","last_undegraded":"2023-10-10T20:24:55.534610+0000","last_fullsized":"2023-10-10T20:24:55.534610+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:50.178429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088507+0000","last_change":"2023-10-10T20:24:55.092557+0000","last_active":"2023-10-10T20:24:56.088507+0000","last_peered":"2023-10-10T20:24:56.088507+0000","last_clean":"2023-10-10T20:24:56.088507+0000","last_became_active":"2023-10-10T20:24:55.091759+0000","last_became_peered":"2023-10-10T20:24:55.091759+0000","last_unstale":"2023-10-10T20:24:56.088507+0000","last_undegraded":"2023-10-10T20:24:56.088507+0000","last_fullsized":"2023-10-10T20:24:56.088507+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:10.381258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.535989+0000","last_change":"2023-10-10T20:24:55.535805+0000","last_active":"2023-10-10T20:24:55.535989+0000","last_peered":"2023-10-10T20:24:55.535989+0000","last_clean":"2023-10-10T20:24:55.535989+0000","last_became_active":"2023-10-10T20:24:55.534918+0000","last_became_peered":"2023-10-10T20:24:55.534918+0000","last_unstale":"2023-10-10T20:24:55.535989+0000","last_undegraded":"2023-10-10T20:24:55.535989+0000","last_fullsized":"2023-10-10T20:24:55.535989+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:23.855778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.524876+0000","last_change":"2023-10-10T20:24:55.082820+0000","last_active":"2023-10-10T20:24:55.524876+0000","last_peered":"2023-10-10T20:24:55.524876+0000","last_clean":"2023-10-10T20:24:55.524876+0000","last_became_active":"2023-10-10T20:24:55.082600+0000","last_became_peered":"2023-10-10T20:24:55.082600+0000","last_unstale":"2023-10-10T20:24:55.524876+0000","last_undegraded":"2023-10-10T20:24:55.524876+0000","last_fullsized":"2023-10-10T20:24:55.524876+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:28:24.174847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530892+0000","last_change":"2023-10-10T20:24:55.530707+0000","last_active":"2023-10-10T20:24:55.530892+0000","last_peered":"2023-10-10T20:24:55.530892+0000","last_clean":"2023-10-10T20:24:55.530892+0000","last_became_active":"2023-10-10T20:24:55.530344+0000","last_became_peered":"2023-10-10T20:24:55.530344+0000","last_unstale":"2023-10-10T20:24:55.530892+0000","last_undegraded":"2023-10-10T20:24:55.530892+0000","last_fullsized":"2023-10-10T20:24:55.530892+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:06.592628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526581+0000","last_change":"2023-10-10T20:24:55.088915+0000","last_active":"2023-10-10T20:24:55.526581+0000","last_peered":"2023-10-10T20:24:55.526581+0000","last_clean":"2023-10-10T20:24:55.526581+0000","last_became_active":"2023-10-10T20:24:55.088688+0000","last_became_peered":"2023-10-10T20:24:55.088688+0000","last_unstale":"2023-10-10T20:24:55.526581+0000","last_undegraded":"2023-10-10T20:24:55.526581+0000","last_fullsized":"2023-10-10T20:24:55.526581+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:49:16.290974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528505+0000","last_change":"2023-10-10T20:24:55.528080+0000","last_active":"2023-10-10T20:24:55.528505+0000","last_peered":"2023-10-10T20:24:55.528505+0000","last_clean":"2023-10-10T20:24:55.528505+0000","last_became_active":"2023-10-10T20:24:55.527599+0000","last_became_peered":"2023-10-10T20:24:55.527599+0000","last_unstale":"2023-10-10T20:24:55.528505+0000","last_undegraded":"2023-10-10T20:24:55.528505+0000","last_fullsized":"2023-10-10T20:24:55.528505+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:37.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"115'8","reported_seq":18,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.089335+0000","last_change":"2023-10-10T20:24:55.090355+0000","last_active":"2023-10-10T20:24:56.089335+0000","last_peered":"2023-10-10T20:24:56.089335+0000","last_clean":"2023-10-10T20:24:56.089335+0000","last_became_active":"2023-10-10T20:24:55.087348+0000","last_became_peered":"2023-10-10T20:24:55.087348+0000","last_unstale":"2023-10-10T20:24:56.089335+0000","last_undegraded":"2023-10-10T20:24:56.089335+0000","last_fullsized":"2023-10-10T20:24:56.089335+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:27.934432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534056+0000","last_change":"2023-10-10T20:24:55.533961+0000","last_active":"2023-10-10T20:24:55.534056+0000","last_peered":"2023-10-10T20:24:55.534056+0000","last_clean":"2023-10-10T20:24:55.534056+0000","last_became_active":"2023-10-10T20:24:55.533734+0000","last_became_peered":"2023-10-10T20:24:55.533734+0000","last_unstale":"2023-10-10T20:24:55.534056+0000","last_undegraded":"2023-10-10T20:24:55.534056+0000","last_fullsized":"2023-10-10T20:24:55.534056+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:48:44.887263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088453+0000","last_change":"2023-10-10T20:24:55.092387+0000","last_active":"2023-10-10T20:24:56.088453+0000","last_peered":"2023-10-10T20:24:56.088453+0000","last_clean":"2023-10-10T20:24:56.088453+0000","last_became_active":"2023-10-10T20:24:55.091474+0000","last_became_peered":"2023-10-10T20:24:55.091474+0000","last_unstale":"2023-10-10T20:24:56.088453+0000","last_undegraded":"2023-10-10T20:24:56.088453+0000","last_fullsized":"2023-10-10T20:24:56.088453+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:37.283070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532770+0000","last_change":"2023-10-10T20:24:55.532710+0000","last_active":"2023-10-10T20:24:55.532770+0000","last_peered":"2023-10-10T20:24:55.532770+0000","last_clean":"2023-10-10T20:24:55.532770+0000","last_became_active":"2023-10-10T20:24:55.532485+0000","last_became_peered":"2023-10-10T20:24:55.532485+0000","last_unstale":"2023-10-10T20:24:55.532770+0000","last_undegraded":"2023-10-10T20:24:55.532770+0000","last_fullsized":"2023-10-10T20:24:55.532770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:41:24.816344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087127+0000","last_change":"2023-10-10T20:24:55.089913+0000","last_active":"2023-10-10T20:24:56.087127+0000","last_peered":"2023-10-10T20:24:56.087127+0000","last_clean":"2023-10-10T20:24:56.087127+0000","last_became_active":"2023-10-10T20:24:55.089687+0000","last_became_peered":"2023-10-10T20:24:55.089687+0000","last_unstale":"2023-10-10T20:24:56.087127+0000","last_undegraded":"2023-10-10T20:24:56.087127+0000","last_fullsized":"2023-10-10T20:24:56.087127+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:19.396498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536764+0000","last_change":"2023-10-10T20:24:55.536692+0000","last_active":"2023-10-10T20:24:55.536764+0000","last_peered":"2023-10-10T20:24:55.536764+0000","last_clean":"2023-10-10T20:24:55.536764+0000","last_became_active":"2023-10-10T20:24:55.536314+0000","last_became_peered":"2023-10-10T20:24:55.536314+0000","last_unstale":"2023-10-10T20:24:55.536764+0000","last_undegraded":"2023-10-10T20:24:55.536764+0000","last_fullsized":"2023-10-10T20:24:55.536764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:16:40.148870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533063+0000","last_change":"2023-10-10T20:24:55.092154+0000","last_active":"2023-10-10T20:24:55.533063+0000","last_peered":"2023-10-10T20:24:55.533063+0000","last_clean":"2023-10-10T20:24:55.533063+0000","last_became_active":"2023-10-10T20:24:55.091723+0000","last_became_peered":"2023-10-10T20:24:55.091723+0000","last_unstale":"2023-10-10T20:24:55.533063+0000","last_undegraded":"2023-10-10T20:24:55.533063+0000","last_fullsized":"2023-10-10T20:24:55.533063+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:05.586990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.532352+0000","last_change":"2023-10-10T20:24:55.532233+0000","last_active":"2023-10-10T20:24:55.532352+0000","last_peered":"2023-10-10T20:24:55.532352+0000","last_clean":"2023-10-10T20:24:55.532352+0000","last_became_active":"2023-10-10T20:24:55.531471+0000","last_became_peered":"2023-10-10T20:24:55.531471+0000","last_unstale":"2023-10-10T20:24:55.532352+0000","last_undegraded":"2023-10-10T20:24:55.532352+0000","last_fullsized":"2023-10-10T20:24:55.532352+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:04:17.022444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.087678+0000","last_change":"2023-10-10T20:24:55.087407+0000","last_active":"2023-10-10T20:24:56.087678+0000","last_peered":"2023-10-10T20:24:56.087678+0000","last_clean":"2023-10-10T20:24:56.087678+0000","last_became_active":"2023-10-10T20:24:55.086741+0000","last_became_peered":"2023-10-10T20:24:55.086741+0000","last_unstale":"2023-10-10T20:24:56.087678+0000","last_undegraded":"2023-10-10T20:24:56.087678+0000","last_fullsized":"2023-10-10T20:24:56.087678+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:01:04.670397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529357+0000","last_change":"2023-10-10T20:24:55.529315+0000","last_active":"2023-10-10T20:24:55.529357+0000","last_peered":"2023-10-10T20:24:55.529357+0000","last_clean":"2023-10-10T20:24:55.529357+0000","last_became_active":"2023-10-10T20:24:55.529151+0000","last_became_peered":"2023-10-10T20:24:55.529151+0000","last_unstale":"2023-10-10T20:24:55.529357+0000","last_undegraded":"2023-10-10T20:24:55.529357+0000","last_fullsized":"2023-10-10T20:24:55.529357+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:55.027809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.525143+0000","last_change":"2023-10-10T20:24:55.092131+0000","last_active":"2023-10-10T20:24:55.525143+0000","last_peered":"2023-10-10T20:24:55.525143+0000","last_clean":"2023-10-10T20:24:55.525143+0000","last_became_active":"2023-10-10T20:24:55.091919+0000","last_became_peered":"2023-10-10T20:24:55.091919+0000","last_unstale":"2023-10-10T20:24:55.525143+0000","last_undegraded":"2023-10-10T20:24:55.525143+0000","last_fullsized":"2023-10-10T20:24:55.525143+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:50.094329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530643+0000","last_change":"2023-10-10T20:24:55.530496+0000","last_active":"2023-10-10T20:24:55.530643+0000","last_peered":"2023-10-10T20:24:55.530643+0000","last_clean":"2023-10-10T20:24:55.530643+0000","last_became_active":"2023-10-10T20:24:55.530283+0000","last_became_peered":"2023-10-10T20:24:55.530283+0000","last_unstale":"2023-10-10T20:24:55.530643+0000","last_undegraded":"2023-10-10T20:24:55.530643+0000","last_fullsized":"2023-10-10T20:24:55.530643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:28.940070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.525381+0000","last_change":"2023-10-10T20:24:55.090532+0000","last_active":"2023-10-10T20:24:55.525381+0000","last_peered":"2023-10-10T20:24:55.525381+0000","last_clean":"2023-10-10T20:24:55.525381+0000","last_became_active":"2023-10-10T20:24:55.090286+0000","last_became_peered":"2023-10-10T20:24:55.090286+0000","last_unstale":"2023-10-10T20:24:55.525381+0000","last_undegraded":"2023-10-10T20:24:55.525381+0000","last_fullsized":"2023-10-10T20:24:55.525381+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:39.162271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.533474+0000","last_change":"2023-10-10T20:24:55.533410+0000","last_active":"2023-10-10T20:24:55.533474+0000","last_peered":"2023-10-10T20:24:55.533474+0000","last_clean":"2023-10-10T20:24:55.533474+0000","last_became_active":"2023-10-10T20:24:55.533181+0000","last_became_peered":"2023-10-10T20:24:55.533181+0000","last_unstale":"2023-10-10T20:24:55.533474+0000","last_undegraded":"2023-10-10T20:24:55.533474+0000","last_fullsized":"2023-10-10T20:24:55.533474+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:19:12.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526811+0000","last_change":"2023-10-10T20:24:55.085375+0000","last_active":"2023-10-10T20:24:55.526811+0000","last_peered":"2023-10-10T20:24:55.526811+0000","last_clean":"2023-10-10T20:24:55.526811+0000","last_became_active":"2023-10-10T20:24:55.085147+0000","last_became_peered":"2023-10-10T20:24:55.085147+0000","last_unstale":"2023-10-10T20:24:55.526811+0000","last_undegraded":"2023-10-10T20:24:55.526811+0000","last_fullsized":"2023-10-10T20:24:55.526811+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:50:49.172791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537007+0000","last_change":"2023-10-10T20:24:55.536769+0000","last_active":"2023-10-10T20:24:55.537007+0000","last_peered":"2023-10-10T20:24:55.537007+0000","last_clean":"2023-10-10T20:24:55.537007+0000","last_became_active":"2023-10-10T20:24:55.536142+0000","last_became_peered":"2023-10-10T20:24:55.536142+0000","last_unstale":"2023-10-10T20:24:55.537007+0000","last_undegraded":"2023-10-10T20:24:55.537007+0000","last_fullsized":"2023-10-10T20:24:55.537007+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:30.129233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"115'2","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.088677+0000","last_change":"2023-10-10T20:24:55.088036+0000","last_active":"2023-10-10T20:24:56.088677+0000","last_peered":"2023-10-10T20:24:56.088677+0000","last_clean":"2023-10-10T20:24:56.088677+0000","last_became_active":"2023-10-10T20:24:55.087449+0000","last_became_peered":"2023-10-10T20:24:55.087449+0000","last_unstale":"2023-10-10T20:24:56.088677+0000","last_undegraded":"2023-10-10T20:24:56.088677+0000","last_fullsized":"2023-10-10T20:24:56.088677+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:22:08.732769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536739+0000","last_change":"2023-10-10T20:24:55.536679+0000","last_active":"2023-10-10T20:24:55.536739+0000","last_peered":"2023-10-10T20:24:55.536739+0000","last_clean":"2023-10-10T20:24:55.536739+0000","last_became_active":"2023-10-10T20:24:55.536469+0000","last_became_peered":"2023-10-10T20:24:55.536469+0000","last_unstale":"2023-10-10T20:24:55.536739+0000","last_undegraded":"2023-10-10T20:24:55.536739+0000","last_fullsized":"2023-10-10T20:24:55.536739+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:37.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528259+0000","last_change":"2023-10-10T20:24:55.087570+0000","last_active":"2023-10-10T20:24:55.528259+0000","last_peered":"2023-10-10T20:24:55.528259+0000","last_clean":"2023-10-10T20:24:55.528259+0000","last_became_active":"2023-10-10T20:24:55.087197+0000","last_became_peered":"2023-10-10T20:24:55.087197+0000","last_unstale":"2023-10-10T20:24:55.528259+0000","last_undegraded":"2023-10-10T20:24:55.528259+0000","last_fullsized":"2023-10-10T20:24:55.528259+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:12.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.536006+0000","last_change":"2023-10-10T20:24:55.535939+0000","last_active":"2023-10-10T20:24:55.536006+0000","last_peered":"2023-10-10T20:24:55.536006+0000","last_clean":"2023-10-10T20:24:55.536006+0000","last_became_active":"2023-10-10T20:24:55.535434+0000","last_became_peered":"2023-10-10T20:24:55.535434+0000","last_unstale":"2023-10-10T20:24:55.536006+0000","last_undegraded":"2023-10-10T20:24:55.536006+0000","last_fullsized":"2023-10-10T20:24:55.536006+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:40.167909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526807+0000","last_change":"2023-10-10T20:24:55.088933+0000","last_active":"2023-10-10T20:24:55.526807+0000","last_peered":"2023-10-10T20:24:55.526807+0000","last_clean":"2023-10-10T20:24:55.526807+0000","last_became_active":"2023-10-10T20:24:55.088558+0000","last_became_peered":"2023-10-10T20:24:55.088558+0000","last_unstale":"2023-10-10T20:24:55.526807+0000","last_undegraded":"2023-10-10T20:24:55.526807+0000","last_fullsized":"2023-10-10T20:24:55.526807+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:42:33.451410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.537272+0000","last_change":"2023-10-10T20:24:55.537209+0000","last_active":"2023-10-10T20:24:55.537272+0000","last_peered":"2023-10-10T20:24:55.537272+0000","last_clean":"2023-10-10T20:24:55.537272+0000","last_became_active":"2023-10-10T20:24:55.536969+0000","last_became_peered":"2023-10-10T20:24:55.536969+0000","last_unstale":"2023-10-10T20:24:55.537272+0000","last_undegraded":"2023-10-10T20:24:55.537272+0000","last_fullsized":"2023-10-10T20:24:55.537272+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:07.481411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531942+0000","last_change":"2023-10-10T20:24:55.087642+0000","last_active":"2023-10-10T20:24:55.531942+0000","last_peered":"2023-10-10T20:24:55.531942+0000","last_clean":"2023-10-10T20:24:55.531942+0000","last_became_active":"2023-10-10T20:24:55.087430+0000","last_became_peered":"2023-10-10T20:24:55.087430+0000","last_unstale":"2023-10-10T20:24:55.531942+0000","last_undegraded":"2023-10-10T20:24:55.531942+0000","last_fullsized":"2023-10-10T20:24:55.531942+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:52:15.303357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528522+0000","last_change":"2023-10-10T20:24:55.528204+0000","last_active":"2023-10-10T20:24:55.528522+0000","last_peered":"2023-10-10T20:24:55.528522+0000","last_clean":"2023-10-10T20:24:55.528522+0000","last_became_active":"2023-10-10T20:24:55.527872+0000","last_became_peered":"2023-10-10T20:24:55.527872+0000","last_unstale":"2023-10-10T20:24:55.528522+0000","last_undegraded":"2023-10-10T20:24:55.528522+0000","last_fullsized":"2023-10-10T20:24:55.528522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:33:44.441784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526901+0000","last_change":"2023-10-10T20:24:55.089694+0000","last_active":"2023-10-10T20:24:55.526901+0000","last_peered":"2023-10-10T20:24:55.526901+0000","last_clean":"2023-10-10T20:24:55.526901+0000","last_became_active":"2023-10-10T20:24:55.089517+0000","last_became_peered":"2023-10-10T20:24:55.089517+0000","last_unstale":"2023-10-10T20:24:55.526901+0000","last_undegraded":"2023-10-10T20:24:55.526901+0000","last_fullsized":"2023-10-10T20:24:55.526901+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:11.615673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534491+0000","last_change":"2023-10-10T20:24:55.534407+0000","last_active":"2023-10-10T20:24:55.534491+0000","last_peered":"2023-10-10T20:24:55.534491+0000","last_clean":"2023-10-10T20:24:55.534491+0000","last_became_active":"2023-10-10T20:24:55.534055+0000","last_became_peered":"2023-10-10T20:24:55.534055+0000","last_unstale":"2023-10-10T20:24:55.534491+0000","last_undegraded":"2023-10-10T20:24:55.534491+0000","last_fullsized":"2023-10-10T20:24:55.534491+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:19.550213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.531640+0000","last_change":"2023-10-10T20:24:55.091836+0000","last_active":"2023-10-10T20:24:55.531640+0000","last_peered":"2023-10-10T20:24:55.531640+0000","last_clean":"2023-10-10T20:24:55.531640+0000","last_became_active":"2023-10-10T20:24:55.091476+0000","last_became_peered":"2023-10-10T20:24:55.091476+0000","last_unstale":"2023-10-10T20:24:55.531640+0000","last_undegraded":"2023-10-10T20:24:55.531640+0000","last_fullsized":"2023-10-10T20:24:55.531640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:30:12.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.528479+0000","last_change":"2023-10-10T20:24:55.527267+0000","last_active":"2023-10-10T20:24:55.528479+0000","last_peered":"2023-10-10T20:24:55.528479+0000","last_clean":"2023-10-10T20:24:55.528479+0000","last_became_active":"2023-10-10T20:24:55.527047+0000","last_became_peered":"2023-10-10T20:24:55.527047+0000","last_unstale":"2023-10-10T20:24:55.528479+0000","last_undegraded":"2023-10-10T20:24:55.528479+0000","last_fullsized":"2023-10-10T20:24:55.528479+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:51:47.021997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.526801+0000","last_change":"2023-10-10T20:24:55.090144+0000","last_active":"2023-10-10T20:24:55.526801+0000","last_peered":"2023-10-10T20:24:55.526801+0000","last_clean":"2023-10-10T20:24:55.526801+0000","last_became_active":"2023-10-10T20:24:55.089464+0000","last_became_peered":"2023-10-10T20:24:55.089464+0000","last_unstale":"2023-10-10T20:24:55.526801+0000","last_undegraded":"2023-10-10T20:24:55.526801+0000","last_fullsized":"2023-10-10T20:24:55.526801+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:26:05.586990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.534016+0000","last_change":"2023-10-10T20:24:55.533786+0000","last_active":"2023-10-10T20:24:55.534016+0000","last_peered":"2023-10-10T20:24:55.534016+0000","last_clean":"2023-10-10T20:24:55.534016+0000","last_became_active":"2023-10-10T20:24:55.533558+0000","last_became_peered":"2023-10-10T20:24:55.533558+0000","last_unstale":"2023-10-10T20:24:55.534016+0000","last_undegraded":"2023-10-10T20:24:55.534016+0000","last_fullsized":"2023-10-10T20:24:55.534016+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:58.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.524043+0000","last_change":"2023-10-10T20:24:55.088405+0000","last_active":"2023-10-10T20:24:55.524043+0000","last_peered":"2023-10-10T20:24:55.524043+0000","last_clean":"2023-10-10T20:24:55.524043+0000","last_became_active":"2023-10-10T20:24:55.087162+0000","last_became_peered":"2023-10-10T20:24:55.087162+0000","last_unstale":"2023-10-10T20:24:55.524043+0000","last_undegraded":"2023-10-10T20:24:55.524043+0000","last_fullsized":"2023-10-10T20:24:55.524043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:18:29.556416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.529967+0000","last_change":"2023-10-10T20:24:55.529634+0000","last_active":"2023-10-10T20:24:55.529967+0000","last_peered":"2023-10-10T20:24:55.529967+0000","last_clean":"2023-10-10T20:24:55.529967+0000","last_became_active":"2023-10-10T20:24:55.529031+0000","last_became_peered":"2023-10-10T20:24:55.529031+0000","last_unstale":"2023-10-10T20:24:55.529967+0000","last_undegraded":"2023-10-10T20:24:55.529967+0000","last_fullsized":"2023-10-10T20:24:55.529967+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:55.074745+0000","last_clean_scrub_stamp":"2023-10-10T20:24:55.074745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:37:45.452411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"115'1","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:56.085983+0000","last_change":"2023-10-10T20:24:55.091660+0000","last_active":"2023-10-10T20:24:56.085983+0000","last_peered":"2023-10-10T20:24:56.085983+0000","last_clean":"2023-10-10T20:24:56.085983+0000","last_became_active":"2023-10-10T20:24:55.091286+0000","last_became_peered":"2023-10-10T20:24:55.091286+0000","last_unstale":"2023-10-10T20:24:56.085983+0000","last_undegraded":"2023-10-10T20:24:56.085983+0000","last_fullsized":"2023-10-10T20:24:56.085983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:24:54.069107+0000","last_clean_scrub_stamp":"2023-10-10T20:24:54.069107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:16:29.018832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":250,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-10-10T20:24:55.530238+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:24:55.530238+0000","last_peered":"2023-10-10T20:24:55.530238+0000","last_clean":"2023-10-10T20:24:55.530238+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:24:55.530238+0000","last_undegraded":"2023-10-10T20:24:55.530238+0000","last_fullsized":"2023-10-10T20:24:55.530238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1188,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194124,"statfs":{"total":200000000,"available":198783464,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.5,"15min":0.47399999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69299999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68700000000000006,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.746,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72099999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.64800000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.52200000000000002},"min":{"1min":0.34300000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71599999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50800000000000001,"15min":0.52000000000000002},"min":{"1min":0.313,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.749,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.50900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":611,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194700,"statfs":{"total":200000000,"available":199373320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69099999999999995,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.35499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72099999999999997,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.47199999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.36799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67000000000000004,"5min":1.865,"15min":1.865},"last":0.438},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.58799999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.65300000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69599999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56399999999999995,"15min":0.53500000000000003},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72999999999999998,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.68000000000000005,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":15.862,"15min":15.862},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.747,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.62},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.29699999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.82699999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.32400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":7,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.75,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":617,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194694,"statfs":{"total":200000000,"available":199367656,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34599999999999997,"15min":0.38400000000000001},"min":{"1min":0.22800000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45800000000000002,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.23499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47299999999999998,"5min":1.647,"15min":1.647},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53400000000000003,"5min":1.524,"15min":1.524},"last":0.316},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.375,"15min":0.42799999999999999},"min":{"1min":0.23200000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.32300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":1.673,"15min":1.673},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80100000000000005,"5min":1.633,"15min":1.633},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.506,"15min":1.506},"last":0.77200000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78800000000000003,"5min":1.696,"15min":1.696},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.67300000000000004},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59599999999999997,"15min":0.63400000000000001},"min":{"1min":0.46899999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.85599999999999998,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":1.585,"15min":1.585},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.622},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74199999999999999,"5min":1.478,"15min":1.478},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60299999999999998,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83599999999999997,"5min":1.722,"15min":1.722},"last":0.56899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1187,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194125,"statfs":{"total":200000000,"available":198784033,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.371,"15min":0.45500000000000002},"min":{"1min":0.22700000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.3260000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.45900000000000002,"15min":0.49099999999999999},"min":{"1min":0.32300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40699999999999997,"15min":0.48499999999999999},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.647,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.42699999999999999,"15min":0.41899999999999998},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":5.891,"5min":5.891,"15min":5.891},"last":0.312}]},{"osd":3,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.40899999999999997,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41599999999999998,"15min":0.435},"min":{"1min":0.308,"5min":0.252,"15min":0.252},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.64800000000000002,"15min":0.626},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64300000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71499999999999997,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.0990000000000002,"5min":9.7260000000000009,"15min":9.7260000000000009},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":5.6379999999999999,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.63200000000000001,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":5.758,"5min":5.758,"15min":5.758},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.0720000000000001,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.59099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":610,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199374921,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.39200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.45400000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.46600000000000003,"15min":0.435},"min":{"1min":0.27800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56399999999999995,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.42199999999999999,"15min":0.39600000000000002},"min":{"1min":0.223,"5min":0.22,"15min":0.22},"max":{"1min":0.60399999999999998,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.496,"15min":0.436},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54600000000000004,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.51500000000000001,"15min":0.44800000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57899999999999996,"5min":11.815,"15min":11.815},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.57799999999999996,"15min":0.46700000000000003},"min":{"1min":0.25800000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56699999999999995,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61499999999999999,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.004,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.627},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.71199999999999997,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.995,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63900000000000001,"15min":0.61499999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97799999999999998,"5min":6.399,"15min":6.399},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0429999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.67600000000000005,"15min":0.60499999999999998},"min":{"1min":0.371,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0189999999999999,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.65400000000000003,"15min":0.61199999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98399999999999999,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68000000000000005,"15min":0.60199999999999998},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92500000000000004,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67500000000000004,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.67100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199366098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.318,"15min":0.30199999999999999},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.314},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35499999999999998,"15min":0.33600000000000002},"min":{"1min":0.25600000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.503,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.318},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.373},"min":{"1min":0.26800000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60499999999999998,"5min":0.751,"15min":0.751},"last":0.371}]},{"osd":2,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.36699999999999999},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.621,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39100000000000001},"min":{"1min":0.29999999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56599999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.45000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.498}]},{"osd":5,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76200000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82799999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199365996,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53100000000000003,"15min":0.48699999999999999},"min":{"1min":0.22700000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.498},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.318,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59299999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68999999999999995,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.52500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.70199999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.498}]},{"osd":6,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49099999999999999,"15min":0.46999999999999997},"min":{"1min":0.30499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.79000000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":617,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194694,"statfs":{"total":200000000,"available":199367586,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.499},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.311,"5min":0.31,"15min":0.31},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52800000000000002,"15min":0.51400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47999999999999998,"15min":0.502},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71999999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.5,"15min":0.53800000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66100000000000003,"5min":0.875,"15min":0.875},"last":0.497}]},{"osd":6,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.54100000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52300000000000002,"15min":0.55900000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69699999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.57699999999999996},"min":{"1min":0.35799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61699999999999999}]}]}],"pool_statfs":[]}} 2023-10-10T20:25:00.514 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:25:00.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:25:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.668+0000 7f54b74db700 1 Processor -- start 2023-10-10T20:25:00.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.668+0000 7f54b74db700 1 -- start start 2023-10-10T20:25:00.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b74db700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54b01542a0 0x7f54b00b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b74db700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b74db700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b006ac70 con 0x7f54b006a830 2023-10-10T20:25:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b74db700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b015b2e0 con 0x7f54b01542a0 2023-10-10T20:25:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b74db700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b006b0a0 con 0x7f54b00cd630 2023-10-10T20:25:00.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54b01542a0 0x7f54b00b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54b01542a0 0x7f54b00b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37112/0 (socket says 172.21.15.148:37112) 2023-10-10T20:25:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 -- 172.21.15.148:0/1695871243 learned_addr learned my addr 172.21.15.148:0/1695871243 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b5cd8700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 -- 172.21.15.148:0/1695871243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 msgr2=0x7f54b006a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b006a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 -- 172.21.15.148:0/1695871243 >> v1:172.21.15.161:6789/0 conn(0x7f54b006a830 legacy=0x7f54b015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 -- 172.21.15.148:0/1695871243 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b015ad20 con 0x7f54b01542a0 2023-10-10T20:25:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b5cd8700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.669+0000 7f54b64d9700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b01542a0 0x7f54b00b65a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009990 tx=0x7f54ac009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215281b3da38fddd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b54d7700 1 -- 172.21.15.148:0/1695871243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005c10 con 0x7f54b01542a0 2023-10-10T20:25:00.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b54d7700 1 -- 172.21.15.148:0/1695871243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac005d70 con 0x7f54b01542a0 2023-10-10T20:25:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b54d7700 1 -- 172.21.15.148:0/1695871243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00b880 con 0x7f54b01542a0 2023-10-10T20:25:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 -- 172.21.15.148:0/1695871243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b01542a0 msgr2=0x7f54b00b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b01542a0 0x7f54b00b65a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009990 tx=0x7f54ac009220 comp rx=0 tx=0).stop 2023-10-10T20:25:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 -- 172.21.15.148:0/1695871243 shutdown_connections 2023-10-10T20:25:00.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b01542a0 0x7f54b00b65a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 --2- 172.21.15.148:0/1695871243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 -- 172.21.15.148:0/1695871243 >> 172.21.15.148:0/1695871243 conn(0x7f54b0068cb0 msgr2=0x7f54b00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 -- 172.21.15.148:0/1695871243 shutdown_connections 2023-10-10T20:25:00.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.670+0000 7f54b74db700 1 -- 172.21.15.148:0/1695871243 wait complete. 2023-10-10T20:25:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.671+0000 7f54b74db700 1 Processor -- start 2023-10-10T20:25:00.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.671+0000 7f54b74db700 1 -- start start 2023-10-10T20:25:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54b01542a0 0x7f54b00c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0157880 con 0x7f54b006a830 2023-10-10T20:25:00.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00c7b80 con 0x7f54b01542a0 2023-10-10T20:25:00.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b74db700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00c7e60 con 0x7f54b00cd630 2023-10-10T20:25:00.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37124/0 (socket says 172.21.15.148:37124) 2023-10-10T20:25:00.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 -- 172.21.15.148:0/3557624370 learned_addr learned my addr 172.21.15.148:0/3557624370 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:00.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b5cd8700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b6cda700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54b01542a0 0x7f54b00c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 -- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 msgr2=0x7f54b0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b0159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 -- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54b01542a0 msgr2=0x7f54b00c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54b01542a0 0x7f54b00c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.672+0000 7f54b64d9700 1 -- 172.21.15.148:0/3557624370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009480 con 0x7f54b006a830 2023-10-10T20:25:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54b64d9700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005d40 tx=0x7f54ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac006b60 con 0x7f54b006a830 2023-10-10T20:25:00.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac0118c0 con 0x7f54b006a830 2023-10-10T20:25:00.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b00c8140 con 0x7f54b006a830 2023-10-10T20:25:00.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac011c40 con 0x7f54b006a830 2023-10-10T20:25:00.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.673+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b0159f40 con 0x7f54b006a830 2023-10-10T20:25:00.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.674+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0157650 con 0x7f54b006a830 2023-10-10T20:25:00.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.674+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54ac01c620 con 0x7f54b006a830 2023-10-10T20:25:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.675+0000 7f54a67fc700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 0x7f54940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.675+0000 7f54b5cd8700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 0x7f54940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.675+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54ac085d50 con 0x7f54b006a830 2023-10-10T20:25:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.675+0000 7f54b5cd8700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 0x7f54940693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f54b00b2120 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:00.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.679+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54b0157650 con 0x7f54b006a830 2023-10-10T20:25:00.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.935+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f54b0157650 con 0x7f54b006a830 2023-10-10T20:25:00.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.938+0000 7f54a67fc700 1 -- 172.21.15.148:0/3557624370 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f54b0157650 con 0x7f54b006a830 2023-10-10T20:25:00.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 msgr2=0x7f54940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 0x7f54940693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f54b00b2120 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2023-10-10T20:25:00.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 msgr2=0x7f54b0153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:00.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f54ac005d40 tx=0x7f54ac016000 comp rx=0 tx=0).stop 2023-10-10T20:25:00.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 shutdown_connections 2023-10-10T20:25:00.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5494066f30 0x7f54940693e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54b01542a0 0x7f54b00c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b006a830 0x7f54b0153650 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 --2- 172.21.15.148:0/3557624370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54b00cd630 0x7f54b0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:00.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 >> 172.21.15.148:0/3557624370 conn(0x7f54b0068cb0 msgr2=0x7f54b00cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:00.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 shutdown_connections 2023-10-10T20:25:00.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:00.940+0000 7f54b74db700 1 -- 172.21.15.148:0/3557624370 wait complete. 2023-10-10T20:25:00.946 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:25:00.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:01.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.066+0000 7fef791ce700 1 Processor -- start 2023-10-10T20:25:01.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.066+0000 7fef791ce700 1 -- start start 2023-10-10T20:25:01.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef740afd90 0x7fef740c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fef7406f1a0 0x7fef7415b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740b0160 0x7fef74070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef740ca270 con 0x7fef740afd90 2023-10-10T20:25:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef740c87c0 con 0x7fef7406f1a0 2023-10-10T20:25:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef791ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7415b770 con 0x7fef740b0160 2023-10-10T20:25:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef72d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef740afd90 0x7fef740c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.067+0000 7fef72d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef740afd90 0x7fef740c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37136/0 (socket says 172.21.15.148:37136) 2023-10-10T20:25:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 -- 172.21.15.148:0/998478207 learned_addr learned my addr 172.21.15.148:0/998478207 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef6bfff700 1 --2- 172.21.15.148:0/998478207 >> v2:172.21.15.161:3300/0 conn(0x7fef7406f1a0 0x7fef7415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef7359e700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740b0160 0x7fef74070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 -- 172.21.15.148:0/998478207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740b0160 msgr2=0x7fef74070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740b0160 0x7fef74070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 -- 172.21.15.148:0/998478207 >> v2:172.21.15.161:3300/0 conn(0x7fef7406f1a0 msgr2=0x7fef7415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 --2- 172.21.15.148:0/998478207 >> v2:172.21.15.161:3300/0 conn(0x7fef7406f1a0 0x7fef7415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 -- 172.21.15.148:0/998478207 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7406e3e0 con 0x7fef740afd90 2023-10-10T20:25:01.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef72d9d700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740afd90 0x7fef740c9c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fef7406d3f0 tx=0x7fef64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1524440e2b4f2d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef7259c700 1 -- 172.21.15.148:0/998478207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400b860 con 0x7fef740afd90 2023-10-10T20:25:01.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef7259c700 1 -- 172.21.15.148:0/998478207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400b9c0 con 0x7fef740afd90 2023-10-10T20:25:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.068+0000 7fef7259c700 1 -- 172.21.15.148:0/998478207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400bd60 con 0x7fef740afd90 2023-10-10T20:25:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 -- 172.21.15.148:0/998478207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740afd90 msgr2=0x7fef740c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740afd90 0x7fef740c9c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fef7406d3f0 tx=0x7fef64009250 comp rx=0 tx=0).stop 2023-10-10T20:25:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 -- 172.21.15.148:0/998478207 shutdown_connections 2023-10-10T20:25:01.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740b0160 0x7fef74070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 --2- 172.21.15.148:0/998478207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740afd90 0x7fef740c9c20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 --2- 172.21.15.148:0/998478207 >> v2:172.21.15.161:3300/0 conn(0x7fef7406f1a0 0x7fef7415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 -- 172.21.15.148:0/998478207 >> 172.21.15.148:0/998478207 conn(0x7fef740ca4f0 msgr2=0x7fef740ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 -- 172.21.15.148:0/998478207 shutdown_connections 2023-10-10T20:25:01.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.069+0000 7fef791ce700 1 -- 172.21.15.148:0/998478207 wait complete. 2023-10-10T20:25:01.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.070+0000 7fef791ce700 1 Processor -- start 2023-10-10T20:25:01.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.070+0000 7fef791ce700 1 -- start start 2023-10-10T20:25:01.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.070+0000 7fef791ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7406f1a0 0x7fef740b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.070+0000 7fef791ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740afd90 0x7fef741583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.070+0000 7fef791ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef791ce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef74158910 con 0x7fef740b0160 2023-10-10T20:25:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef791ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef74158b70 con 0x7fef7406f1a0 2023-10-10T20:25:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef791ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef74158e50 con 0x7fef740afd90 2023-10-10T20:25:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37138/0 (socket says 172.21.15.148:37138) 2023-10-10T20:25:01.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740afd90 0x7fef741583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 -- 172.21.15.148:0/3786105197 learned_addr learned my addr 172.21.15.148:0/3786105197 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef72d9d700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7406f1a0 0x7fef740b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 -- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740afd90 msgr2=0x7fef741583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740afd90 0x7fef741583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 -- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7406f1a0 msgr2=0x7fef740b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7406f1a0 0x7fef740b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 -- 172.21.15.148:0/3786105197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640094f0 con 0x7fef740b0160 2023-10-10T20:25:01.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef7359e700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009fa0 tx=0x7fef6c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00ebc0 con 0x7fef740b0160 2023-10-10T20:25:01.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c00ed20 con 0x7fef740b0160 2023-10-10T20:25:01.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef74159130 con 0x7fef740b0160 2023-10-10T20:25:01.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c014670 con 0x7fef740b0160 2023-10-10T20:25:01.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.071+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef741596f0 con 0x7fef740b0160 2023-10-10T20:25:01.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.072+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef74159910 con 0x7fef740b0160 2023-10-10T20:25:01.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.073+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef6c0149e0 con 0x7fef740b0160 2023-10-10T20:25:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.073+0000 7fef6affd700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.073+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef6c084bc0 con 0x7fef740b0160 2023-10-10T20:25:01.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.073+0000 7fef72d9d700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.073+0000 7fef72d9d700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 0x7fef540693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fef7406d3f0 tx=0x7fef64017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.077+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef6c051e30 con 0x7fef740b0160 2023-10-10T20:25:01.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.301+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7404c3c0 con 0x7fef740b0160 2023-10-10T20:25:01.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.301+0000 7fef6affd700 1 -- 172.21.15.148:0/3786105197 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fef6c055450 con 0x7fef740b0160 2023-10-10T20:25:01.304 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:01.304 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:01.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 msgr2=0x7fef540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 0x7fef540693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fef7406d3f0 tx=0x7fef64017000 comp rx=0 tx=0).stop 2023-10-10T20:25:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 msgr2=0x7fef74154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009fa0 tx=0x7fef6c009900 comp rx=0 tx=0).stop 2023-10-10T20:25:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 shutdown_connections 2023-10-10T20:25:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef54066f30 0x7fef540693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef740b0160 0x7fef74154f10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7406f1a0 0x7fef740b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 --2- 172.21.15.148:0/3786105197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef740afd90 0x7fef741583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 >> 172.21.15.148:0/3786105197 conn(0x7fef740ca4f0 msgr2=0x7fef7415b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 shutdown_connections 2023-10-10T20:25:01.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.304+0000 7fef791ce700 1 -- 172.21.15.148:0/3786105197 wait complete. 2023-10-10T20:25:01.309 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:01.532 INFO:tasks.ceph.mds.a.smithi148.stderr:2023-10-10T20:25:01.530+0000 7fcffe5ef700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:25:01.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:24:55.466159+0000', 'modified': '2023-10-10T20:24:59.099410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6738': {'gid': 6738, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6836/746351743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 746351743}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 746351743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:25:01.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:25:01.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:25:01.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.837+0000 7f435a2d3700 1 Processor -- start 2023-10-10T20:25:01.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f435a2d3700 1 -- start start 2023-10-10T20:25:01.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f435a2d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f43540ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f435a2d3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540c5d80 con 0x7f435415a2b0 2023-10-10T20:25:01.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f435a2d3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540c6060 con 0x7f435415a680 2023-10-10T20:25:01.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f435a2d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4354068470 con 0x7f43540d1c30 2023-10-10T20:25:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f4353fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f435415a2b0 0x7f43540b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40540/0 (socket says 172.21.15.148:40540) 2023-10-10T20:25:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.838+0000 7f4353fff700 1 -- 172.21.15.148:0/3975287797 learned_addr learned my addr 172.21.15.148:0/3975287797 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f43537fe700 1 --2- 172.21.15.148:0/3975287797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f43540ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058226204 0 0) 0x7f43540c5d80 con 0x7f435415a2b0 2023-10-10T20:25:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f433c003570 con 0x7f435415a2b0 2023-10-10T20:25:01.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959786918 0 0) 0x7f43540c6060 con 0x7f435415a680 2023-10-10T20:25:01.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43540c5d80 con 0x7f435415a680 2023-10-10T20:25:01.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f43537fe700 1 -- 172.21.15.148:0/3975287797 >> v1:172.21.15.161:6789/0 conn(0x7f435415a680 legacy=0x7f43540cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f43537fe700 1 -- 172.21.15.148:0/3975287797 >> v1:172.21.15.148:6789/0 conn(0x7f435415a2b0 legacy=0x7f43540b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f43537fe700 1 -- 172.21.15.148:0/3975287797 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540c6340 con 0x7f43540d1c30 2023-10-10T20:25:01.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f43537fe700 1 --2- 172.21.15.148:0/3975287797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f43540ccdd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4348009910 tx=0x7f43480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919647b47c24788e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348011630 con 0x7f43540d1c30 2023-10-10T20:25:01.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348011790 con 0x7f43540d1c30 2023-10-10T20:25:01.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.839+0000 7f4352ffd700 1 -- 172.21.15.148:0/3975287797 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348011b10 con 0x7f43540d1c30 2023-10-10T20:25:01.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 -- 172.21.15.148:0/3975287797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 msgr2=0x7f43540ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 --2- 172.21.15.148:0/3975287797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f43540ccdd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4348009910 tx=0x7f43480091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 -- 172.21.15.148:0/3975287797 shutdown_connections 2023-10-10T20:25:01.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 --2- 172.21.15.148:0/3975287797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f43540ccdd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 -- 172.21.15.148:0/3975287797 >> 172.21.15.148:0/3975287797 conn(0x7f4354067e40 msgr2=0x7f43540d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 -- 172.21.15.148:0/3975287797 shutdown_connections 2023-10-10T20:25:01.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.840+0000 7f435a2d3700 1 -- 172.21.15.148:0/3975287797 wait complete. 2023-10-10T20:25:01.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.841+0000 7f435a2d3700 1 Processor -- start 2023-10-10T20:25:01.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.841+0000 7f435a2d3700 1 -- start start 2023-10-10T20:25:01.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f435415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 0x7f43540c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c90a0 con 0x7f435415a2b0 2023-10-10T20:25:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540ca080 con 0x7f435415a680 2023-10-10T20:25:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f435a2d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435415e560 con 0x7f43540d1c30 2023-10-10T20:25:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37140/0 (socket says 172.21.15.148:37140) 2023-10-10T20:25:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 -- 172.21.15.148:0/866193707 learned_addr learned my addr 172.21.15.148:0/866193707 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f4353fff700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f435415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f4358870700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 0x7f43540c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 -- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 msgr2=0x7f435415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f435415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 -- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 msgr2=0x7f43540c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 0x7f43540c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 -- 172.21.15.148:0/866193707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348009420 con 0x7f435415a2b0 2023-10-10T20:25:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f4353fff700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f435415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f4358870700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 0x7f43540c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f43537fe700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4348013000 tx=0x7f4348017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.842+0000 7f433bfff700 1 -- 172.21.15.148:0/866193707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348011900 con 0x7f435415a2b0 2023-10-10T20:25:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.843+0000 7f433bfff700 1 -- 172.21.15.148:0/866193707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348011a80 con 0x7f435415a2b0 2023-10-10T20:25:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.843+0000 7f433bfff700 1 -- 172.21.15.148:0/866193707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348025670 con 0x7f435415a2b0 2023-10-10T20:25:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.843+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4354148fb0 con 0x7f435415a2b0 2023-10-10T20:25:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.843+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540c8dc0 con 0x7f435415a2b0 2023-10-10T20:25:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.844+0000 7f433bfff700 1 -- 172.21.15.148:0/866193707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4348023070 con 0x7f435415a2b0 2023-10-10T20:25:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.844+0000 7f433bfff700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 0x7f43400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.845+0000 7f4353fff700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 0x7f43400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.845+0000 7f433bfff700 1 -- 172.21.15.148:0/866193707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4348085610 con 0x7f435415a2b0 2023-10-10T20:25:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.845+0000 7f4353fff700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 0x7f43400693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f435409ee80 tx=0x7f4344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.846+0000 7f43635ccb80 1 Processor -- start 2023-10-10T20:25:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.846+0000 7f43635ccb80 1 -- start start 2023-10-10T20:25:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9f8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9f88cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9ee3a70 con 0x5635e9fd6d00 2023-10-10T20:25:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9ee1270 con 0x5635e9f8ca50 2023-10-10T20:25:01.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e9edd060 con 0x5635e9fb5f60 2023-10-10T20:25:01.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43537fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37144/0 (socket says 172.21.15.148:37144) 2023-10-10T20:25:01.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f43537fe700 1 -- 172.21.15.148:0/1535197720 learned_addr learned my addr 172.21.15.148:0/1535197720 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f4358870700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9f88cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.847+0000 7f4353fff700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9f8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 -- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 msgr2=0x5635e9f88cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9f88cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 -- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 msgr2=0x5635e9f8ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9f8ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 -- 172.21.15.148:0/1535197720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e9d77270 con 0x5635e9fd6d00 2023-10-10T20:25:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f4353fff700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9f8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f4358870700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9f88cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43537fe700 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4348000e50 tx=0x7f43480839f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780d1e626a58f35c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43397fa700 1 -- 172.21.15.148:0/1535197720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480596f0 con 0x5635e9fd6d00 2023-10-10T20:25:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43397fa700 1 -- 172.21.15.148:0/1535197720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434805d440 con 0x5635e9fd6d00 2023-10-10T20:25:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43397fa700 1 -- 172.21.15.148:0/1535197720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480548f0 con 0x5635e9fd6d00 2023-10-10T20:25:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43635ccb80 1 -- 172.21.15.148:0/1535197720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 msgr2=0x5635e9fb5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.848+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4348000e50 tx=0x7f43480839f0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 -- 172.21.15.148:0/1535197720 shutdown_connections 2023-10-10T20:25:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9f88cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9f8ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1535197720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fb5a20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 -- 172.21.15.148:0/1535197720 >> 172.21.15.148:0/1535197720 conn(0x5635e9f8e370 msgr2=0x5635e9f8e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 -- 172.21.15.148:0/1535197720 shutdown_connections 2023-10-10T20:25:01.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 -- 172.21.15.148:0/1535197720 wait complete. 2023-10-10T20:25:01.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 -- start start 2023-10-10T20:25:01.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.849+0000 7f43635ccb80 1 Processor -- start 2023-10-10T20:25:01.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9fd6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fd38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9e88890 con 0x5635e9fd6d00 2023-10-10T20:25:01.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9ea3ea0 con 0x5635e9f8ca50 2023-10-10T20:25:01.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e9d5fe30 con 0x5635e9fb5f60 2023-10-10T20:25:01.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43635ccb80 10 client.0 _ll_register_callbacks cb 0x5635e9f50110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:25:01.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37146/0 (socket says 172.21.15.148:37146) 2023-10-10T20:25:01.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 -- 172.21.15.148:0/2081464558 learned_addr learned my addr 172.21.15.148:0/2081464558 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f43537fe700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fd38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4353fff700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9fd6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 -- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 msgr2=0x5635e9fd38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fd38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 -- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 msgr2=0x5635e9fd6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9fd6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.850+0000 7f4358870700 1 -- 172.21.15.148:0/2081464558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540c8dc0 con 0x5635e9fd6d00 2023-10-10T20:25:01.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f4358870700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f434c001fc0 tx=0x7f434c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e2d188eccd0c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:25:01.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c014400 con 0x5635e9fd6d00 2023-10-10T20:25:01.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f434c014560 con 0x5635e9fd6d00 2023-10-10T20:25:01.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e9f81b70 con 0x5635e9fd6d00 2023-10-10T20:25:01.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434c014930 con 0x5635e9fd6d00 2023-10-10T20:25:01.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 10 client.6882 fetch_fsmap learned FSMap version 57 2023-10-10T20:25:01.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5635e9d19fe0 con 0x5635e9fd6d00 2023-10-10T20:25:01.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5635e9f81b70 con 0x5635e9fd6d00 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f434c014a90 con 0x5635e9fd6d00 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 10 client.6882 fetch_fsmap finished waiting for FSMap version 57 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 10 client.6882 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 20 client.6882 populate_metadata read hostname 'smithi148' 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fb4560 0x5635e9fb4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 4 client.6882 mds_command: new command op to 6738 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:25:01.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 --> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e9f5eaa0 con 0x5635e9fb4560 2023-10-10T20:25:01.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.851+0000 7f4353fff700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fb4560 0x5635e9fb4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43327fc700 10 client.6882 ms_handle_connect on v2:172.21.15.148:6834/746351743 2023-10-10T20:25:01.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f4353fff700 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fb4560 0x5635e9fb4930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c352cfddd655a02 server_cookie=519268e70001e20b in_seq=0 out_seq=0 2023-10-10T20:25:01.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43327fc700 1 -- 172.21.15.148:0/2081464558 <== mds.0 v2:172.21.15.148:6834/746351743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5635e9f5eaa0 con 0x5635e9fb4560 2023-10-10T20:25:01.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43327fc700 10 client.6882 handle_command_reply: tid=0 2023-10-10T20:25:01.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 client.6882 shutdown 2023-10-10T20:25:01.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 msgr2=0x5635e9fcec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f434c001fc0 tx=0x7f434c009040 comp rx=0 tx=0).stop 2023-10-10T20:25:01.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 shutdown_connections 2023-10-10T20:25:01.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fb4560 0x5635e9fb4930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9fd6d00 0x5635e9fcec50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43327fc700 0 client.6882 ms_handle_reset on v2:172.21.15.148:6834/746351743 2023-10-10T20:25:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9f8ca50 0x5635e9fd6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 --2- 172.21.15.148:0/2081464558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fd38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.852+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 >> 172.21.15.148:0/2081464558 conn(0x5635e9f8e370 msgr2=0x5635e9fd2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.853+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 shutdown_connections 2023-10-10T20:25:01.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.853+0000 7f43635ccb80 1 -- 172.21.15.148:0/2081464558 wait complete. 2023-10-10T20:25:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.853+0000 7f43635ccb80 20 client.6882 trim_cache size 0 max 16384 2023-10-10T20:25:01.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 Processor -- start 2023-10-10T20:25:01.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 -- start start 2023-10-10T20:25:01.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fe21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 0x5635e9fe2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9f87610 con 0x5635e9f8ca50 2023-10-10T20:25:01.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9fcde30 con 0x5635e9fd6d00 2023-10-10T20:25:01.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.870+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e9fcaa50 con 0x5635e9fb5f60 2023-10-10T20:25:01.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37156/0 (socket says 172.21.15.148:37156) 2023-10-10T20:25:01.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 -- 172.21.15.148:0/3117073575 learned_addr learned my addr 172.21.15.148:0/3117073575 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4358870700 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 0x5635e9fe2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 -- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 msgr2=0x5635e9fe21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fe21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 -- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 msgr2=0x5635e9fe2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 0x5635e9fe2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 -- 172.21.15.148:0/3117073575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e9eab780 con 0x5635e9f8ca50 2023-10-10T20:25:01.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4358870700 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 0x5635e9fe2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:01.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4353fff700 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4344009010 tx=0x7f4344015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc00045ada3ea156 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4339ffb700 1 -- 172.21.15.148:0/3117073575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400e5d0 con 0x5635e9f8ca50 2023-10-10T20:25:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4339ffb700 1 -- 172.21.15.148:0/3117073575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400e730 con 0x5635e9f8ca50 2023-10-10T20:25:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.871+0000 7f4339ffb700 1 -- 172.21.15.148:0/3117073575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400e8a0 con 0x5635e9f8ca50 2023-10-10T20:25:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- 172.21.15.148:0/3117073575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 msgr2=0x5635e9fb3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4344009010 tx=0x7f4344015040 comp rx=0 tx=0).stop 2023-10-10T20:25:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- 172.21.15.148:0/3117073575 shutdown_connections 2023-10-10T20:25:01.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fd6d00 0x5635e9fe2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fb3e30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 --2- 172.21.15.148:0/3117073575 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fb5f60 0x5635e9fe21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- 172.21.15.148:0/3117073575 >> 172.21.15.148:0/3117073575 conn(0x5635e9f8e370 msgr2=0x5635e9fbc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- 172.21.15.148:0/3117073575 shutdown_connections 2023-10-10T20:25:01.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- 172.21.15.148:0/3117073575 wait complete. 2023-10-10T20:25:01.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 -- start start 2023-10-10T20:25:01.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.872+0000 7f43635ccb80 1 Processor -- start 2023-10-10T20:25:01.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fb5f60 0x5635e9fcd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 0x5635e9fe4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9fbc360 con 0x5635e9f8ca50 2023-10-10T20:25:01.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ea03a110 con 0x5635e9fb5f60 2023-10-10T20:25:01.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e9fbc4d0 con 0x5635e9fd6d00 2023-10-10T20:25:01.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43635ccb80 10 client.0 _ll_register_callbacks cb 0x5635e9f50110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:25:01.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f4353fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f4353fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37166/0 (socket says 172.21.15.148:37166) 2023-10-10T20:25:01.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f4353fff700 1 -- 172.21.15.148:0/1453165907 learned_addr learned my addr 172.21.15.148:0/1453165907 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:01.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f43537fe700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fb5f60 0x5635e9fcd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.873+0000 7f4358870700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 0x5635e9fe4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 -- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 msgr2=0x5635e9fe4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 0x5635e9fe4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 -- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fb5f60 msgr2=0x5635e9fcd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:01.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fb5f60 0x5635e9fcd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 -- 172.21.15.148:0/1453165907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434401d040 con 0x5635e9f8ca50 2023-10-10T20:25:01.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4358870700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 0x5635e9fe4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:01.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f4353fff700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4344009010 tx=0x7f4344017030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb20627cc9c58e55 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:01.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:25:01.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344034070 con 0x5635e9f8ca50 2023-10-10T20:25:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4344025540 con 0x5635e9f8ca50 2023-10-10T20:25:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f434805d440 con 0x5635e9f8ca50 2023-10-10T20:25:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434400eea0 con 0x5635e9f8ca50 2023-10-10T20:25:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 10 client.6888 fetch_fsmap learned FSMap version 57 2023-10-10T20:25:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5635e9e92470 con 0x5635e9f8ca50 2023-10-10T20:25:01.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434805d440 con 0x5635e9f8ca50 2023-10-10T20:25:01.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4344025910 con 0x5635e9f8ca50 2023-10-10T20:25:01.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 10 client.6888 fetch_fsmap finished waiting for FSMap version 57 2023-10-10T20:25:01.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 10 client.6888 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:25:01.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 20 client.6888 populate_metadata read hostname 'smithi148' 2023-10-10T20:25:01.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fe5540 0x5635ea038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:01.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 4 client.6888 mds_command: new command op to 6738 tid=0[{"prefix": "status"}] 2023-10-10T20:25:01.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 --> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5635e9e7fe00 con 0x5635e9fe5540 2023-10-10T20:25:01.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.874+0000 7f43537fe700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fe5540 0x5635ea038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:01.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43327fc700 10 client.6888 ms_handle_connect on v2:172.21.15.148:6834/746351743 2023-10-10T20:25:01.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43537fe700 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fe5540 0x5635ea038bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18e3af31b4b2d27 server_cookie=1fc955a837085e77 in_seq=0 out_seq=0 2023-10-10T20:25:01.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43327fc700 1 -- 172.21.15.148:0/1453165907 <== mds.0 v2:172.21.15.148:6834/746351743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5635e9e7fe00 con 0x5635e9fe5540 2023-10-10T20:25:01.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43327fc700 10 client.6888 handle_command_reply: tid=0 2023-10-10T20:25:01.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 client.6888 shutdown 2023-10-10T20:25:01.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 msgr2=0x5635e9fccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4344009010 tx=0x7f4344017030 comp rx=0 tx=0).stop 2023-10-10T20:25:01.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 shutdown_connections 2023-10-10T20:25:01.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:6834/746351743,v1:172.21.15.148:6836/746351743] conn(0x5635e9fe5540 0x5635ea038bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x5635e9fd6d00 0x5635e9fe4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43327fc700 0 client.6888 ms_handle_reset on v2:172.21.15.148:6834/746351743 2023-10-10T20:25:01.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635e9f8ca50 0x5635e9fccc30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 --2- 172.21.15.148:0/1453165907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5635e9fb5f60 0x5635e9fcd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 >> 172.21.15.148:0/1453165907 conn(0x5635e9f8e370 msgr2=0x5635e9fd64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 shutdown_connections 2023-10-10T20:25:01.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 1 -- 172.21.15.148:0/1453165907 wait complete. 2023-10-10T20:25:01.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.875+0000 7f43635ccb80 20 client.6888 trim_cache size 0 max 16384 2023-10-10T20:25:01.905 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:25:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 6738, 2023-10-10T20:25:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:25:01.906 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:25:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:25:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.7960523950000002, 2023-10-10T20:25:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 57, 2023-10-10T20:25:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 116, 2023-10-10T20:25:01.907 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 116, 2023-10-10T20:25:01.908 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.912858573999999 2023-10-10T20:25:01.908 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:01.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 msgr2=0x7f43400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 0x7f43400693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f435409ee80 tx=0x7f4344009040 comp rx=0 tx=0).stop 2023-10-10T20:25:01.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 msgr2=0x7f4354148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:01.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4348013000 tx=0x7f4348017000 comp rx=0 tx=0).stop 2023-10-10T20:25:01.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 shutdown_connections 2023-10-10T20:25:01.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4340066f30 0x7f43400693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f435415a680 0x7f43540c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43540d1c30 0x7f435415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 --2- 172.21.15.148:0/866193707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f435415a2b0 0x7f4354148a70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:01.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 >> 172.21.15.148:0/866193707 conn(0x7f4354067e40 msgr2=0x7f4354069550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:01.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 shutdown_connections 2023-10-10T20:25:01.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:01.876+0000 7f435a2d3700 1 -- 172.21.15.148:0/866193707 wait complete. 2023-10-10T20:25:01.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:25:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.025+0000 7f2499d9e700 1 Processor -- start 2023-10-10T20:25:02.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.025+0000 7f2499d9e700 1 -- start start 2023-10-10T20:25:02.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2499d9e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24940cd6e0 0x7f24940cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2499d9e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f24940cdab0 0x7f249415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2499d9e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24940cb7c0 con 0x7f249415dda0 2023-10-10T20:25:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2499d9e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249415e850 con 0x7f24940cd6e0 2023-10-10T20:25:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2499d9e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940cd030 con 0x7f24940cdab0 2023-10-10T20:25:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2498d9c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24940cd6e0 0x7f24940cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2498d9c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24940cd6e0 0x7f24940cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37176/0 (socket says 172.21.15.148:37176) 2023-10-10T20:25:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.026+0000 7f2498d9c700 1 -- 172.21.15.148:0/1116125040 learned_addr learned my addr 172.21.15.148:0/1116125040 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:02.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2498d9c700 1 -- 172.21.15.148:0/1116125040 >> v2:172.21.15.161:3300/0 conn(0x7f24940cdab0 msgr2=0x7f249415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2498d9c700 1 --2- 172.21.15.148:0/1116125040 >> v2:172.21.15.161:3300/0 conn(0x7f24940cdab0 0x7f249415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2498d9c700 1 -- 172.21.15.148:0/1116125040 >> v1:172.21.15.161:6789/0 conn(0x7f249415dda0 legacy=0x7f249415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2498d9c700 1 -- 172.21.15.148:0/1116125040 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249415e290 con 0x7f24940cd6e0 2023-10-10T20:25:02.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2498d9c700 1 --2- 172.21.15.148:0/1116125040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cd6e0 0x7f24940cbc50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f24940b6c70 tx=0x7f2490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b1e803b729e8d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f248b7fe700 1 -- 172.21.15.148:0/1116125040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000b820 con 0x7f24940cd6e0 2023-10-10T20:25:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f248b7fe700 1 -- 172.21.15.148:0/1116125040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249000b980 con 0x7f24940cd6e0 2023-10-10T20:25:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f248b7fe700 1 -- 172.21.15.148:0/1116125040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000bd20 con 0x7f24940cd6e0 2023-10-10T20:25:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 -- 172.21.15.148:0/1116125040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cd6e0 msgr2=0x7f24940cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 --2- 172.21.15.148:0/1116125040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cd6e0 0x7f24940cbc50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f24940b6c70 tx=0x7f2490009250 comp rx=0 tx=0).stop 2023-10-10T20:25:02.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 -- 172.21.15.148:0/1116125040 shutdown_connections 2023-10-10T20:25:02.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 --2- 172.21.15.148:0/1116125040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cd6e0 0x7f24940cbc50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 --2- 172.21.15.148:0/1116125040 >> v2:172.21.15.161:3300/0 conn(0x7f24940cdab0 0x7f249415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 -- 172.21.15.148:0/1116125040 >> 172.21.15.148:0/1116125040 conn(0x7f2494067660 msgr2=0x7f2494067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:02.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 -- 172.21.15.148:0/1116125040 shutdown_connections 2023-10-10T20:25:02.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.027+0000 7f2499d9e700 1 -- 172.21.15.148:0/1116125040 wait complete. 2023-10-10T20:25:02.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.028+0000 7f2499d9e700 1 Processor -- start 2023-10-10T20:25:02.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 -- start start 2023-10-10T20:25:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24940cd6e0 0x7f24940c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f249415dda0 0x7f249414cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494151490 con 0x7f24940cdab0 2023-10-10T20:25:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249414d260 con 0x7f24940cd6e0 2023-10-10T20:25:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.029+0000 7f2499d9e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24940c9b10 con 0x7f249415dda0 2023-10-10T20:25:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37180/0 (socket says 172.21.15.148:37180) 2023-10-10T20:25:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 -- 172.21.15.148:0/3658322213 learned_addr learned my addr 172.21.15.148:0/3658322213 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2498d9c700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24940cd6e0 0x7f24940c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:02.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f249959d700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f249415dda0 0x7f249414cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 -- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f249415dda0 msgr2=0x7f249414cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f249415dda0 0x7f249414cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 -- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24940cd6e0 msgr2=0x7f24940c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24940cd6e0 0x7f24940c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 -- 172.21.15.148:0/3658322213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900094f0 con 0x7f24940cdab0 2023-10-10T20:25:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f248bfff700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f248400b810 tx=0x7f2484009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400f510 con 0x7f24940cdab0 2023-10-10T20:25:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248400f690 con 0x7f24940cdab0 2023-10-10T20:25:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24940c9df0 con 0x7f24940cdab0 2023-10-10T20:25:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400fa50 con 0x7f24940cdab0 2023-10-10T20:25:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.030+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2494151730 con 0x7f24940cdab0 2023-10-10T20:25:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.031+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2494151920 con 0x7f24940cdab0 2023-10-10T20:25:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.032+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f248401a400 con 0x7f24940cdab0 2023-10-10T20:25:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.032+0000 7f2488ff9700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.032+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2484084000 con 0x7f24940cdab0 2023-10-10T20:25:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.032+0000 7f2498d9c700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.032+0000 7f2498d9c700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f24940b6c70 tx=0x7f2490016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.036+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24840512f0 con 0x7f24940cdab0 2023-10-10T20:25:02.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.260+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f249415e570 con 0x7f24940cdab0 2023-10-10T20:25:02.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.260+0000 7f2488ff9700 1 -- 172.21.15.148:0/3658322213 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2484054910 con 0x7f24940cdab0 2023-10-10T20:25:02.263 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:25:02.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 msgr2=0x7f24800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:02.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f24940b6c70 tx=0x7f2490016000 comp rx=0 tx=0).stop 2023-10-10T20:25:02.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 msgr2=0x7f24940b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:02.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f248400b810 tx=0x7f2484009250 comp rx=0 tx=0).stop 2023-10-10T20:25:02.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 shutdown_connections 2023-10-10T20:25:02.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f249415dda0 0x7f249414cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24940cd6e0 0x7f24940c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 --2- 172.21.15.148:0/3658322213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940cdab0 0x7f24940b4420 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:02.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 >> 172.21.15.148:0/3658322213 conn(0x7f2494067660 msgr2=0x7f24940c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:02.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 shutdown_connections 2023-10-10T20:25:02.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:02.262+0000 7f2499d9e700 1 -- 172.21.15.148:0/3658322213 wait complete. 2023-10-10T20:25:02.277 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:25:02.277 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:25:02.277 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:25:02.278 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:25:02.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:02.279 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:25:02.279 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:25:02.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:25:02.280 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:02.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.280 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:25:02.296 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:25:02.296 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:25:02.297 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:25:02.297 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.297 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:25:02.297 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.297 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:02.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:02.298 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:25:02.298 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:02.298 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:02.298 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:25:02.299 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:02.299 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:02.299 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:25:02.299 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:02.299 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:25:02.300 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42209sec preferred_lft 42209sec 2023-10-10T20:25:02.300 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:25:02.300 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.300 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:25:02.300 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:02.301 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:25:02.301 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.301 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:25:02.301 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.301 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:02.302 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:25:02.302 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:25:02.302 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.302 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:02.302 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:25:02.303 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:25:02.303 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.304 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:25:02.361 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:25:02.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.363 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:25:02.419 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:25:02.420 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:25:02.420 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:25:02.420 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:25:02.421 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:25:02.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.422 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:25:02.423 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:25:02.423 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:25:02.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:25:02.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:25:02.424 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:25:02.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.520 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:25:02.535 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:25:02.535 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:25:02.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:02.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:25:02.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:02.600 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:02.654 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:02.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:02.655 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:02.709 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:25:02.710 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:25:02.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:02.790 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:02.791 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:02.809 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:02.810 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:02.810 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:02.810 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:02.810 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:02.810 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:02.811 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:02.811 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:02.831 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:02.831 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:02.831 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:02.832 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:02.833 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:25:02.833 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:25:02.833 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:25:02.834 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:25:02.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:02.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:02.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:25:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.051+0000 7f32deef3700 1 Processor -- start 2023-10-10T20:25:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.051+0000 7f32deef3700 1 -- start start 2023-10-10T20:25:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32d8157870 0x7f32d815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d815a620 0x7f32d815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d815cbb0 con 0x7f32d815a620 2023-10-10T20:25:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d815bfa0 con 0x7f32d8157870 2023-10-10T20:25:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32deef3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d815c280 con 0x7f32d815d580 2023-10-10T20:25:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d815a620 0x7f32d815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d815a620 0x7f32d815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52910/0 (socket says 172.21.15.148:52910) 2023-10-10T20:25:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 -- 172.21.15.148:0/3090130977 learned_addr learned my addr 172.21.15.148:0/3090130977 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32dcc8f700 1 --2- 172.21.15.148:0/3090130977 >> v2:172.21.15.161:3300/0 conn(0x7f32d8157870 0x7f32d815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32dd490700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 -- 172.21.15.148:0/3090130977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 msgr2=0x7f32d8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 -- 172.21.15.148:0/3090130977 >> v2:172.21.15.161:3300/0 conn(0x7f32d8157870 msgr2=0x7f32d815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 --2- 172.21.15.148:0/3090130977 >> v2:172.21.15.161:3300/0 conn(0x7f32d8157870 0x7f32d815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 -- 172.21.15.148:0/3090130977 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d815c560 con 0x7f32d815a620 2023-10-10T20:25:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.052+0000 7f32d7fff700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d815d040 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f32d8069d80 tx=0x7f32c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdf6f2d6e7c33fc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32d77fe700 1 -- 172.21.15.148:0/3090130977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800e070 con 0x7f32d815a620 2023-10-10T20:25:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32d77fe700 1 -- 172.21.15.148:0/3090130977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800b970 con 0x7f32d815a620 2023-10-10T20:25:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32d77fe700 1 -- 172.21.15.148:0/3090130977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80092e0 con 0x7f32d815a620 2023-10-10T20:25:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 -- 172.21.15.148:0/3090130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 msgr2=0x7f32d815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d815d040 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f32d8069d80 tx=0x7f32c8009840 comp rx=0 tx=0).stop 2023-10-10T20:25:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 -- 172.21.15.148:0/3090130977 shutdown_connections 2023-10-10T20:25:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 --2- 172.21.15.148:0/3090130977 >> v2:172.21.15.161:3300/0 conn(0x7f32d8157870 0x7f32d815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 --2- 172.21.15.148:0/3090130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d815d040 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 -- 172.21.15.148:0/3090130977 >> 172.21.15.148:0/3090130977 conn(0x7f32d8068ce0 msgr2=0x7f32d80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 -- 172.21.15.148:0/3090130977 shutdown_connections 2023-10-10T20:25:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.053+0000 7f32deef3700 1 -- 172.21.15.148:0/3090130977 wait complete. 2023-10-10T20:25:03.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.054+0000 7f32deef3700 1 Processor -- start 2023-10-10T20:25:03.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.054+0000 7f32deef3700 1 -- start start 2023-10-10T20:25:03.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8157870 0x7f32d8154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8155470 con 0x7f32d815a620 2023-10-10T20:25:03.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80ca8c0 con 0x7f32d8157870 2023-10-10T20:25:03.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32deef3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d816de00 con 0x7f32d815d580 2023-10-10T20:25:03.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52926/0 (socket says 172.21.15.148:52926) 2023-10-10T20:25:03.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32d7fff700 1 -- 172.21.15.148:0/358324789 learned_addr learned my addr 172.21.15.148:0/358324789 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:03.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32dcc8f700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8157870 0x7f32d8154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.055+0000 7f32dd490700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 -- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 msgr2=0x7f32d80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:03.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d80b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 -- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8157870 msgr2=0x7f32d8154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:03.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8157870 0x7f32d8154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 -- 172.21.15.148:0/358324789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8009ae0 con 0x7f32d815a620 2023-10-10T20:25:03.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d7fff700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f32c8012000 tx=0x7f32c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:03.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80087f0 con 0x7f32d815a620 2023-10-10T20:25:03.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8008950 con 0x7f32d815a620 2023-10-10T20:25:03.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d816e0e0 con 0x7f32d815a620 2023-10-10T20:25:03.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8008d10 con 0x7f32d815a620 2023-10-10T20:25:03.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.056+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d816e6a0 con 0x7f32d815a620 2023-10-10T20:25:03.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.057+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d804c3c0 con 0x7f32d815a620 2023-10-10T20:25:03.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.057+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32c8003040 con 0x7f32d815a620 2023-10-10T20:25:03.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.058+0000 7f32d4ff9700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 0x7f32b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:03.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.058+0000 7f32dcc8f700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 0x7f32b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:03.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.058+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32c8085b20 con 0x7f32d815a620 2023-10-10T20:25:03.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.058+0000 7f32dcc8f700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 0x7f32b80693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f32d80b34b0 tx=0x7f32d00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:03.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.062+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c8052d90 con 0x7f32d815a620 2023-10-10T20:25:03.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.291+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32d815c840 con 0x7f32d815a620 2023-10-10T20:25:03.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.291+0000 7f32d4ff9700 1 -- 172.21.15.148:0/358324789 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32c80563b0 con 0x7f32d815a620 2023-10-10T20:25:03.294 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:25:03.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 msgr2=0x7f32b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:03.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 0x7f32b80693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f32d80b34b0 tx=0x7f32d00091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 msgr2=0x7f32d80ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:03.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f32c8012000 tx=0x7f32c8018000 comp rx=0 tx=0).stop 2023-10-10T20:25:03.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 shutdown_connections 2023-10-10T20:25:03.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f32b8066f30 0x7f32b80693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32d815d580 0x7f32d80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32d8157870 0x7f32d8154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 --2- 172.21.15.148:0/358324789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d815a620 0x7f32d80ca380 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:03.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 >> 172.21.15.148:0/358324789 conn(0x7f32d8068ce0 msgr2=0x7f32d80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:03.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 shutdown_connections 2023-10-10T20:25:03.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:03.293+0000 7f32deef3700 1 -- 172.21.15.148:0/358324789 wait complete. 2023-10-10T20:25:03.307 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:25:03.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:25:03.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:25:03.308 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:25:03.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:03.309 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:25:03.309 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:25:03.309 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:25:03.310 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:03.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.311 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:25:03.329 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:25:03.329 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:25:03.329 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:25:03.330 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.330 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:25:03.330 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.331 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:03.331 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:03.331 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:25:03.332 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:03.332 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:03.332 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:25:03.332 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:03.333 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:03.333 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:25:03.333 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:03.334 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:25:03.334 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42204sec preferred_lft 42204sec 2023-10-10T20:25:03.335 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:25:03.335 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.336 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:25:03.336 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:03.336 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:25:03.337 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.337 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:25:03.337 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.338 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:03.338 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:25:03.339 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:25:03.339 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.339 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:03.340 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:25:03.340 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:25:03.341 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:03.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.342 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:25:03.398 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:25:03.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.400 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:25:03.459 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:25:03.459 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:25:03.459 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:25:03.459 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:25:03.460 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:25:03.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.460 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:25:03.461 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:25:03.461 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:25:03.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:25:03.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:25:03.462 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:25:03.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.571 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:25:03.586 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:25:03.586 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:25:03.586 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:03.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.587 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:25:03.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:03.652 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:03.707 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:03.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:03.708 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:03.762 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:25:03.763 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:25:03.763 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:03.843 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:03.844 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:03.862 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:03.863 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:03.863 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:03.863 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:03.863 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:03.863 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:03.864 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:03.864 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:03.884 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:03.885 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:03.885 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:03.885 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:03.885 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:03.885 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:25:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:25:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:25:03.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:25:03.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:03.888 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:03.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:04.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.068+0000 7f6d95293700 1 Processor -- start 2023-10-10T20:25:04.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.068+0000 7f6d95293700 1 -- start start 2023-10-10T20:25:04.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d95293700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d900d0970 0x7f6d9015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d95293700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 0x7f6d900d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d95293700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900cf0c0 con 0x7f6d90153e50 2023-10-10T20:25:04.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d95293700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9015e8f0 con 0x7f6d900d0970 2023-10-10T20:25:04.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d95293700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9015b320 con 0x7f6d9015aea0 2023-10-10T20:25:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d900d0970 0x7f6d9015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d900d0970 0x7f6d9015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52936/0 (socket says 172.21.15.148:52936) 2023-10-10T20:25:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 -- 172.21.15.148:0/3473263977 learned_addr learned my addr 172.21.15.148:0/3473263977 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8f7fe700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 0x7f6d900d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 -- 172.21.15.148:0/3473263977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 msgr2=0x7f6d900d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:04.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 0x7f6d900d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 -- 172.21.15.148:0/3473263977 >> v1:172.21.15.161:6789/0 conn(0x7f6d90153e50 legacy=0x7f6d901542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 -- 172.21.15.148:0/3473263977 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900d1280 con 0x7f6d900d0970 2023-10-10T20:25:04.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8f7fe700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 0x7f6d900d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:04.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.069+0000 7f6d8ffff700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d900d0970 0x7f6d9015b540 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6d800099b0 tx=0x7f6d80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc350a9ecbfb2b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d8effd700 1 -- 172.21.15.148:0/3473263977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000b860 con 0x7f6d900d0970 2023-10-10T20:25:04.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d8effd700 1 -- 172.21.15.148:0/3473263977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000b9c0 con 0x7f6d900d0970 2023-10-10T20:25:04.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d8effd700 1 -- 172.21.15.148:0/3473263977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000bd60 con 0x7f6d900d0970 2023-10-10T20:25:04.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 -- 172.21.15.148:0/3473263977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d900d0970 msgr2=0x7f6d9015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d900d0970 0x7f6d9015b540 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6d800099b0 tx=0x7f6d80009250 comp rx=0 tx=0).stop 2023-10-10T20:25:04.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 -- 172.21.15.148:0/3473263977 shutdown_connections 2023-10-10T20:25:04.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d900d0970 0x7f6d9015b540 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 --2- 172.21.15.148:0/3473263977 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d9015aea0 0x7f6d900d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 -- 172.21.15.148:0/3473263977 >> 172.21.15.148:0/3473263977 conn(0x7f6d90068c00 msgr2=0x7f6d90069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:04.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 -- 172.21.15.148:0/3473263977 shutdown_connections 2023-10-10T20:25:04.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.070+0000 7f6d95293700 1 -- 172.21.15.148:0/3473263977 wait complete. 2023-10-10T20:25:04.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.071+0000 7f6d95293700 1 Processor -- start 2023-10-10T20:25:04.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.071+0000 7f6d95293700 1 -- start start 2023-10-10T20:25:04.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d900d0970 0x7f6d90147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d90153e50 0x7f6d900b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90147e60 con 0x7f6d9015aea0 2023-10-10T20:25:04.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9014a120 con 0x7f6d900d0970 2023-10-10T20:25:04.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d95293700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900b1090 con 0x7f6d90153e50 2023-10-10T20:25:04.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52952/0 (socket says 172.21.15.148:52952) 2023-10-10T20:25:04.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 -- 172.21.15.148:0/2930552607 learned_addr learned my addr 172.21.15.148:0/2930552607 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:04.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d8f7fe700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d90153e50 0x7f6d900b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d8ffff700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d900d0970 0x7f6d90147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 -- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d90153e50 msgr2=0x7f6d900b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d90153e50 0x7f6d900b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 -- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d900d0970 msgr2=0x7f6d90147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d900d0970 0x7f6d90147920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.072+0000 7f6d94a92700 1 -- 172.21.15.148:0/2930552607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800094f0 con 0x7f6d9015aea0 2023-10-10T20:25:04.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d94a92700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c8e0 tx=0x7f6d88001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88012630 con 0x7f6d9015aea0 2023-10-10T20:25:04.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88012790 con 0x7f6d9015aea0 2023-10-10T20:25:04.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900b1370 con 0x7f6d9015aea0 2023-10-10T20:25:04.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88012b50 con 0x7f6d9015aea0 2023-10-10T20:25:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.073+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900b1930 con 0x7f6d9015aea0 2023-10-10T20:25:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.074+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90089290 con 0x7f6d9015aea0 2023-10-10T20:25:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.074+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d8801a3a0 con 0x7f6d9015aea0 2023-10-10T20:25:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.074+0000 7f6d77fff700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.074+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d880846a0 con 0x7f6d9015aea0 2023-10-10T20:25:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.075+0000 7f6d8ffff700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.075+0000 7f6d8ffff700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 0x7f6d7c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d800099b0 tx=0x7f6d80017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.079+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d880553a0 con 0x7f6d9015aea0 2023-10-10T20:25:04.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.303+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d9015e050 con 0x7f6d9015aea0 2023-10-10T20:25:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.303+0000 7f6d77fff700 1 -- 172.21.15.148:0/2930552607 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6d88054f30 con 0x7f6d9015aea0 2023-10-10T20:25:04.306 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:04.306 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 msgr2=0x7f6d7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 0x7f6d7c0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d800099b0 tx=0x7f6d80017000 comp rx=0 tx=0).stop 2023-10-10T20:25:04.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 msgr2=0x7f6d9015c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c8e0 tx=0x7f6d88001710 comp rx=0 tx=0).stop 2023-10-10T20:25:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 shutdown_connections 2023-10-10T20:25:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d7c066f30 0x7f6d7c0693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d9015aea0 0x7f6d9015c410 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d900d0970 0x7f6d90147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 --2- 172.21.15.148:0/2930552607 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d90153e50 0x7f6d900b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 >> 172.21.15.148:0/2930552607 conn(0x7f6d90068c00 msgr2=0x7f6d90148160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 shutdown_connections 2023-10-10T20:25:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.306+0000 7f6d95293700 1 -- 172.21.15.148:0/2930552607 wait complete. 2023-10-10T20:25:04.311 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:04.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-10T20:25:04.453 INFO:teuthology.orchestra.run.smithi148.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":6738,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3624759970000007,"mdsmap_epoch":57,"osdmap_epoch":116,"osdmap_epoch_barrier":116,"uptime":13.479284488999999} 2023-10-10T20:25:04.454 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 6738, "mdsmap_epoch": 57, "osdmap_epoch": 116, "osdmap_epoch_barrier": 116, "rank_uptime": 8.362475997, "state": "up:active", "uptime": 13.479284489, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:25:04.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-10T20:25:04.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.571+0000 7fa8fe58c700 1 Processor -- start 2023-10-10T20:25:04.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.571+0000 7fa8fe58c700 1 -- start start 2023-10-10T20:25:04.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fe58c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 0x7fa8f80b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fe58c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8f806a650 0x7fa8f806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fe58c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f815eb10 con 0x7fa8f815a1b0 2023-10-10T20:25:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fe58c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f80b3790 con 0x7fa8f806a650 2023-10-10T20:25:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fe58c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f806b010 con 0x7fa8f8159de0 2023-10-10T20:25:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8f806a650 0x7fa8f806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8f806a650 0x7fa8f806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52958/0 (socket says 172.21.15.148:52958) 2023-10-10T20:25:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 -- 172.21.15.148:0/2257122611 learned_addr learned my addr 172.21.15.148:0/2257122611 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8f7fff700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 0x7fa8f80b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 -- 172.21.15.148:0/2257122611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 msgr2=0x7fa8f80b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 0x7fa8f80b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 -- 172.21.15.148:0/2257122611 >> v1:172.21.15.161:6789/0 conn(0x7fa8f815a1b0 legacy=0x7fa8f80b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8fcb29700 1 -- 172.21.15.148:0/2257122611 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f815e680 con 0x7fa8f806a650 2023-10-10T20:25:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.572+0000 7fa8f7fff700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 0x7fa8f80b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fcb29700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f806a650 0x7fa8f806aad0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009f20 tx=0x7fa8f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6540f66202dd0e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8f6ffd700 1 -- 172.21.15.148:0/2257122611 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000e070 con 0x7fa8f806a650 2023-10-10T20:25:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8f6ffd700 1 -- 172.21.15.148:0/2257122611 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f000ba80 con 0x7fa8f806a650 2023-10-10T20:25:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8f6ffd700 1 -- 172.21.15.148:0/2257122611 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00092e0 con 0x7fa8f806a650 2023-10-10T20:25:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 -- 172.21.15.148:0/2257122611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f806a650 msgr2=0x7fa8f806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f806a650 0x7fa8f806aad0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009f20 tx=0x7fa8f0009840 comp rx=0 tx=0).stop 2023-10-10T20:25:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 -- 172.21.15.148:0/2257122611 shutdown_connections 2023-10-10T20:25:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f806a650 0x7fa8f806aad0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/2257122611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f8159de0 0x7fa8f80b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 -- 172.21.15.148:0/2257122611 >> 172.21.15.148:0/2257122611 conn(0x7fa8f8068d60 msgr2=0x7fa8f8069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 -- 172.21.15.148:0/2257122611 shutdown_connections 2023-10-10T20:25:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.573+0000 7fa8fe58c700 1 -- 172.21.15.148:0/2257122611 wait complete. 2023-10-10T20:25:04.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.574+0000 7fa8fe58c700 1 Processor -- start 2023-10-10T20:25:04.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 -- start start 2023-10-10T20:25:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f806a650 0x7fa8f8146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8159de0 0x7fa8f80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80caad0 con 0x7fa8f815a1b0 2023-10-10T20:25:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80c9c20 con 0x7fa8f8159de0 2023-10-10T20:25:04.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fe58c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80c9ec0 con 0x7fa8f806a650 2023-10-10T20:25:04.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fcb29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fcb29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52972/0 (socket says 172.21.15.148:52972) 2023-10-10T20:25:04.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8fcb29700 1 -- 172.21.15.148:0/921511849 learned_addr learned my addr 172.21.15.148:0/921511849 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:04.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8f7fff700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f806a650 0x7fa8f8146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.575+0000 7fa8f77fe700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8159de0 0x7fa8f80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 -- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f806a650 msgr2=0x7fa8f8146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f806a650 0x7fa8f8146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 -- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8159de0 msgr2=0x7fa8f80c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8159de0 0x7fa8f80c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 -- 172.21.15.148:0/921511849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0009ae0 con 0x7fa8f815a1b0 2023-10-10T20:25:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fcb29700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8072050 tx=0x7fa8f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00032f0 con 0x7fa8f815a1b0 2023-10-10T20:25:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f0008d20 con 0x7fa8f815a1b0 2023-10-10T20:25:04.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f80ca1a0 con 0x7fa8f815a1b0 2023-10-10T20:25:04.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f001c660 con 0x7fa8f815a1b0 2023-10-10T20:25:04.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.576+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f80ca760 con 0x7fa8f815a1b0 2023-10-10T20:25:04.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.577+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f80c4430 con 0x7fa8f815a1b0 2023-10-10T20:25:04.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.577+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8f0003040 con 0x7fa8f815a1b0 2023-10-10T20:25:04.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.578+0000 7fa8dffff700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 0x7fa8e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:04.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.578+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8f0085cf0 con 0x7fa8f815a1b0 2023-10-10T20:25:04.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.578+0000 7fa8f7fff700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 0x7fa8e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:04.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.578+0000 7fa8f7fff700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 0x7fa8e40693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006f00 tx=0x7fa8e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:04.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.582+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8f008a070 con 0x7fa8f815a1b0 2023-10-10T20:25:04.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:04.766+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fa8f804c3c0 con 0x7fa8f815a1b0 2023-10-10T20:25:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.540+0000 7fa8dffff700 1 -- 172.21.15.148:0/921511849 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8f0052f60 con 0x7fa8f815a1b0 2023-10-10T20:25:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 msgr2=0x7fa8e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 0x7fa8e40693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8006f00 tx=0x7fa8e8008040 comp rx=0 tx=0).stop 2023-10-10T20:25:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 msgr2=0x7fa8f80d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:05.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8072050 tx=0x7fa8f0018000 comp rx=0 tx=0).stop 2023-10-10T20:25:05.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 shutdown_connections 2023-10-10T20:25:05.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa8e4066f30 0x7fa8e40693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:05.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f815a1b0 0x7fa8f80d0d30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8f806a650 0x7fa8f8146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 --2- 172.21.15.148:0/921511849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8f8159de0 0x7fa8f80c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.542+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 >> 172.21.15.148:0/921511849 conn(0x7fa8f8068d60 msgr2=0x7fa8f80b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.543+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 shutdown_connections 2023-10-10T20:25:05.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:05.543+0000 7fa8fe58c700 1 -- 172.21.15.148:0/921511849 wait complete. 2023-10-10T20:25:05.548 INFO:teuthology.orchestra.run.smithi148.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-10T20:25:05.558 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:25:05.558 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:25:05.559 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:25:05.559 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:25:05.559 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:25:05.560 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:25:05.560 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:25:05.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:05.816 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:25:05.884 INFO:teuthology.orchestra.run.smithi148.stdout:client5816 2023-10-10T20:25:05.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:05.885 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client5816/osdmap') 2023-10-10T20:25:05.961 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 116 barrier 116 flags 0x588000 2023-10-10T20:25:05.961 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:05.961 INFO:teuthology.orchestra.run.smithi148.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:05.962 INFO:teuthology.orchestra.run.smithi148.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:05.962 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:25:05.962 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:25:05.962 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:25:05.963 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:25:05.963 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:25:05.963 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:25:05.963 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:25:05.963 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:25:05.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:25:05.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:25:05.965 INFO:teuthology.orchestra.run.smithi148.stderr:803 bytes copied, 3.7528e-05 s, 21.4 MB/s 2023-10-10T20:25:05.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-10T20:25:06.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.118+0000 7fa4c1efb700 1 Processor -- start 2023-10-10T20:25:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.118+0000 7fa4c1efb700 1 -- start start 2023-10-10T20:25:06.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c1efb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4bc15a170 0x7fa4bc068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c1efb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4bc0694a0 con 0x7fa4bc159da0 2023-10-10T20:25:06.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c1efb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4bc15ea70 con 0x7fa4bc069040 2023-10-10T20:25:06.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c1efb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4bc0b34b0 con 0x7fa4bc15a170 2023-10-10T20:25:06.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4bbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa4bc15a170 0x7fa4bc068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c0ef9700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4bc159da0 0x7fa4bc0b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58954/0 (socket says 172.21.15.148:58954) 2023-10-10T20:25:06.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4c0ef9700 1 -- 172.21.15.148:0/2453169930 learned_addr learned my addr 172.21.15.148:0/2453169930 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:06.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4bbfff700 1 -- 172.21.15.148:0/2453169930 >> v1:172.21.15.161:6789/0 conn(0x7fa4bc069040 legacy=0x7fa4bc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:06.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4bb7fe700 1 -- 172.21.15.148:0/2453169930 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669593551 0 0) 0x7fa4ac001490 con 0x7fa4bc159da0 2023-10-10T20:25:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4bbfff700 1 -- 172.21.15.148:0/2453169930 >> v1:172.21.15.148:6789/0 conn(0x7fa4bc159da0 legacy=0x7fa4bc0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.119+0000 7fa4bbfff700 1 -- 172.21.15.148:0/2453169930 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc0b3990 con 0x7fa4bc15a170 2023-10-10T20:25:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4bbfff700 1 --2- 172.21.15.148:0/2453169930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc068b00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00098e0 tx=0x7fa4b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b802638fe9a64190 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4bb7fe700 1 -- 172.21.15.148:0/2453169930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0009e80 con 0x7fa4bc15a170 2023-10-10T20:25:06.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4bb7fe700 1 -- 172.21.15.148:0/2453169930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b000b860 con 0x7fa4bc15a170 2023-10-10T20:25:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4bb7fe700 1 -- 172.21.15.148:0/2453169930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000bbe0 con 0x7fa4bc15a170 2023-10-10T20:25:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2453169930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 msgr2=0x7fa4bc068b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2453169930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc068b00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00098e0 tx=0x7fa4b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2453169930 shutdown_connections 2023-10-10T20:25:06.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2453169930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc068b00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2453169930 >> 172.21.15.148:0/2453169930 conn(0x7fa4bc0d0bc0 msgr2=0x7fa4bc0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2453169930 shutdown_connections 2023-10-10T20:25:06.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.120+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2453169930 wait complete. 2023-10-10T20:25:06.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.121+0000 7fa4c1efb700 1 Processor -- start 2023-10-10T20:25:06.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 -- start start 2023-10-10T20:25:06.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4bc069040 0x7fa4bc1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 0x7fa4bc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0c7c80 con 0x7fa4bc15a170 2023-10-10T20:25:06.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0c7f20 con 0x7fa4bc159da0 2023-10-10T20:25:06.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c1efb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc15ce60 con 0x7fa4bc069040 2023-10-10T20:25:06.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c16fa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c16fa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52996/0 (socket says 172.21.15.148:52996) 2023-10-10T20:25:06.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.122+0000 7fa4c16fa700 1 -- 172.21.15.148:0/2244460940 learned_addr learned my addr 172.21.15.148:0/2244460940 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:06.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4bbfff700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 0x7fa4bc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c0ef9700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4bc069040 0x7fa4bc1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 -- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4bc069040 msgr2=0x7fa4bc1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4bc069040 0x7fa4bc1575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 -- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 msgr2=0x7fa4bc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 0x7fa4bc0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 -- 172.21.15.148:0/2244460940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0009420 con 0x7fa4bc15a170 2023-10-10T20:25:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4bbfff700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 0x7fa4bc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c16fa700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c5b0 tx=0x7fa4b4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4011590 con 0x7fa4bc15a170 2023-10-10T20:25:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4011710 con 0x7fa4bc15a170 2023-10-10T20:25:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc15d140 con 0x7fa4bc15a170 2023-10-10T20:25:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4011ad0 con 0x7fa4bc15a170 2023-10-10T20:25:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.123+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc157af0 con 0x7fa4bc15a170 2023-10-10T20:25:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.124+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc15d5e0 con 0x7fa4bc15a170 2023-10-10T20:25:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.124+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4b401b3b0 con 0x7fa4bc15a170 2023-10-10T20:25:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.125+0000 7fa4b8ff9700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 0x7fa4a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.125+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4b40839b0 con 0x7fa4bc15a170 2023-10-10T20:25:06.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.125+0000 7fa4c0ef9700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 0x7fa4a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.125+0000 7fa4c0ef9700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 0x7fa4a80693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000f80 tx=0x7fa4ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.129+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4b4050c20 con 0x7fa4bc15a170 2023-10-10T20:25:06.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.321+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7fa4bc0c8060 con 0x7fa4bc15a170 2023-10-10T20:25:06.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.542+0000 7fa4b8ff9700 1 -- 172.21.15.148:0/2244460940 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v118) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa4b4017070 con 0x7fa4bc15a170 2023-10-10T20:25:06.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 msgr2=0x7fa4a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 0x7fa4a80693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000f80 tx=0x7fa4ac009040 comp rx=0 tx=0).stop 2023-10-10T20:25:06.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 msgr2=0x7fa4bc0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c5b0 tx=0x7fa4b4007f30 comp rx=0 tx=0).stop 2023-10-10T20:25:06.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 shutdown_connections 2023-10-10T20:25:06.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa4a8066f30 0x7fa4a80693e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4bc15a170 0x7fa4bc0d14d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4bc069040 0x7fa4bc1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 --2- 172.21.15.148:0/2244460940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4bc159da0 0x7fa4bc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 >> 172.21.15.148:0/2244460940 conn(0x7fa4bc0d0bc0 msgr2=0x7fa4bc0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:06.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 shutdown_connections 2023-10-10T20:25:06.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.544+0000 7fa4c1efb700 1 -- 172.21.15.148:0/2244460940 wait complete. 2023-10-10T20:25:06.550 INFO:teuthology.orchestra.run.smithi148.stderr:pauserd,pausewr is set 2023-10-10T20:25:06.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-10T20:25:06.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.672+0000 7f468e280700 1 Processor -- start 2023-10-10T20:25:06.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.672+0000 7f468e280700 1 -- start start 2023-10-10T20:25:06.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468e280700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4688159e10 0x7f46880b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468e280700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46880cde50 0x7f46880b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468e280700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468815db80 con 0x7f468815a1e0 2023-10-10T20:25:06.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468e280700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468815e800 con 0x7f46880cde50 2023-10-10T20:25:06.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468e280700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468815de60 con 0x7f4688159e10 2023-10-10T20:25:06.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f468c81d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46880cde50 0x7f46880b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f46877fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f468815a1e0 0x7f46880cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58960/0 (socket says 172.21.15.148:58960) 2023-10-10T20:25:06.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.673+0000 7f46877fe700 1 -- 172.21.15.148:0/4098190815 learned_addr learned my addr 172.21.15.148:0/4098190815 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:06.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f4687fff700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4688159e10 0x7f46880b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468c81d700 1 -- 172.21.15.148:0/4098190815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4688159e10 msgr2=0x7f46880b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:06.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468c81d700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4688159e10 0x7f46880b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468c81d700 1 -- 172.21.15.148:0/4098190815 >> v1:172.21.15.148:6789/0 conn(0x7f468815a1e0 legacy=0x7f46880cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468c81d700 1 -- 172.21.15.148:0/4098190815 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468815e140 con 0x7f46880cde50 2023-10-10T20:25:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f4686ffd700 1 -- 172.21.15.148:0/4098190815 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419166481 0 0) 0x7f468815db80 con 0x7f468815a1e0 2023-10-10T20:25:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468c81d700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880cde50 0x7f46880b3260 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e60 tx=0x7f4680009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1780eb12c38f0641 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f4686ffd700 1 -- 172.21.15.148:0/4098190815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000e070 con 0x7f46880cde50 2023-10-10T20:25:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f4686ffd700 1 -- 172.21.15.148:0/4098190815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000bb60 con 0x7f46880cde50 2023-10-10T20:25:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f4686ffd700 1 -- 172.21.15.148:0/4098190815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680012430 con 0x7f46880cde50 2023-10-10T20:25:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 -- 172.21.15.148:0/4098190815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880cde50 msgr2=0x7f46880b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880cde50 0x7f46880b3260 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e60 tx=0x7f4680009780 comp rx=0 tx=0).stop 2023-10-10T20:25:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 -- 172.21.15.148:0/4098190815 shutdown_connections 2023-10-10T20:25:06.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880cde50 0x7f46880b3260 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 --2- 172.21.15.148:0/4098190815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4688159e10 0x7f46880b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.674+0000 7f468e280700 1 -- 172.21.15.148:0/4098190815 >> 172.21.15.148:0/4098190815 conn(0x7f4688067660 msgr2=0x7f4688067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:06.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.675+0000 7f468e280700 1 -- 172.21.15.148:0/4098190815 shutdown_connections 2023-10-10T20:25:06.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.675+0000 7f468e280700 1 -- 172.21.15.148:0/4098190815 wait complete. 2023-10-10T20:25:06.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 Processor -- start 2023-10-10T20:25:06.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 -- start start 2023-10-10T20:25:06.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 0x7f468815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f468815a1e0 0x7f4688169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c9ab0 con 0x7f4688159e10 2023-10-10T20:25:06.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c94d0 con 0x7f46880cde50 2023-10-10T20:25:06.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.676+0000 7f468e280700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46881597a0 con 0x7f468815a1e0 2023-10-10T20:25:06.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53008/0 (socket says 172.21.15.148:53008) 2023-10-10T20:25:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 -- 172.21.15.148:0/1941255877 learned_addr learned my addr 172.21.15.148:0/1941255877 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f468c81d700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f468815a1e0 0x7f4688169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f4687fff700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 0x7f468815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 -- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f468815a1e0 msgr2=0x7f4688169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f468815a1e0 0x7f4688169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 -- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 msgr2=0x7f468815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:06.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 0x7f468815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:06.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680009a20 con 0x7f4688159e10 2023-10-10T20:25:06.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f4687fff700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 0x7f468815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:06.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f46877fe700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c610 tx=0x7f467c001000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c005b30 con 0x7f4688159e10 2023-10-10T20:25:06.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c005c90 con 0x7f4688159e10 2023-10-10T20:25:06.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468815cd90 con 0x7f4688159e10 2023-10-10T20:25:06.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007300 con 0x7f4688159e10 2023-10-10T20:25:06.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.677+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468815d070 con 0x7f4688159e10 2023-10-10T20:25:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.678+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46880c9c60 con 0x7f4688159e10 2023-10-10T20:25:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f467c01b3b0 con 0x7f4688159e10 2023-10-10T20:25:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f466ffff700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 0x7f46740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f467c083a90 con 0x7f4688159e10 2023-10-10T20:25:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f4687fff700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 0x7f46740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:06.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=119+}) v3 -- 0x7f467406ea50 con 0x7f4688159e10 2023-10-10T20:25:06.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.679+0000 7f4687fff700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 0x7f46740693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4678006f00 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:06.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.683+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f467c017020 con 0x7f4688159e10 2023-10-10T20:25:06.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:06.880+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f468804c3c0 con 0x7f4688159e10 2023-10-10T20:25:07.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.551+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(119..119 src has 1..119) v4 ==== 280+0+0 (secure 0 0 0) 0x7f467c0550c0 con 0x7f4688159e10 2023-10-10T20:25:07.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.551+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=120}) v3 -- 0x7f467406f3a0 con 0x7f4688159e10 2023-10-10T20:25:07.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.555+0000 7f466ffff700 1 -- 172.21.15.148:0/1941255877 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v119) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46880c9c60 con 0x7f4688159e10 2023-10-10T20:25:07.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 msgr2=0x7f46740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 0x7f46740693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4678006f00 tx=0x7f4678008040 comp rx=0 tx=0).stop 2023-10-10T20:25:07.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 msgr2=0x7f468815c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c610 tx=0x7f467c001000 comp rx=0 tx=0).stop 2023-10-10T20:25:07.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 shutdown_connections 2023-10-10T20:25:07.561 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4674066f30 0x7f46740693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f468815a1e0 0x7f4688169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46880cde50 0x7f468815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 --2- 172.21.15.148:0/1941255877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4688159e10 0x7f468815c410 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 >> 172.21.15.148:0/1941255877 conn(0x7f4688067660 msgr2=0x7f46880b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 shutdown_connections 2023-10-10T20:25:07.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.558+0000 7f468e280700 1 -- 172.21.15.148:0/1941255877 wait complete. 2023-10-10T20:25:07.563 INFO:teuthology.orchestra.run.smithi148.stderr:pauserd,pausewr is unset 2023-10-10T20:25:07.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:07.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.687+0000 7fb6e6c8b700 1 Processor -- start 2023-10-10T20:25:07.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.688+0000 7fb6e6c8b700 1 -- start start 2023-10-10T20:25:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.688+0000 7fb6e6c8b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6e015e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.688+0000 7fb6e6c8b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e00cca20 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.688+0000 7fb6e6c8b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e015ea40 con 0x7fb6e015a0c0 2023-10-10T20:25:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.688+0000 7fb6e6c8b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0067660 con 0x7fb6e0159cf0 2023-10-10T20:25:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6e5228700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6e00cf1c0 0x7fb6e0068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58968/0 (socket says 172.21.15.148:58968) 2023-10-10T20:25:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6e5228700 1 -- 172.21.15.148:0/2369008550 learned_addr learned my addr 172.21.15.148:0/2369008550 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6e4a27700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6e015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713231669 0 0) 0x7fb6e00cca20 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c8003570 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697037936 0 0) 0x7fb6d0001110 con 0x7fb6e015a0c0 2023-10-10T20:25:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e00cca20 con 0x7fb6e015a0c0 2023-10-10T20:25:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662621075 0 0) 0x7fb6c8003570 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 msgr2=0x7fb6e015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6e015e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 >> v1:172.21.15.161:6789/0 conn(0x7fb6e015a0c0 legacy=0x7fb6e00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0067f00 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fb6d80013f0 con 0x7fb6e00cf1c0 2023-10-10T20:25:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 >> v1:172.21.15.148:6789/0 conn(0x7fb6e00cf1c0 legacy=0x7fb6e0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6c8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c8004d50 0x7fb6c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0067660 con 0x7fb6c80041d0 2023-10-10T20:25:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c800bbf0 con 0x7fb6c8004d50 2023-10-10T20:25:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c800bdb0 con 0x7fb6e0159cf0 2023-10-10T20:25:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6e4a27700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6c8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.689+0000 7fb6e5228700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c8004d50 0x7fb6c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 -- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 msgr2=0x7fb6c8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e0159cf0 0x7fb6c8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 -- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c8004d50 msgr2=0x7fb6c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c8004d50 0x7fb6c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 -- 172.21.15.148:0/2369008550 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8003570 con 0x7fb6c80041d0 2023-10-10T20:25:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e5228700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c8004d50 0x7fb6c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6c8004810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0001b70 tx=0x7fb6d0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca890b14ac87d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000c6a0 con 0x7fb6c80041d0 2023-10-10T20:25:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d000c800 con 0x7fb6c80041d0 2023-10-10T20:25:07.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6d77fe700 1 -- 172.21.15.148:0/2369008550 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000cba0 con 0x7fb6c80041d0 2023-10-10T20:25:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/2369008550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 msgr2=0x7fb6c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/2369008550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6c8004810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0001b70 tx=0x7fb6d0007d30 comp rx=0 tx=0).stop 2023-10-10T20:25:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e5228700 1 -- 172.21.15.148:0/2369008550 reap_dead start 2023-10-10T20:25:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/2369008550 shutdown_connections 2023-10-10T20:25:07.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/2369008550 >> 172.21.15.148:0/2369008550 conn(0x7fb6e006c6e0 msgr2=0x7fb6e0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/2369008550 shutdown_connections 2023-10-10T20:25:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.690+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/2369008550 wait complete. 2023-10-10T20:25:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.691+0000 7fb6e6c8b700 1 Processor -- start 2023-10-10T20:25:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 -- start start 2023-10-10T20:25:07.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 0x7fb6e00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e00b2640 0x7fb6e00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0153310 con 0x7fb6c80041d0 2023-10-10T20:25:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01506f0 con 0x7fb6e01524b0 2023-10-10T20:25:07.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e6c8b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0152d50 con 0x7fb6e00b2640 2023-10-10T20:25:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e4a27700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e4a27700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53030/0 (socket says 172.21.15.148:53030) 2023-10-10T20:25:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.692+0000 7fb6e4a27700 1 -- 172.21.15.148:0/1846703012 learned_addr learned my addr 172.21.15.148:0/1846703012 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e5228700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e00b2640 0x7fb6e00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 0x7fb6e00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 -- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e00b2640 msgr2=0x7fb6e00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e00b2640 0x7fb6e00b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 -- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 msgr2=0x7fb6e00b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 0x7fb6e00b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 -- 172.21.15.148:0/1846703012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0153030 con 0x7fb6c80041d0 2023-10-10T20:25:07.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 0x7fb6e00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:07.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e4a27700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc011f80 tx=0x7fb6dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:07.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00e040 con 0x7fb6c80041d0 2023-10-10T20:25:07.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00f040 con 0x7fb6c80041d0 2023-10-10T20:25:07.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc015430 con 0x7fb6c80041d0 2023-10-10T20:25:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0068410 con 0x7fb6c80041d0 2023-10-10T20:25:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.693+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e016b0b0 con 0x7fb6c80041d0 2023-10-10T20:25:07.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.694+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e004c3c0 con 0x7fb6c80041d0 2023-10-10T20:25:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.694+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6dc0157e0 con 0x7fb6c80041d0 2023-10-10T20:25:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.695+0000 7fb6d4ff9700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 0x7fb6cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:07.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.695+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6dc085230 con 0x7fb6c80041d0 2023-10-10T20:25:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.695+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 0x7fb6cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.695+0000 7fb6d7fff700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 0x7fb6cc0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6e006a380 tx=0x7fb6d0015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.699+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6dc055f30 con 0x7fb6c80041d0 2023-10-10T20:25:07.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.883+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6e00ce730 con 0x7fb6c80041d0 2023-10-10T20:25:07.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.884+0000 7fb6d4ff9700 1 -- 172.21.15.148:0/1846703012 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15420 (secure 0 0 0) 0x7fb6dc055ac0 con 0x7fb6c80041d0 2023-10-10T20:25:07.886 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:07.886 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:07.547178+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:07.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 msgr2=0x7fb6cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 0x7fb6cc0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6e006a380 tx=0x7fb6d0015000 comp rx=0 tx=0).stop 2023-10-10T20:25:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 msgr2=0x7fb6e01501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc011f80 tx=0x7fb6dc00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 shutdown_connections 2023-10-10T20:25:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6cc066f30 0x7fb6cc0693e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e00b2640 0x7fb6e00b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6c80041d0 0x7fb6e01501b0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 --2- 172.21.15.148:0/1846703012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e01524b0 0x7fb6e00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 >> 172.21.15.148:0/1846703012 conn(0x7fb6e006c6e0 msgr2=0x7fb6e0160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 shutdown_connections 2023-10-10T20:25:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:07.886+0000 7fb6e6c8b700 1 -- 172.21.15.148:0/1846703012 wait complete. 2023-10-10T20:25:07.901 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:25:07.901 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:25:07.902 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:25:07.902 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:25:07.902 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:25:07.903 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:25:07.903 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:25:08.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:08.157 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:25:08.225 INFO:teuthology.orchestra.run.smithi148.stdout:client5816 2023-10-10T20:25:08.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:08.226 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client5816/osdmap') 2023-10-10T20:25:08.300 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 116 barrier 116 flags 0x588000 2023-10-10T20:25:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:25:08.301 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:25:08.302 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:25:08.302 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:25:08.302 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:25:08.302 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:25:08.302 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:25:08.303 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:25:08.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:25:08.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:25:08.304 INFO:teuthology.orchestra.run.smithi148.stderr:803 bytes copied, 5.1689e-05 s, 15.5 MB/s 2023-10-10T20:25:08.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:08.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.455+0000 7f6b097c5700 1 Processor -- start 2023-10-10T20:25:08.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.455+0000 7f6b097c5700 1 -- start start 2023-10-10T20:25:08.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b097c5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b04156570 0x7f6b0406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b097c5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b0415a330 0x7f6b0406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b097c5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b04157c30 con 0x7f6b040c9380 2023-10-10T20:25:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b097c5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b041584f0 con 0x7f6b0415a330 2023-10-10T20:25:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b097c5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04157670 con 0x7f6b04156570 2023-10-10T20:25:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6afa7fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b0415a330 0x7f6b0406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b037fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6b040c9380 0x7f6b04156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58980/0 (socket says 172.21.15.148:58980) 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b037fe700 1 -- 172.21.15.148:0/35299060 learned_addr learned my addr 172.21.15.148:0/35299060 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.456+0000 7f6b02ffd700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b04156570 0x7f6b0406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6afa7fc700 1 -- 172.21.15.148:0/35299060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b04156570 msgr2=0x7f6b0406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6afa7fc700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b04156570 0x7f6b0406d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6afa7fc700 1 -- 172.21.15.148:0/35299060 >> v1:172.21.15.148:6789/0 conn(0x7f6b040c9380 legacy=0x7f6b04156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6afa7fc700 1 -- 172.21.15.148:0/35299060 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04157950 con 0x7f6b0415a330 2023-10-10T20:25:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b027fc700 1 -- 172.21.15.148:0/35299060 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055567014 0 0) 0x7f6b04157c30 con 0x7f6b040c9380 2023-10-10T20:25:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6afa7fc700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b0406d8c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0098e0 tx=0x7f6aec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59494ccc51b55d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b027fc700 1 -- 172.21.15.148:0/35299060 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00b640 con 0x7f6b0415a330 2023-10-10T20:25:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b027fc700 1 -- 172.21.15.148:0/35299060 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00b7a0 con 0x7f6b0415a330 2023-10-10T20:25:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b027fc700 1 -- 172.21.15.148:0/35299060 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00bb40 con 0x7f6b0415a330 2023-10-10T20:25:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 -- 172.21.15.148:0/35299060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 msgr2=0x7f6b0406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b0406d8c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0098e0 tx=0x7f6aec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 -- 172.21.15.148:0/35299060 shutdown_connections 2023-10-10T20:25:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b04156570 0x7f6b0406d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 --2- 172.21.15.148:0/35299060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b0406d8c0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 -- 172.21.15.148:0/35299060 >> 172.21.15.148:0/35299060 conn(0x7f6b04067660 msgr2=0x7f6b04067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 -- 172.21.15.148:0/35299060 shutdown_connections 2023-10-10T20:25:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.457+0000 7f6b097c5700 1 -- 172.21.15.148:0/35299060 wait complete. 2023-10-10T20:25:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 Processor -- start 2023-10-10T20:25:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 -- start start 2023-10-10T20:25:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b040c9380 0x7f6b040c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b04156570 0x7f6b040c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040c3220 con 0x7f6b0415a330 2023-10-10T20:25:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04169930 con 0x7f6b04156570 2023-10-10T20:25:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.459+0000 7f6b097c5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040c2980 con 0x7f6b040c9380 2023-10-10T20:25:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53044/0 (socket says 172.21.15.148:53044) 2023-10-10T20:25:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 -- 172.21.15.148:0/2347781955 learned_addr learned my addr 172.21.15.148:0/2347781955 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b02ffd700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b040c9380 0x7f6b040c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 -- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b040c9380 msgr2=0x7f6b040c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b040c9380 0x7f6b040c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 -- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b04156570 msgr2=0x7f6b040c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b04156570 0x7f6b040c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 -- 172.21.15.148:0/2347781955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009420 con 0x7f6b0415a330 2023-10-10T20:25:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b037fe700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00cbf0 tx=0x7f6afc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00a9c0 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00ab40 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0070f0 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b040c2c60 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.460+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04159470 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.461+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04169aa0 con 0x7f6b0415a330 2023-10-10T20:25:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.462+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6afc007520 con 0x7f6b0415a330 2023-10-10T20:25:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.462+0000 7f6afb7fe700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 0x7f6ae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.462+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6afc0848d0 con 0x7f6b0415a330 2023-10-10T20:25:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.462+0000 7f6b02ffd700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 0x7f6ae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.462+0000 7f6b02ffd700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 0x7f6ae40693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6af40093f0 tx=0x7f6af4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:08.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.466+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6afc051bc0 con 0x7f6b0415a330 2023-10-10T20:25:08.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.691+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b0404c3c0 con 0x7f6b0415a330 2023-10-10T20:25:08.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.692+0000 7f6afb7fe700 1 -- 172.21.15.148:0/2347781955 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6afc0551e0 con 0x7f6b0415a330 2023-10-10T20:25:08.695 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:08.695 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:08.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 msgr2=0x7f6ae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 0x7f6ae40693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6af40093f0 tx=0x7f6af4008040 comp rx=0 tx=0).stop 2023-10-10T20:25:08.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 msgr2=0x7f6b040b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:08.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00cbf0 tx=0x7f6afc0019b0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 shutdown_connections 2023-10-10T20:25:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6ae4066f30 0x7f6ae40693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b0415a330 0x7f6b040b18d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b040c9380 0x7f6b040c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 --2- 172.21.15.148:0/2347781955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b04156570 0x7f6b040c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:08.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 >> 172.21.15.148:0/2347781955 conn(0x7f6b04067660 msgr2=0x7f6b040cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:08.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 shutdown_connections 2023-10-10T20:25:08.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:08.694+0000 7f6b097c5700 1 -- 172.21.15.148:0/2347781955 wait complete. 2023-10-10T20:25:08.699 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:08.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 119 2023-10-10T20:25:08.839 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-10T20:25:08.840 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:25:08.841 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:25:08.841 DEBUG:teuthology.orchestra.run.smithi148:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-10T20:25:08.841 DEBUG:teuthology.orchestra.run.smithi148:> os.fchmod(fd, 0O666) 2023-10-10T20:25:08.842 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(fd) 2023-10-10T20:25:08.842 DEBUG:teuthology.orchestra.run.smithi148:> os.close(fd) 2023-10-10T20:25:08.842 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:25:09.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:09.097 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:25:09.164 INFO:teuthology.orchestra.run.smithi148.stdout:client5816 2023-10-10T20:25:09.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:09.165 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client5816/osdmap') 2023-10-10T20:25:09.239 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 119 barrier 119 flags 0x588000 2023-10-10T20:25:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.240 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:25:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:25:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:25:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:25:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:25:09.241 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:25:09.242 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:25:09.242 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:25:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:25:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:25:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:803 bytes copied, 3.2369e-05 s, 24.8 MB/s 2023-10-10T20:25:09.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:09.244 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-10T20:25:09.310 INFO:teuthology.orchestra.run.smithi148.stdout:client5816 2023-10-10T20:25:09.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:09.311 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/374035e8-0861-4d26-af5d-2eadca412b24.client5816/osdmap') 2023-10-10T20:25:09.386 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 119 barrier 119 flags 0x588000 2023-10-10T20:25:09.387 INFO:teuthology.orchestra.run.smithi148.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.387 INFO:teuthology.orchestra.run.smithi148.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.387 INFO:teuthology.orchestra.run.smithi148.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-10T20:25:09.387 INFO:teuthology.orchestra.run.smithi148.stdout:osd0 (1)172.21.15.148:6801 100% (exists, up) 100% -1 2023-10-10T20:25:09.388 INFO:teuthology.orchestra.run.smithi148.stdout:osd1 (1)172.21.15.148:6817 100% (exists, up) 100% -1 2023-10-10T20:25:09.388 INFO:teuthology.orchestra.run.smithi148.stdout:osd2 (1)172.21.15.148:6809 100% (exists, up) 100% -1 2023-10-10T20:25:09.388 INFO:teuthology.orchestra.run.smithi148.stdout:osd3 (1)172.21.15.148:6825 100% (exists, up) 100% -1 2023-10-10T20:25:09.388 INFO:teuthology.orchestra.run.smithi148.stdout:osd4 (1)172.21.15.161:6806 100% (exists, up) 100% -1 2023-10-10T20:25:09.388 INFO:teuthology.orchestra.run.smithi148.stdout:osd5 (1)172.21.15.161:6804 100% (exists, up) 100% -1 2023-10-10T20:25:09.389 INFO:teuthology.orchestra.run.smithi148.stdout:osd6 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2023-10-10T20:25:09.389 INFO:teuthology.orchestra.run.smithi148.stdout:osd7 (1)172.21.15.161:6803 100% (exists, up) 100% -1 2023-10-10T20:25:09.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records in 2023-10-10T20:25:09.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+1 records out 2023-10-10T20:25:09.390 INFO:teuthology.orchestra.run.smithi148.stderr:803 bytes copied, 3.3425e-05 s, 24.0 MB/s 2023-10-10T20:25:09.391 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:25:09.391 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:25:09.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:25:09.469 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:25:09.512 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:25:09.513 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:25:09.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:09.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:09.567 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:25:09.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:09.568 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:25:09.592 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:25:09.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:09.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:09.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:25:09.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:09.637 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:25:09.692 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:09.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.805+0000 7f686e7ed700 1 Processor -- start 2023-10-10T20:25:09.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.805+0000 7f686e7ed700 1 -- start start 2023-10-10T20:25:09.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686e7ed700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f686815a620 0x7f686815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686e7ed700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686815cbb0 con 0x7f686815d580 2023-10-10T20:25:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686e7ed700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686815bfa0 con 0x7f6868157870 2023-10-10T20:25:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686e7ed700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686815bdb0 con 0x7f686815a620 2023-10-10T20:25:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f68677fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f686815a620 0x7f686815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:09.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686cd8a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f686815d580 0x7f6868158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58994/0 (socket says 172.21.15.148:58994) 2023-10-10T20:25:09.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f686cd8a700 1 -- 172.21.15.148:0/2418797877 learned_addr learned my addr 172.21.15.148:0/2418797877 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:09.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f68677fe700 1 -- 172.21.15.148:0/2418797877 >> v1:172.21.15.161:6789/0 conn(0x7f6868157870 legacy=0x7f686815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:09.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f68677fe700 1 -- 172.21.15.148:0/2418797877 >> v1:172.21.15.148:6789/0 conn(0x7f686815d580 legacy=0x7f6868158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:09.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f68677fe700 1 -- 172.21.15.148:0/2418797877 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686815c280 con 0x7f686815a620 2023-10-10T20:25:09.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f6866ffd700 1 -- 172.21.15.148:0/2418797877 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220913274 0 0) 0x7f686815cbb0 con 0x7f686815d580 2023-10-10T20:25:09.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.806+0000 7f68677fe700 1 --2- 172.21.15.148:0/2418797877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815a620 0x7f686815d040 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098e0 tx=0x7f685c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d305d502ff05e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:09.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f6866ffd700 1 -- 172.21.15.148:0/2418797877 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c009e80 con 0x7f686815a620 2023-10-10T20:25:09.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f6866ffd700 1 -- 172.21.15.148:0/2418797877 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c00b820 con 0x7f686815a620 2023-10-10T20:25:09.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f6866ffd700 1 -- 172.21.15.148:0/2418797877 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c00bba0 con 0x7f686815a620 2023-10-10T20:25:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 -- 172.21.15.148:0/2418797877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815a620 msgr2=0x7f686815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 --2- 172.21.15.148:0/2418797877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815a620 0x7f686815d040 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098e0 tx=0x7f685c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 -- 172.21.15.148:0/2418797877 shutdown_connections 2023-10-10T20:25:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 --2- 172.21.15.148:0/2418797877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815a620 0x7f686815d040 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 -- 172.21.15.148:0/2418797877 >> 172.21.15.148:0/2418797877 conn(0x7f6868068ce0 msgr2=0x7f68680690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:09.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 -- 172.21.15.148:0/2418797877 shutdown_connections 2023-10-10T20:25:09.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.807+0000 7f686e7ed700 1 -- 172.21.15.148:0/2418797877 wait complete. 2023-10-10T20:25:09.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.808+0000 7f686e7ed700 1 Processor -- start 2023-10-10T20:25:09.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.808+0000 7f686e7ed700 1 -- start start 2023-10-10T20:25:09.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868157870 0x7f686814dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:09.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f686815a620 0x7f68681543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:09.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:09.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68680b1900 con 0x7f686815d580 2023-10-10T20:25:09.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68680b2740 con 0x7f6868157870 2023-10-10T20:25:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686e7ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68680b1b30 con 0x7f686815a620 2023-10-10T20:25:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53064/0 (socket says 172.21.15.148:53064) 2023-10-10T20:25:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 -- 172.21.15.148:0/4098325198 learned_addr learned my addr 172.21.15.148:0/4098325198 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 -- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f686815a620 msgr2=0x7f68681543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:09.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f686815a620 0x7f68681543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:09.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 -- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868157870 msgr2=0x7f686814dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:09.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.809+0000 7f686cd8a700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868157870 0x7f686814dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:09.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f686cd8a700 1 -- 172.21.15.148:0/4098325198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009420 con 0x7f686815d580 2023-10-10T20:25:09.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f686cd8a700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f686000cbc0 tx=0x7f6860001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000a940 con 0x7f686815d580 2023-10-10T20:25:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686000aaa0 con 0x7f686815d580 2023-10-10T20:25:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68680b1e10 con 0x7f686815d580 2023-10-10T20:25:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000ae60 con 0x7f686815d580 2023-10-10T20:25:09.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.810+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68680b23d0 con 0x7f686815d580 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.811+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686804c3c0 con 0x7f686815d580 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.811+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6860007300 con 0x7f686815d580 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.812+0000 7f684ffff700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 0x7f68540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.812+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6860084910 con 0x7f686815d580 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.812+0000 7f6867fff700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 0x7f68540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.812+0000 7f6867fff700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 0x7f68540693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6858007fd0 tx=0x7f6858007b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:09.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:09.816+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6860051c00 con 0x7f686815d580 2023-10-10T20:25:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.043+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f686815c560 con 0x7f686815d580 2023-10-10T20:25:10.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.044+0000 7f684ffff700 1 -- 172.21.15.148:0/4098325198 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6860055220 con 0x7f686815d580 2023-10-10T20:25:10.046 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:10.047 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 msgr2=0x7f68540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 0x7f68540693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6858007fd0 tx=0x7f6858007b20 comp rx=0 tx=0).stop 2023-10-10T20:25:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 msgr2=0x7f6868154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f686000cbc0 tx=0x7f6860001980 comp rx=0 tx=0).stop 2023-10-10T20:25:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 shutdown_connections 2023-10-10T20:25:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6854066f30 0x7f68540693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f686815d580 0x7f6868154930 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868157870 0x7f686814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 --2- 172.21.15.148:0/4098325198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f686815a620 0x7f68681543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 >> 172.21.15.148:0/4098325198 conn(0x7f6868068ce0 msgr2=0x7f686814fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 shutdown_connections 2023-10-10T20:25:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.046+0000 7f686e7ed700 1 -- 172.21.15.148:0/4098325198 wait complete. 2023-10-10T20:25:10.051 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:10.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.174+0000 7fa886153700 1 Processor -- start 2023-10-10T20:25:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.174+0000 7fa886153700 1 -- start start 2023-10-10T20:25:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa886153700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800b1220 0x7fa8800c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa886153700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 0x7fa8800cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa886153700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88015de60 con 0x7fa8800b0e50 2023-10-10T20:25:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa886153700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800cbeb0 con 0x7fa8800b1220 2023-10-10T20:25:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa886153700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88015e140 con 0x7fa8800cb4f0 2023-10-10T20:25:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800b1220 0x7fa8800c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800b1220 0x7fa8800c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53074/0 (socket says 172.21.15.148:53074) 2023-10-10T20:25:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 -- 172.21.15.148:0/4059474737 learned_addr learned my addr 172.21.15.148:0/4059474737 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87ffff700 1 --2- 172.21.15.148:0/4059474737 >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 0x7fa8800cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 -- 172.21.15.148:0/4059474737 >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 msgr2=0x7fa8800cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 --2- 172.21.15.148:0/4059474737 >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 0x7fa8800cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 -- 172.21.15.148:0/4059474737 >> v1:172.21.15.161:6789/0 conn(0x7fa8800b0e50 legacy=0x7fa88006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 -- 172.21.15.148:0/4059474737 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88015e420 con 0x7fa8800b1220 2023-10-10T20:25:10.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87ffff700 1 --2- 172.21.15.148:0/4059474737 >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 0x7fa8800cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:10.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87effd700 1 --2- 172.21.15.148:0/4059474737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa8800c9180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa88015e9e0 tx=0x7fa870009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e928de83521435 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87e7fc700 1 -- 172.21.15.148:0/4059474737 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000b820 con 0x7fa8800b1220 2023-10-10T20:25:10.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87e7fc700 1 -- 172.21.15.148:0/4059474737 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000b980 con 0x7fa8800b1220 2023-10-10T20:25:10.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.175+0000 7fa87e7fc700 1 -- 172.21.15.148:0/4059474737 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000bd40 con 0x7fa8800b1220 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 -- 172.21.15.148:0/4059474737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 msgr2=0x7fa8800c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 --2- 172.21.15.148:0/4059474737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa8800c9180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa88015e9e0 tx=0x7fa870009250 comp rx=0 tx=0).stop 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 -- 172.21.15.148:0/4059474737 shutdown_connections 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 --2- 172.21.15.148:0/4059474737 >> v2:172.21.15.161:3300/0 conn(0x7fa8800cb4f0 0x7fa8800cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 --2- 172.21.15.148:0/4059474737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa8800c9180 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 -- 172.21.15.148:0/4059474737 >> 172.21.15.148:0/4059474737 conn(0x7fa880069870 msgr2=0x7fa880069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 -- 172.21.15.148:0/4059474737 shutdown_connections 2023-10-10T20:25:10.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.176+0000 7fa886153700 1 -- 172.21.15.148:0/4059474737 wait complete. 2023-10-10T20:25:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.177+0000 7fa886153700 1 Processor -- start 2023-10-10T20:25:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.177+0000 7fa886153700 1 -- start start 2023-10-10T20:25:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 0x7fa88016b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8800cb4f0 0x7fa8800b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880154260 con 0x7fa8800b1220 2023-10-10T20:25:10.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88016e4d0 con 0x7fa8800b0e50 2023-10-10T20:25:10.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa886153700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88016dbe0 con 0x7fa8800cb4f0 2023-10-10T20:25:10.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53078/0 (socket says 172.21.15.148:53078) 2023-10-10T20:25:10.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 -- 172.21.15.148:0/605212924 learned_addr learned my addr 172.21.15.148:0/605212924 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:10.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87f7fe700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 0x7fa88016b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 -- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8800cb4f0 msgr2=0x7fa8800b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8800cb4f0 0x7fa8800b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 -- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 msgr2=0x7fa88016b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 0x7fa88016b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 -- 172.21.15.148:0/605212924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700094f0 con 0x7fa8800b1220 2023-10-10T20:25:10.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87f7fe700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 0x7fa88016b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:10.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.178+0000 7fa87effd700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa8800c59e0 tx=0x7fa870016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.179+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870006ab0 con 0x7fa8800b1220 2023-10-10T20:25:10.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.179+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa870020070 con 0x7fa8800b1220 2023-10-10T20:25:10.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.179+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88016de80 con 0x7fa8800b1220 2023-10-10T20:25:10.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.179+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87001b450 con 0x7fa8800b1220 2023-10-10T20:25:10.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.179+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa880154490 con 0x7fa8800b1220 2023-10-10T20:25:10.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.180+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88016e640 con 0x7fa8800b1220 2023-10-10T20:25:10.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.180+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa870022030 con 0x7fa8800b1220 2023-10-10T20:25:10.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.180+0000 7fa8777fe700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 0x7fa8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.180+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8700863e0 con 0x7fa8800b1220 2023-10-10T20:25:10.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.180+0000 7fa87f7fe700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 0x7fa8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.181+0000 7fa87f7fe700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 0x7fa8600693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa8800b4950 tx=0x7fa868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.185+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa870053650 con 0x7fa8800b1220 2023-10-10T20:25:10.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.371+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa88016e640 con 0x7fa8800b1220 2023-10-10T20:25:10.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.371+0000 7fa8777fe700 1 -- 172.21.15.148:0/605212924 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15420 (secure 0 0 0) 0x7fa8700092d0 con 0x7fa8800b1220 2023-10-10T20:25:10.374 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:10.374 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:07.547178+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:10.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 msgr2=0x7fa8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 0x7fa8600693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa8800b4950 tx=0x7fa868009040 comp rx=0 tx=0).stop 2023-10-10T20:25:10.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 msgr2=0x7fa880153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa8800c59e0 tx=0x7fa870016000 comp rx=0 tx=0).stop 2023-10-10T20:25:10.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 shutdown_connections 2023-10-10T20:25:10.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa860066f30 0x7fa8600693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8800cb4f0 0x7fa8800b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8800b0e50 0x7fa88016b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.373+0000 7fa886153700 1 --2- 172.21.15.148:0/605212924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800b1220 0x7fa880153d20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.374+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 >> 172.21.15.148:0/605212924 conn(0x7fa880069870 msgr2=0x7fa8800ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.374+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 shutdown_connections 2023-10-10T20:25:10.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.374+0000 7fa886153700 1 -- 172.21.15.148:0/605212924 wait complete. 2023-10-10T20:25:10.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:24:53.859471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:24:54.424055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:25:10.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:25:10.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.504+0000 7f0267fff700 1 Processor -- start 2023-10-10T20:25:10.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.504+0000 7f0267fff700 1 -- start start 2023-10-10T20:25:10.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0267fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0267fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0267fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680b4d70 con 0x7f026806a830 2023-10-10T20:25:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0267fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026806ac70 con 0x7f02680cce30 2023-10-10T20:25:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0267fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680cd8d0 con 0x7f02680b6580 2023-10-10T20:25:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0266ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0266ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:37978/0 (socket says 172.21.15.148:37978) 2023-10-10T20:25:10.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02677fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f026806a830 0x7f026815d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:58172/0 (socket says 172.21.15.148:58172) 2023-10-10T20:25:10.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:41664/0 (socket says 172.21.15.148:41664) 2023-10-10T20:25:10.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0266ffd700 1 -- 172.21.15.148:0/3987170083 learned_addr learned my addr 172.21.15.148:0/3987170083 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:10.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 -- 172.21.15.148:0/3987170083 >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 msgr2=0x7f026815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:10.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 --2- 172.21.15.148:0/3987170083 >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 -- 172.21.15.148:0/3987170083 >> v1:172.21.15.161:6789/0 conn(0x7f026806a830 legacy=0x7f026815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0265ffb700 1 -- 172.21.15.148:0/3987170083 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766104230 0 0) 0x7f02680b4d70 con 0x7f026806a830 2023-10-10T20:25:10.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f02667fc700 1 -- 172.21.15.148:0/3987170083 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680b4440 con 0x7f02680b6580 2023-10-10T20:25:10.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.505+0000 7f0266ffd700 1 --2- 172.21.15.148:0/3987170083 >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:10.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f02667fc700 1 --2- 172.21.15.148:0/3987170083 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02500098e0 tx=0x7f02500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5a55d4ca0b3d92 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0265ffb700 1 -- 172.21.15.148:0/3987170083 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000b640 con 0x7f02680b6580 2023-10-10T20:25:10.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0265ffb700 1 -- 172.21.15.148:0/3987170083 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000b7a0 con 0x7f02680b6580 2023-10-10T20:25:10.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0265ffb700 1 -- 172.21.15.148:0/3987170083 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000bb40 con 0x7f02680b6580 2023-10-10T20:25:10.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 -- 172.21.15.148:0/3987170083 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 msgr2=0x7f026806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 --2- 172.21.15.148:0/3987170083 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02500098e0 tx=0x7f02500091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 -- 172.21.15.148:0/3987170083 shutdown_connections 2023-10-10T20:25:10.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 --2- 172.21.15.148:0/3987170083 >> v2:172.21.15.161:3300/0 conn(0x7f02680cce30 0x7f026815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 --2- 172.21.15.148:0/3987170083 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026806a2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 -- 172.21.15.148:0/3987170083 >> 172.21.15.148:0/3987170083 conn(0x7f0268068cb0 msgr2=0x7f02680690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 -- 172.21.15.148:0/3987170083 shutdown_connections 2023-10-10T20:25:10.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.506+0000 7f0267fff700 1 -- 172.21.15.148:0/3987170083 wait complete. 2023-10-10T20:25:10.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.507+0000 7f0267fff700 1 Processor -- start 2023-10-10T20:25:10.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 -- start start 2023-10-10T20:25:10.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026813c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02680cce30 0x7f026813c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680b3440 con 0x7f026806a830 2023-10-10T20:25:10.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680b16b0 con 0x7f02680cce30 2023-10-10T20:25:10.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0267fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680b3160 con 0x7f02680b6580 2023-10-10T20:25:10.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0266ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0266ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53090/0 (socket says 172.21.15.148:53090) 2023-10-10T20:25:10.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.508+0000 7f0266ffd700 1 -- 172.21.15.148:0/1354947400 learned_addr learned my addr 172.21.15.148:0/1354947400 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:10.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f02677fe700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02680cce30 0x7f026813c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f02667fc700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026813c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 -- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 msgr2=0x7f026813c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026813c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 -- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02680cce30 msgr2=0x7f026813c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02680cce30 0x7f026813c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 -- 172.21.15.148:0/1354947400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258009ae0 con 0x7f026806a830 2023-10-10T20:25:10.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0266ffd700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0258009fc0 tx=0x7f025800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800c9c0 con 0x7f026806a830 2023-10-10T20:25:10.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025800f040 con 0x7f026806a830 2023-10-10T20:25:10.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0250009420 con 0x7f026806a830 2023-10-10T20:25:10.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025800cd50 con 0x7f026806a830 2023-10-10T20:25:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.509+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02680b1b80 con 0x7f026806a830 2023-10-10T20:25:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.510+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680b21d0 con 0x7f026806a830 2023-10-10T20:25:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.511+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0258014690 con 0x7f026806a830 2023-10-10T20:25:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.511+0000 7f025effd700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.511+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0258084b90 con 0x7f026806a830 2023-10-10T20:25:10.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.511+0000 7f02667fc700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.511+0000 7f02667fc700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 0x7f02480693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f02680b6990 tx=0x7f0250017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.515+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0258051e00 con 0x7f026806a830 2023-10-10T20:25:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.698+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02680b4720 con 0x7f026806a830 2023-10-10T20:25:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.698+0000 7f025effd700 1 -- 172.21.15.148:0/1354947400 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0258055420 con 0x7f026806a830 2023-10-10T20:25:10.701 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:25:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 msgr2=0x7f02480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 0x7f02480693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f02680b6990 tx=0x7f0250017000 comp rx=0 tx=0).stop 2023-10-10T20:25:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 msgr2=0x7f02680b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0258009fc0 tx=0x7f025800b040 comp rx=0 tx=0).stop 2023-10-10T20:25:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 shutdown_connections 2023-10-10T20:25:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0248066f30 0x7f02480693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02680cce30 0x7f026813c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f026806a830 0x7f02680b0480 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 --2- 172.21.15.148:0/1354947400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02680b6580 0x7f026813c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 >> 172.21.15.148:0/1354947400 conn(0x7f0268068cb0 msgr2=0x7f0268150480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.700+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 shutdown_connections 2023-10-10T20:25:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.701+0000 7f0267fff700 1 -- 172.21.15.148:0/1354947400 wait complete. 2023-10-10T20:25:10.714 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:25:10.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.830+0000 7f3eebfff700 1 Processor -- start 2023-10-10T20:25:10.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.830+0000 7f3eebfff700 1 -- start start 2023-10-10T20:25:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eebfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3eec0cce30 0x7f3eec15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 0x7f3eec0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eebfff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec0b4d70 con 0x7f3eec0b6580 2023-10-10T20:25:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eebfff700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec0cd8d0 con 0x7f3eec0cce30 2023-10-10T20:25:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eebfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b4440 con 0x7f3eec06aa00 2023-10-10T20:25:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3eec0cce30 0x7f3eec15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3ee27fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3eec0b6580 0x7f3eec06a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59010/0 (socket says 172.21.15.148:59010) 2023-10-10T20:25:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3ee27fc700 1 -- 172.21.15.148:0/4152603005 learned_addr learned my addr 172.21.15.148:0/4152603005 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeb7fe700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 0x7f3eec0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 -- 172.21.15.148:0/4152603005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 msgr2=0x7f3eec0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 0x7f3eec0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 -- 172.21.15.148:0/4152603005 >> v1:172.21.15.148:6789/0 conn(0x7f3eec0b6580 legacy=0x7f3eec06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eea7fc700 1 -- 172.21.15.148:0/4152603005 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551304192 0 0) 0x7f3eec0b4d70 con 0x7f3eec0b6580 2023-10-10T20:25:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 -- 172.21.15.148:0/4152603005 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0b4720 con 0x7f3eec0cce30 2023-10-10T20:25:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeb7fe700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 0x7f3eec0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.831+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0cce30 0x7f3eec15aca0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098e0 tx=0x7f3edc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f19bb27c836452d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eea7fc700 1 -- 172.21.15.148:0/4152603005 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc009e80 con 0x7f3eec0cce30 2023-10-10T20:25:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eea7fc700 1 -- 172.21.15.148:0/4152603005 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc00b860 con 0x7f3eec0cce30 2023-10-10T20:25:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eea7fc700 1 -- 172.21.15.148:0/4152603005 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00bbe0 con 0x7f3eec0cce30 2023-10-10T20:25:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 -- 172.21.15.148:0/4152603005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0cce30 msgr2=0x7f3eec15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0cce30 0x7f3eec15aca0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098e0 tx=0x7f3edc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 -- 172.21.15.148:0/4152603005 shutdown_connections 2023-10-10T20:25:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec06aa00 0x7f3eec0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 --2- 172.21.15.148:0/4152603005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0cce30 0x7f3eec15aca0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 -- 172.21.15.148:0/4152603005 >> 172.21.15.148:0/4152603005 conn(0x7f3eec068cb0 msgr2=0x7f3eec0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 -- 172.21.15.148:0/4152603005 shutdown_connections 2023-10-10T20:25:10.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.832+0000 7f3eebfff700 1 -- 172.21.15.148:0/4152603005 wait complete. 2023-10-10T20:25:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.833+0000 7f3eebfff700 1 Processor -- start 2023-10-10T20:25:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.833+0000 7f3eebfff700 1 -- start start 2023-10-10T20:25:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 0x7f3eec153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec0cce30 0x7f3eec16d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec157740 con 0x7f3eec0b6580 2023-10-10T20:25:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec155600 con 0x7f3eec06aa00 2023-10-10T20:25:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eebfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec154a30 con 0x7f3eec0cce30 2023-10-10T20:25:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3ee27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3ee27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53098/0 (socket says 172.21.15.148:53098) 2023-10-10T20:25:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3ee27fc700 1 -- 172.21.15.148:0/3261312311 learned_addr learned my addr 172.21.15.148:0/3261312311 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eeb7fe700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec0cce30 0x7f3eec16d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.834+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 0x7f3eec153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 -- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec0cce30 msgr2=0x7f3eec16d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec0cce30 0x7f3eec16d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 -- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 msgr2=0x7f3eec153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 0x7f3eec153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 -- 172.21.15.148:0/3261312311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc009420 con 0x7f3eec0b6580 2023-10-10T20:25:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 0x7f3eec153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee27fc700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c7e0 tx=0x7f3ed4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4005b30 con 0x7f3eec0b6580 2023-10-10T20:25:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed4005c90 con 0x7f3eec0b6580 2023-10-10T20:25:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec154cd0 con 0x7f3eec0b6580 2023-10-10T20:25:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4007300 con 0x7f3eec0b6580 2023-10-10T20:25:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.835+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec155290 con 0x7f3eec0b6580 2023-10-10T20:25:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.836+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec155790 con 0x7f3eec0b6580 2023-10-10T20:25:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.836+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ed401b400 con 0x7f3eec0b6580 2023-10-10T20:25:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.837+0000 7f3ee37fe700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 0x7f3ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.837+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ed4084050 con 0x7f3eec0b6580 2023-10-10T20:25:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.837+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 0x7f3ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.837+0000 7f3eeaffd700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 0x7f3ecc0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0091e0 tx=0x7f3edc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:10.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:10.841+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ed4051340 con 0x7f3eec0b6580 2023-10-10T20:25:11.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.023+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3eec0b4a00 con 0x7f3eec0b6580 2023-10-10T20:25:11.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.023+0000 7f3ee37fe700 1 -- 172.21.15.148:0/3261312311 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ed4054960 con 0x7f3eec0b6580 2023-10-10T20:25:11.026 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:25:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 msgr2=0x7f3ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 0x7f3ecc0693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0091e0 tx=0x7f3edc017000 comp rx=0 tx=0).stop 2023-10-10T20:25:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 msgr2=0x7f3eec155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c7e0 tx=0x7f3ed4007df0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 shutdown_connections 2023-10-10T20:25:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3ecc066f30 0x7f3ecc0693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3eec0cce30 0x7f3eec16d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3eec06aa00 0x7f3eec153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 --2- 172.21.15.148:0/3261312311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3eec0b6580 0x7f3eec155a90 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 >> 172.21.15.148:0/3261312311 conn(0x7f3eec068cb0 msgr2=0x7f3eec0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 shutdown_connections 2023-10-10T20:25:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.025+0000 7f3eebfff700 1 -- 172.21.15.148:0/3261312311 wait complete. 2023-10-10T20:25:11.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:25:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.156+0000 7f1bdf10c700 1 Processor -- start 2023-10-10T20:25:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.156+0000 7f1bdf10c700 1 -- start start 2023-10-10T20:25:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdf10c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdf10c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1bd80cc160 0x7f1bd80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdf10c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd8157010 con 0x7f1bd80cb670 2023-10-10T20:25:11.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdf10c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd80cbdf0 con 0x7f1bd80cc160 2023-10-10T20:25:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdf10c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd80ccb20 con 0x7f1bd8156c40 2023-10-10T20:25:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdd6a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1bd80cc160 0x7f1bd80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdd6a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1bd80cc160 0x7f1bd80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53102/0 (socket says 172.21.15.148:53102) 2023-10-10T20:25:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.157+0000 7f1bdd6a9700 1 -- 172.21.15.148:0/260568115 learned_addr learned my addr 172.21.15.148:0/260568115 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdcea8700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdd6a9700 1 -- 172.21.15.148:0/260568115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 msgr2=0x7f1bd815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdd6a9700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdd6a9700 1 -- 172.21.15.148:0/260568115 >> v1:172.21.15.161:6789/0 conn(0x7f1bd80cb670 legacy=0x7f1bd80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdd6a9700 1 -- 172.21.15.148:0/260568115 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80c9410 con 0x7f1bd80cc160 2023-10-10T20:25:11.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdd6a9700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd80cc5e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00099e0 tx=0x7f1bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd4f34d781159ee server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bd77fe700 1 -- 172.21.15.148:0/260568115 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000b820 con 0x7f1bd80cc160 2023-10-10T20:25:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bd77fe700 1 -- 172.21.15.148:0/260568115 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd000b980 con 0x7f1bd80cc160 2023-10-10T20:25:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bd77fe700 1 -- 172.21.15.148:0/260568115 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000bd20 con 0x7f1bd80cc160 2023-10-10T20:25:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 -- 172.21.15.148:0/260568115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 msgr2=0x7f1bd80cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd80cc5e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00099e0 tx=0x7f1bd0009250 comp rx=0 tx=0).stop 2023-10-10T20:25:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 -- 172.21.15.148:0/260568115 shutdown_connections 2023-10-10T20:25:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd80cc5e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/260568115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.158+0000 7f1bdf10c700 1 -- 172.21.15.148:0/260568115 >> 172.21.15.148:0/260568115 conn(0x7f1bd80ce570 msgr2=0x7f1bd80ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.159+0000 7f1bdf10c700 1 -- 172.21.15.148:0/260568115 shutdown_connections 2023-10-10T20:25:11.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.159+0000 7f1bdf10c700 1 -- 172.21.15.148:0/260568115 wait complete. 2023-10-10T20:25:11.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 Processor -- start 2023-10-10T20:25:11.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 -- start start 2023-10-10T20:25:11.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd80cb670 0x7f1bd8169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81576b0 con 0x7f1bd80cc160 2023-10-10T20:25:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8155780 con 0x7f1bd80cb670 2023-10-10T20:25:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.160+0000 7f1bdf10c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81551c0 con 0x7f1bd8156c40 2023-10-10T20:25:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53110/0 (socket says 172.21.15.148:53110) 2023-10-10T20:25:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 -- 172.21.15.148:0/3720992484 learned_addr learned my addr 172.21.15.148:0/3720992484 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bdcea8700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd80cb670 0x7f1bd8169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bdd6a9700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 -- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 msgr2=0x7f1bd8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 -- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd80cb670 msgr2=0x7f1bd8169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd80cb670 0x7f1bd8169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 -- 172.21.15.148:0/3720992484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd00094f0 con 0x7f1bd80cc160 2023-10-10T20:25:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd7fff700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c590 tx=0x7f1bc000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0011590 con 0x7f1bd80cc160 2023-10-10T20:25:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0011710 con 0x7f1bd80cc160 2023-10-10T20:25:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd81554a0 con 0x7f1bd80cc160 2023-10-10T20:25:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0011ad0 con 0x7f1bd80cc160 2023-10-10T20:25:11.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.161+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd816a0e0 con 0x7f1bd80cc160 2023-10-10T20:25:11.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.162+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd81577f0 con 0x7f1bd80cc160 2023-10-10T20:25:11.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.163+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1bc001b3b0 con 0x7f1bd80cc160 2023-10-10T20:25:11.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.163+0000 7f1bd4ff9700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 0x7f1bb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.163+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bc00839c0 con 0x7f1bd80cc160 2023-10-10T20:25:11.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.163+0000 7f1bdcea8700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 0x7f1bb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.163+0000 7f1bdcea8700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 0x7f1bb80693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80075a0 tx=0x7f1bc80097e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.167+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bc0050c30 con 0x7f1bd80cc160 2023-10-10T20:25:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.347+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1bd804c3c0 con 0x7f1bd80cc160 2023-10-10T20:25:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.347+0000 7f1bd4ff9700 1 -- 172.21.15.148:0/3720992484 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1bc0017070 con 0x7f1bd80cc160 2023-10-10T20:25:11.350 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:25:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 msgr2=0x7f1bb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 0x7f1bb80693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80075a0 tx=0x7f1bc80097e0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 msgr2=0x7f1bd8154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c590 tx=0x7f1bc000c8a0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 shutdown_connections 2023-10-10T20:25:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1bb8066f30 0x7f1bb80693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1bd8156c40 0x7f1bd8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd80cb670 0x7f1bd8169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 --2- 172.21.15.148:0/3720992484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bd80cc160 0x7f1bd8154870 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 >> 172.21.15.148:0/3720992484 conn(0x7f1bd80ce570 msgr2=0x7f1bd8069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 shutdown_connections 2023-10-10T20:25:11.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.349+0000 7f1bdf10c700 1 -- 172.21.15.148:0/3720992484 wait complete. 2023-10-10T20:25:11.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:11.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.476+0000 7fb84938b700 1 Processor -- start 2023-10-10T20:25:11.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.476+0000 7fb84938b700 1 -- start start 2023-10-10T20:25:11.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8440cd6e0 0x7fb8440cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb84415dda0 0x7fb84415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84415e850 con 0x7fb8440cd6e0 2023-10-10T20:25:11.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8440cd030 con 0x7fb84415dda0 2023-10-10T20:25:11.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb84938b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440cb7c0 con 0x7fb8440cdab0 2023-10-10T20:25:11.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8440cd6e0 0x7fb8440cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8440cd6e0 0x7fb8440cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53112/0 (socket says 172.21.15.148:53112) 2023-10-10T20:25:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 -- 172.21.15.148:0/2749167085 learned_addr learned my addr 172.21.15.148:0/2749167085 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb848b8a700 1 --2- 172.21.15.148:0/2749167085 >> v2:172.21.15.161:3300/0 conn(0x7fb84415dda0 0x7fb84415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb8437fe700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 -- 172.21.15.148:0/2749167085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 msgr2=0x7fb84415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:11.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 -- 172.21.15.148:0/2749167085 >> v2:172.21.15.161:3300/0 conn(0x7fb84415dda0 msgr2=0x7fb84415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:11.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.477+0000 7fb843fff700 1 --2- 172.21.15.148:0/2749167085 >> v2:172.21.15.161:3300/0 conn(0x7fb84415dda0 0x7fb84415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb843fff700 1 -- 172.21.15.148:0/2749167085 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84415e290 con 0x7fb8440cd6e0 2023-10-10T20:25:11.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb8437fe700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:11.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb843fff700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8440cd6e0 0x7fb8440cbc50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb844089ac0 tx=0x7fb834009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c72caecf7085b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb842ffd700 1 -- 172.21.15.148:0/2749167085 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834009e70 con 0x7fb8440cd6e0 2023-10-10T20:25:11.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb842ffd700 1 -- 172.21.15.148:0/2749167085 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400b860 con 0x7fb8440cd6e0 2023-10-10T20:25:11.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb842ffd700 1 -- 172.21.15.148:0/2749167085 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400bc00 con 0x7fb8440cd6e0 2023-10-10T20:25:11.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 -- 172.21.15.148:0/2749167085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8440cd6e0 msgr2=0x7fb8440cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8440cd6e0 0x7fb8440cbc50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb844089ac0 tx=0x7fb834009250 comp rx=0 tx=0).stop 2023-10-10T20:25:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 -- 172.21.15.148:0/2749167085 shutdown_connections 2023-10-10T20:25:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 --2- 172.21.15.148:0/2749167085 >> v2:172.21.15.161:3300/0 conn(0x7fb84415dda0 0x7fb84415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8440cd6e0 0x7fb8440cbc50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 --2- 172.21.15.148:0/2749167085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 -- 172.21.15.148:0/2749167085 >> 172.21.15.148:0/2749167085 conn(0x7fb844067660 msgr2=0x7fb844067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 -- 172.21.15.148:0/2749167085 shutdown_connections 2023-10-10T20:25:11.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.478+0000 7fb84938b700 1 -- 172.21.15.148:0/2749167085 wait complete. 2023-10-10T20:25:11.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.479+0000 7fb84938b700 1 Processor -- start 2023-10-10T20:25:11.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 -- start start 2023-10-10T20:25:11.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8440cd6e0 0x7fb8440b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440b60b0 con 0x7fb84415dda0 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440b54a0 con 0x7fb8440cd6e0 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb84938b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440b5780 con 0x7fb8440cdab0 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb848b8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb848b8a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53126/0 (socket says 172.21.15.148:53126) 2023-10-10T20:25:11.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.480+0000 7fb848b8a700 1 -- 172.21.15.148:0/3260298208 learned_addr learned my addr 172.21.15.148:0/3260298208 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb843fff700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8440cd6e0 0x7fb8440b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb8437fe700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 -- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 msgr2=0x7fb84415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 -- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8440cd6e0 msgr2=0x7fb8440b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8440cd6e0 0x7fb8440b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 -- 172.21.15.148:0/3260298208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340094f0 con 0x7fb84415dda0 2023-10-10T20:25:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb848b8a700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009e90 tx=0x7fb83c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00cc20 con 0x7fb84415dda0 2023-10-10T20:25:11.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c00f040 con 0x7fb84415dda0 2023-10-10T20:25:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c014480 con 0x7fb84415dda0 2023-10-10T20:25:11.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8440b5a60 con 0x7fb84415dda0 2023-10-10T20:25:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.481+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8440caae0 con 0x7fb84415dda0 2023-10-10T20:25:11.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.482+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb844089280 con 0x7fb84415dda0 2023-10-10T20:25:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.483+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb83c014830 con 0x7fb84415dda0 2023-10-10T20:25:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.483+0000 7fb82bfff700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 0x7fb8300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.483+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb83c085010 con 0x7fb84415dda0 2023-10-10T20:25:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.483+0000 7fb843fff700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 0x7fb8300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.483+0000 7fb843fff700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 0x7fb8300693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb844089ac0 tx=0x7fb834017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.487+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb83c055d10 con 0x7fb84415dda0 2023-10-10T20:25:11.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.716+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb84415e570 con 0x7fb84415dda0 2023-10-10T20:25:11.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.717+0000 7fb82bfff700 1 -- 172.21.15.148:0/3260298208 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb83c0558a0 con 0x7fb84415dda0 2023-10-10T20:25:11.720 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:11.720 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:11.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 msgr2=0x7fb8300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 0x7fb8300693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb844089ac0 tx=0x7fb834017000 comp rx=0 tx=0).stop 2023-10-10T20:25:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 msgr2=0x7fb84415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009e90 tx=0x7fb83c00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 shutdown_connections 2023-10-10T20:25:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb830066f30 0x7fb8300693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb84415dda0 0x7fb84415b3a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8440cd6e0 0x7fb8440b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 --2- 172.21.15.148:0/3260298208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8440cdab0 0x7fb84415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 >> 172.21.15.148:0/3260298208 conn(0x7fb844067660 msgr2=0x7fb8440d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 shutdown_connections 2023-10-10T20:25:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.719+0000 7fb84938b700 1 -- 172.21.15.148:0/3260298208 wait complete. 2023-10-10T20:25:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:11.733 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:11.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.846+0000 7fc213fff700 1 Processor -- start 2023-10-10T20:25:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.846+0000 7fc213fff700 1 -- start start 2023-10-10T20:25:11.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc213fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc213fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc213fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140cb7c0 con 0x7fc2140cdab0 2023-10-10T20:25:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc213fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21415e850 con 0x7fc2140cd6e0 2023-10-10T20:25:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc213fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140cd030 con 0x7fc21415cbd0 2023-10-10T20:25:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc212ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc212ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:38050/0 (socket says 172.21.15.148:38050) 2023-10-10T20:25:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:41730/0 (socket says 172.21.15.148:41730) 2023-10-10T20:25:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc212ffd700 1 -- 172.21.15.148:0/1390063482 learned_addr learned my addr 172.21.15.148:0/1390063482 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.847+0000 7fc20a7fc700 1 --1- 172.21.15.148:0/1390063482 >> v1:172.21.15.161:6789/0 conn(0x7fc2140cdab0 0x7fc21415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:58182/0 (socket says 172.21.15.148:58182) 2023-10-10T20:25:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2137fe700 1 -- 172.21.15.148:0/1390063482 >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 msgr2=0x7fc2140cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2137fe700 1 --2- 172.21.15.148:0/1390063482 >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2137fe700 1 -- 172.21.15.148:0/1390063482 >> v1:172.21.15.161:6789/0 conn(0x7fc2140cdab0 legacy=0x7fc21415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2137fe700 1 -- 172.21.15.148:0/1390063482 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21415e290 con 0x7fc21415cbd0 2023-10-10T20:25:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc212ffd700 1 --2- 172.21.15.148:0/1390063482 >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2137fe700 1 --2- 172.21.15.148:0/1390063482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009e60 tx=0x7fc20c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb40848e049176e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2127fc700 1 -- 172.21.15.148:0/1390063482 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00e070 con 0x7fc21415cbd0 2023-10-10T20:25:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2127fc700 1 -- 172.21.15.148:0/1390063482 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00bb60 con 0x7fc21415cbd0 2023-10-10T20:25:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc2127fc700 1 -- 172.21.15.148:0/1390063482 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c012430 con 0x7fc21415cbd0 2023-10-10T20:25:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.848+0000 7fc213fff700 1 -- 172.21.15.148:0/1390063482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 msgr2=0x7fc21415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 --2- 172.21.15.148:0/1390063482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009e60 tx=0x7fc20c009780 comp rx=0 tx=0).stop 2023-10-10T20:25:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 -- 172.21.15.148:0/1390063482 shutdown_connections 2023-10-10T20:25:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 --2- 172.21.15.148:0/1390063482 >> v2:172.21.15.161:3300/0 conn(0x7fc2140cd6e0 0x7fc2140cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 --2- 172.21.15.148:0/1390063482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc21415d020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 -- 172.21.15.148:0/1390063482 >> 172.21.15.148:0/1390063482 conn(0x7fc214067660 msgr2=0x7fc214067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 -- 172.21.15.148:0/1390063482 shutdown_connections 2023-10-10T20:25:11.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.849+0000 7fc213fff700 1 -- 172.21.15.148:0/1390063482 wait complete. 2023-10-10T20:25:11.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.850+0000 7fc213fff700 1 Processor -- start 2023-10-10T20:25:11.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.850+0000 7fc213fff700 1 -- start start 2023-10-10T20:25:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2140cd6e0 0x7fc21414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc2140b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21414ce30 con 0x7fc2140cdab0 2023-10-10T20:25:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140b4a40 con 0x7fc2140cd6e0 2023-10-10T20:25:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140b4d20 con 0x7fc21415cbd0 2023-10-10T20:25:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53130/0 (socket says 172.21.15.148:53130) 2023-10-10T20:25:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 -- 172.21.15.148:0/4126054411 learned_addr learned my addr 172.21.15.148:0/4126054411 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc212ffd700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2140cd6e0 0x7fc21414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc2137fe700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc2140b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 -- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 msgr2=0x7fc2140b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc2140b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 -- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2140cd6e0 msgr2=0x7fc21414c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2140cd6e0 0x7fc21414c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 -- 172.21.15.148:0/4126054411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040094f0 con 0x7fc2140cdab0 2023-10-10T20:25:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20a7fc700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c7c0 tx=0x7fc1fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc005a70 con 0x7fc2140cdab0 2023-10-10T20:25:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc005bf0 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c009a20 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.851+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0070f0 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.852+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2140b52e0 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.852+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214089280 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.853+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1fc007460 con 0x7fc2140cdab0 2023-10-10T20:25:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.853+0000 7fc20b7fe700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 0x7fc1f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.854+0000 7fc212ffd700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 0x7fc1f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.854+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc1fc083f60 con 0x7fc2140cdab0 2023-10-10T20:25:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.854+0000 7fc212ffd700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 0x7fc1f40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc204006d40 tx=0x7fc204008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:11.858+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1fc051250 con 0x7fc2140cdab0 2023-10-10T20:25:12.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.040+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc21415e570 con 0x7fc2140cdab0 2023-10-10T20:25:12.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.041+0000 7fc20b7fe700 1 -- 172.21.15.148:0/4126054411 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15420 (secure 0 0 0) 0x7fc1fc054870 con 0x7fc2140cdab0 2023-10-10T20:25:12.043 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:12.044 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:07.547178+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:12.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 msgr2=0x7fc1f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 0x7fc1f40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc204006d40 tx=0x7fc204008040 comp rx=0 tx=0).stop 2023-10-10T20:25:12.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 msgr2=0x7fc2140b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c7c0 tx=0x7fc1fc007f70 comp rx=0 tx=0).stop 2023-10-10T20:25:12.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 shutdown_connections 2023-10-10T20:25:12.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc1f4066f30 0x7fc1f40693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc21415cbd0 0x7fc2140b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2140cd6e0 0x7fc21414c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 --2- 172.21.15.148:0/4126054411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2140cdab0 0x7fc2140b5ae0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 >> 172.21.15.148:0/4126054411 conn(0x7fc214067660 msgr2=0x7fc2141576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 shutdown_connections 2023-10-10T20:25:12.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.043+0000 7fc213fff700 1 -- 172.21.15.148:0/4126054411 wait complete. 2023-10-10T20:25:12.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:25:12.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:12.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.170+0000 7f2f47fff700 1 Processor -- start 2023-10-10T20:25:12.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.170+0000 7f2f47fff700 1 -- start start 2023-10-10T20:25:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f47fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2f480cdb80 0x7f2f4815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f47fff700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4815db30 con 0x7f2f4815e500 2023-10-10T20:25:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f47fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480cd690 con 0x7f2f480d0430 2023-10-10T20:25:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f47fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4815b2d0 con 0x7f2f480cdb80 2023-10-10T20:25:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f477fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2f4815e500 0x7f2f480cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59024/0 (socket says 172.21.15.148:59024) 2023-10-10T20:25:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f477fe700 1 -- 172.21.15.148:0/3303928548 learned_addr learned my addr 172.21.15.148:0/3303928548 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.171+0000 7f2f3e7fc700 1 --2- 172.21.15.148:0/3303928548 >> v2:172.21.15.161:3300/0 conn(0x7f2f480cdb80 0x7f2f4815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f467fc700 1 -- 172.21.15.148:0/3303928548 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506797380 0 0) 0x7f2f4815db30 con 0x7f2f4815e500 2023-10-10T20:25:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f467fc700 1 -- 172.21.15.148:0/3303928548 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f34003570 con 0x7f2f4815e500 2023-10-10T20:25:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f3e7fc700 1 -- 172.21.15.148:0/3303928548 >> v1:172.21.15.161:6789/0 conn(0x7f2f480d0430 legacy=0x7f2f4815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f3e7fc700 1 -- 172.21.15.148:0/3303928548 >> v1:172.21.15.148:6789/0 conn(0x7f2f4815e500 legacy=0x7f2f480cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f3e7fc700 1 -- 172.21.15.148:0/3303928548 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480cdfd0 con 0x7f2f480cdb80 2023-10-10T20:25:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f3e7fc700 1 --2- 172.21.15.148:0/3303928548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f480cdb80 0x7f2f4815dfc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f4815dae0 tx=0x7f2f300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed03cd78f542153b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f467fc700 1 -- 172.21.15.148:0/3303928548 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30011630 con 0x7f2f480cdb80 2023-10-10T20:25:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f467fc700 1 -- 172.21.15.148:0/3303928548 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30011790 con 0x7f2f480cdb80 2023-10-10T20:25:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f467fc700 1 -- 172.21.15.148:0/3303928548 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30011b10 con 0x7f2f480cdb80 2023-10-10T20:25:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f47fff700 1 -- 172.21.15.148:0/3303928548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f480cdb80 msgr2=0x7f2f4815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.172+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3303928548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f480cdb80 0x7f2f4815dfc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2f4815dae0 tx=0x7f2f300091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.173+0000 7f2f47fff700 1 -- 172.21.15.148:0/3303928548 shutdown_connections 2023-10-10T20:25:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.173+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3303928548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f480cdb80 0x7f2f4815dfc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.173+0000 7f2f47fff700 1 -- 172.21.15.148:0/3303928548 >> 172.21.15.148:0/3303928548 conn(0x7f2f48068b40 msgr2=0x7f2f48068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.173+0000 7f2f47fff700 1 -- 172.21.15.148:0/3303928548 shutdown_connections 2023-10-10T20:25:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.173+0000 7f2f47fff700 1 -- 172.21.15.148:0/3303928548 wait complete. 2023-10-10T20:25:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.174+0000 7f2f47fff700 1 Processor -- start 2023-10-10T20:25:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.174+0000 7f2f47fff700 1 -- start start 2023-10-10T20:25:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.174+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.174+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f480d0430 0x7f2f480afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.174+0000 7f2f47fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f4815e500 0x7f2f4816def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f47fff700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4816b860 con 0x7f2f480cdb80 2023-10-10T20:25:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f47fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b0190 con 0x7f2f4815e500 2023-10-10T20:25:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f47fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b49a0 con 0x7f2f480d0430 2023-10-10T20:25:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53140/0 (socket says 172.21.15.148:53140) 2023-10-10T20:25:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 -- 172.21.15.148:0/3097602997 learned_addr learned my addr 172.21.15.148:0/3097602997 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f477fe700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f4815e500 0x7f2f4816def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f3e7fc700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f480d0430 0x7f2f480afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 -- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f480d0430 msgr2=0x7f2f480afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f480d0430 0x7f2f480afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 -- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f4815e500 msgr2=0x7f2f4816def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f4815e500 0x7f2f4816def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 -- 172.21.15.148:0/3097602997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30009420 con 0x7f2f480cdb80 2023-10-10T20:25:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f46ffd700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c7c0 tx=0x7f2f38007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011590 con 0x7f2f480cdb80 2023-10-10T20:25:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38011710 con 0x7f2f480cdb80 2023-10-10T20:25:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480b3d90 con 0x7f2f480cdb80 2023-10-10T20:25:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011ad0 con 0x7f2f480cdb80 2023-10-10T20:25:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.175+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480b4350 con 0x7f2f480cdb80 2023-10-10T20:25:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.176+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f480b4b10 con 0x7f2f480cdb80 2023-10-10T20:25:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.177+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f3801b3b0 con 0x7f2f480cdb80 2023-10-10T20:25:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.177+0000 7f2f3f7fe700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 0x7f2f280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.177+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f38083950 con 0x7f2f480cdb80 2023-10-10T20:25:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.177+0000 7f2f3e7fc700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 0x7f2f280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.177+0000 7f2f3e7fc700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 0x7f2f280693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f300091e0 tx=0x7f2f30016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.181+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f38050bc0 con 0x7f2f480cdb80 2023-10-10T20:25:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.406+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f4815aa30 con 0x7f2f480cdb80 2023-10-10T20:25:12.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.407+0000 7f2f3f7fe700 1 -- 172.21.15.148:0/3097602997 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2f38017070 con 0x7f2f480cdb80 2023-10-10T20:25:12.409 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:12.410 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:12.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 msgr2=0x7f2f280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 0x7f2f280693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f300091e0 tx=0x7f2f30016000 comp rx=0 tx=0).stop 2023-10-10T20:25:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 msgr2=0x7f2f4816b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c7c0 tx=0x7f2f38007f40 comp rx=0 tx=0).stop 2023-10-10T20:25:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 shutdown_connections 2023-10-10T20:25:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2f28066f30 0x7f2f280693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f4815e500 0x7f2f4816def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f480cdb80 0x7f2f4816b320 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 --2- 172.21.15.148:0/3097602997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f480d0430 0x7f2f480afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 >> 172.21.15.148:0/3097602997 conn(0x7f2f48068b40 msgr2=0x7f2f48158870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 shutdown_connections 2023-10-10T20:25:12.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.409+0000 7f2f47fff700 1 -- 172.21.15.148:0/3097602997 wait complete. 2023-10-10T20:25:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:12.423 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:12.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.537+0000 7f86b1251700 1 Processor -- start 2023-10-10T20:25:12.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.537+0000 7f86b1251700 1 -- start start 2023-10-10T20:25:12.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.537+0000 7f86b1251700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ac157870 0x7f86ac15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.537+0000 7f86b1251700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86ac15a620 0x7f86ac15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.537+0000 7f86b1251700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ac15cbb0 con 0x7f86ac15d580 2023-10-10T20:25:12.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86b1251700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ac15bfa0 con 0x7f86ac157870 2023-10-10T20:25:12.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86b1251700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ac15c280 con 0x7f86ac15a620 2023-10-10T20:25:12.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ac157870 0x7f86ac15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f86ac157870 0x7f86ac15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53156/0 (socket says 172.21.15.148:53156) 2023-10-10T20:25:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 -- 172.21.15.148:0/1933688019 learned_addr learned my addr 172.21.15.148:0/1933688019 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aa59c700 1 --2- 172.21.15.148:0/1933688019 >> v2:172.21.15.161:3300/0 conn(0x7f86ac15a620 0x7f86ac15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 -- 172.21.15.148:0/1933688019 >> v2:172.21.15.161:3300/0 conn(0x7f86ac15a620 msgr2=0x7f86ac15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 --2- 172.21.15.148:0/1933688019 >> v2:172.21.15.161:3300/0 conn(0x7f86ac15a620 0x7f86ac15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 -- 172.21.15.148:0/1933688019 >> v1:172.21.15.161:6789/0 conn(0x7f86ac15d580 legacy=0x7f86ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:12.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 -- 172.21.15.148:0/1933688019 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac15c560 con 0x7f86ac157870 2023-10-10T20:25:12.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86aad9d700 1 --2- 172.21.15.148:0/1933688019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac157870 0x7f86ac15b6a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f86ac15cb20 tx=0x7f869c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ebfac4ab8cb9eb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86a9d9b700 1 -- 172.21.15.148:0/1933688019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c00b820 con 0x7f86ac157870 2023-10-10T20:25:12.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86a9d9b700 1 -- 172.21.15.148:0/1933688019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c00b980 con 0x7f86ac157870 2023-10-10T20:25:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.538+0000 7f86a9d9b700 1 -- 172.21.15.148:0/1933688019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c00bd20 con 0x7f86ac157870 2023-10-10T20:25:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 -- 172.21.15.148:0/1933688019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac157870 msgr2=0x7f86ac15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 --2- 172.21.15.148:0/1933688019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac157870 0x7f86ac15b6a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f86ac15cb20 tx=0x7f869c009250 comp rx=0 tx=0).stop 2023-10-10T20:25:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 -- 172.21.15.148:0/1933688019 shutdown_connections 2023-10-10T20:25:12.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 --2- 172.21.15.148:0/1933688019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac157870 0x7f86ac15b6a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 --2- 172.21.15.148:0/1933688019 >> v2:172.21.15.161:3300/0 conn(0x7f86ac15a620 0x7f86ac15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 -- 172.21.15.148:0/1933688019 >> 172.21.15.148:0/1933688019 conn(0x7f86ac068ce0 msgr2=0x7f86ac0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 -- 172.21.15.148:0/1933688019 shutdown_connections 2023-10-10T20:25:12.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.539+0000 7f86b1251700 1 -- 172.21.15.148:0/1933688019 wait complete. 2023-10-10T20:25:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.540+0000 7f86b1251700 1 Processor -- start 2023-10-10T20:25:12.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.540+0000 7f86b1251700 1 -- start start 2023-10-10T20:25:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 0x7f86ac145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 0x7f86ac0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0b1670 con 0x7f86ac15d580 2023-10-10T20:25:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0b0610 con 0x7f86ac15a620 2023-10-10T20:25:12.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0b08b0 con 0x7f86ac157870 2023-10-10T20:25:12.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53168/0 (socket says 172.21.15.148:53168) 2023-10-10T20:25:12.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 -- 172.21.15.148:0/712821291 learned_addr learned my addr 172.21.15.148:0/712821291 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:12.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86aa59c700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 0x7f86ac0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86aad9d700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 0x7f86ac145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 -- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 msgr2=0x7f86ac145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 0x7f86ac145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 -- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 msgr2=0x7f86ac0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 0x7f86ac0cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 -- 172.21.15.148:0/712821291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0094f0 con 0x7f86ac15d580 2023-10-10T20:25:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86aad9d700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 0x7f86ac145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:12.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86aa59c700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 0x7f86ac0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86ab59e700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f86a400bb60 tx=0x7f86a40097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400c940 con 0x7f86ac15d580 2023-10-10T20:25:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a400caa0 con 0x7f86ac15d580 2023-10-10T20:25:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ac0b0b90 con 0x7f86ac15d580 2023-10-10T20:25:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a400ce60 con 0x7f86ac15d580 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.541+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac0b1150 con 0x7f86ac15d580 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.542+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac0b1370 con 0x7f86ac15d580 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.543+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86a4014620 con 0x7f86ac15d580 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.543+0000 7f8692ffd700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.543+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86a4084550 con 0x7f86ac15d580 2023-10-10T20:25:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.543+0000 7f86aad9d700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.543+0000 7f86aad9d700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 0x7f86980693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86ac15cb20 tx=0x7f869c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.547+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86a404c5a0 con 0x7f86ac15d580 2023-10-10T20:25:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.773+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86ac04c3c0 con 0x7f86ac15d580 2023-10-10T20:25:12.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.774+0000 7f8692ffd700 1 -- 172.21.15.148:0/712821291 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f86a40517c0 con 0x7f86ac15d580 2023-10-10T20:25:12.777 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:12.777 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:24:55.466159+0000","modified":"2023-10-10T20:24:59.099410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/746351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":746351743},{"type":"v1","addr":"172.21.15.148:6836","nonce":746351743}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-10T20:25:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 msgr2=0x7f86980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 0x7f86980693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86ac15cb20 tx=0x7f869c016000 comp rx=0 tx=0).stop 2023-10-10T20:25:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 msgr2=0x7f86ac0d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f86a400bb60 tx=0x7f86a40097f0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 shutdown_connections 2023-10-10T20:25:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8698066f30 0x7f86980693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86ac15d580 0x7f86ac0d0470 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86ac157870 0x7f86ac145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 --2- 172.21.15.148:0/712821291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86ac15a620 0x7f86ac0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 >> 172.21.15.148:0/712821291 conn(0x7f86ac068ce0 msgr2=0x7f86ac155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 shutdown_connections 2023-10-10T20:25:12.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.776+0000 7f86b1251700 1 -- 172.21.15.148:0/712821291 wait complete. 2023-10-10T20:25:12.781 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 57 2023-10-10T20:25:12.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:12.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.902+0000 7f489b7c3700 1 Processor -- start 2023-10-10T20:25:12.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 -- start start 2023-10-10T20:25:12.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4894159ef0 0x7f48940ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f489415a2c0 0x7f48940cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f4894067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489415dc60 con 0x7f489415a2c0 2023-10-10T20:25:12.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48940c91d0 con 0x7f4894159ef0 2023-10-10T20:25:12.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f489b7c3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940cbf30 con 0x7f48940cbad0 2023-10-10T20:25:12.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.903+0000 7f4898d5e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f489415a2c0 0x7f48940cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f489415a2c0 0x7f48940cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43716/0 (socket says 172.21.15.148:43716) 2023-10-10T20:25:12.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 -- 172.21.15.148:0/2253138505 learned_addr learned my addr 172.21.15.148:0/2253138505 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:12.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f489955f700 1 --2- 172.21.15.148:0/2253138505 >> v2:172.21.15.161:3300/0 conn(0x7f4894159ef0 0x7f48940ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 -- 172.21.15.148:0/2253138505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 msgr2=0x7f4894067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:12.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4899d60700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f4894067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f4894067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 -- 172.21.15.148:0/2253138505 >> v2:172.21.15.161:3300/0 conn(0x7f4894159ef0 msgr2=0x7f48940ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:12.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 --2- 172.21.15.148:0/2253138505 >> v2:172.21.15.161:3300/0 conn(0x7f4894159ef0 0x7f48940ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 -- 172.21.15.148:0/2253138505 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489415de70 con 0x7f489415a2c0 2023-10-10T20:25:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f4898d5e700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940cb590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f489406a8c0 tx=0x7f4884009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921c4e0e38225d1c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f488bfff700 1 -- 172.21.15.148:0/2253138505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400b860 con 0x7f489415a2c0 2023-10-10T20:25:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f488bfff700 1 -- 172.21.15.148:0/2253138505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b9c0 con 0x7f489415a2c0 2023-10-10T20:25:12.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.904+0000 7f488bfff700 1 -- 172.21.15.148:0/2253138505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bd60 con 0x7f489415a2c0 2023-10-10T20:25:12.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 -- 172.21.15.148:0/2253138505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 msgr2=0x7f48940cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940cb590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f489406a8c0 tx=0x7f4884009250 comp rx=0 tx=0).stop 2023-10-10T20:25:12.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 -- 172.21.15.148:0/2253138505 shutdown_connections 2023-10-10T20:25:12.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f4894067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 --2- 172.21.15.148:0/2253138505 >> v2:172.21.15.161:3300/0 conn(0x7f4894159ef0 0x7f48940ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 --2- 172.21.15.148:0/2253138505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940cb590 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 -- 172.21.15.148:0/2253138505 >> 172.21.15.148:0/2253138505 conn(0x7f4894069870 msgr2=0x7f4894069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:12.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 -- 172.21.15.148:0/2253138505 shutdown_connections 2023-10-10T20:25:12.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.905+0000 7f489b7c3700 1 -- 172.21.15.148:0/2253138505 wait complete. 2023-10-10T20:25:12.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.906+0000 7f489b7c3700 1 Processor -- start 2023-10-10T20:25:12.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.906+0000 7f489b7c3700 1 -- start start 2023-10-10T20:25:12.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f48940c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4894159ef0 0x7f48940c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b22c0 con 0x7f489415a2c0 2023-10-10T20:25:12.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b25a0 con 0x7f4894159ef0 2023-10-10T20:25:12.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b2880 con 0x7f48940cbad0 2023-10-10T20:25:12.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43722/0 (socket says 172.21.15.148:43722) 2023-10-10T20:25:12.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 -- 172.21.15.148:0/224781797 learned_addr learned my addr 172.21.15.148:0/224781797 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:12.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489955f700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f48940c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4898d5e700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4894159ef0 0x7f48940c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 -- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 msgr2=0x7f48940c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f48940c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 -- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4894159ef0 msgr2=0x7f48940c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:12.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4894159ef0 0x7f48940c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:12.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 -- 172.21.15.148:0/224781797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840094f0 con 0x7f489415a2c0 2023-10-10T20:25:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f4899d60700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f488c00cdb0 tx=0x7f488c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00a9d0 con 0x7f489415a2c0 2023-10-10T20:25:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00ab30 con 0x7f489415a2c0 2023-10-10T20:25:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48940b2b60 con 0x7f489415a2c0 2023-10-10T20:25:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c018480 con 0x7f489415a2c0 2023-10-10T20:25:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.907+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48940b1d90 con 0x7f489415a2c0 2023-10-10T20:25:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.908+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48940b2010 con 0x7f489415a2c0 2023-10-10T20:25:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.909+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f488c00d040 con 0x7f489415a2c0 2023-10-10T20:25:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.909+0000 7f48897fa700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 0x7f48780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:12.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.909+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f488c013070 con 0x7f489415a2c0 2023-10-10T20:25:12.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.909+0000 7f489955f700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 0x7f48780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:12.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.910+0000 7f489955f700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 0x7f48780693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:12.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:12.914+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f488c051bf0 con 0x7f489415a2c0 2023-10-10T20:25:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.096+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f489404c3c0 con 0x7f489415a2c0 2023-10-10T20:25:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.097+0000 7f48897fa700 1 -- 172.21.15.148:0/224781797 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f488c055210 con 0x7f489415a2c0 2023-10-10T20:25:13.099 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:13.099 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:07.547178+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:24:53.859471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-10T20:24:54.424055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 msgr2=0x7f48780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 0x7f48780693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4890006fd0 tx=0x7f4890008040 comp rx=0 tx=0).stop 2023-10-10T20:25:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 msgr2=0x7f48940c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f488c00cdb0 tx=0x7f488c0077d0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 shutdown_connections 2023-10-10T20:25:13.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4878066f30 0x7f48780693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415a2c0 0x7f48940c7ee0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48940cbad0 0x7f48940c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 --2- 172.21.15.148:0/224781797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4894159ef0 0x7f48940c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 >> 172.21.15.148:0/224781797 conn(0x7f4894069870 msgr2=0x7f4894155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 shutdown_connections 2023-10-10T20:25:13.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.099+0000 7f489b7c3700 1 -- 172.21.15.148:0/224781797 wait complete. 2023-10-10T20:25:13.113 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:25:13.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.227+0000 7f19a6d47700 1 Processor -- start 2023-10-10T20:25:13.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.227+0000 7f19a6d47700 1 -- start start 2023-10-10T20:25:13.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f19a6d47700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19a006c630 0x7f19a0067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f19a6d47700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a0067ba0 0x7f19a0159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f19a6d47700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00cb370 con 0x7f19a00b0890 2023-10-10T20:25:13.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f19a6d47700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a0067fe0 con 0x7f19a006c630 2023-10-10T20:25:13.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f19a6d47700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00b0c60 con 0x7f19a0067ba0 2023-10-10T20:25:13.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f1997fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19a006c630 0x7f19a0067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f1997fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19a006c630 0x7f19a0067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43724/0 (socket says 172.21.15.148:43724) 2023-10-10T20:25:13.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.228+0000 7f1997fff700 1 -- 172.21.15.148:0/1837907509 learned_addr learned my addr 172.21.15.148:0/1837907509 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:13.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f19a52e4700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a0067ba0 0x7f19a0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f1997fff700 1 -- 172.21.15.148:0/1837907509 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a0067ba0 msgr2=0x7f19a0159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:13.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f1997fff700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a0067ba0 0x7f19a0159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f1997fff700 1 -- 172.21.15.148:0/1837907509 >> v1:172.21.15.161:6789/0 conn(0x7f19a00b0890 legacy=0x7f19a00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f1997fff700 1 -- 172.21.15.148:0/1837907509 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a006a1e0 con 0x7f19a006c630 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f19977fe700 1 -- 172.21.15.148:0/1837907509 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217548484 0 0) 0x7f19a00cb370 con 0x7f19a00b0890 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f1997fff700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a006c630 0x7f19a0067660 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f19900099b0 tx=0x7f1990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a10358ad416fe29 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f19977fe700 1 -- 172.21.15.148:0/1837907509 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990009ea0 con 0x7f19a006c630 2023-10-10T20:25:13.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f19977fe700 1 -- 172.21.15.148:0/1837907509 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199000b860 con 0x7f19a006c630 2023-10-10T20:25:13.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.229+0000 7f19977fe700 1 -- 172.21.15.148:0/1837907509 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000bc00 con 0x7f19a006c630 2023-10-10T20:25:13.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 -- 172.21.15.148:0/1837907509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a006c630 msgr2=0x7f19a0067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a006c630 0x7f19a0067660 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f19900099b0 tx=0x7f1990009250 comp rx=0 tx=0).stop 2023-10-10T20:25:13.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 -- 172.21.15.148:0/1837907509 shutdown_connections 2023-10-10T20:25:13.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a0067ba0 0x7f19a0159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 --2- 172.21.15.148:0/1837907509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a006c630 0x7f19a0067660 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 -- 172.21.15.148:0/1837907509 >> 172.21.15.148:0/1837907509 conn(0x7f19a006fe50 msgr2=0x7f19a0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:13.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 -- 172.21.15.148:0/1837907509 shutdown_connections 2023-10-10T20:25:13.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.230+0000 7f19a6d47700 1 -- 172.21.15.148:0/1837907509 wait complete. 2023-10-10T20:25:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.231+0000 7f19a6d47700 1 Processor -- start 2023-10-10T20:25:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.231+0000 7f19a6d47700 1 -- start start 2023-10-10T20:25:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19a006c630 0x7f19a00c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a00b0890 0x7f19a014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a014e560 con 0x7f19a0067ba0 2023-10-10T20:25:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00c2d40 con 0x7f19a006c630 2023-10-10T20:25:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00c2410 con 0x7f19a00b0890 2023-10-10T20:25:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43732/0 (socket says 172.21.15.148:43732) 2023-10-10T20:25:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 -- 172.21.15.148:0/4180211589 learned_addr learned my addr 172.21.15.148:0/4180211589 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f1997fff700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19a006c630 0x7f19a00c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a52e4700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a00b0890 0x7f19a014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 -- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a00b0890 msgr2=0x7f19a014e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a00b0890 0x7f19a014e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 -- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19a006c630 msgr2=0x7f19a00c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19a006c630 0x7f19a00c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 -- 172.21.15.148:0/4180211589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19900094f0 con 0x7f19a0067ba0 2023-10-10T20:25:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a4ae3700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c910 tx=0x7f199c001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c012630 con 0x7f19a0067ba0 2023-10-10T20:25:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c0127b0 con 0x7f19a0067ba0 2023-10-10T20:25:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a00c26f0 con 0x7f19a0067ba0 2023-10-10T20:25:13.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c012b70 con 0x7f19a0067ba0 2023-10-10T20:25:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.232+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a015a640 con 0x7f19a0067ba0 2023-10-10T20:25:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.233+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a004c3c0 con 0x7f19a0067ba0 2023-10-10T20:25:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.234+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f199c003030 con 0x7f19a0067ba0 2023-10-10T20:25:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.234+0000 7f1994ff9700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 0x7f198c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.234+0000 7f1997fff700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 0x7f198c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.234+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f199c084480 con 0x7f19a0067ba0 2023-10-10T20:25:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.235+0000 7f1997fff700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 0x7f198c0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19900099b0 tx=0x7f1990017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.238+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f199c0516f0 con 0x7f19a0067ba0 2023-10-10T20:25:13.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.464+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f19a006cdc0 con 0x7f19a0067ba0 2023-10-10T20:25:13.469 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:25:13.464+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:25:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.469+0000 7f1994ff9700 1 -- 172.21.15.148:0/4180211589 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f199c054d10 con 0x7f19a0067ba0 2023-10-10T20:25:13.473 INFO:tasks.ceph.mds.a.smithi148.stderr: -3313> 2023-10-10T20:25:01.530+0000 7fcffe5ef700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:25:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 msgr2=0x7f198c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 0x7f198c0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19900099b0 tx=0x7f1990017000 comp rx=0 tx=0).stop 2023-10-10T20:25:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 msgr2=0x7f19a00c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c910 tx=0x7f199c001700 comp rx=0 tx=0).stop 2023-10-10T20:25:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 shutdown_connections 2023-10-10T20:25:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f198c066f30 0x7f198c0693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19a00b0890 0x7f19a014e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19a0067ba0 0x7f19a00c31d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 --2- 172.21.15.148:0/4180211589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19a006c630 0x7f19a00c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 >> 172.21.15.148:0/4180211589 conn(0x7f19a006fe50 msgr2=0x7f19a015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 shutdown_connections 2023-10-10T20:25:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.472+0000 7f19a6d47700 1 -- 172.21.15.148:0/4180211589 wait complete. 2023-10-10T20:25:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:25:13.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:25:13.497 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:25:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.604+0000 7f394aa4e700 1 Processor -- start 2023-10-10T20:25:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.604+0000 7f394aa4e700 1 -- start start 2023-10-10T20:25:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f394aa4e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3944159ec0 0x7f39440af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f394aa4e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440b0790 con 0x7f394406acb0 2023-10-10T20:25:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f394aa4e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440ce1f0 con 0x7f394415a290 2023-10-10T20:25:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f394aa4e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440b0b20 con 0x7f3944159ec0 2023-10-10T20:25:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f3948feb700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f394406acb0 0x7f39440cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48366/0 (socket says 172.21.15.148:48366) 2023-10-10T20:25:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.605+0000 7f3948feb700 1 -- 172.21.15.148:0/3769671616 learned_addr learned my addr 172.21.15.148:0/3769671616 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:13.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3943fff700 1 --2- 172.21.15.148:0/3769671616 >> v2:172.21.15.161:3300/0 conn(0x7f3944159ec0 0x7f39440af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655901987 0 0) 0x7f39440b0790 con 0x7f394406acb0 2023-10-10T20:25:13.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f392c003570 con 0x7f394406acb0 2023-10-10T20:25:13.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3481146742 0 0) 0x7f392c003570 con 0x7f394406acb0 2023-10-10T20:25:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 >> v2:172.21.15.161:3300/0 conn(0x7f3944159ec0 msgr2=0x7f39440af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 --2- 172.21.15.148:0/3769671616 >> v2:172.21.15.161:3300/0 conn(0x7f3944159ec0 0x7f39440af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 >> v1:172.21.15.161:6789/0 conn(0x7f394415a290 legacy=0x7f394406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440b04b0 con 0x7f394406acb0 2023-10-10T20:25:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f393c0017f0 con 0x7f394406acb0 2023-10-10T20:25:13.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 >> v1:172.21.15.148:6789/0 conn(0x7f394406acb0 legacy=0x7f39440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a60 0x7f392c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f392c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0051c0 0x7f392c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440b0b20 con 0x7f392c004810 2023-10-10T20:25:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00c160 con 0x7f392c0051c0 2023-10-10T20:25:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00c320 con 0x7f392c003a60 2023-10-10T20:25:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f392c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3943fff700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a60 0x7f392c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3948feb700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0051c0 0x7f392c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 -- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a60 msgr2=0x7f392c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a60 0x7f392c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 -- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0051c0 msgr2=0x7f392c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0051c0 0x7f392c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f39437fe700 1 -- 172.21.15.148:0/3769671616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c003570 con 0x7f392c004810 2023-10-10T20:25:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.606+0000 7f3948feb700 1 -- 172.21.15.148:0/3769671616 reap_dead start 2023-10-10T20:25:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f39437fe700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f392c004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f39380018b0 tx=0x7f3938007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54da8914db52bcb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938012420 con 0x7f392c004810 2023-10-10T20:25:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39380125a0 con 0x7f392c004810 2023-10-10T20:25:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f3942ffd700 1 -- 172.21.15.148:0/3769671616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938012960 con 0x7f392c004810 2023-10-10T20:25:13.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 -- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 msgr2=0x7f392c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f392c004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f39380018b0 tx=0x7f3938007d30 comp rx=0 tx=0).stop 2023-10-10T20:25:13.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 -- 172.21.15.148:0/3769671616 shutdown_connections 2023-10-10T20:25:13.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3769671616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f392c004c80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 -- 172.21.15.148:0/3769671616 >> 172.21.15.148:0/3769671616 conn(0x7f39440d1880 msgr2=0x7f39440cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:13.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 -- 172.21.15.148:0/3769671616 shutdown_connections 2023-10-10T20:25:13.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.607+0000 7f394aa4e700 1 -- 172.21.15.148:0/3769671616 wait complete. 2023-10-10T20:25:13.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.608+0000 7f394aa4e700 1 Processor -- start 2023-10-10T20:25:13.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 -- start start 2023-10-10T20:25:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3944157f40 0x7f39440b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39440b6260 0x7f3944153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944158ac0 con 0x7f392c004810 2023-10-10T20:25:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394415a660 con 0x7f3944157f40 2023-10-10T20:25:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f394aa4e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394416e2e0 con 0x7f39440b6260 2023-10-10T20:25:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f3943fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f3943fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43768/0 (socket says 172.21.15.148:43768) 2023-10-10T20:25:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f3943fff700 1 -- 172.21.15.148:0/3003636129 learned_addr learned my addr 172.21.15.148:0/3003636129 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f39437fe700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3944157f40 0x7f39440b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.609+0000 7f3948feb700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39440b6260 0x7f3944153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 -- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39440b6260 msgr2=0x7f3944153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39440b6260 0x7f3944153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 -- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3944157f40 msgr2=0x7f39440b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3944157f40 0x7f39440b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 -- 172.21.15.148:0/3003636129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39441583d0 con 0x7f392c004810 2023-10-10T20:25:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f3943fff700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3934011d40 tx=0x7f39340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393400fac0 con 0x7f392c004810 2023-10-10T20:25:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393400fc20 con 0x7f392c004810 2023-10-10T20:25:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394415a8b0 con 0x7f392c004810 2023-10-10T20:25:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934014440 con 0x7f392c004810 2023-10-10T20:25:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.610+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394415ae70 con 0x7f392c004810 2023-10-10T20:25:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.611+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394404c3c0 con 0x7f392c004810 2023-10-10T20:25:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.611+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3934014870 con 0x7f392c004810 2023-10-10T20:25:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.612+0000 7f392bfff700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 0x7f39300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.612+0000 7f39437fe700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 0x7f39300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.612+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3934018030 con 0x7f392c004810 2023-10-10T20:25:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.612+0000 7f39437fe700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 0x7f39300693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394415dac0 tx=0x7f3938015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.616+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3934052260 con 0x7f392c004810 2023-10-10T20:25:13.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:13.841+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f394415b1e0 con 0x7f392c004810 2023-10-10T20:25:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.469+0000 7f392bfff700 1 -- 172.21.15.148:0/3003636129 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3934055880 con 0x7f392c004810 2023-10-10T20:25:14.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 msgr2=0x7f39300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 0x7f39300693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394415dac0 tx=0x7f3938015000 comp rx=0 tx=0).stop 2023-10-10T20:25:14.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 msgr2=0x7f394416dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3934011d40 tx=0x7f39340091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 shutdown_connections 2023-10-10T20:25:14.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3930066f30 0x7f39300693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39440b6260 0x7f3944153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f392c004810 0x7f394416dda0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 --2- 172.21.15.148:0/3003636129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3944157f40 0x7f39440b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 >> 172.21.15.148:0/3003636129 conn(0x7f39440d1880 msgr2=0x7f39441555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:14.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 shutdown_connections 2023-10-10T20:25:14.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.472+0000 7f394aa4e700 1 -- 172.21.15.148:0/3003636129 wait complete. 2023-10-10T20:25:14.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:25:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.645+0000 7fcc64c62700 1 Processor -- start 2023-10-10T20:25:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.645+0000 7fcc64c62700 1 -- start start 2023-10-10T20:25:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc64c62700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc600d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc64c62700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc600d08e0 con 0x7fcc600d0d40 2023-10-10T20:25:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc64c62700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc60157e90 con 0x7fcc6015ac90 2023-10-10T20:25:14.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc64c62700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6015b0d0 con 0x7fcc60157ac0 2023-10-10T20:25:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc55d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcc600d0d40 0x7fcc6015a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48380/0 (socket says 172.21.15.148:48380) 2023-10-10T20:25:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.646+0000 7fcc55d9b700 1 -- 172.21.15.148:0/3223355533 learned_addr learned my addr 172.21.15.148:0/3223355533 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc600d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640467627 0 0) 0x7fcc48001110 con 0x7fcc600d0d40 2023-10-10T20:25:14.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc4c003570 con 0x7fcc600d0d40 2023-10-10T20:25:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 919341612 0 0) 0x7fcc4c003570 con 0x7fcc600d0d40 2023-10-10T20:25:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 msgr2=0x7fcc600d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc600d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 >> v1:172.21.15.161:6789/0 conn(0x7fcc6015ac90 legacy=0x7fcc6015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6015b2f0 con 0x7fcc600d0d40 2023-10-10T20:25:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fcc48001600 con 0x7fcc600d0d40 2023-10-10T20:25:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 >> v1:172.21.15.148:6789/0 conn(0x7fcc600d0d40 legacy=0x7fcc6015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c003a90 0x7fcc4c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc4c004810 0x7fcc4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6015b0d0 con 0x7fcc4c004810 2023-10-10T20:25:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00bb10 con 0x7fcc4c003a90 2023-10-10T20:25:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00bcd0 con 0x7fcc60157ac0 2023-10-10T20:25:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc4c004810 0x7fcc4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c003a90 0x7fcc4c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc5ed9d700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc55d9b700 1 -- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 msgr2=0x7fcc4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc60157ac0 0x7fcc4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc55d9b700 1 -- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c003a90 msgr2=0x7fcc4c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:14.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.647+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c003a90 0x7fcc4c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc55d9b700 1 -- 172.21.15.148:0/3223355533 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c003570 con 0x7fcc4c004810 2023-10-10T20:25:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc4c004810 0x7fcc4c004c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c5f0 tx=0x7fcc48007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e287d3a87c9622b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800cdf0 con 0x7fcc4c004810 2023-10-10T20:25:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48005920 con 0x7fcc4c004810 2023-10-10T20:25:14.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc5dd9b700 1 -- 172.21.15.148:0/3223355533 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48005ce0 con 0x7fcc4c004810 2023-10-10T20:25:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 -- 172.21.15.148:0/3223355533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc4c004810 msgr2=0x7fcc4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 --2- 172.21.15.148:0/3223355533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc4c004810 0x7fcc4c004c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c5f0 tx=0x7fcc48007a50 comp rx=0 tx=0).stop 2023-10-10T20:25:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc5ed9d700 1 -- 172.21.15.148:0/3223355533 reap_dead start 2023-10-10T20:25:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 -- 172.21.15.148:0/3223355533 shutdown_connections 2023-10-10T20:25:14.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 -- 172.21.15.148:0/3223355533 >> 172.21.15.148:0/3223355533 conn(0x7fcc600680f0 msgr2=0x7fcc600b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 -- 172.21.15.148:0/3223355533 shutdown_connections 2023-10-10T20:25:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.648+0000 7fcc64c62700 1 -- 172.21.15.148:0/3223355533 wait complete. 2023-10-10T20:25:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.649+0000 7fcc64c62700 1 Processor -- start 2023-10-10T20:25:14.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 -- start start 2023-10-10T20:25:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 0x7fcc6015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc600b5880 0x7fcc6015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6015cd80 con 0x7fcc6013f5b0 2023-10-10T20:25:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc600b5d20 con 0x7fcc4c004810 2023-10-10T20:25:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc64c62700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6015df80 con 0x7fcc600b5880 2023-10-10T20:25:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc55d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc55d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43784/0 (socket says 172.21.15.148:43784) 2023-10-10T20:25:14.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc55d9b700 1 -- 172.21.15.148:0/36003609 learned_addr learned my addr 172.21.15.148:0/36003609 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.650+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 0x7fcc6015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc5ed9d700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc600b5880 0x7fcc6015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 -- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc600b5880 msgr2=0x7fcc6015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc600b5880 0x7fcc6015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 -- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 msgr2=0x7fcc6015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 0x7fcc6015da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 -- 172.21.15.148:0/36003609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6015eea0 con 0x7fcc6013f5b0 2023-10-10T20:25:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 0x7fcc6015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc55d9b700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c004770 tx=0x7fcc48015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc480072e0 con 0x7fcc6013f5b0 2023-10-10T20:25:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48007480 con 0x7fcc6013f5b0 2023-10-10T20:25:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6015cf60 con 0x7fcc6013f5b0 2023-10-10T20:25:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4801e400 con 0x7fcc6013f5b0 2023-10-10T20:25:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.651+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6015d520 con 0x7fcc6013f5b0 2023-10-10T20:25:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.652+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6015cd80 con 0x7fcc6013f5b0 2023-10-10T20:25:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.652+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc48019030 con 0x7fcc6013f5b0 2023-10-10T20:25:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.653+0000 7fcc56ffd700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.653+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fcc48085650 con 0x7fcc6013f5b0 2023-10-10T20:25:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.653+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.653+0000 7fcc5e59c700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006f00 tx=0x7fcc50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.657+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc48052870 con 0x7fcc6013f5b0 2023-10-10T20:25:14.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:14.840+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcc600d08e0 con 0x7fcc6013f5b0 2023-10-10T20:25:15.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.595+0000 7fcc56ffd700 1 -- 172.21.15.148:0/36003609 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcc600d08e0 con 0x7fcc6013f5b0 2023-10-10T20:25:15.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 msgr2=0x7fcc400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006f00 tx=0x7fcc50008040 comp rx=0 tx=0).stop 2023-10-10T20:25:15.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 msgr2=0x7fcc600b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c004770 tx=0x7fcc48015000 comp rx=0 tx=0).stop 2023-10-10T20:25:15.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 shutdown_connections 2023-10-10T20:25:15.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcc600b5880 0x7fcc6015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc4c004810 0x7fcc6015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 --2- 172.21.15.148:0/36003609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6013f5b0 0x7fcc600b5340 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 >> 172.21.15.148:0/36003609 conn(0x7fcc600680f0 msgr2=0x7fcc6015b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:15.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 shutdown_connections 2023-10-10T20:25:15.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.598+0000 7fcc64c62700 1 -- 172.21.15.148:0/36003609 wait complete. 2023-10-10T20:25:15.603 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:25:15.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:25:15.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 Processor -- start 2023-10-10T20:25:15.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 -- start start 2023-10-10T20:25:15.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 0x7f2a9c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2a9c1595f0 0x7f2a9c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c15e5b0 con 0x7f2a9c0cdbc0 2023-10-10T20:25:15.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c15e890 con 0x7f2a9c1595f0 2023-10-10T20:25:15.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.734+0000 7f2aa0f36700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0ceea0 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2a9c1595f0 0x7f2a9c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9affd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2a9c0cdbc0 0x7f2a9c15de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48396/0 (socket says 172.21.15.148:48396) 2023-10-10T20:25:15.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9affd700 1 -- 172.21.15.148:0/1719874302 learned_addr learned my addr 172.21.15.148:0/1719874302 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:15.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 -- 172.21.15.148:0/1719874302 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 msgr2=0x7f2a9c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:15.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9b7fe700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 0x7f2a9c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 0x7f2a9c15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 -- 172.21.15.148:0/1719874302 >> v1:172.21.15.148:6789/0 conn(0x7f2a9c0cdbc0 legacy=0x7f2a9c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9a7fc700 1 -- 172.21.15.148:0/1719874302 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240357279 0 0) 0x7f2a9c15e5b0 con 0x7f2a9c0cdbc0 2023-10-10T20:25:15.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 -- 172.21.15.148:0/1719874302 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0cf180 con 0x7f2a9c1595f0 2023-10-10T20:25:15.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9b7fe700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 0x7f2a9c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:15.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9bfff700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c159a40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009e60 tx=0x7f2a94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a556ae31292382 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:15.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9a7fc700 1 -- 172.21.15.148:0/1719874302 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400e070 con 0x7f2a9c1595f0 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9a7fc700 1 -- 172.21.15.148:0/1719874302 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9400bc30 con 0x7f2a9c1595f0 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.735+0000 7f2a9a7fc700 1 -- 172.21.15.148:0/1719874302 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94012430 con 0x7f2a9c1595f0 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1719874302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c1595f0 msgr2=0x7f2a9c159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c159a40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009e60 tx=0x7f2a94009780 comp rx=0 tx=0).stop 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1719874302 shutdown_connections 2023-10-10T20:25:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c159a40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1719874302 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cd7f0 0x7f2a9c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1719874302 >> 172.21.15.148:0/1719874302 conn(0x7f2a9c068bf0 msgr2=0x7f2a9c068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:15.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1719874302 shutdown_connections 2023-10-10T20:25:15.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.736+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1719874302 wait complete. 2023-10-10T20:25:15.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.737+0000 7f2aa0f36700 1 Processor -- start 2023-10-10T20:25:15.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.737+0000 7f2aa0f36700 1 -- start start 2023-10-10T20:25:15.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cdbc0 0x7f2a9c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c157b20 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b50a0 con 0x7f2a9c1595f0 2023-10-10T20:25:15.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2aa0f36700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0b5380 con 0x7f2a9c0cdbc0 2023-10-10T20:25:15.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43802/0 (socket says 172.21.15.148:43802) 2023-10-10T20:25:15.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 -- 172.21.15.148:0/1211257972 learned_addr learned my addr 172.21.15.148:0/1211257972 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:15.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9affd700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cdbc0 0x7f2a9c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9bfff700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 -- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cdbc0 msgr2=0x7f2a9c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cdbc0 0x7f2a9c0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 -- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a9c1595f0 msgr2=0x7f2a9c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 -- 172.21.15.148:0/1211257972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009a20 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.738+0000 7f2a9b7fe700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c830 tx=0x7f2a8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:15.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c011600 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c011760 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c0c7e50 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c011b20 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c0b43b0 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.739+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c0b55a0 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.740+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a8c00a3e0 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.740+0000 7f2a937fe700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:15.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.740+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f2a8c084890 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.740+0000 7f2a9affd700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:15.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.741+0000 7f2a9affd700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15d2b0 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:15.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.745+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a8c051c60 con 0x7f2a9c0cd7f0 2023-10-10T20:25:15.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:15.927+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a9c0c7c90 con 0x7f2a9c0cd7f0 2023-10-10T20:25:16.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.605+0000 7f2a937fe700 1 -- 172.21.15.148:0/1211257972 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2a8c0150d0 con 0x7f2a9c0cd7f0 2023-10-10T20:25:16.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 msgr2=0x7f2a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c15d2b0 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2023-10-10T20:25:16.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 msgr2=0x7f2a9c1575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c830 tx=0x7f2a8c008040 comp rx=0 tx=0).stop 2023-10-10T20:25:16.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 shutdown_connections 2023-10-10T20:25:16.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a9c1595f0 0x7f2a9c0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0cd7f0 0x7f2a9c1575e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 --2- 172.21.15.148:0/1211257972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a9c0cdbc0 0x7f2a9c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 >> 172.21.15.148:0/1211257972 conn(0x7f2a9c068bf0 msgr2=0x7f2a9c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 shutdown_connections 2023-10-10T20:25:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.608+0000 7f2aa0f36700 1 -- 172.21.15.148:0/1211257972 wait complete. 2023-10-10T20:25:16.613 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:25:16.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:25:16.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.741+0000 7f11ae22f700 1 Processor -- start 2023-10-10T20:25:16.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.741+0000 7f11ae22f700 1 -- start start 2023-10-10T20:25:16.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ae22f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f11a80ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ae22f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11a80acb00 0x7f11a80ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ae22f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80cae80 con 0x7f11a8155f70 2023-10-10T20:25:16.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ae22f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a80ad080 con 0x7f11a80acb00 2023-10-10T20:25:16.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ae22f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80ca340 con 0x7f11a8156340 2023-10-10T20:25:16.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ad22d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f11a8155f70 0x7f11a806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48402/0 (socket says 172.21.15.148:48402) 2023-10-10T20:25:16.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ad22d700 1 -- 172.21.15.148:0/2647212190 learned_addr learned my addr 172.21.15.148:0/2647212190 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:16.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236338255 0 0) 0x7f11a80cae80 con 0x7f11a8155f70 2023-10-10T20:25:16.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1190004680 con 0x7f11a8155f70 2023-10-10T20:25:16.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2647212190 >> v2:172.21.15.161:3300/0 conn(0x7f11a80acb00 0x7f11a80ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11aca2c700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f11a80ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2212104038 0 0) 0x7f1190004680 con 0x7f11a8155f70 2023-10-10T20:25:16.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 msgr2=0x7f11a80ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:16.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f11a80ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 >> v2:172.21.15.161:3300/0 conn(0x7f11a80acb00 msgr2=0x7f11a80ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:16.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 --2- 172.21.15.148:0/2647212190 >> v2:172.21.15.161:3300/0 conn(0x7f11a80acb00 0x7f11a80ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80ca550 con 0x7f11a8155f70 2023-10-10T20:25:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.742+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f11a0001970 con 0x7f11a8155f70 2023-10-10T20:25:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 >> v1:172.21.15.148:6789/0 conn(0x7f11a8155f70 legacy=0x7f11a806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190004b70 0x7f1190004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f1190005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190005e60 0x7f11900062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80ca340 con 0x7f1190005e60 2023-10-10T20:25:16.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119000cb70 con 0x7f1190004b70 2023-10-10T20:25:16.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119000ccb0 con 0x7f11a8156340 2023-10-10T20:25:16.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190005e60 0x7f11900062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ad22d700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190004b70 0x7f1190004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11aca2c700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f1190005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 -- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 msgr2=0x7f1190005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:16.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8156340 0x7f1190005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 -- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190004b70 msgr2=0x7f1190004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:16.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1190004b70 0x7f1190004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 -- 172.21.15.148:0/2647212190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190004680 con 0x7f1190005e60 2023-10-10T20:25:16.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190005e60 0x7f11900062d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1198009c00 tx=0x7f11980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ef5ac1d2a1da3b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:16.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800e6b0 con 0x7f1190005e60 2023-10-10T20:25:16.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119800e810 con 0x7f1190005e60 2023-10-10T20:25:16.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.743+0000 7f11a7fff700 1 -- 172.21.15.148:0/2647212190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800ebd0 con 0x7f1190005e60 2023-10-10T20:25:16.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 -- 172.21.15.148:0/2647212190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190005e60 msgr2=0x7f11900062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2647212190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190005e60 0x7f11900062d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1198009c00 tx=0x7f11980091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ada2e700 1 -- 172.21.15.148:0/2647212190 reap_dead start 2023-10-10T20:25:16.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 -- 172.21.15.148:0/2647212190 shutdown_connections 2023-10-10T20:25:16.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 -- 172.21.15.148:0/2647212190 >> 172.21.15.148:0/2647212190 conn(0x7f11a8067660 msgr2=0x7f11a8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:16.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 -- 172.21.15.148:0/2647212190 shutdown_connections 2023-10-10T20:25:16.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.744+0000 7f11ae22f700 1 -- 172.21.15.148:0/2647212190 wait complete. 2023-10-10T20:25:16.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.745+0000 7f11ae22f700 1 Processor -- start 2023-10-10T20:25:16.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.745+0000 7f11ae22f700 1 -- start start 2023-10-10T20:25:16.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8151a00 0x7f11a80695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 0x7f11a8171df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8069f50 con 0x7f11a8149760 2023-10-10T20:25:16.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81517a0 con 0x7f11a8151a00 2023-10-10T20:25:16.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ae22f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8150940 con 0x7f11a8069ae0 2023-10-10T20:25:16.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43820/0 (socket says 172.21.15.148:43820) 2023-10-10T20:25:16.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 -- 172.21.15.148:0/2550171830 learned_addr learned my addr 172.21.15.148:0/2550171830 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:16.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11aca2c700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8151a00 0x7f11a80695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 0x7f11a8171df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 -- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 msgr2=0x7f11a8171df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 0x7f11a8171df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 -- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8151a00 msgr2=0x7f11a80695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8151a00 0x7f11a80695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 -- 172.21.15.148:0/2550171830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8150c20 con 0x7f11a8149760 2023-10-10T20:25:16.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ada2e700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 0x7f11a8171df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:16.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.746+0000 7f11ad22d700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f11a000ff50 tx=0x7f11a0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:16.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.747+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0005d80 con 0x7f11a8149760 2023-10-10T20:25:16.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.747+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a000da20 con 0x7f11a8149760 2023-10-10T20:25:16.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.747+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000dde0 con 0x7f11a8149760 2023-10-10T20:25:16.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.747+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a8150f00 con 0x7f11a8149760 2023-10-10T20:25:16.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.747+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a81514c0 con 0x7f11a8149760 2023-10-10T20:25:16.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.748+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a8171b10 con 0x7f11a8149760 2023-10-10T20:25:16.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.748+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11a0007320 con 0x7f11a8149760 2023-10-10T20:25:16.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.748+0000 7f11a57fa700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 0x7f1194069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.748+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f11a00845a0 con 0x7f11a8149760 2023-10-10T20:25:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.748+0000 7f11aca2c700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 0x7f1194069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.749+0000 7f11aca2c700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 0x7f1194069460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f11a806a620 tx=0x7f119c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.753+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11a8171b10 con 0x7f11a8149760 2023-10-10T20:25:16.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.936+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f11a8171b10 con 0x7f11a8149760 2023-10-10T20:25:16.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.939+0000 7f11a57fa700 1 -- 172.21.15.148:0/2550171830 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11a8171b10 con 0x7f11a8149760 2023-10-10T20:25:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.941+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 msgr2=0x7f1194069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.941+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 0x7f1194069460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f11a806a620 tx=0x7f119c008040 comp rx=0 tx=0).stop 2023-10-10T20:25:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 msgr2=0x7f11a816e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f11a000ff50 tx=0x7f11a0009040 comp rx=0 tx=0).stop 2023-10-10T20:25:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 shutdown_connections 2023-10-10T20:25:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1194066fb0 0x7f1194069460 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11a8069ae0 0x7f11a8171df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11a8149760 0x7f11a816e510 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 --2- 172.21.15.148:0/2550171830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11a8151a00 0x7f11a80695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:16.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 >> 172.21.15.148:0/2550171830 conn(0x7f11a8067660 msgr2=0x7f11a806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:16.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 shutdown_connections 2023-10-10T20:25:16.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:16.942+0000 7f11ae22f700 1 -- 172.21.15.148:0/2550171830 wait complete. 2023-10-10T20:25:16.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-10T20:25:17.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.073+0000 7f4573050700 1 Processor -- start 2023-10-10T20:25:17.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4573050700 1 -- start start 2023-10-10T20:25:17.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4573050700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f456c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4573050700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c15cbb0 con 0x7f456c157870 2023-10-10T20:25:17.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4573050700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c15bfa0 con 0x7f456c15d580 2023-10-10T20:25:17.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4573050700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c15bdb0 con 0x7f456c15a620 2023-10-10T20:25:17.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.074+0000 7f4570dec700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f456c157870 0x7f456c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48410/0 (socket says 172.21.15.148:48410) 2023-10-10T20:25:17.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f4570dec700 1 -- 172.21.15.148:0/3386301841 learned_addr learned my addr 172.21.15.148:0/3386301841 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:17.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f4563fff700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f456c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647563440 0 0) 0x7f456c15cbb0 con 0x7f456c157870 2023-10-10T20:25:17.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4554003570 con 0x7f456c157870 2023-10-10T20:25:17.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2929242489 0 0) 0x7f4554003570 con 0x7f456c157870 2023-10-10T20:25:17.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 msgr2=0x7f456c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:17.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f456c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 >> v1:172.21.15.161:6789/0 conn(0x7f456c15d580 legacy=0x7f456c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c15c280 con 0x7f456c157870 2023-10-10T20:25:17.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f4568001470 con 0x7f456c157870 2023-10-10T20:25:17.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 >> v1:172.21.15.148:6789/0 conn(0x7f456c157870 legacy=0x7f456c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f4554003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554004010 0x7f4554004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4554004d50 0x7f45540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455400bbf0 con 0x7f4554004d50 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455400bd30 con 0x7f4554004010 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455400bef0 con 0x7f456c15a620 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f45715ed700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4554004d50 0x7f45540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f4570dec700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f4554003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.075+0000 7f4563fff700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554004010 0x7f4554004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 -- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 msgr2=0x7f4554003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c15a620 0x7f4554003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 -- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554004010 msgr2=0x7f4554004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554004010 0x7f4554004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 -- 172.21.15.148:0/3386301841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554003570 con 0x7f4554004d50 2023-10-10T20:25:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4554004d50 0x7f45540051c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f456c15d9e0 tx=0x7f4564001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7610d118bc1fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564001950 con 0x7f4554004d50 2023-10-10T20:25:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4564004030 con 0x7f4554004d50 2023-10-10T20:25:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45637fe700 1 -- 172.21.15.148:0/3386301841 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456400cb00 con 0x7f4554004d50 2023-10-10T20:25:17.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 -- 172.21.15.148:0/3386301841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4554004d50 msgr2=0x7f45540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 --2- 172.21.15.148:0/3386301841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4554004d50 0x7f45540051c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f456c15d9e0 tx=0x7f4564001640 comp rx=0 tx=0).stop 2023-10-10T20:25:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f45715ed700 1 -- 172.21.15.148:0/3386301841 reap_dead start 2023-10-10T20:25:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 -- 172.21.15.148:0/3386301841 shutdown_connections 2023-10-10T20:25:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 -- 172.21.15.148:0/3386301841 >> 172.21.15.148:0/3386301841 conn(0x7f456c068ce0 msgr2=0x7f456c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:17.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 -- 172.21.15.148:0/3386301841 shutdown_connections 2023-10-10T20:25:17.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.076+0000 7f4573050700 1 -- 172.21.15.148:0/3386301841 wait complete. 2023-10-10T20:25:17.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.077+0000 7f4573050700 1 Processor -- start 2023-10-10T20:25:17.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 -- start start 2023-10-10T20:25:17.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c149680 0x7f456c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f456c1593d0 0x7f456c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c159db0 con 0x7f456c0b1520 2023-10-10T20:25:17.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c14b060 con 0x7f456c1593d0 2023-10-10T20:25:17.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4573050700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c14a450 con 0x7f456c149680 2023-10-10T20:25:17.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.078+0000 7f4563fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43832/0 (socket says 172.21.15.148:43832) 2023-10-10T20:25:17.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 -- 172.21.15.148:0/610657053 learned_addr learned my addr 172.21.15.148:0/610657053 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4570dec700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c149680 0x7f456c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f45715ed700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f456c1593d0 0x7f456c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 -- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c149680 msgr2=0x7f456c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c149680 0x7f456c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 -- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f456c1593d0 msgr2=0x7f456c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f456c1593d0 0x7f456c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 -- 172.21.15.148:0/610657053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c14a730 con 0x7f456c0b1520 2023-10-10T20:25:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4563fff700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f455c011b40 tx=0x7f455c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00f8e0 con 0x7f456c0b1520 2023-10-10T20:25:17.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c00fa40 con 0x7f456c0b1520 2023-10-10T20:25:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c14aa10 con 0x7f456c0b1520 2023-10-10T20:25:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00fdc0 con 0x7f456c0b1520 2023-10-10T20:25:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.079+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c149a50 con 0x7f456c0b1520 2023-10-10T20:25:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.080+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c04c3c0 con 0x7f456c0b1520 2023-10-10T20:25:17.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.081+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f455c014650 con 0x7f456c0b1520 2023-10-10T20:25:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.081+0000 7f4560ff9700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.081+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f455c018030 con 0x7f456c0b1520 2023-10-10T20:25:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.081+0000 7f4570dec700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.081+0000 7f4570dec700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 0x7f45580693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f456c0cf2c0 tx=0x7f4568001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.085+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f455c0521c0 con 0x7f456c0b1520 2023-10-10T20:25:17.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.345+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f456c14b1a0 con 0x7f456c0b1520 2023-10-10T20:25:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.606+0000 7f4560ff9700 1 -- 172.21.15.148:0/610657053 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v308) v1 ==== 128+0+0 (secure 0 0 0) 0x7f455c0220c0 con 0x7f456c0b1520 2023-10-10T20:25:17.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 msgr2=0x7f45580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 0x7f45580693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f456c0cf2c0 tx=0x7f4568001e50 comp rx=0 tx=0).stop 2023-10-10T20:25:17.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 msgr2=0x7f456c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f455c011b40 tx=0x7f455c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 shutdown_connections 2023-10-10T20:25:17.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4558066f30 0x7f45580693e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f456c1593d0 0x7f456c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f456c149680 0x7f456c0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 --2- 172.21.15.148:0/610657053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456c0b1520 0x7f456c0b1990 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 >> 172.21.15.148:0/610657053 conn(0x7f456c068ce0 msgr2=0x7f456c0b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:17.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 shutdown_connections 2023-10-10T20:25:17.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.608+0000 7f4573050700 1 -- 172.21.15.148:0/610657053 wait complete. 2023-10-10T20:25:17.623 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-10T20:25:17.623 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-10T20:25:17.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-10T20:25:17.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 Processor -- start 2023-10-10T20:25:17.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 -- start start 2023-10-10T20:25:17.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69f8157c40 0x7f69f8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f815cbb0 con 0x7f69f8157870 2023-10-10T20:25:17.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f815bfa0 con 0x7f69f815a620 2023-10-10T20:25:17.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.736+0000 7f69fd0ad700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f815bdb0 con 0x7f69f8157c40 2023-10-10T20:25:17.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69f8157c40 0x7f69f8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f6d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f69f8157870 0x7f69f815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48424/0 (socket says 172.21.15.148:48424) 2023-10-10T20:25:17.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69f8157c40 0x7f69f8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43840/0 (socket says 172.21.15.148:43840) 2023-10-10T20:25:17.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f6d9d700 1 -- 172.21.15.148:0/2074449990 learned_addr learned my addr 172.21.15.148:0/2074449990 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:17.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 -- 172.21.15.148:0/2074449990 >> v1:172.21.15.161:6789/0 conn(0x7f69f815a620 legacy=0x7f69f815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:17.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 -- 172.21.15.148:0/2074449990 >> v1:172.21.15.148:6789/0 conn(0x7f69f8157870 legacy=0x7f69f815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 -- 172.21.15.148:0/2074449990 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f815c280 con 0x7f69f8157c40 2023-10-10T20:25:17.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f5d9b700 1 -- 172.21.15.148:0/2074449990 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191662959 0 0) 0x7f69f815cbb0 con 0x7f69f8157870 2023-10-10T20:25:17.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f759e700 1 --2- 172.21.15.148:0/2074449990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157c40 0x7f69f8158110 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009e60 tx=0x7f69f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee3a989ed91013a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f5d9b700 1 -- 172.21.15.148:0/2074449990 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000e070 con 0x7f69f8157c40 2023-10-10T20:25:17.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f5d9b700 1 -- 172.21.15.148:0/2074449990 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f000bbf0 con 0x7f69f8157c40 2023-10-10T20:25:17.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.737+0000 7f69f5d9b700 1 -- 172.21.15.148:0/2074449990 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0012430 con 0x7f69f8157c40 2023-10-10T20:25:17.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 -- 172.21.15.148:0/2074449990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157c40 msgr2=0x7f69f8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/2074449990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157c40 0x7f69f8158110 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009e60 tx=0x7f69f0009780 comp rx=0 tx=0).stop 2023-10-10T20:25:17.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 -- 172.21.15.148:0/2074449990 shutdown_connections 2023-10-10T20:25:17.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/2074449990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157c40 0x7f69f8158110 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 -- 172.21.15.148:0/2074449990 >> 172.21.15.148:0/2074449990 conn(0x7f69f8068ce0 msgr2=0x7f69f80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:17.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 -- 172.21.15.148:0/2074449990 shutdown_connections 2023-10-10T20:25:17.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.738+0000 7f69fd0ad700 1 -- 172.21.15.148:0/2074449990 wait complete. 2023-10-10T20:25:17.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.739+0000 7f69fd0ad700 1 Processor -- start 2023-10-10T20:25:17.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.739+0000 7f69fd0ad700 1 -- start start 2023-10-10T20:25:17.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f8157c40 0x7f69f80b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f815a620 0x7f69f8151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81511e0 con 0x7f69f8157870 2023-10-10T20:25:17.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80b2610 con 0x7f69f815a620 2023-10-10T20:25:17.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80b2050 con 0x7f69f8157c40 2023-10-10T20:25:17.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43846/0 (socket says 172.21.15.148:43846) 2023-10-10T20:25:17.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 -- 172.21.15.148:0/1631604210 learned_addr learned my addr 172.21.15.148:0/1631604210 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:17.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 -- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f8157c40 msgr2=0x7f69f80b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f8157c40 0x7f69f80b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 -- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f815a620 msgr2=0x7f69f8151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:17.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f815a620 0x7f69f8151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 -- 172.21.15.148:0/1631604210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009a20 con 0x7f69f8157870 2023-10-10T20:25:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69f6d9d700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c810 tx=0x7f69e8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005b30 con 0x7f69f8157870 2023-10-10T20:25:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8005c90 con 0x7f69f8157870 2023-10-10T20:25:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80b2330 con 0x7f69f8157870 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8007300 con 0x7f69f8157870 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.740+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f814f1e0 con 0x7f69f8157870 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.741+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f81514c0 con 0x7f69f8157870 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.742+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69e801b400 con 0x7f69f8157870 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.742+0000 7f69deffd700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 0x7f69e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.742+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f69e8083d30 con 0x7f69f8157870 2023-10-10T20:25:17.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.742+0000 7f69f659c700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 0x7f69e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:17.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.743+0000 7f69f659c700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 0x7f69e40693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f69f80d0f50 tx=0x7f69ec006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:17.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:17.747+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e8051320 con 0x7f69f8157870 2023-10-10T20:25:18.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.006+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f69f804c3c0 con 0x7f69f8157870 2023-10-10T20:25:18.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.607+0000 7f69deffd700 1 -- 172.21.15.148:0/1631604210 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v309) v1 ==== 143+0+0 (secure 0 0 0) 0x7f69e8054940 con 0x7f69f8157870 2023-10-10T20:25:18.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 msgr2=0x7f69e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 0x7f69e40693e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f69f80d0f50 tx=0x7f69ec006c60 comp rx=0 tx=0).stop 2023-10-10T20:25:18.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 msgr2=0x7f69f814eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c810 tx=0x7f69e8007df0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 shutdown_connections 2023-10-10T20:25:18.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f69e4066f30 0x7f69e40693e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f815a620 0x7f69f8151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69f8157870 0x7f69f814eca0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 --2- 172.21.15.148:0/1631604210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f8157c40 0x7f69f80b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.609+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 >> 172.21.15.148:0/1631604210 conn(0x7f69f8068ce0 msgr2=0x7f69f80b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:18.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.610+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 shutdown_connections 2023-10-10T20:25:18.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.610+0000 7f69fd0ad700 1 -- 172.21.15.148:0/1631604210 wait complete. 2023-10-10T20:25:18.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:25:18.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.739+0000 7f5427ffc700 1 Processor -- start 2023-10-10T20:25:18.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 -- start start 2023-10-10T20:25:18.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 0x7f542015da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 0x7f542015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542015a6d0 con 0x7f542015b0a0 2023-10-10T20:25:18.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542015e960 con 0x7f5420150d30 2023-10-10T20:25:18.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.740+0000 7f5427ffc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200ad290 con 0x7f54200aa870 2023-10-10T20:25:18.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 0x7f542015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425d98700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 0x7f542015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5426599700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f542015b0a0 0x7f5420070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48770/0 (socket says 172.21.15.148:48770) 2023-10-10T20:25:18.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5426599700 1 -- 172.21.15.148:0/3913373275 learned_addr learned my addr 172.21.15.148:0/3913373275 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:18.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 -- 172.21.15.148:0/3913373275 >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 msgr2=0x7f542015da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:18.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 --2- 172.21.15.148:0/3913373275 >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 0x7f542015da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 -- 172.21.15.148:0/3913373275 >> v1:172.21.15.161:6789/0 conn(0x7f542015b0a0 legacy=0x7f5420070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 -- 172.21.15.148:0/3913373275 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542015e0c0 con 0x7f54200aa870 2023-10-10T20:25:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425d98700 1 --2- 172.21.15.148:0/3913373275 >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 0x7f542015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5425597700 1 --2- 172.21.15.148:0/3913373275 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 0x7f542015ab60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54100098e0 tx=0x7f54100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dba2354d7e41c4f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:18.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5424d96700 1 -- 172.21.15.148:0/3913373275 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000b640 con 0x7f54200aa870 2023-10-10T20:25:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5424d96700 1 -- 172.21.15.148:0/3913373275 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000b7a0 con 0x7f54200aa870 2023-10-10T20:25:18.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.741+0000 7f5424d96700 1 -- 172.21.15.148:0/3913373275 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000bb40 con 0x7f54200aa870 2023-10-10T20:25:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 -- 172.21.15.148:0/3913373275 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 msgr2=0x7f542015ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 --2- 172.21.15.148:0/3913373275 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 0x7f542015ab60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54100098e0 tx=0x7f54100091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 -- 172.21.15.148:0/3913373275 shutdown_connections 2023-10-10T20:25:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 --2- 172.21.15.148:0/3913373275 >> v2:172.21.15.161:3300/0 conn(0x7f5420150d30 0x7f542015da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 --2- 172.21.15.148:0/3913373275 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54200aa870 0x7f542015ab60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 -- 172.21.15.148:0/3913373275 >> 172.21.15.148:0/3913373275 conn(0x7f54200ce580 msgr2=0x7f54200af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:18.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 -- 172.21.15.148:0/3913373275 shutdown_connections 2023-10-10T20:25:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.742+0000 7f5427ffc700 1 -- 172.21.15.148:0/3913373275 wait complete. 2023-10-10T20:25:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.743+0000 7f5427ffc700 1 Processor -- start 2023-10-10T20:25:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.743+0000 7f5427ffc700 1 -- start start 2023-10-10T20:25:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5420150d30 0x7f542014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f542015b0a0 0x7f542014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420148610 con 0x7f54200aa870 2023-10-10T20:25:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542016ada0 con 0x7f5420150d30 2023-10-10T20:25:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5427ffc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5420147fc0 con 0x7f542015b0a0 2023-10-10T20:25:18.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43862/0 (socket says 172.21.15.148:43862) 2023-10-10T20:25:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 -- 172.21.15.148:0/1188190335 learned_addr learned my addr 172.21.15.148:0/1188190335 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425597700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5420150d30 0x7f542014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5426599700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f542015b0a0 0x7f542014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 -- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f542015b0a0 msgr2=0x7f542014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f542015b0a0 0x7f542014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 -- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5420150d30 msgr2=0x7f542014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5420150d30 0x7f542014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 -- 172.21.15.148:0/1188190335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c009a20 con 0x7f54200aa870 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5425d98700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c170 tx=0x7f541c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.744+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00cc50 con 0x7f54200aa870 2023-10-10T20:25:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.745+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00f040 con 0x7f54200aa870 2023-10-10T20:25:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.745+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410009420 con 0x7f54200aa870 2023-10-10T20:25:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.745+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c014480 con 0x7f54200aa870 2023-10-10T20:25:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.745+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5420139960 con 0x7f54200aa870 2023-10-10T20:25:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.746+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542004c3c0 con 0x7f54200aa870 2023-10-10T20:25:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.746+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f541c0147f0 con 0x7f54200aa870 2023-10-10T20:25:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.747+0000 7f5415ffb700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 0x7f54040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.747+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f541c0850b0 con 0x7f54200aa870 2023-10-10T20:25:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.747+0000 7f5425597700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 0x7f54040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.747+0000 7f5425597700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 0x7f54040693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f54100093f0 tx=0x7f5410017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:18.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.751+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f541c0560b0 con 0x7f54200aa870 2023-10-10T20:25:18.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.934+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f54200b0240 con 0x7f54200aa870 2023-10-10T20:25:18.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.938+0000 7f5415ffb700 1 -- 172.21.15.148:0/1188190335 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f541c055c40 con 0x7f54200aa870 2023-10-10T20:25:18.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.940+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 msgr2=0x7f54040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.940+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 0x7f54040693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f54100093f0 tx=0x7f5410017000 comp rx=0 tx=0).stop 2023-10-10T20:25:18.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.940+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 msgr2=0x7f5420139420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:18.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.940+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c170 tx=0x7f541c00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:18.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 shutdown_connections 2023-10-10T20:25:18.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5404066f30 0x7f54040693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f542015b0a0 0x7f542014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54200aa870 0x7f5420139420 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 --2- 172.21.15.148:0/1188190335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5420150d30 0x7f542014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 >> 172.21.15.148:0/1188190335 conn(0x7f54200ce580 msgr2=0x7f542015bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 shutdown_connections 2023-10-10T20:25:18.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:18.941+0000 7f5427ffc700 1 -- 172.21.15.148:0/1188190335 wait complete. 2023-10-10T20:25:18.955 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:25:18.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:25:18.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:25:19.036 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:25:19.036 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:25:19.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:19.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.149+0000 7fd254195700 1 Processor -- start 2023-10-10T20:25:19.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 -- start start 2023-10-10T20:25:19.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd24c1568d0 0x7fd24c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd24c0ad9b0 0x7fd24c06a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c067ba0 con 0x7fd24c156ca0 2023-10-10T20:25:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c15aa90 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd254195700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c067870 con 0x7fd24c1568d0 2023-10-10T20:25:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.150+0000 7fd251730700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd24c0ad9b0 0x7fd24c06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd252732700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd24c156ca0 0x7fd24c0c8090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48434/0 (socket says 172.21.15.148:48434) 2023-10-10T20:25:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd252732700 1 -- 172.21.15.148:0/1576328527 learned_addr learned my addr 172.21.15.148:0/1576328527 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd251730700 1 -- 172.21.15.148:0/1576328527 >> v2:172.21.15.161:3300/0 conn(0x7fd24c1568d0 msgr2=0x7fd24c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd251730700 1 --2- 172.21.15.148:0/1576328527 >> v2:172.21.15.161:3300/0 conn(0x7fd24c1568d0 0x7fd24c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd251730700 1 -- 172.21.15.148:0/1576328527 >> v1:172.21.15.148:6789/0 conn(0x7fd24c156ca0 legacy=0x7fd24c0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd251730700 1 -- 172.21.15.148:0/1576328527 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0c8ca0 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd250f2f700 1 -- 172.21.15.148:0/1576328527 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072028025 0 0) 0x7fd24c067ba0 con 0x7fd24c156ca0 2023-10-10T20:25:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd251730700 1 --2- 172.21.15.148:0/1576328527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c0ad9b0 0x7fd24c06a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0b3bd0 tx=0x7fd23c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b8ecf068b6fabc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd250f2f700 1 -- 172.21.15.148:0/1576328527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00b6b0 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd250f2f700 1 -- 172.21.15.148:0/1576328527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00b810 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.151+0000 7fd250f2f700 1 -- 172.21.15.148:0/1576328527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00bbb0 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 -- 172.21.15.148:0/1576328527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c0ad9b0 msgr2=0x7fd24c06a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 --2- 172.21.15.148:0/1576328527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c0ad9b0 0x7fd24c06a360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0b3bd0 tx=0x7fd23c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 -- 172.21.15.148:0/1576328527 shutdown_connections 2023-10-10T20:25:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 --2- 172.21.15.148:0/1576328527 >> v2:172.21.15.161:3300/0 conn(0x7fd24c1568d0 0x7fd24c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 --2- 172.21.15.148:0/1576328527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c0ad9b0 0x7fd24c06a360 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 -- 172.21.15.148:0/1576328527 >> 172.21.15.148:0/1576328527 conn(0x7fd24c0cda50 msgr2=0x7fd24c0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 -- 172.21.15.148:0/1576328527 shutdown_connections 2023-10-10T20:25:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.152+0000 7fd254195700 1 -- 172.21.15.148:0/1576328527 wait complete. 2023-10-10T20:25:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.153+0000 7fd254195700 1 Processor -- start 2023-10-10T20:25:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.153+0000 7fd254195700 1 -- start start 2023-10-10T20:25:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.153+0000 7fd254195700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 0x7fd24c0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.153+0000 7fd254195700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd24c1568d0 0x7fd24c0c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b2350 con 0x7fd24c156ca0 2023-10-10T20:25:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0b2840 con 0x7fd24c1568d0 2023-10-10T20:25:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c4470 con 0x7fd24c0ad9b0 2023-10-10T20:25:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43878/0 (socket says 172.21.15.148:43878) 2023-10-10T20:25:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 -- 172.21.15.148:0/4285143002 learned_addr learned my addr 172.21.15.148:0/4285143002 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:19.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd251730700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd24c1568d0 0x7fd24c0c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd251f31700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 0x7fd24c0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 -- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 msgr2=0x7fd24c0c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 0x7fd24c0c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 -- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd24c1568d0 msgr2=0x7fd24c0c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd24c1568d0 0x7fd24c0c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 -- 172.21.15.148:0/4285143002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c009420 con 0x7fd24c156ca0 2023-10-10T20:25:19.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd251f31700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 0x7fd24c0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:19.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd252732700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd24800cc90 tx=0x7fd248001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24800cea0 con 0x7fd24c156ca0 2023-10-10T20:25:19.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24800ab30 con 0x7fd24c156ca0 2023-10-10T20:25:19.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c16ba00 con 0x7fd24c156ca0 2023-10-10T20:25:19.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480070f0 con 0x7fd24c156ca0 2023-10-10T20:25:19.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.154+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c16c240 con 0x7fd24c156ca0 2023-10-10T20:25:19.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.155+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c04c3c0 con 0x7fd24c156ca0 2023-10-10T20:25:19.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.156+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2480074a0 con 0x7fd24c156ca0 2023-10-10T20:25:19.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.156+0000 7fd241ffb700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.156+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd2480846a0 con 0x7fd24c156ca0 2023-10-10T20:25:19.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.156+0000 7fd251f31700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.156+0000 7fd251f31700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 0x7fd2300693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd244006fd0 tx=0x7fd244008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.160+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd248051c90 con 0x7fd24c156ca0 2023-10-10T20:25:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.383+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd24c0c8f80 con 0x7fd24c156ca0 2023-10-10T20:25:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.384+0000 7fd241ffb700 1 -- 172.21.15.148:0/4285143002 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fd2480552b0 con 0x7fd24c156ca0 2023-10-10T20:25:19.387 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:19.387 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/2180774111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2180774111},{"type":"v1","addr":"172.21.15.161:6834","nonce":2180774111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3750569024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":3750569024},{"type":"v1","addr":"172.21.15.161:6835","nonce":3750569024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/262614475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":262614475},{"type":"v1","addr":"172.21.15.148:6837","nonce":262614475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4189260437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4189260437},{"type":"v1","addr":"172.21.15.148:6836","nonce":4189260437}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[]} 2023-10-10T20:25:19.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 msgr2=0x7fd2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 0x7fd2300693e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd244006fd0 tx=0x7fd244008040 comp rx=0 tx=0).stop 2023-10-10T20:25:19.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 msgr2=0x7fd24c1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd24800cc90 tx=0x7fd248001a20 comp rx=0 tx=0).stop 2023-10-10T20:25:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 shutdown_connections 2023-10-10T20:25:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd230066f30 0x7fd2300693e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd24c156ca0 0x7fd24c1593c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd24c0ad9b0 0x7fd24c0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 --2- 172.21.15.148:0/4285143002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd24c1568d0 0x7fd24c0c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 >> 172.21.15.148:0/4285143002 conn(0x7fd24c0cda50 msgr2=0x7fd24c155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 shutdown_connections 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.386+0000 7fd254195700 1 -- 172.21.15.148:0/4285143002 wait complete. 2023-10-10T20:25:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 59 2023-10-10T20:25:19.400 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:25:19.401 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:25:19.401 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:25:19.401 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:25:19.402 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:25:19.402 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:25:19.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:25:19.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:25:19.404 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:25:19.404 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:25:19.405 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:25:19.405 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:25:19.405 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:25:19.406 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:25:19.406 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:25:19.407 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:25:19.411 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:25:19.411 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:25:19.413 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:25:19.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:25:19.416 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:25:19.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:25:19.419 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:25:19.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:25:19.455 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:25:19.455 DEBUG:teuthology.parallel:result is None 2023-10-10T20:25:19.456 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:25:19.457 DEBUG:teuthology.parallel:result is None 2023-10-10T20:25:19.461 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:25:19.461 DEBUG:teuthology.parallel:result is None 2023-10-10T20:25:19.462 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:25:19.463 DEBUG:teuthology.parallel:result is None 2023-10-10T20:25:19.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:25:19.529 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:25:19.530 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:25:19.542 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:25:19.543 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:25:19.544 INFO:tasks.ceph.mds.d.smithi161.stderr:2023-10-10T20:25:19.541+0000 7fcfb5003700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:25:19.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.585+0000 7f1099ed8700 1 Processor -- start 2023-10-10T20:25:19.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.585+0000 7f1099ed8700 1 -- start start 2023-10-10T20:25:19.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1099ed8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f109415a170 0x7f1094068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1099ed8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10940694a0 con 0x7f1094159da0 2023-10-10T20:25:19.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1099ed8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109415ea70 con 0x7f1094069040 2023-10-10T20:25:19.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1099ed8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10940b34b0 con 0x7f109415a170 2023-10-10T20:25:19.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1098ed6700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1094159da0 0x7f10940b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48450/0 (socket says 172.21.15.148:48450) 2023-10-10T20:25:19.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f1098ed6700 1 -- 172.21.15.148:0/2010490924 learned_addr learned my addr 172.21.15.148:0/2010490924 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:19.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f108bfff700 1 --2- 172.21.15.148:0/2010490924 >> v2:172.21.15.161:3300/0 conn(0x7f109415a170 0x7f1094068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220020350 0 0) 0x7f1090001110 con 0x7f1094159da0 2023-10-10T20:25:19.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.586+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c003570 con 0x7f1094159da0 2023-10-10T20:25:19.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218803784 0 0) 0x7f109415ea70 con 0x7f1094069040 2023-10-10T20:25:19.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1090001110 con 0x7f1094069040 2023-10-10T20:25:19.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161224025 0 0) 0x7f107c003570 con 0x7f1094159da0 2023-10-10T20:25:19.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 >> v2:172.21.15.161:3300/0 conn(0x7f109415a170 msgr2=0x7f1094068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:19.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 --2- 172.21.15.148:0/2010490924 >> v2:172.21.15.161:3300/0 conn(0x7f109415a170 0x7f1094068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 >> v1:172.21.15.161:6789/0 conn(0x7f1094069040 legacy=0x7f10940af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940b3990 con 0x7f1094159da0 2023-10-10T20:25:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f1090001600 con 0x7f1094159da0 2023-10-10T20:25:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 >> v1:172.21.15.148:6789/0 conn(0x7f1094159da0 legacy=0x7f10940b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f107c003c40 0x7f107c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c004810 0x7f107c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 0x7f107c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940b34b0 con 0x7f107c005280 2023-10-10T20:25:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00c250 con 0x7f107c004810 2023-10-10T20:25:19.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00c410 con 0x7f107c003c40 2023-10-10T20:25:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108bfff700 1 --2- 172.21.15.148:0/2010490924 >> v2:172.21.15.161:3300/0 conn(0x7f109415a170 0x7f1094068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 0x7f107c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f1098ed6700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f107c003c40 0x7f107c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108bfff700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c004810 0x7f107c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 -- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f107c003c40 msgr2=0x7f107c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f107c003c40 0x7f107c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 -- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c004810 msgr2=0x7f107c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c004810 0x7f107c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 -- 172.21.15.148:0/2010490924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c003570 con 0x7f107c005280 2023-10-10T20:25:19.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f1098ed6700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f107c003c40 0x7f107c004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:19.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 -- 172.21.15.148:0/2010490924 reap_dead start 2023-10-10T20:25:19.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f10996d7700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 0x7f107c00ba60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f108c002160 tx=0x7f108c0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d92ba71a1eb7ae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00c980 con 0x7f107c005280 2023-10-10T20:25:19.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c00cae0 con 0x7f107c005280 2023-10-10T20:25:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.587+0000 7f108b7fe700 1 -- 172.21.15.148:0/2010490924 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00ce80 con 0x7f107c005280 2023-10-10T20:25:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 -- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 msgr2=0x7f107c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 0x7f107c00ba60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f108c002160 tx=0x7f108c0017e0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 -- 172.21.15.148:0/2010490924 shutdown_connections 2023-10-10T20:25:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 --2- 172.21.15.148:0/2010490924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f107c005280 0x7f107c00ba60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 -- 172.21.15.148:0/2010490924 >> 172.21.15.148:0/2010490924 conn(0x7f10940d0bc0 msgr2=0x7f10940d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 -- 172.21.15.148:0/2010490924 shutdown_connections 2023-10-10T20:25:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.588+0000 7f1099ed8700 1 -- 172.21.15.148:0/2010490924 wait complete. 2023-10-10T20:25:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.589+0000 7f1099ed8700 1 Processor -- start 2023-10-10T20:25:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.589+0000 7f1099ed8700 1 -- start start 2023-10-10T20:25:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c005280 0x7f10940aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1094158cf0 0x7f1094159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10941596a0 con 0x7f10940c5de0 2023-10-10T20:25:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940c8f60 con 0x7f107c005280 2023-10-10T20:25:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1099ed8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940afa60 con 0x7f1094158cf0 2023-10-10T20:25:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43924/0 (socket says 172.21.15.148:43924) 2023-10-10T20:25:19.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 -- 172.21.15.148:0/1762516183 learned_addr learned my addr 172.21.15.148:0/1762516183 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f1098ed6700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c005280 0x7f10940aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f10996d7700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1094158cf0 0x7f1094159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 -- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1094158cf0 msgr2=0x7f1094159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1094158cf0 0x7f1094159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 -- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c005280 msgr2=0x7f10940aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c005280 0x7f10940aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.590+0000 7f108bfff700 1 -- 172.21.15.148:0/1762516183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940af780 con 0x7f10940c5de0 2023-10-10T20:25:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f108bfff700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1084011da0 tx=0x7f10840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084011ea0 con 0x7f10940c5de0 2023-10-10T20:25:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400fbc0 con 0x7f10940c5de0 2023-10-10T20:25:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940c8910 con 0x7f10940c5de0 2023-10-10T20:25:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108401d480 con 0x7f10940c5de0 2023-10-10T20:25:19.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.591+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10940af470 con 0x7f10940c5de0 2023-10-10T20:25:19.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.592+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10940b0500 con 0x7f10940c5de0 2023-10-10T20:25:19.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.592+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f108401d830 con 0x7f10940c5de0 2023-10-10T20:25:19.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.593+0000 7f1088ff9700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 0x7f10800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:19.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.593+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1084018070 con 0x7f10940c5de0 2023-10-10T20:25:19.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.593+0000 7f1098ed6700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 0x7f10800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:19.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.593+0000 7f1098ed6700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 0x7f10800693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1090000ff0 tx=0x7f1090007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:19.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.597+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1084052520 con 0x7f10940c5de0 2023-10-10T20:25:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:19.798+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f10940c90a0 con 0x7f10940c5de0 2023-10-10T20:25:20.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.621+0000 7f1088ff9700 1 -- 172.21.15.148:0/1762516183 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f108405b070 con 0x7f10940c5de0 2023-10-10T20:25:20.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.623+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 msgr2=0x7f10800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.623+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 0x7f10800693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1090000ff0 tx=0x7f1090007040 comp rx=0 tx=0).stop 2023-10-10T20:25:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.623+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 msgr2=0x7f1094157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.623+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1084011da0 tx=0x7f10840091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 shutdown_connections 2023-10-10T20:25:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1080066f30 0x7f10800693e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1094158cf0 0x7f1094159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f107c005280 0x7f10940aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 --2- 172.21.15.148:0/1762516183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940c5de0 0x7f1094157fb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 >> 172.21.15.148:0/1762516183 conn(0x7f10940d0bc0 msgr2=0x7f10940ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 shutdown_connections 2023-10-10T20:25:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.624+0000 7f1099ed8700 1 -- 172.21.15.148:0/1762516183 wait complete. 2023-10-10T20:25:20.629 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:25:20.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:25:20.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.753+0000 7fe197c5c700 1 Processor -- start 2023-10-10T20:25:20.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe197c5c700 1 -- start start 2023-10-10T20:25:20.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe197c5c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe19006ac30 0x7fe190164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe197c5c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19015c360 con 0x7fe190157ba0 2023-10-10T20:25:20.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe197c5c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19015b790 con 0x7fe1901577d0 2023-10-10T20:25:20.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe197c5c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19015d800 con 0x7fe19006ac30 2023-10-10T20:25:20.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.754+0000 7fe1951f7700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe190157ba0 0x7fe19006a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48466/0 (socket says 172.21.15.148:48466) 2023-10-10T20:25:20.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1951f7700 1 -- 172.21.15.148:0/769587929 learned_addr learned my addr 172.21.15.148:0/769587929 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:20.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1961f9700 1 --2- 172.21.15.148:0/769587929 >> v2:172.21.15.161:3300/0 conn(0x7fe19006ac30 0x7fe190164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895452049 0 0) 0x7fe19015c360 con 0x7fe190157ba0 2023-10-10T20:25:20.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003570 con 0x7fe190157ba0 2023-10-10T20:25:20.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438491520 0 0) 0x7fe17c003570 con 0x7fe190157ba0 2023-10-10T20:25:20.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 >> v2:172.21.15.161:3300/0 conn(0x7fe19006ac30 msgr2=0x7fe190164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:20.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 --2- 172.21.15.148:0/769587929 >> v2:172.21.15.161:3300/0 conn(0x7fe19006ac30 0x7fe190164e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 >> v1:172.21.15.161:6789/0 conn(0x7fe1901577d0 legacy=0x7fe1900cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19015ba30 con 0x7fe190157ba0 2023-10-10T20:25:20.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fe180001470 con 0x7fe190157ba0 2023-10-10T20:25:20.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 >> v1:172.21.15.148:6789/0 conn(0x7fe190157ba0 legacy=0x7fe19006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 0x7fe17c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe17c004810 0x7fe17c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c0051c0 0x7fe17c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19015d800 con 0x7fe17c003a60 2023-10-10T20:25:20.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00c160 con 0x7fe17c0051c0 2023-10-10T20:25:20.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00c320 con 0x7fe17c004810 2023-10-10T20:25:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1959f8700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 0x7fe17c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1951f7700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe17c004810 0x7fe17c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1961f9700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c0051c0 0x7fe17c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1959f8700 1 -- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe17c004810 msgr2=0x7fe17c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:20.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1959f8700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe17c004810 0x7fe17c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1959f8700 1 -- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c0051c0 msgr2=0x7fe17c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:20.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.755+0000 7fe1959f8700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c0051c0 0x7fe17c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1961f9700 1 -- 172.21.15.148:0/769587929 reap_dead start 2023-10-10T20:25:20.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1959f8700 1 -- 172.21.15.148:0/769587929 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c003570 con 0x7fe17c003a60 2023-10-10T20:25:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1951f7700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe17c004810 0x7fe17c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1959f8700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 0x7fe17c003e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001fc0 tx=0x7fe18c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c67f7266986836b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c001950 con 0x7fe17c003a60 2023-10-10T20:25:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c00c8b0 con 0x7fe17c003a60 2023-10-10T20:25:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe1949f6700 1 -- 172.21.15.148:0/769587929 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00cc50 con 0x7fe17c003a60 2023-10-10T20:25:20.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 -- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 msgr2=0x7fe17c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 0x7fe17c003e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001fc0 tx=0x7fe18c001640 comp rx=0 tx=0).stop 2023-10-10T20:25:20.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 -- 172.21.15.148:0/769587929 shutdown_connections 2023-10-10T20:25:20.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 --2- 172.21.15.148:0/769587929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe17c003a60 0x7fe17c003e70 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 -- 172.21.15.148:0/769587929 >> 172.21.15.148:0/769587929 conn(0x7fe190068ce0 msgr2=0x7fe1900690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 -- 172.21.15.148:0/769587929 shutdown_connections 2023-10-10T20:25:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.756+0000 7fe197c5c700 1 -- 172.21.15.148:0/769587929 wait complete. 2023-10-10T20:25:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 Processor -- start 2023-10-10T20:25:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 -- start start 2023-10-10T20:25:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c003a60 0x7fe190152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe19014d8d0 0x7fe19014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1900b05c0 con 0x7fe1900d0c60 2023-10-10T20:25:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19014e2b0 con 0x7fe17c003a60 2023-10-10T20:25:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe197c5c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe190153460 con 0x7fe19014d8d0 2023-10-10T20:25:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe1951f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe1951f7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43944/0 (socket says 172.21.15.148:43944) 2023-10-10T20:25:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.758+0000 7fe1951f7700 1 -- 172.21.15.148:0/2609721873 learned_addr learned my addr 172.21.15.148:0/2609721873 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1961f9700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe19014d8d0 0x7fe19014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1959f8700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c003a60 0x7fe190152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 -- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe19014d8d0 msgr2=0x7fe19014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe19014d8d0 0x7fe19014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 -- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c003a60 msgr2=0x7fe190152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c003a60 0x7fe190152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 -- 172.21.15.148:0/2609721873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190153740 con 0x7fe1900d0c60 2023-10-10T20:25:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe1951f7700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe180000e10 tx=0x7fe180001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180012590 con 0x7fe1900d0c60 2023-10-10T20:25:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1800126f0 con 0x7fe1900d0c60 2023-10-10T20:25:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190153a20 con 0x7fe1900d0c60 2023-10-10T20:25:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180012a70 con 0x7fe1900d0c60 2023-10-10T20:25:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.759+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190153030 con 0x7fe1900d0c60 2023-10-10T20:25:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.760+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1900b0ac0 con 0x7fe1900d0c60 2023-10-10T20:25:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.760+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe18001d460 con 0x7fe1900d0c60 2023-10-10T20:25:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.761+0000 7fe185ffb700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 0x7fe1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.761+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe1800840a0 con 0x7fe1900d0c60 2023-10-10T20:25:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.761+0000 7fe1959f8700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 0x7fe1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.761+0000 7fe1959f8700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 0x7fe1740693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001390 tx=0x7fe18c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:20.765+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1900b0ac0 con 0x7fe1900d0c60 2023-10-10T20:25:21.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.018+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe1900b0ac0 con 0x7fe1900d0c60 2023-10-10T20:25:21.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.018+0000 7fe185ffb700 1 -- 172.21.15.148:0/2609721873 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe1900b0ac0 con 0x7fe1900d0c60 2023-10-10T20:25:21.021 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:21.021 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:25:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:25:21.022 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:25:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:25:21.023 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:25:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:25:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:25:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.025 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:25:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:25:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:25:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:25:21.027 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:25:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:25:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:25:21.029 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:25:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:25:21.030 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:25:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.031 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:25:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:25:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.032 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:25:21.033 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.034 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:25:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:25:21.035 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:25:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:25:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:25:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:25:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:25:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:25:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:25:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:25:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:25:21.044 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:25:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:25:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:25:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.045 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:25:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:25:21.046 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:25:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:25:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:25:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:25:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.048 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:25:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:25:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.050 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:25:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:25:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:25:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:25:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:25:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:25:21.052 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:25:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:25:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:25:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:25:21.054 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:25:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.055 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:25:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:25:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:25:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:25:21.056 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:25:21.057 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:25:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:25:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:25:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:25:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:25:21.058 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:21.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:21.059 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 msgr2=0x7fe1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 0x7fe1740693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001390 tx=0x7fe18c015000 comp rx=0 tx=0).stop 2023-10-10T20:25:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 msgr2=0x7fe190151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe180000e10 tx=0x7fe180001e50 comp rx=0 tx=0).stop 2023-10-10T20:25:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 shutdown_connections 2023-10-10T20:25:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe174066f30 0x7fe1740693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe19014d8d0 0x7fe19014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe17c003a60 0x7fe190152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 --2- 172.21.15.148:0/2609721873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1900d0c60 0x7fe190151110 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.020+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 >> 172.21.15.148:0/2609721873 conn(0x7fe190068ce0 msgr2=0x7fe1900692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.021+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 shutdown_connections 2023-10-10T20:25:21.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.021+0000 7fe197c5c700 1 -- 172.21.15.148:0/2609721873 wait complete. 2023-10-10T20:25:21.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:21.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.176+0000 7f1668c34700 1 Processor -- start 2023-10-10T20:25:21.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.176+0000 7f1668c34700 1 -- start start 2023-10-10T20:25:21.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1668c34700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1664069de0 0x7f166415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1668c34700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1664158160 0x7f166415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1668c34700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1664069060 con 0x7f16640c9560 2023-10-10T20:25:21.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1668c34700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166415c6e0 con 0x7f1664158160 2023-10-10T20:25:21.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1668c34700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1664069300 con 0x7f1664069de0 2023-10-10T20:25:21.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1662d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1664158160 0x7f166415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1662d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1664158160 0x7f166415c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43958/0 (socket says 172.21.15.148:43958) 2023-10-10T20:25:21.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.177+0000 7f1662d9d700 1 -- 172.21.15.148:0/523423832 learned_addr learned my addr 172.21.15.148:0/523423832 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:21.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1659d9b700 1 --2- 172.21.15.148:0/523423832 >> v2:172.21.15.161:3300/0 conn(0x7f1664069de0 0x7f166415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1662d9d700 1 -- 172.21.15.148:0/523423832 >> v2:172.21.15.161:3300/0 conn(0x7f1664069de0 msgr2=0x7f166415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1662d9d700 1 --2- 172.21.15.148:0/523423832 >> v2:172.21.15.161:3300/0 conn(0x7f1664069de0 0x7f166415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1662d9d700 1 -- 172.21.15.148:0/523423832 >> v1:172.21.15.161:6789/0 conn(0x7f16640c9560 legacy=0x7f1664157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1662d9d700 1 -- 172.21.15.148:0/523423832 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640695e0 con 0x7f1664158160 2023-10-10T20:25:21.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1662d9d700 1 --2- 172.21.15.148:0/523423832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664158160 0x7f166415c1a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f16640aeff0 tx=0x7f165c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb369ee05152ec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1661d9b700 1 -- 172.21.15.148:0/523423832 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c00e070 con 0x7f1664158160 2023-10-10T20:25:21.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1661d9b700 1 -- 172.21.15.148:0/523423832 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c00ba10 con 0x7f1664158160 2023-10-10T20:25:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1661d9b700 1 -- 172.21.15.148:0/523423832 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0092e0 con 0x7f1664158160 2023-10-10T20:25:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 -- 172.21.15.148:0/523423832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664158160 msgr2=0x7f166415c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 --2- 172.21.15.148:0/523423832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664158160 0x7f166415c1a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f16640aeff0 tx=0x7f165c009840 comp rx=0 tx=0).stop 2023-10-10T20:25:21.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 -- 172.21.15.148:0/523423832 shutdown_connections 2023-10-10T20:25:21.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 --2- 172.21.15.148:0/523423832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664158160 0x7f166415c1a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 --2- 172.21.15.148:0/523423832 >> v2:172.21.15.161:3300/0 conn(0x7f1664069de0 0x7f166415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.178+0000 7f1668c34700 1 -- 172.21.15.148:0/523423832 >> 172.21.15.148:0/523423832 conn(0x7f1664067660 msgr2=0x7f1664067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.179+0000 7f1668c34700 1 -- 172.21.15.148:0/523423832 shutdown_connections 2023-10-10T20:25:21.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.179+0000 7f1668c34700 1 -- 172.21.15.148:0/523423832 wait complete. 2023-10-10T20:25:21.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 Processor -- start 2023-10-10T20:25:21.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 -- start start 2023-10-10T20:25:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16640c9560 0x7f16640c0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1664158160 0x7f16640b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c1e20 con 0x7f1664069de0 2023-10-10T20:25:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664158600 con 0x7f1664158160 2023-10-10T20:25:21.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.180+0000 7f1668c34700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16640c0d80 con 0x7f16640c9560 2023-10-10T20:25:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43974/0 (socket says 172.21.15.148:43974) 2023-10-10T20:25:21.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 -- 172.21.15.148:0/4152549819 learned_addr learned my addr 172.21.15.148:0/4152549819 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:21.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f1662d9d700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1664158160 0x7f16640b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f1659d9b700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16640c9560 0x7f16640c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 -- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16640c9560 msgr2=0x7f16640c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16640c9560 0x7f16640c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 -- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1664158160 msgr2=0x7f16640b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1664158160 0x7f16640b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 -- 172.21.15.148:0/4152549819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c009ae0 con 0x7f1664069de0 2023-10-10T20:25:21.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f166259c700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f165400c5d0 tx=0x7f1654007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654011590 con 0x7f1664069de0 2023-10-10T20:25:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1654011710 con 0x7f1664069de0 2023-10-10T20:25:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16640c1060 con 0x7f1664069de0 2023-10-10T20:25:21.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654011ad0 con 0x7f1664069de0 2023-10-10T20:25:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.181+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664149340 con 0x7f1664069de0 2023-10-10T20:25:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.182+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16640c12a0 con 0x7f1664069de0 2023-10-10T20:25:21.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.183+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f165401c460 con 0x7f1664069de0 2023-10-10T20:25:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.183+0000 7f165affd700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 0x7f16440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.183+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1654083e60 con 0x7f1664069de0 2023-10-10T20:25:21.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.183+0000 7f1659d9b700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 0x7f16440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.183+0000 7f1659d9b700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 0x7f16440693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f16640c4180 tx=0x7f164c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.187+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16640c12a0 con 0x7f1664069de0 2023-10-10T20:25:21.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.371+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16640c1910 con 0x7f1664069de0 2023-10-10T20:25:21.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.371+0000 7f165affd700 1 -- 172.21.15.148:0/4152549819 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+12193 (secure 0 0 0) 0x7f16640c1910 con 0x7f1664069de0 2023-10-10T20:25:21.374 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:21.374 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":127,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:20.620288+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:21.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 msgr2=0x7f16440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 0x7f16440693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f16640c4180 tx=0x7f164c008040 comp rx=0 tx=0).stop 2023-10-10T20:25:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 msgr2=0x7f1664148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f165400c5d0 tx=0x7f1654007f60 comp rx=0 tx=0).stop 2023-10-10T20:25:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 shutdown_connections 2023-10-10T20:25:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1644066f30 0x7f16440693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1664158160 0x7f16640b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1664069de0 0x7f1664148e00 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 --2- 172.21.15.148:0/4152549819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16640c9560 0x7f16640c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 >> 172.21.15.148:0/4152549819 conn(0x7f1664067660 msgr2=0x7f16640cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 shutdown_connections 2023-10-10T20:25:21.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.374+0000 7f1668c34700 1 -- 172.21.15.148:0/4152549819 wait complete. 2023-10-10T20:25:21.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:25:21.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:25:21.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.504+0000 7f9868caf700 1 Processor -- start 2023-10-10T20:25:21.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.504+0000 7f9868caf700 1 -- start start 2023-10-10T20:25:21.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9868caf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f986415e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9868caf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640cae70 con 0x7f98640678b0 2023-10-10T20:25:21.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9868caf700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640cb150 con 0x7f98640c9390 2023-10-10T20:25:21.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9868caf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98640cb430 con 0x7f98640cbf10 2023-10-10T20:25:21.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f986259c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f98640678b0 0x7f986415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48482/0 (socket says 172.21.15.148:48482) 2023-10-10T20:25:21.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f986259c700 1 -- 172.21.15.148:0/1172446605 learned_addr learned my addr 172.21.15.148:0/1172446605 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9859d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f986415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360007551 0 0) 0x7f98640cae70 con 0x7f98640678b0 2023-10-10T20:25:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.505+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9850003570 con 0x7f98640678b0 2023-10-10T20:25:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964445205 0 0) 0x7f9850003570 con 0x7f98640678b0 2023-10-10T20:25:21.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 msgr2=0x7f986415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f986415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 >> v1:172.21.15.161:6789/0 conn(0x7f98640c9390 legacy=0x7f986415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640cb710 con 0x7f98640678b0 2023-10-10T20:25:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9854001470 con 0x7f98640678b0 2023-10-10T20:25:21.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 >> v1:172.21.15.148:6789/0 conn(0x7f98640678b0 legacy=0x7f986415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850003a90 0x7f9850003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850004810 0x7f9850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f98500051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640cb430 con 0x7f9850003a90 2023-10-10T20:25:21.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985000bb90 con 0x7f9850004810 2023-10-10T20:25:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985000bd50 con 0x7f98640cbf10 2023-10-10T20:25:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850003a90 0x7f9850003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9859d9b700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850004810 0x7f9850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9862d9d700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f98500051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 -- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 msgr2=0x7f98500051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640cbf10 0x7f98500051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 -- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850004810 msgr2=0x7f9850004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850004810 0x7f9850004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 -- 172.21.15.148:0/1172446605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850003570 con 0x7f9850003a90 2023-10-10T20:25:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f986259c700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850003a90 0x7f9850003ea0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f985400c3c0 tx=0x7f9854001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412cc0843d747b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400cc90 con 0x7f9850003a90 2023-10-10T20:25:21.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400cdf0 con 0x7f9850003a90 2023-10-10T20:25:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.506+0000 7f9861d9b700 1 -- 172.21.15.148:0/1172446605 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854005b50 con 0x7f9850003a90 2023-10-10T20:25:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 -- 172.21.15.148:0/1172446605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850003a90 msgr2=0x7f9850003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 --2- 172.21.15.148:0/1172446605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9850003a90 0x7f9850003ea0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f985400c3c0 tx=0x7f9854001e50 comp rx=0 tx=0).stop 2023-10-10T20:25:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9862d9d700 1 -- 172.21.15.148:0/1172446605 reap_dead start 2023-10-10T20:25:21.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 -- 172.21.15.148:0/1172446605 shutdown_connections 2023-10-10T20:25:21.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 -- 172.21.15.148:0/1172446605 >> 172.21.15.148:0/1172446605 conn(0x7f9864069870 msgr2=0x7f9864069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 -- 172.21.15.148:0/1172446605 shutdown_connections 2023-10-10T20:25:21.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.507+0000 7f9868caf700 1 -- 172.21.15.148:0/1172446605 wait complete. 2023-10-10T20:25:21.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.508+0000 7f9868caf700 1 Processor -- start 2023-10-10T20:25:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.508+0000 7f9868caf700 1 -- start start 2023-10-10T20:25:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850003a90 0x7f9864169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640b07c0 0x7f98640cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640cf030 con 0x7f98640cf4c0 2023-10-10T20:25:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986416a0c0 con 0x7f9850003a90 2023-10-10T20:25:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9868caf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986414ecd0 con 0x7f98640b07c0 2023-10-10T20:25:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43996/0 (socket says 172.21.15.148:43996) 2023-10-10T20:25:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 -- 172.21.15.148:0/2515658200 learned_addr learned my addr 172.21.15.148:0/2515658200 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f986259c700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850003a90 0x7f9864169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 -- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640b07c0 msgr2=0x7f98640cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9859d9b700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640b07c0 0x7f98640cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640b07c0 0x7f98640cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 -- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850003a90 msgr2=0x7f9864169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850003a90 0x7f9864169b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.509+0000 7f9862d9d700 1 -- 172.21.15.148:0/2515658200 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640ce420 con 0x7f98640cf4c0 2023-10-10T20:25:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f9862d9d700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f986406a380 tx=0x7f985c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c006db0 con 0x7f98640cf4c0 2023-10-10T20:25:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c003110 con 0x7f98640cf4c0 2023-10-10T20:25:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98640ce700 con 0x7f98640cf4c0 2023-10-10T20:25:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c001580 con 0x7f98640cf4c0 2023-10-10T20:25:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.510+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98640cecc0 con 0x7f98640cf4c0 2023-10-10T20:25:21.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.511+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98640cf1e0 con 0x7f98640cf4c0 2023-10-10T20:25:21.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.511+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f985c0019b0 con 0x7f98640cf4c0 2023-10-10T20:25:21.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.511+0000 7f985affd700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.511+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f985c0839f0 con 0x7f98640cf4c0 2023-10-10T20:25:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.511+0000 7f986259c700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.512+0000 7f986259c700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 0x7f98440693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9854001ed0 tx=0x7f9854015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.516+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f985c04c5a0 con 0x7f98640cf4c0 2023-10-10T20:25:21.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.700+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f986404c3c0 con 0x7f98640cf4c0 2023-10-10T20:25:21.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.700+0000 7f985affd700 1 -- 172.21.15.148:0/2515658200 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98640cf1e0 con 0x7f98640cf4c0 2023-10-10T20:25:21.703 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:25:21.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 msgr2=0x7f98440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 0x7f98440693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9854001ed0 tx=0x7f9854015000 comp rx=0 tx=0).stop 2023-10-10T20:25:21.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 msgr2=0x7f986414e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f986406a380 tx=0x7f985c009040 comp rx=0 tx=0).stop 2023-10-10T20:25:21.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 shutdown_connections 2023-10-10T20:25:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9844066f30 0x7f98440693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98640cf4c0 0x7f986414e270 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9850003a90 0x7f9864169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 --2- 172.21.15.148:0/2515658200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98640b07c0 0x7f98640cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 >> 172.21.15.148:0/2515658200 conn(0x7f9864069870 msgr2=0x7f98640b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.702+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 shutdown_connections 2023-10-10T20:25:21.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.703+0000 7f9868caf700 1 -- 172.21.15.148:0/2515658200 wait complete. 2023-10-10T20:25:21.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:25:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.831+0000 7fa9446fa700 1 Processor -- start 2023-10-10T20:25:21.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.831+0000 7fa9446fa700 1 -- start start 2023-10-10T20:25:21.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 0x7fa93c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa93c15a620 0x7fa93c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa93c15d580 0x7fa93c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c15cbb0 con 0x7fa93c15d580 2023-10-10T20:25:21.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c15bfa0 con 0x7fa93c15a620 2023-10-10T20:25:21.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa9446fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c15c280 con 0x7fa93c157870 2023-10-10T20:25:21.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa93c15d580 0x7fa93c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa93c15d580 0x7fa93c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44008/0 (socket says 172.21.15.148:44008) 2023-10-10T20:25:21.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 -- 172.21.15.148:0/2158493383 learned_addr learned my addr 172.21.15.148:0/2158493383 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:21.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa941c95700 1 --2- 172.21.15.148:0/2158493383 >> v2:172.21.15.161:3300/0 conn(0x7fa93c15a620 0x7fa93c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942496700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 0x7fa93c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 -- 172.21.15.148:0/2158493383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 msgr2=0x7fa93c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 0x7fa93c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 -- 172.21.15.148:0/2158493383 >> v2:172.21.15.161:3300/0 conn(0x7fa93c15a620 msgr2=0x7fa93c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:21.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 --2- 172.21.15.148:0/2158493383 >> v2:172.21.15.161:3300/0 conn(0x7fa93c15a620 0x7fa93c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 -- 172.21.15.148:0/2158493383 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c15c560 con 0x7fa93c15d580 2023-10-10T20:25:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942496700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 0x7fa93c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa942c97700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c158110 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0897d0 tx=0x7fa934009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dbaa506fa7ca69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.832+0000 7fa941494700 1 -- 172.21.15.148:0/2158493383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400b860 con 0x7fa93c15d580 2023-10-10T20:25:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa941494700 1 -- 172.21.15.148:0/2158493383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93400b9c0 con 0x7fa93c15d580 2023-10-10T20:25:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa941494700 1 -- 172.21.15.148:0/2158493383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400bd60 con 0x7fa93c15d580 2023-10-10T20:25:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 -- 172.21.15.148:0/2158493383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 msgr2=0x7fa93c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c158110 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0897d0 tx=0x7fa934009250 comp rx=0 tx=0).stop 2023-10-10T20:25:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 -- 172.21.15.148:0/2158493383 shutdown_connections 2023-10-10T20:25:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c158110 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 --2- 172.21.15.148:0/2158493383 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c157870 0x7fa93c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 --2- 172.21.15.148:0/2158493383 >> v2:172.21.15.161:3300/0 conn(0x7fa93c15a620 0x7fa93c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 -- 172.21.15.148:0/2158493383 >> 172.21.15.148:0/2158493383 conn(0x7fa93c068ce0 msgr2=0x7fa93c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:21.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 -- 172.21.15.148:0/2158493383 shutdown_connections 2023-10-10T20:25:21.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.833+0000 7fa9446fa700 1 -- 172.21.15.148:0/2158493383 wait complete. 2023-10-10T20:25:21.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.834+0000 7fa9446fa700 1 Processor -- start 2023-10-10T20:25:21.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.834+0000 7fa9446fa700 1 -- start start 2023-10-10T20:25:21.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93c157870 0x7fa93c16e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c15a620 0x7fa93c154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c16e1e0 con 0x7fa93c15d580 2023-10-10T20:25:21.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0b1f40 con 0x7fa93c157870 2023-10-10T20:25:21.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa9446fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0b21a0 con 0x7fa93c15a620 2023-10-10T20:25:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44016/0 (socket says 172.21.15.148:44016) 2023-10-10T20:25:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 -- 172.21.15.148:0/3085405468 learned_addr learned my addr 172.21.15.148:0/3085405468 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa941c95700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c15a620 0x7fa93c154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942496700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93c157870 0x7fa93c16e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 -- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c15a620 msgr2=0x7fa93c154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c15a620 0x7fa93c154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 -- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93c157870 msgr2=0x7fa93c16e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:21.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93c157870 0x7fa93c16e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:21.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.835+0000 7fa942c97700 1 -- 172.21.15.148:0/3085405468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9340094f0 con 0x7fa93c15d580 2023-10-10T20:25:21.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa942c97700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa93c06b530 tx=0x7fa934017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934011ea0 con 0x7fa93c15d580 2023-10-10T20:25:21.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93401a630 con 0x7fa93c15d580 2023-10-10T20:25:21.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93401aa10 con 0x7fa93c15d580 2023-10-10T20:25:21.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c0b2480 con 0x7fa93c15d580 2023-10-10T20:25:21.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.836+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c16de70 con 0x7fa93c15d580 2023-10-10T20:25:21.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.837+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c04c3c0 con 0x7fa93c15d580 2023-10-10T20:25:21.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.837+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa93402d440 con 0x7fa93c15d580 2023-10-10T20:25:21.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.837+0000 7fa9327fc700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 0x7fa9200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:21.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.837+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa934085d90 con 0x7fa93c15d580 2023-10-10T20:25:21.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.837+0000 7fa942496700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 0x7fa9200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:21.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.838+0000 7fa942496700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 0x7fa9200693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa93800d6e0 tx=0x7fa938007850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:21.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:21.842+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa934053350 con 0x7fa93c15d580 2023-10-10T20:25:22.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.024+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa93c15c840 con 0x7fa93c15d580 2023-10-10T20:25:22.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.025+0000 7fa9327fc700 1 -- 172.21.15.148:0/3085405468 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v127) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa934006d70 con 0x7fa93c15d580 2023-10-10T20:25:22.028 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:22.028 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:25:22.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:22.029 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:25:22.029 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:25:22.029 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:22.030 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:25:22.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 msgr2=0x7fa9200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 0x7fa9200693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa93800d6e0 tx=0x7fa938007850 comp rx=0 tx=0).stop 2023-10-10T20:25:22.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 msgr2=0x7fa93c0b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa93c06b530 tx=0x7fa934017000 comp rx=0 tx=0).stop 2023-10-10T20:25:22.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 shutdown_connections 2023-10-10T20:25:22.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa920066f30 0x7fa9200693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa93c15d580 0x7fa93c0b1a00 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa93c157870 0x7fa93c16e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 --2- 172.21.15.148:0/3085405468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa93c15a620 0x7fa93c154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 >> 172.21.15.148:0/3085405468 conn(0x7fa93c068ce0 msgr2=0x7fa93c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:22.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 shutdown_connections 2023-10-10T20:25:22.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.027+0000 7fa9446fa700 1 -- 172.21.15.148:0/3085405468 wait complete. 2023-10-10T20:25:22.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:25:22.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:25:22.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.157+0000 7f22f6fcd700 1 Processor -- start 2023-10-10T20:25:22.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.157+0000 7f22f6fcd700 1 -- start start 2023-10-10T20:25:22.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f6fcd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 0x7f22f006f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f6fcd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22f00cf2b0 0x7f22f006f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f6fcd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f00cb3c0 con 0x7f22f0158040 2023-10-10T20:25:22.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f6fcd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f015c310 con 0x7f22f00cf2b0 2023-10-10T20:25:22.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f6fcd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f015b9e0 con 0x7f22f0157c70 2023-10-10T20:25:22.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f556a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22f00cf2b0 0x7f22f006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22effff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f22f0158040 0x7f22f00ceba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48498/0 (socket says 172.21.15.148:48498) 2023-10-10T20:25:22.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22effff700 1 -- 172.21.15.148:0/2141263630 learned_addr learned my addr 172.21.15.148:0/2141263630 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:22.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.158+0000 7f22f556a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22f00cf2b0 0x7f22f006f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44028/0 (socket says 172.21.15.148:44028) 2023-10-10T20:25:22.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f4d69700 1 --2- 172.21.15.148:0/2141263630 >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 0x7f22f006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f556a700 1 -- 172.21.15.148:0/2141263630 >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 msgr2=0x7f22f006f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:22.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f556a700 1 --2- 172.21.15.148:0/2141263630 >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 0x7f22f006f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f556a700 1 -- 172.21.15.148:0/2141263630 >> v1:172.21.15.148:6789/0 conn(0x7f22f0158040 legacy=0x7f22f00ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f556a700 1 -- 172.21.15.148:0/2141263630 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f015bcc0 con 0x7f22f00cf2b0 2023-10-10T20:25:22.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22ef7fe700 1 -- 172.21.15.148:0/2141263630 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970401523 0 0) 0x7f22e4001110 con 0x7f22f0158040 2023-10-10T20:25:22.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f4d69700 1 --2- 172.21.15.148:0/2141263630 >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 0x7f22f006f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:25:22.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f556a700 1 --2- 172.21.15.148:0/2141263630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f00cf2b0 0x7f22f006f770 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f22f006e850 tx=0x7f22e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191070ca28789566 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22ef7fe700 1 -- 172.21.15.148:0/2141263630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e800b6b0 con 0x7f22f00cf2b0 2023-10-10T20:25:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22ef7fe700 1 -- 172.21.15.148:0/2141263630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e800b810 con 0x7f22f00cf2b0 2023-10-10T20:25:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22ef7fe700 1 -- 172.21.15.148:0/2141263630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e800bbb0 con 0x7f22f00cf2b0 2023-10-10T20:25:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f6fcd700 1 -- 172.21.15.148:0/2141263630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f00cf2b0 msgr2=0x7f22f006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.159+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/2141263630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f00cf2b0 0x7f22f006f770 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f22f006e850 tx=0x7f22e80091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 -- 172.21.15.148:0/2141263630 shutdown_connections 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/2141263630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f00cf2b0 0x7f22f006f770 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/2141263630 >> v2:172.21.15.161:3300/0 conn(0x7f22f0157c70 0x7f22f006f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 -- 172.21.15.148:0/2141263630 >> 172.21.15.148:0/2141263630 conn(0x7f22f006bc70 msgr2=0x7f22f006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 -- 172.21.15.148:0/2141263630 shutdown_connections 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.160+0000 7f22f6fcd700 1 -- 172.21.15.148:0/2141263630 wait complete. 2023-10-10T20:25:22.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.161+0000 7f22f6fcd700 1 Processor -- start 2023-10-10T20:25:22.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.161+0000 7f22f6fcd700 1 -- start start 2023-10-10T20:25:22.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22f00cf2b0 0x7f22f0158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22f0158040 0x7f22f00b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0159870 con 0x7f22f0157c70 2023-10-10T20:25:22.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f01565d0 con 0x7f22f0158040 2023-10-10T20:25:22.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0159360 con 0x7f22f00cf2b0 2023-10-10T20:25:22.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44032/0 (socket says 172.21.15.148:44032) 2023-10-10T20:25:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 -- 172.21.15.148:0/3686320148 learned_addr learned my addr 172.21.15.148:0/3686320148 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f4d69700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22f00cf2b0 0x7f22f0158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f556a700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22f0158040 0x7f22f00b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 -- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22f00cf2b0 msgr2=0x7f22f0158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22f00cf2b0 0x7f22f0158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 -- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22f0158040 msgr2=0x7f22f00b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22f0158040 0x7f22f00b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 -- 172.21.15.148:0/3686320148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8009420 con 0x7f22f0157c70 2023-10-10T20:25:22.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22effff700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7e0 tx=0x7f22e4007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4005b30 con 0x7f22f0157c70 2023-10-10T20:25:22.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4005c90 con 0x7f22f0157c70 2023-10-10T20:25:22.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f0158ac0 con 0x7f22f0157c70 2023-10-10T20:25:22.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40072e0 con 0x7f22f0157c70 2023-10-10T20:25:22.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.162+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0159080 con 0x7f22f0157c70 2023-10-10T20:25:22.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.163+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f004c3c0 con 0x7f22f0157c70 2023-10-10T20:25:22.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.164+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22e4007690 con 0x7f22f0157c70 2023-10-10T20:25:22.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.164+0000 7f22ecff9700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.164+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f22e4083350 con 0x7f22f0157c70 2023-10-10T20:25:22.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.164+0000 7f22f4d69700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.164+0000 7f22f4d69700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 0x7f22dc0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d6e0 tx=0x7f22e0007850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.168+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22e404b5a0 con 0x7f22f0157c70 2023-10-10T20:25:22.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.350+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f22f00cb3a0 con 0x7f22f0157c70 2023-10-10T20:25:22.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.629+0000 7f22ecff9700 1 -- 172.21.15.148:0/3686320148 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v128) v1 ==== 138+0+0 (secure 0 0 0) 0x7f22e40509c0 con 0x7f22f0157c70 2023-10-10T20:25:22.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.633+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 msgr2=0x7f22dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.633+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 0x7f22dc0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d6e0 tx=0x7f22e0007850 comp rx=0 tx=0).stop 2023-10-10T20:25:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.633+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 msgr2=0x7f22f0156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.633+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c7e0 tx=0x7f22e4007c30 comp rx=0 tx=0).stop 2023-10-10T20:25:22.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 shutdown_connections 2023-10-10T20:25:22.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22f0158040 0x7f22f00b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22f00cf2b0 0x7f22f0158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 --2- 172.21.15.148:0/3686320148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22f0157c70 0x7f22f0156090 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 >> 172.21.15.148:0/3686320148 conn(0x7f22f006bc70 msgr2=0x7f22f0156e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 shutdown_connections 2023-10-10T20:25:22.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.634+0000 7f22f6fcd700 1 -- 172.21.15.148:0/3686320148 wait complete. 2023-10-10T20:25:22.639 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:25:22.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:25:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.769+0000 7fb1e3674700 1 Processor -- start 2023-10-10T20:25:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.769+0000 7fb1e3674700 1 -- start start 2023-10-10T20:25:22.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1dc159ec0 0x7fb1dc0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb1dc15a290 0x7fb1dc0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc06aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc06af60 con 0x7fb1dc159ec0 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc0b0790 con 0x7fb1dc15a290 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e3674700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0ce1f0 con 0x7fb1dc06a5a0 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e1410700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1dc159ec0 0x7fb1dc0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e1410700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1dc159ec0 0x7fb1dc0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57650/0 (socket says 172.21.15.148:57650) 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e1410700 1 -- 172.21.15.148:0/1410282213 learned_addr learned my addr 172.21.15.148:0/1410282213 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e0c0f700 1 --2- 172.21.15.148:0/1410282213 >> v2:172.21.15.161:3300/0 conn(0x7fb1dc15a290 0x7fb1dc0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.770+0000 7fb1e1c11700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc06aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 -- 172.21.15.148:0/1410282213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 msgr2=0x7fb1dc06aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:22.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc06aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 -- 172.21.15.148:0/1410282213 >> v2:172.21.15.161:3300/0 conn(0x7fb1dc15a290 msgr2=0x7fb1dc0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:22.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1410282213 >> v2:172.21.15.161:3300/0 conn(0x7fb1dc15a290 0x7fb1dc0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 -- 172.21.15.148:0/1410282213 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0b0b20 con 0x7fb1dc159ec0 2023-10-10T20:25:22.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc0af940 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc089910 tx=0x7fb1d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca75769b21a8f9f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1d3fff700 1 -- 172.21.15.148:0/1410282213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800e070 con 0x7fb1dc159ec0 2023-10-10T20:25:22.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1d3fff700 1 -- 172.21.15.148:0/1410282213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d800b970 con 0x7fb1dc159ec0 2023-10-10T20:25:22.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1d3fff700 1 -- 172.21.15.148:0/1410282213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80092e0 con 0x7fb1dc159ec0 2023-10-10T20:25:22.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 -- 172.21.15.148:0/1410282213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc159ec0 msgr2=0x7fb1dc0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc0af940 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc089910 tx=0x7fb1d8009840 comp rx=0 tx=0).stop 2023-10-10T20:25:22.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 -- 172.21.15.148:0/1410282213 shutdown_connections 2023-10-10T20:25:22.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc06aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1410282213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc0af940 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.771+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1410282213 >> v2:172.21.15.161:3300/0 conn(0x7fb1dc15a290 0x7fb1dc0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.772+0000 7fb1e3674700 1 -- 172.21.15.148:0/1410282213 >> 172.21.15.148:0/1410282213 conn(0x7fb1dc0d1880 msgr2=0x7fb1dc0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:22.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.772+0000 7fb1e3674700 1 -- 172.21.15.148:0/1410282213 shutdown_connections 2023-10-10T20:25:22.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.772+0000 7fb1e3674700 1 -- 172.21.15.148:0/1410282213 wait complete. 2023-10-10T20:25:22.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.773+0000 7fb1e3674700 1 Processor -- start 2023-10-10T20:25:22.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.773+0000 7fb1e3674700 1 -- start start 2023-10-10T20:25:22.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc157cf0 con 0x7fb1dc15a290 2023-10-10T20:25:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc1570e0 con 0x7fb1dc159ec0 2023-10-10T20:25:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc1573c0 con 0x7fb1dc06a5a0 2023-10-10T20:25:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57660/0 (socket says 172.21.15.148:57660) 2023-10-10T20:25:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 -- 172.21.15.148:0/1445199620 learned_addr learned my addr 172.21.15.148:0/1445199620 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e0c0f700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 -- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 msgr2=0x7fb1dc158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 -- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc159ec0 msgr2=0x7fb1dc1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc1534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 -- 172.21.15.148:0/1445199620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8009ae0 con 0x7fb1dc15a290 2023-10-10T20:25:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e1c11700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400ea50 tx=0x7fb1d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400ee90 con 0x7fb1dc15a290 2023-10-10T20:25:22.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400c850 con 0x7fb1dc15a290 2023-10-10T20:25:22.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc1576a0 con 0x7fb1dc15a290 2023-10-10T20:25:22.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400cc10 con 0x7fb1dc15a290 2023-10-10T20:25:22.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.774+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc1539e0 con 0x7fb1dc15a290 2023-10-10T20:25:22.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.775+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc16a270 con 0x7fb1dc15a290 2023-10-10T20:25:22.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.776+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1d4013470 con 0x7fb1dc15a290 2023-10-10T20:25:22.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.776+0000 7fb1d17fa700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 0x7fb1c0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:22.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.776+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fb1d4084580 con 0x7fb1dc15a290 2023-10-10T20:25:22.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.776+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 0x7fb1c0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:22.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.777+0000 7fb1e1410700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 0x7fb1c0069460 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009880 tx=0x7fb1d8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:22.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.781+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1d4051930 con 0x7fb1dc15a290 2023-10-10T20:25:22.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:22.962+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb1dc04c3c0 con 0x7fb1dc15a290 2023-10-10T20:25:23.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:25:23.159 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:25:23.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.633+0000 7fb1d17fa700 1 -- 172.21.15.148:0/1445199620 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v129) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb1d4054f50 con 0x7fb1dc15a290 2023-10-10T20:25:23.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 msgr2=0x7fb1c0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 0x7fb1c0069460 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009880 tx=0x7fb1d8018000 comp rx=0 tx=0).stop 2023-10-10T20:25:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 msgr2=0x7fb1dc15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400ea50 tx=0x7fb1d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 shutdown_connections 2023-10-10T20:25:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb1c0066fb0 0x7fb1c0069460 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009880 tx=0x7fb1d8018000 comp rx=0 tx=0).stop 2023-10-10T20:25:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1dc15a290 0x7fb1dc15bfc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1dc06a5a0 0x7fb1dc158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.639+0000 7fb1e3674700 1 --2- 172.21.15.148:0/1445199620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1dc159ec0 0x7fb1dc1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.640+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 >> 172.21.15.148:0/1445199620 conn(0x7fb1dc0d1880 msgr2=0x7fb1dc0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:23.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.643+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 shutdown_connections 2023-10-10T20:25:23.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.644+0000 7fb1e3674700 1 -- 172.21.15.148:0/1445199620 wait complete. 2023-10-10T20:25:23.651 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:25:23.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:25:23.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.779+0000 7f8725500700 1 Processor -- start 2023-10-10T20:25:23.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.779+0000 7f8725500700 1 -- start start 2023-10-10T20:25:23.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f8725500700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8720152300 0x7f872015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f8725500700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f872006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f8725500700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872015c8d0 con 0x7f8720158df0 2023-10-10T20:25:23.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f8725500700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87200b1400 con 0x7f8720152300 2023-10-10T20:25:23.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f8725500700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87200c9370 con 0x7f872006d370 2023-10-10T20:25:23.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f871effd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8720152300 0x7f872015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f87167fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8720158df0 0x7f87200caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57734/0 (socket says 172.21.15.148:57734) 2023-10-10T20:25:23.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.780+0000 7f87167fc700 1 -- 172.21.15.148:0/1184020650 learned_addr learned my addr 172.21.15.148:0/1184020650 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:23.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871f7fe700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f872006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871effd700 1 -- 172.21.15.148:0/1184020650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 msgr2=0x7f872006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:23.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871effd700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f872006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871effd700 1 -- 172.21.15.148:0/1184020650 >> v1:172.21.15.148:6789/0 conn(0x7f8720158df0 legacy=0x7f87200caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871effd700 1 -- 172.21.15.148:0/1184020650 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872015c030 con 0x7f8720152300 2023-10-10T20:25:23.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871e7fc700 1 -- 172.21.15.148:0/1184020650 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583610185 0 0) 0x7f872015c8d0 con 0x7f8720158df0 2023-10-10T20:25:23.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871effd700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f872015b9a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f87100098e0 tx=0x7f87100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad65706b07e7cad server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:23.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871e7fc700 1 -- 172.21.15.148:0/1184020650 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710009e80 con 0x7f8720152300 2023-10-10T20:25:23.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871e7fc700 1 -- 172.21.15.148:0/1184020650 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871000b860 con 0x7f8720152300 2023-10-10T20:25:23.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f871e7fc700 1 -- 172.21.15.148:0/1184020650 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000bbe0 con 0x7f8720152300 2023-10-10T20:25:23.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 -- 172.21.15.148:0/1184020650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 msgr2=0x7f872015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f872015b9a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f87100098e0 tx=0x7f87100091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 -- 172.21.15.148:0/1184020650 shutdown_connections 2023-10-10T20:25:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f872006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 --2- 172.21.15.148:0/1184020650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f872015b9a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.781+0000 7f8725500700 1 -- 172.21.15.148:0/1184020650 >> 172.21.15.148:0/1184020650 conn(0x7f8720069870 msgr2=0x7f8720069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:23.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.782+0000 7f8725500700 1 -- 172.21.15.148:0/1184020650 shutdown_connections 2023-10-10T20:25:23.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.782+0000 7f8725500700 1 -- 172.21.15.148:0/1184020650 wait complete. 2023-10-10T20:25:23.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 Processor -- start 2023-10-10T20:25:23.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 -- start start 2023-10-10T20:25:23.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f8720158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720158df0 0x7f87201535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b3c90 con 0x7f8720152300 2023-10-10T20:25:23.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b3f70 con 0x7f8720158df0 2023-10-10T20:25:23.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.783+0000 7f8725500700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87200b4250 con 0x7f872006d370 2023-10-10T20:25:23.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57686/0 (socket says 172.21.15.148:57686) 2023-10-10T20:25:23.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 -- 172.21.15.148:0/2470353863 learned_addr learned my addr 172.21.15.148:0/2470353863 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:23.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f871effd700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f8720158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f871f7fe700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720158df0 0x7f87201535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 -- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 msgr2=0x7f8720158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f8720158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 -- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720158df0 msgr2=0x7f87201535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:23.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720158df0 0x7f87201535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 -- 172.21.15.148:0/2470353863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710009420 con 0x7f8720152300 2023-10-10T20:25:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87167fc700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f870800c7e0 tx=0x7f8708007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708005b30 con 0x7f8720152300 2023-10-10T20:25:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708005c90 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87200b4530 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708007300 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.784+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87200b3880 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.785+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872004c3c0 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.786+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f870801b3b0 con 0x7f8720152300 2023-10-10T20:25:23.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.786+0000 7f87177fe700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 0x7f87000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:23.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.786+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f87080839f0 con 0x7f8720152300 2023-10-10T20:25:23.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.786+0000 7f871effd700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 0x7f87000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:23.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.786+0000 7f871effd700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 0x7f87000693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87100091e0 tx=0x7f8710017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:23.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:23.791+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8708050cc0 con 0x7f8720152300 2023-10-10T20:25:24.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.011+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f87200b0ae0 con 0x7f8720152300 2023-10-10T20:25:24.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.660+0000 7f87177fe700 1 -- 172.21.15.148:0/2470353863 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8708017020 con 0x7f8720152300 2023-10-10T20:25:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.665+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 msgr2=0x7f87000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.665+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 0x7f87000693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87100091e0 tx=0x7f8710017000 comp rx=0 tx=0).stop 2023-10-10T20:25:24.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.665+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 msgr2=0x7f8720153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.665+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f870800c7e0 tx=0x7f8708007df0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 shutdown_connections 2023-10-10T20:25:24.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8700066f30 0x7f87000693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720158df0 0x7f87201535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f872006d370 0x7f8720158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 --2- 172.21.15.148:0/2470353863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8720152300 0x7f8720153070 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 >> 172.21.15.148:0/2470353863 conn(0x7f8720069870 msgr2=0x7f87201514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:24.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 shutdown_connections 2023-10-10T20:25:24.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.666+0000 7f8725500700 1 -- 172.21.15.148:0/2470353863 wait complete. 2023-10-10T20:25:24.669 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 16 and data pool 17 2023-10-10T20:25:24.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:24.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.795+0000 7f455e197700 1 Processor -- start 2023-10-10T20:25:24.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.795+0000 7f455e197700 1 -- start start 2023-10-10T20:25:24.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455e197700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45580cdab0 0x7f455815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:24.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455e197700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580cb7c0 con 0x7f455815dda0 2023-10-10T20:25:24.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455e197700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455815e290 con 0x7f45580cd6e0 2023-10-10T20:25:24.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455e197700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455815e850 con 0x7f45580cdab0 2023-10-10T20:25:24.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455c994700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45580cdab0 0x7f455815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:24.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455d996700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f455815dda0 0x7f455815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57748/0 (socket says 172.21.15.148:57748) 2023-10-10T20:25:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455d996700 1 -- 172.21.15.148:0/4154814826 learned_addr learned my addr 172.21.15.148:0/4154814826 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455c994700 1 -- 172.21.15.148:0/4154814826 >> v1:172.21.15.161:6789/0 conn(0x7f45580cd6e0 legacy=0x7f45580cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455c994700 1 -- 172.21.15.148:0/4154814826 >> v1:172.21.15.148:6789/0 conn(0x7f455815dda0 legacy=0x7f455815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f454ffff700 1 -- 172.21.15.148:0/4154814826 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130603869 0 0) 0x7f45580cb7c0 con 0x7f455815dda0 2023-10-10T20:25:24.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455c994700 1 -- 172.21.15.148:0/4154814826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455815e570 con 0x7f45580cdab0 2023-10-10T20:25:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f455c994700 1 --2- 172.21.15.148:0/4154814826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455815d860 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f45480098e0 tx=0x7f45480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e936a9f762ca8966 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.796+0000 7f454ffff700 1 -- 172.21.15.148:0/4154814826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548009e80 con 0x7f45580cdab0 2023-10-10T20:25:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f454ffff700 1 -- 172.21.15.148:0/4154814826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454800b820 con 0x7f45580cdab0 2023-10-10T20:25:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f454ffff700 1 -- 172.21.15.148:0/4154814826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800bba0 con 0x7f45580cdab0 2023-10-10T20:25:24.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 -- 172.21.15.148:0/4154814826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 msgr2=0x7f455815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 --2- 172.21.15.148:0/4154814826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455815d860 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f45480098e0 tx=0x7f45480091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 -- 172.21.15.148:0/4154814826 shutdown_connections 2023-10-10T20:25:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 --2- 172.21.15.148:0/4154814826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455815d860 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 -- 172.21.15.148:0/4154814826 >> 172.21.15.148:0/4154814826 conn(0x7f4558067660 msgr2=0x7f4558067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:24.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 -- 172.21.15.148:0/4154814826 shutdown_connections 2023-10-10T20:25:24.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.797+0000 7f455e197700 1 -- 172.21.15.148:0/4154814826 wait complete. 2023-10-10T20:25:24.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.798+0000 7f455e197700 1 Processor -- start 2023-10-10T20:25:24.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.798+0000 7f455e197700 1 -- start start 2023-10-10T20:25:24.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45580cd6e0 0x7f455814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:24.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:24.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f455815dda0 0x7f455815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:24.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580cc360 con 0x7f45580cdab0 2023-10-10T20:25:24.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455816ea70 con 0x7f45580cd6e0 2023-10-10T20:25:24.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455e197700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45580b3bb0 con 0x7f455815dda0 2023-10-10T20:25:24.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455c994700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:24.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455c994700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57710/0 (socket says 172.21.15.148:57710) 2023-10-10T20:25:24.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455c994700 1 -- 172.21.15.148:0/2133771869 learned_addr learned my addr 172.21.15.148:0/2133771869 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:24.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455d195700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45580cd6e0 0x7f455814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:24.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.799+0000 7f455d996700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f455815dda0 0x7f455815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:24.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 -- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f455815dda0 msgr2=0x7f455815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f455815dda0 0x7f455815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 -- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45580cd6e0 msgr2=0x7f455814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45580cd6e0 0x7f455814a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 -- 172.21.15.148:0/2133771869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009420 con 0x7f45580cdab0 2023-10-10T20:25:24.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455c994700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f45480030c0 tx=0x7f4548016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:24.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800b990 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4548020070 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45580b3e90 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454801b450 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.800+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455814ae10 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.801+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45580b3a10 con 0x7f45580cdab0 2023-10-10T20:25:24.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.801+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4548022030 con 0x7f45580cdab0 2023-10-10T20:25:24.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.802+0000 7f454d7fa700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 0x7f453c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:24.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.802+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4548086490 con 0x7f45580cdab0 2023-10-10T20:25:24.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.802+0000 7f455d195700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 0x7f453c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:24.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.802+0000 7f455d195700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 0x7f453c0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4554003a30 tx=0x7f4554007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:24.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.806+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45580b3a10 con 0x7f45580cdab0 2023-10-10T20:25:24.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.990+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45580b43b0 con 0x7f45580cdab0 2023-10-10T20:25:24.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.990+0000 7f454d7fa700 1 -- 172.21.15.148:0/2133771869 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f45580b43b0 con 0x7f45580cdab0 2023-10-10T20:25:24.992 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:24.992 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":130,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:24.013269+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:24.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 msgr2=0x7f453c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 0x7f453c0693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4554003a30 tx=0x7f4554007ea0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 msgr2=0x7f455814f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:24.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f45480030c0 tx=0x7f4548016000 comp rx=0 tx=0).stop 2023-10-10T20:25:24.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 shutdown_connections 2023-10-10T20:25:24.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f453c066f30 0x7f453c0693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f455815dda0 0x7f455815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45580cd6e0 0x7f455814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 --2- 172.21.15.148:0/2133771869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45580cdab0 0x7f455814f630 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:24.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.992+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 >> 172.21.15.148:0/2133771869 conn(0x7f4558067660 msgr2=0x7f45580b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:24.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.993+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 shutdown_connections 2023-10-10T20:25:24.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:24.993+0000 7f455e197700 1 -- 172.21.15.148:0/2133771869 wait complete. 2023-10-10T20:25:25.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:25.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.119+0000 7fc01c3ec700 1 Processor -- start 2023-10-10T20:25:25.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.119+0000 7fc01c3ec700 1 -- start start 2023-10-10T20:25:25.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01c3ec700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc01415a620 0x7fc01415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01c3ec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 0x7fc014158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01c3ec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01415cbb0 con 0x7fc014157870 2023-10-10T20:25:25.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01c3ec700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01415bdb0 con 0x7fc01415a620 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01c3ec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415bfa0 con 0x7fc01415d580 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 0x7fc014158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a188700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc014157870 0x7fc01415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:36856/0 (socket says 172.21.15.148:36856) 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a188700 1 -- 172.21.15.148:0/3219000060 learned_addr learned my addr 172.21.15.148:0/3219000060 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc019987700 1 --2- 172.21.15.148:0/3219000060 >> v2:172.21.15.161:3300/0 conn(0x7fc01415a620 0x7fc01415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 -- 172.21.15.148:0/3219000060 >> v2:172.21.15.161:3300/0 conn(0x7fc01415a620 msgr2=0x7fc01415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:25.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 --2- 172.21.15.148:0/3219000060 >> v2:172.21.15.161:3300/0 conn(0x7fc01415a620 0x7fc01415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 -- 172.21.15.148:0/3219000060 >> v1:172.21.15.161:6789/0 conn(0x7fc014157870 legacy=0x7fc01415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc019186700 1 -- 172.21.15.148:0/3219000060 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013675070 0 0) 0x7fc01415cbb0 con 0x7fc014157870 2023-10-10T20:25:25.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 -- 172.21.15.148:0/3219000060 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01415c280 con 0x7fc01415d580 2023-10-10T20:25:25.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.120+0000 7fc01a989700 1 --2- 172.21.15.148:0/3219000060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 0x7fc014158110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009e60 tx=0x7fc00c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6302b42700f9b614 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc019186700 1 -- 172.21.15.148:0/3219000060 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00e070 con 0x7fc01415d580 2023-10-10T20:25:25.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc019186700 1 -- 172.21.15.148:0/3219000060 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c00bb60 con 0x7fc01415d580 2023-10-10T20:25:25.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc019186700 1 -- 172.21.15.148:0/3219000060 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c012430 con 0x7fc01415d580 2023-10-10T20:25:25.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 -- 172.21.15.148:0/3219000060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 msgr2=0x7fc014158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/3219000060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 0x7fc014158110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009e60 tx=0x7fc00c009780 comp rx=0 tx=0).stop 2023-10-10T20:25:25.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 -- 172.21.15.148:0/3219000060 shutdown_connections 2023-10-10T20:25:25.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/3219000060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415d580 0x7fc014158110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/3219000060 >> v2:172.21.15.161:3300/0 conn(0x7fc01415a620 0x7fc01415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 -- 172.21.15.148:0/3219000060 >> 172.21.15.148:0/3219000060 conn(0x7fc014068ce0 msgr2=0x7fc0140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:25.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 -- 172.21.15.148:0/3219000060 shutdown_connections 2023-10-10T20:25:25.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.121+0000 7fc01c3ec700 1 -- 172.21.15.148:0/3219000060 wait complete. 2023-10-10T20:25:25.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.122+0000 7fc01c3ec700 1 Processor -- start 2023-10-10T20:25:25.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.122+0000 7fc01c3ec700 1 -- start start 2023-10-10T20:25:25.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc014157870 0x7fc0140cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415a620 0x7fc0140c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140cdb40 con 0x7fc01415d580 2023-10-10T20:25:25.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140b19f0 con 0x7fc014157870 2023-10-10T20:25:25.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01c3ec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141599e0 con 0x7fc01415a620 2023-10-10T20:25:25.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01a989700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01a989700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57714/0 (socket says 172.21.15.148:57714) 2023-10-10T20:25:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01a989700 1 -- 172.21.15.148:0/842771667 learned_addr learned my addr 172.21.15.148:0/842771667 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc019987700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415a620 0x7fc0140c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.123+0000 7fc01a188700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc014157870 0x7fc0140cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 -- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415a620 msgr2=0x7fc0140c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415a620 0x7fc0140c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 -- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc014157870 msgr2=0x7fc0140cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc014157870 0x7fc0140cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 -- 172.21.15.148:0/842771667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009a20 con 0x7fc01415d580 2023-10-10T20:25:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01a989700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc00c013000 tx=0x7fc00c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c012ea0 con 0x7fc01415d580 2023-10-10T20:25:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c027470 con 0x7fc01415d580 2023-10-10T20:25:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0140b10c0 con 0x7fc01415d580 2023-10-10T20:25:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c027830 con 0x7fc01415d580 2023-10-10T20:25:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.124+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0140b1680 con 0x7fc01415d580 2023-10-10T20:25:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.125+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01404c3c0 con 0x7fc01415d580 2023-10-10T20:25:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.125+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc00c024070 con 0x7fc01415d580 2023-10-10T20:25:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.126+0000 7fc00a7fc700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 0x7fbff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.126+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc00c086c30 con 0x7fc01415d580 2023-10-10T20:25:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.126+0000 7fc01a188700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 0x7fbff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.126+0000 7fc01a188700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 0x7fbff80693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc010007fd0 tx=0x7fc010007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.130+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc00c053ea0 con 0x7fc01415d580 2023-10-10T20:25:25.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.312+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc01415c560 con 0x7fc01415d580 2023-10-10T20:25:25.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.313+0000 7fc00a7fc700 1 -- 172.21.15.148:0/842771667 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+15426 (secure 0 0 0) 0x7fc00c02c090 con 0x7fc01415d580 2023-10-10T20:25:25.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:25.315 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":130,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:24.013269+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:25.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 msgr2=0x7fbff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 0x7fbff80693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc010007fd0 tx=0x7fc010007d30 comp rx=0 tx=0).stop 2023-10-10T20:25:25.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 msgr2=0x7fc0140c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc00c013000 tx=0x7fc00c019000 comp rx=0 tx=0).stop 2023-10-10T20:25:25.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 shutdown_connections 2023-10-10T20:25:25.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbff8066f30 0x7fbff80693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc01415d580 0x7fc0140c6050 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc014157870 0x7fc0140cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 --2- 172.21.15.148:0/842771667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc01415a620 0x7fc0140c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 >> 172.21.15.148:0/842771667 conn(0x7fc014068ce0 msgr2=0x7fc0140b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:25.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 shutdown_connections 2023-10-10T20:25:25.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.315+0000 7fc01c3ec700 1 -- 172.21.15.148:0/842771667 wait complete. 2023-10-10T20:25:25.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:25:25.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.446+0000 7feb86630700 1 Processor -- start 2023-10-10T20:25:25.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.446+0000 7feb86630700 1 -- start start 2023-10-10T20:25:25.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb86630700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb86630700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb800b68e0 0x7feb8006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb86630700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb800b46b0 con 0x7feb8015d560 2023-10-10T20:25:25.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb86630700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb800b54f0 con 0x7feb800b68e0 2023-10-10T20:25:25.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb86630700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb800b48e0 con 0x7feb8015d190 2023-10-10T20:25:25.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb7ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb7f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb800b68e0 0x7feb8006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb7f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb800b68e0 0x7feb8006aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:51716/0 (socket says 172.21.15.148:51716) 2023-10-10T20:25:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb7ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:50030/0 (socket says 172.21.15.148:50030) 2023-10-10T20:25:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb84bcd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7feb8015d560 0x7feb8015aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:36862/0 (socket says 172.21.15.148:36862) 2023-10-10T20:25:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.447+0000 7feb7f7fe700 1 -- 172.21.15.148:0/2158105600 learned_addr learned my addr 172.21.15.148:0/2158105600 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:25.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7f7fe700 1 -- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 msgr2=0x7feb8006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:25.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7f7fe700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7f7fe700 1 -- 172.21.15.148:0/2158105600 >> v1:172.21.15.161:6789/0 conn(0x7feb8015d560 legacy=0x7feb8015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7f7fe700 1 -- 172.21.15.148:0/2158105600 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800b4bc0 con 0x7feb800b68e0 2023-10-10T20:25:25.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7ffff700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:25.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7effd700 1 -- 172.21.15.148:0/2158105600 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016586271 0 0) 0x7feb800b46b0 con 0x7feb8015d560 2023-10-10T20:25:25.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7f7fe700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb8006aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb740099b0 tx=0x7feb74009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e001d2cfaf6ad664 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7effd700 1 -- 172.21.15.148:0/2158105600 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74009ea0 con 0x7feb800b68e0 2023-10-10T20:25:25.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7effd700 1 -- 172.21.15.148:0/2158105600 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7400b860 con 0x7feb800b68e0 2023-10-10T20:25:25.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.448+0000 7feb7effd700 1 -- 172.21.15.148:0/2158105600 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400bc00 con 0x7feb800b68e0 2023-10-10T20:25:25.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 -- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 msgr2=0x7feb8006aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb8006aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb740099b0 tx=0x7feb74009250 comp rx=0 tx=0).stop 2023-10-10T20:25:25.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 -- 172.21.15.148:0/2158105600 shutdown_connections 2023-10-10T20:25:25.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8006a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 --2- 172.21.15.148:0/2158105600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb8006aa60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 -- 172.21.15.148:0/2158105600 >> 172.21.15.148:0/2158105600 conn(0x7feb80068ce0 msgr2=0x7feb800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:25.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 -- 172.21.15.148:0/2158105600 shutdown_connections 2023-10-10T20:25:25.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.449+0000 7feb86630700 1 -- 172.21.15.148:0/2158105600 wait complete. 2023-10-10T20:25:25.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.450+0000 7feb86630700 1 Processor -- start 2023-10-10T20:25:25.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.450+0000 7feb86630700 1 -- start start 2023-10-10T20:25:25.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb80149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8014e600 con 0x7feb8015d560 2023-10-10T20:25:25.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8014e8e0 con 0x7feb800b68e0 2023-10-10T20:25:25.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8014ebc0 con 0x7feb8015d190 2023-10-10T20:25:25.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb7f7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb7ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb80149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57716/0 (socket says 172.21.15.148:57716) 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 -- 172.21.15.148:0/2044596709 learned_addr learned my addr 172.21.15.148:0/2044596709 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 -- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 msgr2=0x7feb8014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8014b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 -- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 msgr2=0x7feb80149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:25.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb80149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:25.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 -- 172.21.15.148:0/2044596709 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009420 con 0x7feb8015d560 2023-10-10T20:25:25.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb84bcd700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feb7800cd50 tx=0x7feb78001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78005df0 con 0x7feb8015d560 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7800a8b0 con 0x7feb8015d560 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7800ac70 con 0x7feb8015d560 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb740094f0 con 0x7feb8015d560 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.451+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb801516d0 con 0x7feb8015d560 2023-10-10T20:25:25.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.452+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8014f060 con 0x7feb8015d560 2023-10-10T20:25:25.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.453+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb78007300 con 0x7feb8015d560 2023-10-10T20:25:25.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.453+0000 7feb67fff700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 0x7feb6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:25.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.453+0000 7feb7ffff700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 0x7feb6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:25.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.453+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb780846d0 con 0x7feb8015d560 2023-10-10T20:25:25.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.454+0000 7feb7ffff700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 0x7feb6c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb70009f40 tx=0x7feb700091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:25.465 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.458+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb78051920 con 0x7feb8015d560 2023-10-10T20:25:25.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:25.683+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb80151980 con 0x7feb8015d560 2023-10-10T20:25:26.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.662+0000 7feb67fff700 1 -- 172.21.15.148:0/2044596709 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb80151980 con 0x7feb8015d560 2023-10-10T20:25:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 msgr2=0x7feb6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 0x7feb6c0693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb70009f40 tx=0x7feb700091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 msgr2=0x7feb8014bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feb7800cd50 tx=0x7feb78001b10 comp rx=0 tx=0).stop 2023-10-10T20:25:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 shutdown_connections 2023-10-10T20:25:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feb6c066f30 0x7feb6c0693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8015d560 0x7feb8014bcf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb800b68e0 0x7feb80149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 --2- 172.21.15.148:0/2044596709 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feb8015d190 0x7feb8014b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 >> 172.21.15.148:0/2044596709 conn(0x7feb80068ce0 msgr2=0x7feb8006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 shutdown_connections 2023-10-10T20:25:26.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.665+0000 7feb86630700 1 -- 172.21.15.148:0/2044596709 wait complete. 2023-10-10T20:25:26.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:25:26.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.798+0000 7f9064a50700 1 Processor -- start 2023-10-10T20:25:26.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.798+0000 7f9064a50700 1 -- start start 2023-10-10T20:25:26.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9064a50700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f9060164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9064a50700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600b0790 con 0x7f906015a290 2023-10-10T20:25:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9064a50700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600ce1f0 con 0x7f9060159ec0 2023-10-10T20:25:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9064a50700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b0b20 con 0x7f90600d05a0 2023-10-10T20:25:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9055d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f906015a290 0x7f906006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57760/0 (socket says 172.21.15.148:57760) 2023-10-10T20:25:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f9055d9b700 1 -- 172.21.15.148:0/4135706858 learned_addr learned my addr 172.21.15.148:0/4135706858 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:26.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.799+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f9060164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911721338 0 0) 0x7f90600b0790 con 0x7f906015a290 2023-10-10T20:25:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f904c003570 con 0x7f906015a290 2023-10-10T20:25:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736188098 0 0) 0x7f90600ce1f0 con 0x7f9060159ec0 2023-10-10T20:25:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90600b0790 con 0x7f9060159ec0 2023-10-10T20:25:26.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1519278791 0 0) 0x7f904c003570 con 0x7f906015a290 2023-10-10T20:25:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 msgr2=0x7f9060164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f9060164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 >> v1:172.21.15.161:6789/0 conn(0x7f9060159ec0 legacy=0x7f90600af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600b04b0 con 0x7f906015a290 2023-10-10T20:25:26.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9048001470 con 0x7f906015a290 2023-10-10T20:25:26.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f9060164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:26.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 >> v1:172.21.15.148:6789/0 conn(0x7f906015a290 legacy=0x7f906006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f904c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904c004160 0x7f904c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f904c004d50 0x7f904c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b0b20 con 0x7f904c004d50 2023-10-10T20:25:26.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c00bba0 con 0x7f904c004160 2023-10-10T20:25:26.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c00bd60 con 0x7f90600d05a0 2023-10-10T20:25:26.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f904c004d50 0x7f904c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905e59c700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f904c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f9055d9b700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904c004160 0x7f904c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 -- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 msgr2=0x7f904c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:26.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90600d05a0 0x7f904c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 -- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904c004160 msgr2=0x7f904c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:26.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904c004160 0x7f904c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.800+0000 7f905ed9d700 1 -- 172.21.15.148:0/4135706858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009a20 con 0x7f904c004d50 2023-10-10T20:25:26.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f905ed9d700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f904c004d50 0x7f904c0051c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9060071210 tx=0x7f905800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40261284067c2e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:26.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058009c50 con 0x7f904c004d50 2023-10-10T20:25:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905800f040 con 0x7f904c004d50 2023-10-10T20:25:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f905dd9b700 1 -- 172.21.15.148:0/4135706858 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800ee60 con 0x7f904c004d50 2023-10-10T20:25:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 -- 172.21.15.148:0/4135706858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f904c004d50 msgr2=0x7f904c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 --2- 172.21.15.148:0/4135706858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f904c004d50 0x7f904c0051c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9060071210 tx=0x7f905800b040 comp rx=0 tx=0).stop 2023-10-10T20:25:26.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f905ed9d700 1 -- 172.21.15.148:0/4135706858 reap_dead start 2023-10-10T20:25:26.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 -- 172.21.15.148:0/4135706858 shutdown_connections 2023-10-10T20:25:26.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 -- 172.21.15.148:0/4135706858 >> 172.21.15.148:0/4135706858 conn(0x7f90600d1880 msgr2=0x7f90600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:26.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 -- 172.21.15.148:0/4135706858 shutdown_connections 2023-10-10T20:25:26.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.801+0000 7f9064a50700 1 -- 172.21.15.148:0/4135706858 wait complete. 2023-10-10T20:25:26.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.802+0000 7f9064a50700 1 Processor -- start 2023-10-10T20:25:26.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 -- start start 2023-10-10T20:25:26.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f906016a660 0x7f90600c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 0x7f906016be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c9080 con 0x7f90600ca600 2023-10-10T20:25:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c9360 con 0x7f906016af70 2023-10-10T20:25:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9064a50700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600c9640 con 0x7f906016a660 2023-10-10T20:25:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9055d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9055d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57732/0 (socket says 172.21.15.148:57732) 2023-10-10T20:25:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f9055d9b700 1 -- 172.21.15.148:0/1452735255 learned_addr learned my addr 172.21.15.148:0/1452735255 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.803+0000 7f905ed9d700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 0x7f906016be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f905e59c700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f906016a660 0x7f90600c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 -- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f906016a660 msgr2=0x7f90600c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f906016a660 0x7f90600c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 -- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 msgr2=0x7f906016be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 0x7f906016be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 -- 172.21.15.148:0/1452735255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906016b960 con 0x7f90600ca600 2023-10-10T20:25:26.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f905ed9d700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 0x7f906016be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:26.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9055d9b700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f90480073c0 tx=0x7f9048001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:26.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048018030 con 0x7f90600ca600 2023-10-10T20:25:26.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048007830 con 0x7f90600ca600 2023-10-10T20:25:26.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904800fb20 con 0x7f90600ca600 2023-10-10T20:25:26.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906016b3e0 con 0x7f90600ca600 2023-10-10T20:25:26.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.804+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906016c3c0 con 0x7f90600ca600 2023-10-10T20:25:26.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.805+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906016b7c0 con 0x7f90600ca600 2023-10-10T20:25:26.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.805+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f904801b070 con 0x7f90600ca600 2023-10-10T20:25:26.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.806+0000 7f9056ffd700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 0x7f90400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.806+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9048019070 con 0x7f90600ca600 2023-10-10T20:25:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.806+0000 7f905e59c700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 0x7f90400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.806+0000 7f905e59c700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 0x7f90400693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90500010e0 tx=0x7f9050009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:26.810+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f906016b7c0 con 0x7f90600ca600 2023-10-10T20:25:27.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.040+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f906016bba0 con 0x7f90600ca600 2023-10-10T20:25:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.666+0000 7f9056ffd700 1 -- 172.21.15.148:0/1452735255 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f906016bba0 con 0x7f90600ca600 2023-10-10T20:25:27.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 msgr2=0x7f90400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 0x7f90400693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f90500010e0 tx=0x7f9050009040 comp rx=0 tx=0).stop 2023-10-10T20:25:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 msgr2=0x7f906016aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f90480073c0 tx=0x7f9048001df0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 shutdown_connections 2023-10-10T20:25:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9040066f30 0x7f90400693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f906016af70 0x7f906016be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f906016a660 0x7f90600c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 --2- 172.21.15.148:0/1452735255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90600ca600 0x7f906016aa30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 >> 172.21.15.148:0/1452735255 conn(0x7f90600d1880 msgr2=0x7f90600cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 shutdown_connections 2023-10-10T20:25:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.669+0000 7f9064a50700 1 -- 172.21.15.148:0/1452735255 wait complete. 2023-10-10T20:25:27.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:27.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.797+0000 7f980a664700 1 Processor -- start 2023-10-10T20:25:27.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.797+0000 7f980a664700 1 -- start start 2023-10-10T20:25:27.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f980a664700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98040cdab0 0x7f980415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f980a664700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f980415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f980a664700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98040cb7c0 con 0x7f98040cd6e0 2023-10-10T20:25:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f980a664700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980415e850 con 0x7f98040cdab0 2023-10-10T20:25:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f980a664700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98040cc360 con 0x7f980415dda0 2023-10-10T20:25:27.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f9809662700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f98040cd6e0 0x7f98040cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57770/0 (socket says 172.21.15.148:57770) 2023-10-10T20:25:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f9809662700 1 -- 172.21.15.148:0/3019298916 learned_addr learned my addr 172.21.15.148:0/3019298916 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f9808e61700 1 --2- 172.21.15.148:0/3019298916 >> v2:172.21.15.161:3300/0 conn(0x7f98040cdab0 0x7f980415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331414160 0 0) 0x7f98040cb7c0 con 0x7f98040cd6e0 2023-10-10T20:25:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f9809e63700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f980415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.798+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f0003570 con 0x7f98040cd6e0 2023-10-10T20:25:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346601901 0 0) 0x7f97f0003570 con 0x7f98040cd6e0 2023-10-10T20:25:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 msgr2=0x7f980415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f980415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 >> v2:172.21.15.161:3300/0 conn(0x7f98040cdab0 msgr2=0x7f980415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:27.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 --2- 172.21.15.148:0/3019298916 >> v2:172.21.15.161:3300/0 conn(0x7f98040cdab0 0x7f980415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980415e290 con 0x7f98040cd6e0 2023-10-10T20:25:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f98000017f0 con 0x7f98040cd6e0 2023-10-10T20:25:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 >> v1:172.21.15.148:6789/0 conn(0x7f98040cd6e0 legacy=0x7f98040cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0003a60 0x7f97f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f0004810 0x7f97f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f97f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980415e850 con 0x7f97f0004810 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98040cc360 con 0x7f97f0003a60 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f000ba60 con 0x7f980415dda0 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f0004810 0x7f97f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9809662700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0003a60 0x7f97f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9809e63700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f97f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 -- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 msgr2=0x7f97f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f980415dda0 0x7f97f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 -- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0003a60 msgr2=0x7f97f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:27.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0003a60 0x7f97f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 -- 172.21.15.148:0/3019298916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0003570 con 0x7f97f0004810 2023-10-10T20:25:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f9808e61700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f0004810 0x7f97f0004c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009c30 tx=0x7f97f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b588fef094118a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400e710 con 0x7f97f0004810 2023-10-10T20:25:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f400e870 con 0x7f97f0004810 2023-10-10T20:25:27.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.799+0000 7f97fbfff700 1 -- 172.21.15.148:0/3019298916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400ec30 con 0x7f97f0004810 2023-10-10T20:25:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 -- 172.21.15.148:0/3019298916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f0004810 msgr2=0x7f97f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 --2- 172.21.15.148:0/3019298916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f0004810 0x7f97f0004c80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009c30 tx=0x7f97f40091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f9809e63700 1 -- 172.21.15.148:0/3019298916 reap_dead start 2023-10-10T20:25:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 -- 172.21.15.148:0/3019298916 shutdown_connections 2023-10-10T20:25:27.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 -- 172.21.15.148:0/3019298916 >> 172.21.15.148:0/3019298916 conn(0x7f9804067660 msgr2=0x7f9804067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 -- 172.21.15.148:0/3019298916 shutdown_connections 2023-10-10T20:25:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.800+0000 7f980a664700 1 -- 172.21.15.148:0/3019298916 wait complete. 2023-10-10T20:25:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.801+0000 7f980a664700 1 Processor -- start 2023-10-10T20:25:27.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.801+0000 7f980a664700 1 -- start start 2023-10-10T20:25:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0004810 0x7f98040b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98040ca1e0 0x7f980416df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98040c7d20 con 0x7f98040ca650 2023-10-10T20:25:27.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98040c8b80 con 0x7f97f0004810 2023-10-10T20:25:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f980a664700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98040c9010 con 0x7f98040ca1e0 2023-10-10T20:25:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57746/0 (socket says 172.21.15.148:57746) 2023-10-10T20:25:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 -- 172.21.15.148:0/3637112813 learned_addr learned my addr 172.21.15.148:0/3637112813 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:27.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9808e61700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98040ca1e0 0x7f980416df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809662700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0004810 0x7f98040b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 -- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98040ca1e0 msgr2=0x7f980416df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98040ca1e0 0x7f980416df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 -- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0004810 msgr2=0x7f98040b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:27.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0004810 0x7f98040b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:27.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.802+0000 7f9809e63700 1 -- 172.21.15.148:0/3637112813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040c7f70 con 0x7f98040ca650 2023-10-10T20:25:27.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f9809e63700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f98040c7cd0 tx=0x7f97fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:27.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc013750 con 0x7f98040ca650 2023-10-10T20:25:27.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc0138b0 con 0x7f98040ca650 2023-10-10T20:25:27.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98040c8250 con 0x7f98040ca650 2023-10-10T20:25:27.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc013c50 con 0x7f98040ca650 2023-10-10T20:25:27.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.803+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98040c8810 con 0x7f98040ca650 2023-10-10T20:25:27.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.804+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98040c8cf0 con 0x7f98040ca650 2023-10-10T20:25:27.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.804+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97fc014650 con 0x7f98040ca650 2023-10-10T20:25:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.805+0000 7f97f97fa700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 0x7f97e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.805+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f97fc084930 con 0x7f98040ca650 2023-10-10T20:25:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.805+0000 7f9809662700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 0x7f97e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:27.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.805+0000 7f9809662700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 0x7f97e80693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f98000020a0 tx=0x7f9800006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:27.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:27.809+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97fc051c20 con 0x7f98040ca650 2023-10-10T20:25:28.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.032+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98040c6dd0 con 0x7f98040ca650 2023-10-10T20:25:28.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.032+0000 7f97f97fa700 1 -- 172.21.15.148:0/3637112813 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f97fc022090 con 0x7f98040ca650 2023-10-10T20:25:28.034 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:28.034 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:25:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 msgr2=0x7f97e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 0x7f97e80693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f98000020a0 tx=0x7f9800006040 comp rx=0 tx=0).stop 2023-10-10T20:25:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 msgr2=0x7f980416e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f98040c7cd0 tx=0x7f97fc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 shutdown_connections 2023-10-10T20:25:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97e8066f30 0x7f97e80693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040ca650 0x7f980416e490 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f0004810 0x7f98040b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 --2- 172.21.15.148:0/3637112813 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98040ca1e0 0x7f980416df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 >> 172.21.15.148:0/3637112813 conn(0x7f9804067660 msgr2=0x7f98040c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 shutdown_connections 2023-10-10T20:25:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.034+0000 7f980a664700 1 -- 172.21.15.148:0/3637112813 wait complete. 2023-10-10T20:25:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:25:28.047 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:25:28.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.162+0000 7f96923a6700 1 Processor -- start 2023-10-10T20:25:28.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.162+0000 7f96923a6700 1 -- start start 2023-10-10T20:25:28.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f96923a6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f96923a6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0a9d30 con 0x7f968c0c8440 2023-10-10T20:25:28.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f96923a6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0c9030 con 0x7f968c0c7ff0 2023-10-10T20:25:28.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f96923a6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c06e040 con 0x7f968c06d730 2023-10-10T20:25:28.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f9691ba5700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f968c0c8440 0x7f968c0aaed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57778/0 (socket says 172.21.15.148:57778) 2023-10-10T20:25:28.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f9691ba5700 1 -- 172.21.15.148:0/523823251 learned_addr learned my addr 172.21.15.148:0/523823251 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f96913a4700 1 --2- 172.21.15.148:0/523823251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.163+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903550243 0 0) 0x7f968c0a9d30 con 0x7f968c0c8440 2023-10-10T20:25:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9678003570 con 0x7f968c0c8440 2023-10-10T20:25:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628309245 0 0) 0x7f968c0c9030 con 0x7f968c0c7ff0 2023-10-10T20:25:28.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f968c0a9d30 con 0x7f968c0c7ff0 2023-10-10T20:25:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f96913a4700 1 -- 172.21.15.148:0/523823251 >> v1:172.21.15.161:6789/0 conn(0x7f968c0c7ff0 legacy=0x7f968c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f96913a4700 1 -- 172.21.15.148:0/523823251 >> v1:172.21.15.148:6789/0 conn(0x7f968c0c8440 legacy=0x7f968c0aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f96913a4700 1 -- 172.21.15.148:0/523823251 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0c8a70 con 0x7f968c06d730 2023-10-10T20:25:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f96913a4700 1 --2- 172.21.15.148:0/523823251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c06db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f967c009910 tx=0x7f967c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb3be4cba577ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c011630 con 0x7f968c06d730 2023-10-10T20:25:28.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c011790 con 0x7f968c06d730 2023-10-10T20:25:28.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.164+0000 7f9683fff700 1 -- 172.21.15.148:0/523823251 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c011b30 con 0x7f968c06d730 2023-10-10T20:25:28.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 -- 172.21.15.148:0/523823251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 msgr2=0x7f968c06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 --2- 172.21.15.148:0/523823251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c06db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f967c009910 tx=0x7f967c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 -- 172.21.15.148:0/523823251 shutdown_connections 2023-10-10T20:25:28.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 --2- 172.21.15.148:0/523823251 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c06db00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 -- 172.21.15.148:0/523823251 >> 172.21.15.148:0/523823251 conn(0x7f968c0cd040 msgr2=0x7f968c0cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 -- 172.21.15.148:0/523823251 shutdown_connections 2023-10-10T20:25:28.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.165+0000 7f96923a6700 1 -- 172.21.15.148:0/523823251 wait complete. 2023-10-10T20:25:28.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.166+0000 7f96923a6700 1 Processor -- start 2023-10-10T20:25:28.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.166+0000 7f96923a6700 1 -- start start 2023-10-10T20:25:28.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f968c0c7ff0 0x7f968c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c166c10 con 0x7f968c0c8440 2023-10-10T20:25:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0c4c40 con 0x7f968c0c7ff0 2023-10-10T20:25:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1665c0 con 0x7f968c06d730 2023-10-10T20:25:28.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57748/0 (socket says 172.21.15.148:57748) 2023-10-10T20:25:28.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 -- 172.21.15.148:0/562633921 learned_addr learned my addr 172.21.15.148:0/562633921 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:28.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96913a4700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9690ba3700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f968c0c7ff0 0x7f968c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 -- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 msgr2=0x7f968c1513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c1513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 -- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f968c0c7ff0 msgr2=0x7f968c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f968c0c7ff0 0x7f968c0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 -- 172.21.15.148:0/562633921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c009420 con 0x7f968c0c8440 2023-10-10T20:25:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f9691ba5700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f968400cd00 tx=0x7f9684001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400ce00 con 0x7f968c0c8440 2023-10-10T20:25:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968400aac0 con 0x7f968c0c8440 2023-10-10T20:25:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c1668a0 con 0x7f968c0c8440 2023-10-10T20:25:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400ae80 con 0x7f968c0c8440 2023-10-10T20:25:28.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.167+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1670a0 con 0x7f968c0c8440 2023-10-10T20:25:28.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.168+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c06c790 con 0x7f968c0c8440 2023-10-10T20:25:28.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.169+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9684007300 con 0x7f968c0c8440 2023-10-10T20:25:28.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.169+0000 7f96817fa700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 0x7f96700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.169+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9684084880 con 0x7f968c0c8440 2023-10-10T20:25:28.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.169+0000 7f96913a4700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 0x7f96700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.170+0000 7f96913a4700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 0x7f96700693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f967c009860 tx=0x7f967c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.173+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9684051b70 con 0x7f968c0c8440 2023-10-10T20:25:28.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.357+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f968c166d50 con 0x7f968c0c8440 2023-10-10T20:25:28.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.358+0000 7f96817fa700 1 -- 172.21.15.148:0/562633921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+15413 (secure 0 0 0) 0x7f9684055190 con 0x7f968c0c8440 2023-10-10T20:25:28.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:28.360 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":131,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:25.642388+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:25:28.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 msgr2=0x7f96700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 0x7f96700693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f967c009860 tx=0x7f967c017000 comp rx=0 tx=0).stop 2023-10-10T20:25:28.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 msgr2=0x7f968c152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f968400cd00 tx=0x7f9684001ac0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 shutdown_connections 2023-10-10T20:25:28.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9670066f30 0x7f96700693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f968c0c8440 0x7f968c152b00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f968c06d730 0x7f968c1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 --2- 172.21.15.148:0/562633921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f968c0c7ff0 0x7f968c0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 >> 172.21.15.148:0/562633921 conn(0x7f968c0cd040 msgr2=0x7f968c0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 shutdown_connections 2023-10-10T20:25:28.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.360+0000 7f96923a6700 1 -- 172.21.15.148:0/562633921 wait complete. 2023-10-10T20:25:28.374 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:25:28.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:25:28.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.487+0000 7f7d43c3e700 1 Processor -- start 2023-10-10T20:25:28.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.487+0000 7f7d43c3e700 1 -- start start 2023-10-10T20:25:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d43c3e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d3c157870 0x7f7d3c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d43c3e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c15a620 0x7f7d3c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d43c3e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c15cbb0 con 0x7f7d3c15d580 2023-10-10T20:25:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d43c3e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c15bfa0 con 0x7f7d3c157870 2023-10-10T20:25:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d43c3e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c15c280 con 0x7f7d3c15a620 2023-10-10T20:25:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d3c157870 0x7f7d3c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d3c157870 0x7f7d3c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57752/0 (socket says 172.21.15.148:57752) 2023-10-10T20:25:28.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 -- 172.21.15.148:0/1197898567 learned_addr learned my addr 172.21.15.148:0/1197898567 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 -- 172.21.15.148:0/1197898567 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c15a620 msgr2=0x7f7d3c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 --2- 172.21.15.148:0/1197898567 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c15a620 0x7f7d3c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 -- 172.21.15.148:0/1197898567 >> v1:172.21.15.161:6789/0 conn(0x7f7d3c15d580 legacy=0x7f7d3c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.488+0000 7f7d419da700 1 -- 172.21.15.148:0/1197898567 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c15c560 con 0x7f7d3c157870 2023-10-10T20:25:28.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d419da700 1 --2- 172.21.15.148:0/1197898567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c157870 0x7f7d3c15b6a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7d380099b0 tx=0x7f7d38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fabeeab43039d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d409d8700 1 -- 172.21.15.148:0/1197898567 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800b820 con 0x7f7d3c157870 2023-10-10T20:25:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d409d8700 1 -- 172.21.15.148:0/1197898567 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800b980 con 0x7f7d3c157870 2023-10-10T20:25:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d409d8700 1 -- 172.21.15.148:0/1197898567 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800bd20 con 0x7f7d3c157870 2023-10-10T20:25:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 -- 172.21.15.148:0/1197898567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c157870 msgr2=0x7f7d3c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/1197898567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c157870 0x7f7d3c15b6a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7d380099b0 tx=0x7f7d38009250 comp rx=0 tx=0).stop 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 -- 172.21.15.148:0/1197898567 shutdown_connections 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/1197898567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c157870 0x7f7d3c15b6a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/1197898567 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c15a620 0x7f7d3c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 -- 172.21.15.148:0/1197898567 >> 172.21.15.148:0/1197898567 conn(0x7f7d3c068ce0 msgr2=0x7f7d3c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 -- 172.21.15.148:0/1197898567 shutdown_connections 2023-10-10T20:25:28.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.489+0000 7f7d43c3e700 1 -- 172.21.15.148:0/1197898567 wait complete. 2023-10-10T20:25:28.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.490+0000 7f7d43c3e700 1 Processor -- start 2023-10-10T20:25:28.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.490+0000 7f7d43c3e700 1 -- start start 2023-10-10T20:25:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c157870 0x7f7d3c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3c15a620 0x7f7d3c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b0e40 con 0x7f7d3c15d580 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b1120 con 0x7f7d3c15a620 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d43c3e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b1400 con 0x7f7d3c157870 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d421db700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d421db700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57764/0 (socket says 172.21.15.148:57764) 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d421db700 1 -- 172.21.15.148:0/2402927636 learned_addr learned my addr 172.21.15.148:0/2402927636 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:28.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d419da700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c157870 0x7f7d3c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.491+0000 7f7d411d9700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3c15a620 0x7f7d3c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 -- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c157870 msgr2=0x7f7d3c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c157870 0x7f7d3c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 -- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3c15a620 msgr2=0x7f7d3c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3c15a620 0x7f7d3c0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 -- 172.21.15.148:0/2402927636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380094f0 con 0x7f7d3c15d580 2023-10-10T20:25:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d421db700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400bb60 tx=0x7f7d340097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400c7e0 con 0x7f7d3c15d580 2023-10-10T20:25:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400c940 con 0x7f7d3c15d580 2023-10-10T20:25:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0b16e0 con 0x7f7d3c15d580 2023-10-10T20:25:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400cd00 con 0x7f7d3c15d580 2023-10-10T20:25:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.492+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0b0b50 con 0x7f7d3c15d580 2023-10-10T20:25:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.493+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c142fe0 con 0x7f7d3c15d580 2023-10-10T20:25:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.493+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d34014620 con 0x7f7d3c15d580 2023-10-10T20:25:28.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.494+0000 7f7d31ffb700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.494+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7d34084420 con 0x7f7d3c15d580 2023-10-10T20:25:28.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.494+0000 7f7d419da700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.494+0000 7f7d419da700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7d380099b0 tx=0x7f7d38016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.498+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d3404c5a0 con 0x7f7d3c15d580 2023-10-10T20:25:28.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.677+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d3c04c3c0 con 0x7f7d20066f30 2023-10-10T20:25:28.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.685+0000 7f7d31ffb700 1 -- 172.21.15.148:0/2402927636 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285493 (secure 0 0 0) 0x7f7d3c04c3c0 con 0x7f7d20066f30 2023-10-10T20:25:28.690 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 msgr2=0x7f7d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7d380099b0 tx=0x7f7d38016000 comp rx=0 tx=0).stop 2023-10-10T20:25:28.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 msgr2=0x7f7d3c0b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400bb60 tx=0x7f7d340097f0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 shutdown_connections 2023-10-10T20:25:28.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d3c15d580 0x7f7d3c0b2600 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d3c157870 0x7f7d3c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 --2- 172.21.15.148:0/2402927636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3c15a620 0x7f7d3c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 >> 172.21.15.148:0/2402927636 conn(0x7f7d3c068ce0 msgr2=0x7f7d3c159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 shutdown_connections 2023-10-10T20:25:28.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.688+0000 7f7d43c3e700 1 -- 172.21.15.148:0/2402927636 wait complete. 2023-10-10T20:25:28.695 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:25:28.701 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2023-10-10T20:25:27.401262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555728,"statfs":{"total":1600000000,"available":1593066298,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.771321"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651713+0000","last_change":"2023-10-10T20:25:24.651440+0000","last_active":"2023-10-10T20:25:24.651713+0000","last_peered":"2023-10-10T20:25:24.651713+0000","last_clean":"2023-10-10T20:25:24.651713+0000","last_became_active":"2023-10-10T20:25:24.650772+0000","last_became_peered":"2023-10-10T20:25:24.650772+0000","last_unstale":"2023-10-10T20:25:24.651713+0000","last_undegraded":"2023-10-10T20:25:24.651713+0000","last_fullsized":"2023-10-10T20:25:24.651713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:30:29.602041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657264+0000","last_change":"2023-10-10T20:25:24.657220+0000","last_active":"2023-10-10T20:25:24.657264+0000","last_peered":"2023-10-10T20:25:24.657264+0000","last_clean":"2023-10-10T20:25:24.657264+0000","last_became_active":"2023-10-10T20:25:24.655581+0000","last_became_peered":"2023-10-10T20:25:24.655581+0000","last_unstale":"2023-10-10T20:25:24.657264+0000","last_undegraded":"2023-10-10T20:25:24.657264+0000","last_fullsized":"2023-10-10T20:25:24.657264+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:26.014253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654097+0000","last_change":"2023-10-10T20:25:24.653768+0000","last_active":"2023-10-10T20:25:24.654097+0000","last_peered":"2023-10-10T20:25:24.654097+0000","last_clean":"2023-10-10T20:25:24.654097+0000","last_became_active":"2023-10-10T20:25:24.652640+0000","last_became_peered":"2023-10-10T20:25:24.652640+0000","last_unstale":"2023-10-10T20:25:24.654097+0000","last_undegraded":"2023-10-10T20:25:24.654097+0000","last_fullsized":"2023-10-10T20:25:24.654097+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:46.680434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655938+0000","last_change":"2023-10-10T20:25:24.655589+0000","last_active":"2023-10-10T20:25:24.655938+0000","last_peered":"2023-10-10T20:25:24.655938+0000","last_clean":"2023-10-10T20:25:24.655938+0000","last_became_active":"2023-10-10T20:25:24.654857+0000","last_became_peered":"2023-10-10T20:25:24.654857+0000","last_unstale":"2023-10-10T20:25:24.655938+0000","last_undegraded":"2023-10-10T20:25:24.655938+0000","last_fullsized":"2023-10-10T20:25:24.655938+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:01:29.670966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657708+0000","last_change":"2023-10-10T20:25:24.657649+0000","last_active":"2023-10-10T20:25:24.657708+0000","last_peered":"2023-10-10T20:25:24.657708+0000","last_clean":"2023-10-10T20:25:24.657708+0000","last_became_active":"2023-10-10T20:25:24.657205+0000","last_became_peered":"2023-10-10T20:25:24.657205+0000","last_unstale":"2023-10-10T20:25:24.657708+0000","last_undegraded":"2023-10-10T20:25:24.657708+0000","last_fullsized":"2023-10-10T20:25:24.657708+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:53.834324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654132+0000","last_change":"2023-10-10T20:25:24.652946+0000","last_active":"2023-10-10T20:25:24.654132+0000","last_peered":"2023-10-10T20:25:24.654132+0000","last_clean":"2023-10-10T20:25:24.654132+0000","last_became_active":"2023-10-10T20:25:24.652406+0000","last_became_peered":"2023-10-10T20:25:24.652406+0000","last_unstale":"2023-10-10T20:25:24.654132+0000","last_undegraded":"2023-10-10T20:25:24.654132+0000","last_fullsized":"2023-10-10T20:25:24.654132+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:17:24.859536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.646593+0000","last_change":"2023-10-10T20:25:24.646346+0000","last_active":"2023-10-10T20:25:24.646593+0000","last_peered":"2023-10-10T20:25:24.646593+0000","last_clean":"2023-10-10T20:25:24.646593+0000","last_became_active":"2023-10-10T20:25:24.646103+0000","last_became_peered":"2023-10-10T20:25:24.646103+0000","last_unstale":"2023-10-10T20:25:24.646593+0000","last_undegraded":"2023-10-10T20:25:24.646593+0000","last_fullsized":"2023-10-10T20:25:24.646593+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:53.512364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654142+0000","last_change":"2023-10-10T20:25:24.654076+0000","last_active":"2023-10-10T20:25:24.654142+0000","last_peered":"2023-10-10T20:25:24.654142+0000","last_clean":"2023-10-10T20:25:24.654142+0000","last_became_active":"2023-10-10T20:25:24.653533+0000","last_became_peered":"2023-10-10T20:25:24.653533+0000","last_unstale":"2023-10-10T20:25:24.654142+0000","last_undegraded":"2023-10-10T20:25:24.654142+0000","last_fullsized":"2023-10-10T20:25:24.654142+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:59.171859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655956+0000","last_change":"2023-10-10T20:25:24.655714+0000","last_active":"2023-10-10T20:25:24.655956+0000","last_peered":"2023-10-10T20:25:24.655956+0000","last_clean":"2023-10-10T20:25:24.655956+0000","last_became_active":"2023-10-10T20:25:24.655355+0000","last_became_peered":"2023-10-10T20:25:24.655355+0000","last_unstale":"2023-10-10T20:25:24.655956+0000","last_undegraded":"2023-10-10T20:25:24.655956+0000","last_fullsized":"2023-10-10T20:25:24.655956+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:55:47.564111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650356+0000","last_change":"2023-10-10T20:25:24.650192+0000","last_active":"2023-10-10T20:25:24.650356+0000","last_peered":"2023-10-10T20:25:24.650356+0000","last_clean":"2023-10-10T20:25:24.650356+0000","last_became_active":"2023-10-10T20:25:24.649973+0000","last_became_peered":"2023-10-10T20:25:24.649973+0000","last_unstale":"2023-10-10T20:25:24.650356+0000","last_undegraded":"2023-10-10T20:25:24.650356+0000","last_fullsized":"2023-10-10T20:25:24.650356+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:35.603986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654205+0000","last_change":"2023-10-10T20:25:24.654010+0000","last_active":"2023-10-10T20:25:24.654205+0000","last_peered":"2023-10-10T20:25:24.654205+0000","last_clean":"2023-10-10T20:25:24.654205+0000","last_became_active":"2023-10-10T20:25:24.653757+0000","last_became_peered":"2023-10-10T20:25:24.653757+0000","last_unstale":"2023-10-10T20:25:24.654205+0000","last_undegraded":"2023-10-10T20:25:24.654205+0000","last_fullsized":"2023-10-10T20:25:24.654205+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:32.738474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650374+0000","last_change":"2023-10-10T20:25:24.650221+0000","last_active":"2023-10-10T20:25:24.650374+0000","last_peered":"2023-10-10T20:25:24.650374+0000","last_clean":"2023-10-10T20:25:24.650374+0000","last_became_active":"2023-10-10T20:25:24.649595+0000","last_became_peered":"2023-10-10T20:25:24.649595+0000","last_unstale":"2023-10-10T20:25:24.650374+0000","last_undegraded":"2023-10-10T20:25:24.650374+0000","last_fullsized":"2023-10-10T20:25:24.650374+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:10.834318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654128+0000","last_change":"2023-10-10T20:25:24.653941+0000","last_active":"2023-10-10T20:25:24.654128+0000","last_peered":"2023-10-10T20:25:24.654128+0000","last_clean":"2023-10-10T20:25:24.654128+0000","last_became_active":"2023-10-10T20:25:24.652935+0000","last_became_peered":"2023-10-10T20:25:24.652935+0000","last_unstale":"2023-10-10T20:25:24.654128+0000","last_undegraded":"2023-10-10T20:25:24.654128+0000","last_fullsized":"2023-10-10T20:25:24.654128+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655617+0000","last_change":"2023-10-10T20:25:24.655422+0000","last_active":"2023-10-10T20:25:24.655617+0000","last_peered":"2023-10-10T20:25:24.655617+0000","last_clean":"2023-10-10T20:25:24.655617+0000","last_became_active":"2023-10-10T20:25:24.654904+0000","last_became_peered":"2023-10-10T20:25:24.654904+0000","last_unstale":"2023-10-10T20:25:24.655617+0000","last_undegraded":"2023-10-10T20:25:24.655617+0000","last_fullsized":"2023-10-10T20:25:24.655617+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.563127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656959+0000","last_change":"2023-10-10T20:25:24.656910+0000","last_active":"2023-10-10T20:25:24.656959+0000","last_peered":"2023-10-10T20:25:24.656959+0000","last_clean":"2023-10-10T20:25:24.656959+0000","last_became_active":"2023-10-10T20:25:24.656728+0000","last_became_peered":"2023-10-10T20:25:24.656728+0000","last_unstale":"2023-10-10T20:25:24.656959+0000","last_undegraded":"2023-10-10T20:25:24.656959+0000","last_fullsized":"2023-10-10T20:25:24.656959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:31.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651745+0000","last_change":"2023-10-10T20:25:24.651657+0000","last_active":"2023-10-10T20:25:24.651745+0000","last_peered":"2023-10-10T20:25:24.651745+0000","last_clean":"2023-10-10T20:25:24.651745+0000","last_became_active":"2023-10-10T20:25:24.651077+0000","last_became_peered":"2023-10-10T20:25:24.651077+0000","last_unstale":"2023-10-10T20:25:24.651745+0000","last_undegraded":"2023-10-10T20:25:24.651745+0000","last_fullsized":"2023-10-10T20:25:24.651745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:48:22.954740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657745+0000","last_change":"2023-10-10T20:25:24.657348+0000","last_active":"2023-10-10T20:25:24.657745+0000","last_peered":"2023-10-10T20:25:24.657745+0000","last_clean":"2023-10-10T20:25:24.657745+0000","last_became_active":"2023-10-10T20:25:24.656104+0000","last_became_peered":"2023-10-10T20:25:24.656104+0000","last_unstale":"2023-10-10T20:25:24.657745+0000","last_undegraded":"2023-10-10T20:25:24.657745+0000","last_fullsized":"2023-10-10T20:25:24.657745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:04:24.784388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651379+0000","last_change":"2023-10-10T20:25:24.650698+0000","last_active":"2023-10-10T20:25:24.651379+0000","last_peered":"2023-10-10T20:25:24.651379+0000","last_clean":"2023-10-10T20:25:24.651379+0000","last_became_active":"2023-10-10T20:25:24.650452+0000","last_became_peered":"2023-10-10T20:25:24.650452+0000","last_unstale":"2023-10-10T20:25:24.651379+0000","last_undegraded":"2023-10-10T20:25:24.651379+0000","last_fullsized":"2023-10-10T20:25:24.651379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:41.591721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654732+0000","last_change":"2023-10-10T20:25:24.654673+0000","last_active":"2023-10-10T20:25:24.654732+0000","last_peered":"2023-10-10T20:25:24.654732+0000","last_clean":"2023-10-10T20:25:24.654732+0000","last_became_active":"2023-10-10T20:25:24.652973+0000","last_became_peered":"2023-10-10T20:25:24.652973+0000","last_unstale":"2023-10-10T20:25:24.654732+0000","last_undegraded":"2023-10-10T20:25:24.654732+0000","last_fullsized":"2023-10-10T20:25:24.654732+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:34.264510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654189+0000","last_change":"2023-10-10T20:25:24.653998+0000","last_active":"2023-10-10T20:25:24.654189+0000","last_peered":"2023-10-10T20:25:24.654189+0000","last_clean":"2023-10-10T20:25:24.654189+0000","last_became_active":"2023-10-10T20:25:24.653574+0000","last_became_peered":"2023-10-10T20:25:24.653574+0000","last_unstale":"2023-10-10T20:25:24.654189+0000","last_undegraded":"2023-10-10T20:25:24.654189+0000","last_fullsized":"2023-10-10T20:25:24.654189+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:21.626071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657835+0000","last_change":"2023-10-10T20:25:24.657785+0000","last_active":"2023-10-10T20:25:24.657835+0000","last_peered":"2023-10-10T20:25:24.657835+0000","last_clean":"2023-10-10T20:25:24.657835+0000","last_became_active":"2023-10-10T20:25:24.657545+0000","last_became_peered":"2023-10-10T20:25:24.657545+0000","last_unstale":"2023-10-10T20:25:24.657835+0000","last_undegraded":"2023-10-10T20:25:24.657835+0000","last_fullsized":"2023-10-10T20:25:24.657835+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:49:06.922751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655191+0000","last_change":"2023-10-10T20:25:24.655129+0000","last_active":"2023-10-10T20:25:24.655191+0000","last_peered":"2023-10-10T20:25:24.655191+0000","last_clean":"2023-10-10T20:25:24.655191+0000","last_became_active":"2023-10-10T20:25:24.654743+0000","last_became_peered":"2023-10-10T20:25:24.654743+0000","last_unstale":"2023-10-10T20:25:24.655191+0000","last_undegraded":"2023-10-10T20:25:24.655191+0000","last_fullsized":"2023-10-10T20:25:24.655191+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:36.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654858+0000","last_change":"2023-10-10T20:25:24.654724+0000","last_active":"2023-10-10T20:25:24.654858+0000","last_peered":"2023-10-10T20:25:24.654858+0000","last_clean":"2023-10-10T20:25:24.654858+0000","last_became_active":"2023-10-10T20:25:24.654409+0000","last_became_peered":"2023-10-10T20:25:24.654409+0000","last_unstale":"2023-10-10T20:25:24.654858+0000","last_undegraded":"2023-10-10T20:25:24.654858+0000","last_fullsized":"2023-10-10T20:25:24.654858+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:51:49.367967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655605+0000","last_change":"2023-10-10T20:25:24.655518+0000","last_active":"2023-10-10T20:25:24.655605+0000","last_peered":"2023-10-10T20:25:24.655605+0000","last_clean":"2023-10-10T20:25:24.655605+0000","last_became_active":"2023-10-10T20:25:24.655283+0000","last_became_peered":"2023-10-10T20:25:24.655283+0000","last_unstale":"2023-10-10T20:25:24.655605+0000","last_undegraded":"2023-10-10T20:25:24.655605+0000","last_fullsized":"2023-10-10T20:25:24.655605+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.563127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654171+0000","last_change":"2023-10-10T20:25:24.653836+0000","last_active":"2023-10-10T20:25:24.654171+0000","last_peered":"2023-10-10T20:25:24.654171+0000","last_clean":"2023-10-10T20:25:24.654171+0000","last_became_active":"2023-10-10T20:25:24.653219+0000","last_became_peered":"2023-10-10T20:25:24.653219+0000","last_unstale":"2023-10-10T20:25:24.654171+0000","last_undegraded":"2023-10-10T20:25:24.654171+0000","last_fullsized":"2023-10-10T20:25:24.654171+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:20.958148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648825+0000","last_change":"2023-10-10T20:25:24.647543+0000","last_active":"2023-10-10T20:25:24.648825+0000","last_peered":"2023-10-10T20:25:24.648825+0000","last_clean":"2023-10-10T20:25:24.648825+0000","last_became_active":"2023-10-10T20:25:24.647049+0000","last_became_peered":"2023-10-10T20:25:24.647049+0000","last_unstale":"2023-10-10T20:25:24.648825+0000","last_undegraded":"2023-10-10T20:25:24.648825+0000","last_fullsized":"2023-10-10T20:25:24.648825+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:30:12.692223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652193+0000","last_change":"2023-10-10T20:25:24.652097+0000","last_active":"2023-10-10T20:25:24.652193+0000","last_peered":"2023-10-10T20:25:24.652193+0000","last_clean":"2023-10-10T20:25:24.652193+0000","last_became_active":"2023-10-10T20:25:24.651778+0000","last_became_peered":"2023-10-10T20:25:24.651778+0000","last_unstale":"2023-10-10T20:25:24.652193+0000","last_undegraded":"2023-10-10T20:25:24.652193+0000","last_fullsized":"2023-10-10T20:25:24.652193+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:50.971429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657479+0000","last_change":"2023-10-10T20:25:24.657418+0000","last_active":"2023-10-10T20:25:24.657479+0000","last_peered":"2023-10-10T20:25:24.657479+0000","last_clean":"2023-10-10T20:25:24.657479+0000","last_became_active":"2023-10-10T20:25:24.657220+0000","last_became_peered":"2023-10-10T20:25:24.657220+0000","last_unstale":"2023-10-10T20:25:24.657479+0000","last_undegraded":"2023-10-10T20:25:24.657479+0000","last_fullsized":"2023-10-10T20:25:24.657479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:04.606687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655367+0000","last_change":"2023-10-10T20:25:24.655305+0000","last_active":"2023-10-10T20:25:24.655367+0000","last_peered":"2023-10-10T20:25:24.655367+0000","last_clean":"2023-10-10T20:25:24.655367+0000","last_became_active":"2023-10-10T20:25:24.655026+0000","last_became_peered":"2023-10-10T20:25:24.655026+0000","last_unstale":"2023-10-10T20:25:24.655367+0000","last_undegraded":"2023-10-10T20:25:24.655367+0000","last_fullsized":"2023-10-10T20:25:24.655367+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:26.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657716+0000","last_change":"2023-10-10T20:25:24.655372+0000","last_active":"2023-10-10T20:25:24.657716+0000","last_peered":"2023-10-10T20:25:24.657716+0000","last_clean":"2023-10-10T20:25:24.657716+0000","last_became_active":"2023-10-10T20:25:24.655112+0000","last_became_peered":"2023-10-10T20:25:24.655112+0000","last_unstale":"2023-10-10T20:25:24.657716+0000","last_undegraded":"2023-10-10T20:25:24.657716+0000","last_fullsized":"2023-10-10T20:25:24.657716+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:36.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648844+0000","last_change":"2023-10-10T20:25:24.647676+0000","last_active":"2023-10-10T20:25:24.648844+0000","last_peered":"2023-10-10T20:25:24.648844+0000","last_clean":"2023-10-10T20:25:24.648844+0000","last_became_active":"2023-10-10T20:25:24.647327+0000","last_became_peered":"2023-10-10T20:25:24.647327+0000","last_unstale":"2023-10-10T20:25:24.648844+0000","last_undegraded":"2023-10-10T20:25:24.648844+0000","last_fullsized":"2023-10-10T20:25:24.648844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:58:13.613317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654166+0000","last_change":"2023-10-10T20:25:24.653160+0000","last_active":"2023-10-10T20:25:24.654166+0000","last_peered":"2023-10-10T20:25:24.654166+0000","last_clean":"2023-10-10T20:25:24.654166+0000","last_became_active":"2023-10-10T20:25:24.652936+0000","last_became_peered":"2023-10-10T20:25:24.652936+0000","last_unstale":"2023-10-10T20:25:24.654166+0000","last_undegraded":"2023-10-10T20:25:24.654166+0000","last_fullsized":"2023-10-10T20:25:24.654166+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:46:56.089847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651502+0000","last_change":"2023-10-10T20:25:24.651254+0000","last_active":"2023-10-10T20:25:24.651502+0000","last_peered":"2023-10-10T20:25:24.651502+0000","last_clean":"2023-10-10T20:25:24.651502+0000","last_became_active":"2023-10-10T20:25:24.650710+0000","last_became_peered":"2023-10-10T20:25:24.650710+0000","last_unstale":"2023-10-10T20:25:24.651502+0000","last_undegraded":"2023-10-10T20:25:24.651502+0000","last_fullsized":"2023-10-10T20:25:24.651502+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:04:46.775057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"130'4","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664508+0000","last_change":"2023-10-10T20:25:23.640916+0000","last_active":"2023-10-10T20:25:24.664508+0000","last_peered":"2023-10-10T20:25:24.664508+0000","last_clean":"2023-10-10T20:25:24.664508+0000","last_became_active":"2023-10-10T20:25:23.640695+0000","last_became_peered":"2023-10-10T20:25:23.640695+0000","last_unstale":"2023-10-10T20:25:24.664508+0000","last_undegraded":"2023-10-10T20:25:24.664508+0000","last_fullsized":"2023-10-10T20:25:24.664508+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:28.224223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652177+0000","last_change":"2023-10-10T20:25:24.652057+0000","last_active":"2023-10-10T20:25:24.652177+0000","last_peered":"2023-10-10T20:25:24.652177+0000","last_clean":"2023-10-10T20:25:24.652177+0000","last_became_active":"2023-10-10T20:25:24.651827+0000","last_became_peered":"2023-10-10T20:25:24.651827+0000","last_unstale":"2023-10-10T20:25:24.652177+0000","last_undegraded":"2023-10-10T20:25:24.652177+0000","last_fullsized":"2023-10-10T20:25:24.652177+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:46.680434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664235+0000","last_change":"2023-10-10T20:25:23.642233+0000","last_active":"2023-10-10T20:25:24.664235+0000","last_peered":"2023-10-10T20:25:24.664235+0000","last_clean":"2023-10-10T20:25:24.664235+0000","last_became_active":"2023-10-10T20:25:23.641603+0000","last_became_peered":"2023-10-10T20:25:23.641603+0000","last_unstale":"2023-10-10T20:25:24.664235+0000","last_undegraded":"2023-10-10T20:25:24.664235+0000","last_fullsized":"2023-10-10T20:25:24.664235+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:20.621784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654150+0000","last_change":"2023-10-10T20:25:24.653083+0000","last_active":"2023-10-10T20:25:24.654150+0000","last_peered":"2023-10-10T20:25:24.654150+0000","last_clean":"2023-10-10T20:25:24.654150+0000","last_became_active":"2023-10-10T20:25:24.652717+0000","last_became_peered":"2023-10-10T20:25:24.652717+0000","last_unstale":"2023-10-10T20:25:24.654150+0000","last_undegraded":"2023-10-10T20:25:24.654150+0000","last_fullsized":"2023-10-10T20:25:24.654150+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:41.204103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667191+0000","last_change":"2023-10-10T20:25:23.641354+0000","last_active":"2023-10-10T20:25:24.667191+0000","last_peered":"2023-10-10T20:25:24.667191+0000","last_clean":"2023-10-10T20:25:24.667191+0000","last_became_active":"2023-10-10T20:25:23.641151+0000","last_became_peered":"2023-10-10T20:25:23.641151+0000","last_unstale":"2023-10-10T20:25:24.667191+0000","last_undegraded":"2023-10-10T20:25:24.667191+0000","last_fullsized":"2023-10-10T20:25:24.667191+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:54.891283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657080+0000","last_change":"2023-10-10T20:25:24.657017+0000","last_active":"2023-10-10T20:25:24.657080+0000","last_peered":"2023-10-10T20:25:24.657080+0000","last_clean":"2023-10-10T20:25:24.657080+0000","last_became_active":"2023-10-10T20:25:24.656787+0000","last_became_peered":"2023-10-10T20:25:24.656787+0000","last_unstale":"2023-10-10T20:25:24.657080+0000","last_undegraded":"2023-10-10T20:25:24.657080+0000","last_fullsized":"2023-10-10T20:25:24.657080+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666897+0000","last_change":"2023-10-10T20:25:23.642924+0000","last_active":"2023-10-10T20:25:24.666897+0000","last_peered":"2023-10-10T20:25:24.666897+0000","last_clean":"2023-10-10T20:25:24.666897+0000","last_became_active":"2023-10-10T20:25:23.642462+0000","last_became_peered":"2023-10-10T20:25:23.642462+0000","last_unstale":"2023-10-10T20:25:24.666897+0000","last_undegraded":"2023-10-10T20:25:24.666897+0000","last_fullsized":"2023-10-10T20:25:24.666897+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:18.488919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656486+0000","last_change":"2023-10-10T20:25:24.656278+0000","last_active":"2023-10-10T20:25:24.656486+0000","last_peered":"2023-10-10T20:25:24.656486+0000","last_clean":"2023-10-10T20:25:24.656486+0000","last_became_active":"2023-10-10T20:25:24.655773+0000","last_became_peered":"2023-10-10T20:25:24.655773+0000","last_unstale":"2023-10-10T20:25:24.656486+0000","last_undegraded":"2023-10-10T20:25:24.656486+0000","last_fullsized":"2023-10-10T20:25:24.656486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:21:54.876803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664039+0000","last_change":"2023-10-10T20:25:23.642833+0000","last_active":"2023-10-10T20:25:24.664039+0000","last_peered":"2023-10-10T20:25:24.664039+0000","last_clean":"2023-10-10T20:25:24.664039+0000","last_became_active":"2023-10-10T20:25:23.642581+0000","last_became_peered":"2023-10-10T20:25:23.642581+0000","last_unstale":"2023-10-10T20:25:24.664039+0000","last_undegraded":"2023-10-10T20:25:24.664039+0000","last_fullsized":"2023-10-10T20:25:24.664039+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:51.227809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657149+0000","last_change":"2023-10-10T20:25:24.657084+0000","last_active":"2023-10-10T20:25:24.657149+0000","last_peered":"2023-10-10T20:25:24.657149+0000","last_clean":"2023-10-10T20:25:24.657149+0000","last_became_active":"2023-10-10T20:25:24.655273+0000","last_became_peered":"2023-10-10T20:25:24.655273+0000","last_unstale":"2023-10-10T20:25:24.657149+0000","last_undegraded":"2023-10-10T20:25:24.657149+0000","last_fullsized":"2023-10-10T20:25:24.657149+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:40.723203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666067+0000","last_change":"2023-10-10T20:25:23.647030+0000","last_active":"2023-10-10T20:25:24.666067+0000","last_peered":"2023-10-10T20:25:24.666067+0000","last_clean":"2023-10-10T20:25:24.666067+0000","last_became_active":"2023-10-10T20:25:23.646801+0000","last_became_peered":"2023-10-10T20:25:23.646801+0000","last_unstale":"2023-10-10T20:25:24.666067+0000","last_undegraded":"2023-10-10T20:25:24.666067+0000","last_fullsized":"2023-10-10T20:25:24.666067+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:45.130605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652175+0000","last_change":"2023-10-10T20:25:24.651771+0000","last_active":"2023-10-10T20:25:24.652175+0000","last_peered":"2023-10-10T20:25:24.652175+0000","last_clean":"2023-10-10T20:25:24.652175+0000","last_became_active":"2023-10-10T20:25:24.651424+0000","last_became_peered":"2023-10-10T20:25:24.651424+0000","last_unstale":"2023-10-10T20:25:24.652175+0000","last_undegraded":"2023-10-10T20:25:24.652175+0000","last_fullsized":"2023-10-10T20:25:24.652175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:40.048292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648930+0000","last_change":"2023-10-10T20:25:23.647122+0000","last_active":"2023-10-10T20:25:24.648930+0000","last_peered":"2023-10-10T20:25:24.648930+0000","last_clean":"2023-10-10T20:25:24.648930+0000","last_became_active":"2023-10-10T20:25:23.646944+0000","last_became_peered":"2023-10-10T20:25:23.646944+0000","last_unstale":"2023-10-10T20:25:24.648930+0000","last_undegraded":"2023-10-10T20:25:24.648930+0000","last_fullsized":"2023-10-10T20:25:24.648930+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:52.508077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.646493+0000","last_change":"2023-10-10T20:25:24.646348+0000","last_active":"2023-10-10T20:25:24.646493+0000","last_peered":"2023-10-10T20:25:24.646493+0000","last_clean":"2023-10-10T20:25:24.646493+0000","last_became_active":"2023-10-10T20:25:24.646112+0000","last_became_peered":"2023-10-10T20:25:24.646112+0000","last_unstale":"2023-10-10T20:25:24.646493+0000","last_undegraded":"2023-10-10T20:25:24.646493+0000","last_fullsized":"2023-10-10T20:25:24.646493+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:42.256826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645173+0000","last_change":"2023-10-10T20:25:23.646537+0000","last_active":"2023-10-10T20:25:24.645173+0000","last_peered":"2023-10-10T20:25:24.645173+0000","last_clean":"2023-10-10T20:25:24.645173+0000","last_became_active":"2023-10-10T20:25:23.645588+0000","last_became_peered":"2023-10-10T20:25:23.645588+0000","last_unstale":"2023-10-10T20:25:24.645173+0000","last_undegraded":"2023-10-10T20:25:24.645173+0000","last_fullsized":"2023-10-10T20:25:24.645173+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:15.558840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.653697+0000","last_change":"2023-10-10T20:25:24.653498+0000","last_active":"2023-10-10T20:25:24.653697+0000","last_peered":"2023-10-10T20:25:24.653697+0000","last_clean":"2023-10-10T20:25:24.653697+0000","last_became_active":"2023-10-10T20:25:24.653221+0000","last_became_peered":"2023-10-10T20:25:24.653221+0000","last_unstale":"2023-10-10T20:25:24.653697+0000","last_undegraded":"2023-10-10T20:25:24.653697+0000","last_fullsized":"2023-10-10T20:25:24.653697+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:02.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648528+0000","last_change":"2023-10-10T20:25:23.643011+0000","last_active":"2023-10-10T20:25:24.648528+0000","last_peered":"2023-10-10T20:25:24.648528+0000","last_clean":"2023-10-10T20:25:24.648528+0000","last_became_active":"2023-10-10T20:25:23.642736+0000","last_became_peered":"2023-10-10T20:25:23.642736+0000","last_unstale":"2023-10-10T20:25:24.648528+0000","last_undegraded":"2023-10-10T20:25:24.648528+0000","last_fullsized":"2023-10-10T20:25:24.648528+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:57.679851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647328+0000","last_change":"2023-10-10T20:25:24.647280+0000","last_active":"2023-10-10T20:25:24.647328+0000","last_peered":"2023-10-10T20:25:24.647328+0000","last_clean":"2023-10-10T20:25:24.647328+0000","last_became_active":"2023-10-10T20:25:24.647065+0000","last_became_peered":"2023-10-10T20:25:24.647065+0000","last_unstale":"2023-10-10T20:25:24.647328+0000","last_undegraded":"2023-10-10T20:25:24.647328+0000","last_fullsized":"2023-10-10T20:25:24.647328+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.649479+0000","last_change":"2023-10-10T20:25:23.647452+0000","last_active":"2023-10-10T20:25:24.649479+0000","last_peered":"2023-10-10T20:25:24.649479+0000","last_clean":"2023-10-10T20:25:24.649479+0000","last_became_active":"2023-10-10T20:25:23.646987+0000","last_became_peered":"2023-10-10T20:25:23.646987+0000","last_unstale":"2023-10-10T20:25:24.649479+0000","last_undegraded":"2023-10-10T20:25:24.649479+0000","last_fullsized":"2023-10-10T20:25:24.649479+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:13:55.035835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658276+0000","last_change":"2023-10-10T20:25:24.658109+0000","last_active":"2023-10-10T20:25:24.658276+0000","last_peered":"2023-10-10T20:25:24.658276+0000","last_clean":"2023-10-10T20:25:24.658276+0000","last_became_active":"2023-10-10T20:25:24.657882+0000","last_became_peered":"2023-10-10T20:25:24.657882+0000","last_unstale":"2023-10-10T20:25:24.658276+0000","last_undegraded":"2023-10-10T20:25:24.658276+0000","last_fullsized":"2023-10-10T20:25:24.658276+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:40.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"130'3","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665317+0000","last_change":"2023-10-10T20:25:23.646024+0000","last_active":"2023-10-10T20:25:24.665317+0000","last_peered":"2023-10-10T20:25:24.665317+0000","last_clean":"2023-10-10T20:25:24.665317+0000","last_became_active":"2023-10-10T20:25:23.645822+0000","last_became_peered":"2023-10-10T20:25:23.645822+0000","last_unstale":"2023-10-10T20:25:24.665317+0000","last_undegraded":"2023-10-10T20:25:24.665317+0000","last_fullsized":"2023-10-10T20:25:24.665317+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:33.260222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652608+0000","last_change":"2023-10-10T20:25:24.652544+0000","last_active":"2023-10-10T20:25:24.652608+0000","last_peered":"2023-10-10T20:25:24.652608+0000","last_clean":"2023-10-10T20:25:24.652608+0000","last_became_active":"2023-10-10T20:25:24.652277+0000","last_became_peered":"2023-10-10T20:25:24.652277+0000","last_unstale":"2023-10-10T20:25:24.652608+0000","last_undegraded":"2023-10-10T20:25:24.652608+0000","last_fullsized":"2023-10-10T20:25:24.652608+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:04.444619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"130'6","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.668691+0000","last_change":"2023-10-10T20:25:23.647101+0000","last_active":"2023-10-10T20:25:24.668691+0000","last_peered":"2023-10-10T20:25:24.668691+0000","last_clean":"2023-10-10T20:25:24.668691+0000","last_became_active":"2023-10-10T20:25:23.646884+0000","last_became_peered":"2023-10-10T20:25:23.646884+0000","last_unstale":"2023-10-10T20:25:24.668691+0000","last_undegraded":"2023-10-10T20:25:24.668691+0000","last_fullsized":"2023-10-10T20:25:24.668691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:39.540397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655425+0000","last_change":"2023-10-10T20:25:24.655343+0000","last_active":"2023-10-10T20:25:24.655425+0000","last_peered":"2023-10-10T20:25:24.655425+0000","last_clean":"2023-10-10T20:25:24.655425+0000","last_became_active":"2023-10-10T20:25:24.654972+0000","last_became_peered":"2023-10-10T20:25:24.654972+0000","last_unstale":"2023-10-10T20:25:24.655425+0000","last_undegraded":"2023-10-10T20:25:24.655425+0000","last_fullsized":"2023-10-10T20:25:24.655425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:59.171859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666959+0000","last_change":"2023-10-10T20:25:23.642543+0000","last_active":"2023-10-10T20:25:24.666959+0000","last_peered":"2023-10-10T20:25:24.666959+0000","last_clean":"2023-10-10T20:25:24.666959+0000","last_became_active":"2023-10-10T20:25:23.642160+0000","last_became_peered":"2023-10-10T20:25:23.642160+0000","last_unstale":"2023-10-10T20:25:24.666959+0000","last_undegraded":"2023-10-10T20:25:24.666959+0000","last_fullsized":"2023-10-10T20:25:24.666959+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:52.552940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.653278+0000","last_change":"2023-10-10T20:25:24.653045+0000","last_active":"2023-10-10T20:25:24.653278+0000","last_peered":"2023-10-10T20:25:24.653278+0000","last_clean":"2023-10-10T20:25:24.653278+0000","last_became_active":"2023-10-10T20:25:24.652833+0000","last_became_peered":"2023-10-10T20:25:24.652833+0000","last_unstale":"2023-10-10T20:25:24.653278+0000","last_undegraded":"2023-10-10T20:25:24.653278+0000","last_fullsized":"2023-10-10T20:25:24.653278+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644406+0000","last_change":"2023-10-10T20:25:23.648449+0000","last_active":"2023-10-10T20:25:24.644406+0000","last_peered":"2023-10-10T20:25:24.644406+0000","last_clean":"2023-10-10T20:25:24.644406+0000","last_became_active":"2023-10-10T20:25:23.648279+0000","last_became_peered":"2023-10-10T20:25:23.648279+0000","last_unstale":"2023-10-10T20:25:24.644406+0000","last_undegraded":"2023-10-10T20:25:24.644406+0000","last_fullsized":"2023-10-10T20:25:24.644406+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:03.131761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658238+0000","last_change":"2023-10-10T20:25:24.657996+0000","last_active":"2023-10-10T20:25:24.658238+0000","last_peered":"2023-10-10T20:25:24.658238+0000","last_clean":"2023-10-10T20:25:24.658238+0000","last_became_active":"2023-10-10T20:25:24.657783+0000","last_became_peered":"2023-10-10T20:25:24.657783+0000","last_unstale":"2023-10-10T20:25:24.658238+0000","last_undegraded":"2023-10-10T20:25:24.658238+0000","last_fullsized":"2023-10-10T20:25:24.658238+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:22.406668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"130'8","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667763+0000","last_change":"2023-10-10T20:25:23.646310+0000","last_active":"2023-10-10T20:25:24.667763+0000","last_peered":"2023-10-10T20:25:24.667763+0000","last_clean":"2023-10-10T20:25:24.667763+0000","last_became_active":"2023-10-10T20:25:23.646083+0000","last_became_peered":"2023-10-10T20:25:23.646083+0000","last_unstale":"2023-10-10T20:25:24.667763+0000","last_undegraded":"2023-10-10T20:25:24.667763+0000","last_fullsized":"2023-10-10T20:25:24.667763+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:09.433016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654157+0000","last_change":"2023-10-10T20:25:24.653709+0000","last_active":"2023-10-10T20:25:24.654157+0000","last_peered":"2023-10-10T20:25:24.654157+0000","last_clean":"2023-10-10T20:25:24.654157+0000","last_became_active":"2023-10-10T20:25:24.653487+0000","last_became_peered":"2023-10-10T20:25:24.653487+0000","last_unstale":"2023-10-10T20:25:24.654157+0000","last_undegraded":"2023-10-10T20:25:24.654157+0000","last_fullsized":"2023-10-10T20:25:24.654157+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:20.396556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665817+0000","last_change":"2023-10-10T20:25:23.638278+0000","last_active":"2023-10-10T20:25:24.665817+0000","last_peered":"2023-10-10T20:25:24.665817+0000","last_clean":"2023-10-10T20:25:24.665817+0000","last_became_active":"2023-10-10T20:25:23.638042+0000","last_became_peered":"2023-10-10T20:25:23.638042+0000","last_unstale":"2023-10-10T20:25:24.665817+0000","last_undegraded":"2023-10-10T20:25:24.665817+0000","last_fullsized":"2023-10-10T20:25:24.665817+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:40.587433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651719+0000","last_change":"2023-10-10T20:25:24.651414+0000","last_active":"2023-10-10T20:25:24.651719+0000","last_peered":"2023-10-10T20:25:24.651719+0000","last_clean":"2023-10-10T20:25:24.651719+0000","last_became_active":"2023-10-10T20:25:24.651032+0000","last_became_peered":"2023-10-10T20:25:24.651032+0000","last_unstale":"2023-10-10T20:25:24.651719+0000","last_undegraded":"2023-10-10T20:25:24.651719+0000","last_fullsized":"2023-10-10T20:25:24.651719+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:58.969237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666669+0000","last_change":"2023-10-10T20:25:23.644481+0000","last_active":"2023-10-10T20:25:24.666669+0000","last_peered":"2023-10-10T20:25:24.666669+0000","last_clean":"2023-10-10T20:25:24.666669+0000","last_became_active":"2023-10-10T20:25:23.644277+0000","last_became_peered":"2023-10-10T20:25:23.644277+0000","last_unstale":"2023-10-10T20:25:24.666669+0000","last_undegraded":"2023-10-10T20:25:24.666669+0000","last_fullsized":"2023-10-10T20:25:24.666669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:52.552940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652250+0000","last_change":"2023-10-10T20:25:24.651532+0000","last_active":"2023-10-10T20:25:24.652250+0000","last_peered":"2023-10-10T20:25:24.652250+0000","last_clean":"2023-10-10T20:25:24.652250+0000","last_became_active":"2023-10-10T20:25:24.651293+0000","last_became_peered":"2023-10-10T20:25:24.651293+0000","last_unstale":"2023-10-10T20:25:24.652250+0000","last_undegraded":"2023-10-10T20:25:24.652250+0000","last_fullsized":"2023-10-10T20:25:24.652250+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:04.136048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665833+0000","last_change":"2023-10-10T20:25:23.642313+0000","last_active":"2023-10-10T20:25:24.665833+0000","last_peered":"2023-10-10T20:25:24.665833+0000","last_clean":"2023-10-10T20:25:24.665833+0000","last_became_active":"2023-10-10T20:25:23.642058+0000","last_became_peered":"2023-10-10T20:25:23.642058+0000","last_unstale":"2023-10-10T20:25:24.665833+0000","last_undegraded":"2023-10-10T20:25:24.665833+0000","last_fullsized":"2023-10-10T20:25:24.665833+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:25.873244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656997+0000","last_change":"2023-10-10T20:25:24.656929+0000","last_active":"2023-10-10T20:25:24.656997+0000","last_peered":"2023-10-10T20:25:24.656997+0000","last_clean":"2023-10-10T20:25:24.656997+0000","last_became_active":"2023-10-10T20:25:24.656657+0000","last_became_peered":"2023-10-10T20:25:24.656657+0000","last_unstale":"2023-10-10T20:25:24.656997+0000","last_undegraded":"2023-10-10T20:25:24.656997+0000","last_fullsized":"2023-10-10T20:25:24.656997+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:41.591721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644766+0000","last_change":"2023-10-10T20:25:23.639175+0000","last_active":"2023-10-10T20:25:24.644766+0000","last_peered":"2023-10-10T20:25:24.644766+0000","last_clean":"2023-10-10T20:25:24.644766+0000","last_became_active":"2023-10-10T20:25:23.638827+0000","last_became_peered":"2023-10-10T20:25:23.638827+0000","last_unstale":"2023-10-10T20:25:24.644766+0000","last_undegraded":"2023-10-10T20:25:24.644766+0000","last_fullsized":"2023-10-10T20:25:24.644766+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:40.626519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655580+0000","last_change":"2023-10-10T20:25:24.655114+0000","last_active":"2023-10-10T20:25:24.655580+0000","last_peered":"2023-10-10T20:25:24.655580+0000","last_clean":"2023-10-10T20:25:24.655580+0000","last_became_active":"2023-10-10T20:25:24.653274+0000","last_became_peered":"2023-10-10T20:25:24.653274+0000","last_unstale":"2023-10-10T20:25:24.655580+0000","last_undegraded":"2023-10-10T20:25:24.655580+0000","last_fullsized":"2023-10-10T20:25:24.655580+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:40.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667129+0000","last_change":"2023-10-10T20:25:23.640525+0000","last_active":"2023-10-10T20:25:24.667129+0000","last_peered":"2023-10-10T20:25:24.667129+0000","last_clean":"2023-10-10T20:25:24.667129+0000","last_became_active":"2023-10-10T20:25:23.640317+0000","last_became_peered":"2023-10-10T20:25:23.640317+0000","last_unstale":"2023-10-10T20:25:24.667129+0000","last_undegraded":"2023-10-10T20:25:24.667129+0000","last_fullsized":"2023-10-10T20:25:24.667129+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:18.488919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656755+0000","last_change":"2023-10-10T20:25:24.656657+0000","last_active":"2023-10-10T20:25:24.656755+0000","last_peered":"2023-10-10T20:25:24.656755+0000","last_clean":"2023-10-10T20:25:24.656755+0000","last_became_active":"2023-10-10T20:25:24.656046+0000","last_became_peered":"2023-10-10T20:25:24.656046+0000","last_unstale":"2023-10-10T20:25:24.656755+0000","last_undegraded":"2023-10-10T20:25:24.656755+0000","last_fullsized":"2023-10-10T20:25:24.656755+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:37:14.768034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647459+0000","last_change":"2023-10-10T20:25:23.642577+0000","last_active":"2023-10-10T20:25:24.647459+0000","last_peered":"2023-10-10T20:25:24.647459+0000","last_clean":"2023-10-10T20:25:24.647459+0000","last_became_active":"2023-10-10T20:25:23.642349+0000","last_became_peered":"2023-10-10T20:25:23.642349+0000","last_unstale":"2023-10-10T20:25:24.647459+0000","last_undegraded":"2023-10-10T20:25:24.647459+0000","last_fullsized":"2023-10-10T20:25:24.647459+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:11.677340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656439+0000","last_change":"2023-10-10T20:25:24.655979+0000","last_active":"2023-10-10T20:25:24.656439+0000","last_peered":"2023-10-10T20:25:24.656439+0000","last_clean":"2023-10-10T20:25:24.656439+0000","last_became_active":"2023-10-10T20:25:24.655547+0000","last_became_peered":"2023-10-10T20:25:24.655547+0000","last_unstale":"2023-10-10T20:25:24.656439+0000","last_undegraded":"2023-10-10T20:25:24.656439+0000","last_fullsized":"2023-10-10T20:25:24.656439+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:26.014253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650314+0000","last_change":"2023-10-10T20:25:23.639906+0000","last_active":"2023-10-10T20:25:24.650314+0000","last_peered":"2023-10-10T20:25:24.650314+0000","last_clean":"2023-10-10T20:25:24.650314+0000","last_became_active":"2023-10-10T20:25:23.639669+0000","last_became_peered":"2023-10-10T20:25:23.639669+0000","last_unstale":"2023-10-10T20:25:24.650314+0000","last_undegraded":"2023-10-10T20:25:24.650314+0000","last_fullsized":"2023-10-10T20:25:24.650314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:07.874385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658258+0000","last_change":"2023-10-10T20:25:24.658070+0000","last_active":"2023-10-10T20:25:24.658258+0000","last_peered":"2023-10-10T20:25:24.658258+0000","last_clean":"2023-10-10T20:25:24.658258+0000","last_became_active":"2023-10-10T20:25:24.657533+0000","last_became_peered":"2023-10-10T20:25:24.657533+0000","last_unstale":"2023-10-10T20:25:24.658258+0000","last_undegraded":"2023-10-10T20:25:24.658258+0000","last_fullsized":"2023-10-10T20:25:24.658258+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:52.273593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645222+0000","last_change":"2023-10-10T20:25:23.649325+0000","last_active":"2023-10-10T20:25:24.645222+0000","last_peered":"2023-10-10T20:25:24.645222+0000","last_clean":"2023-10-10T20:25:24.645222+0000","last_became_active":"2023-10-10T20:25:23.649093+0000","last_became_peered":"2023-10-10T20:25:23.649093+0000","last_unstale":"2023-10-10T20:25:24.645222+0000","last_undegraded":"2023-10-10T20:25:24.645222+0000","last_fullsized":"2023-10-10T20:25:24.645222+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:00.349551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647298+0000","last_change":"2023-10-10T20:25:24.647240+0000","last_active":"2023-10-10T20:25:24.647298+0000","last_peered":"2023-10-10T20:25:24.647298+0000","last_clean":"2023-10-10T20:25:24.647298+0000","last_became_active":"2023-10-10T20:25:24.647015+0000","last_became_peered":"2023-10-10T20:25:24.647015+0000","last_unstale":"2023-10-10T20:25:24.647298+0000","last_undegraded":"2023-10-10T20:25:24.647298+0000","last_fullsized":"2023-10-10T20:25:24.647298+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:34.264510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667052+0000","last_change":"2023-10-10T20:25:23.648889+0000","last_active":"2023-10-10T20:25:24.667052+0000","last_peered":"2023-10-10T20:25:24.667052+0000","last_clean":"2023-10-10T20:25:24.667052+0000","last_became_active":"2023-10-10T20:25:23.648672+0000","last_became_peered":"2023-10-10T20:25:23.648672+0000","last_unstale":"2023-10-10T20:25:24.667052+0000","last_undegraded":"2023-10-10T20:25:24.667052+0000","last_fullsized":"2023-10-10T20:25:24.667052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:25.873244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652509+0000","last_change":"2023-10-10T20:25:24.652305+0000","last_active":"2023-10-10T20:25:24.652509+0000","last_peered":"2023-10-10T20:25:24.652509+0000","last_clean":"2023-10-10T20:25:24.652509+0000","last_became_active":"2023-10-10T20:25:24.651922+0000","last_became_peered":"2023-10-10T20:25:24.651922+0000","last_unstale":"2023-10-10T20:25:24.652509+0000","last_undegraded":"2023-10-10T20:25:24.652509+0000","last_fullsized":"2023-10-10T20:25:24.652509+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:07:06.620203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645430+0000","last_change":"2023-10-10T20:25:23.638429+0000","last_active":"2023-10-10T20:25:24.645430+0000","last_peered":"2023-10-10T20:25:24.645430+0000","last_clean":"2023-10-10T20:25:24.645430+0000","last_became_active":"2023-10-10T20:25:23.638232+0000","last_became_peered":"2023-10-10T20:25:23.638232+0000","last_unstale":"2023-10-10T20:25:24.645430+0000","last_undegraded":"2023-10-10T20:25:24.645430+0000","last_fullsized":"2023-10-10T20:25:24.645430+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:13:55.035835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651730+0000","last_change":"2023-10-10T20:25:24.651637+0000","last_active":"2023-10-10T20:25:24.651730+0000","last_peered":"2023-10-10T20:25:24.651730+0000","last_clean":"2023-10-10T20:25:24.651730+0000","last_became_active":"2023-10-10T20:25:24.651129+0000","last_became_peered":"2023-10-10T20:25:24.651129+0000","last_unstale":"2023-10-10T20:25:24.651730+0000","last_undegraded":"2023-10-10T20:25:24.651730+0000","last_fullsized":"2023-10-10T20:25:24.651730+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:55.985324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644563+0000","last_change":"2023-10-10T20:25:23.650971+0000","last_active":"2023-10-10T20:25:24.644563+0000","last_peered":"2023-10-10T20:25:24.644563+0000","last_clean":"2023-10-10T20:25:24.644563+0000","last_became_active":"2023-10-10T20:25:23.650304+0000","last_became_peered":"2023-10-10T20:25:23.650304+0000","last_unstale":"2023-10-10T20:25:24.644563+0000","last_undegraded":"2023-10-10T20:25:24.644563+0000","last_fullsized":"2023-10-10T20:25:24.644563+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:45.676146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647260+0000","last_change":"2023-10-10T20:25:24.647201+0000","last_active":"2023-10-10T20:25:24.647260+0000","last_peered":"2023-10-10T20:25:24.647260+0000","last_clean":"2023-10-10T20:25:24.647260+0000","last_became_active":"2023-10-10T20:25:24.646964+0000","last_became_peered":"2023-10-10T20:25:24.646964+0000","last_unstale":"2023-10-10T20:25:24.647260+0000","last_undegraded":"2023-10-10T20:25:24.647260+0000","last_fullsized":"2023-10-10T20:25:24.647260+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651419+0000","last_change":"2023-10-10T20:25:23.647969+0000","last_active":"2023-10-10T20:25:24.651419+0000","last_peered":"2023-10-10T20:25:24.651419+0000","last_clean":"2023-10-10T20:25:24.651419+0000","last_became_active":"2023-10-10T20:25:23.647721+0000","last_became_peered":"2023-10-10T20:25:23.647721+0000","last_unstale":"2023-10-10T20:25:24.651419+0000","last_undegraded":"2023-10-10T20:25:24.651419+0000","last_fullsized":"2023-10-10T20:25:24.651419+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:13.948239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657763+0000","last_change":"2023-10-10T20:25:24.657531+0000","last_active":"2023-10-10T20:25:24.657763+0000","last_peered":"2023-10-10T20:25:24.657763+0000","last_clean":"2023-10-10T20:25:24.657763+0000","last_became_active":"2023-10-10T20:25:24.656409+0000","last_became_peered":"2023-10-10T20:25:24.656409+0000","last_unstale":"2023-10-10T20:25:24.657763+0000","last_undegraded":"2023-10-10T20:25:24.657763+0000","last_fullsized":"2023-10-10T20:25:24.657763+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:49.348279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644498+0000","last_change":"2023-10-10T20:25:23.644312+0000","last_active":"2023-10-10T20:25:24.644498+0000","last_peered":"2023-10-10T20:25:24.644498+0000","last_clean":"2023-10-10T20:25:24.644498+0000","last_became_active":"2023-10-10T20:25:23.644073+0000","last_became_peered":"2023-10-10T20:25:23.644073+0000","last_unstale":"2023-10-10T20:25:24.644498+0000","last_undegraded":"2023-10-10T20:25:24.644498+0000","last_fullsized":"2023-10-10T20:25:24.644498+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:34.550368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652621+0000","last_change":"2023-10-10T20:25:24.652562+0000","last_active":"2023-10-10T20:25:24.652621+0000","last_peered":"2023-10-10T20:25:24.652621+0000","last_clean":"2023-10-10T20:25:24.652621+0000","last_became_active":"2023-10-10T20:25:24.652326+0000","last_became_peered":"2023-10-10T20:25:24.652326+0000","last_unstale":"2023-10-10T20:25:24.652621+0000","last_undegraded":"2023-10-10T20:25:24.652621+0000","last_fullsized":"2023-10-10T20:25:24.652621+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:01.638962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650054+0000","last_change":"2023-10-10T20:25:23.643319+0000","last_active":"2023-10-10T20:25:24.650054+0000","last_peered":"2023-10-10T20:25:24.650054+0000","last_clean":"2023-10-10T20:25:24.650054+0000","last_became_active":"2023-10-10T20:25:23.643108+0000","last_became_peered":"2023-10-10T20:25:23.643108+0000","last_unstale":"2023-10-10T20:25:24.650054+0000","last_undegraded":"2023-10-10T20:25:24.650054+0000","last_fullsized":"2023-10-10T20:25:24.650054+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:03.440331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655599+0000","last_change":"2023-10-10T20:25:24.655265+0000","last_active":"2023-10-10T20:25:24.655599+0000","last_peered":"2023-10-10T20:25:24.655599+0000","last_clean":"2023-10-10T20:25:24.655599+0000","last_became_active":"2023-10-10T20:25:24.654435+0000","last_became_peered":"2023-10-10T20:25:24.654435+0000","last_unstale":"2023-10-10T20:25:24.655599+0000","last_undegraded":"2023-10-10T20:25:24.655599+0000","last_fullsized":"2023-10-10T20:25:24.655599+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:22.406668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647654+0000","last_change":"2023-10-10T20:25:23.643033+0000","last_active":"2023-10-10T20:25:24.647654+0000","last_peered":"2023-10-10T20:25:24.647654+0000","last_clean":"2023-10-10T20:25:24.647654+0000","last_became_active":"2023-10-10T20:25:23.642803+0000","last_became_peered":"2023-10-10T20:25:23.642803+0000","last_unstale":"2023-10-10T20:25:24.647654+0000","last_undegraded":"2023-10-10T20:25:24.647654+0000","last_fullsized":"2023-10-10T20:25:24.647654+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:35.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652691+0000","last_change":"2023-10-10T20:25:24.652634+0000","last_active":"2023-10-10T20:25:24.652691+0000","last_peered":"2023-10-10T20:25:24.652691+0000","last_clean":"2023-10-10T20:25:24.652691+0000","last_became_active":"2023-10-10T20:25:24.652165+0000","last_became_peered":"2023-10-10T20:25:24.652165+0000","last_unstale":"2023-10-10T20:25:24.652691+0000","last_undegraded":"2023-10-10T20:25:24.652691+0000","last_fullsized":"2023-10-10T20:25:24.652691+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:17.152615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644637+0000","last_change":"2023-10-10T20:25:23.651071+0000","last_active":"2023-10-10T20:25:24.644637+0000","last_peered":"2023-10-10T20:25:24.644637+0000","last_clean":"2023-10-10T20:25:24.644637+0000","last_became_active":"2023-10-10T20:25:23.650533+0000","last_became_peered":"2023-10-10T20:25:23.650533+0000","last_unstale":"2023-10-10T20:25:24.644637+0000","last_undegraded":"2023-10-10T20:25:24.644637+0000","last_fullsized":"2023-10-10T20:25:24.644637+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:12.179132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655588+0000","last_change":"2023-10-10T20:25:24.655399+0000","last_active":"2023-10-10T20:25:24.655588+0000","last_peered":"2023-10-10T20:25:24.655588+0000","last_clean":"2023-10-10T20:25:24.655588+0000","last_became_active":"2023-10-10T20:25:24.655152+0000","last_became_peered":"2023-10-10T20:25:24.655152+0000","last_unstale":"2023-10-10T20:25:24.655588+0000","last_undegraded":"2023-10-10T20:25:24.655588+0000","last_fullsized":"2023-10-10T20:25:24.655588+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644584+0000","last_change":"2023-10-10T20:25:23.639303+0000","last_active":"2023-10-10T20:25:24.644584+0000","last_peered":"2023-10-10T20:25:24.644584+0000","last_clean":"2023-10-10T20:25:24.644584+0000","last_became_active":"2023-10-10T20:25:23.639030+0000","last_became_peered":"2023-10-10T20:25:23.639030+0000","last_unstale":"2023-10-10T20:25:24.644584+0000","last_undegraded":"2023-10-10T20:25:24.644584+0000","last_fullsized":"2023-10-10T20:25:24.644584+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:45.096131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":262,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651072+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:25:24.651072+0000","last_peered":"2023-10-10T20:25:24.651072+0000","last_clean":"2023-10-10T20:25:24.651072+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:25:24.651072+0000","last_undegraded":"2023-10-10T20:25:24.651072+0000","last_fullsized":"2023-10-10T20:25:24.651072+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1279,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194033,"statfs":{"total":200000000,"available":198690247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70299999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40999999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.439}]},{"osd":4,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.74199999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.318},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.376},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281133,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.53500000000000003},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":1.865,"15min":1.865},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64200000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72799999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.44}]},{"osd":3,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69099999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71499999999999997,"5min":15.862,"15min":15.862},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45300000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.62,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.313},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.46200000000000002},"min":{"1min":0.28000000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.39700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.50700000000000001,"15min":1.6100000000000001},"last":0.317},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.38300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":1.647},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38700000000000001},"min":{"1min":0.24199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.627,"15min":1.524},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.58799999999999997,"15min":1.3959999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38800000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":1.4470000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63,"15min":1.673},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.90900000000000003,"15min":1.633},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.506},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.696},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.9359999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.97499999999999998,"15min":1.7490000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.872,"15min":1.585},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.478},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.63},"min":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42299999999999999},"max":{"1min":0.75900000000000001,"5min":0.91000000000000003,"15min":1.722},"last":0.73499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1279,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194033,"statfs":{"total":200000000,"available":198690198,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.44500000000000001},"min":{"1min":0.216,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.379},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.435,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58099999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.313}]},{"osd":2,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.47399999999999998},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52400000000000002,"5min":1.647,"15min":2.4830000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":5.891,"15min":5.891},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.433},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.57999999999999996,"5min":1.462,"15min":1.462},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.433}]},{"osd":5,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.92600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77800000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.443}]},{"osd":7,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":5.758,"15min":5.758},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.79200000000000004,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":700,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194611,"statfs":{"total":200000000,"available":199282214,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.38600000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.54200000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53100000000000003,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.22},"max":{"1min":0.42299999999999999,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.48899999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.378}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.50600000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.49399999999999999,"5min":11.815,"15min":11.815},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.56699999999999995,"15min":0.46300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.47899999999999998,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.32300000000000001,"15min":0.28100000000000003},"max":{"1min":0.82899999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70199999999999996,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.72499999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":6.399,"15min":6.399},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.66700000000000004,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75900000000000001,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.67300000000000004,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":703,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194608,"statfs":{"total":200000000,"available":199279597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.30299999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.34200000000000003,"15min":0.315},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.44,"5min":0.57099999999999995,"15min":0.65600000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.55200000000000005,"15min":0.67900000000000005},"last":0.313},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53700000000000003,"5min":0.63200000000000001,"15min":0.751},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.25,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.58199999999999996,"5min":0.621,"15min":0.74099999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.63500000000000001,"15min":0.80700000000000005},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.376},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.436}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.35599999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42299999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.499}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85099999999999998,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":701,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281203,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.496},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.41299999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73999999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.49399999999999999},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.628}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"max":{"1min":0.92600000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194609,"statfs":{"total":200000000,"available":199280627,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.35399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.91200000000000003},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.79900000000000004},"last":0.496}]},{"osd":2,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.35399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.72599999999999998,"5min":0.83599999999999997,"15min":0.89000000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.78000000000000003,"15min":0.96299999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.752,"15min":0.78200000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.621},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.67500000000000004}]}]}],"pool_statfs":[]}} 2023-10-10T20:25:28.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:25:28.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.856+0000 7fb9c6841700 1 Processor -- start 2023-10-10T20:25:28.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.856+0000 7fb9c6841700 1 -- start start 2023-10-10T20:25:28.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9c6841700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9c0157870 0x7fb9c015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9c6841700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb9c0157c40 0x7fb9c0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9c6841700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c015cbb0 con 0x7fb9c015a620 2023-10-10T20:25:28.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9c6841700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c015bfa0 con 0x7fb9c0157870 2023-10-10T20:25:28.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9c6841700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c015c280 con 0x7fb9c0157c40 2023-10-10T20:25:28.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9bffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9c0157870 0x7fb9c015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9bf7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb9c015a620 0x7fb9c015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57794/0 (socket says 172.21.15.148:57794) 2023-10-10T20:25:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.857+0000 7fb9bf7fe700 1 -- 172.21.15.148:0/2977107192 learned_addr learned my addr 172.21.15.148:0/2977107192 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9bffff700 1 -- 172.21.15.148:0/2977107192 >> v2:172.21.15.161:3300/0 conn(0x7fb9c0157c40 msgr2=0x7fb9c0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2977107192 >> v2:172.21.15.161:3300/0 conn(0x7fb9c0157c40 0x7fb9c0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9bffff700 1 -- 172.21.15.148:0/2977107192 >> v1:172.21.15.148:6789/0 conn(0x7fb9c015a620 legacy=0x7fb9c015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9bffff700 1 -- 172.21.15.148:0/2977107192 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c015c560 con 0x7fb9c0157870 2023-10-10T20:25:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9beffd700 1 -- 172.21.15.148:0/2977107192 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940481736 0 0) 0x7fb9c015cbb0 con 0x7fb9c015a620 2023-10-10T20:25:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2977107192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0157870 0x7fb9c015b6a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb9c006b4e0 tx=0x7fb9b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49895d228c98152 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9beffd700 1 -- 172.21.15.148:0/2977107192 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000b6b0 con 0x7fb9c0157870 2023-10-10T20:25:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9beffd700 1 -- 172.21.15.148:0/2977107192 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b000b810 con 0x7fb9c0157870 2023-10-10T20:25:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9beffd700 1 -- 172.21.15.148:0/2977107192 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000bbb0 con 0x7fb9c0157870 2023-10-10T20:25:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 -- 172.21.15.148:0/2977107192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0157870 msgr2=0x7fb9c015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2977107192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0157870 0x7fb9c015b6a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb9c006b4e0 tx=0x7fb9b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 -- 172.21.15.148:0/2977107192 shutdown_connections 2023-10-10T20:25:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2977107192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0157870 0x7fb9c015b6a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2977107192 >> v2:172.21.15.161:3300/0 conn(0x7fb9c0157c40 0x7fb9c0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 -- 172.21.15.148:0/2977107192 >> 172.21.15.148:0/2977107192 conn(0x7fb9c0068ce0 msgr2=0x7fb9c00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:28.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 -- 172.21.15.148:0/2977107192 shutdown_connections 2023-10-10T20:25:28.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.858+0000 7fb9c6841700 1 -- 172.21.15.148:0/2977107192 wait complete. 2023-10-10T20:25:28.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 Processor -- start 2023-10-10T20:25:28.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 -- start start 2023-10-10T20:25:28.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 0x7fb9c014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 0x7fb9c0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c014b920 con 0x7fb9c015a620 2023-10-10T20:25:28.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0151430 con 0x7fb9c0157870 2023-10-10T20:25:28.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.860+0000 7fb9c6841700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01542c0 con 0x7fb9c0157c40 2023-10-10T20:25:28.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57774/0 (socket says 172.21.15.148:57774) 2023-10-10T20:25:28.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 -- 172.21.15.148:0/2094801730 learned_addr learned my addr 172.21.15.148:0/2094801730 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:28.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 0x7fb9c014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9bf7fe700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 0x7fb9c0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 -- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 msgr2=0x7fb9c0153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 0x7fb9c0153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 -- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 msgr2=0x7fb9c014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:28.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 0x7fb9c014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:28.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 -- 172.21.15.148:0/2094801730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009420 con 0x7fb9c015a620 2023-10-10T20:25:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 0x7fb9c014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9bf7fe700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 0x7fb9c0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c4dde700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800bb60 tx=0x7fb9b8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800c940 con 0x7fb9c015a620 2023-10-10T20:25:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800caa0 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c0153a70 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800ce60 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.861+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0153d00 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.862+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c01515e0 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.863+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9b8014620 con 0x7fb9c015a620 2023-10-10T20:25:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.863+0000 7fb9a7fff700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:28.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.863+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9b80845a0 con 0x7fb9c015a620 2023-10-10T20:25:28.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.863+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:28.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.863+0000 7fb9bffff700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 0x7fb9ac0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00093f0 tx=0x7fb9b0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:28.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:28.867+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9b804c5a0 con 0x7fb9c015a620 2023-10-10T20:25:29.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.043+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9c0150f60 con 0x7fb9ac066f30 2023-10-10T20:25:29.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.051+0000 7fb9a7fff700 1 -- 172.21.15.148:0/2094801730 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285493 (secure 0 0 0) 0x7fb9b0002f30 con 0x7fb9ac066f30 2023-10-10T20:25:29.055 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:29.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 msgr2=0x7fb9ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 0x7fb9ac0693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00093f0 tx=0x7fb9b0017000 comp rx=0 tx=0).stop 2023-10-10T20:25:29.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 msgr2=0x7fb9c01518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800bb60 tx=0x7fb9b8009810 comp rx=0 tx=0).stop 2023-10-10T20:25:29.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 shutdown_connections 2023-10-10T20:25:29.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c015a620 0x7fb9c01518c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9c0157870 0x7fb9c014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 --2- 172.21.15.148:0/2094801730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9c0157c40 0x7fb9c0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 >> 172.21.15.148:0/2094801730 conn(0x7fb9c0068ce0 msgr2=0x7fb9c00d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:29.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 shutdown_connections 2023-10-10T20:25:29.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.054+0000 7fb9c6841700 1 -- 172.21.15.148:0/2094801730 wait complete. 2023-10-10T20:25:29.060 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:25:29.067 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2023-10-10T20:25:27.401262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555728,"statfs":{"total":1600000000,"available":1593066298,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.771321"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651713+0000","last_change":"2023-10-10T20:25:24.651440+0000","last_active":"2023-10-10T20:25:24.651713+0000","last_peered":"2023-10-10T20:25:24.651713+0000","last_clean":"2023-10-10T20:25:24.651713+0000","last_became_active":"2023-10-10T20:25:24.650772+0000","last_became_peered":"2023-10-10T20:25:24.650772+0000","last_unstale":"2023-10-10T20:25:24.651713+0000","last_undegraded":"2023-10-10T20:25:24.651713+0000","last_fullsized":"2023-10-10T20:25:24.651713+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:30:29.602041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657264+0000","last_change":"2023-10-10T20:25:24.657220+0000","last_active":"2023-10-10T20:25:24.657264+0000","last_peered":"2023-10-10T20:25:24.657264+0000","last_clean":"2023-10-10T20:25:24.657264+0000","last_became_active":"2023-10-10T20:25:24.655581+0000","last_became_peered":"2023-10-10T20:25:24.655581+0000","last_unstale":"2023-10-10T20:25:24.657264+0000","last_undegraded":"2023-10-10T20:25:24.657264+0000","last_fullsized":"2023-10-10T20:25:24.657264+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:26.014253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654097+0000","last_change":"2023-10-10T20:25:24.653768+0000","last_active":"2023-10-10T20:25:24.654097+0000","last_peered":"2023-10-10T20:25:24.654097+0000","last_clean":"2023-10-10T20:25:24.654097+0000","last_became_active":"2023-10-10T20:25:24.652640+0000","last_became_peered":"2023-10-10T20:25:24.652640+0000","last_unstale":"2023-10-10T20:25:24.654097+0000","last_undegraded":"2023-10-10T20:25:24.654097+0000","last_fullsized":"2023-10-10T20:25:24.654097+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:46.680434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655938+0000","last_change":"2023-10-10T20:25:24.655589+0000","last_active":"2023-10-10T20:25:24.655938+0000","last_peered":"2023-10-10T20:25:24.655938+0000","last_clean":"2023-10-10T20:25:24.655938+0000","last_became_active":"2023-10-10T20:25:24.654857+0000","last_became_peered":"2023-10-10T20:25:24.654857+0000","last_unstale":"2023-10-10T20:25:24.655938+0000","last_undegraded":"2023-10-10T20:25:24.655938+0000","last_fullsized":"2023-10-10T20:25:24.655938+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:01:29.670966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657708+0000","last_change":"2023-10-10T20:25:24.657649+0000","last_active":"2023-10-10T20:25:24.657708+0000","last_peered":"2023-10-10T20:25:24.657708+0000","last_clean":"2023-10-10T20:25:24.657708+0000","last_became_active":"2023-10-10T20:25:24.657205+0000","last_became_peered":"2023-10-10T20:25:24.657205+0000","last_unstale":"2023-10-10T20:25:24.657708+0000","last_undegraded":"2023-10-10T20:25:24.657708+0000","last_fullsized":"2023-10-10T20:25:24.657708+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:57:53.834324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654132+0000","last_change":"2023-10-10T20:25:24.652946+0000","last_active":"2023-10-10T20:25:24.654132+0000","last_peered":"2023-10-10T20:25:24.654132+0000","last_clean":"2023-10-10T20:25:24.654132+0000","last_became_active":"2023-10-10T20:25:24.652406+0000","last_became_peered":"2023-10-10T20:25:24.652406+0000","last_unstale":"2023-10-10T20:25:24.654132+0000","last_undegraded":"2023-10-10T20:25:24.654132+0000","last_fullsized":"2023-10-10T20:25:24.654132+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:17:24.859536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.646593+0000","last_change":"2023-10-10T20:25:24.646346+0000","last_active":"2023-10-10T20:25:24.646593+0000","last_peered":"2023-10-10T20:25:24.646593+0000","last_clean":"2023-10-10T20:25:24.646593+0000","last_became_active":"2023-10-10T20:25:24.646103+0000","last_became_peered":"2023-10-10T20:25:24.646103+0000","last_unstale":"2023-10-10T20:25:24.646593+0000","last_undegraded":"2023-10-10T20:25:24.646593+0000","last_fullsized":"2023-10-10T20:25:24.646593+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:53.512364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654142+0000","last_change":"2023-10-10T20:25:24.654076+0000","last_active":"2023-10-10T20:25:24.654142+0000","last_peered":"2023-10-10T20:25:24.654142+0000","last_clean":"2023-10-10T20:25:24.654142+0000","last_became_active":"2023-10-10T20:25:24.653533+0000","last_became_peered":"2023-10-10T20:25:24.653533+0000","last_unstale":"2023-10-10T20:25:24.654142+0000","last_undegraded":"2023-10-10T20:25:24.654142+0000","last_fullsized":"2023-10-10T20:25:24.654142+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:59.171859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655956+0000","last_change":"2023-10-10T20:25:24.655714+0000","last_active":"2023-10-10T20:25:24.655956+0000","last_peered":"2023-10-10T20:25:24.655956+0000","last_clean":"2023-10-10T20:25:24.655956+0000","last_became_active":"2023-10-10T20:25:24.655355+0000","last_became_peered":"2023-10-10T20:25:24.655355+0000","last_unstale":"2023-10-10T20:25:24.655956+0000","last_undegraded":"2023-10-10T20:25:24.655956+0000","last_fullsized":"2023-10-10T20:25:24.655956+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:55:47.564111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650356+0000","last_change":"2023-10-10T20:25:24.650192+0000","last_active":"2023-10-10T20:25:24.650356+0000","last_peered":"2023-10-10T20:25:24.650356+0000","last_clean":"2023-10-10T20:25:24.650356+0000","last_became_active":"2023-10-10T20:25:24.649973+0000","last_became_peered":"2023-10-10T20:25:24.649973+0000","last_unstale":"2023-10-10T20:25:24.650356+0000","last_undegraded":"2023-10-10T20:25:24.650356+0000","last_fullsized":"2023-10-10T20:25:24.650356+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:58:35.603986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654205+0000","last_change":"2023-10-10T20:25:24.654010+0000","last_active":"2023-10-10T20:25:24.654205+0000","last_peered":"2023-10-10T20:25:24.654205+0000","last_clean":"2023-10-10T20:25:24.654205+0000","last_became_active":"2023-10-10T20:25:24.653757+0000","last_became_peered":"2023-10-10T20:25:24.653757+0000","last_unstale":"2023-10-10T20:25:24.654205+0000","last_undegraded":"2023-10-10T20:25:24.654205+0000","last_fullsized":"2023-10-10T20:25:24.654205+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:46:32.738474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650374+0000","last_change":"2023-10-10T20:25:24.650221+0000","last_active":"2023-10-10T20:25:24.650374+0000","last_peered":"2023-10-10T20:25:24.650374+0000","last_clean":"2023-10-10T20:25:24.650374+0000","last_became_active":"2023-10-10T20:25:24.649595+0000","last_became_peered":"2023-10-10T20:25:24.649595+0000","last_unstale":"2023-10-10T20:25:24.650374+0000","last_undegraded":"2023-10-10T20:25:24.650374+0000","last_fullsized":"2023-10-10T20:25:24.650374+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:43:10.834318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654128+0000","last_change":"2023-10-10T20:25:24.653941+0000","last_active":"2023-10-10T20:25:24.654128+0000","last_peered":"2023-10-10T20:25:24.654128+0000","last_clean":"2023-10-10T20:25:24.654128+0000","last_became_active":"2023-10-10T20:25:24.652935+0000","last_became_peered":"2023-10-10T20:25:24.652935+0000","last_unstale":"2023-10-10T20:25:24.654128+0000","last_undegraded":"2023-10-10T20:25:24.654128+0000","last_fullsized":"2023-10-10T20:25:24.654128+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655617+0000","last_change":"2023-10-10T20:25:24.655422+0000","last_active":"2023-10-10T20:25:24.655617+0000","last_peered":"2023-10-10T20:25:24.655617+0000","last_clean":"2023-10-10T20:25:24.655617+0000","last_became_active":"2023-10-10T20:25:24.654904+0000","last_became_peered":"2023-10-10T20:25:24.654904+0000","last_unstale":"2023-10-10T20:25:24.655617+0000","last_undegraded":"2023-10-10T20:25:24.655617+0000","last_fullsized":"2023-10-10T20:25:24.655617+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.563127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656959+0000","last_change":"2023-10-10T20:25:24.656910+0000","last_active":"2023-10-10T20:25:24.656959+0000","last_peered":"2023-10-10T20:25:24.656959+0000","last_clean":"2023-10-10T20:25:24.656959+0000","last_became_active":"2023-10-10T20:25:24.656728+0000","last_became_peered":"2023-10-10T20:25:24.656728+0000","last_unstale":"2023-10-10T20:25:24.656959+0000","last_undegraded":"2023-10-10T20:25:24.656959+0000","last_fullsized":"2023-10-10T20:25:24.656959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:14:31.614400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651745+0000","last_change":"2023-10-10T20:25:24.651657+0000","last_active":"2023-10-10T20:25:24.651745+0000","last_peered":"2023-10-10T20:25:24.651745+0000","last_clean":"2023-10-10T20:25:24.651745+0000","last_became_active":"2023-10-10T20:25:24.651077+0000","last_became_peered":"2023-10-10T20:25:24.651077+0000","last_unstale":"2023-10-10T20:25:24.651745+0000","last_undegraded":"2023-10-10T20:25:24.651745+0000","last_fullsized":"2023-10-10T20:25:24.651745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:48:22.954740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657745+0000","last_change":"2023-10-10T20:25:24.657348+0000","last_active":"2023-10-10T20:25:24.657745+0000","last_peered":"2023-10-10T20:25:24.657745+0000","last_clean":"2023-10-10T20:25:24.657745+0000","last_became_active":"2023-10-10T20:25:24.656104+0000","last_became_peered":"2023-10-10T20:25:24.656104+0000","last_unstale":"2023-10-10T20:25:24.657745+0000","last_undegraded":"2023-10-10T20:25:24.657745+0000","last_fullsized":"2023-10-10T20:25:24.657745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:04:24.784388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651379+0000","last_change":"2023-10-10T20:25:24.650698+0000","last_active":"2023-10-10T20:25:24.651379+0000","last_peered":"2023-10-10T20:25:24.651379+0000","last_clean":"2023-10-10T20:25:24.651379+0000","last_became_active":"2023-10-10T20:25:24.650452+0000","last_became_peered":"2023-10-10T20:25:24.650452+0000","last_unstale":"2023-10-10T20:25:24.651379+0000","last_undegraded":"2023-10-10T20:25:24.651379+0000","last_fullsized":"2023-10-10T20:25:24.651379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:41.591721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654732+0000","last_change":"2023-10-10T20:25:24.654673+0000","last_active":"2023-10-10T20:25:24.654732+0000","last_peered":"2023-10-10T20:25:24.654732+0000","last_clean":"2023-10-10T20:25:24.654732+0000","last_became_active":"2023-10-10T20:25:24.652973+0000","last_became_peered":"2023-10-10T20:25:24.652973+0000","last_unstale":"2023-10-10T20:25:24.654732+0000","last_undegraded":"2023-10-10T20:25:24.654732+0000","last_fullsized":"2023-10-10T20:25:24.654732+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:34.264510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654189+0000","last_change":"2023-10-10T20:25:24.653998+0000","last_active":"2023-10-10T20:25:24.654189+0000","last_peered":"2023-10-10T20:25:24.654189+0000","last_clean":"2023-10-10T20:25:24.654189+0000","last_became_active":"2023-10-10T20:25:24.653574+0000","last_became_peered":"2023-10-10T20:25:24.653574+0000","last_unstale":"2023-10-10T20:25:24.654189+0000","last_undegraded":"2023-10-10T20:25:24.654189+0000","last_fullsized":"2023-10-10T20:25:24.654189+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:21.626071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657835+0000","last_change":"2023-10-10T20:25:24.657785+0000","last_active":"2023-10-10T20:25:24.657835+0000","last_peered":"2023-10-10T20:25:24.657835+0000","last_clean":"2023-10-10T20:25:24.657835+0000","last_became_active":"2023-10-10T20:25:24.657545+0000","last_became_peered":"2023-10-10T20:25:24.657545+0000","last_unstale":"2023-10-10T20:25:24.657835+0000","last_undegraded":"2023-10-10T20:25:24.657835+0000","last_fullsized":"2023-10-10T20:25:24.657835+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:49:06.922751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655191+0000","last_change":"2023-10-10T20:25:24.655129+0000","last_active":"2023-10-10T20:25:24.655191+0000","last_peered":"2023-10-10T20:25:24.655191+0000","last_clean":"2023-10-10T20:25:24.655191+0000","last_became_active":"2023-10-10T20:25:24.654743+0000","last_became_peered":"2023-10-10T20:25:24.654743+0000","last_unstale":"2023-10-10T20:25:24.655191+0000","last_undegraded":"2023-10-10T20:25:24.655191+0000","last_fullsized":"2023-10-10T20:25:24.655191+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:36.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654858+0000","last_change":"2023-10-10T20:25:24.654724+0000","last_active":"2023-10-10T20:25:24.654858+0000","last_peered":"2023-10-10T20:25:24.654858+0000","last_clean":"2023-10-10T20:25:24.654858+0000","last_became_active":"2023-10-10T20:25:24.654409+0000","last_became_peered":"2023-10-10T20:25:24.654409+0000","last_unstale":"2023-10-10T20:25:24.654858+0000","last_undegraded":"2023-10-10T20:25:24.654858+0000","last_fullsized":"2023-10-10T20:25:24.654858+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:51:49.367967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655605+0000","last_change":"2023-10-10T20:25:24.655518+0000","last_active":"2023-10-10T20:25:24.655605+0000","last_peered":"2023-10-10T20:25:24.655605+0000","last_clean":"2023-10-10T20:25:24.655605+0000","last_became_active":"2023-10-10T20:25:24.655283+0000","last_became_peered":"2023-10-10T20:25:24.655283+0000","last_unstale":"2023-10-10T20:25:24.655605+0000","last_undegraded":"2023-10-10T20:25:24.655605+0000","last_fullsized":"2023-10-10T20:25:24.655605+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:16.563127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654171+0000","last_change":"2023-10-10T20:25:24.653836+0000","last_active":"2023-10-10T20:25:24.654171+0000","last_peered":"2023-10-10T20:25:24.654171+0000","last_clean":"2023-10-10T20:25:24.654171+0000","last_became_active":"2023-10-10T20:25:24.653219+0000","last_became_peered":"2023-10-10T20:25:24.653219+0000","last_unstale":"2023-10-10T20:25:24.654171+0000","last_undegraded":"2023-10-10T20:25:24.654171+0000","last_fullsized":"2023-10-10T20:25:24.654171+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:13:20.958148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648825+0000","last_change":"2023-10-10T20:25:24.647543+0000","last_active":"2023-10-10T20:25:24.648825+0000","last_peered":"2023-10-10T20:25:24.648825+0000","last_clean":"2023-10-10T20:25:24.648825+0000","last_became_active":"2023-10-10T20:25:24.647049+0000","last_became_peered":"2023-10-10T20:25:24.647049+0000","last_unstale":"2023-10-10T20:25:24.648825+0000","last_undegraded":"2023-10-10T20:25:24.648825+0000","last_fullsized":"2023-10-10T20:25:24.648825+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:30:12.692223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652193+0000","last_change":"2023-10-10T20:25:24.652097+0000","last_active":"2023-10-10T20:25:24.652193+0000","last_peered":"2023-10-10T20:25:24.652193+0000","last_clean":"2023-10-10T20:25:24.652193+0000","last_became_active":"2023-10-10T20:25:24.651778+0000","last_became_peered":"2023-10-10T20:25:24.651778+0000","last_unstale":"2023-10-10T20:25:24.652193+0000","last_undegraded":"2023-10-10T20:25:24.652193+0000","last_fullsized":"2023-10-10T20:25:24.652193+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:50.971429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657479+0000","last_change":"2023-10-10T20:25:24.657418+0000","last_active":"2023-10-10T20:25:24.657479+0000","last_peered":"2023-10-10T20:25:24.657479+0000","last_clean":"2023-10-10T20:25:24.657479+0000","last_became_active":"2023-10-10T20:25:24.657220+0000","last_became_peered":"2023-10-10T20:25:24.657220+0000","last_unstale":"2023-10-10T20:25:24.657479+0000","last_undegraded":"2023-10-10T20:25:24.657479+0000","last_fullsized":"2023-10-10T20:25:24.657479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:17:04.606687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655367+0000","last_change":"2023-10-10T20:25:24.655305+0000","last_active":"2023-10-10T20:25:24.655367+0000","last_peered":"2023-10-10T20:25:24.655367+0000","last_clean":"2023-10-10T20:25:24.655367+0000","last_became_active":"2023-10-10T20:25:24.655026+0000","last_became_peered":"2023-10-10T20:25:24.655026+0000","last_unstale":"2023-10-10T20:25:24.655367+0000","last_undegraded":"2023-10-10T20:25:24.655367+0000","last_fullsized":"2023-10-10T20:25:24.655367+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:26.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657716+0000","last_change":"2023-10-10T20:25:24.655372+0000","last_active":"2023-10-10T20:25:24.657716+0000","last_peered":"2023-10-10T20:25:24.657716+0000","last_clean":"2023-10-10T20:25:24.657716+0000","last_became_active":"2023-10-10T20:25:24.655112+0000","last_became_peered":"2023-10-10T20:25:24.655112+0000","last_unstale":"2023-10-10T20:25:24.657716+0000","last_undegraded":"2023-10-10T20:25:24.657716+0000","last_fullsized":"2023-10-10T20:25:24.657716+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:36.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648844+0000","last_change":"2023-10-10T20:25:24.647676+0000","last_active":"2023-10-10T20:25:24.648844+0000","last_peered":"2023-10-10T20:25:24.648844+0000","last_clean":"2023-10-10T20:25:24.648844+0000","last_became_active":"2023-10-10T20:25:24.647327+0000","last_became_peered":"2023-10-10T20:25:24.647327+0000","last_unstale":"2023-10-10T20:25:24.648844+0000","last_undegraded":"2023-10-10T20:25:24.648844+0000","last_fullsized":"2023-10-10T20:25:24.648844+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:58:13.613317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654166+0000","last_change":"2023-10-10T20:25:24.653160+0000","last_active":"2023-10-10T20:25:24.654166+0000","last_peered":"2023-10-10T20:25:24.654166+0000","last_clean":"2023-10-10T20:25:24.654166+0000","last_became_active":"2023-10-10T20:25:24.652936+0000","last_became_peered":"2023-10-10T20:25:24.652936+0000","last_unstale":"2023-10-10T20:25:24.654166+0000","last_undegraded":"2023-10-10T20:25:24.654166+0000","last_fullsized":"2023-10-10T20:25:24.654166+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:46:56.089847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651502+0000","last_change":"2023-10-10T20:25:24.651254+0000","last_active":"2023-10-10T20:25:24.651502+0000","last_peered":"2023-10-10T20:25:24.651502+0000","last_clean":"2023-10-10T20:25:24.651502+0000","last_became_active":"2023-10-10T20:25:24.650710+0000","last_became_peered":"2023-10-10T20:25:24.650710+0000","last_unstale":"2023-10-10T20:25:24.651502+0000","last_undegraded":"2023-10-10T20:25:24.651502+0000","last_fullsized":"2023-10-10T20:25:24.651502+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:04:46.775057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"130'4","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664508+0000","last_change":"2023-10-10T20:25:23.640916+0000","last_active":"2023-10-10T20:25:24.664508+0000","last_peered":"2023-10-10T20:25:24.664508+0000","last_clean":"2023-10-10T20:25:24.664508+0000","last_became_active":"2023-10-10T20:25:23.640695+0000","last_became_peered":"2023-10-10T20:25:23.640695+0000","last_unstale":"2023-10-10T20:25:24.664508+0000","last_undegraded":"2023-10-10T20:25:24.664508+0000","last_fullsized":"2023-10-10T20:25:24.664508+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:36:28.224223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652177+0000","last_change":"2023-10-10T20:25:24.652057+0000","last_active":"2023-10-10T20:25:24.652177+0000","last_peered":"2023-10-10T20:25:24.652177+0000","last_clean":"2023-10-10T20:25:24.652177+0000","last_became_active":"2023-10-10T20:25:24.651827+0000","last_became_peered":"2023-10-10T20:25:24.651827+0000","last_unstale":"2023-10-10T20:25:24.652177+0000","last_undegraded":"2023-10-10T20:25:24.652177+0000","last_fullsized":"2023-10-10T20:25:24.652177+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:46.680434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664235+0000","last_change":"2023-10-10T20:25:23.642233+0000","last_active":"2023-10-10T20:25:24.664235+0000","last_peered":"2023-10-10T20:25:24.664235+0000","last_clean":"2023-10-10T20:25:24.664235+0000","last_became_active":"2023-10-10T20:25:23.641603+0000","last_became_peered":"2023-10-10T20:25:23.641603+0000","last_unstale":"2023-10-10T20:25:24.664235+0000","last_undegraded":"2023-10-10T20:25:24.664235+0000","last_fullsized":"2023-10-10T20:25:24.664235+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:20.621784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654150+0000","last_change":"2023-10-10T20:25:24.653083+0000","last_active":"2023-10-10T20:25:24.654150+0000","last_peered":"2023-10-10T20:25:24.654150+0000","last_clean":"2023-10-10T20:25:24.654150+0000","last_became_active":"2023-10-10T20:25:24.652717+0000","last_became_peered":"2023-10-10T20:25:24.652717+0000","last_unstale":"2023-10-10T20:25:24.654150+0000","last_undegraded":"2023-10-10T20:25:24.654150+0000","last_fullsized":"2023-10-10T20:25:24.654150+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:51:41.204103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667191+0000","last_change":"2023-10-10T20:25:23.641354+0000","last_active":"2023-10-10T20:25:24.667191+0000","last_peered":"2023-10-10T20:25:24.667191+0000","last_clean":"2023-10-10T20:25:24.667191+0000","last_became_active":"2023-10-10T20:25:23.641151+0000","last_became_peered":"2023-10-10T20:25:23.641151+0000","last_unstale":"2023-10-10T20:25:24.667191+0000","last_undegraded":"2023-10-10T20:25:24.667191+0000","last_fullsized":"2023-10-10T20:25:24.667191+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:41:54.891283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657080+0000","last_change":"2023-10-10T20:25:24.657017+0000","last_active":"2023-10-10T20:25:24.657080+0000","last_peered":"2023-10-10T20:25:24.657080+0000","last_clean":"2023-10-10T20:25:24.657080+0000","last_became_active":"2023-10-10T20:25:24.656787+0000","last_became_peered":"2023-10-10T20:25:24.656787+0000","last_unstale":"2023-10-10T20:25:24.657080+0000","last_undegraded":"2023-10-10T20:25:24.657080+0000","last_fullsized":"2023-10-10T20:25:24.657080+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666897+0000","last_change":"2023-10-10T20:25:23.642924+0000","last_active":"2023-10-10T20:25:24.666897+0000","last_peered":"2023-10-10T20:25:24.666897+0000","last_clean":"2023-10-10T20:25:24.666897+0000","last_became_active":"2023-10-10T20:25:23.642462+0000","last_became_peered":"2023-10-10T20:25:23.642462+0000","last_unstale":"2023-10-10T20:25:24.666897+0000","last_undegraded":"2023-10-10T20:25:24.666897+0000","last_fullsized":"2023-10-10T20:25:24.666897+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:18.488919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656486+0000","last_change":"2023-10-10T20:25:24.656278+0000","last_active":"2023-10-10T20:25:24.656486+0000","last_peered":"2023-10-10T20:25:24.656486+0000","last_clean":"2023-10-10T20:25:24.656486+0000","last_became_active":"2023-10-10T20:25:24.655773+0000","last_became_peered":"2023-10-10T20:25:24.655773+0000","last_unstale":"2023-10-10T20:25:24.656486+0000","last_undegraded":"2023-10-10T20:25:24.656486+0000","last_fullsized":"2023-10-10T20:25:24.656486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:21:54.876803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.664039+0000","last_change":"2023-10-10T20:25:23.642833+0000","last_active":"2023-10-10T20:25:24.664039+0000","last_peered":"2023-10-10T20:25:24.664039+0000","last_clean":"2023-10-10T20:25:24.664039+0000","last_became_active":"2023-10-10T20:25:23.642581+0000","last_became_peered":"2023-10-10T20:25:23.642581+0000","last_unstale":"2023-10-10T20:25:24.664039+0000","last_undegraded":"2023-10-10T20:25:24.664039+0000","last_fullsized":"2023-10-10T20:25:24.664039+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:56:51.227809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657149+0000","last_change":"2023-10-10T20:25:24.657084+0000","last_active":"2023-10-10T20:25:24.657149+0000","last_peered":"2023-10-10T20:25:24.657149+0000","last_clean":"2023-10-10T20:25:24.657149+0000","last_became_active":"2023-10-10T20:25:24.655273+0000","last_became_peered":"2023-10-10T20:25:24.655273+0000","last_unstale":"2023-10-10T20:25:24.657149+0000","last_undegraded":"2023-10-10T20:25:24.657149+0000","last_fullsized":"2023-10-10T20:25:24.657149+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:40.723203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666067+0000","last_change":"2023-10-10T20:25:23.647030+0000","last_active":"2023-10-10T20:25:24.666067+0000","last_peered":"2023-10-10T20:25:24.666067+0000","last_clean":"2023-10-10T20:25:24.666067+0000","last_became_active":"2023-10-10T20:25:23.646801+0000","last_became_peered":"2023-10-10T20:25:23.646801+0000","last_unstale":"2023-10-10T20:25:24.666067+0000","last_undegraded":"2023-10-10T20:25:24.666067+0000","last_fullsized":"2023-10-10T20:25:24.666067+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:45.130605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652175+0000","last_change":"2023-10-10T20:25:24.651771+0000","last_active":"2023-10-10T20:25:24.652175+0000","last_peered":"2023-10-10T20:25:24.652175+0000","last_clean":"2023-10-10T20:25:24.652175+0000","last_became_active":"2023-10-10T20:25:24.651424+0000","last_became_peered":"2023-10-10T20:25:24.651424+0000","last_unstale":"2023-10-10T20:25:24.652175+0000","last_undegraded":"2023-10-10T20:25:24.652175+0000","last_fullsized":"2023-10-10T20:25:24.652175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:58:40.048292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648930+0000","last_change":"2023-10-10T20:25:23.647122+0000","last_active":"2023-10-10T20:25:24.648930+0000","last_peered":"2023-10-10T20:25:24.648930+0000","last_clean":"2023-10-10T20:25:24.648930+0000","last_became_active":"2023-10-10T20:25:23.646944+0000","last_became_peered":"2023-10-10T20:25:23.646944+0000","last_unstale":"2023-10-10T20:25:24.648930+0000","last_undegraded":"2023-10-10T20:25:24.648930+0000","last_fullsized":"2023-10-10T20:25:24.648930+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:52.508077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.646493+0000","last_change":"2023-10-10T20:25:24.646348+0000","last_active":"2023-10-10T20:25:24.646493+0000","last_peered":"2023-10-10T20:25:24.646493+0000","last_clean":"2023-10-10T20:25:24.646493+0000","last_became_active":"2023-10-10T20:25:24.646112+0000","last_became_peered":"2023-10-10T20:25:24.646112+0000","last_unstale":"2023-10-10T20:25:24.646493+0000","last_undegraded":"2023-10-10T20:25:24.646493+0000","last_fullsized":"2023-10-10T20:25:24.646493+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:42.256826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645173+0000","last_change":"2023-10-10T20:25:23.646537+0000","last_active":"2023-10-10T20:25:24.645173+0000","last_peered":"2023-10-10T20:25:24.645173+0000","last_clean":"2023-10-10T20:25:24.645173+0000","last_became_active":"2023-10-10T20:25:23.645588+0000","last_became_peered":"2023-10-10T20:25:23.645588+0000","last_unstale":"2023-10-10T20:25:24.645173+0000","last_undegraded":"2023-10-10T20:25:24.645173+0000","last_fullsized":"2023-10-10T20:25:24.645173+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:55:15.558840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.653697+0000","last_change":"2023-10-10T20:25:24.653498+0000","last_active":"2023-10-10T20:25:24.653697+0000","last_peered":"2023-10-10T20:25:24.653697+0000","last_clean":"2023-10-10T20:25:24.653697+0000","last_became_active":"2023-10-10T20:25:24.653221+0000","last_became_peered":"2023-10-10T20:25:24.653221+0000","last_unstale":"2023-10-10T20:25:24.653697+0000","last_undegraded":"2023-10-10T20:25:24.653697+0000","last_fullsized":"2023-10-10T20:25:24.653697+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:02.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.648528+0000","last_change":"2023-10-10T20:25:23.643011+0000","last_active":"2023-10-10T20:25:24.648528+0000","last_peered":"2023-10-10T20:25:24.648528+0000","last_clean":"2023-10-10T20:25:24.648528+0000","last_became_active":"2023-10-10T20:25:23.642736+0000","last_became_peered":"2023-10-10T20:25:23.642736+0000","last_unstale":"2023-10-10T20:25:24.648528+0000","last_undegraded":"2023-10-10T20:25:24.648528+0000","last_fullsized":"2023-10-10T20:25:24.648528+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:39:57.679851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647328+0000","last_change":"2023-10-10T20:25:24.647280+0000","last_active":"2023-10-10T20:25:24.647328+0000","last_peered":"2023-10-10T20:25:24.647328+0000","last_clean":"2023-10-10T20:25:24.647328+0000","last_became_active":"2023-10-10T20:25:24.647065+0000","last_became_peered":"2023-10-10T20:25:24.647065+0000","last_unstale":"2023-10-10T20:25:24.647328+0000","last_undegraded":"2023-10-10T20:25:24.647328+0000","last_fullsized":"2023-10-10T20:25:24.647328+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.649479+0000","last_change":"2023-10-10T20:25:23.647452+0000","last_active":"2023-10-10T20:25:24.649479+0000","last_peered":"2023-10-10T20:25:24.649479+0000","last_clean":"2023-10-10T20:25:24.649479+0000","last_became_active":"2023-10-10T20:25:23.646987+0000","last_became_peered":"2023-10-10T20:25:23.646987+0000","last_unstale":"2023-10-10T20:25:24.649479+0000","last_undegraded":"2023-10-10T20:25:24.649479+0000","last_fullsized":"2023-10-10T20:25:24.649479+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:13:55.035835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658276+0000","last_change":"2023-10-10T20:25:24.658109+0000","last_active":"2023-10-10T20:25:24.658276+0000","last_peered":"2023-10-10T20:25:24.658276+0000","last_clean":"2023-10-10T20:25:24.658276+0000","last_became_active":"2023-10-10T20:25:24.657882+0000","last_became_peered":"2023-10-10T20:25:24.657882+0000","last_unstale":"2023-10-10T20:25:24.658276+0000","last_undegraded":"2023-10-10T20:25:24.658276+0000","last_fullsized":"2023-10-10T20:25:24.658276+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:40.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"130'3","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665317+0000","last_change":"2023-10-10T20:25:23.646024+0000","last_active":"2023-10-10T20:25:24.665317+0000","last_peered":"2023-10-10T20:25:24.665317+0000","last_clean":"2023-10-10T20:25:24.665317+0000","last_became_active":"2023-10-10T20:25:23.645822+0000","last_became_peered":"2023-10-10T20:25:23.645822+0000","last_unstale":"2023-10-10T20:25:24.665317+0000","last_undegraded":"2023-10-10T20:25:24.665317+0000","last_fullsized":"2023-10-10T20:25:24.665317+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:33.260222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652608+0000","last_change":"2023-10-10T20:25:24.652544+0000","last_active":"2023-10-10T20:25:24.652608+0000","last_peered":"2023-10-10T20:25:24.652608+0000","last_clean":"2023-10-10T20:25:24.652608+0000","last_became_active":"2023-10-10T20:25:24.652277+0000","last_became_peered":"2023-10-10T20:25:24.652277+0000","last_unstale":"2023-10-10T20:25:24.652608+0000","last_undegraded":"2023-10-10T20:25:24.652608+0000","last_fullsized":"2023-10-10T20:25:24.652608+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:04.444619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"130'6","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.668691+0000","last_change":"2023-10-10T20:25:23.647101+0000","last_active":"2023-10-10T20:25:24.668691+0000","last_peered":"2023-10-10T20:25:24.668691+0000","last_clean":"2023-10-10T20:25:24.668691+0000","last_became_active":"2023-10-10T20:25:23.646884+0000","last_became_peered":"2023-10-10T20:25:23.646884+0000","last_unstale":"2023-10-10T20:25:24.668691+0000","last_undegraded":"2023-10-10T20:25:24.668691+0000","last_fullsized":"2023-10-10T20:25:24.668691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:39.540397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655425+0000","last_change":"2023-10-10T20:25:24.655343+0000","last_active":"2023-10-10T20:25:24.655425+0000","last_peered":"2023-10-10T20:25:24.655425+0000","last_clean":"2023-10-10T20:25:24.655425+0000","last_became_active":"2023-10-10T20:25:24.654972+0000","last_became_peered":"2023-10-10T20:25:24.654972+0000","last_unstale":"2023-10-10T20:25:24.655425+0000","last_undegraded":"2023-10-10T20:25:24.655425+0000","last_fullsized":"2023-10-10T20:25:24.655425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:59.171859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666959+0000","last_change":"2023-10-10T20:25:23.642543+0000","last_active":"2023-10-10T20:25:24.666959+0000","last_peered":"2023-10-10T20:25:24.666959+0000","last_clean":"2023-10-10T20:25:24.666959+0000","last_became_active":"2023-10-10T20:25:23.642160+0000","last_became_peered":"2023-10-10T20:25:23.642160+0000","last_unstale":"2023-10-10T20:25:24.666959+0000","last_undegraded":"2023-10-10T20:25:24.666959+0000","last_fullsized":"2023-10-10T20:25:24.666959+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:52.552940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.653278+0000","last_change":"2023-10-10T20:25:24.653045+0000","last_active":"2023-10-10T20:25:24.653278+0000","last_peered":"2023-10-10T20:25:24.653278+0000","last_clean":"2023-10-10T20:25:24.653278+0000","last_became_active":"2023-10-10T20:25:24.652833+0000","last_became_peered":"2023-10-10T20:25:24.652833+0000","last_unstale":"2023-10-10T20:25:24.653278+0000","last_undegraded":"2023-10-10T20:25:24.653278+0000","last_fullsized":"2023-10-10T20:25:24.653278+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644406+0000","last_change":"2023-10-10T20:25:23.648449+0000","last_active":"2023-10-10T20:25:24.644406+0000","last_peered":"2023-10-10T20:25:24.644406+0000","last_clean":"2023-10-10T20:25:24.644406+0000","last_became_active":"2023-10-10T20:25:23.648279+0000","last_became_peered":"2023-10-10T20:25:23.648279+0000","last_unstale":"2023-10-10T20:25:24.644406+0000","last_undegraded":"2023-10-10T20:25:24.644406+0000","last_fullsized":"2023-10-10T20:25:24.644406+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:03.131761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658238+0000","last_change":"2023-10-10T20:25:24.657996+0000","last_active":"2023-10-10T20:25:24.658238+0000","last_peered":"2023-10-10T20:25:24.658238+0000","last_clean":"2023-10-10T20:25:24.658238+0000","last_became_active":"2023-10-10T20:25:24.657783+0000","last_became_peered":"2023-10-10T20:25:24.657783+0000","last_unstale":"2023-10-10T20:25:24.658238+0000","last_undegraded":"2023-10-10T20:25:24.658238+0000","last_fullsized":"2023-10-10T20:25:24.658238+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:22.406668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"130'8","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667763+0000","last_change":"2023-10-10T20:25:23.646310+0000","last_active":"2023-10-10T20:25:24.667763+0000","last_peered":"2023-10-10T20:25:24.667763+0000","last_clean":"2023-10-10T20:25:24.667763+0000","last_became_active":"2023-10-10T20:25:23.646083+0000","last_became_peered":"2023-10-10T20:25:23.646083+0000","last_unstale":"2023-10-10T20:25:24.667763+0000","last_undegraded":"2023-10-10T20:25:24.667763+0000","last_fullsized":"2023-10-10T20:25:24.667763+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:09.433016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.654157+0000","last_change":"2023-10-10T20:25:24.653709+0000","last_active":"2023-10-10T20:25:24.654157+0000","last_peered":"2023-10-10T20:25:24.654157+0000","last_clean":"2023-10-10T20:25:24.654157+0000","last_became_active":"2023-10-10T20:25:24.653487+0000","last_became_peered":"2023-10-10T20:25:24.653487+0000","last_unstale":"2023-10-10T20:25:24.654157+0000","last_undegraded":"2023-10-10T20:25:24.654157+0000","last_fullsized":"2023-10-10T20:25:24.654157+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:20.396556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665817+0000","last_change":"2023-10-10T20:25:23.638278+0000","last_active":"2023-10-10T20:25:24.665817+0000","last_peered":"2023-10-10T20:25:24.665817+0000","last_clean":"2023-10-10T20:25:24.665817+0000","last_became_active":"2023-10-10T20:25:23.638042+0000","last_became_peered":"2023-10-10T20:25:23.638042+0000","last_unstale":"2023-10-10T20:25:24.665817+0000","last_undegraded":"2023-10-10T20:25:24.665817+0000","last_fullsized":"2023-10-10T20:25:24.665817+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:40.587433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651719+0000","last_change":"2023-10-10T20:25:24.651414+0000","last_active":"2023-10-10T20:25:24.651719+0000","last_peered":"2023-10-10T20:25:24.651719+0000","last_clean":"2023-10-10T20:25:24.651719+0000","last_became_active":"2023-10-10T20:25:24.651032+0000","last_became_peered":"2023-10-10T20:25:24.651032+0000","last_unstale":"2023-10-10T20:25:24.651719+0000","last_undegraded":"2023-10-10T20:25:24.651719+0000","last_fullsized":"2023-10-10T20:25:24.651719+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:07:58.969237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.666669+0000","last_change":"2023-10-10T20:25:23.644481+0000","last_active":"2023-10-10T20:25:24.666669+0000","last_peered":"2023-10-10T20:25:24.666669+0000","last_clean":"2023-10-10T20:25:24.666669+0000","last_became_active":"2023-10-10T20:25:23.644277+0000","last_became_peered":"2023-10-10T20:25:23.644277+0000","last_unstale":"2023-10-10T20:25:24.666669+0000","last_undegraded":"2023-10-10T20:25:24.666669+0000","last_fullsized":"2023-10-10T20:25:24.666669+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:52.552940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652250+0000","last_change":"2023-10-10T20:25:24.651532+0000","last_active":"2023-10-10T20:25:24.652250+0000","last_peered":"2023-10-10T20:25:24.652250+0000","last_clean":"2023-10-10T20:25:24.652250+0000","last_became_active":"2023-10-10T20:25:24.651293+0000","last_became_peered":"2023-10-10T20:25:24.651293+0000","last_unstale":"2023-10-10T20:25:24.652250+0000","last_undegraded":"2023-10-10T20:25:24.652250+0000","last_fullsized":"2023-10-10T20:25:24.652250+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:04.136048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"130'1","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.665833+0000","last_change":"2023-10-10T20:25:23.642313+0000","last_active":"2023-10-10T20:25:24.665833+0000","last_peered":"2023-10-10T20:25:24.665833+0000","last_clean":"2023-10-10T20:25:24.665833+0000","last_became_active":"2023-10-10T20:25:23.642058+0000","last_became_peered":"2023-10-10T20:25:23.642058+0000","last_unstale":"2023-10-10T20:25:24.665833+0000","last_undegraded":"2023-10-10T20:25:24.665833+0000","last_fullsized":"2023-10-10T20:25:24.665833+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:25.873244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656997+0000","last_change":"2023-10-10T20:25:24.656929+0000","last_active":"2023-10-10T20:25:24.656997+0000","last_peered":"2023-10-10T20:25:24.656997+0000","last_clean":"2023-10-10T20:25:24.656997+0000","last_became_active":"2023-10-10T20:25:24.656657+0000","last_became_peered":"2023-10-10T20:25:24.656657+0000","last_unstale":"2023-10-10T20:25:24.656997+0000","last_undegraded":"2023-10-10T20:25:24.656997+0000","last_fullsized":"2023-10-10T20:25:24.656997+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:50:41.591721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644766+0000","last_change":"2023-10-10T20:25:23.639175+0000","last_active":"2023-10-10T20:25:24.644766+0000","last_peered":"2023-10-10T20:25:24.644766+0000","last_clean":"2023-10-10T20:25:24.644766+0000","last_became_active":"2023-10-10T20:25:23.638827+0000","last_became_peered":"2023-10-10T20:25:23.638827+0000","last_unstale":"2023-10-10T20:25:24.644766+0000","last_undegraded":"2023-10-10T20:25:24.644766+0000","last_fullsized":"2023-10-10T20:25:24.644766+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:40.626519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655580+0000","last_change":"2023-10-10T20:25:24.655114+0000","last_active":"2023-10-10T20:25:24.655580+0000","last_peered":"2023-10-10T20:25:24.655580+0000","last_clean":"2023-10-10T20:25:24.655580+0000","last_became_active":"2023-10-10T20:25:24.653274+0000","last_became_peered":"2023-10-10T20:25:24.653274+0000","last_unstale":"2023-10-10T20:25:24.655580+0000","last_undegraded":"2023-10-10T20:25:24.655580+0000","last_fullsized":"2023-10-10T20:25:24.655580+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:28:40.544684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667129+0000","last_change":"2023-10-10T20:25:23.640525+0000","last_active":"2023-10-10T20:25:24.667129+0000","last_peered":"2023-10-10T20:25:24.667129+0000","last_clean":"2023-10-10T20:25:24.667129+0000","last_became_active":"2023-10-10T20:25:23.640317+0000","last_became_peered":"2023-10-10T20:25:23.640317+0000","last_unstale":"2023-10-10T20:25:24.667129+0000","last_undegraded":"2023-10-10T20:25:24.667129+0000","last_fullsized":"2023-10-10T20:25:24.667129+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:18.488919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656755+0000","last_change":"2023-10-10T20:25:24.656657+0000","last_active":"2023-10-10T20:25:24.656755+0000","last_peered":"2023-10-10T20:25:24.656755+0000","last_clean":"2023-10-10T20:25:24.656755+0000","last_became_active":"2023-10-10T20:25:24.656046+0000","last_became_peered":"2023-10-10T20:25:24.656046+0000","last_unstale":"2023-10-10T20:25:24.656755+0000","last_undegraded":"2023-10-10T20:25:24.656755+0000","last_fullsized":"2023-10-10T20:25:24.656755+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:37:14.768034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647459+0000","last_change":"2023-10-10T20:25:23.642577+0000","last_active":"2023-10-10T20:25:24.647459+0000","last_peered":"2023-10-10T20:25:24.647459+0000","last_clean":"2023-10-10T20:25:24.647459+0000","last_became_active":"2023-10-10T20:25:23.642349+0000","last_became_peered":"2023-10-10T20:25:23.642349+0000","last_unstale":"2023-10-10T20:25:24.647459+0000","last_undegraded":"2023-10-10T20:25:24.647459+0000","last_fullsized":"2023-10-10T20:25:24.647459+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:51:11.677340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.656439+0000","last_change":"2023-10-10T20:25:24.655979+0000","last_active":"2023-10-10T20:25:24.656439+0000","last_peered":"2023-10-10T20:25:24.656439+0000","last_clean":"2023-10-10T20:25:24.656439+0000","last_became_active":"2023-10-10T20:25:24.655547+0000","last_became_peered":"2023-10-10T20:25:24.655547+0000","last_unstale":"2023-10-10T20:25:24.656439+0000","last_undegraded":"2023-10-10T20:25:24.656439+0000","last_fullsized":"2023-10-10T20:25:24.656439+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:56:26.014253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650314+0000","last_change":"2023-10-10T20:25:23.639906+0000","last_active":"2023-10-10T20:25:24.650314+0000","last_peered":"2023-10-10T20:25:24.650314+0000","last_clean":"2023-10-10T20:25:24.650314+0000","last_became_active":"2023-10-10T20:25:23.639669+0000","last_became_peered":"2023-10-10T20:25:23.639669+0000","last_unstale":"2023-10-10T20:25:24.650314+0000","last_undegraded":"2023-10-10T20:25:24.650314+0000","last_fullsized":"2023-10-10T20:25:24.650314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:52:07.874385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.658258+0000","last_change":"2023-10-10T20:25:24.658070+0000","last_active":"2023-10-10T20:25:24.658258+0000","last_peered":"2023-10-10T20:25:24.658258+0000","last_clean":"2023-10-10T20:25:24.658258+0000","last_became_active":"2023-10-10T20:25:24.657533+0000","last_became_peered":"2023-10-10T20:25:24.657533+0000","last_unstale":"2023-10-10T20:25:24.658258+0000","last_undegraded":"2023-10-10T20:25:24.658258+0000","last_fullsized":"2023-10-10T20:25:24.658258+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:57:52.273593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645222+0000","last_change":"2023-10-10T20:25:23.649325+0000","last_active":"2023-10-10T20:25:24.645222+0000","last_peered":"2023-10-10T20:25:24.645222+0000","last_clean":"2023-10-10T20:25:24.645222+0000","last_became_active":"2023-10-10T20:25:23.649093+0000","last_became_peered":"2023-10-10T20:25:23.649093+0000","last_unstale":"2023-10-10T20:25:24.645222+0000","last_undegraded":"2023-10-10T20:25:24.645222+0000","last_fullsized":"2023-10-10T20:25:24.645222+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:00.349551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647298+0000","last_change":"2023-10-10T20:25:24.647240+0000","last_active":"2023-10-10T20:25:24.647298+0000","last_peered":"2023-10-10T20:25:24.647298+0000","last_clean":"2023-10-10T20:25:24.647298+0000","last_became_active":"2023-10-10T20:25:24.647015+0000","last_became_peered":"2023-10-10T20:25:24.647015+0000","last_unstale":"2023-10-10T20:25:24.647298+0000","last_undegraded":"2023-10-10T20:25:24.647298+0000","last_fullsized":"2023-10-10T20:25:24.647298+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:34.264510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"130'2","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.667052+0000","last_change":"2023-10-10T20:25:23.648889+0000","last_active":"2023-10-10T20:25:24.667052+0000","last_peered":"2023-10-10T20:25:24.667052+0000","last_clean":"2023-10-10T20:25:24.667052+0000","last_became_active":"2023-10-10T20:25:23.648672+0000","last_became_peered":"2023-10-10T20:25:23.648672+0000","last_unstale":"2023-10-10T20:25:24.667052+0000","last_undegraded":"2023-10-10T20:25:24.667052+0000","last_fullsized":"2023-10-10T20:25:24.667052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:25.873244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652509+0000","last_change":"2023-10-10T20:25:24.652305+0000","last_active":"2023-10-10T20:25:24.652509+0000","last_peered":"2023-10-10T20:25:24.652509+0000","last_clean":"2023-10-10T20:25:24.652509+0000","last_became_active":"2023-10-10T20:25:24.651922+0000","last_became_peered":"2023-10-10T20:25:24.651922+0000","last_unstale":"2023-10-10T20:25:24.652509+0000","last_undegraded":"2023-10-10T20:25:24.652509+0000","last_fullsized":"2023-10-10T20:25:24.652509+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:07:06.620203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.645430+0000","last_change":"2023-10-10T20:25:23.638429+0000","last_active":"2023-10-10T20:25:24.645430+0000","last_peered":"2023-10-10T20:25:24.645430+0000","last_clean":"2023-10-10T20:25:24.645430+0000","last_became_active":"2023-10-10T20:25:23.638232+0000","last_became_peered":"2023-10-10T20:25:23.638232+0000","last_unstale":"2023-10-10T20:25:24.645430+0000","last_undegraded":"2023-10-10T20:25:24.645430+0000","last_fullsized":"2023-10-10T20:25:24.645430+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:13:55.035835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651730+0000","last_change":"2023-10-10T20:25:24.651637+0000","last_active":"2023-10-10T20:25:24.651730+0000","last_peered":"2023-10-10T20:25:24.651730+0000","last_clean":"2023-10-10T20:25:24.651730+0000","last_became_active":"2023-10-10T20:25:24.651129+0000","last_became_peered":"2023-10-10T20:25:24.651129+0000","last_unstale":"2023-10-10T20:25:24.651730+0000","last_undegraded":"2023-10-10T20:25:24.651730+0000","last_fullsized":"2023-10-10T20:25:24.651730+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:55.985324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644563+0000","last_change":"2023-10-10T20:25:23.650971+0000","last_active":"2023-10-10T20:25:24.644563+0000","last_peered":"2023-10-10T20:25:24.644563+0000","last_clean":"2023-10-10T20:25:24.644563+0000","last_became_active":"2023-10-10T20:25:23.650304+0000","last_became_peered":"2023-10-10T20:25:23.650304+0000","last_unstale":"2023-10-10T20:25:24.644563+0000","last_undegraded":"2023-10-10T20:25:24.644563+0000","last_fullsized":"2023-10-10T20:25:24.644563+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:19:45.676146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647260+0000","last_change":"2023-10-10T20:25:24.647201+0000","last_active":"2023-10-10T20:25:24.647260+0000","last_peered":"2023-10-10T20:25:24.647260+0000","last_clean":"2023-10-10T20:25:24.647260+0000","last_became_active":"2023-10-10T20:25:24.646964+0000","last_became_peered":"2023-10-10T20:25:24.646964+0000","last_unstale":"2023-10-10T20:25:24.647260+0000","last_undegraded":"2023-10-10T20:25:24.647260+0000","last_fullsized":"2023-10-10T20:25:24.647260+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:13.183419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651419+0000","last_change":"2023-10-10T20:25:23.647969+0000","last_active":"2023-10-10T20:25:24.651419+0000","last_peered":"2023-10-10T20:25:24.651419+0000","last_clean":"2023-10-10T20:25:24.651419+0000","last_became_active":"2023-10-10T20:25:23.647721+0000","last_became_peered":"2023-10-10T20:25:23.647721+0000","last_unstale":"2023-10-10T20:25:24.651419+0000","last_undegraded":"2023-10-10T20:25:24.651419+0000","last_fullsized":"2023-10-10T20:25:24.651419+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:54:13.948239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.657763+0000","last_change":"2023-10-10T20:25:24.657531+0000","last_active":"2023-10-10T20:25:24.657763+0000","last_peered":"2023-10-10T20:25:24.657763+0000","last_clean":"2023-10-10T20:25:24.657763+0000","last_became_active":"2023-10-10T20:25:24.656409+0000","last_became_peered":"2023-10-10T20:25:24.656409+0000","last_unstale":"2023-10-10T20:25:24.657763+0000","last_undegraded":"2023-10-10T20:25:24.657763+0000","last_fullsized":"2023-10-10T20:25:24.657763+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:22:49.348279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644498+0000","last_change":"2023-10-10T20:25:23.644312+0000","last_active":"2023-10-10T20:25:24.644498+0000","last_peered":"2023-10-10T20:25:24.644498+0000","last_clean":"2023-10-10T20:25:24.644498+0000","last_became_active":"2023-10-10T20:25:23.644073+0000","last_became_peered":"2023-10-10T20:25:23.644073+0000","last_unstale":"2023-10-10T20:25:24.644498+0000","last_undegraded":"2023-10-10T20:25:24.644498+0000","last_fullsized":"2023-10-10T20:25:24.644498+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:53:34.550368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652621+0000","last_change":"2023-10-10T20:25:24.652562+0000","last_active":"2023-10-10T20:25:24.652621+0000","last_peered":"2023-10-10T20:25:24.652621+0000","last_clean":"2023-10-10T20:25:24.652621+0000","last_became_active":"2023-10-10T20:25:24.652326+0000","last_became_peered":"2023-10-10T20:25:24.652326+0000","last_unstale":"2023-10-10T20:25:24.652621+0000","last_undegraded":"2023-10-10T20:25:24.652621+0000","last_fullsized":"2023-10-10T20:25:24.652621+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:01.638962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.650054+0000","last_change":"2023-10-10T20:25:23.643319+0000","last_active":"2023-10-10T20:25:24.650054+0000","last_peered":"2023-10-10T20:25:24.650054+0000","last_clean":"2023-10-10T20:25:24.650054+0000","last_became_active":"2023-10-10T20:25:23.643108+0000","last_became_peered":"2023-10-10T20:25:23.643108+0000","last_unstale":"2023-10-10T20:25:24.650054+0000","last_undegraded":"2023-10-10T20:25:24.650054+0000","last_fullsized":"2023-10-10T20:25:24.650054+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:46:03.440331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655599+0000","last_change":"2023-10-10T20:25:24.655265+0000","last_active":"2023-10-10T20:25:24.655599+0000","last_peered":"2023-10-10T20:25:24.655599+0000","last_clean":"2023-10-10T20:25:24.655599+0000","last_became_active":"2023-10-10T20:25:24.654435+0000","last_became_peered":"2023-10-10T20:25:24.654435+0000","last_unstale":"2023-10-10T20:25:24.655599+0000","last_undegraded":"2023-10-10T20:25:24.655599+0000","last_fullsized":"2023-10-10T20:25:24.655599+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:11:22.406668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.647654+0000","last_change":"2023-10-10T20:25:23.643033+0000","last_active":"2023-10-10T20:25:24.647654+0000","last_peered":"2023-10-10T20:25:24.647654+0000","last_clean":"2023-10-10T20:25:24.647654+0000","last_became_active":"2023-10-10T20:25:23.642803+0000","last_became_peered":"2023-10-10T20:25:23.642803+0000","last_unstale":"2023-10-10T20:25:24.647654+0000","last_undegraded":"2023-10-10T20:25:24.647654+0000","last_fullsized":"2023-10-10T20:25:24.647654+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:52:35.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.652691+0000","last_change":"2023-10-10T20:25:24.652634+0000","last_active":"2023-10-10T20:25:24.652691+0000","last_peered":"2023-10-10T20:25:24.652691+0000","last_clean":"2023-10-10T20:25:24.652691+0000","last_became_active":"2023-10-10T20:25:24.652165+0000","last_became_peered":"2023-10-10T20:25:24.652165+0000","last_unstale":"2023-10-10T20:25:24.652691+0000","last_undegraded":"2023-10-10T20:25:24.652691+0000","last_fullsized":"2023-10-10T20:25:24.652691+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:08:17.152615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644637+0000","last_change":"2023-10-10T20:25:23.651071+0000","last_active":"2023-10-10T20:25:24.644637+0000","last_peered":"2023-10-10T20:25:24.644637+0000","last_clean":"2023-10-10T20:25:24.644637+0000","last_became_active":"2023-10-10T20:25:23.650533+0000","last_became_peered":"2023-10-10T20:25:23.650533+0000","last_unstale":"2023-10-10T20:25:24.644637+0000","last_undegraded":"2023-10-10T20:25:24.644637+0000","last_fullsized":"2023-10-10T20:25:24.644637+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:57:12.179132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.655588+0000","last_change":"2023-10-10T20:25:24.655399+0000","last_active":"2023-10-10T20:25:24.655588+0000","last_peered":"2023-10-10T20:25:24.655588+0000","last_clean":"2023-10-10T20:25:24.655588+0000","last_became_active":"2023-10-10T20:25:24.655152+0000","last_became_peered":"2023-10-10T20:25:24.655152+0000","last_unstale":"2023-10-10T20:25:24.655588+0000","last_undegraded":"2023-10-10T20:25:24.655588+0000","last_fullsized":"2023-10-10T20:25:24.655588+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:23.629651+0000","last_clean_scrub_stamp":"2023-10-10T20:25:23.629651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:01.353838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.644584+0000","last_change":"2023-10-10T20:25:23.639303+0000","last_active":"2023-10-10T20:25:24.644584+0000","last_peered":"2023-10-10T20:25:24.644584+0000","last_clean":"2023-10-10T20:25:24.644584+0000","last_became_active":"2023-10-10T20:25:23.639030+0000","last_became_peered":"2023-10-10T20:25:23.639030+0000","last_unstale":"2023-10-10T20:25:24.644584+0000","last_undegraded":"2023-10-10T20:25:24.644584+0000","last_fullsized":"2023-10-10T20:25:24.644584+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:25:22.625363+0000","last_clean_scrub_stamp":"2023-10-10T20:25:22.625363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:45.096131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":262,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-10T20:25:24.651072+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:25:24.651072+0000","last_peered":"2023-10-10T20:25:24.651072+0000","last_clean":"2023-10-10T20:25:24.651072+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:25:24.651072+0000","last_undegraded":"2023-10-10T20:25:24.651072+0000","last_fullsized":"2023-10-10T20:25:24.651072+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1279,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194033,"statfs":{"total":200000000,"available":198690247,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70299999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40999999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.439}]},{"osd":4,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.74199999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.318},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.376},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281133,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.53500000000000003},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":1.865,"15min":1.865},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64200000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72799999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.44}]},{"osd":3,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69099999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71499999999999997,"5min":15.862,"15min":15.862},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45300000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.62,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.313},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.46200000000000002},"min":{"1min":0.28000000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.39700000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.503},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.50700000000000001,"15min":1.6100000000000001},"last":0.317},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.38300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":1.647},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38700000000000001},"min":{"1min":0.24199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.627,"15min":1.524},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.58799999999999997,"15min":1.3959999999999999},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38800000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":1.4470000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63,"15min":1.673},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.90900000000000003,"15min":1.633},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.506},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.696},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.9359999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.97499999999999998,"15min":1.7490000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.872,"15min":1.585},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.478},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.63},"min":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42299999999999999},"max":{"1min":0.75900000000000001,"5min":0.91000000000000003,"15min":1.722},"last":0.73499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1279,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194033,"statfs":{"total":200000000,"available":198690198,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.44500000000000001},"min":{"1min":0.216,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.379},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.435,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58099999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.313}]},{"osd":2,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.47399999999999998},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52400000000000002,"5min":1.647,"15min":2.4830000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":5.891,"15min":5.891},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.433},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.57999999999999996,"5min":1.462,"15min":1.462},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.433}]},{"osd":5,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.92600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77800000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.443}]},{"osd":7,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":5.758,"15min":5.758},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.79200000000000004,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.40500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":700,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194611,"statfs":{"total":200000000,"available":199282214,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.38600000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.54200000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53100000000000003,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.22},"max":{"1min":0.42299999999999999,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.48899999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.378}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.50600000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.49399999999999999,"5min":11.815,"15min":11.815},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.56699999999999995,"15min":0.46300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.47899999999999998,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.32300000000000001,"15min":0.28100000000000003},"max":{"1min":0.82899999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70199999999999996,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.72499999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":6.399,"15min":6.399},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.66700000000000004,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75900000000000001,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.67300000000000004,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":703,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194608,"statfs":{"total":200000000,"available":199279597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.30299999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.34200000000000003,"15min":0.315},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.44,"5min":0.57099999999999995,"15min":0.65600000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.55200000000000005,"15min":0.67900000000000005},"last":0.313},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53700000000000003,"5min":0.63200000000000001,"15min":0.751},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.25,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.58199999999999996,"5min":0.621,"15min":0.74099999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.63500000000000001,"15min":0.80700000000000005},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.376},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.436}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.35599999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42299999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.499}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85099999999999998,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":701,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194610,"statfs":{"total":200000000,"available":199281203,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.496},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.41299999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73999999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.49399999999999999},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.628}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"max":{"1min":0.92600000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":702,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194609,"statfs":{"total":200000000,"available":199280627,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.35399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.91200000000000003},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.79900000000000004},"last":0.496}]},{"osd":2,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.35399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.72599999999999998,"5min":0.83599999999999997,"15min":0.89000000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.78000000000000003,"15min":0.96299999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52200000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.752,"15min":0.78200000000000003},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.621},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.67500000000000004}]}]}],"pool_statfs":[]}} 2023-10-10T20:25:29.073 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:25:29.074 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:25:29.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.228+0000 7f0116ba5700 1 Processor -- start 2023-10-10T20:25:29.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.228+0000 7f0116ba5700 1 -- start start 2023-10-10T20:25:29.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0110159ec0 0x7f01100af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f011015a290 0x7f01100afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011006a5a0 0x7f011006aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011006af60 con 0x7f011015a290 2023-10-10T20:25:29.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01100b0790 con 0x7f0110159ec0 2023-10-10T20:25:29.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0116ba5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01100ce1f0 con 0x7f011006a5a0 2023-10-10T20:25:29.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f011015a290 0x7f01100afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f011015a290 0x7f01100afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57786/0 (socket says 172.21.15.148:57786) 2023-10-10T20:25:29.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 -- 172.21.15.148:0/60453291 learned_addr learned my addr 172.21.15.148:0/60453291 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:29.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0115142700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011006a5a0 0x7f011006aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0114941700 1 --2- 172.21.15.148:0/60453291 >> v2:172.21.15.161:3300/0 conn(0x7f0110159ec0 0x7f01100af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 -- 172.21.15.148:0/60453291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011006a5a0 msgr2=0x7f011006aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011006a5a0 0x7f011006aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 -- 172.21.15.148:0/60453291 >> v2:172.21.15.161:3300/0 conn(0x7f0110159ec0 msgr2=0x7f01100af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 --2- 172.21.15.148:0/60453291 >> v2:172.21.15.161:3300/0 conn(0x7f0110159ec0 0x7f01100af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 -- 172.21.15.148:0/60453291 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100b0b20 con 0x7f011015a290 2023-10-10T20:25:29.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.229+0000 7f0107fff700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011015a290 0x7f01100afe80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f01100898e0 tx=0x7f0100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e0e72fbf53df23 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f01077fe700 1 -- 172.21.15.148:0/60453291 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000b640 con 0x7f011015a290 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f01077fe700 1 -- 172.21.15.148:0/60453291 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000b7a0 con 0x7f011015a290 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f01077fe700 1 -- 172.21.15.148:0/60453291 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000bb60 con 0x7f011015a290 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 -- 172.21.15.148:0/60453291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011015a290 msgr2=0x7f01100afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011015a290 0x7f01100afe80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f01100898e0 tx=0x7f0100009250 comp rx=0 tx=0).stop 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 -- 172.21.15.148:0/60453291 shutdown_connections 2023-10-10T20:25:29.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011006a5a0 0x7f011006aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 --2- 172.21.15.148:0/60453291 >> v2:172.21.15.161:3300/0 conn(0x7f0110159ec0 0x7f01100af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 --2- 172.21.15.148:0/60453291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011015a290 0x7f01100afe80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 -- 172.21.15.148:0/60453291 >> 172.21.15.148:0/60453291 conn(0x7f01100d1880 msgr2=0x7f01100cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:29.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 -- 172.21.15.148:0/60453291 shutdown_connections 2023-10-10T20:25:29.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.230+0000 7f0116ba5700 1 -- 172.21.15.148:0/60453291 wait complete. 2023-10-10T20:25:29.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.231+0000 7f0116ba5700 1 Processor -- start 2023-10-10T20:25:29.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.231+0000 7f0116ba5700 1 -- start start 2023-10-10T20:25:29.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0110159ec0 0x7f011016dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011015a290 0x7f011015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100b4590 con 0x7f011006a5a0 2023-10-10T20:25:29.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011015ea50 con 0x7f0110159ec0 2023-10-10T20:25:29.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0116ba5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011015ed30 con 0x7f011015a290 2023-10-10T20:25:29.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0114941700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0114941700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57800/0 (socket says 172.21.15.148:57800) 2023-10-10T20:25:29.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0114941700 1 -- 172.21.15.148:0/3582431756 learned_addr learned my addr 172.21.15.148:0/3582431756 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:29.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0107fff700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0110159ec0 0x7f011016dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.232+0000 7f0115142700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011015a290 0x7f011015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 -- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011015a290 msgr2=0x7f011015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011015a290 0x7f011015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 -- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0110159ec0 msgr2=0x7f011016dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0110159ec0 0x7f011016dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 -- 172.21.15.148:0/3582431756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000094f0 con 0x7f011006a5a0 2023-10-10T20:25:29.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0114941700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ef0 tx=0x7f010c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00cb70 con 0x7f011006a5a0 2023-10-10T20:25:29.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c00f040 con 0x7f011006a5a0 2023-10-10T20:25:29.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011015f010 con 0x7f011006a5a0 2023-10-10T20:25:29.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c014480 con 0x7f011006a5a0 2023-10-10T20:25:29.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.233+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01100b68f0 con 0x7f011006a5a0 2023-10-10T20:25:29.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.234+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011015f4b0 con 0x7f011006a5a0 2023-10-10T20:25:29.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.234+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f010c0147f0 con 0x7f011006a5a0 2023-10-10T20:25:29.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.235+0000 7f0104ff9700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 0x7f00fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.235+0000 7f0107fff700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 0x7f00fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.235+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f010c085130 con 0x7f011006a5a0 2023-10-10T20:25:29.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.235+0000 7f0107fff700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 0x7f00fc0693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f01100898e0 tx=0x7f0100017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.239+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f010c0523a0 con 0x7f011006a5a0 2023-10-10T20:25:29.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.494+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f011004c3c0 con 0x7f011006a5a0 2023-10-10T20:25:29.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.498+0000 7f0104ff9700 1 -- 172.21.15.148:0/3582431756 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f010c0559c0 con 0x7f011006a5a0 2023-10-10T20:25:29.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 msgr2=0x7f00fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 0x7f00fc0693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f01100898e0 tx=0x7f0100017000 comp rx=0 tx=0).stop 2023-10-10T20:25:29.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 msgr2=0x7f011016d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f010c009ef0 tx=0x7f010c00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:29.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 shutdown_connections 2023-10-10T20:25:29.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f00fc066f30 0x7f00fc0693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f011015a290 0x7f011015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f011006a5a0 0x7f011016d7b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 --2- 172.21.15.148:0/3582431756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0110159ec0 0x7f011016dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 >> 172.21.15.148:0/3582431756 conn(0x7f01100d1880 msgr2=0x7f01100cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:29.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 shutdown_connections 2023-10-10T20:25:29.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.500+0000 7f0116ba5700 1 -- 172.21.15.148:0/3582431756 wait complete. 2023-10-10T20:25:29.505 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:25:29.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.628+0000 7f9761d6f700 1 Processor -- start 2023-10-10T20:25:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.628+0000 7f9761d6f700 1 -- start start 2023-10-10T20:25:29.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9761d6f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f975c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9761d6f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f975c0cbf10 0x7f975c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9761d6f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f975c0ad5c0 con 0x7f975c0cd420 2023-10-10T20:25:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9761d6f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c0ac9b0 con 0x7f975c0cbf10 2023-10-10T20:25:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9761d6f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0acc90 con 0x7f975c156980 2023-10-10T20:25:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f975c0cd420 0x7f975c0cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57802/0 (socket says 172.21.15.148:57802) 2023-10-10T20:25:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975bfff700 1 -- 172.21.15.148:0/663310520 learned_addr learned my addr 172.21.15.148:0/663310520 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f976156e700 1 --2- 172.21.15.148:0/663310520 >> v2:172.21.15.161:3300/0 conn(0x7f975c0cbf10 0x7f975c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f9760d6d700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f975c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107643006 0 0) 0x7f975c0ad5c0 con 0x7f975c0cd420 2023-10-10T20:25:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f974c003570 con 0x7f975c0cd420 2023-10-10T20:25:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3845190485 0 0) 0x7f974c003570 con 0x7f975c0cd420 2023-10-10T20:25:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.629+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 msgr2=0x7f975c0ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f975c0ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 >> v2:172.21.15.161:3300/0 conn(0x7f975c0cbf10 msgr2=0x7f975c06d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 --2- 172.21.15.148:0/663310520 >> v2:172.21.15.161:3300/0 conn(0x7f975c0cbf10 0x7f975c06d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c0acf70 con 0x7f975c0cd420 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f9748001470 con 0x7f975c0cd420 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f9760d6d700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f975c0ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f976156e700 1 --2- 172.21.15.148:0/663310520 >> v2:172.21.15.161:3300/0 conn(0x7f975c0cbf10 0x7f975c06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 >> v1:172.21.15.148:6789/0 conn(0x7f975c0cd420 legacy=0x7f975c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c003a60 0x7f974c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974c004810 0x7f974c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f974c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0ac9b0 con 0x7f974c004810 2023-10-10T20:25:29.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0acc90 con 0x7f974c003a60 2023-10-10T20:25:29.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c00ba60 con 0x7f975c156980 2023-10-10T20:25:29.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974c004810 0x7f974c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f9760d6d700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c003a60 0x7f974c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f976156e700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f974c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 -- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 msgr2=0x7f974c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c156980 0x7f974c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 -- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c003a60 msgr2=0x7f974c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:29.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c003a60 0x7f974c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 -- 172.21.15.148:0/663310520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009420 con 0x7f974c004810 2023-10-10T20:25:29.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975bfff700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974c004810 0x7f974c004c80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f974800c3c0 tx=0x7f9748001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539c8d9147e7485 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974800cc90 con 0x7f974c004810 2023-10-10T20:25:29.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974800ce30 con 0x7f974c004810 2023-10-10T20:25:29.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.630+0000 7f975b7fe700 1 -- 172.21.15.148:0/663310520 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748005b70 con 0x7f974c004810 2023-10-10T20:25:29.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 -- 172.21.15.148:0/663310520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974c004810 msgr2=0x7f974c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 --2- 172.21.15.148:0/663310520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f974c004810 0x7f974c004c80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f974800c3c0 tx=0x7f9748001e90 comp rx=0 tx=0).stop 2023-10-10T20:25:29.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f976156e700 1 -- 172.21.15.148:0/663310520 reap_dead start 2023-10-10T20:25:29.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 -- 172.21.15.148:0/663310520 shutdown_connections 2023-10-10T20:25:29.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 -- 172.21.15.148:0/663310520 >> 172.21.15.148:0/663310520 conn(0x7f975c0680e0 msgr2=0x7f975c0cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:29.644 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 -- 172.21.15.148:0/663310520 shutdown_connections 2023-10-10T20:25:29.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.631+0000 7f9761d6f700 1 -- 172.21.15.148:0/663310520 wait complete. 2023-10-10T20:25:29.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.632+0000 7f9761d6f700 1 Processor -- start 2023-10-10T20:25:29.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.632+0000 7f9761d6f700 1 -- start start 2023-10-10T20:25:29.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c004810 0x7f975c16da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.645 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c069f90 0x7f975c0ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0ac200 con 0x7f975c0abb20 2023-10-10T20:25:29.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0ca820 con 0x7f974c004810 2023-10-10T20:25:29.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9761d6f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0cab00 con 0x7f975c069f90 2023-10-10T20:25:29.646 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57812/0 (socket says 172.21.15.148:57812) 2023-10-10T20:25:29.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 -- 172.21.15.148:0/3535451943 learned_addr learned my addr 172.21.15.148:0/3535451943 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:29.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9760d6d700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c004810 0x7f975c16da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f975bfff700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c069f90 0x7f975c0ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 -- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c069f90 msgr2=0x7f975c0ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c069f90 0x7f975c0ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 -- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c004810 msgr2=0x7f975c16da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c004810 0x7f975c16da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 -- 172.21.15.148:0/3535451943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c0cade0 con 0x7f975c0abb20 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f976156e700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f975c06cc80 tx=0x7f975400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.633+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400e040 con 0x7f975c0abb20 2023-10-10T20:25:29.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.634+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975400f040 con 0x7f975c0abb20 2023-10-10T20:25:29.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.634+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c0cb0c0 con 0x7f975c0abb20 2023-10-10T20:25:29.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.634+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400fdb0 con 0x7f975c0abb20 2023-10-10T20:25:29.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.634+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c06a400 con 0x7f975c0abb20 2023-10-10T20:25:29.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.635+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c0cb410 con 0x7f975c0abb20 2023-10-10T20:25:29.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.635+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9754015640 con 0x7f975c0abb20 2023-10-10T20:25:29.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.635+0000 7f9758ff9700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 0x7f97400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:29.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.636+0000 7f9760d6d700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 0x7f97400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:29.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.636+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9754084f70 con 0x7f975c0abb20 2023-10-10T20:25:29.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.636+0000 7f9760d6d700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 0x7f97400693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9750006f00 tx=0x7f97500091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:29.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.640+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97540521e0 con 0x7f975c0abb20 2023-10-10T20:25:29.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.865+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f975c06a680 con 0x7f975c0abb20 2023-10-10T20:25:29.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.866+0000 7f9758ff9700 1 -- 172.21.15.148:0/3535451943 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9754055800 con 0x7f975c0abb20 2023-10-10T20:25:29.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:29.868 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:25:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 msgr2=0x7f97400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 0x7f97400693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9750006f00 tx=0x7f97500091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 msgr2=0x7f975c149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f975c06cc80 tx=0x7f975400b040 comp rx=0 tx=0).stop 2023-10-10T20:25:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 shutdown_connections 2023-10-10T20:25:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9740066f30 0x7f97400693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f975c0abb20 0x7f975c149c40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974c004810 0x7f975c16da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 --2- 172.21.15.148:0/3535451943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c069f90 0x7f975c0ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 >> 172.21.15.148:0/3535451943 conn(0x7f975c0680e0 msgr2=0x7f975c1525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 shutdown_connections 2023-10-10T20:25:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:29.868+0000 7f9761d6f700 1 -- 172.21.15.148:0/3535451943 wait complete. 2023-10-10T20:25:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:25:29.948 INFO:tasks.ceph.mds.d.smithi161.stderr:2023-10-10T20:25:29.947+0000 7fcfaeff7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:25:30.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:25:24.013584+0000', 'modified': '2023-10-10T20:25:27.661435+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7095}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7095': {'gid': 7095, 'name': 'd', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/2686518159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 2686518159}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2686518159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:25:30.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:25:30.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-10T20:25:30.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.403+0000 7f5e78aa1700 1 Processor -- start 2023-10-10T20:25:30.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.403+0000 7f5e78aa1700 1 -- start start 2023-10-10T20:25:30.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e78aa1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 0x7f5e7415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e78aa1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7415cbb0 con 0x7f5e7415a620 2023-10-10T20:25:30.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e78aa1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7415bfa0 con 0x7f5e74157c40 2023-10-10T20:25:30.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e78aa1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7415c280 con 0x7f5e74157870 2023-10-10T20:25:30.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e71d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5e7415a620 0x7f5e7415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57804/0 (socket says 172.21.15.148:57804) 2023-10-10T20:25:30.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e71d9b700 1 -- 172.21.15.148:0/815423765 learned_addr learned my addr 172.21.15.148:0/815423765 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:30.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.404+0000 7f5e7259c700 1 --2- 172.21.15.148:0/815423765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 0x7f5e7415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806233127 0 0) 0x7f5e7415cbb0 con 0x7f5e7415a620 2023-10-10T20:25:30.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e60003570 con 0x7f5e7415a620 2023-10-10T20:25:30.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231675844 0 0) 0x7f5e7415bfa0 con 0x7f5e74157c40 2023-10-10T20:25:30.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e7415cbb0 con 0x7f5e74157c40 2023-10-10T20:25:30.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7259c700 1 -- 172.21.15.148:0/815423765 >> v1:172.21.15.161:6789/0 conn(0x7f5e74157c40 legacy=0x7f5e74158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7259c700 1 -- 172.21.15.148:0/815423765 >> v1:172.21.15.148:6789/0 conn(0x7f5e7415a620 legacy=0x7f5e7415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7259c700 1 -- 172.21.15.148:0/815423765 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7415c560 con 0x7f5e74157870 2023-10-10T20:25:30.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7259c700 1 --2- 172.21.15.148:0/815423765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 0x7f5e7415b6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009e90 tx=0x7f5e6c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5316feb8c8ffa84e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00e070 con 0x7f5e74157870 2023-10-10T20:25:30.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c00bb80 con 0x7f5e74157870 2023-10-10T20:25:30.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.405+0000 7f5e7159a700 1 -- 172.21.15.148:0/815423765 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c012430 con 0x7f5e74157870 2023-10-10T20:25:30.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 -- 172.21.15.148:0/815423765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 msgr2=0x7f5e7415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/815423765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 0x7f5e7415b6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009e90 tx=0x7f5e6c009780 comp rx=0 tx=0).stop 2023-10-10T20:25:30.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 -- 172.21.15.148:0/815423765 shutdown_connections 2023-10-10T20:25:30.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/815423765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157870 0x7f5e7415b6a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 -- 172.21.15.148:0/815423765 >> 172.21.15.148:0/815423765 conn(0x7f5e74068ce0 msgr2=0x7f5e740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 -- 172.21.15.148:0/815423765 shutdown_connections 2023-10-10T20:25:30.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.406+0000 7f5e78aa1700 1 -- 172.21.15.148:0/815423765 wait complete. 2023-10-10T20:25:30.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 Processor -- start 2023-10-10T20:25:30.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 -- start start 2023-10-10T20:25:30.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74157870 0x7f5e74152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157c40 0x7f5e74150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.407+0000 7f5e78aa1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7414ffc0 con 0x7f5e7415a620 2023-10-10T20:25:30.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e78aa1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b2fb0 con 0x7f5e74157870 2023-10-10T20:25:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e78aa1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7414f3b0 con 0x7f5e74157c40 2023-10-10T20:25:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57824/0 (socket says 172.21.15.148:57824) 2023-10-10T20:25:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e71d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157c40 0x7f5e74150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 -- 172.21.15.148:0/1933348279 learned_addr learned my addr 172.21.15.148:0/1933348279 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74157870 0x7f5e74152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 -- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157c40 msgr2=0x7f5e74150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157c40 0x7f5e74150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 -- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74157870 msgr2=0x7f5e74152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74157870 0x7f5e74152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 -- 172.21.15.148:0/1933348279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009a20 con 0x7f5e7415a620 2023-10-10T20:25:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c5c0 tx=0x7f5e64007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e6a7fc700 1 -- 172.21.15.148:0/1933348279 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64016070 con 0x7f5e7415a620 2023-10-10T20:25:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e6a7fc700 1 -- 172.21.15.148:0/1933348279 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64011590 con 0x7f5e7415a620 2023-10-10T20:25:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7414f690 con 0x7f5e7415a620 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e6a7fc700 1 -- 172.21.15.148:0/1933348279 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64011970 con 0x7f5e7415a620 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.408+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7414fc50 con 0x7f5e7415a620 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.410+0000 7f5e6a7fc700 1 -- 172.21.15.148:0/1933348279 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e64012420 con 0x7f5e7415a620 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.410+0000 7f5e6a7fc700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 0x7f5e540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.410+0000 7f5e6a7fc700 1 -- 172.21.15.148:0/1933348279 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e64083f90 con 0x7f5e7415a620 2023-10-10T20:25:30.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.410+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 0x7f5e540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.410+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 0x7f5e540693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009de0 tx=0x7f5e6c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.412+0000 7f5e81d9ab80 1 Processor -- start 2023-10-10T20:25:30.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.412+0000 7f5e81d9ab80 1 -- start start 2023-10-10T20:25:30.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336be2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bb5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336b10a70 con 0x55d336bb9a50 2023-10-10T20:25:30.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336b0e270 con 0x55d336be2f60 2023-10-10T20:25:30.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d336b0a060 con 0x55d336c03d00 2023-10-10T20:25:30.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57826/0 (socket says 172.21.15.148:57826) 2023-10-10T20:25:30.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 -- 172.21.15.148:0/1869968398 learned_addr learned my addr 172.21.15.148:0/1869968398 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bb5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336be2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 -- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 msgr2=0x55d336be2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336be2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 -- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 msgr2=0x55d336bb5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bb5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 -- 172.21.15.148:0/1869968398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3369a4270 con 0x55d336bb9a50 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bb5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:30.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.413+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b610 tx=0x7f5e6c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f985275c43a58852 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e537fe700 1 -- 172.21.15.148:0/1869968398 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c022070 con 0x55d336bb9a50 2023-10-10T20:25:30.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e537fe700 1 -- 172.21.15.148:0/1869968398 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c033480 con 0x55d336bb9a50 2023-10-10T20:25:30.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e537fe700 1 -- 172.21.15.148:0/1869968398 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c033610 con 0x55d336bb9a50 2023-10-10T20:25:30.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1869968398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 msgr2=0x55d336bb9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b610 tx=0x7f5e6c022000 comp rx=0 tx=0).stop 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1869968398 shutdown_connections 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bb5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bb9e20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1869968398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336be2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1869968398 >> 172.21.15.148:0/1869968398 conn(0x55d336bbb370 msgr2=0x55d336bbb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1869968398 shutdown_connections 2023-10-10T20:25:30.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.414+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1869968398 wait complete. 2023-10-10T20:25:30.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.415+0000 7f5e81d9ab80 1 -- start start 2023-10-10T20:25:30.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.415+0000 7f5e81d9ab80 1 Processor -- start 2023-10-10T20:25:30.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bfb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336c000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336ab5890 con 0x55d336bb9a50 2023-10-10T20:25:30.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336ad0ea0 con 0x55d336be2f60 2023-10-10T20:25:30.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d33698ce30 con 0x55d336c03d00 2023-10-10T20:25:30.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e81d9ab80 10 client.0 _ll_register_callbacks cb 0x55d336b7d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:25:30.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57838/0 (socket says 172.21.15.148:57838) 2023-10-10T20:25:30.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 -- 172.21.15.148:0/985719010 learned_addr learned my addr 172.21.15.148:0/985719010 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bfb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336c000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 -- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 msgr2=0x55d336c000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336c000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 -- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 msgr2=0x55d336bfb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bfb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e7259c700 1 -- 172.21.15.148:0/985719010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7415c560 con 0x55d336bb9a50 2023-10-10T20:25:30.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.416+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336c000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:30.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e7259c700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c029c20 tx=0x7f5e6c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108d7765acfdc678 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:25:30.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c022040 con 0x55d336bb9a50 2023-10-10T20:25:30.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e6c033a40 con 0x55d336bb9a50 2023-10-10T20:25:30.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d336baeb70 con 0x55d336bb9a50 2023-10-10T20:25:30.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6c033e10 con 0x55d336bb9a50 2023-10-10T20:25:30.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 10 client.7224 fetch_fsmap learned FSMap version 64 2023-10-10T20:25:30.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55d336946fe0 con 0x55d336bb9a50 2023-10-10T20:25:30.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d336baeb70 con 0x55d336bb9a50 2023-10-10T20:25:30.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5e6c039020 con 0x55d336bb9a50 2023-10-10T20:25:30.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 10 client.7224 fetch_fsmap finished waiting for FSMap version 64 2023-10-10T20:25:30.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 10 client.7224 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:25:30.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 20 client.7224 populate_metadata read hostname 'smithi148' 2023-10-10T20:25:30.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336bb4c20 0x55d336bb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 4 client.7224 mds_command: new command op to 7095 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:25:30.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 --> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d336b8baa0 con 0x55d336bb4c20 2023-10-10T20:25:30.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.417+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336bb4c20 0x55d336bb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e50ff9700 10 client.7224 ms_handle_connect on v2:172.21.15.161:6833/2686518159 2023-10-10T20:25:30.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336bb4c20 0x55d336bb4ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67de5242ca9b907 server_cookie=2c483c8543da3e2a in_seq=0 out_seq=0 2023-10-10T20:25:30.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e50ff9700 1 -- 172.21.15.148:0/985719010 <== mds.0 v2:172.21.15.161:6833/2686518159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d336b8baa0 con 0x55d336bb4c20 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e50ff9700 10 client.7224 handle_command_reply: tid=0 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 client.7224 shutdown 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 msgr2=0x55d336bdf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c029c20 tx=0x7f5e6c02c040 comp rx=0 tx=0).stop 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 shutdown_connections 2023-10-10T20:25:30.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336bb4c20 0x55d336bb4ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336c03d00 0x55d336c000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb9a50 0x55d336bdf040 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.418+0000 7f5e50ff9700 0 client.7224 ms_handle_reset on v2:172.21.15.161:6833/2686518159 2023-10-10T20:25:30.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.419+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/985719010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336be2f60 0x55d336bfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.419+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 >> 172.21.15.148:0/985719010 conn(0x55d336bbb370 msgr2=0x55d336bff550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.419+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 shutdown_connections 2023-10-10T20:25:30.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.419+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/985719010 wait complete. 2023-10-10T20:25:30.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.419+0000 7f5e81d9ab80 20 client.7224 trim_cache size 0 max 16384 2023-10-10T20:25:30.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.435+0000 7f5e81d9ab80 1 Processor -- start 2023-10-10T20:25:30.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 -- start start 2023-10-10T20:25:30.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bb4c20 0x55d336bb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336be2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336bb8c70 con 0x55d336bbb370 2023-10-10T20:25:30.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336bf9fb0 con 0x55d336bb4c20 2023-10-10T20:25:30.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d336bfba40 con 0x55d336bb9a50 2023-10-10T20:25:30.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57852/0 (socket says 172.21.15.148:57852) 2023-10-10T20:25:30.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.436+0000 7f5e7259c700 1 -- 172.21.15.148:0/481158101 learned_addr learned my addr 172.21.15.148:0/481158101 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 -- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 msgr2=0x55d336be2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336be2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 -- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bb4c20 msgr2=0x55d336bb4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bb4c20 0x55d336bb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 -- 172.21.15.148:0/481158101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d336ad8780 con 0x55d336bbb370 2023-10-10T20:25:30.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e7259c700 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c033800 tx=0x7f5e6c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e35f8b9d9e17468 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e53fff700 1 -- 172.21.15.148:0/481158101 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c022070 con 0x55d336bbb370 2023-10-10T20:25:30.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e53fff700 1 -- 172.21.15.148:0/481158101 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c03b400 con 0x55d336bbb370 2023-10-10T20:25:30.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e53fff700 1 -- 172.21.15.148:0/481158101 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0125e0 con 0x55d336bbb370 2023-10-10T20:25:30.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/481158101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 msgr2=0x55d336c12bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c033800 tx=0x7f5e6c034000 comp rx=0 tx=0).stop 2023-10-10T20:25:30.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/481158101 shutdown_connections 2023-10-10T20:25:30.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336be2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bbb370 0x55d336c12bc0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/481158101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bb4c20 0x55d336bb4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/481158101 >> 172.21.15.148:0/481158101 conn(0x55d336bb8fa0 msgr2=0x55d336bb24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/481158101 shutdown_connections 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.437+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/481158101 wait complete. 2023-10-10T20:25:30.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.438+0000 7f5e81d9ab80 1 -- start start 2023-10-10T20:25:30.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.438+0000 7f5e81d9ab80 1 Processor -- start 2023-10-10T20:25:30.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336c65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 0x55d336c00ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336c670f0 con 0x55d336bb4c20 2023-10-10T20:25:30.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d336bfb1b0 con 0x55d336bbb370 2023-10-10T20:25:30.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d336bf6c50 con 0x55d336bb9a50 2023-10-10T20:25:30.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e81d9ab80 10 client.0 _ll_register_callbacks cb 0x55d336b7d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:25:30.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57862/0 (socket says 172.21.15.148:57862) 2023-10-10T20:25:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 -- 172.21.15.148:0/1285860610 learned_addr learned my addr 172.21.15.148:0/1285860610 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336c65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 0x55d336c00ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 -- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 msgr2=0x55d336c65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336c65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 -- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 msgr2=0x55d336c00ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 0x55d336c00ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e7259c700 1 -- 172.21.15.148:0/1285860610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d336946fe0 con 0x55d336bb4c20 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.439+0000 7f5e72d9d700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 0x55d336c00ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e7259c700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001050 tx=0x7f5e6c02b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f58f430f4d129a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:25:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c022070 con 0x55d336bb4c20 2023-10-10T20:25:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e6c040820 con 0x55d336bb4c20 2023-10-10T20:25:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e6c033480 con 0x55d336bb4c20 2023-10-10T20:25:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6c040bf0 con 0x55d336bb4c20 2023-10-10T20:25:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 10 client.7230 fetch_fsmap learned FSMap version 64 2023-10-10T20:25:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55d336abf470 con 0x55d336bb4c20 2023-10-10T20:25:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c033480 con 0x55d336bb4c20 2023-10-10T20:25:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5e6c04e020 con 0x55d336bb4c20 2023-10-10T20:25:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 10 client.7230 fetch_fsmap finished waiting for FSMap version 64 2023-10-10T20:25:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 10 client.7230 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-10T20:25:30.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 20 client.7230 populate_metadata read hostname 'smithi148' 2023-10-10T20:25:30.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336be2f60 0x55d336bf9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 4 client.7230 mds_command: new command op to 7095 tid=0[{"prefix": "status"}] 2023-10-10T20:25:30.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 --> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d336aace00 con 0x55d336be2f60 2023-10-10T20:25:30.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.440+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336be2f60 0x55d336bf9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e50ff9700 10 client.7230 ms_handle_connect on v2:172.21.15.161:6833/2686518159 2023-10-10T20:25:30.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e71d9b700 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336be2f60 0x55d336bf9fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4758a204fac5bc9 server_cookie=37250e0be6e6ef55 in_seq=0 out_seq=0 2023-10-10T20:25:30.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e50ff9700 1 -- 172.21.15.148:0/1285860610 <== mds.0 v2:172.21.15.161:6833/2686518159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55d336aace00 con 0x55d336be2f60 2023-10-10T20:25:30.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e50ff9700 10 client.7230 handle_command_reply: tid=0 2023-10-10T20:25:30.461 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 client.7230 shutdown 2023-10-10T20:25:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 msgr2=0x55d336bde340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001050 tx=0x7f5e6c02b070 comp rx=0 tx=0).stop 2023-10-10T20:25:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 shutdown_connections 2023-10-10T20:25:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:6833/2686518159,v1:172.21.15.161:6835/2686518159] conn(0x55d336be2f60 0x55d336bf9fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d336bbb370 0x55d336c00ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e50ff9700 0 client.7230 ms_handle_reset on v2:172.21.15.161:6833/2686518159 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d336bb4c20 0x55d336bde340 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 --2- 172.21.15.148:0/1285860610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d336bb9a50 0x55d336c65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 >> 172.21.15.148:0/1285860610 conn(0x55d336bb8fa0 msgr2=0x55d336bffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 shutdown_connections 2023-10-10T20:25:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 1 -- 172.21.15.148:0/1285860610 wait complete. 2023-10-10T20:25:30.464 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.441+0000 7f5e81d9ab80 20 client.7230 trim_cache size 0 max 16384 2023-10-10T20:25:30.464 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:30.464 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 7095, 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:25:30.465 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.7821278080000003, 2023-10-10T20:25:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 64, 2023-10-10T20:25:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 131, 2023-10-10T20:25:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 131, 2023-10-10T20:25:30.466 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.898337634000001 2023-10-10T20:25:30.466 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 msgr2=0x7f5e540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 0x7f5e540693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009de0 tx=0x7f5e6c019000 comp rx=0 tx=0).stop 2023-10-10T20:25:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 msgr2=0x7f5e740b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.468 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c5c0 tx=0x7f5e64007d80 comp rx=0 tx=0).stop 2023-10-10T20:25:30.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 shutdown_connections 2023-10-10T20:25:30.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5e54066f30 0x7f5e540693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e7415a620 0x7f5e740b15e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e74157870 0x7f5e74152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 --2- 172.21.15.148:0/1933348279 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e74157c40 0x7f5e74150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 >> 172.21.15.148:0/1933348279 conn(0x7f5e74068ce0 msgr2=0x7f5e740b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 shutdown_connections 2023-10-10T20:25:30.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.442+0000 7f5e78aa1700 1 -- 172.21.15.148:0/1933348279 wait complete. 2023-10-10T20:25:30.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:25:30.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.585+0000 7f466f59e700 1 Processor -- start 2023-10-10T20:25:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.585+0000 7f466f59e700 1 -- start start 2023-10-10T20:25:30.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466f59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f46700cd6e0 0x7f46700cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f467015d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466f59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46700cb7c0 con 0x7f46700cdab0 2023-10-10T20:25:30.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466f59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467015e850 con 0x7f46700cd6e0 2023-10-10T20:25:30.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466f59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46700cd030 con 0x7f467015cbd0 2023-10-10T20:25:30.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f4665d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46700cdab0 0x7f467015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57806/0 (socket says 172.21.15.148:57806) 2023-10-10T20:25:30.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f4665d9b700 1 -- 172.21.15.148:0/2925654172 learned_addr learned my addr 172.21.15.148:0/2925654172 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:30.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466e59c700 1 --2- 172.21.15.148:0/2925654172 >> v2:172.21.15.161:3300/0 conn(0x7f46700cd6e0 0x7f46700cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099449011 0 0) 0x7f46700cb7c0 con 0x7f46700cdab0 2023-10-10T20:25:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466ed9d700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f467015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f465c003570 con 0x7f46700cdab0 2023-10-10T20:25:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609755782 0 0) 0x7f465c003570 con 0x7f46700cdab0 2023-10-10T20:25:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.586+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 msgr2=0x7f467015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f467015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 >> v2:172.21.15.161:3300/0 conn(0x7f46700cd6e0 msgr2=0x7f46700cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 --2- 172.21.15.148:0/2925654172 >> v2:172.21.15.161:3300/0 conn(0x7f46700cd6e0 0x7f46700cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467015e290 con 0x7f46700cdab0 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f4658001470 con 0x7f46700cdab0 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 >> v1:172.21.15.148:6789/0 conn(0x7f46700cdab0 legacy=0x7f467015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f465c003a60 0x7f465c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f465c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f465c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467015e850 con 0x7f465c004d50 2023-10-10T20:25:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46700cd030 con 0x7f465c003a60 2023-10-10T20:25:30.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c00ba60 con 0x7f467015cbd0 2023-10-10T20:25:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f465c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466e59c700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f465c003a60 0x7f465c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f4665d9b700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f465c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 -- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 msgr2=0x7f465c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f467015cbd0 0x7f465c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 -- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f465c003a60 msgr2=0x7f465c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f465c003a60 0x7f465c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 -- 172.21.15.148:0/2925654172 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c003570 con 0x7f465c004d50 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466ed9d700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f465c0051c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f466800e210 tx=0x7f466800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f610fcabd632ab server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800ea60 con 0x7f465c004d50 2023-10-10T20:25:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466800f040 con 0x7f465c004d50 2023-10-10T20:25:30.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.587+0000 7f466dd9b700 1 -- 172.21.15.148:0/2925654172 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800ee10 con 0x7f465c004d50 2023-10-10T20:25:30.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 -- 172.21.15.148:0/2925654172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 msgr2=0x7f465c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 --2- 172.21.15.148:0/2925654172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f465c0051c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f466800e210 tx=0x7f466800b040 comp rx=0 tx=0).stop 2023-10-10T20:25:30.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466ed9d700 1 -- 172.21.15.148:0/2925654172 reap_dead start 2023-10-10T20:25:30.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 -- 172.21.15.148:0/2925654172 shutdown_connections 2023-10-10T20:25:30.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 -- 172.21.15.148:0/2925654172 >> 172.21.15.148:0/2925654172 conn(0x7f4670067660 msgr2=0x7f4670067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 -- 172.21.15.148:0/2925654172 shutdown_connections 2023-10-10T20:25:30.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.588+0000 7f466f59e700 1 -- 172.21.15.148:0/2925654172 wait complete. 2023-10-10T20:25:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.589+0000 7f466f59e700 1 Processor -- start 2023-10-10T20:25:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.589+0000 7f466f59e700 1 -- start start 2023-10-10T20:25:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46700b6610 0x7f46700b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 0x7f46700b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467016a570 con 0x7f465c004d50 2023-10-10T20:25:30.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467016b370 con 0x7f467014c990 2023-10-10T20:25:30.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466f59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467016b740 con 0x7f46700b6610 2023-10-10T20:25:30.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57878/0 (socket says 172.21.15.148:57878) 2023-10-10T20:25:30.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 -- 172.21.15.148:0/690831011 learned_addr learned my addr 172.21.15.148:0/690831011 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:30.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f4665d9b700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46700b6610 0x7f46700b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466ed9d700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 0x7f46700b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 -- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46700b6610 msgr2=0x7f46700b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46700b6610 0x7f46700b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 -- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 msgr2=0x7f46700b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 0x7f46700b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 -- 172.21.15.148:0/690831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467016a760 con 0x7f465c004d50 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466ed9d700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 0x7f46700b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:30.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.590+0000 7f466e59c700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4660006ac0 tx=0x7f46600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000f940 con 0x7f465c004d50 2023-10-10T20:25:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000faa0 con 0x7f465c004d50 2023-10-10T20:25:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467016aa40 con 0x7f465c004d50 2023-10-10T20:25:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000fe20 con 0x7f465c004d50 2023-10-10T20:25:30.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467016b000 con 0x7f465c004d50 2023-10-10T20:25:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.591+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4670089280 con 0x7f465c004d50 2023-10-10T20:25:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.592+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4660014650 con 0x7f465c004d50 2023-10-10T20:25:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.592+0000 7f4666ffd700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 0x7f46500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.592+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4660018030 con 0x7f465c004d50 2023-10-10T20:25:30.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.592+0000 7f4665d9b700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 0x7f46500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:30.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.593+0000 7f4665d9b700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 0x7f46500693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f467015e110 tx=0x7f4658001e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:30.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.597+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f466008a020 con 0x7f465c004d50 2023-10-10T20:25:30.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.822+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f467015e570 con 0x7f465c004d50 2023-10-10T20:25:30.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.822+0000 7f4666ffd700 1 -- 172.21.15.148:0/690831011 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4660022070 con 0x7f465c004d50 2023-10-10T20:25:30.824 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:25:30.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 msgr2=0x7f46500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 0x7f46500693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f467015e110 tx=0x7f4658001e90 comp rx=0 tx=0).stop 2023-10-10T20:25:30.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 msgr2=0x7f46700b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:30.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4660006ac0 tx=0x7f46600091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 shutdown_connections 2023-10-10T20:25:30.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4650066f30 0x7f46500693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f467014c990 0x7f46700b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f465c004d50 0x7f46700b60d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.824+0000 7f466f59e700 1 --2- 172.21.15.148:0/690831011 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46700b6610 0x7f46700b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:30.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.825+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 >> 172.21.15.148:0/690831011 conn(0x7f4670067660 msgr2=0x7f467014d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:30.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.825+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 shutdown_connections 2023-10-10T20:25:30.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:30.825+0000 7f466f59e700 1 -- 172.21.15.148:0/690831011 wait complete. 2023-10-10T20:25:30.838 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:25:30.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:25:30.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:25:30.839 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:25:30.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:30.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:25:30.840 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:25:30.840 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:25:30.840 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:30.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:30.841 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:25:30.857 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:25:30.857 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:25:30.858 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:25:30.858 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.858 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:25:30.858 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.858 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:30.859 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:30.859 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:25:30.859 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:30.859 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:30.859 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:25:30.860 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42180sec preferred_lft 42180sec 2023-10-10T20:25:30.861 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:25:30.861 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.861 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:25:30.861 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:30.861 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:25:30.862 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.862 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:25:30.862 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.862 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:30.862 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:25:30.863 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:25:30.863 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.863 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:30.863 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:25:30.863 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:25:30.864 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:30.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:30.864 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:25:30.920 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:25:30.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:30.921 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:25:30.977 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:25:30.977 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:25:30.978 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:25:30.978 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:25:30.978 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:25:30.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:30.979 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:25:30.979 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:25:30.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:25:30.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:25:30.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:25:30.981 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:25:31.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.077 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:25:31.092 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:25:31.092 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:25:31.093 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:31.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:25:31.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:31.158 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:31.211 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:25:31.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:31.212 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:31.266 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:25:31.267 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:25:31.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:31.346 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:31.347 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:31.365 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:31.366 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:31.366 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:31.366 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:31.366 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:31.367 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:31.367 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:31.367 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:31.387 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:31.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:31.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:31.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:31.388 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:25:31.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:25:31.390 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:25:31.391 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:25:31.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.391 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:25:31.456 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:25:31.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.610+0000 7efed4ea6700 1 Processor -- start 2023-10-10T20:25:31.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.610+0000 7efed4ea6700 1 -- start start 2023-10-10T20:25:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efed4ea6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efed00cd6e0 0x7efed00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efed4ea6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed00cb7c0 con 0x7efed015cbd0 2023-10-10T20:25:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efed4ea6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed015e290 con 0x7efed00cdab0 2023-10-10T20:25:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efed4ea6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed015e850 con 0x7efed00cd6e0 2023-10-10T20:25:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecf7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efed00cd6e0 0x7efed00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efed015cbd0 0x7efed015d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57816/0 (socket says 172.21.15.148:57816) 2023-10-10T20:25:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecffff700 1 -- 172.21.15.148:0/3811923745 learned_addr learned my addr 172.21.15.148:0/3811923745 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecf7fe700 1 -- 172.21.15.148:0/3811923745 >> v1:172.21.15.161:6789/0 conn(0x7efed00cdab0 legacy=0x7efed015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecf7fe700 1 -- 172.21.15.148:0/3811923745 >> v1:172.21.15.148:6789/0 conn(0x7efed015cbd0 legacy=0x7efed015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efecf7fe700 1 -- 172.21.15.148:0/3811923745 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed015e570 con 0x7efed00cd6e0 2023-10-10T20:25:31.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.611+0000 7efece7fc700 1 -- 172.21.15.148:0/3811923745 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980057068 0 0) 0x7efed00cb7c0 con 0x7efed015cbd0 2023-10-10T20:25:31.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efecf7fe700 1 --2- 172.21.15.148:0/3811923745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed00cbc50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e60 tx=0x7efec0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342cb72df6aa361d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:31.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efece7fc700 1 -- 172.21.15.148:0/3811923745 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000e070 con 0x7efed00cd6e0 2023-10-10T20:25:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efece7fc700 1 -- 172.21.15.148:0/3811923745 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000bc30 con 0x7efed00cd6e0 2023-10-10T20:25:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efece7fc700 1 -- 172.21.15.148:0/3811923745 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0012430 con 0x7efed00cd6e0 2023-10-10T20:25:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 -- 172.21.15.148:0/3811923745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 msgr2=0x7efed00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 --2- 172.21.15.148:0/3811923745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed00cbc50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e60 tx=0x7efec0009780 comp rx=0 tx=0).stop 2023-10-10T20:25:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 -- 172.21.15.148:0/3811923745 shutdown_connections 2023-10-10T20:25:31.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 --2- 172.21.15.148:0/3811923745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed00cbc50 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 -- 172.21.15.148:0/3811923745 >> 172.21.15.148:0/3811923745 conn(0x7efed0067660 msgr2=0x7efed0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:31.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 -- 172.21.15.148:0/3811923745 shutdown_connections 2023-10-10T20:25:31.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.612+0000 7efed4ea6700 1 -- 172.21.15.148:0/3811923745 wait complete. 2023-10-10T20:25:31.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 Processor -- start 2023-10-10T20:25:31.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 -- start start 2023-10-10T20:25:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efed00cdab0 0x7efed00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 0x7efed016a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed00c8260 con 0x7efed00cd6e0 2023-10-10T20:25:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed00b3a10 con 0x7efed015cbd0 2023-10-10T20:25:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efed4ea6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed00c8500 con 0x7efed00cdab0 2023-10-10T20:25:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efecf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efecf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57910/0 (socket says 172.21.15.148:57910) 2023-10-10T20:25:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.614+0000 7efecf7fe700 1 -- 172.21.15.148:0/2405459881 learned_addr learned my addr 172.21.15.148:0/2405459881 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecffff700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 0x7efed016a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efeceffd700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efed00cdab0 0x7efed00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 -- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efed00cdab0 msgr2=0x7efed00c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efed00cdab0 0x7efed00c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 -- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 msgr2=0x7efed016a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 0x7efed016a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 -- 172.21.15.148:0/2405459881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009a20 con 0x7efed00cd6e0 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecffff700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 0x7efed016a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efecf7fe700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efec0012580 tx=0x7efec0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0027470 con 0x7efed00cd6e0 2023-10-10T20:25:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec00275f0 con 0x7efed00cd6e0 2023-10-10T20:25:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed00c87e0 con 0x7efed00cd6e0 2023-10-10T20:25:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00279b0 con 0x7efed00cd6e0 2023-10-10T20:25:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.615+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed00c8da0 con 0x7efed00cd6e0 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.616+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed00b3b80 con 0x7efed00cd6e0 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.616+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efec0024070 con 0x7efed00cd6e0 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.617+0000 7efeb77fe700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.617+0000 7efeceffd700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.617+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efec0086c80 con 0x7efed00cd6e0 2023-10-10T20:25:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.617+0000 7efeceffd700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 0x7efebc0693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed00b05e0 tx=0x7efec4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:31.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.621+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efec0053ef0 con 0x7efed00cd6e0 2023-10-10T20:25:31.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.850+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efed00cf7c0 con 0x7efed00cd6e0 2023-10-10T20:25:31.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.850+0000 7efeb77fe700 1 -- 172.21.15.148:0/2405459881 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7efec0057510 con 0x7efed00cd6e0 2023-10-10T20:25:31.852 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:25:31.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 msgr2=0x7efebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 0x7efebc0693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed00b05e0 tx=0x7efec4009040 comp rx=0 tx=0).stop 2023-10-10T20:25:31.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 msgr2=0x7efed0142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:31.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efec0012580 tx=0x7efec0019000 comp rx=0 tx=0).stop 2023-10-10T20:25:31.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 shutdown_connections 2023-10-10T20:25:31.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed015cbd0 0x7efed016a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efed00cd6e0 0x7efed0142200 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 --2- 172.21.15.148:0/2405459881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efed00cdab0 0x7efed00c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:31.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 >> 172.21.15.148:0/2405459881 conn(0x7efed0067660 msgr2=0x7efed00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:31.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 shutdown_connections 2023-10-10T20:25:31.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:31.852+0000 7efed4ea6700 1 -- 172.21.15.148:0/2405459881 wait complete. 2023-10-10T20:25:31.866 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:25:31.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:25:31.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:25:31.867 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:25:31.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:31.868 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:25:31.868 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:25:31.868 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:25:31.869 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:31.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.869 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:25:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:25:31.888 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:25:31.888 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:25:31.888 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.889 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:25:31.889 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.889 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:31.889 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:31.889 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:25:31.890 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:31.890 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:31.890 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:25:31.890 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42176sec preferred_lft 42176sec 2023-10-10T20:25:31.891 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:25:31.892 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.892 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:25:31.892 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:25:31.892 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:25:31.892 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.893 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:25:31.893 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.893 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:31.893 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:25:31.893 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:25:31.894 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.894 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:25:31.894 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:25:31.894 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:25:31.894 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:25:31.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.895 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:25:31.954 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:25:31.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:31.956 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:25:32.014 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:25:32.014 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:25:32.015 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:25:32.015 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:25:32.015 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:25:32.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:32.016 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:25:32.016 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:25:32.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:25:32.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:25:32.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:25:32.018 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:25:32.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:32.119 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:25:32.134 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:25:32.134 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:25:32.134 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:32.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:32.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:25:32.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:32.199 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:32.253 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:25:32.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:25:32.254 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:32.307 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:25:32.308 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:25:32.308 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:32.387 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:32.388 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:32.406 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:32.406 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:32.406 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:32.407 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:32.407 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:32.407 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:32.407 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:32.407 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:32.428 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:25:32.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:25:32.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:25:32.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:25:32.429 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:25:32.429 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:25:32.429 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:25:32.429 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:25:32.429 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:25:32.430 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:25:32.430 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:25:32.431 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:25:32.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:25:32.432 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:25:32.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:32.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.610+0000 7f97bee2e700 1 Processor -- start 2023-10-10T20:25:32.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.610+0000 7f97bee2e700 1 -- start start 2023-10-10T20:25:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bee2e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97b815a620 0x7f97b815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bee2e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b815cbb0 con 0x7f97b815d580 2023-10-10T20:25:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bee2e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b815bfa0 con 0x7f97b8157870 2023-10-10T20:25:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bee2e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b815bdb0 con 0x7f97b815a620 2023-10-10T20:25:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bd3cb700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97b815d580 0x7f97b8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43388/0 (socket says 172.21.15.148:43388) 2023-10-10T20:25:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97bd3cb700 1 -- 172.21.15.148:0/3970653704 learned_addr learned my addr 172.21.15.148:0/3970653704 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721582934 0 0) 0x7f97b815cbb0 con 0x7f97b815d580 2023-10-10T20:25:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a8003570 con 0x7f97b815d580 2023-10-10T20:25:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.611+0000 7f97b7fff700 1 --2- 172.21.15.148:0/3970653704 >> v2:172.21.15.161:3300/0 conn(0x7f97b815a620 0x7f97b815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4039884967 0 0) 0x7f97a8003570 con 0x7f97b815d580 2023-10-10T20:25:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 >> v2:172.21.15.161:3300/0 conn(0x7f97b815a620 msgr2=0x7f97b815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:32.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 --2- 172.21.15.148:0/3970653704 >> v2:172.21.15.161:3300/0 conn(0x7f97b815a620 0x7f97b815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 >> v1:172.21.15.161:6789/0 conn(0x7f97b8157870 legacy=0x7f97b815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b815c280 con 0x7f97b815d580 2023-10-10T20:25:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f97b00017f0 con 0x7f97b815d580 2023-10-10T20:25:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 >> v1:172.21.15.148:6789/0 conn(0x7f97b815d580 legacy=0x7f97b8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8003a60 0x7f97a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97a8004810 0x7f97a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a800c200 con 0x7f97a80052e0 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a800c370 con 0x7f97a8003a60 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a800c530 con 0x7f97a8004810 2023-10-10T20:25:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bcbca700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8003a60 0x7f97a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b7fff700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97a8004810 0x7f97a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 -- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97a8004810 msgr2=0x7f97a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97a8004810 0x7f97a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 -- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8003a60 msgr2=0x7f97a8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a8003a60 0x7f97a8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 -- 172.21.15.148:0/3970653704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8003570 con 0x7f97a80052e0 2023-10-10T20:25:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 -- 172.21.15.148:0/3970653704 reap_dead start 2023-10-10T20:25:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97bd3cb700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97a800ba60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f97b000a900 tx=0x7f97b00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854e9247bd933373 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:32.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00070d0 con 0x7f97a80052e0 2023-10-10T20:25:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0007230 con 0x7f97a80052e0 2023-10-10T20:25:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.612+0000 7f97b77fe700 1 -- 172.21.15.148:0/3970653704 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00075d0 con 0x7f97a80052e0 2023-10-10T20:25:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 -- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 msgr2=0x7f97a800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97a800ba60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f97b000a900 tx=0x7f97b00021d0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 -- 172.21.15.148:0/3970653704 shutdown_connections 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 --2- 172.21.15.148:0/3970653704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97a800ba60 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 -- 172.21.15.148:0/3970653704 >> 172.21.15.148:0/3970653704 conn(0x7f97b8068ce0 msgr2=0x7f97b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 -- 172.21.15.148:0/3970653704 shutdown_connections 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.613+0000 7f97bee2e700 1 -- 172.21.15.148:0/3970653704 wait complete. 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.614+0000 7f97bee2e700 1 Processor -- start 2023-10-10T20:25:32.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.614+0000 7f97bee2e700 1 -- start start 2023-10-10T20:25:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 0x7f97b8159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b80cfdb0 0x7f97b80d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80d0790 con 0x7f97a80052e0 2023-10-10T20:25:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8158910 con 0x7f97b80cfdb0 2023-10-10T20:25:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bee2e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8157d20 con 0x7f97b816b640 2023-10-10T20:25:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36294/0 (socket says 172.21.15.148:36294) 2023-10-10T20:25:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 -- 172.21.15.148:0/2676058704 learned_addr learned my addr 172.21.15.148:0/2676058704 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bd3cb700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b80cfdb0 0x7f97b80d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97b7fff700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 0x7f97b8159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 -- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 msgr2=0x7f97b8159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 0x7f97b8159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 -- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b80cfdb0 msgr2=0x7f97b80d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b80cfdb0 0x7f97b80d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97bcbca700 1 -- 172.21.15.148:0/2676058704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8157fe0 con 0x7f97a80052e0 2023-10-10T20:25:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.615+0000 7f97b7fff700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 0x7f97b8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:32.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97bcbca700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f97ac002950 tx=0x7f97ac007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac006820 con 0x7f97a80052e0 2023-10-10T20:25:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac006980 con 0x7f97a80052e0 2023-10-10T20:25:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac006d40 con 0x7f97a80052e0 2023-10-10T20:25:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b81582c0 con 0x7f97a80052e0 2023-10-10T20:25:32.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.616+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b814e1e0 con 0x7f97a80052e0 2023-10-10T20:25:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.617+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b804c3c0 con 0x7f97a80052e0 2023-10-10T20:25:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.617+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97ac007240 con 0x7f97a80052e0 2023-10-10T20:25:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.617+0000 7f97b4ff9700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 0x7f979c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.617+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f97ac0845f0 con 0x7f97a80052e0 2023-10-10T20:25:32.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.617+0000 7f97b7fff700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 0x7f979c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.618+0000 7f97b7fff700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 0x7f979c0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b0b0 tx=0x7f97a400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:32.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.622+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97ac051860 con 0x7f97a80052e0 2023-10-10T20:25:32.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.851+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97b815c560 con 0x7f97a80052e0 2023-10-10T20:25:32.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.851+0000 7f97b4ff9700 1 -- 172.21.15.148:0/2676058704 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f97ac054e80 con 0x7f97a80052e0 2023-10-10T20:25:32.853 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:32.853 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:25:32.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 msgr2=0x7f979c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 0x7f979c0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b0b0 tx=0x7f97a400b040 comp rx=0 tx=0).stop 2023-10-10T20:25:32.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 msgr2=0x7f97b814dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:32.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f97ac002950 tx=0x7f97ac007d30 comp rx=0 tx=0).stop 2023-10-10T20:25:32.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 shutdown_connections 2023-10-10T20:25:32.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f979c066f30 0x7f979c0693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b80cfdb0 0x7f97b80d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97a80052e0 0x7f97b814dca0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 --2- 172.21.15.148:0/2676058704 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97b816b640 0x7f97b8159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:32.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 >> 172.21.15.148:0/2676058704 conn(0x7f97b8068ce0 msgr2=0x7f97b80b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:32.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 shutdown_connections 2023-10-10T20:25:32.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:32.853+0000 7f97bee2e700 1 -- 172.21.15.148:0/2676058704 wait complete. 2023-10-10T20:25:32.858 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:25:32.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-10T20:25:33.000 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":7095,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3301286829999999,"mdsmap_epoch":64,"osdmap_epoch":131,"osdmap_epoch_barrier":131,"uptime":13.446339826999999} 2023-10-10T20:25:33.000 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 7095, "mdsmap_epoch": 64, "osdmap_epoch": 131, "osdmap_epoch_barrier": 131, "rank_uptime": 8.330128683, "state": "up:active", "uptime": 13.446339827, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:25:33.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-10T20:25:33.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.119+0000 7fa752dbb700 1 Processor -- start 2023-10-10T20:25:33.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.119+0000 7fa752dbb700 1 -- start start 2023-10-10T20:25:33.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa752dbb700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa74c15a1b0 0x7fa74c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa752dbb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa752dbb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c15eb10 con 0x7fa74c159de0 2023-10-10T20:25:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa752dbb700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c0b3790 con 0x7fa74c15a1b0 2023-10-10T20:25:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa752dbb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c06b010 con 0x7fa74c06a650 2023-10-10T20:25:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa74c15a1b0 0x7fa74c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa74c15a1b0 0x7fa74c0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36310/0 (socket says 172.21.15.148:36310) 2023-10-10T20:25:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 -- 172.21.15.148:0/2540065723 learned_addr learned my addr 172.21.15.148:0/2540065723 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa750b57700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa74c159de0 0x7fa74c0b2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43398/0 (socket says 172.21.15.148:43398) 2023-10-10T20:25:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa751358700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 -- 172.21.15.148:0/2540065723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 msgr2=0x7fa74c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 -- 172.21.15.148:0/2540065723 >> v1:172.21.15.148:6789/0 conn(0x7fa74c159de0 legacy=0x7fa74c0b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74bfff700 1 -- 172.21.15.148:0/2540065723 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c15e680 con 0x7fa74c15a1b0 2023-10-10T20:25:33.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa74b7fe700 1 -- 172.21.15.148:0/2540065723 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693134837 0 0) 0x7fa73c001110 con 0x7fa74c159de0 2023-10-10T20:25:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.120+0000 7fa751358700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa74bfff700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0b14e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa7400099e0 tx=0x7fa740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ea78abdd8852ce server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa74b7fe700 1 -- 172.21.15.148:0/2540065723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000b820 con 0x7fa74c15a1b0 2023-10-10T20:25:33.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa74b7fe700 1 -- 172.21.15.148:0/2540065723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000b980 con 0x7fa74c15a1b0 2023-10-10T20:25:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa74b7fe700 1 -- 172.21.15.148:0/2540065723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000bd20 con 0x7fa74c15a1b0 2023-10-10T20:25:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 -- 172.21.15.148:0/2540065723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c15a1b0 msgr2=0x7fa74c0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0b14e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa7400099e0 tx=0x7fa740009250 comp rx=0 tx=0).stop 2023-10-10T20:25:33.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 -- 172.21.15.148:0/2540065723 shutdown_connections 2023-10-10T20:25:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 --2- 172.21.15.148:0/2540065723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0b14e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 -- 172.21.15.148:0/2540065723 >> 172.21.15.148:0/2540065723 conn(0x7fa74c068d60 msgr2=0x7fa74c069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 -- 172.21.15.148:0/2540065723 shutdown_connections 2023-10-10T20:25:33.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.121+0000 7fa752dbb700 1 -- 172.21.15.148:0/2540065723 wait complete. 2023-10-10T20:25:33.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.122+0000 7fa752dbb700 1 Processor -- start 2023-10-10T20:25:33.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 -- start start 2023-10-10T20:25:33.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0c8640 con 0x7fa74c159de0 2023-10-10T20:25:33.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0cffc0 con 0x7fa74c15a1b0 2023-10-10T20:25:33.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa752dbb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0cae60 con 0x7fa74c06a650 2023-10-10T20:25:33.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa74bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36326/0 (socket says 172.21.15.148:36326) 2023-10-10T20:25:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.123+0000 7fa74bfff700 1 -- 172.21.15.148:0/1051735601 learned_addr learned my addr 172.21.15.148:0/1051735601 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa750b57700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa751358700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 -- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 msgr2=0x7fa74c146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 -- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 msgr2=0x7fa74c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 -- 172.21.15.148:0/1051735601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400094f0 con 0x7fa74c159de0 2023-10-10T20:25:33.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa751358700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:33.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa74bfff700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa74c072050 tx=0x7fa740017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:33.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740011d80 con 0x7fa74c159de0 2023-10-10T20:25:33.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740026450 con 0x7fa74c159de0 2023-10-10T20:25:33.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0ca250 con 0x7fa74c159de0 2023-10-10T20:25:33.139 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740026810 con 0x7fa74c159de0 2023-10-10T20:25:33.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.124+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c0ca810 con 0x7fa74c159de0 2023-10-10T20:25:33.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.125+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c0cafa0 con 0x7fa74c159de0 2023-10-10T20:25:33.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.125+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa74001a420 con 0x7fa74c159de0 2023-10-10T20:25:33.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.126+0000 7fa748ff9700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 0x7fa7380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:33.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.126+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa740086490 con 0x7fa74c159de0 2023-10-10T20:25:33.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.126+0000 7fa750b57700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 0x7fa7380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:33.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.126+0000 7fa750b57700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 0x7fa7380693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007fd0 tx=0x7fa73c007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:33.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.130+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa74c0cafa0 con 0x7fa74c159de0 2023-10-10T20:25:33.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.314+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fa74c0cafa0 con 0x7fa74c159de0 2023-10-10T20:25:33.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.683+0000 7fa748ff9700 1 -- 172.21.15.148:0/1051735601 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v132) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa74c0cafa0 con 0x7fa74c159de0 2023-10-10T20:25:33.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 msgr2=0x7fa7380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 0x7fa7380693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa73c007fd0 tx=0x7fa73c007b90 comp rx=0 tx=0).stop 2023-10-10T20:25:33.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 msgr2=0x7fa74c0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:33.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa74c072050 tx=0x7fa740017000 comp rx=0 tx=0).stop 2023-10-10T20:25:33.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 shutdown_connections 2023-10-10T20:25:33.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa738066f30 0x7fa7380693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa74c15a1b0 0x7fa74c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa74c06a650 0x7fa74c146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 --2- 172.21.15.148:0/1051735601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa74c159de0 0x7fa74c0c8100 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 >> 172.21.15.148:0/1051735601 conn(0x7fa74c068d60 msgr2=0x7fa74c0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 shutdown_connections 2023-10-10T20:25:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:33.686+0000 7fa752dbb700 1 -- 172.21.15.148:0/1051735601 wait complete. 2023-10-10T20:25:33.693 INFO:teuthology.orchestra.run.smithi148.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-10T20:25:33.701 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:25:33.833 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:25:33.833 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:25:33.834 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-10T20:25:33.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:33.834 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-10T20:25:33.946 INFO:teuthology.orchestra.run.smithi148.stderr:16+0 records in 2023-10-10T20:25:33.947 INFO:teuthology.orchestra.run.smithi148.stderr:16+0 records out 2023-10-10T20:25:33.947 INFO:teuthology.orchestra.run.smithi148.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.09244 s, 181 MB/s 2023-10-10T20:25:33.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:33.948 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-10T20:25:34.071 INFO:teuthology.orchestra.run.smithi148.stderr:17+0 records in 2023-10-10T20:25:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:17+0 records out 2023-10-10T20:25:34.072 INFO:teuthology.orchestra.run.smithi148.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.105274 s, 169 MB/s 2023-10-10T20:25:34.072 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-10T20:25:34.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.185+0000 7fee3a241700 1 Processor -- start 2023-10-10T20:25:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.185+0000 7fee3a241700 1 -- start start 2023-10-10T20:25:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3a241700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fee3415cbd0 0x7fee3415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3a241700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee340cb7c0 con 0x7fee340cd6e0 2023-10-10T20:25:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3a241700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3415e290 con 0x7fee340cdab0 2023-10-10T20:25:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3a241700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3415e850 con 0x7fee3415cbd0 2023-10-10T20:25:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3923f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee340cd6e0 0x7fee340cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43406/0 (socket says 172.21.15.148:43406) 2023-10-10T20:25:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee3923f700 1 -- 172.21.15.148:0/2502786016 learned_addr learned my addr 172.21.15.148:0/2502786016 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:34.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee39a40700 1 --2- 172.21.15.148:0/2502786016 >> v2:172.21.15.161:3300/0 conn(0x7fee3415cbd0 0x7fee3415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816044135 0 0) 0x7fee340cb7c0 con 0x7fee340cd6e0 2023-10-10T20:25:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.186+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003570 con 0x7fee340cd6e0 2023-10-10T20:25:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166815237 0 0) 0x7fee20003570 con 0x7fee340cd6e0 2023-10-10T20:25:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee39a40700 1 -- 172.21.15.148:0/2502786016 >> v1:172.21.15.161:6789/0 conn(0x7fee340cdab0 legacy=0x7fee3415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee39a40700 1 -- 172.21.15.148:0/2502786016 >> v1:172.21.15.148:6789/0 conn(0x7fee340cd6e0 legacy=0x7fee340cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee39a40700 1 -- 172.21.15.148:0/2502786016 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415e570 con 0x7fee3415cbd0 2023-10-10T20:25:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee39a40700 1 --2- 172.21.15.148:0/2502786016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee3415d050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009e60 tx=0x7fee2c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75161a44a7d665a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00e070 con 0x7fee3415cbd0 2023-10-10T20:25:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00b850 con 0x7fee3415cbd0 2023-10-10T20:25:34.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee2bfff700 1 -- 172.21.15.148:0/2502786016 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30001470 con 0x7fee3415cbd0 2023-10-10T20:25:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee3a241700 1 -- 172.21.15.148:0/2502786016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 msgr2=0x7fee3415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.187+0000 7fee3a241700 1 --2- 172.21.15.148:0/2502786016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee3415d050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009e60 tx=0x7fee2c009780 comp rx=0 tx=0).stop 2023-10-10T20:25:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.188+0000 7fee3a241700 1 -- 172.21.15.148:0/2502786016 shutdown_connections 2023-10-10T20:25:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.188+0000 7fee3a241700 1 --2- 172.21.15.148:0/2502786016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee3415d050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.188+0000 7fee3a241700 1 -- 172.21.15.148:0/2502786016 >> 172.21.15.148:0/2502786016 conn(0x7fee34067660 msgr2=0x7fee34067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.188+0000 7fee3a241700 1 -- 172.21.15.148:0/2502786016 shutdown_connections 2023-10-10T20:25:34.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.188+0000 7fee3a241700 1 -- 172.21.15.148:0/2502786016 wait complete. 2023-10-10T20:25:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.189+0000 7fee3a241700 1 Processor -- start 2023-10-10T20:25:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.189+0000 7fee3a241700 1 -- start start 2023-10-10T20:25:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:34.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee340cdab0 0x7fee3416baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee340b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340cc360 con 0x7fee340cd6e0 2023-10-10T20:25:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34139ef0 con 0x7fee3415cbd0 2023-10-10T20:25:34.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b1150 con 0x7fee340cdab0 2023-10-10T20:25:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36336/0 (socket says 172.21.15.148:36336) 2023-10-10T20:25:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 -- 172.21.15.148:0/2544601276 learned_addr learned my addr 172.21.15.148:0/2544601276 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:34.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee39a40700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee340b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee38a3e700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee340cdab0 0x7fee3416baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 -- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee340cdab0 msgr2=0x7fee3416baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee340cdab0 0x7fee3416baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 -- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 msgr2=0x7fee340b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee340b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 -- 172.21.15.148:0/2544601276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c009a20 con 0x7fee340cd6e0 2023-10-10T20:25:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3923f700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c7f0 tx=0x7fee30001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30007360 con 0x7fee340cd6e0 2023-10-10T20:25:34.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300074e0 con 0x7fee340cd6e0 2023-10-10T20:25:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee340b1cd0 con 0x7fee340cd6e0 2023-10-10T20:25:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30004830 con 0x7fee340cd6e0 2023-10-10T20:25:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.190+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee340b1960 con 0x7fee340cd6e0 2023-10-10T20:25:34.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.191+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34089280 con 0x7fee340cd6e0 2023-10-10T20:25:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.192+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee30004c00 con 0x7fee340cd6e0 2023-10-10T20:25:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.192+0000 7fee297fa700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 0x7fee180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.192+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee30083b00 con 0x7fee340cd6e0 2023-10-10T20:25:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.192+0000 7fee38a3e700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 0x7fee180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:34.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.192+0000 7fee38a3e700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 0x7fee180693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fee24003a80 tx=0x7fee24007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:34.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.196+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee30050df0 con 0x7fee340cd6e0 2023-10-10T20:25:34.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.381+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fee340c5ab0 con 0x7fee340cd6e0 2023-10-10T20:25:34.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.382+0000 7fee297fa700 1 -- 172.21.15.148:0/2544601276 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+32330 (secure 0 0 0) 0x7fee30018020 con 0x7fee340cd6e0 2023-10-10T20:25:34.384 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:34.384 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:34.385 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 132, 2023-10-10T20:25:34.385 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:34.385 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:34.385 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:33.677368+0000", 2023-10-10T20:25:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:34.387 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:34.388 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:34.388 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:34.388 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:34.388 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:34.389 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:34.389 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:34.389 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:34.389 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:34.389 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:34.390 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:34.390 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:34.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.390 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:34.390 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:34.391 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:34.391 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:34.391 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:34.391 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:34.392 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:34.392 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:34.392 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:34.392 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:34.393 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:34.393 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:34.393 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:34.393 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:34.394 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:34.394 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:34.394 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:34.394 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:34.394 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:34.395 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:34.395 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:34.395 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:34.395 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:34.395 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:34.396 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:34.396 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:34.396 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:34.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.396 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:34.397 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:34.397 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:34.397 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:34.397 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:34.397 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:34.398 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:34.398 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:34.398 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:34.398 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:34.399 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:34.400 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:34.400 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:34.400 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:34.400 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:34.400 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:34.401 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:34.401 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:34.401 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:34.401 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:34.401 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:34.402 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:34.402 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:34.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.402 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:34.403 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:34.403 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:34.403 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:34.403 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:34.403 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:34.404 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:34.404 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:34.404 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:34.404 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:34.404 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:34.405 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:34.405 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:34.405 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:34.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.406 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:34.406 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:34.406 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:34.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:34.407 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:34.407 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:34.407 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:34.408 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:34.408 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:34.408 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:34.408 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:34.408 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:34.409 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:34.409 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:34.409 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:34.409 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:34.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:34.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:34.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:34.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:34.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:34.411 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:34.411 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:34.411 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:34.411 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:34.412 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:34.412 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:34.412 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:34.412 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:34.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.413 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:34.413 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:34.413 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:34.413 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:34.413 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:34.414 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:34.414 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:34.414 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:34.414 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:34.415 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:34.415 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:34.415 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:34.415 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:34.416 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:34.416 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:34.416 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:34.416 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:34.417 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:34.417 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:34.417 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:34.417 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:34.417 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:34.418 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:34.418 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:34.418 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:34.418 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:34.419 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:34.419 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.419 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:34.419 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:34.420 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:34.420 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:34.420 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:34.420 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:34.421 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:34.421 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:34.421 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:34.421 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:34.421 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:34.424 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:34.425 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:34.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:34.425 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:34.425 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:34.425 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:34.426 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:34.426 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:34.426 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:34.426 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:34.427 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:34.427 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:34.427 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:34.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:34.428 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:34.428 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:34.428 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:34.428 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:34.428 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:34.429 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:34.429 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:34.429 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:34.429 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:34.430 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:34.430 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:34.430 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:34.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.430 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "132", 2023-10-10T20:25:34.431 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:34.431 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:34.431 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:34.431 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:34.431 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:34.432 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:34.432 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:34.432 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:34.432 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:34.433 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:34.433 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:34.433 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:34.433 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:34.433 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:34.434 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:34.434 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:34.434 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:34.434 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:34.435 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:34.435 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:34.435 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:34.435 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:34.435 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:34.436 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:34.436 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:34.436 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:34.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:34.437 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:34.438 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:34.439 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:34.439 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:34.439 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:34.439 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:34.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.441 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:34.441 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:34.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.442 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:34.442 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.442 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.442 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.442 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.443 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.443 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.443 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.443 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.443 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.445 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.445 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:34.445 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.445 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.446 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:34.446 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.446 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.447 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.447 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.447 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.448 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.448 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:34.448 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.448 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.449 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.449 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:34.449 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.449 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.450 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.450 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.450 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.450 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.451 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.451 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.451 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:34.451 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.451 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.452 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:34.452 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.452 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.453 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.453 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.453 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.453 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.454 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.454 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:34.454 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.455 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.455 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:34.455 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:34.455 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.456 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.456 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:34.456 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:34.457 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:34.457 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:34.457 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.457 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.457 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.458 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.458 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:34.459 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.459 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.459 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.459 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.459 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.460 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.460 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.460 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.460 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.461 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.461 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.461 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.461 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.462 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.463 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.463 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:34.463 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.463 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.464 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.464 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.464 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.464 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.465 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.467 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.467 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.468 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:34.468 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.468 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.469 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.469 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:34.469 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.469 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.470 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.470 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.471 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:34.471 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.471 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:34.473 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.474 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.474 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.474 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.475 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:34.475 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.475 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.475 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.476 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.476 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.476 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.476 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.476 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.477 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.477 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.477 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.477 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.477 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.478 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.478 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:34.478 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.479 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.479 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:34.479 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.479 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.480 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.480 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.480 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.480 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.481 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.481 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:34.481 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.481 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.481 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.484 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.484 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.484 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:34.484 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.485 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.485 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:34.486 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.486 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.486 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.487 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.487 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:34.488 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.488 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:34.489 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:34.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.489 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.489 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:34.490 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:34.490 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:34.490 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:34.490 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.491 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.491 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.492 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:34.492 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.492 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.492 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.493 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.493 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.493 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.493 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.493 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.494 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.494 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.494 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.494 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.494 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.495 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.495 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:34.495 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.496 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.496 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:34.496 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.497 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.497 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.497 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.498 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.498 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:34.498 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.499 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:34.499 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.501 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:34.501 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.502 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.502 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:34.502 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.504 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:34.504 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.505 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.505 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:34.505 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:34.505 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.506 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.506 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:34.506 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.508 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.508 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:34.508 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.510 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.510 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.510 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.510 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.512 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:34.512 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.512 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.512 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.513 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:34.513 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.513 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.514 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:34.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:34.516 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.517 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:34.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.518 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.518 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:34.519 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.519 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:34.521 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.521 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.521 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.521 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:34.522 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:34.522 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.522 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.523 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:34.523 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:34.523 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:34.523 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:34.523 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.524 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.524 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.524 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.525 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:34.525 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.525 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.525 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.525 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.526 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.526 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.526 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:34.528 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:34.529 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.529 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:34.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.532 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.532 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.533 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:34.534 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.535 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:34.535 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.536 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.536 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.536 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.536 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.537 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:34.537 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:34.538 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:34.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.538 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:34.539 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:34.540 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.540 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.540 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.540 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:34.541 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.541 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.541 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.542 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.542 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.543 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.543 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.543 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.543 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.543 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:34.544 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.545 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.545 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:34.545 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.546 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.546 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:34.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:34.548 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.549 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.549 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.550 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.550 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:34.550 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:34.551 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.552 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.552 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.552 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.553 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.553 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:34.553 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.554 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:34.554 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:34.555 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.555 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.555 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:34.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:34.556 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:34.556 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:34.556 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.556 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.557 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.557 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.557 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:34.558 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:34.558 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:34.558 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:34.558 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:34.558 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:34.559 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:34.559 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:34.559 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:34.559 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:34.560 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:34.560 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:34.560 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:34.560 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:34.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:34.562 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.562 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.563 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.563 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:34.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.565 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.565 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:34.565 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.565 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.565 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.566 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:34.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.567 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.567 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:34.567 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:34.568 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.568 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.568 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.569 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:34.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:34.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:34.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:34.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.571 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:34.571 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:34.571 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:34.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.572 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:34.572 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:34.572 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:34.572 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:34.573 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:34.573 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:34.573 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:34.573 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:34.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.574 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:34.574 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:34.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.574 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:34.575 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.575 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.575 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.575 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.575 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.576 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:34.576 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:34.577 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.577 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.577 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.577 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.577 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.578 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:34.578 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.579 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:34.579 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.579 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.579 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.579 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.580 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.580 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:34.580 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.581 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:34.581 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.581 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.582 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.582 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.582 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:34.582 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.583 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:34.583 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.583 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:34.585 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.585 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:34.585 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.586 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.586 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.586 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.586 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.586 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:34.587 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.587 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:34.587 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.588 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:34.589 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:34.589 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:34.590 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:34.590 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:34.590 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:34.590 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:34.590 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:34.591 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:34.591 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:34.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.591 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:34.592 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:34.592 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:34.592 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:34.592 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:34.592 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:34.593 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:34.593 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:34.593 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:34.593 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:34.593 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:34.594 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:34.594 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:34.594 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:34.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:34.595 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:34.595 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:34.595 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:34.595 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:34.596 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:34.596 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:34.596 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:34.596 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:34.596 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:34.597 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:34.597 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:34.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:34.597 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 msgr2=0x7fee180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 0x7fee180693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fee24003a80 tx=0x7fee24007ec0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 msgr2=0x7fee340c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:34.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c7f0 tx=0x7fee30001a00 comp rx=0 tx=0).stop 2023-10-10T20:25:34.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 shutdown_connections 2023-10-10T20:25:34.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee18066f30 0x7fee180693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee3415cbd0 0x7fee340b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340cd6e0 0x7fee340c7eb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 --2- 172.21.15.148:0/2544601276 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee340cdab0 0x7fee3416baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:34.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.384+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 >> 172.21.15.148:0/2544601276 conn(0x7fee34067660 msgr2=0x7fee341577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:34.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.385+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 shutdown_connections 2023-10-10T20:25:34.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:34.385+0000 7fee3a241700 1 -- 172.21.15.148:0/2544601276 wait complete. 2023-10-10T20:25:34.602 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:25:39.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:39.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.730+0000 7f2cbaa0d700 1 Processor -- start 2023-10-10T20:25:39.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.730+0000 7f2cbaa0d700 1 -- start start 2023-10-10T20:25:39.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cbaa0d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2cb40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cbaa0d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2cb406ac30 0x7f2cb415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cbaa0d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb415c360 con 0x7f2cb4157ba0 2023-10-10T20:25:39.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cbaa0d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb415b790 con 0x7f2cb406ac30 2023-10-10T20:25:39.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cbaa0d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415ba30 con 0x7f2cb41577d0 2023-10-10T20:25:39.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cb37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2cb4157ba0 0x7f2cb406a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43412/0 (socket says 172.21.15.148:43412) 2023-10-10T20:25:39.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.731+0000 7f2cb37fe700 1 -- 172.21.15.148:0/564940814 learned_addr learned my addr 172.21.15.148:0/564940814 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:39.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/564940814 >> v2:172.21.15.161:3300/0 conn(0x7f2cb406ac30 0x7f2cb415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2cb40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719981099 0 0) 0x7f2cb415c360 con 0x7f2cb4157ba0 2023-10-10T20:25:39.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c9c003570 con 0x7f2cb4157ba0 2023-10-10T20:25:39.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3981141098 0 0) 0x7f2c9c003570 con 0x7f2cb4157ba0 2023-10-10T20:25:39.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 msgr2=0x7f2cb40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:39.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2cb40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 >> v2:172.21.15.161:3300/0 conn(0x7f2cb406ac30 msgr2=0x7f2cb415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:39.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 --2- 172.21.15.148:0/564940814 >> v2:172.21.15.161:3300/0 conn(0x7f2cb406ac30 0x7f2cb415c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb415bd10 con 0x7f2cb4157ba0 2023-10-10T20:25:39.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f2ca8001470 con 0x7f2cb4157ba0 2023-10-10T20:25:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 >> v1:172.21.15.148:6789/0 conn(0x7f2cb4157ba0 legacy=0x7f2cb406a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2c9c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c003fe0 0x7f2c9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c9c004d50 0x7f2c9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415b790 con 0x7f2c9c004d50 2023-10-10T20:25:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415ba30 con 0x7f2c9c003fe0 2023-10-10T20:25:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c00ba90 con 0x7f2cb41577d0 2023-10-10T20:25:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c9c004d50 0x7f2c9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2c9c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb37fe700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c003fe0 0x7f2c9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.732+0000 7f2cb8faa700 1 -- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 msgr2=0x7f2c9c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:39.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2c9c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 -- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c003fe0 msgr2=0x7f2c9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:39.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c9c003fe0 0x7f2c9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 -- 172.21.15.148:0/564940814 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c003570 con 0x7f2c9c004d50 2023-10-10T20:25:39.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41577d0 0x7f2c9c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:39.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c9c004d50 0x7f2c9c0051c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406b030 tx=0x7f2cac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cbc8b9d2a04363 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:39.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00ea30 con 0x7f2c9c004d50 2023-10-10T20:25:39.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00f040 con 0x7f2c9c004d50 2023-10-10T20:25:39.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb2ffd700 1 -- 172.21.15.148:0/564940814 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00ede0 con 0x7f2c9c004d50 2023-10-10T20:25:39.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/564940814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c9c004d50 msgr2=0x7f2c9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:39.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/564940814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c9c004d50 0x7f2c9c0051c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2cb406b030 tx=0x7f2cac00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:39.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cb8faa700 1 -- 172.21.15.148:0/564940814 reap_dead start 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/564940814 shutdown_connections 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/564940814 >> 172.21.15.148:0/564940814 conn(0x7f2cb4068ce0 msgr2=0x7f2cb40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/564940814 shutdown_connections 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.733+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/564940814 wait complete. 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.735+0000 7f2cbaa0d700 1 Processor -- start 2023-10-10T20:25:39.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.735+0000 7f2cbaa0d700 1 -- start start 2023-10-10T20:25:39.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.735+0000 7f2cbaa0d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.735+0000 7f2cbaa0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb414f130 0x7f2cb4154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 0x7f2cb416d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb4154b50 con 0x7f2cb414cd40 2023-10-10T20:25:39.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415e120 con 0x7f2cb414f130 2023-10-10T20:25:39.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415d510 con 0x7f2cb41546b0 2023-10-10T20:25:39.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36360/0 (socket says 172.21.15.148:36360) 2023-10-10T20:25:39.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 -- 172.21.15.148:0/4258774553 learned_addr learned my addr 172.21.15.148:0/4258774553 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:39.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb37fe700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb414f130 0x7f2cb4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 0x7f2cb416d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 -- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 msgr2=0x7f2cb416d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 0x7f2cb416d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 -- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb414f130 msgr2=0x7f2cb4154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb414f130 0x7f2cb4154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:39.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 -- 172.21.15.148:0/4258774553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb415d7f0 con 0x7f2cb414cd40 2023-10-10T20:25:39.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb8faa700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 0x7f2cb416d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:39.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cb3fff700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011d70 tx=0x7f2ca40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:39.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4011e70 con 0x7f2cb414cd40 2023-10-10T20:25:39.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400fc70 con 0x7f2cb414cd40 2023-10-10T20:25:39.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4014440 con 0x7f2cb414cd40 2023-10-10T20:25:39.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb415dad0 con 0x7f2cb414cd40 2023-10-10T20:25:39.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.736+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb415e5b0 con 0x7f2cb414cd40 2023-10-10T20:25:39.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.737+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb404c3c0 con 0x7f2cb414cd40 2023-10-10T20:25:39.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.738+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ca4014870 con 0x7f2cb414cd40 2023-10-10T20:25:39.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.738+0000 7f2c9bfff700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.738+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ca4018070 con 0x7f2cb414cd40 2023-10-10T20:25:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.738+0000 7f2cb37fe700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.738+0000 7f2cb37fe700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8001010 tx=0x7f2ca8007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.742+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ca4052390 con 0x7f2cb414cd40 2023-10-10T20:25:39.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.925+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2cb415bff0 con 0x7f2cb414cd40 2023-10-10T20:25:39.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.926+0000 7f2c9bfff700 1 -- 172.21.15.148:0/4258774553 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f2ca40559b0 con 0x7f2cb414cd40 2023-10-10T20:25:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:39.929 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 132, 2023-10-10T20:25:39.929 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:33.677368+0000", 2023-10-10T20:25:39.930 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:39.931 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:39.932 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:39.933 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:39.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:39.935 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:39.936 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:39.937 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:39.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:39.939 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:39.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:39.942 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:39.943 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:39.944 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:39.946 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:39.947 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:39.948 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:39.949 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:39.950 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:39.951 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:39.952 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:39.953 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:39.954 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:39.955 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:39.956 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:39.957 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:39.958 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:39.959 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:39.960 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.961 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:39.962 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:39.963 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:39.964 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:39.965 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:39.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:39.967 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:39.968 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:39.969 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:39.970 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:39.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "132", 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:39.972 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:39.973 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:39.974 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:39.975 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:39.976 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:39.977 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:39.978 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:39.979 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:39.982 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:39.983 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:39.984 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:39.985 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:39.987 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.989 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:39.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.991 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.992 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.993 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:39.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.996 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:39.996 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:39.996 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:39.996 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:39.996 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:39.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:39.998 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:39.998 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:39.998 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:39.998 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:39.998 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:39.999 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:39.999 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:39.999 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:39.999 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:39.999 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.001 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.002 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:40.002 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.002 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.002 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.003 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.004 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.005 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:40.006 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:40.007 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.008 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:40.009 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:40.010 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:40.011 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.012 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.014 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.015 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.016 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.017 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.018 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:40.019 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.020 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.023 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.024 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.025 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.026 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:40.027 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.029 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.030 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.031 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.032 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.033 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:40.034 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:40.035 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.040 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:40.042 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.045 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.046 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.050 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.051 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.057 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:40.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.060 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.061 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.063 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.064 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.067 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:40.068 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.069 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:40.072 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:40.073 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.074 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.075 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.076 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.077 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:40.079 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.081 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.082 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.084 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.085 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.087 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.088 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.089 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:40.091 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:40.092 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.095 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:40.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:40.097 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.100 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:40.101 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:40.102 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.103 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:40.104 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:40.105 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.106 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.107 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.108 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.109 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.110 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.111 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.112 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:40.113 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.114 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.115 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.116 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.116 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.116 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.116 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:40.116 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.117 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:40.118 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:40.121 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:40.122 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:40.123 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:40.124 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:40.125 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:40.126 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:40.127 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:40.128 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 msgr2=0x7f2ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8001010 tx=0x7f2ca8007940 comp rx=0 tx=0).stop 2023-10-10T20:25:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 msgr2=0x7f2cb415cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011d70 tx=0x7f2ca40091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 shutdown_connections 2023-10-10T20:25:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cb41546b0 0x7f2cb416d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb414cd40 0x7f2cb415cc10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 --2- 172.21.15.148:0/4258774553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb414f130 0x7f2cb4154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 >> 172.21.15.148:0/4258774553 conn(0x7f2cb4068ce0 msgr2=0x7f2cb414d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 shutdown_connections 2023-10-10T20:25:40.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:39.928+0000 7f2cbaa0d700 1 -- 172.21.15.148:0/4258774553 wait complete. 2023-10-10T20:25:40.132 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:25:45.134 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:45.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.254+0000 7fd307cbe700 1 Processor -- start 2023-10-10T20:25:45.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 -- start start 2023-10-10T20:25:45.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 0x7fd30015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30015e280 con 0x7fd300070700 2023-10-10T20:25:45.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000ca0b0 con 0x7fd3000c9af0 2023-10-10T20:25:45.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.255+0000 7fd307cbe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd300068a10 con 0x7fd3001510f0 2023-10-10T20:25:45.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305259700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 0x7fd30015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:36978/0 (socket says 172.21.15.148:36978) 2023-10-10T20:25:45.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 -- 172.21.15.148:0/2123270488 learned_addr learned my addr 172.21.15.148:0/2123270488 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:45.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd30625b700 1 --1- 172.21.15.148:0/2123270488 >> v1:172.21.15.161:6789/0 conn(0x7fd300070700 0x7fd3000afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:55498/0 (socket says 172.21.15.148:55498) 2023-10-10T20:25:45.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 -- 172.21.15.148:0/2123270488 >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 msgr2=0x7fd30015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:45.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 --2- 172.21.15.148:0/2123270488 >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 0x7fd30015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 -- 172.21.15.148:0/2123270488 >> v1:172.21.15.161:6789/0 conn(0x7fd300070700 legacy=0x7fd3000afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 -- 172.21.15.148:0/2123270488 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300068cf0 con 0x7fd3001510f0 2023-10-10T20:25:45.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd304a58700 1 -- 172.21.15.148:0/2123270488 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746371374 0 0) 0x7fd30015e280 con 0x7fd300070700 2023-10-10T20:25:45.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305259700 1 --2- 172.21.15.148:0/2123270488 >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 0x7fd30015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:45.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd305a5a700 1 --2- 172.21.15.148:0/2123270488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009f20 tx=0x7fd2fc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ec0035da5572d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:45.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.256+0000 7fd304a58700 1 -- 172.21.15.148:0/2123270488 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00e070 con 0x7fd3001510f0 2023-10-10T20:25:45.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd304a58700 1 -- 172.21.15.148:0/2123270488 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00b9d0 con 0x7fd3001510f0 2023-10-10T20:25:45.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd304a58700 1 -- 172.21.15.148:0/2123270488 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0092e0 con 0x7fd3001510f0 2023-10-10T20:25:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 -- 172.21.15.148:0/2123270488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 msgr2=0x7fd30006ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 --2- 172.21.15.148:0/2123270488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009f20 tx=0x7fd2fc009840 comp rx=0 tx=0).stop 2023-10-10T20:25:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 -- 172.21.15.148:0/2123270488 shutdown_connections 2023-10-10T20:25:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 --2- 172.21.15.148:0/2123270488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3001510f0 0x7fd30006ecc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 --2- 172.21.15.148:0/2123270488 >> v2:172.21.15.161:3300/0 conn(0x7fd3000c9af0 0x7fd30015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 -- 172.21.15.148:0/2123270488 >> 172.21.15.148:0/2123270488 conn(0x7fd3000ca4f0 msgr2=0x7fd3000ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 -- 172.21.15.148:0/2123270488 shutdown_connections 2023-10-10T20:25:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.257+0000 7fd307cbe700 1 -- 172.21.15.148:0/2123270488 wait complete. 2023-10-10T20:25:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.258+0000 7fd307cbe700 1 Processor -- start 2023-10-10T20:25:45.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.258+0000 7fd307cbe700 1 -- start start 2023-10-10T20:25:45.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3000c9af0 0x7fd30016af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 0x7fd3000b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b38e0 con 0x7fd300070700 2023-10-10T20:25:45.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b1790 con 0x7fd3001510f0 2023-10-10T20:25:45.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd307cbe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30016aad0 con 0x7fd3000c9af0 2023-10-10T20:25:45.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40508/0 (socket says 172.21.15.148:40508) 2023-10-10T20:25:45.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 -- 172.21.15.148:0/1987774154 learned_addr learned my addr 172.21.15.148:0/1987774154 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:45.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd30625b700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 0x7fd3000b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305259700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3000c9af0 0x7fd30016af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 -- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3000c9af0 msgr2=0x7fd30016af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3000c9af0 0x7fd30016af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 -- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 msgr2=0x7fd3000b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 0x7fd3000b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd305a5a700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0009420 con 0x7fd300070700 2023-10-10T20:25:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.259+0000 7fd30625b700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 0x7fd3000b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd305a5a700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b3a80 tx=0x7fd2fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0079c0 con 0x7fd300070700 2023-10-10T20:25:45.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc007b20 con 0x7fd300070700 2023-10-10T20:25:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc009ae0 con 0x7fd300070700 2023-10-10T20:25:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc01d420 con 0x7fd300070700 2023-10-10T20:25:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.260+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd300157ab0 con 0x7fd300070700 2023-10-10T20:25:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.261+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30004c3c0 con 0x7fd300070700 2023-10-10T20:25:45.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.261+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2fc003040 con 0x7fd300070700 2023-10-10T20:25:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.261+0000 7fd2f5ffb700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 0x7fd2e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.262+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2fc086160 con 0x7fd300070700 2023-10-10T20:25:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.262+0000 7fd305259700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 0x7fd2e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.262+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7fd2e406ea50 con 0x7fd300070700 2023-10-10T20:25:45.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.262+0000 7fd305259700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 0x7fd2e40693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b2a90 tx=0x7fd2f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:45.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.266+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2fc0533d0 con 0x7fd300070700 2023-10-10T20:25:45.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.448+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd30016adb0 con 0x7fd300070700 2023-10-10T20:25:45.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.449+0000 7fd2f5ffb700 1 -- 172.21.15.148:0/1987774154 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+32349 (secure 0 0 0) 0x7fd2fc0569f0 con 0x7fd300070700 2023-10-10T20:25:45.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:45.452 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:45.452 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 133, 2023-10-10T20:25:45.452 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:45.453 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:45.453 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:40.529706+0000", 2023-10-10T20:25:45.453 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:45.453 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:45.453 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:45.454 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:45.455 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:45.455 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:45.455 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:45.455 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:45.456 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:45.457 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:45.457 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:45.457 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:45.457 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:45.458 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:45.458 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:45.458 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:45.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:45.459 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:45.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:45.460 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:45.460 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:45.460 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:45.460 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:45.460 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:45.461 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:45.461 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:45.461 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:45.461 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:45.461 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:45.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:45.462 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:45.462 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:45.462 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:45.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:45.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:45.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:45.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:45.463 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:45.464 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:45.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:45.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:45.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:45.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:45.466 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:45.466 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:45.466 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:45.466 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:45.466 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:45.467 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:45.468 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:45.468 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:45.468 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:45.469 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:45.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:45.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:45.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:45.470 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:45.470 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:45.470 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:45.471 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:45.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:45.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:45.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:45.472 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:45.473 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:45.473 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:45.473 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:45.474 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:45.474 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:45.474 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:45.475 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:45.475 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:45.475 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:45.476 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:45.476 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:45.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.477 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:45.477 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:45.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.478 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.478 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.479 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:45.479 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:45.479 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:45.479 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:45.480 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:45.480 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:45.480 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:45.480 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:45.480 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:45.481 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:45.481 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:45.481 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:45.481 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:45.481 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:45.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:45.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:45.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:45.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:45.482 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:45.483 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:45.483 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:45.483 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:45.483 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:45.483 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:45.484 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:45.484 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:45.484 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:45.484 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.484 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:45.485 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:45.485 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:45.485 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:45.485 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:45.486 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:45.486 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:45.486 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:45.486 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:45.487 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:45.487 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:45.487 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:45.487 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:45.487 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:45.488 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:45.488 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:45.488 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:45.488 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:45.488 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:45.489 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:45.489 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:45.489 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:45.489 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:45.489 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:45.490 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:45.490 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:45.490 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:45.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.490 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:45.491 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:45.492 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:45.492 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:45.492 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:45.492 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:45.492 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:45.493 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:45.494 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:45.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.495 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:45.495 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:45.495 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:45.495 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:25:45.495 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:25:45.496 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:45.496 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:45.496 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:45.496 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:45.496 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:45.497 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:45.497 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:45.497 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:45.497 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:45.497 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:45.500 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:45.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.500 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "133", 2023-10-10T20:25:45.500 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:45.500 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:45.501 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:45.502 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:45.502 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:45.502 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:45.502 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:45.503 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:45.503 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:45.503 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:45.503 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:45.503 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:45.504 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:45.504 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:45.504 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:45.504 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:45.504 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.505 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:45.506 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:45.507 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:45.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.509 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:45.509 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:45.509 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:45.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.509 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.510 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.510 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:45.510 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:45.510 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.511 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:45.511 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.511 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.512 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.512 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.512 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.513 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.514 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:45.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:45.516 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.516 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.519 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.521 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:45.521 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.522 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.522 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:45.523 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.524 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:45.524 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:45.524 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.525 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:45.525 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:45.526 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:45.526 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:45.526 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.528 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.529 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:45.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.533 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.534 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.535 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.536 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.587 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.587 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.588 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:45.589 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:45.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.591 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:45.594 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.595 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.595 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.595 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.596 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:45.596 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.598 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:45.598 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:45.598 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:45.599 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:45.599 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.599 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.599 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.600 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.601 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:45.601 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.601 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.602 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.602 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.603 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.603 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.603 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.604 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.604 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.605 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.605 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.611 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.612 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.612 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:45.614 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.615 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.616 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:45.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.616 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.616 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.617 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:45.618 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:45.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:45.618 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:45.618 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.619 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:45.620 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.620 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.620 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.620 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.620 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.621 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.621 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.621 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.622 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.622 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.622 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:45.624 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.625 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.625 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.626 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:45.626 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.627 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.627 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.628 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.629 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:45.630 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.631 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.631 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:45.631 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.632 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.633 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.633 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.634 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.634 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:45.635 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.635 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.636 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.636 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:45.637 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.637 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.637 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.638 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.638 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.638 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.639 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.639 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:45.640 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.641 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:45.641 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:45.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.642 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.642 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:45.643 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:45.643 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:45.643 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:45.643 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.643 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.644 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.644 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.644 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:45.645 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:45.645 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:45.645 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:45.645 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:45.645 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:45.646 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:45.647 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:45.647 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:45.647 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.647 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.647 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.649 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:45.649 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.650 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.651 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:45.652 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.652 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.652 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:45.653 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.654 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.654 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:45.654 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.656 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:45.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:45.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:45.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:45.657 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.657 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:45.658 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:45.658 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:45.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.658 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.659 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:45.659 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:45.659 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:45.659 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:45.659 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:45.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.661 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:45.661 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.661 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.661 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.663 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:45.663 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.663 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.663 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.663 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.664 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:45.665 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.665 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.665 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.665 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.665 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.666 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:45.666 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.666 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.666 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.667 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:45.668 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.669 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.669 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.669 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.669 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.669 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:45.670 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.670 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.670 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.670 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:45.670 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:45.671 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.672 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:45.672 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.672 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:45.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:45.674 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:45.674 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:45.674 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:45.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.676 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:45.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:45.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:45.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:45.676 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:45.677 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:45.678 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:45.678 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:45.678 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:45.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:45.679 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:45.680 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:45.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:45.681 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:45.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 msgr2=0x7fd2e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 0x7fd2e40693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b2a90 tx=0x7fd2f0008040 comp rx=0 tx=0).stop 2023-10-10T20:25:45.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 msgr2=0x7fd300150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:45.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b3a80 tx=0x7fd2fc018000 comp rx=0 tx=0).stop 2023-10-10T20:25:45.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 shutdown_connections 2023-10-10T20:25:45.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd2e4066f30 0x7fd2e40693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3001510f0 0x7fd3000b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd300070700 0x7fd300150580 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 --2- 172.21.15.148:0/1987774154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3000c9af0 0x7fd30016af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:45.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 >> 172.21.15.148:0/1987774154 conn(0x7fd3000ca4f0 msgr2=0x7fd3000ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:45.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 shutdown_connections 2023-10-10T20:25:45.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:45.451+0000 7fd307cbe700 1 -- 172.21.15.148:0/1987774154 wait complete. 2023-10-10T20:25:45.685 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:25:45.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:45.686 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-10T20:25:45.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:25:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-10T20:25:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:25:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:25:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.229855 s, 228 MB/s 2023-10-10T20:25:45.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:46.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 Processor -- start 2023-10-10T20:25:46.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 -- start start 2023-10-10T20:25:46.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f4194158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419415cbb0 con 0x7f4194157870 2023-10-10T20:25:46.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419415bfa0 con 0x7f419415a620 2023-10-10T20:25:46.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.048+0000 7f419aca3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419415bdb0 con 0x7f4194157c40 2023-10-10T20:25:46.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f4198a3f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4194157870 0x7f419415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49042/0 (socket says 172.21.15.148:49042) 2023-10-10T20:25:46.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f4198a3f700 1 -- 172.21.15.148:0/4179737634 learned_addr learned my addr 172.21.15.148:0/4179737634 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f4199240700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f4194158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177775442 0 0) 0x7f419415cbb0 con 0x7f4194157870 2023-10-10T20:25:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f417c003570 con 0x7f4194157870 2023-10-10T20:25:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726597800 0 0) 0x7f417c003570 con 0x7f4194157870 2023-10-10T20:25:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 msgr2=0x7f4194158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:46.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f4194158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 >> v1:172.21.15.161:6789/0 conn(0x7f419415a620 legacy=0x7f419415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419415c280 con 0x7f4194157870 2023-10-10T20:25:46.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f41900017f0 con 0x7f4194157870 2023-10-10T20:25:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 >> v1:172.21.15.148:6789/0 conn(0x7f4194157870 legacy=0x7f419415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417c003a90 0x7f417c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f417c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f417c004d50 0x7f417c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c00bb40 con 0x7f417c004d50 2023-10-10T20:25:46.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c00bcb0 con 0x7f417c003a90 2023-10-10T20:25:46.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c00be70 con 0x7f4194157c40 2023-10-10T20:25:46.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f4199240700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f417c004d50 0x7f417c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f4198a3f700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417c003a90 0x7f417c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.049+0000 7f418bfff700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f417c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 -- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 msgr2=0x7f417c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:46.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4194157c40 0x7f417c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 -- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417c003a90 msgr2=0x7f417c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:46.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417c003a90 0x7f417c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 -- 172.21.15.148:0/4179737634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c003570 con 0x7f417c004d50 2023-10-10T20:25:46.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f417c004d50 0x7f417c0051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41941580c0 tx=0x7f418c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e496b6eb333f057 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00e710 con 0x7f417c004d50 2023-10-10T20:25:46.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c00e870 con 0x7f417c004d50 2023-10-10T20:25:46.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f418b7fe700 1 -- 172.21.15.148:0/4179737634 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00ec30 con 0x7f417c004d50 2023-10-10T20:25:46.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f419aca3700 1 -- 172.21.15.148:0/4179737634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f417c004d50 msgr2=0x7f417c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f419aca3700 1 --2- 172.21.15.148:0/4179737634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f417c004d50 0x7f417c0051c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41941580c0 tx=0x7f418c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.050+0000 7f4199240700 1 -- 172.21.15.148:0/4179737634 reap_dead start 2023-10-10T20:25:46.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.051+0000 7f419aca3700 1 -- 172.21.15.148:0/4179737634 shutdown_connections 2023-10-10T20:25:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.051+0000 7f419aca3700 1 -- 172.21.15.148:0/4179737634 >> 172.21.15.148:0/4179737634 conn(0x7f4194068ce0 msgr2=0x7f41940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.051+0000 7f419aca3700 1 -- 172.21.15.148:0/4179737634 shutdown_connections 2023-10-10T20:25:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.051+0000 7f419aca3700 1 -- 172.21.15.148:0/4179737634 wait complete. 2023-10-10T20:25:46.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 Processor -- start 2023-10-10T20:25:46.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 -- start start 2023-10-10T20:25:46.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f419414beb0 0x7f4194154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41941581e0 0x7f4194158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41940a7960 con 0x7f419414bae0 2023-10-10T20:25:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4194158b70 con 0x7f41941581e0 2023-10-10T20:25:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.052+0000 7f419aca3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419416b900 con 0x7f419414beb0 2023-10-10T20:25:46.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40514/0 (socket says 172.21.15.148:40514) 2023-10-10T20:25:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 -- 172.21.15.148:0/3531506363 learned_addr learned my addr 172.21.15.148:0/3531506363 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f418bfff700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f419414beb0 0x7f4194154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4199240700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41941581e0 0x7f4194158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 -- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f419414beb0 msgr2=0x7f4194154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f419414beb0 0x7f4194154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 -- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41941581e0 msgr2=0x7f4194158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41941581e0 0x7f4194158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419415f740 con 0x7f419414bae0 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4198a3f700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f419000a610 tx=0x7f41900021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190000fc0 con 0x7f419414bae0 2023-10-10T20:25:46.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4190013600 con 0x7f419414bae0 2023-10-10T20:25:46.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900139c0 con 0x7f419414bae0 2023-10-10T20:25:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419416ad80 con 0x7f419414bae0 2023-10-10T20:25:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.053+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419416b340 con 0x7f419414bae0 2023-10-10T20:25:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.054+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419416e010 con 0x7f419414bae0 2023-10-10T20:25:46.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4190013d30 con 0x7f419414bae0 2023-10-10T20:25:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f4188ff9700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4190085520 con 0x7f419414bae0 2023-10-10T20:25:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f418bfff700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7f418006ea50 con 0x7f419414bae0 2023-10-10T20:25:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.055+0000 7f418bfff700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f419415ea50 tx=0x7f4184007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.059+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4190017020 con 0x7f419414bae0 2023-10-10T20:25:46.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.243+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f419414c280 con 0x7f419414bae0 2023-10-10T20:25:46.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.244+0000 7f4188ff9700 1 -- 172.21.15.148:0/3531506363 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+32349 (secure 0 0 0) 0x7f419414c280 con 0x7f419414bae0 2023-10-10T20:25:46.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:46.247 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:46.247 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 133, 2023-10-10T20:25:46.247 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:46.248 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:46.248 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:40.529706+0000", 2023-10-10T20:25:46.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:46.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:46.248 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:46.249 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:46.249 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:46.249 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:46.249 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:46.249 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:46.250 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:46.250 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:46.250 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:46.250 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:46.250 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:46.251 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:46.251 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:46.251 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:46.251 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:46.252 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:46.252 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:46.252 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:46.252 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:46.252 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:46.253 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:46.254 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:46.254 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:46.254 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:46.254 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:46.254 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:46.255 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:46.255 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:46.255 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:46.255 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:46.255 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:46.256 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:46.256 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:46.256 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:46.256 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:46.256 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:46.257 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:46.257 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:46.257 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:46.257 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:46.258 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:46.259 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:46.259 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:46.259 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:46.259 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:46.259 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:46.260 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:46.260 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:46.262 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:46.262 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:46.262 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:46.262 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:46.263 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:46.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:46.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:46.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:46.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:46.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:46.265 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:46.265 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:46.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:46.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.266 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:46.266 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:46.266 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:46.266 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:46.266 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:46.267 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:46.268 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:46.268 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:46.268 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:46.268 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:46.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.269 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:46.269 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:46.269 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:46.271 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:46.271 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:46.271 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:46.271 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:46.272 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:46.272 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:46.272 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:46.273 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:46.273 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:46.273 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:46.273 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:46.273 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:46.274 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:46.275 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:46.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:46.275 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:46.275 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:46.275 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:46.276 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:46.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.276 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:46.276 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:46.276 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:46.277 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:46.278 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:46.278 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:46.278 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:46.278 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:46.278 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:46.279 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:46.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:46.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:46.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:46.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:46.280 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:46.281 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:46.281 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:46.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.281 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:46.281 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:46.282 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:46.283 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:46.283 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:46.283 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:46.283 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:46.283 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:46.284 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:46.284 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:46.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.284 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:46.284 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.285 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:46.286 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:46.286 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:46.286 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:25:46.286 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:25:46.286 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:46.287 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:46.287 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:46.287 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:46.287 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:46.287 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:46.288 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:46.288 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:46.288 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:46.288 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:46.288 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:46.289 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:46.290 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:46.290 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:46.290 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:46.290 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:46.290 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:46.291 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:46.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "133", 2023-10-10T20:25:46.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:46.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:46.292 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:46.292 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:46.292 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:46.292 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:46.293 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:46.294 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:46.294 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:46.294 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:46.294 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:46.294 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:46.295 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:46.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:46.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:46.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:46.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:46.296 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:46.296 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:46.296 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:46.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:46.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.297 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:46.297 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:46.297 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:46.297 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:46.298 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:46.299 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:46.299 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:46.299 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:46.299 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:46.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.300 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:46.300 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:46.300 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:46.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.301 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:46.301 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:46.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.302 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.303 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.304 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.305 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.306 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:46.307 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:46.308 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.311 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.312 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.313 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:46.314 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.315 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:46.316 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.317 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.318 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:46.319 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.322 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:46.324 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:46.325 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.326 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.328 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:46.329 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.331 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.332 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.335 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.336 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:46.337 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.338 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.340 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.342 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:46.344 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.345 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.346 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.347 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.348 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.349 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.350 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:46.351 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.356 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:46.357 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:46.358 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.359 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.360 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.361 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.362 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.364 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.365 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.366 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.367 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:46.368 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:46.369 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:46.373 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.375 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.376 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.377 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.378 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.379 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.380 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.382 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.383 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:46.385 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:46.386 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:46.387 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.388 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.389 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.390 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.391 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.393 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.394 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.395 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.396 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:46.397 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:46.398 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.399 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.400 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.401 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:46.402 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:46.404 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:46.405 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.406 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.407 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.408 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:46.409 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:46.410 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.411 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.412 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:46.415 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:46.416 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:46.417 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.418 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:46.419 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.419 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.419 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.419 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.419 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:46.420 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.421 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.421 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.421 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.421 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.421 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:46.422 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.422 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.422 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:46.422 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:46.423 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.424 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:46.424 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.424 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.426 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.426 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:46.426 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.426 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.428 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.428 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:46.428 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.428 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.428 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.429 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.430 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:46.430 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.430 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.430 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.430 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.431 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.431 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:46.431 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.431 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:46.432 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:46.433 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:46.434 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:46.434 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:46.434 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:46.434 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:46.434 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:46.435 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:46.436 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:46.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:46.436 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:46.436 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:46.437 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:46.438 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:46.438 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:46.438 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:46.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:46.438 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:46.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 msgr2=0x7f41800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f419415ea50 tx=0x7f4184007d30 comp rx=0 tx=0).stop 2023-10-10T20:25:46.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 msgr2=0x7f419415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f419000a610 tx=0x7f41900021d0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 shutdown_connections 2023-10-10T20:25:46.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41941581e0 0x7f4194158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f419414bae0 0x7f419415f200 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 --2- 172.21.15.148:0/3531506363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f419414beb0 0x7f4194154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 >> 172.21.15.148:0/3531506363 conn(0x7f4194068ce0 msgr2=0x7f41940b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:46.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 shutdown_connections 2023-10-10T20:25:46.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.246+0000 7f419aca3700 1 -- 172.21.15.148:0/3531506363 wait complete. 2023-10-10T20:25:46.443 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:46.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.557+0000 7f5260457700 1 Processor -- start 2023-10-10T20:25:46.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.557+0000 7f5260457700 1 -- start start 2023-10-10T20:25:46.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f5260457700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f52581577d0 0x7f52580cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f5260457700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5258157ba0 0x7f525806a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f5260457700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525815c360 con 0x7f525815b0d0 2023-10-10T20:25:46.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f5260457700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525815b790 con 0x7f5258157ba0 2023-10-10T20:25:46.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f5260457700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525815d800 con 0x7f52581577d0 2023-10-10T20:25:46.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5258157ba0 0x7f525806a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525e9f4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f525815b0d0 0x7f525815da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49056/0 (socket says 172.21.15.148:49056) 2023-10-10T20:25:46.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5258157ba0 0x7f525806a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40524/0 (socket says 172.21.15.148:40524) 2023-10-10T20:25:46.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525e9f4700 1 -- 172.21.15.148:0/1376425388 learned_addr learned my addr 172.21.15.148:0/1376425388 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:46.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525e1f3700 1 --2- 172.21.15.148:0/1376425388 >> v2:172.21.15.161:3300/0 conn(0x7f52581577d0 0x7f52580cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 -- 172.21.15.148:0/1376425388 >> v2:172.21.15.161:3300/0 conn(0x7f52581577d0 msgr2=0x7f52580cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:46.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 --2- 172.21.15.148:0/1376425388 >> v2:172.21.15.161:3300/0 conn(0x7f52581577d0 0x7f52580cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 -- 172.21.15.148:0/1376425388 >> v1:172.21.15.148:6789/0 conn(0x7f525815b0d0 legacy=0x7f525815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d1f1700 1 -- 172.21.15.148:0/1376425388 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670255019 0 0) 0x7f525815c360 con 0x7f525815b0d0 2023-10-10T20:25:46.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 -- 172.21.15.148:0/1376425388 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525815ba30 con 0x7f5258157ba0 2023-10-10T20:25:46.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d9f2700 1 --2- 172.21.15.148:0/1376425388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f525806a520 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f52580b6dc0 tx=0x7f52480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f713c32889cdbb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d1f1700 1 -- 172.21.15.148:0/1376425388 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800b6b0 con 0x7f5258157ba0 2023-10-10T20:25:46.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d1f1700 1 -- 172.21.15.148:0/1376425388 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800b810 con 0x7f5258157ba0 2023-10-10T20:25:46.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.558+0000 7f525d1f1700 1 -- 172.21.15.148:0/1376425388 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800bbb0 con 0x7f5258157ba0 2023-10-10T20:25:46.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 -- 172.21.15.148:0/1376425388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 msgr2=0x7f525806a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 --2- 172.21.15.148:0/1376425388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f525806a520 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f52580b6dc0 tx=0x7f52480091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 -- 172.21.15.148:0/1376425388 shutdown_connections 2023-10-10T20:25:46.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 --2- 172.21.15.148:0/1376425388 >> v2:172.21.15.161:3300/0 conn(0x7f52581577d0 0x7f52580cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 --2- 172.21.15.148:0/1376425388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f525806a520 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 -- 172.21.15.148:0/1376425388 >> 172.21.15.148:0/1376425388 conn(0x7f5258068ce0 msgr2=0x7f52580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:46.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 -- 172.21.15.148:0/1376425388 shutdown_connections 2023-10-10T20:25:46.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.559+0000 7f5260457700 1 -- 172.21.15.148:0/1376425388 wait complete. 2023-10-10T20:25:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.560+0000 7f5260457700 1 Processor -- start 2023-10-10T20:25:46.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.560+0000 7f5260457700 1 -- start start 2023-10-10T20:25:46.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 0x7f52581477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f525815b0d0 0x7f52580d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b1fb0 con 0x7f5258157ba0 2023-10-10T20:25:46.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525816a610 con 0x7f52581577d0 2023-10-10T20:25:46.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f5260457700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b13a0 con 0x7f525815b0d0 2023-10-10T20:25:46.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40526/0 (socket says 172.21.15.148:40526) 2023-10-10T20:25:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 -- 172.21.15.148:0/522980320 learned_addr learned my addr 172.21.15.148:0/522980320 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525e1f3700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 0x7f52581477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525e9f4700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f525815b0d0 0x7f52580d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 -- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f525815b0d0 msgr2=0x7f52580d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f525815b0d0 0x7f52580d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 -- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 msgr2=0x7f52581477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 0x7f52581477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525d9f2700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248009420 con 0x7f5258157ba0 2023-10-10T20:25:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.561+0000 7f525e1f3700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 0x7f52581477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f525d9f2700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f525806b580 tx=0x7f5248017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248011420 con 0x7f5258157ba0 2023-10-10T20:25:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248011580 con 0x7f5258157ba0 2023-10-10T20:25:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52580b1680 con 0x7f5258157ba0 2023-10-10T20:25:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248011940 con 0x7f5258157ba0 2023-10-10T20:25:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.562+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52580b1c40 con 0x7f5258157ba0 2023-10-10T20:25:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.563+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525804c3c0 con 0x7f5258157ba0 2023-10-10T20:25:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.563+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5248011d30 con 0x7f5258157ba0 2023-10-10T20:25:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.563+0000 7f524e7fc700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 0x7f523c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.563+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52480859e0 con 0x7f5258157ba0 2023-10-10T20:25:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.563+0000 7f525e1f3700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 0x7f523c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:46.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.564+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7f523c06ea50 con 0x7f5258157ba0 2023-10-10T20:25:46.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.564+0000 7f525e1f3700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 0x7f523c0693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5254007540 tx=0x7f525400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:46.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.568+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5248015020 con 0x7f5258157ba0 2023-10-10T20:25:46.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.793+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52580b20f0 con 0x7f5258157ba0 2023-10-10T20:25:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.794+0000 7f524e7fc700 1 -- 172.21.15.148:0/522980320 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5248052c50 con 0x7f5258157ba0 2023-10-10T20:25:46.796 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:46.796 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:25:46.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 msgr2=0x7f523c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 0x7f523c0693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5254007540 tx=0x7f525400b040 comp rx=0 tx=0).stop 2023-10-10T20:25:46.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 msgr2=0x7f52580d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:46.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f525806b580 tx=0x7f5248017000 comp rx=0 tx=0).stop 2023-10-10T20:25:46.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 shutdown_connections 2023-10-10T20:25:46.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f523c066f30 0x7f523c0693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f525815b0d0 0x7f52580d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52581577d0 0x7f52581477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 --2- 172.21.15.148:0/522980320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258157ba0 0x7f52580d02d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:46.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 >> 172.21.15.148:0/522980320 conn(0x7f5258068ce0 msgr2=0x7f5258155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:46.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 shutdown_connections 2023-10-10T20:25:46.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:46.796+0000 7f5260457700 1 -- 172.21.15.148:0/522980320 wait complete. 2023-10-10T20:25:46.801 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:25:46.810 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-10T20:25:46.955 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":7095,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.285153255000001,"mdsmap_epoch":64,"osdmap_epoch":133,"osdmap_epoch_barrier":133,"uptime":27.401365307999999} 2023-10-10T20:25:46.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 7095, "mdsmap_epoch": 64, "osdmap_epoch": 133, "osdmap_epoch_barrier": 133, "rank_uptime": 22.285153255, "state": "up:active", "uptime": 27.401365308, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:25:46.956 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:25:46.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:46.957 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-10T20:25:46.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:46.975 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-10T20:25:47.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:47.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.184+0000 7f2e0e4bf700 1 Processor -- start 2023-10-10T20:25:47.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.184+0000 7f2e0e4bf700 1 -- start start 2023-10-10T20:25:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e0e4bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 0x7f2e080b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e0e4bf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2e080b1bf0 0x7f2e080c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e0e4bf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0806efc0 con 0x7f2e080aedb0 2023-10-10T20:25:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e0e4bf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e08158470 con 0x7f2e080b1bf0 2023-10-10T20:25:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e0e4bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080b2100 con 0x7f2e0806f450 2023-10-10T20:25:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e07fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2e080aedb0 0x7f2e080af180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49072/0 (socket says 172.21.15.148:49072) 2023-10-10T20:25:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.185+0000 7f2e07fff700 1 -- 172.21.15.148:0/874017525 learned_addr learned my addr 172.21.15.148:0/874017525 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 --2- 172.21.15.148:0/874017525 >> v2:172.21.15.161:3300/0 conn(0x7f2e080b1bf0 0x7f2e080c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e077fe700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 0x7f2e080b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230313001 0 0) 0x7f2e0806efc0 con 0x7f2e080aedb0 2023-10-10T20:25:47.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df0003570 con 0x7f2e080aedb0 2023-10-10T20:25:47.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 -- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 msgr2=0x7f2e080b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:47.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 0x7f2e080b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163420789 0 0) 0x7f2df0003570 con 0x7f2e080aedb0 2023-10-10T20:25:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 -- 172.21.15.148:0/874017525 >> v1:172.21.15.148:6789/0 conn(0x7f2e080aedb0 legacy=0x7f2e080af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 -- 172.21.15.148:0/874017525 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e080b23e0 con 0x7f2e080b1bf0 2023-10-10T20:25:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e077fe700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 0x7f2e080b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e0ca5c700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080b1bf0 0x7f2e080c90f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e60 tx=0x7f2e00009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca37d80b07b57418 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:47.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000e070 con 0x7f2e080b1bf0 2023-10-10T20:25:47.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0000b7a0 con 0x7f2e080b1bf0 2023-10-10T20:25:47.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.186+0000 7f2e06ffd700 1 -- 172.21.15.148:0/874017525 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8001470 con 0x7f2e080b1bf0 2023-10-10T20:25:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080b1bf0 msgr2=0x7f2e080c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080b1bf0 0x7f2e080c90f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e60 tx=0x7f2e00009780 comp rx=0 tx=0).stop 2023-10-10T20:25:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/874017525 shutdown_connections 2023-10-10T20:25:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080b1bf0 0x7f2e080c90f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/874017525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0806f450 0x7f2e080b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/874017525 >> 172.21.15.148:0/874017525 conn(0x7f2e080cd1e0 msgr2=0x7f2e080cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:47.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/874017525 shutdown_connections 2023-10-10T20:25:47.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.187+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/874017525 wait complete. 2023-10-10T20:25:47.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.188+0000 7f2e0e4bf700 1 Processor -- start 2023-10-10T20:25:47.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.188+0000 7f2e0e4bf700 1 -- start start 2023-10-10T20:25:47.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 0x7f2e0814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 0x7f2e0814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0816b480 con 0x7f2e0806f450 2023-10-10T20:25:47.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08159790 con 0x7f2e080aedb0 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0e4bf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0815a030 con 0x7f2e080b1bf0 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40532/0 (socket says 172.21.15.148:40532) 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 -- 172.21.15.148:0/151440256 learned_addr learned my addr 172.21.15.148:0/151440256 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e077fe700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 0x7f2e0814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0ca5c700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 0x7f2e0814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 -- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 msgr2=0x7f2e0814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 0x7f2e0814ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 -- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 msgr2=0x7f2e0814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 0x7f2e0814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc009420 con 0x7f2e0806f450 2023-10-10T20:25:47.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e077fe700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 0x7f2e0814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:47.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e0ca5c700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 0x7f2e0814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:47.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2e07fff700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2df800cb90 tx=0x7f2df8001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.189+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8007360 con 0x7f2e0806f450 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.190+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80074e0 con 0x7f2e0806f450 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.190+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e00009a20 con 0x7f2e0806f450 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.190+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004830 con 0x7f2e0806f450 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.190+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0814d820 con 0x7f2e0806f450 2023-10-10T20:25:47.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.190+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0804c3c0 con 0x7f2e0806f450 2023-10-10T20:25:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.191+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2df8004ba0 con 0x7f2e0806f450 2023-10-10T20:25:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.191+0000 7f2deffff700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 0x7f2df40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.191+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2df8083e30 con 0x7f2e0806f450 2023-10-10T20:25:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.191+0000 7f2e077fe700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 0x7f2df40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.191+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7f2df406ea50 con 0x7f2e0806f450 2023-10-10T20:25:47.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.192+0000 7f2e077fe700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 0x7f2df40693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0093f0 tx=0x7f2dfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:47.206 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.195+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2df8018020 con 0x7f2e0806f450 2023-10-10T20:25:47.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.379+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2e0806ec50 con 0x7f2e0806f450 2023-10-10T20:25:47.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.380+0000 7f2deffff700 1 -- 172.21.15.148:0/151440256 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+32349 (secure 0 0 0) 0x7f2df8051120 con 0x7f2e0806f450 2023-10-10T20:25:47.383 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:47.383 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:47.383 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 133, 2023-10-10T20:25:47.383 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:47.384 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:47.384 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:40.529706+0000", 2023-10-10T20:25:47.384 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:47.384 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:47.385 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:47.386 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:47.386 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:47.386 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:47.386 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:47.386 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:47.387 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:47.387 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:47.387 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:47.387 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:47.387 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:47.388 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:47.388 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:47.388 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:47.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.388 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:47.389 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:47.389 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:47.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:47.389 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:47.389 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:47.390 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:47.390 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:47.390 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:47.390 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:47.390 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:47.391 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:47.391 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:47.391 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:47.391 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:47.391 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:47.392 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:47.392 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:47.392 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:47.392 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:47.392 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:47.393 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:47.393 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:47.393 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:47.393 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:47.393 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:47.394 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:47.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.394 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:47.394 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:47.394 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:47.395 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:47.395 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:47.395 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:47.395 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:47.395 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:47.396 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:47.396 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:47.396 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:47.396 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:47.396 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:47.397 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:47.397 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:47.397 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:47.397 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:47.397 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:47.398 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:47.398 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:47.398 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:47.398 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:47.398 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:47.399 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:47.399 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:47.399 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:47.399 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:47.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:47.400 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:47.401 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:47.401 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:47.401 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:47.401 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:47.401 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:47.403 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:47.403 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.403 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.403 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:47.404 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:47.404 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:47.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:47.404 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:47.404 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:47.405 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:47.405 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:47.405 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:47.405 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:47.405 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:47.406 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:47.406 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:47.406 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:47.406 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:47.406 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:47.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:47.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:47.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:47.407 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:47.407 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:47.408 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:47.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.409 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:47.409 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:47.409 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:47.409 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:47.410 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:47.410 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:47.410 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:47.410 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:47.410 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:47.411 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:47.411 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:47.411 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:47.411 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:47.411 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:47.412 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:47.412 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:47.412 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:47.412 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:47.412 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:47.413 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:47.414 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:47.414 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:47.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.414 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:47.414 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:47.415 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:47.415 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:47.415 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:47.415 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:47.415 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:47.416 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:47.417 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:47.417 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:47.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.417 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:47.417 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:47.418 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:47.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.418 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.418 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.418 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.419 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:47.419 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:47.419 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:47.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:25:47.419 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:25:47.420 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:47.420 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:47.420 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:47.420 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:47.420 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:47.421 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:47.421 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:47.421 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:47.421 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:47.421 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:47.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:47.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:47.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:47.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:47.422 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:47.423 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:47.423 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:47.423 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:47.423 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:47.423 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:47.424 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:47.424 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:47.424 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.424 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "133", 2023-10-10T20:25:47.424 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:47.425 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:47.425 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:47.425 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:47.425 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:47.425 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:47.426 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:47.426 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:47.426 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:47.426 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:47.426 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:47.427 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:47.428 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:47.429 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:47.429 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:47.429 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:47.429 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:47.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.430 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:47.430 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:47.430 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:47.430 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:47.430 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:47.431 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:47.431 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:47.431 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:47.431 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:47.431 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:47.432 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:47.432 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:47.432 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:47.432 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:47.432 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.434 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:47.434 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:47.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.434 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:47.434 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.435 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.435 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.435 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.435 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.435 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.436 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.437 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.437 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.437 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:47.437 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.438 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.438 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:47.438 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.439 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.439 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.439 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.440 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.440 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:47.440 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.441 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.442 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.442 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.442 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.443 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:47.443 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.443 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.444 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.445 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.445 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.445 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.445 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:47.445 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.446 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.446 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.446 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.446 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:47.446 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:47.447 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.447 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.447 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.447 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:47.447 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:47.448 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:47.448 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:47.448 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.448 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.448 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.449 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.450 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.450 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.450 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.450 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.451 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.452 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.453 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.453 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:47.453 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.453 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.453 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.454 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.454 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.454 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.454 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.454 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.455 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:47.456 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.456 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.456 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.456 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.456 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.457 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.457 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.457 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.457 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:47.457 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.458 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.458 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.458 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.458 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:47.458 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.459 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.459 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.459 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.459 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.459 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.460 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.460 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:47.460 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.460 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.461 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.461 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:47.461 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:47.461 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.462 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.462 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.462 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:47.462 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:47.462 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.463 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.464 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.464 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:47.464 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.464 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.464 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.465 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.465 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.465 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.466 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.466 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.466 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.466 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.466 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.467 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.467 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.467 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:47.467 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.467 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.468 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.468 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.468 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:47.468 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.468 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.469 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.470 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.471 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.471 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:47.471 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.471 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.471 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.472 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.472 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.472 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.472 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.472 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.473 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:47.473 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.473 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.473 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.474 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:47.474 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.474 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.475 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.475 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.475 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.475 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.475 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:47.476 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.476 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.476 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.476 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.476 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:47.477 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:47.477 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.477 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.477 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.477 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.478 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.479 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.479 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.479 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.479 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.479 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:47.480 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.480 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.480 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.480 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.480 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.481 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.482 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.482 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.482 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.482 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.482 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:47.483 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.483 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.483 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.485 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:47.485 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.486 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.486 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.486 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.486 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.486 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.487 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.488 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.489 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.489 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.489 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.490 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:47.490 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.490 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.490 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.490 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.491 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.492 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.492 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.492 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:47.492 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.493 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.493 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:47.493 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:47.494 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.495 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.496 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.497 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.497 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.497 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.497 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.497 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.498 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.499 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.499 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:47.499 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.500 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.500 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:47.500 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.501 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.502 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:47.502 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.502 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.503 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:47.503 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.503 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.504 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.504 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.504 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.504 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.505 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:47.506 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.506 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.506 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.507 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.507 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.507 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.507 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:47.507 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:47.508 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.509 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.509 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:47.509 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:47.509 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:47.510 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:47.510 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.510 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.510 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.510 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.511 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:47.511 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.511 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.512 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.512 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.512 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.512 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.513 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.513 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.513 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.513 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.513 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.515 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.515 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:47.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.515 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.516 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.516 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:47.517 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.519 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.519 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.519 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:47.520 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.520 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.520 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:47.521 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.521 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.521 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.521 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:47.522 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:47.523 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:47.524 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:47.525 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:47.525 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.525 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.525 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.526 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:47.526 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.526 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.527 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.527 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.527 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.527 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.527 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:47.529 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.530 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.530 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:47.530 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:47.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.534 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.534 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.534 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.534 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.535 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.536 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.537 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.537 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.537 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:47.537 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.537 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:47.538 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:47.539 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:47.540 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.540 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.540 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.540 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:47.541 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:47.541 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:47.541 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:47.542 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:47.543 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:47.543 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:47.543 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:47.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.545 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.545 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:47.545 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:47.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:47.548 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.548 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.548 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.548 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.550 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.550 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.550 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:47.550 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.551 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:47.551 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.553 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:47.553 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:47.553 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:47.553 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.553 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.554 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:47.554 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:47.554 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:47.554 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:47.555 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:47.555 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:47.555 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:47.555 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:47.555 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:47.556 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.557 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.557 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.557 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.557 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.557 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:47.558 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.558 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:47.558 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:47.559 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.560 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:47.560 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.560 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.562 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:47.562 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.562 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.564 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.564 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:47.564 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.564 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.564 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.566 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.567 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.567 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:47.567 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.567 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.568 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:47.568 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.568 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.568 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.568 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.569 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.569 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:47.569 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:47.570 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:47.571 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:47.571 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:47.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.571 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:47.571 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:47.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:47.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:47.572 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:47.572 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:47.572 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:47.573 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:47.574 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:47.574 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:47.574 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:47.574 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:47.575 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:47.575 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:47.575 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:47.575 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:47.575 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:47.576 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:47.576 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:47.576 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:47.576 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:47.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:47.577 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 msgr2=0x7f2df40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 0x7f2df40693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0093f0 tx=0x7f2dfc008040 comp rx=0 tx=0).stop 2023-10-10T20:25:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 msgr2=0x7f2e081680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2df800cb90 tx=0x7f2df8001b50 comp rx=0 tx=0).stop 2023-10-10T20:25:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 shutdown_connections 2023-10-10T20:25:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2df4066f30 0x7f2df40693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e080b1bf0 0x7f2e0814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e0806f450 0x7f2e081680f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 --2- 172.21.15.148:0/151440256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e080aedb0 0x7f2e0814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 >> 172.21.15.148:0/151440256 conn(0x7f2e080cd1e0 msgr2=0x7f2e081559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 shutdown_connections 2023-10-10T20:25:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:47.382+0000 7f2e0e4bf700 1 -- 172.21.15.148:0/151440256 wait complete. 2023-10-10T20:25:47.581 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:25:52.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:52.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.703+0000 7fddb8214700 1 Processor -- start 2023-10-10T20:25:52.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 -- start start 2023-10-10T20:25:52.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fddb0157ba0 0x7fddb006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fddb015b0d0 0x7fddb006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb015c360 con 0x7fddb01577d0 2023-10-10T20:25:52.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb015b790 con 0x7fddb0157ba0 2023-10-10T20:25:52.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.704+0000 7fddb8214700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb015d800 con 0x7fddb015b0d0 2023-10-10T20:25:52.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fddb0157ba0 0x7fddb006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fddb0157ba0 0x7fddb006a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33806/0 (socket says 172.21.15.148:33806) 2023-10-10T20:25:52.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 -- 172.21.15.148:0/2614175955 learned_addr learned my addr 172.21.15.148:0/2614175955 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:52.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb67b1700 1 --2- 172.21.15.148:0/2614175955 >> v2:172.21.15.161:3300/0 conn(0x7fddb015b0d0 0x7fddb006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 -- 172.21.15.148:0/2614175955 >> v2:172.21.15.161:3300/0 conn(0x7fddb015b0d0 msgr2=0x7fddb006aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:52.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 --2- 172.21.15.148:0/2614175955 >> v2:172.21.15.161:3300/0 conn(0x7fddb015b0d0 0x7fddb006aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:52.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 -- 172.21.15.148:0/2614175955 >> v1:172.21.15.161:6789/0 conn(0x7fddb01577d0 legacy=0x7fddb00cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:52.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 -- 172.21.15.148:0/2614175955 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb015ba30 con 0x7fddb0157ba0 2023-10-10T20:25:52.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb57af700 1 --2- 172.21.15.148:0/2614175955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb006a520 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fddb00b6d90 tx=0x7fdda0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e4f48edb574d4e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:52.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb4fae700 1 -- 172.21.15.148:0/2614175955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0009e70 con 0x7fddb0157ba0 2023-10-10T20:25:52.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb4fae700 1 -- 172.21.15.148:0/2614175955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda000b860 con 0x7fddb0157ba0 2023-10-10T20:25:52.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.705+0000 7fddb4fae700 1 -- 172.21.15.148:0/2614175955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000bc00 con 0x7fddb0157ba0 2023-10-10T20:25:52.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 -- 172.21.15.148:0/2614175955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 msgr2=0x7fddb006a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:52.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 --2- 172.21.15.148:0/2614175955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb006a520 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fddb00b6d90 tx=0x7fdda0009250 comp rx=0 tx=0).stop 2023-10-10T20:25:52.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 -- 172.21.15.148:0/2614175955 shutdown_connections 2023-10-10T20:25:52.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 --2- 172.21.15.148:0/2614175955 >> v2:172.21.15.161:3300/0 conn(0x7fddb015b0d0 0x7fddb006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:52.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 --2- 172.21.15.148:0/2614175955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb006a520 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:52.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 -- 172.21.15.148:0/2614175955 >> 172.21.15.148:0/2614175955 conn(0x7fddb0068ce0 msgr2=0x7fddb00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:52.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 -- 172.21.15.148:0/2614175955 shutdown_connections 2023-10-10T20:25:52.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.706+0000 7fddb8214700 1 -- 172.21.15.148:0/2614175955 wait complete. 2023-10-10T20:25:52.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.707+0000 7fddb8214700 1 Processor -- start 2023-10-10T20:25:52.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.707+0000 7fddb8214700 1 -- start start 2023-10-10T20:25:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb01577d0 0x7fddb00b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb015b0d0 0x7fddb016e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b22d0 con 0x7fddb0157ba0 2023-10-10T20:25:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b16c0 con 0x7fddb015b0d0 2023-10-10T20:25:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb8214700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00b19a0 con 0x7fddb01577d0 2023-10-10T20:25:52.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33816/0 (socket says 172.21.15.148:33816) 2023-10-10T20:25:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 -- 172.21.15.148:0/3190017640 learned_addr learned my addr 172.21.15.148:0/3190017640 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb67b1700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb015b0d0 0x7fddb016e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb5fb0700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb01577d0 0x7fddb00b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 -- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb01577d0 msgr2=0x7fddb00b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:52.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb01577d0 0x7fddb00b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:52.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 -- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb015b0d0 msgr2=0x7fddb016e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:52.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb015b0d0 0x7fddb016e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00094f0 con 0x7fddb0157ba0 2023-10-10T20:25:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fddb57af700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fddb006b580 tx=0x7fdda0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.708+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda000bbe0 con 0x7fddb0157ba0 2023-10-10T20:25:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.709+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda0011420 con 0x7fddb0157ba0 2023-10-10T20:25:52.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.709+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00b1c80 con 0x7fddb0157ba0 2023-10-10T20:25:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.709+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda00117c0 con 0x7fddb0157ba0 2023-10-10T20:25:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.709+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb00b2ca0 con 0x7fddb0157ba0 2023-10-10T20:25:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.710+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb004c3c0 con 0x7fddb0157ba0 2023-10-10T20:25:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdda0011bf0 con 0x7fddb0157ba0 2023-10-10T20:25:52.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fdda5ffb700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 0x7fdd940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdda0085990 con 0x7fddb0157ba0 2023-10-10T20:25:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fddb5fb0700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 0x7fdd940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7fdd9406ea50 con 0x7fddb0157ba0 2023-10-10T20:25:52.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.711+0000 7fddb5fb0700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 0x7fdd940693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fddb0155170 tx=0x7fddac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:52.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.716+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdda0052c00 con 0x7fddb0157ba0 2023-10-10T20:25:52.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.902+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fddb00b2410 con 0x7fddb0157ba0 2023-10-10T20:25:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.903+0000 7fdda5ffb700 1 -- 172.21.15.148:0/3190017640 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+32349 (secure 0 0 0) 0x7fdda0028020 con 0x7fddb0157ba0 2023-10-10T20:25:52.906 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:52.906 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 133, 2023-10-10T20:25:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:52.907 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:40.529706+0000", 2023-10-10T20:25:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:52.908 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:52.909 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:52.910 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:52.911 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:52.912 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:52.913 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:52.914 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:52.915 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:52.916 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:52.917 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:52.918 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:52.919 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:52.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:52.921 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:52.922 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:52.923 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:52.924 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:52.947 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:25:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:25:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:52.948 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:52.949 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:52.950 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:52.951 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:52.952 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "133", 2023-10-10T20:25:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:52.953 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:52.954 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:52.955 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:52.956 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:52.958 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:52.959 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:52.960 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:52.961 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:52.962 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.963 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:52.964 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:52.965 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:52.966 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.967 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.968 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.969 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:52.970 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.970 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.970 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.971 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:52.972 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.972 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.972 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.972 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.972 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:52.973 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.973 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.973 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:52.973 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.974 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.974 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:52.974 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.975 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.975 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:52.975 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.976 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.976 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:52.976 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.977 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.977 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:52.977 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:52.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:52.978 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:52.979 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:52.979 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:52.979 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:52.979 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:52.980 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.980 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:52.980 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:52.981 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:52.981 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:52.981 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:52.981 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:52.982 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:52.982 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:52.982 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:52.982 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:52.983 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:52.983 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:52.983 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:52.983 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.984 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.984 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:52.984 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.985 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.985 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:52.985 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.985 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.989 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.989 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:52.989 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.989 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.990 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.990 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:52.990 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.990 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.990 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.991 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.992 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:52.992 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:52.992 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.992 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.993 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:52.994 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:52.994 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:52.994 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:52.994 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.995 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.995 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:52.995 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:52.995 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:52.995 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:52.996 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:52.996 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:52.996 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:52.996 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:52.996 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:52.997 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:52.997 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:52.997 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:52.997 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:52.997 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:52.998 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:52.998 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:52.998 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:52.998 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:52.999 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:52.999 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:52.999 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:52.999 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:52.999 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.000 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.000 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.000 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.000 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:53.001 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.001 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.001 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.001 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.002 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:53.002 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.002 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.002 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.003 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.003 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.003 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.004 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:53.004 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.004 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.004 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.005 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.006 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.006 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.006 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.006 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.007 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:53.007 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.007 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.007 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.007 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.008 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:53.008 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.008 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.008 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.008 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.009 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:53.010 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.010 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.010 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.010 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.012 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:53.012 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:53.012 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:53.012 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:53.012 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.013 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.013 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.013 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.013 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.013 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.014 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:53.014 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:53.014 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:53.017 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:53.017 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:53.017 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:53.017 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:53.018 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:53.018 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:53.018 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:53.018 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:53.018 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:53.019 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.019 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.019 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.019 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:53.020 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.020 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.020 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.020 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.021 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:53.021 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.021 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.022 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.022 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.022 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:53.023 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.023 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.023 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:53.024 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.024 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.024 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.024 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:53.025 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.026 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.026 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.026 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:53.027 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.027 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.027 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.027 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.028 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:53.028 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.029 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:53.029 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:53.030 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.030 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.030 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:53.030 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:53.031 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:53.031 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:53.031 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.031 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.032 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.032 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.032 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:53.033 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:53.034 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:53.034 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:53.034 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:53.034 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:53.034 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:53.035 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:53.035 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.035 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.037 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:53.037 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.038 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.038 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.038 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.039 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.039 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:53.039 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.039 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.040 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:53.040 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.041 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.041 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.041 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.042 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.042 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:53.042 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.043 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.043 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:53.043 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.059 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:53.059 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:53.060 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.060 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.060 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.061 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.061 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.061 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.061 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.061 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.062 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:53.062 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.063 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.063 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:53.063 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:53.063 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.063 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.064 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.064 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:53.064 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:53.064 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:53.064 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:53.065 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.065 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.065 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.065 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:53.067 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:53.067 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:53.067 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:53.067 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:53.067 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:53.068 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:53.068 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:53.068 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:53.068 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:53.068 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:53.070 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.070 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.070 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:53.071 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.071 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.071 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.072 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.072 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.072 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:53.073 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.073 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.073 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:53.074 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.074 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.074 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.075 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.075 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.075 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.075 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.075 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:53.076 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.076 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.076 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.076 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.077 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:53.077 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.077 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.077 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.078 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.078 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.078 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.078 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:53.079 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.079 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.079 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.079 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:53.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:53.080 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.080 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.081 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:53.081 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:53.081 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:53.081 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:53.081 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.083 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:53.083 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:53.083 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:53.083 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:53.083 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:53.084 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:53.084 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:53.084 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:53.084 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:53.085 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:53.085 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:53.085 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:53.085 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:53.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:53.086 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.087 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.087 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:53.087 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.087 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.088 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.088 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:53.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:53.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.090 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.090 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.090 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:53.090 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.090 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.091 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.091 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:53.091 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.092 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.092 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:53.092 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.092 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.092 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.093 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.093 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:53.093 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.093 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.093 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.094 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.094 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:53.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:53.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:53.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:53.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.097 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:53.097 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:53.097 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:53.097 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:53.098 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:53.098 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:53.098 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:53.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:53.099 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.099 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:53.099 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:53.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.099 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:53.100 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.100 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.100 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.100 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.100 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.101 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:53.101 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.101 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.102 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:53.102 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.102 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.102 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.102 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.103 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.103 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:53.103 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.104 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.105 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.105 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.105 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:53.105 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.105 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:53.106 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.106 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.107 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.107 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.107 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.107 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:53.107 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.108 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.108 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.108 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:53.108 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.108 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.109 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.109 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.109 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.109 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:53.109 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.110 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.110 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:53.110 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.111 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.111 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.111 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.111 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.111 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:53.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.112 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:53.112 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:53.114 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:53.114 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:53.114 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:53.115 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:53.116 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:53.118 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:53.118 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:53.118 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:53.118 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:53.118 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:53.119 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:53.119 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:53.119 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:53.120 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:53.120 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:53.120 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:53.120 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:53.120 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:53.121 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:53.121 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:53.121 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:53.121 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:53.121 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:53.122 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:53.122 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:53.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 msgr2=0x7fdd940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:53.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 0x7fdd940693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fddb0155170 tx=0x7fddac009040 comp rx=0 tx=0).stop 2023-10-10T20:25:53.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 msgr2=0x7fddb016ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:53.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fddb006b580 tx=0x7fdda0017000 comp rx=0 tx=0).stop 2023-10-10T20:25:53.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 shutdown_connections 2023-10-10T20:25:53.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fdd94066f30 0x7fdd940693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:53.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb015b0d0 0x7fddb016e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:53.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb01577d0 0x7fddb00b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:53.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 --2- 172.21.15.148:0/3190017640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddb0157ba0 0x7fddb016ded0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:53.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 >> 172.21.15.148:0/3190017640 conn(0x7fddb0068ce0 msgr2=0x7fddb006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:53.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 shutdown_connections 2023-10-10T20:25:53.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:52.905+0000 7fddb8214700 1 -- 172.21.15.148:0/3190017640 wait complete. 2023-10-10T20:25:53.127 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:25:53.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:25:53.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:25:58.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:58.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.254+0000 7f5691178700 1 Processor -- start 2023-10-10T20:25:58.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.254+0000 7f5691178700 1 -- start start 2023-10-10T20:25:58.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f5691178700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f568c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f5691178700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0cca20 con 0x7f568c15a0c0 2023-10-10T20:25:58.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f5691178700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c15ea40 con 0x7f568c0cf1c0 2023-10-10T20:25:58.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f5691178700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c067660 con 0x7f568c159cf0 2023-10-10T20:25:58.261 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f568259c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f568c15a0c0 0x7f568c0ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33546/0 (socket says 172.21.15.148:33546) 2023-10-10T20:25:58.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.255+0000 7f568259c700 1 -- 172.21.15.148:0/3010603163 learned_addr learned my addr 172.21.15.148:0/3010603163 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:58.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f568c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058157996 0 0) 0x7f568c0cca20 con 0x7f568c15a0c0 2023-10-10T20:25:58.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5678003570 con 0x7f568c15a0c0 2023-10-10T20:25:58.262 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451472557 0 0) 0x7f5684001490 con 0x7f568c0cf1c0 2023-10-10T20:25:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568c0cca20 con 0x7f568c0cf1c0 2023-10-10T20:25:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428178338 0 0) 0x7f5678003570 con 0x7f568c15a0c0 2023-10-10T20:25:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 msgr2=0x7f568c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f568c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 >> v1:172.21.15.161:6789/0 conn(0x7f568c0cf1c0 legacy=0x7f568c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c067f00 con 0x7f568c15a0c0 2023-10-10T20:25:58.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f5674001470 con 0x7f568c15a0c0 2023-10-10T20:25:58.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 >> v1:172.21.15.148:6789/0 conn(0x7f568c15a0c0 legacy=0x7f568c0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678003c20 0x7f5678004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678004810 0x7f5678004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f5678005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c067660 con 0x7f5678003c20 2023-10-10T20:25:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567800bbf0 con 0x7f5678004810 2023-10-10T20:25:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567800bdb0 con 0x7f568c159cf0 2023-10-10T20:25:58.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678003c20 0x7f5678004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568259c700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678004810 0x7f5678004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568b59e700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f5678005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 -- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 msgr2=0x7f5678005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f5678005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 -- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678004810 msgr2=0x7f5678004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678004810 0x7f5678004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568ad9d700 1 -- 172.21.15.148:0/3010603163 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678003570 con 0x7f5678003c20 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.256+0000 7f568b59e700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c159cf0 0x7f5678005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f568ad9d700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678003c20 0x7f5678004010 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f567c009c00 tx=0x7f567c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb3273818ddbc86 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00e710 con 0x7f5678003c20 2023-10-10T20:25:58.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00e870 con 0x7f5678003c20 2023-10-10T20:25:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f568a59c700 1 -- 172.21.15.148:0/3010603163 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00ec30 con 0x7f5678003c20 2023-10-10T20:25:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 -- 172.21.15.148:0/3010603163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678003c20 msgr2=0x7f5678004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 --2- 172.21.15.148:0/3010603163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5678003c20 0x7f5678004010 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f567c009c00 tx=0x7f567c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f568b59e700 1 -- 172.21.15.148:0/3010603163 reap_dead start 2023-10-10T20:25:58.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 -- 172.21.15.148:0/3010603163 shutdown_connections 2023-10-10T20:25:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 -- 172.21.15.148:0/3010603163 >> 172.21.15.148:0/3010603163 conn(0x7f568c06c6e0 msgr2=0x7f568c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 -- 172.21.15.148:0/3010603163 shutdown_connections 2023-10-10T20:25:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.257+0000 7f5691178700 1 -- 172.21.15.148:0/3010603163 wait complete. 2023-10-10T20:25:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 Processor -- start 2023-10-10T20:25:58.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 -- start start 2023-10-10T20:25:58.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 0x7f568c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b3960 0x7f568c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c157e70 con 0x7f568c15c8d0 2023-10-10T20:25:58.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c6d60 con 0x7f5678003c20 2023-10-10T20:25:58.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f5691178700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0b28c0 con 0x7f568c0b3960 2023-10-10T20:25:58.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f568259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f568259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33840/0 (socket says 172.21.15.148:33840) 2023-10-10T20:25:58.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.259+0000 7f568259c700 1 -- 172.21.15.148:0/4203870723 learned_addr learned my addr 172.21.15.148:0/4203870723 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:58.273 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568ad9d700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 0x7f568c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568b59e700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b3960 0x7f568c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 -- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b3960 msgr2=0x7f568c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b3960 0x7f568c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 -- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 msgr2=0x7f568c157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.274 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 0x7f568c157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 -- 172.21.15.148:0/4203870723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0b2ba0 con 0x7f568c15c8d0 2023-10-10T20:25:58.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568ad9d700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 0x7f568c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:58.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f568259c700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5674001120 tx=0x7f5674001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674018030 con 0x7f568c15c8d0 2023-10-10T20:25:58.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567400fa70 con 0x7f568c15c8d0 2023-10-10T20:25:58.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0b2e80 con 0x7f568c15c8d0 2023-10-10T20:25:58.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400fe10 con 0x7f568c15c8d0 2023-10-10T20:25:58.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.260+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c0b3440 con 0x7f568c15c8d0 2023-10-10T20:25:58.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.261+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c04c3c0 con 0x7f568c15c8d0 2023-10-10T20:25:58.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.261+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f567401b070 con 0x7f568c15c8d0 2023-10-10T20:25:58.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.262+0000 7f56837fe700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.262+0000 7f568ad9d700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.262+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5674019070 con 0x7f568c15c8d0 2023-10-10T20:25:58.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.262+0000 7f568ad9d700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f567c0091e0 tx=0x7f567c01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.266+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5674051860 con 0x7f568c15c8d0 2023-10-10T20:25:58.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.451+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f568c0ce730 con 0x7f568c15c8d0 2023-10-10T20:25:58.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.452+0000 7f56837fe700 1 -- 172.21.15.148:0/4203870723 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f5674054e80 con 0x7f568c15c8d0 2023-10-10T20:25:58.454 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:58.455 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:58.455 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 134, 2023-10-10T20:25:58.455 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:58.455 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:58.455 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:55.532661+0000", 2023-10-10T20:25:58.456 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:58.456 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:58.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:58.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:58.456 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:58.457 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:58.457 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:58.457 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:58.457 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:58.457 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:58.458 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:58.459 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:58.459 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:58.459 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:58.459 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:58.459 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:58.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:58.460 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:58.460 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:58.460 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:58.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:58.461 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:58.461 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:58.461 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:58.461 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:58.462 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:58.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:58.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:58.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:58.462 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:58.463 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:58.464 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:58.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:58.464 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:58.464 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:58.464 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:58.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:58.465 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:58.465 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:58.465 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.465 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:58.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:58.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:58.466 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:58.466 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:58.466 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:58.467 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:58.467 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:58.467 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:58.467 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:58.467 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:58.468 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:58.472 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:58.473 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:58.473 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:58.473 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:58.473 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:58.473 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.474 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:58.474 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:58.474 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.474 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.474 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:58.475 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:58.476 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:58.476 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:58.476 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:58.476 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:58.476 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:58.477 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:58.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:58.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:58.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:58.478 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:58.478 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:58.479 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:58.480 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:58.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:58.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:58.480 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:58.481 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:58.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:58.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:58.481 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:58.481 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:58.482 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:58.482 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:58.482 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:58.482 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:58.482 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:58.483 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:58.483 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:58.483 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:58.483 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:58.483 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:58.484 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:58.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:58.485 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:58.485 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:58.485 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:58.486 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:58.486 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:58.486 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:58.486 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:58.486 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:58.487 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:58.487 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:58.487 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:58.487 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:58.487 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:58.488 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.489 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:58.491 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:58.491 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:58.491 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:58.491 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:58.491 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:58.492 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:58.492 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:58.492 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:58.492 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:58.492 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:58.493 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:58.494 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:58.495 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:58.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.495 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "134", 2023-10-10T20:25:58.495 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:58.495 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:58.496 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:58.496 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:58.496 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:58.496 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:58.496 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:58.497 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:58.498 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:58.498 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:58.498 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:58.498 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:58.498 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:58.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:58.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:58.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:58.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:58.499 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:58.500 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:58.500 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:58.500 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:58.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.500 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:58.501 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:58.501 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:58.501 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:58.501 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:58.501 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:58.502 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:58.502 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:58.502 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:58.502 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:58.502 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.503 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.505 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:58.505 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:58.505 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.505 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:58.505 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.506 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.506 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.506 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.506 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.506 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.507 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.507 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.507 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.507 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.508 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.508 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.508 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.508 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.508 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:58.509 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.510 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.510 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.510 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.510 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.510 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.511 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.511 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.511 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:58.511 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.512 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.512 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.512 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:58.512 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.512 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.515 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:58.517 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.517 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.517 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.517 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.517 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:58.518 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:58.518 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.518 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.518 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.519 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.520 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.520 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.520 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:58.520 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.521 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.521 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.521 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.521 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.521 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.522 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.522 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.522 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.522 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.522 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.523 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.523 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.523 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:58.524 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.524 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.524 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.524 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.525 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.526 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.526 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.526 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:58.526 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.527 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.527 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:58.527 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.527 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.529 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:58.529 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.529 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.529 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.531 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.531 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.531 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.531 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:58.532 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:58.533 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.533 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.533 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.533 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:58.533 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:58.534 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:58.534 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:58.534 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.534 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.534 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.535 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.536 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.536 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.536 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.536 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.536 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.537 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.537 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.537 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.537 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.537 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.539 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.539 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.539 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:58.539 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.540 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.541 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.542 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:58.542 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.542 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.542 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.543 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.543 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.543 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.543 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:58.544 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.544 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.544 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.544 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.545 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.546 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.546 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:58.546 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.547 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.547 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:58.547 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:58.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.549 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.549 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.550 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:58.551 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.552 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.552 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.552 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.552 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.552 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.553 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.553 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.553 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.553 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.553 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.554 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.554 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.554 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.554 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.554 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:58.555 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.555 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.555 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.555 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.556 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.557 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.557 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.557 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:58.557 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.558 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.558 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.558 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:58.558 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.558 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.559 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.559 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.559 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.559 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.560 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.560 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:58.560 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.560 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.561 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.561 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:58.561 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.561 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.562 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.562 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.562 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.562 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.563 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:58.564 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:58.564 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.564 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.564 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:58.565 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:58.565 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:58.565 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:58.565 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.565 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.566 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.566 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.566 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:58.567 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.567 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.567 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.567 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.567 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.568 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.569 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.569 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:58.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.570 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.570 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.571 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.572 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.573 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.573 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:58.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.573 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.574 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.574 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.575 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.575 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:58.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.575 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.575 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.577 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.578 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:58.579 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:58.580 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:58.580 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:58.580 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.580 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.580 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.581 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.581 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.581 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:58.581 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.582 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.583 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.583 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.583 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.584 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.584 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:58.584 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.585 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.585 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.585 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:58.585 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.586 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.586 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.586 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:58.589 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.589 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.589 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.589 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:58.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.592 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.592 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:58.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.594 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:58.594 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:58.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:58.595 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.596 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.596 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.596 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.596 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.597 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.598 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.598 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.598 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.598 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.598 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.599 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.600 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:58.600 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.600 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.600 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.601 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:58.602 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.603 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.603 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:58.603 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.604 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.604 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.604 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.604 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.605 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:58.605 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.606 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:58.606 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.606 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.608 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.609 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:58.609 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:58.609 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.609 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.610 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:58.610 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:58.610 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:58.610 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:58.610 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.611 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:58.612 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:58.613 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:58.613 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:58.613 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:58.613 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:58.613 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:58.614 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:58.614 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:58.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.614 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.614 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:58.616 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.616 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.616 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.616 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.618 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.618 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.618 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:58.618 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.618 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.619 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.619 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.619 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:58.619 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.619 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.621 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:58.621 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.621 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.621 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.622 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:58.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:58.622 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.622 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:58.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:58.623 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:58.624 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.624 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.624 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:58.624 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.626 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:58.627 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.627 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.627 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.627 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.628 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:58.628 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.628 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.629 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:58.630 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.630 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.630 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:58.630 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.631 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.631 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.631 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.631 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:58.632 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:58.633 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.634 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.634 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:58.634 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.634 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.635 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.635 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.635 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:58.635 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.636 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.637 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.637 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.637 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:58.637 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.638 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:58.638 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.638 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.638 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:58.640 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.641 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:58.642 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:58.642 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:58.642 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:58.642 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:58.642 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:58.643 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:58.643 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:58.643 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:58.643 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:58.643 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:58.645 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:58.645 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:58.645 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:58.645 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:58.645 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:58.646 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:58.646 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:58.646 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:58.646 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:58.646 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:58.647 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.647 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:58.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 msgr2=0x7f566c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f567c0091e0 tx=0x7f567c01a000 comp rx=0 tx=0).stop 2023-10-10T20:25:58.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 msgr2=0x7f568c15cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5674001120 tx=0x7f5674001d40 comp rx=0 tx=0).stop 2023-10-10T20:25:58.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 shutdown_connections 2023-10-10T20:25:58.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b3960 0x7f568c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5678003c20 0x7f568c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 --2- 172.21.15.148:0/4203870723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f568c15c8d0 0x7f568c15cd40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 >> 172.21.15.148:0/4203870723 conn(0x7f568c06c6e0 msgr2=0x7f568c0c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:58.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 shutdown_connections 2023-10-10T20:25:58.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.454+0000 7f5691178700 1 -- 172.21.15.148:0/4203870723 wait complete. 2023-10-10T20:25:58.651 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:25:58.652 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:25:58.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.764+0000 7f43fd1ce700 1 Processor -- start 2023-10-10T20:25:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.764+0000 7f43fd1ce700 1 -- start start 2023-10-10T20:25:58.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43fd1ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43f815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43fd1ce700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f815cbb0 con 0x7f43f815d580 2023-10-10T20:25:58.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43fd1ce700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f815bfa0 con 0x7f43f8157870 2023-10-10T20:25:58.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43fd1ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f815bdb0 con 0x7f43f815a620 2023-10-10T20:25:58.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43f759e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f43f815d580 0x7f43f8158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33556/0 (socket says 172.21.15.148:33556) 2023-10-10T20:25:58.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43f759e700 1 -- 172.21.15.148:0/454240488 learned_addr learned my addr 172.21.15.148:0/454240488 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:25:58.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43ee59c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43f815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229984902 0 0) 0x7f43f815cbb0 con 0x7f43f815d580 2023-10-10T20:25:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.765+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e4003570 con 0x7f43f815d580 2023-10-10T20:25:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164955144 0 0) 0x7f43e4003570 con 0x7f43f815d580 2023-10-10T20:25:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 msgr2=0x7f43f815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43f815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 >> v1:172.21.15.161:6789/0 conn(0x7f43f8157870 legacy=0x7f43f815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f815c280 con 0x7f43f815d580 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f43f00017f0 con 0x7f43f815d580 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 >> v1:172.21.15.148:6789/0 conn(0x7f43f815d580 legacy=0x7f43f8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43e4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4004010 0x7f43e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004d50 0x7f43e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e400bbf0 con 0x7f43e4004010 2023-10-10T20:25:58.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e400bd30 con 0x7f43e4004d50 2023-10-10T20:25:58.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e400bef0 con 0x7f43f815a620 2023-10-10T20:25:58.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4004010 0x7f43e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f759e700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004d50 0x7f43e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f6d9d700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43e4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 -- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 msgr2=0x7f43e4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f815a620 0x7f43e4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 -- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004d50 msgr2=0x7f43e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:58.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004d50 0x7f43e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 -- 172.21.15.148:0/454240488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4003570 con 0x7f43e4004010 2023-10-10T20:25:58.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f759e700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004d50 0x7f43e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:25:58.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43ee59c700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4004010 0x7f43e4004810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006f00 tx=0x7f43e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f9392a2a4cb916 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000e6e0 con 0x7f43e4004010 2023-10-10T20:25:58.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000e840 con 0x7f43e4004010 2023-10-10T20:25:58.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.766+0000 7f43f659c700 1 -- 172.21.15.148:0/454240488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000ec00 con 0x7f43e4004010 2023-10-10T20:25:58.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 -- 172.21.15.148:0/454240488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4004010 msgr2=0x7f43e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/454240488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4004010 0x7f43e4004810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006f00 tx=0x7f43e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43f759e700 1 -- 172.21.15.148:0/454240488 reap_dead start 2023-10-10T20:25:58.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 -- 172.21.15.148:0/454240488 shutdown_connections 2023-10-10T20:25:58.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 -- 172.21.15.148:0/454240488 >> 172.21.15.148:0/454240488 conn(0x7f43f8068ce0 msgr2=0x7f43f80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:58.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 -- 172.21.15.148:0/454240488 shutdown_connections 2023-10-10T20:25:58.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.767+0000 7f43fd1ce700 1 -- 172.21.15.148:0/454240488 wait complete. 2023-10-10T20:25:58.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.768+0000 7f43fd1ce700 1 Processor -- start 2023-10-10T20:25:58.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.768+0000 7f43fd1ce700 1 -- start start 2023-10-10T20:25:58.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004010 0x7f43f814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f816d7d0 0x7f43f80d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f814b6f0 con 0x7f43f814b920 2023-10-10T20:25:58.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f816dd90 con 0x7f43e4004010 2023-10-10T20:25:58.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43fd1ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f816e070 con 0x7f43f816d7d0 2023-10-10T20:25:58.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33856/0 (socket says 172.21.15.148:33856) 2023-10-10T20:25:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 -- 172.21.15.148:0/1526419816 learned_addr learned my addr 172.21.15.148:0/1526419816 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43f6d9d700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004010 0x7f43f814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43f759e700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f816d7d0 0x7f43f80d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 -- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f816d7d0 msgr2=0x7f43f80d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f816d7d0 0x7f43f80d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 -- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004010 msgr2=0x7f43f814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:58.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004010 0x7f43f814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:58.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.769+0000 7f43ee59c700 1 -- 172.21.15.148:0/1526419816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f816e350 con 0x7f43f814b920 2023-10-10T20:25:58.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43ee59c700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f43e00091e0 tx=0x7f43e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0022070 con 0x7f43f814b920 2023-10-10T20:25:58.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000e9b0 con 0x7f43f814b920 2023-10-10T20:25:58.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f814b6f0 con 0x7f43f814b920 2023-10-10T20:25:58.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0027470 con 0x7f43f814b920 2023-10-10T20:25:58.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.770+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f8159280 con 0x7f43f814b920 2023-10-10T20:25:58.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.771+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f80b0a20 con 0x7f43f814b920 2023-10-10T20:25:58.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.771+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43e00278c0 con 0x7f43f814b920 2023-10-10T20:25:58.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.772+0000 7f43ef7fe700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:58.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.772+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43e0085f00 con 0x7f43f814b920 2023-10-10T20:25:58.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.772+0000 7f43f6d9d700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:58.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.772+0000 7f43f6d9d700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f43e8007fd0 tx=0x7f43e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:58.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.776+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e00532e0 con 0x7f43f814b920 2023-10-10T20:25:58.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.960+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f43f804c3c0 con 0x7f43f814b920 2023-10-10T20:25:58.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.961+0000 7f43ef7fe700 1 -- 172.21.15.148:0/1526419816 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f43e0017090 con 0x7f43f814b920 2023-10-10T20:25:58.964 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:58.964 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:25:58.964 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 134, 2023-10-10T20:25:58.964 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:25:58.964 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:25:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:25:55.532661+0000", 2023-10-10T20:25:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:25:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:25:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:25:58.965 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:25:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:25:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:25:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:25:58.966 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:25:58.967 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:25:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:25:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 17, 2023-10-10T20:25:58.968 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:25:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:25:58.969 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:25:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:25:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:25:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:25:58.970 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:25:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:25:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:58.971 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:25:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:25:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:25:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:25:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:25:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:25:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:58.976 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:58.977 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 16, 2023-10-10T20:25:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:25:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.369290+0000", 2023-10-10T20:25:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:25:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:25:58.989 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:25:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:25:58.990 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:25:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:58.991 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "130", 2023-10-10T20:25:58.992 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:58.993 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:25:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:58.994 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:58.995 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:58.996 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:58.997 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:58.998 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:58.999 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:59.000 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:59.001 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:25:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:25:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:25:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:59.002 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.003 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 17, 2023-10-10T20:25:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:25:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:25:22.981299+0000", 2023-10-10T20:25:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:25:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:25:59.004 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:25:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:25:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:25:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:25:59.005 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:25:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:25:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:25:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:25:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:25:59.006 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:25:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:25:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:25:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:25:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:25:59.007 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:25:59.008 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:25:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:25:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "134", 2023-10-10T20:25:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:25:59.009 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:25:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:25:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:25:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:25:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:25:59.010 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:25:59.011 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:25:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:25:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:25:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:25:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:25:59.012 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:25:59.013 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:25:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:25:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:25:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:25:59.014 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:25:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:25:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:25:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:25:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:25:59.015 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:25:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:25:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:25:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:25:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:25:59.016 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:25:59.017 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:25:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:59.018 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:25:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.019 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.020 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:25:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.022 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:25:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.023 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.024 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:25:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.025 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.026 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:25:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:25:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.029 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:25:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.030 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.031 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:25:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:25:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:25:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:25:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.032 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.033 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.034 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.035 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:25:59.036 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.037 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:25:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.038 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:25:59.039 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:25:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.042 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.044 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:25:59.045 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:25:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:25:59.046 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:25:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:25:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:25:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.047 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.048 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.049 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.050 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:25:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.051 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:25:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.052 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.053 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:25:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.054 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.055 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:25:59.056 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.057 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.058 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:25:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.059 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.060 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:25:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:25:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:25:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:25:59.061 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.062 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.063 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.064 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.065 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:25:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.066 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.067 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:25:59.068 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:25:59.069 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.070 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:25:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.071 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.072 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.073 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:25:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.074 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:25:59.075 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:25:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:25:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:25:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.076 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.077 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.078 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.079 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:25:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.080 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:25:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.081 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.082 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:25:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.083 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.084 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:25:59.085 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.086 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.087 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:25:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.088 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.089 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:25:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:25:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:25:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:25:59.090 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.091 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.092 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.093 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.094 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:25:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.095 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.096 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:25:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.098 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.099 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.100 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:25:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.101 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.102 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:25:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.103 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:25:59.104 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:25:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:25:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:25:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.105 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:59.106 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.107 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.108 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:25:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.110 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.111 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.112 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:25:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:25:59.114 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:25:59.115 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.116 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:25:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.117 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:25:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:25:59.118 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:25:59.119 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:25:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:25:59.121 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:25:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:25:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:25:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:25:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:25:59.122 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 129, 2023-10-10T20:25:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:25:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:25:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:25:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.123 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.124 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:25:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.126 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:25:59.127 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:25:59.128 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.129 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:25:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.130 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:25:59.131 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:25:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:25:59.132 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:25:59.133 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:25:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:25:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:25:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:25:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:25:59.134 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:25:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:25:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:25:59.135 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.136 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:25:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.137 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:25:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.138 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:25:59.139 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.140 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:25:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:25:59.141 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.142 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.143 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:25:59.144 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:25:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.145 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:25:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.147 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:25:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:25:59.149 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:25:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:25:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:25:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:25:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.150 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:25:59.151 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:25:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:25:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:25:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:25:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:25:59.152 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:25:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:25:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:25:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:25:59.153 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:25:59.154 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:25:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:25:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:25:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:25:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:25:59.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:25:59.156 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:25:59.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 msgr2=0x7f43d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f43e8007fd0 tx=0x7f43e8009040 comp rx=0 tx=0).stop 2023-10-10T20:25:59.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 msgr2=0x7f43f80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f43e00091e0 tx=0x7f43e001a000 comp rx=0 tx=0).stop 2023-10-10T20:25:59.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 shutdown_connections 2023-10-10T20:25:59.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43f816d7d0 0x7f43f80d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4004010 0x7f43f814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 --2- 172.21.15.148:0/1526419816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43f814b920 0x7f43f80cfb20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 >> 172.21.15.148:0/1526419816 conn(0x7f43f8068ce0 msgr2=0x7f43f80b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 shutdown_connections 2023-10-10T20:25:59.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:58.963+0000 7f43fd1ce700 1 -- 172.21.15.148:0/1526419816 wait complete. 2023-10-10T20:25:59.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:25:59.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.272+0000 7fc266f79700 1 Processor -- start 2023-10-10T20:25:59.275 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.272+0000 7fc266f79700 1 -- start start 2023-10-10T20:25:59.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.273+0000 7fc266f79700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 0x7fc26006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.273+0000 7fc266f79700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2600c9af0 0x7fc26015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.273+0000 7fc266f79700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26015e280 con 0x7fc260070700 2023-10-10T20:25:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.273+0000 7fc266f79700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2600ca0b0 con 0x7fc2600c9af0 2023-10-10T20:25:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.273+0000 7fc266f79700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc260068a10 con 0x7fc2601510f0 2023-10-10T20:25:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc264d15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 0x7fc26006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2600c9af0 0x7fc26015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2600c9af0 0x7fc26015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:40230/0 (socket says 172.21.15.148:40230) 2023-10-10T20:25:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc264d15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 0x7fc26006ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:43074/0 (socket says 172.21.15.148:43074) 2023-10-10T20:25:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc265516700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc260070700 0x7fc2600afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:52146/0 (socket says 172.21.15.148:52146) 2023-10-10T20:25:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 -- 172.21.15.148:0/441831694 learned_addr learned my addr 172.21.15.148:0/441831694 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 -- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 msgr2=0x7fc26006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:25:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 --2- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 0x7fc26006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 -- 172.21.15.148:0/441831694 >> v1:172.21.15.161:6789/0 conn(0x7fc260070700 legacy=0x7fc2600afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 -- 172.21.15.148:0/441831694 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260068cf0 con 0x7fc2600c9af0 2023-10-10T20:25:59.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc2577fe700 1 -- 172.21.15.148:0/441831694 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088159361 0 0) 0x7fc26015e280 con 0x7fc260070700 2023-10-10T20:25:59.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc257fff700 1 --2- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2600c9af0 0x7fc26015dd40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2500099b0 tx=0x7fc250009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da2bd6b57fb30d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:59.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc2577fe700 1 -- 172.21.15.148:0/441831694 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250009ea0 con 0x7fc2600c9af0 2023-10-10T20:25:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc2577fe700 1 -- 172.21.15.148:0/441831694 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000b860 con 0x7fc2600c9af0 2023-10-10T20:25:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.274+0000 7fc2577fe700 1 -- 172.21.15.148:0/441831694 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000bc00 con 0x7fc2600c9af0 2023-10-10T20:25:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 -- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2600c9af0 msgr2=0x7fc26015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 --2- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2600c9af0 0x7fc26015dd40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2500099b0 tx=0x7fc250009250 comp rx=0 tx=0).stop 2023-10-10T20:25:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 -- 172.21.15.148:0/441831694 shutdown_connections 2023-10-10T20:25:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 --2- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2601510f0 0x7fc26006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 --2- 172.21.15.148:0/441831694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2600c9af0 0x7fc26015dd40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 -- 172.21.15.148:0/441831694 >> 172.21.15.148:0/441831694 conn(0x7fc2600ca4f0 msgr2=0x7fc2600ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 -- 172.21.15.148:0/441831694 shutdown_connections 2023-10-10T20:25:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.275+0000 7fc266f79700 1 -- 172.21.15.148:0/441831694 wait complete. 2023-10-10T20:25:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.276+0000 7fc266f79700 1 Processor -- start 2023-10-10T20:25:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.276+0000 7fc266f79700 1 -- start start 2023-10-10T20:25:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 0x7fc2600b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2601510f0 0x7fc26016dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600b3830 con 0x7fc2600c9af0 2023-10-10T20:25:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2601599f0 con 0x7fc2601510f0 2023-10-10T20:25:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc266f79700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc260159e80 con 0x7fc260070700 2023-10-10T20:25:59.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc257fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc257fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33864/0 (socket says 172.21.15.148:33864) 2023-10-10T20:25:59.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 0x7fc2600b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc257fff700 1 -- 172.21.15.148:0/1600395034 learned_addr learned my addr 172.21.15.148:0/1600395034 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:25:59.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc265516700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2601510f0 0x7fc26016dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 -- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2601510f0 msgr2=0x7fc26016dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2601510f0 0x7fc26016dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 -- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 msgr2=0x7fc260154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc264d15700 1 -- 172.21.15.148:0/1600395034 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500094f0 con 0x7fc260070700 2023-10-10T20:25:59.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.277+0000 7fc257fff700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:25:59.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc264d15700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 0x7fc2600b32f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ed60 tx=0x7fc25c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:59.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00eec0 con 0x7fc260070700 2023-10-10T20:25:59.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c00f040 con 0x7fc260070700 2023-10-10T20:25:59.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc260158de0 con 0x7fc260070700 2023-10-10T20:25:59.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00ccd0 con 0x7fc260070700 2023-10-10T20:25:59.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.278+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2601593a0 con 0x7fc260070700 2023-10-10T20:25:59.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.279+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc260159cd0 con 0x7fc260070700 2023-10-10T20:25:59.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.279+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc25c014690 con 0x7fc260070700 2023-10-10T20:25:59.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.279+0000 7fc254ff9700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 0x7fc24c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:25:59.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.279+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc25c084f90 con 0x7fc260070700 2023-10-10T20:25:59.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.280+0000 7fc257fff700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 0x7fc24c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:25:59.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.280+0000 7fc257fff700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 0x7fc24c0693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b0b30 tx=0x7fc250017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:25:59.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.284+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc25c052200 con 0x7fc260070700 2023-10-10T20:25:59.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.508+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc26004c3c0 con 0x7fc260070700 2023-10-10T20:25:59.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.508+0000 7fc254ff9700 1 -- 172.21.15.148:0/1600395034 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fc25c055820 con 0x7fc260070700 2023-10-10T20:25:59.511 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:25:59.511 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:25:59.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 msgr2=0x7fc24c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 0x7fc24c0693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b0b30 tx=0x7fc250017000 comp rx=0 tx=0).stop 2023-10-10T20:25:59.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 msgr2=0x7fc2600b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:25:59.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 0x7fc2600b32f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ed60 tx=0x7fc25c00b040 comp rx=0 tx=0).stop 2023-10-10T20:25:59.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 shutdown_connections 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc24c066f30 0x7fc24c0693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2601510f0 0x7fc26016dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc260070700 0x7fc2600b32f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 --2- 172.21.15.148:0/1600395034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2600c9af0 0x7fc260154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 >> 172.21.15.148:0/1600395034 conn(0x7fc2600ca4f0 msgr2=0x7fc26015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:25:59.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 shutdown_connections 2023-10-10T20:25:59.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:25:59.511+0000 7fc266f79700 1 -- 172.21.15.148:0/1600395034 wait complete. 2023-10-10T20:25:59.516 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:25:59.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-10T20:25:59.667 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":7095,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.997164224999999,"mdsmap_epoch":64,"osdmap_epoch":134,"osdmap_epoch_barrier":133,"uptime":40.113377876999998} 2023-10-10T20:25:59.668 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 7095, "mdsmap_epoch": 64, "osdmap_epoch": 134, "osdmap_epoch_barrier": 133, "rank_uptime": 34.997164225, "state": "up:active", "uptime": 40.113377877, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:25:59.668 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:25:59.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:25:59.669 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-10T20:25:59.949 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:25:59.950 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:25:59.951 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.26351 s, 199 MB/s 2023-10-10T20:25:59.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:25:59.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:25:59.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:26:00.031 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:26:00.032 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:26:00.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:26:00.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:26:00.089 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:26:00.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:00.090 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:26:00.112 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:26:00.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:26:00.112 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:26:00.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:26:00.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:00.154 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:26:00.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 Processor -- start 2023-10-10T20:26:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 -- start start 2023-10-10T20:26:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f50bc15a2b0 0x7f50bc0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50bc0d1c30 0x7f50bc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0c5d80 con 0x7f50bc15a680 2023-10-10T20:26:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc068470 con 0x7f50bc0d1c30 2023-10-10T20:26:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.322+0000 7f50c226f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0c6060 con 0x7f50bc15a2b0 2023-10-10T20:26:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50bc0d1c30 0x7f50bc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50c080c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50bc15a680 0x7f50bc0c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33558/0 (socket says 172.21.15.148:33558) 2023-10-10T20:26:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50c080c700 1 -- 172.21.15.148:0/1806005034 learned_addr learned my addr 172.21.15.148:0/1806005034 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bbfff700 1 --2- 172.21.15.148:0/1806005034 >> v2:172.21.15.161:3300/0 conn(0x7f50bc15a2b0 0x7f50bc0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 -- 172.21.15.148:0/1806005034 >> v2:172.21.15.161:3300/0 conn(0x7f50bc15a2b0 msgr2=0x7f50bc0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 --2- 172.21.15.148:0/1806005034 >> v2:172.21.15.161:3300/0 conn(0x7f50bc15a2b0 0x7f50bc0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 -- 172.21.15.148:0/1806005034 >> v1:172.21.15.148:6789/0 conn(0x7f50bc15a680 legacy=0x7f50bc0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 -- 172.21.15.148:0/1806005034 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0c6340 con 0x7f50bc0d1c30 2023-10-10T20:26:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50baffd700 1 -- 172.21.15.148:0/1806005034 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586584836 0 0) 0x7f50bc0c5d80 con 0x7f50bc15a680 2023-10-10T20:26:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50bb7fe700 1 --2- 172.21.15.148:0/1806005034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc0d1c30 0x7f50bc0cf710 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f50bc089220 tx=0x7f50a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1656ec2ef0334a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50baffd700 1 -- 172.21.15.148:0/1806005034 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011630 con 0x7f50bc0d1c30 2023-10-10T20:26:00.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50baffd700 1 -- 172.21.15.148:0/1806005034 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a4011790 con 0x7f50bc0d1c30 2023-10-10T20:26:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.323+0000 7f50baffd700 1 -- 172.21.15.148:0/1806005034 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011b30 con 0x7f50bc0d1c30 2023-10-10T20:26:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 -- 172.21.15.148:0/1806005034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc0d1c30 msgr2=0x7f50bc0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 --2- 172.21.15.148:0/1806005034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc0d1c30 0x7f50bc0cf710 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f50bc089220 tx=0x7f50a40091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 -- 172.21.15.148:0/1806005034 shutdown_connections 2023-10-10T20:26:00.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 --2- 172.21.15.148:0/1806005034 >> v2:172.21.15.161:3300/0 conn(0x7f50bc15a2b0 0x7f50bc0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 --2- 172.21.15.148:0/1806005034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc0d1c30 0x7f50bc0cf710 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 -- 172.21.15.148:0/1806005034 >> 172.21.15.148:0/1806005034 conn(0x7f50bc067e40 msgr2=0x7f50bc0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 -- 172.21.15.148:0/1806005034 shutdown_connections 2023-10-10T20:26:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.324+0000 7f50c226f700 1 -- 172.21.15.148:0/1806005034 wait complete. 2023-10-10T20:26:00.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.325+0000 7f50c226f700 1 Processor -- start 2023-10-10T20:26:00.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.325+0000 7f50c226f700 1 -- start start 2023-10-10T20:26:00.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 0x7f50bc15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc15a680 0x7f50bc16a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15db80 con 0x7f50bc15a2b0 2023-10-10T20:26:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15f880 con 0x7f50bc15a680 2023-10-10T20:26:00.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c226f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc15bf50 con 0x7f50bc0d1c30 2023-10-10T20:26:00.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33874/0 (socket says 172.21.15.148:33874) 2023-10-10T20:26:00.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 -- 172.21.15.148:0/2244912238 learned_addr learned my addr 172.21.15.148:0/2244912238 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:00.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50c080c700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc15a680 0x7f50bc16a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bbfff700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 0x7f50bc15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 -- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 msgr2=0x7f50bc15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 0x7f50bc15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 -- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc15a680 msgr2=0x7f50bc16a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc15a680 0x7f50bc16a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 -- 172.21.15.148:0/2244912238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4009420 con 0x7f50bc15a2b0 2023-10-10T20:26:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bbfff700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 0x7f50bc15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.326+0000 7f50bb7fe700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f50a4013000 tx=0x7f50a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011b10 con 0x7f50bc15a2b0 2023-10-10T20:26:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a40253f0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc15c230 con 0x7f50bc15a2b0 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4025790 con 0x7f50bc15a2b0 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1584e0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.327+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc15f6e0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.328+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50a4025b80 con 0x7f50bc15a2b0 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.328+0000 7f50b3fff700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.328+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50a4085d50 con 0x7f50bc15a2b0 2023-10-10T20:26:00.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.328+0000 7f50bbfff700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.329+0000 7f50bbfff700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 0x7f509c0693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50bc069f90 tx=0x7f50ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.333+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50bc15f6e0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.557+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50bc15f6e0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.557+0000 7f50b3fff700 1 -- 172.21.15.148:0/2244912238 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f50bc15f6e0 con 0x7f50bc15a2b0 2023-10-10T20:26:00.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:00.560 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:26:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 msgr2=0x7f509c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 0x7f509c0693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50bc069f90 tx=0x7f50ac008040 comp rx=0 tx=0).stop 2023-10-10T20:26:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 msgr2=0x7f50bc169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f50a4013000 tx=0x7f50a4017000 comp rx=0 tx=0).stop 2023-10-10T20:26:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 shutdown_connections 2023-10-10T20:26:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f509c066f30 0x7f509c0693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc15a680 0x7f50bc16a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50bc0d1c30 0x7f50bc15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 --2- 172.21.15.148:0/2244912238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc15a2b0 0x7f50bc169fe0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 >> 172.21.15.148:0/2244912238 conn(0x7f50bc067e40 msgr2=0x7f50bc0cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 shutdown_connections 2023-10-10T20:26:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.559+0000 7f50c226f700 1 -- 172.21.15.148:0/2244912238 wait complete. 2023-10-10T20:26:00.564 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:26:00.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:00.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.686+0000 7f966081b700 1 Processor -- start 2023-10-10T20:26:00.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.686+0000 7f966081b700 1 -- start start 2023-10-10T20:26:00.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f966081b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f965c0cd6e0 0x7f965c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f966081b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f966081b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0cb7c0 con 0x7f965c0cdab0 2023-10-10T20:26:00.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f966081b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c15e850 con 0x7f965c0cd6e0 2023-10-10T20:26:00.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f966081b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0cd030 con 0x7f965c15cbd0 2023-10-10T20:26:00.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f965c0cd6e0 0x7f965c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:43098/0 (socket says 172.21.15.148:43098) 2023-10-10T20:26:00.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965affd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f965c0cdab0 0x7f965c15d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:52148/0 (socket says 172.21.15.148:52148) 2023-10-10T20:26:00.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965bfff700 1 -- 172.21.15.148:0/3849624643 learned_addr learned my addr 172.21.15.148:0/3849624643 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:00.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965b7fe700 1 -- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 msgr2=0x7f965c15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:00.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965b7fe700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965b7fe700 1 -- 172.21.15.148:0/3849624643 >> v1:172.21.15.161:6789/0 conn(0x7f965c0cdab0 legacy=0x7f965c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965b7fe700 1 -- 172.21.15.148:0/3849624643 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c15e290 con 0x7f965c0cd6e0 2023-10-10T20:26:00.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965bfff700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:00.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.687+0000 7f965a7fc700 1 -- 172.21.15.148:0/3849624643 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005128221 0 0) 0x7f965c0cb7c0 con 0x7f965c0cdab0 2023-10-10T20:26:00.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f965b7fe700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c0cd6e0 0x7f965c0cbc50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f964c009e60 tx=0x7f964c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9476729886e38d69 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f965a7fc700 1 -- 172.21.15.148:0/3849624643 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00e070 con 0x7f965c0cd6e0 2023-10-10T20:26:00.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f965a7fc700 1 -- 172.21.15.148:0/3849624643 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00bb60 con 0x7f965c0cd6e0 2023-10-10T20:26:00.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f965a7fc700 1 -- 172.21.15.148:0/3849624643 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c012430 con 0x7f965c0cd6e0 2023-10-10T20:26:00.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 -- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c0cd6e0 msgr2=0x7f965c0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c0cd6e0 0x7f965c0cbc50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f964c009e60 tx=0x7f964c009780 comp rx=0 tx=0).stop 2023-10-10T20:26:00.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 -- 172.21.15.148:0/3849624643 shutdown_connections 2023-10-10T20:26:00.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c0cd6e0 0x7f965c0cbc50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 --2- 172.21.15.148:0/3849624643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c15cbd0 0x7f965c15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 -- 172.21.15.148:0/3849624643 >> 172.21.15.148:0/3849624643 conn(0x7f965c067660 msgr2=0x7f965c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:00.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 -- 172.21.15.148:0/3849624643 shutdown_connections 2023-10-10T20:26:00.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.688+0000 7f966081b700 1 -- 172.21.15.148:0/3849624643 wait complete. 2023-10-10T20:26:00.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.689+0000 7f966081b700 1 Processor -- start 2023-10-10T20:26:00.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 -- start start 2023-10-10T20:26:00.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c0cdab0 0x7f965c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c15cbd0 0x7f965c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c155bf0 con 0x7f965c0cd6e0 2023-10-10T20:26:00.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15c4c0 con 0x7f965c15cbd0 2023-10-10T20:26:00.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f966081b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15baf0 con 0x7f965c0cdab0 2023-10-10T20:26:00.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f965b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f965b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33884/0 (socket says 172.21.15.148:33884) 2023-10-10T20:26:00.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f965b7fe700 1 -- 172.21.15.148:0/2989933366 learned_addr learned my addr 172.21.15.148:0/2989933366 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:00.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f965affd700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c0cdab0 0x7f965c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.690+0000 7f965bfff700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c15cbd0 0x7f965c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 -- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c0cdab0 msgr2=0x7f965c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c0cdab0 0x7f965c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 -- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c15cbd0 msgr2=0x7f965c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c15cbd0 0x7f965c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 -- 172.21.15.148:0/2989933366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540094f0 con 0x7f965c0cd6e0 2023-10-10T20:26:00.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f965b7fe700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b6c0 tx=0x7f964c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c012cb0 con 0x7f965c0cd6e0 2023-10-10T20:26:00.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c012e30 con 0x7f965c0cd6e0 2023-10-10T20:26:00.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c009a20 con 0x7f965c0cd6e0 2023-10-10T20:26:00.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0276c0 con 0x7f965c0cd6e0 2023-10-10T20:26:00.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.691+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c15b250 con 0x7f965c0cd6e0 2023-10-10T20:26:00.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.692+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c0b0260 con 0x7f965c0cd6e0 2023-10-10T20:26:00.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.692+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f964c024070 con 0x7f965c0cd6e0 2023-10-10T20:26:00.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.693+0000 7f96437fe700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:00.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.693+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f964c086a20 con 0x7f965c0cd6e0 2023-10-10T20:26:00.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.693+0000 7f965affd700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:00.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.693+0000 7f965affd700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 0x7f96480693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f965c16d9a0 tx=0x7f9650007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:00.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.697+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f964c053c90 con 0x7f965c0cd6e0 2023-10-10T20:26:00.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.881+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f965c15e570 con 0x7f965c0cd6e0 2023-10-10T20:26:00.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.882+0000 7f96437fe700 1 -- 172.21.15.148:0/2989933366 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f964c0572b0 con 0x7f965c0cd6e0 2023-10-10T20:26:00.884 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:00.884 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:55.532661+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.883+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 msgr2=0x7f96480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 0x7f96480693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f965c16d9a0 tx=0x7f9650007d50 comp rx=0 tx=0).stop 2023-10-10T20:26:00.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 msgr2=0x7f965c15bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b6c0 tx=0x7f964c019000 comp rx=0 tx=0).stop 2023-10-10T20:26:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 shutdown_connections 2023-10-10T20:26:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9648066f30 0x7f96480693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c15cbd0 0x7f965c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f965c0cd6e0 0x7f965c15bf80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 --2- 172.21.15.148:0/2989933366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f965c0cdab0 0x7f965c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 >> 172.21.15.148:0/2989933366 conn(0x7f965c067660 msgr2=0x7f965c157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 shutdown_connections 2023-10-10T20:26:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:00.884+0000 7f966081b700 1 -- 172.21.15.148:0/2989933366 wait complete. 2023-10-10T20:26:00.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:25:22.369290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:25:22.981299+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:26:00.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:26:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.014+0000 7ff2e23a7700 1 Processor -- start 2023-10-10T20:26:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.014+0000 7ff2e23a7700 1 -- start start 2023-10-10T20:26:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e23a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 0x7ff2dc06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e23a7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff2dc06aae0 0x7ff2dc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e23a7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0b0790 con 0x7ff2dc159ec0 2023-10-10T20:26:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e23a7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc06af60 con 0x7ff2dc06aae0 2023-10-10T20:26:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e23a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0ce1f0 con 0x7ff2dc15a290 2023-10-10T20:26:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2dbfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff2dc159ec0 0x7ff2dc0af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33568/0 (socket says 172.21.15.148:33568) 2023-10-10T20:26:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2dbfff700 1 -- 172.21.15.148:0/1627974297 learned_addr learned my addr 172.21.15.148:0/1627974297 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2db7fe700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 0x7ff2dc06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2e0944700 1 --2- 172.21.15.148:0/1627974297 >> v2:172.21.15.161:3300/0 conn(0x7ff2dc06aae0 0x7ff2dc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.015+0000 7ff2daffd700 1 -- 172.21.15.148:0/1627974297 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106036242 0 0) 0x7ff2dc0b0790 con 0x7ff2dc159ec0 2023-10-10T20:26:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2daffd700 1 -- 172.21.15.148:0/1627974297 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c4003570 con 0x7ff2dc159ec0 2023-10-10T20:26:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e0944700 1 -- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 msgr2=0x7ff2dc06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e0944700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 0x7ff2dc06a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e0944700 1 -- 172.21.15.148:0/1627974297 >> v1:172.21.15.148:6789/0 conn(0x7ff2dc159ec0 legacy=0x7ff2dc0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e0944700 1 -- 172.21.15.148:0/1627974297 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0b0b20 con 0x7ff2dc06aae0 2023-10-10T20:26:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2db7fe700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 0x7ff2dc06a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e0944700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc0cf590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e60 tx=0x7ff2d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666d8219a53c592e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2daffd700 1 -- 172.21.15.148:0/1627974297 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400e070 con 0x7ff2dc06aae0 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2daffd700 1 -- 172.21.15.148:0/1627974297 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400bab0 con 0x7ff2dc06aae0 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2daffd700 1 -- 172.21.15.148:0/1627974297 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400be70 con 0x7ff2dc06aae0 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e23a7700 1 -- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 msgr2=0x7ff2dc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc0cf590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e60 tx=0x7ff2d4009780 comp rx=0 tx=0).stop 2023-10-10T20:26:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e23a7700 1 -- 172.21.15.148:0/1627974297 shutdown_connections 2023-10-10T20:26:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.016+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc0cf590 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.017+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/1627974297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc15a290 0x7ff2dc06a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.017+0000 7ff2e23a7700 1 -- 172.21.15.148:0/1627974297 >> 172.21.15.148:0/1627974297 conn(0x7ff2dc0d1880 msgr2=0x7ff2dc0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.017+0000 7ff2e23a7700 1 -- 172.21.15.148:0/1627974297 shutdown_connections 2023-10-10T20:26:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.017+0000 7ff2e23a7700 1 -- 172.21.15.148:0/1627974297 wait complete. 2023-10-10T20:26:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.018+0000 7ff2e23a7700 1 Processor -- start 2023-10-10T20:26:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.018+0000 7ff2e23a7700 1 -- start start 2023-10-10T20:26:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.018+0000 7ff2e23a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.018+0000 7ff2e23a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc159ec0 0x7ff2dc15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc159630 con 0x7ff2dc15a290 2023-10-10T20:26:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc15c7b0 con 0x7ff2dc06aae0 2023-10-10T20:26:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc15c9c0 con 0x7ff2dc159ec0 2023-10-10T20:26:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33886/0 (socket says 172.21.15.148:33886) 2023-10-10T20:26:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 -- 172.21.15.148:0/3647945665 learned_addr learned my addr 172.21.15.148:0/3647945665 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2dbfff700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2db7fe700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc159ec0 0x7ff2dc15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 -- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc159ec0 msgr2=0x7ff2dc15c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc159ec0 0x7ff2dc15c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 -- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 msgr2=0x7ff2dc1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc1590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 -- 172.21.15.148:0/3647945665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0009420 con 0x7ff2dc15a290 2023-10-10T20:26:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2dbfff700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e0944700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc071eb0 tx=0x7ff2d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4012e30 con 0x7ff2dc15a290 2023-10-10T20:26:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4027470 con 0x7ff2dc15a290 2023-10-10T20:26:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4009a20 con 0x7ff2dc15a290 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4027830 con 0x7ff2dc15a290 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.019+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc15d260 con 0x7ff2dc15a290 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.020+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc04c3c0 con 0x7ff2dc15a290 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.021+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2d4024070 con 0x7ff2dc15a290 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.021+0000 7ff2c3fff700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 0x7ff2c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.021+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2d4086b10 con 0x7ff2dc15a290 2023-10-10T20:26:01.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.021+0000 7ff2dbfff700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 0x7ff2c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.021+0000 7ff2dbfff700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 0x7ff2c80693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007fd0 tx=0x7ff2cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.025+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2d4053d80 con 0x7ff2dc15a290 2023-10-10T20:26:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.206+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2dc0b04b0 con 0x7ff2dc15a290 2023-10-10T20:26:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.207+0000 7ff2c3fff700 1 -- 172.21.15.148:0/3647945665 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2d40573a0 con 0x7ff2dc15a290 2023-10-10T20:26:01.209 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:26:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 msgr2=0x7ff2c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.211 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 0x7ff2c80693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007fd0 tx=0x7ff2cc009040 comp rx=0 tx=0).stop 2023-10-10T20:26:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 msgr2=0x7ff2dc156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc071eb0 tx=0x7ff2d4019000 comp rx=0 tx=0).stop 2023-10-10T20:26:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 shutdown_connections 2023-10-10T20:26:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff2c8066f30 0x7ff2c80693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2dc15a290 0x7ff2dc156cd0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc06aae0 0x7ff2dc1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 --2- 172.21.15.148:0/3647945665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2dc159ec0 0x7ff2dc15c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 >> 172.21.15.148:0/3647945665 conn(0x7ff2dc0d1880 msgr2=0x7ff2dc0c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 shutdown_connections 2023-10-10T20:26:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.209+0000 7ff2e23a7700 1 -- 172.21.15.148:0/3647945665 wait complete. 2023-10-10T20:26:01.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:26:01.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.339+0000 7f484359e700 1 Processor -- start 2023-10-10T20:26:01.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.339+0000 7f484359e700 1 -- start start 2023-10-10T20:26:01.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48440b6580 0x7f484406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484359e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f484406a830 0x7f48440b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484359e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48440b4d70 con 0x7f48440cce30 2023-10-10T20:26:01.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484359e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484406ac70 con 0x7f484406a830 2023-10-10T20:26:01.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484415b3b0 con 0x7f48440b6580 2023-10-10T20:26:01.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484259c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f48440cce30 0x7f484415aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33582/0 (socket says 172.21.15.148:33582) 2023-10-10T20:26:01.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f484406a830 0x7f48440b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484259c700 1 -- 172.21.15.148:0/2928894078 learned_addr learned my addr 172.21.15.148:0/2928894078 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:01.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4841d9b700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48440b6580 0x7f484406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 -- 172.21.15.148:0/2928894078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48440b6580 msgr2=0x7f484406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:01.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48440b6580 0x7f484406a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 -- 172.21.15.148:0/2928894078 >> v1:172.21.15.148:6789/0 conn(0x7f48440cce30 legacy=0x7f484415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 -- 172.21.15.148:0/2928894078 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440cd8d0 con 0x7f484406a830 2023-10-10T20:26:01.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484159a700 1 -- 172.21.15.148:0/2928894078 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147487213 0 0) 0x7f48440b4d70 con 0x7f48440cce30 2023-10-10T20:26:01.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f4842d9d700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f484406a830 0x7f48440b5200 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098e0 tx=0x7f483c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d7ef1b7dc9e938 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484159a700 1 -- 172.21.15.148:0/2928894078 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c009e80 con 0x7f484406a830 2023-10-10T20:26:01.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484159a700 1 -- 172.21.15.148:0/2928894078 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c00b860 con 0x7f484406a830 2023-10-10T20:26:01.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.340+0000 7f484159a700 1 -- 172.21.15.148:0/2928894078 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00bbe0 con 0x7f484406a830 2023-10-10T20:26:01.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 -- 172.21.15.148:0/2928894078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f484406a830 msgr2=0x7f48440b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f484406a830 0x7f48440b5200 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098e0 tx=0x7f483c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 -- 172.21.15.148:0/2928894078 shutdown_connections 2023-10-10T20:26:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f484406a830 0x7f48440b5200 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 --2- 172.21.15.148:0/2928894078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48440b6580 0x7f484406a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 -- 172.21.15.148:0/2928894078 >> 172.21.15.148:0/2928894078 conn(0x7f4844068cb0 msgr2=0x7f48440690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 -- 172.21.15.148:0/2928894078 shutdown_connections 2023-10-10T20:26:01.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.341+0000 7f484359e700 1 -- 172.21.15.148:0/2928894078 wait complete. 2023-10-10T20:26:01.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.342+0000 7f484359e700 1 Processor -- start 2023-10-10T20:26:01.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.342+0000 7f484359e700 1 -- start start 2023-10-10T20:26:01.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484406a830 0x7f48441521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48440b6580 0x7f4844147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441529f0 con 0x7f48440cce30 2023-10-10T20:26:01.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48440b2090 con 0x7f48440b6580 2023-10-10T20:26:01.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48440b14c0 con 0x7f484406a830 2023-10-10T20:26:01.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484406a830 0x7f48441521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33902/0 (socket says 172.21.15.148:33902) 2023-10-10T20:26:01.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 -- 172.21.15.148:0/4088465852 learned_addr learned my addr 172.21.15.148:0/4088465852 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:01.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4841d9b700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48440b6580 0x7f4844147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 -- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484406a830 msgr2=0x7f48441521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484406a830 0x7f48441521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 -- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48440b6580 msgr2=0x7f4844147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48440b6580 0x7f4844147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.355 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 -- 172.21.15.148:0/4088465852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c009420 con 0x7f48440cce30 2023-10-10T20:26:01.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f4842d9d700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f483c00b3c0 tx=0x7f483c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c011420 con 0x7f48440cce30 2023-10-10T20:26:01.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0115a0 con 0x7f48440cce30 2023-10-10T20:26:01.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c011960 con 0x7f48440cce30 2023-10-10T20:26:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48440b1760 con 0x7f48440cce30 2023-10-10T20:26:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.343+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48440b1d20 con 0x7f48440cce30 2023-10-10T20:26:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.344+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48440b2240 con 0x7f48440cce30 2023-10-10T20:26:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.345+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f483c011cd0 con 0x7f48440cce30 2023-10-10T20:26:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.345+0000 7f483a7fc700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 0x7f48240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.345+0000 7f484259c700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 0x7f48240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.345+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f483c085bc0 con 0x7f48440cce30 2023-10-10T20:26:01.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.346+0000 7f484259c700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 0x7f48240693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4834007240 tx=0x7f4834001970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.350+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48440b2240 con 0x7f48440cce30 2023-10-10T20:26:01.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.532+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48440b2200 con 0x7f48440cce30 2023-10-10T20:26:01.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.532+0000 7f483a7fc700 1 -- 172.21.15.148:0/4088465852 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f483c028020 con 0x7f48440cce30 2023-10-10T20:26:01.535 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:26:01.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 msgr2=0x7f48240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 0x7f48240693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4834007240 tx=0x7f4834001970 comp rx=0 tx=0).stop 2023-10-10T20:26:01.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 msgr2=0x7f4844153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f483c00b3c0 tx=0x7f483c017000 comp rx=0 tx=0).stop 2023-10-10T20:26:01.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 shutdown_connections 2023-10-10T20:26:01.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4824066f30 0x7f48240693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48440cce30 0x7f4844153350 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484406a830 0x7f48441521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 --2- 172.21.15.148:0/4088465852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48440b6580 0x7f4844147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 >> 172.21.15.148:0/4088465852 conn(0x7f4844068cb0 msgr2=0x7f48441497c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 shutdown_connections 2023-10-10T20:26:01.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.534+0000 7f484359e700 1 -- 172.21.15.148:0/4088465852 wait complete. 2023-10-10T20:26:01.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:26:01.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.664+0000 7fb335f76700 1 Processor -- start 2023-10-10T20:26:01.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.664+0000 7fb335f76700 1 -- start start 2023-10-10T20:26:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb335f76700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb330068880 0x7fb3300c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb335f76700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33006f980 con 0x7fb3300affe0 2023-10-10T20:26:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb335f76700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300cd8c0 con 0x7fb330070e10 2023-10-10T20:26:01.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb335f76700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300ccfd0 con 0x7fb330068880 2023-10-10T20:26:01.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32f7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb3300affe0 0x7fb330070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33596/0 (socket says 172.21.15.148:33596) 2023-10-10T20:26:01.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32f7fe700 1 -- 172.21.15.148:0/1127565953 learned_addr learned my addr 172.21.15.148:0/1127565953 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:01.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32ffff700 1 --2- 172.21.15.148:0/1127565953 >> v2:172.21.15.161:3300/0 conn(0x7fb330068880 0x7fb3300c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217813519 0 0) 0x7fb33006f980 con 0x7fb3300affe0 2023-10-10T20:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003570 con 0x7fb3300affe0 2023-10-10T20:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533447689 0 0) 0x7fb3300cd8c0 con 0x7fb330070e10 2023-10-10T20:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb33006f980 con 0x7fb330070e10 2023-10-10T20:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32ffff700 1 -- 172.21.15.148:0/1127565953 >> v1:172.21.15.161:6789/0 conn(0x7fb330070e10 legacy=0x7fb33015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2094339898 0 0) 0x7fb31c003570 con 0x7fb3300affe0 2023-10-10T20:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32ffff700 1 -- 172.21.15.148:0/1127565953 >> v1:172.21.15.148:6789/0 conn(0x7fb3300affe0 legacy=0x7fb330070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32ffff700 1 -- 172.21.15.148:0/1127565953 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330068d80 con 0x7fb330068880 2023-10-10T20:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.665+0000 7fb32ffff700 1 --2- 172.21.15.148:0/1127565953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb330068880 0x7fb3300c80e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3200098e0 tx=0x7fb3200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e2c82313fa7c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280017f0 con 0x7fb330068880 2023-10-10T20:26:01.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320011420 con 0x7fb330068880 2023-10-10T20:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb32e7fc700 1 -- 172.21.15.148:0/1127565953 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200117c0 con 0x7fb330068880 2023-10-10T20:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 -- 172.21.15.148:0/1127565953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb330068880 msgr2=0x7fb3300c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 --2- 172.21.15.148:0/1127565953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb330068880 0x7fb3300c80e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3200098e0 tx=0x7fb3200091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 -- 172.21.15.148:0/1127565953 shutdown_connections 2023-10-10T20:26:01.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 --2- 172.21.15.148:0/1127565953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb330068880 0x7fb3300c80e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 -- 172.21.15.148:0/1127565953 >> 172.21.15.148:0/1127565953 conn(0x7fb3300ce570 msgr2=0x7fb3300ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 -- 172.21.15.148:0/1127565953 shutdown_connections 2023-10-10T20:26:01.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.666+0000 7fb335f76700 1 -- 172.21.15.148:0/1127565953 wait complete. 2023-10-10T20:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.667+0000 7fb335f76700 1 Processor -- start 2023-10-10T20:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.667+0000 7fb335f76700 1 -- start start 2023-10-10T20:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb330068880 0x7fb3300b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3300affe0 0x7fb3300c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300b1d60 con 0x7fb330070e10 2023-10-10T20:26:01.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300b0eb0 con 0x7fb3300affe0 2023-10-10T20:26:01.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb335f76700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300b1150 con 0x7fb330068880 2023-10-10T20:26:01.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb32effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb32effd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33912/0 (socket says 172.21.15.148:33912) 2023-10-10T20:26:01.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb32effd700 1 -- 172.21.15.148:0/3605802771 learned_addr learned my addr 172.21.15.148:0/3605802771 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:01.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb32ffff700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3300affe0 0x7fb3300c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.668+0000 7fb32f7fe700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb330068880 0x7fb3300b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 -- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb330068880 msgr2=0x7fb3300b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb330068880 0x7fb3300b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 -- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3300affe0 msgr2=0x7fb3300c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3300affe0 0x7fb3300c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 -- 172.21.15.148:0/3605802771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb320009420 con 0x7fb330070e10 2023-10-10T20:26:01.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb32effd700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c790 tx=0x7fb318007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318010510 con 0x7fb330070e10 2023-10-10T20:26:01.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318010670 con 0x7fb330070e10 2023-10-10T20:26:01.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318010a10 con 0x7fb330070e10 2023-10-10T20:26:01.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300b1430 con 0x7fb330070e10 2023-10-10T20:26:01.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.669+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3300b19f0 con 0x7fb330070e10 2023-10-10T20:26:01.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.670+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3300b1ed0 con 0x7fb330070e10 2023-10-10T20:26:01.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.670+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb31801b400 con 0x7fb330070e10 2023-10-10T20:26:01.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.670+0000 7fb3277fe700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 0x7fb3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.671+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3180842c0 con 0x7fb330070e10 2023-10-10T20:26:01.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.671+0000 7fb32f7fe700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 0x7fb3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.671+0000 7fb32f7fe700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 0x7fb3100693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb328007fd0 tx=0x7fb328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.675+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb318051530 con 0x7fb330070e10 2023-10-10T20:26:01.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.857+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb33004c3c0 con 0x7fb330070e10 2023-10-10T20:26:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.857+0000 7fb3277fe700 1 -- 172.21.15.148:0/3605802771 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb318054b50 con 0x7fb330070e10 2023-10-10T20:26:01.860 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:26:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 msgr2=0x7fb3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 0x7fb3100693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb328007fd0 tx=0x7fb328009040 comp rx=0 tx=0).stop 2023-10-10T20:26:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 msgr2=0x7fb33014a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c790 tx=0x7fb318007e10 comp rx=0 tx=0).stop 2023-10-10T20:26:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 shutdown_connections 2023-10-10T20:26:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb310066f30 0x7fb3100693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3300affe0 0x7fb3300c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb330068880 0x7fb3300b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 --2- 172.21.15.148:0/3605802771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb330070e10 0x7fb33014a360 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 >> 172.21.15.148:0/3605802771 conn(0x7fb3300ce570 msgr2=0x7fb3300ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 shutdown_connections 2023-10-10T20:26:01.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.859+0000 7fb335f76700 1 -- 172.21.15.148:0/3605802771 wait complete. 2023-10-10T20:26:01.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:01.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.986+0000 7fbeed42c700 1 Processor -- start 2023-10-10T20:26:01.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.986+0000 7fbeed42c700 1 -- start start 2023-10-10T20:26:01.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbeed42c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbee80cdab0 0x7fbee815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbeed42c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee80cb7c0 con 0x7fbee80cd6e0 2023-10-10T20:26:01.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbeed42c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee815e290 con 0x7fbee815dda0 2023-10-10T20:26:01.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbeed42c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee815e850 con 0x7fbee80cdab0 2023-10-10T20:26:01.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbee80cdab0 0x7fbee815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee7fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbee80cd6e0 0x7fbee80cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33604/0 (socket says 172.21.15.148:33604) 2023-10-10T20:26:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee7fff700 1 -- 172.21.15.148:0/1173025451 learned_addr learned my addr 172.21.15.148:0/1173025451 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee77fe700 1 -- 172.21.15.148:0/1173025451 >> v1:172.21.15.161:6789/0 conn(0x7fbee815dda0 legacy=0x7fbee815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee77fe700 1 -- 172.21.15.148:0/1173025451 >> v1:172.21.15.148:6789/0 conn(0x7fbee80cd6e0 legacy=0x7fbee80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee77fe700 1 -- 172.21.15.148:0/1173025451 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee815e570 con 0x7fbee80cdab0 2023-10-10T20:26:01.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.987+0000 7fbee6ffd700 1 -- 172.21.15.148:0/1173025451 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936636613 0 0) 0x7fbee80cb7c0 con 0x7fbee80cd6e0 2023-10-10T20:26:01.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbee77fe700 1 --2- 172.21.15.148:0/1173025451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cdab0 0x7fbee815d860 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098e0 tx=0x7fbedc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ffd3a76d42413d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:01.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbee6ffd700 1 -- 172.21.15.148:0/1173025451 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc009e80 con 0x7fbee80cdab0 2023-10-10T20:26:01.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbee6ffd700 1 -- 172.21.15.148:0/1173025451 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc00b860 con 0x7fbee80cdab0 2023-10-10T20:26:01.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbee6ffd700 1 -- 172.21.15.148:0/1173025451 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00bbe0 con 0x7fbee80cdab0 2023-10-10T20:26:01.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 -- 172.21.15.148:0/1173025451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cdab0 msgr2=0x7fbee815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:01.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1173025451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cdab0 0x7fbee815d860 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098e0 tx=0x7fbedc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 -- 172.21.15.148:0/1173025451 shutdown_connections 2023-10-10T20:26:01.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1173025451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cdab0 0x7fbee815d860 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:01.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 -- 172.21.15.148:0/1173025451 >> 172.21.15.148:0/1173025451 conn(0x7fbee8067660 msgr2=0x7fbee8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:01.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 -- 172.21.15.148:0/1173025451 shutdown_connections 2023-10-10T20:26:01.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.988+0000 7fbeed42c700 1 -- 172.21.15.148:0/1173025451 wait complete. 2023-10-10T20:26:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.989+0000 7fbeed42c700 1 Processor -- start 2023-10-10T20:26:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.989+0000 7fbeed42c700 1 -- start start 2023-10-10T20:26:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee80cdab0 0x7fbee814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee815dda0 0x7fbee814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80cc360 con 0x7fbee80cd6e0 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80c7eb0 con 0x7fbee815dda0 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbeed42c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8151560 con 0x7fbee80cdab0 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbee7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbee7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33922/0 (socket says 172.21.15.148:33922) 2023-10-10T20:26:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbee7fff700 1 -- 172.21.15.148:0/1885274602 learned_addr learned my addr 172.21.15.148:0/1885274602 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbee7fff700 1 -- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee80cdab0 msgr2=0x7fbee814f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.990+0000 7fbee7fff700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee80cdab0 0x7fbee814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbee7fff700 1 -- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee815dda0 msgr2=0x7fbee814fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbeecc2b700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee815dda0 0x7fbee814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbee7fff700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee815dda0 0x7fbee814fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbee7fff700 1 -- 172.21.15.148:0/1885274602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc009420 con 0x7fbee80cd6e0 2023-10-10T20:26:02.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbee7fff700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c810 tx=0x7fbed8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8005c00 con 0x7fbee80cd6e0 2023-10-10T20:26:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8005d60 con 0x7fbee80cd6e0 2023-10-10T20:26:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee80b4990 con 0x7fbee80cd6e0 2023-10-10T20:26:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8007340 con 0x7fbee80cd6e0 2023-10-10T20:26:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.991+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee80b4f50 con 0x7fbee80cd6e0 2023-10-10T20:26:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.992+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee80c8040 con 0x7fbee80cd6e0 2023-10-10T20:26:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.992+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbed801b400 con 0x7fbee80cd6e0 2023-10-10T20:26:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.992+0000 7fbecffff700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.993+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbed8084010 con 0x7fbee80cd6e0 2023-10-10T20:26:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.993+0000 7fbee77fe700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.993+0000 7fbee77fe700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 0x7fbed40693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbee8157010 tx=0x7fbedc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:01.997+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbed8051300 con 0x7fbee80cd6e0 2023-10-10T20:26:02.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.218+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbee80b52f0 con 0x7fbee80cd6e0 2023-10-10T20:26:02.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.219+0000 7fbecffff700 1 -- 172.21.15.148:0/1885274602 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fbed8054920 con 0x7fbee80cd6e0 2023-10-10T20:26:02.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:02.222 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:26:02.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 msgr2=0x7fbed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 0x7fbed40693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbee8157010 tx=0x7fbedc017000 comp rx=0 tx=0).stop 2023-10-10T20:26:02.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 msgr2=0x7fbee8151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c810 tx=0x7fbed8007d90 comp rx=0 tx=0).stop 2023-10-10T20:26:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 shutdown_connections 2023-10-10T20:26:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbed4066f30 0x7fbed40693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee815dda0 0x7fbee814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee80cd6e0 0x7fbee8151020 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 --2- 172.21.15.148:0/1885274602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee80cdab0 0x7fbee814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.221+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 >> 172.21.15.148:0/1885274602 conn(0x7fbee8067660 msgr2=0x7fbee80c6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:02.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.222+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 shutdown_connections 2023-10-10T20:26:02.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.222+0000 7fbeed42c700 1 -- 172.21.15.148:0/1885274602 wait complete. 2023-10-10T20:26:02.226 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:26:02.236 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.348+0000 7fab15c4c700 1 Processor -- start 2023-10-10T20:26:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.348+0000 7fab15c4c700 1 -- start start 2023-10-10T20:26:02.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab15c4c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab15c4c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab101438f0 0x7fab100b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab15c4c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab100d0870 con 0x7fab10143520 2023-10-10T20:26:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab15c4c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab100cdfb0 con 0x7fab101438f0 2023-10-10T20:26:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab15c4c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab100cfc60 con 0x7fab1006ab50 2023-10-10T20:26:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab1544b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab101438f0 0x7fab100b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab14c4a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fab10143520 0x7fab1006a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33614/0 (socket says 172.21.15.148:33614) 2023-10-10T20:26:02.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab14c4a700 1 -- 172.21.15.148:0/3569672224 learned_addr learned my addr 172.21.15.148:0/3569672224 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:02.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab1544b700 1 -- 172.21.15.148:0/3569672224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 msgr2=0x7fab100cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:02.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab0ffff700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.349+0000 7fab0f7fe700 1 -- 172.21.15.148:0/3569672224 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361928249 0 0) 0x7fab100d0870 con 0x7fab10143520 2023-10-10T20:26:02.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab1544b700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab1544b700 1 -- 172.21.15.148:0/3569672224 >> v1:172.21.15.148:6789/0 conn(0x7fab10143520 legacy=0x7fab1006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab1544b700 1 -- 172.21.15.148:0/3569672224 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100cff40 con 0x7fab101438f0 2023-10-10T20:26:02.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab0ffff700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:02.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab1544b700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab101438f0 0x7fab100b3ac0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fab080098e0 tx=0x7fab080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a98fb7bf90c216 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab0f7fe700 1 -- 172.21.15.148:0/3569672224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08009e80 con 0x7fab101438f0 2023-10-10T20:26:02.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab0f7fe700 1 -- 172.21.15.148:0/3569672224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800b860 con 0x7fab101438f0 2023-10-10T20:26:02.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.350+0000 7fab0f7fe700 1 -- 172.21.15.148:0/3569672224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800bc00 con 0x7fab101438f0 2023-10-10T20:26:02.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 -- 172.21.15.148:0/3569672224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab101438f0 msgr2=0x7fab100b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab101438f0 0x7fab100b3ac0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fab080098e0 tx=0x7fab080091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 -- 172.21.15.148:0/3569672224 shutdown_connections 2023-10-10T20:26:02.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab101438f0 0x7fab100b3ac0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 --2- 172.21.15.148:0/3569672224 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 -- 172.21.15.148:0/3569672224 >> 172.21.15.148:0/3569672224 conn(0x7fab10069750 msgr2=0x7fab100b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 -- 172.21.15.148:0/3569672224 shutdown_connections 2023-10-10T20:26:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.351+0000 7fab15c4c700 1 -- 172.21.15.148:0/3569672224 wait complete. 2023-10-10T20:26:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.352+0000 7fab15c4c700 1 Processor -- start 2023-10-10T20:26:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.352+0000 7fab15c4c700 1 -- start start 2023-10-10T20:26:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab101438f0 0x7fab10169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1016a920 con 0x7fab10143520 2023-10-10T20:26:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015d580 con 0x7fab101438f0 2023-10-10T20:26:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab15c4c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015d860 con 0x7fab1006ab50 2023-10-10T20:26:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33942/0 (socket says 172.21.15.148:33942) 2023-10-10T20:26:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 -- 172.21.15.148:0/2115907503 learned_addr learned my addr 172.21.15.148:0/2115907503 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab1544b700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab101438f0 0x7fab10169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab14c4a700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 -- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 msgr2=0x7fab100b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 -- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab101438f0 msgr2=0x7fab10169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab101438f0 0x7fab10169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 -- 172.21.15.148:0/2115907503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08009420 con 0x7fab10143520 2023-10-10T20:26:02.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab14c4a700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:02.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.353+0000 7fab0ffff700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c830 tx=0x7faaf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.354+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011600 con 0x7fab10143520 2023-10-10T20:26:02.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.354+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf8011760 con 0x7fab10143520 2023-10-10T20:26:02.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.354+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1015db40 con 0x7fab10143520 2023-10-10T20:26:02.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.354+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011b20 con 0x7fab10143520 2023-10-10T20:26:02.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.354+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1015e340 con 0x7fab10143520 2023-10-10T20:26:02.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.355+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1015e560 con 0x7fab10143520 2023-10-10T20:26:02.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.355+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faaf800a3e0 con 0x7fab10143520 2023-10-10T20:26:02.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.356+0000 7fab0cff9700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.356+0000 7fab14c4a700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.356+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faaf8084a90 con 0x7fab10143520 2023-10-10T20:26:02.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.356+0000 7fab14c4a700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 0x7faaf00693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d270 tx=0x7fab00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.360+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaf8051c60 con 0x7fab10143520 2023-10-10T20:26:02.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.545+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab10089030 con 0x7fab10143520 2023-10-10T20:26:02.548 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.545+0000 7fab0cff9700 1 -- 172.21.15.148:0/2115907503 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+15420 (secure 0 0 0) 0x7faaf8015090 con 0x7fab10143520 2023-10-10T20:26:02.548 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:02.548 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:55.532661+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:02.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 msgr2=0x7faaf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 0x7faaf00693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d270 tx=0x7fab00009040 comp rx=0 tx=0).stop 2023-10-10T20:26:02.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 msgr2=0x7fab100b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c830 tx=0x7faaf8008040 comp rx=0 tx=0).stop 2023-10-10T20:26:02.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 shutdown_connections 2023-10-10T20:26:02.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab101438f0 0x7fab10169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1006ab50 0x7fab100b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 --2- 172.21.15.148:0/2115907503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab10143520 0x7fab100b1ad0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 >> 172.21.15.148:0/2115907503 conn(0x7fab10069750 msgr2=0x7fab10141c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:02.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.547+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 shutdown_connections 2023-10-10T20:26:02.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.548+0000 7fab15c4c700 1 -- 172.21.15.148:0/2115907503 wait complete. 2023-10-10T20:26:02.562 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:26:02.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.674+0000 7fa8bedab700 1 Processor -- start 2023-10-10T20:26:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.674+0000 7fa8bedab700 1 -- start start 2023-10-10T20:26:02.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bedab700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8b80cd6e0 0x7fa8b80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bedab700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8b815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bedab700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b80cb7c0 con 0x7fa8b815dda0 2023-10-10T20:26:02.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bedab700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b815e850 con 0x7fa8b80cd6e0 2023-10-10T20:26:02.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bedab700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80cd030 con 0x7fa8b80cdab0 2023-10-10T20:26:02.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8be5aa700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa8b815dda0 0x7fa8b815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40094/0 (socket says 172.21.15.148:40094) 2023-10-10T20:26:02.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8be5aa700 1 -- 172.21.15.148:0/4197271191 learned_addr learned my addr 172.21.15.148:0/4197271191 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:02.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bdda9700 1 --2- 172.21.15.148:0/4197271191 >> v2:172.21.15.161:3300/0 conn(0x7fa8b80cd6e0 0x7fa8b80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bd5a8700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8b815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974442468 0 0) 0x7fa8b80cb7c0 con 0x7fa8b815dda0 2023-10-10T20:26:02.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.675+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a4003570 con 0x7fa8b815dda0 2023-10-10T20:26:02.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524844626 0 0) 0x7fa8a4003570 con 0x7fa8b815dda0 2023-10-10T20:26:02.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 msgr2=0x7fa8b815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:02.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8b815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 >> v2:172.21.15.161:3300/0 conn(0x7fa8b80cd6e0 msgr2=0x7fa8b80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 --2- 172.21.15.148:0/4197271191 >> v2:172.21.15.161:3300/0 conn(0x7fa8b80cd6e0 0x7fa8b80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b815e290 con 0x7fa8b815dda0 2023-10-10T20:26:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa8b00017f0 con 0x7fa8b815dda0 2023-10-10T20:26:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 >> v1:172.21.15.148:6789/0 conn(0x7fa8b815dda0 legacy=0x7fa8b815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a4003a60 0x7fa8a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a4004d50 0x7fa8a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b815e850 con 0x7fa8a4004d50 2023-10-10T20:26:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80cd030 con 0x7fa8a4003a60 2023-10-10T20:26:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a400ba60 con 0x7fa8b80cdab0 2023-10-10T20:26:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a4004d50 0x7fa8a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bdda9700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a4003a60 0x7fa8a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bd5a8700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 -- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 msgr2=0x7fa8a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80cdab0 0x7fa8a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 -- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a4003a60 msgr2=0x7fa8a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:02.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8a4003a60 0x7fa8a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 -- 172.21.15.148:0/4197271191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4003570 con 0x7fa8a4004d50 2023-10-10T20:26:02.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8be5aa700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a4004d50 0x7fa8a40051c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000a970 tx=0x7fa8b80ccaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5662ef719921aa84 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00070d0 con 0x7fa8a4004d50 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0007270 con 0x7fa8a4004d50 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.676+0000 7fa8bcda7700 1 -- 172.21.15.148:0/4197271191 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0007630 con 0x7fa8a4004d50 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 -- 172.21.15.148:0/4197271191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a4004d50 msgr2=0x7fa8a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 --2- 172.21.15.148:0/4197271191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8a4004d50 0x7fa8a40051c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000a970 tx=0x7fa8b80ccaf0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8be5aa700 1 -- 172.21.15.148:0/4197271191 reap_dead start 2023-10-10T20:26:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 -- 172.21.15.148:0/4197271191 shutdown_connections 2023-10-10T20:26:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 -- 172.21.15.148:0/4197271191 >> 172.21.15.148:0/4197271191 conn(0x7fa8b8067660 msgr2=0x7fa8b8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 -- 172.21.15.148:0/4197271191 shutdown_connections 2023-10-10T20:26:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.677+0000 7fa8bedab700 1 -- 172.21.15.148:0/4197271191 wait complete. 2023-10-10T20:26:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.678+0000 7fa8bedab700 1 Processor -- start 2023-10-10T20:26:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.678+0000 7fa8bedab700 1 -- start start 2023-10-10T20:26:02.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8b814f730 0x7fa8b80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80c6300 0x7fa8b814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80c7d20 con 0x7fa8b814ccb0 2023-10-10T20:26:02.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80c6770 con 0x7fa8b814f730 2023-10-10T20:26:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80b4900 con 0x7fa8b80c6300 2023-10-10T20:26:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37398/0 (socket says 172.21.15.148:37398) 2023-10-10T20:26:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 -- 172.21.15.148:0/1515153689 learned_addr learned my addr 172.21.15.148:0/1515153689 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8be5aa700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80c6300 0x7fa8b814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bd5a8700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8b814f730 0x7fa8b80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 -- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80c6300 msgr2=0x7fa8b814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80c6300 0x7fa8b814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 -- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8b814f730 msgr2=0x7fa8b80c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8b814f730 0x7fa8b80c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 -- 172.21.15.148:0/1515153689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80b3a10 con 0x7fa8b814ccb0 2023-10-10T20:26:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bdda9700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006ac0 tx=0x7fa8b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400fa10 con 0x7fa8b814ccb0 2023-10-10T20:26:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b400fb70 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b80b3cf0 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.679+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4014440 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.680+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b80b42b0 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.681+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b8089280 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.681+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8b4014830 con 0x7fa8b814ccb0 2023-10-10T20:26:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.681+0000 7fa8adffb700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 0x7fa89c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.681+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8b4018030 con 0x7fa8b814ccb0 2023-10-10T20:26:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.682+0000 7fa8bd5a8700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 0x7fa89c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.682+0000 7fa8bd5a8700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 0x7fa89c069410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80b5080 tx=0x7fa8a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.686+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8b4052220 con 0x7fa8b814ccb0 2023-10-10T20:26:02.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.911+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8b815e570 con 0x7fa8b814ccb0 2023-10-10T20:26:02.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.911+0000 7fa8adffb700 1 -- 172.21.15.148:0/1515153689 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fa8b4055840 con 0x7fa8b814ccb0 2023-10-10T20:26:02.915 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:02.915 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:26:02.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.913+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 msgr2=0x7fa89c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.913+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 0x7fa89c069410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80b5080 tx=0x7fa8a8008040 comp rx=0 tx=0).stop 2023-10-10T20:26:02.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.913+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 msgr2=0x7fa8b814f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:02.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.913+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006ac0 tx=0x7fa8b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 shutdown_connections 2023-10-10T20:26:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa89c066f60 0x7fa89c069410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8b80c6300 0x7fa8b814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b814ccb0 0x7fa8b814f1f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 --2- 172.21.15.148:0/1515153689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8b814f730 0x7fa8b80c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 >> 172.21.15.148:0/1515153689 conn(0x7fa8b8067660 msgr2=0x7fa8b80c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:02.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 shutdown_connections 2023-10-10T20:26:02.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:02.914+0000 7fa8bedab700 1 -- 172.21.15.148:0/1515153689 wait complete. 2023-10-10T20:26:02.919 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:26:02.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:03.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.039+0000 7f40a4094700 1 Processor -- start 2023-10-10T20:26:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.039+0000 7f40a4094700 1 -- start start 2023-10-10T20:26:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.040+0000 7f40a4094700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 0x7f409c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.040+0000 7f40a4094700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.040+0000 7f40a4094700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c15cbb0 con 0x7f409c157870 2023-10-10T20:26:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.040+0000 7f40a4094700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15bdb0 con 0x7f409c15a620 2023-10-10T20:26:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.040+0000 7f40a4094700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c15bfa0 con 0x7f409c15d580 2023-10-10T20:26:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a162f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 0x7f409c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a1e30700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f409c157870 0x7f409c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43938/0 (socket says 172.21.15.148:43938) 2023-10-10T20:26:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a1e30700 1 -- 172.21.15.148:0/2114033164 learned_addr learned my addr 172.21.15.148:0/2114033164 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:03.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 -- 172.21.15.148:0/2114033164 >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 msgr2=0x7f409c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 --2- 172.21.15.148:0/2114033164 >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 0x7f409c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 -- 172.21.15.148:0/2114033164 >> v1:172.21.15.161:6789/0 conn(0x7f409c157870 legacy=0x7f409c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 -- 172.21.15.148:0/2114033164 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c15c280 con 0x7f409c15d580 2023-10-10T20:26:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a162f700 1 --2- 172.21.15.148:0/2114033164 >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 0x7f409c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.041+0000 7f40a2631700 1 --2- 172.21.15.148:0/2114033164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c158110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4094009e60 tx=0x7f4094009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fefc190ab863bd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a0e2e700 1 -- 172.21.15.148:0/2114033164 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400e070 con 0x7f409c15d580 2023-10-10T20:26:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a0e2e700 1 -- 172.21.15.148:0/2114033164 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400ba60 con 0x7f409c15d580 2023-10-10T20:26:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a0e2e700 1 -- 172.21.15.148:0/2114033164 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400be20 con 0x7f409c15d580 2023-10-10T20:26:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 -- 172.21.15.148:0/2114033164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 msgr2=0x7f409c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 --2- 172.21.15.148:0/2114033164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c158110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4094009e60 tx=0x7f4094009780 comp rx=0 tx=0).stop 2023-10-10T20:26:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 -- 172.21.15.148:0/2114033164 shutdown_connections 2023-10-10T20:26:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 --2- 172.21.15.148:0/2114033164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c158110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 --2- 172.21.15.148:0/2114033164 >> v2:172.21.15.161:3300/0 conn(0x7f409c15a620 0x7f409c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 -- 172.21.15.148:0/2114033164 >> 172.21.15.148:0/2114033164 conn(0x7f409c068ce0 msgr2=0x7f409c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 -- 172.21.15.148:0/2114033164 shutdown_connections 2023-10-10T20:26:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.042+0000 7f40a4094700 1 -- 172.21.15.148:0/2114033164 wait complete. 2023-10-10T20:26:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.043+0000 7f40a4094700 1 Processor -- start 2023-10-10T20:26:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.043+0000 7f40a4094700 1 -- start start 2023-10-10T20:26:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c16bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c158740 con 0x7f409c157870 2023-10-10T20:26:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c153490 con 0x7f409c15a620 2023-10-10T20:26:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a4094700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c153710 con 0x7f409c15d580 2023-10-10T20:26:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a1e30700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a1e30700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37404/0 (socket says 172.21.15.148:37404) 2023-10-10T20:26:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a1e30700 1 -- 172.21.15.148:0/1607154338 learned_addr learned my addr 172.21.15.148:0/1607154338 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a162f700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c16bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.044+0000 7f40a2631700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 -- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 msgr2=0x7f409c152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 -- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 msgr2=0x7f409c16bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c16bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 -- 172.21.15.148:0/1607154338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c009420 con 0x7f409c157870 2023-10-10T20:26:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a1e30700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f409800c810 tx=0x7f4098007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098005b30 con 0x7f409c157870 2023-10-10T20:26:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098005c90 con 0x7f409c157870 2023-10-10T20:26:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4094009a20 con 0x7f409c157870 2023-10-10T20:26:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098007320 con 0x7f409c157870 2023-10-10T20:26:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.045+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c1583d0 con 0x7f409c157870 2023-10-10T20:26:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.046+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0d1c50 con 0x7f409c157870 2023-10-10T20:26:03.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.046+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f409801b3b0 con 0x7f409c157870 2023-10-10T20:26:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.047+0000 7f4091ffb700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 0x7f40800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.047+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4098083b90 con 0x7f409c157870 2023-10-10T20:26:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.047+0000 7f40a162f700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 0x7f40800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.047+0000 7f40a162f700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 0x7f40800693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f409c0633c0 tx=0x7f408c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.051+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4098050e00 con 0x7f409c157870 2023-10-10T20:26:03.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.274+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409c06b720 con 0x7f409c157870 2023-10-10T20:26:03.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.274+0000 7f4091ffb700 1 -- 172.21.15.148:0/1607154338 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f4098017020 con 0x7f409c157870 2023-10-10T20:26:03.277 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:03.277 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":64,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:25:24.013584+0000","modified":"2023-10-10T20:25:27.661435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/2686518159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2686518159},{"type":"v1","addr":"172.21.15.161:6835","nonce":2686518159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-10T20:26:03.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 msgr2=0x7f40800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 0x7f40800693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f409c0633c0 tx=0x7f408c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 msgr2=0x7f409c16b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f409800c810 tx=0x7f4098007f30 comp rx=0 tx=0).stop 2023-10-10T20:26:03.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 shutdown_connections 2023-10-10T20:26:03.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4080066f30 0x7f40800693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f409c15d580 0x7f409c152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c157870 0x7f409c16b5b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 --2- 172.21.15.148:0/1607154338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f409c15a620 0x7f409c16bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 >> 172.21.15.148:0/1607154338 conn(0x7f409c068ce0 msgr2=0x7f409c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 shutdown_connections 2023-10-10T20:26:03.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.277+0000 7f40a4094700 1 -- 172.21.15.148:0/1607154338 wait complete. 2023-10-10T20:26:03.282 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 64 2023-10-10T20:26:03.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:03.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.403+0000 7fc474207700 1 Processor -- start 2023-10-10T20:26:03.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 -- start start 2023-10-10T20:26:03.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46c157870 0x7fc46c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc46c15a620 0x7fc46c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c15cbb0 con 0x7fc46c15d580 2023-10-10T20:26:03.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c15bfa0 con 0x7fc46c157870 2023-10-10T20:26:03.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.404+0000 7fc474207700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c15c280 con 0x7fc46c15a620 2023-10-10T20:26:03.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46c157870 0x7fc46c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc46c157870 0x7fc46c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37416/0 (socket says 172.21.15.148:37416) 2023-10-10T20:26:03.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 -- 172.21.15.148:0/1120954046 learned_addr learned my addr 172.21.15.148:0/1120954046 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:03.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc4717a2700 1 --2- 172.21.15.148:0/1120954046 >> v2:172.21.15.161:3300/0 conn(0x7fc46c15a620 0x7fc46c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 -- 172.21.15.148:0/1120954046 >> v2:172.21.15.161:3300/0 conn(0x7fc46c15a620 msgr2=0x7fc46c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:03.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 --2- 172.21.15.148:0/1120954046 >> v2:172.21.15.161:3300/0 conn(0x7fc46c15a620 0x7fc46c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 -- 172.21.15.148:0/1120954046 >> v1:172.21.15.161:6789/0 conn(0x7fc46c15d580 legacy=0x7fc46c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 -- 172.21.15.148:0/1120954046 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c15c560 con 0x7fc46c157870 2023-10-10T20:26:03.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc471fa3700 1 --2- 172.21.15.148:0/1120954046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c157870 0x7fc46c15b6a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc46c15cb20 tx=0x7fc468009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cd8de9df0d8ed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc470fa1700 1 -- 172.21.15.148:0/1120954046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800e070 con 0x7fc46c157870 2023-10-10T20:26:03.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc470fa1700 1 -- 172.21.15.148:0/1120954046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800ba50 con 0x7fc46c157870 2023-10-10T20:26:03.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.405+0000 7fc470fa1700 1 -- 172.21.15.148:0/1120954046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4680092e0 con 0x7fc46c157870 2023-10-10T20:26:03.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 -- 172.21.15.148:0/1120954046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c157870 msgr2=0x7fc46c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 --2- 172.21.15.148:0/1120954046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c157870 0x7fc46c15b6a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc46c15cb20 tx=0x7fc468009840 comp rx=0 tx=0).stop 2023-10-10T20:26:03.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 -- 172.21.15.148:0/1120954046 shutdown_connections 2023-10-10T20:26:03.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 --2- 172.21.15.148:0/1120954046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c157870 0x7fc46c15b6a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 --2- 172.21.15.148:0/1120954046 >> v2:172.21.15.161:3300/0 conn(0x7fc46c15a620 0x7fc46c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 -- 172.21.15.148:0/1120954046 >> 172.21.15.148:0/1120954046 conn(0x7fc46c068ce0 msgr2=0x7fc46c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 -- 172.21.15.148:0/1120954046 shutdown_connections 2023-10-10T20:26:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.406+0000 7fc474207700 1 -- 172.21.15.148:0/1120954046 wait complete. 2023-10-10T20:26:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.407+0000 7fc474207700 1 Processor -- start 2023-10-10T20:26:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.407+0000 7fc474207700 1 -- start start 2023-10-10T20:26:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc46c157870 0x7fc46c149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c15a620 0x7fc46c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c16eb30 con 0x7fc46c15d580 2023-10-10T20:26:03.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c15f470 con 0x7fc46c15a620 2023-10-10T20:26:03.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc474207700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c16dd70 con 0x7fc46c157870 2023-10-10T20:26:03.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37424/0 (socket says 172.21.15.148:37424) 2023-10-10T20:26:03.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 -- 172.21.15.148:0/795974806 learned_addr learned my addr 172.21.15.148:0/795974806 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:03.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4717a2700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c15a620 0x7fc46c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc471fa3700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc46c157870 0x7fc46c149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 -- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc46c157870 msgr2=0x7fc46c149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc46c157870 0x7fc46c149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 -- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c15a620 msgr2=0x7fc46c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c15a620 0x7fc46c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 -- 172.21.15.148:0/795974806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009ae0 con 0x7fc46c15d580 2023-10-10T20:26:03.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc4727a4700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c490 tx=0x7fc464007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.408+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46400ce40 con 0x7fc46c15d580 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.409+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c16e050 con 0x7fc46c15d580 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.409+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4640117a0 con 0x7fc46c15d580 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.409+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464011910 con 0x7fc46c15d580 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.409+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c16e610 con 0x7fc46c15d580 2023-10-10T20:26:03.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.410+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c16e830 con 0x7fc46c15d580 2023-10-10T20:26:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.410+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc464011cc0 con 0x7fc46c15d580 2023-10-10T20:26:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.411+0000 7fc461ffb700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.411+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc464083720 con 0x7fc46c15d580 2023-10-10T20:26:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.411+0000 7fc471fa3700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.411+0000 7fc471fa3700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 0x7fc4500693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc468009880 tx=0x7fc468018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.415+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc464050a40 con 0x7fc46c15d580 2023-10-10T20:26:03.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.599+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc46c04c3c0 con 0x7fc46c15d580 2023-10-10T20:26:03.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.599+0000 7fc461ffb700 1 -- 172.21.15.148:0/795974806 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+15420 (secure 0 0 0) 0x7fc464017090 con 0x7fc46c15d580 2023-10-10T20:26:03.602 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:03.602 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:25:55.532661+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:25:22.369290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-10T20:25:22.981299+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:03.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 msgr2=0x7fc4500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 0x7fc4500693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc468009880 tx=0x7fc468018000 comp rx=0 tx=0).stop 2023-10-10T20:26:03.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 msgr2=0x7fc46c149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c490 tx=0x7fc464007f20 comp rx=0 tx=0).stop 2023-10-10T20:26:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 shutdown_connections 2023-10-10T20:26:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc450066f30 0x7fc4500693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46c15d580 0x7fc46c149190 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc46c157870 0x7fc46c149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 --2- 172.21.15.148:0/795974806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c15a620 0x7fc46c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 >> 172.21.15.148:0/795974806 conn(0x7fc46c068ce0 msgr2=0x7fc46c158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 shutdown_connections 2023-10-10T20:26:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.602+0000 7fc474207700 1 -- 172.21.15.148:0/795974806 wait complete. 2023-10-10T20:26:03.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:26:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.732+0000 7f5f97416700 1 Processor -- start 2023-10-10T20:26:03.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.732+0000 7f5f97416700 1 -- start start 2023-10-10T20:26:03.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.733+0000 7f5f97416700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5f90152010 0x7f5f9015b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.733+0000 7f5f97416700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.733+0000 7f5f97416700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9006c010 con 0x7f5f901523e0 2023-10-10T20:26:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.733+0000 7f5f97416700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9006bd10 con 0x7f5f90152010 2023-10-10T20:26:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.733+0000 7f5f97416700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015c060 con 0x7f5f90158e50 2023-10-10T20:26:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:57004/0 (socket says 172.21.15.148:57004) 2023-10-10T20:26:03.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 -- 172.21.15.148:0/1488798207 learned_addr learned my addr 172.21.15.148:0/1488798207 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:03.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f951b2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5f90152010 0x7f5f9015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f949b1700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5f901523e0 0x7f5f9006c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43950/0 (socket says 172.21.15.148:43950) 2023-10-10T20:26:03.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 -- 172.21.15.148:0/1488798207 >> v2:172.21.15.161:3300/0 conn(0x7f5f90152010 msgr2=0x7f5f9015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:03.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 --2- 172.21.15.148:0/1488798207 >> v2:172.21.15.161:3300/0 conn(0x7f5f90152010 0x7f5f9015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 -- 172.21.15.148:0/1488798207 >> v1:172.21.15.161:6789/0 conn(0x7f5f901523e0 legacy=0x7f5f9006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 -- 172.21.15.148:0/1488798207 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9015c340 con 0x7f5f90158e50 2023-10-10T20:26:03.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.734+0000 7f5f959b3700 1 --2- 172.21.15.148:0/1488798207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f880099b0 tx=0x7f5f88009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb7d4cd130a2412 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f87fff700 1 -- 172.21.15.148:0/1488798207 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88009ea0 con 0x7f5f90158e50 2023-10-10T20:26:03.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f87fff700 1 -- 172.21.15.148:0/1488798207 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8800b860 con 0x7f5f90158e50 2023-10-10T20:26:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f87fff700 1 -- 172.21.15.148:0/1488798207 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800bc00 con 0x7f5f90158e50 2023-10-10T20:26:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 -- 172.21.15.148:0/1488798207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 msgr2=0x7f5f900cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 --2- 172.21.15.148:0/1488798207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5f880099b0 tx=0x7f5f88009250 comp rx=0 tx=0).stop 2023-10-10T20:26:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 -- 172.21.15.148:0/1488798207 shutdown_connections 2023-10-10T20:26:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 --2- 172.21.15.148:0/1488798207 >> v2:172.21.15.161:3300/0 conn(0x7f5f90152010 0x7f5f9015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 --2- 172.21.15.148:0/1488798207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f90158e50 0x7f5f900cdf30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.737+0000 7f5f97416700 1 -- 172.21.15.148:0/1488798207 >> 172.21.15.148:0/1488798207 conn(0x7f5f90069870 msgr2=0x7f5f90069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.738+0000 7f5f97416700 1 -- 172.21.15.148:0/1488798207 shutdown_connections 2023-10-10T20:26:03.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.738+0000 7f5f97416700 1 -- 172.21.15.148:0/1488798207 wait complete. 2023-10-10T20:26:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.739+0000 7f5f97416700 1 Processor -- start 2023-10-10T20:26:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.739+0000 7f5f97416700 1 -- start start 2023-10-10T20:26:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f901523e0 0x7f5f900c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f90158e50 0x7f5f9015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9013c7c0 con 0x7f5f90152010 2023-10-10T20:26:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900c7100 con 0x7f5f90158e50 2023-10-10T20:26:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900c64f0 con 0x7f5f901523e0 2023-10-10T20:26:03.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37438/0 (socket says 172.21.15.148:37438) 2023-10-10T20:26:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 -- 172.21.15.148:0/2422399039 learned_addr learned my addr 172.21.15.148:0/2422399039 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f949b1700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f901523e0 0x7f5f900c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f959b3700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f90158e50 0x7f5f9015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 -- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f901523e0 msgr2=0x7f5f900c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f901523e0 0x7f5f900c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 -- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f90158e50 msgr2=0x7f5f9015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f90158e50 0x7f5f9015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 -- 172.21.15.148:0/2422399039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880094f0 con 0x7f5f90152010 2023-10-10T20:26:03.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f951b2700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ef0 tx=0x7f5f8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00cb70 con 0x7f5f90152010 2023-10-10T20:26:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f90152010 2023-10-10T20:26:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f900c67d0 con 0x7f5f90152010 2023-10-10T20:26:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.740+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c014480 con 0x7f5f90152010 2023-10-10T20:26:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.741+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f900c6d90 con 0x7f5f90152010 2023-10-10T20:26:03.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.742+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9004c3c0 con 0x7f5f90152010 2023-10-10T20:26:03.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.742+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f8c0147f0 con 0x7f5f90152010 2023-10-10T20:26:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.742+0000 7f5f857fa700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 0x7f5f74069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.742+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f8c085210 con 0x7f5f90152010 2023-10-10T20:26:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.742+0000 7f5f949b1700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 0x7f5f74069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.743+0000 7f5f949b1700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 0x7f5f74069460 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5f90064230 tx=0x7f5f80007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.747+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f8c0523e0 con 0x7f5f90152010 2023-10-10T20:26:03.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.973+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f900c8ff0 con 0x7f5f90152010 2023-10-10T20:26:03.976 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:26:03.974+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:26:03.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.978+0000 7f5f857fa700 1 -- 172.21.15.148:0/2422399039 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f8c055a00 con 0x7f5f90152010 2023-10-10T20:26:03.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 msgr2=0x7f5f74069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 0x7f5f74069460 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5f90064230 tx=0x7f5f80007eb0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 msgr2=0x7f5f9013c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ef0 tx=0x7f5f8c00b040 comp rx=0 tx=0).stop 2023-10-10T20:26:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 shutdown_connections 2023-10-10T20:26:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f74066fb0 0x7f5f74069460 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f90158e50 0x7f5f9015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90152010 0x7f5f9013c280 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 --2- 172.21.15.148:0/2422399039 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f901523e0 0x7f5f900c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 >> 172.21.15.148:0/2422399039 conn(0x7f5f90069870 msgr2=0x7f5f900cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 shutdown_connections 2023-10-10T20:26:03.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:03.981+0000 7f5f97416700 1 -- 172.21.15.148:0/2422399039 wait complete. 2023-10-10T20:26:03.986 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:26:03.987 INFO:tasks.ceph.mds.d.smithi161.stderr: -8908> 2023-10-10T20:25:19.541+0000 7fcfb5003700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:26:03.987 INFO:tasks.ceph.mds.d.smithi161.stderr: -7709> 2023-10-10T20:25:29.947+0000 7fcfaeff7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:26:03.995 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:26:04.014 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:26:04.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.109+0000 7ff76ef7d700 1 Processor -- start 2023-10-10T20:26:04.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.109+0000 7ff76ef7d700 1 -- start start 2023-10-10T20:26:04.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff768159ec0 0x7ff7680af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff76815a290 0x7ff7680afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff76806a5a0 0x7ff76806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76806af60 con 0x7ff76806a5a0 2023-10-10T20:26:04.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680b0790 con 0x7ff76815a290 2023-10-10T20:26:04.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76ef7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680ce1f0 con 0x7ff768159ec0 2023-10-10T20:26:04.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76d51a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff76806a5a0 0x7ff76806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76d51a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff76806a5a0 0x7ff76806aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37450/0 (socket says 172.21.15.148:37450) 2023-10-10T20:26:04.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76d51a700 1 -- 172.21.15.148:0/3514118613 learned_addr learned my addr 172.21.15.148:0/3514118613 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:04.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff75ffff700 1 --2- 172.21.15.148:0/3514118613 >> v2:172.21.15.161:3300/0 conn(0x7ff76815a290 0x7ff7680afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.110+0000 7ff76cd19700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff768159ec0 0x7ff7680af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 -- 172.21.15.148:0/3514118613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff768159ec0 msgr2=0x7ff7680af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:04.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff768159ec0 0x7ff7680af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 -- 172.21.15.148:0/3514118613 >> v2:172.21.15.161:3300/0 conn(0x7ff76815a290 msgr2=0x7ff7680afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:04.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 --2- 172.21.15.148:0/3514118613 >> v2:172.21.15.161:3300/0 conn(0x7ff76815a290 0x7ff7680afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 -- 172.21.15.148:0/3514118613 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680b0b20 con 0x7ff76806a5a0 2023-10-10T20:26:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76d51a700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff76806a5a0 0x7ff76806aa20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff768088fc0 tx=0x7ff760009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db4014bb2b46fb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff75f7fe700 1 -- 172.21.15.148:0/3514118613 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000b640 con 0x7ff76806a5a0 2023-10-10T20:26:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff75f7fe700 1 -- 172.21.15.148:0/3514118613 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76000b7a0 con 0x7ff76806a5a0 2023-10-10T20:26:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff75f7fe700 1 -- 172.21.15.148:0/3514118613 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000bb60 con 0x7ff76806a5a0 2023-10-10T20:26:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3514118613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff76806a5a0 msgr2=0x7ff76806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:04.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff76806a5a0 0x7ff76806aa20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff768088fc0 tx=0x7ff760009250 comp rx=0 tx=0).stop 2023-10-10T20:26:04.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3514118613 shutdown_connections 2023-10-10T20:26:04.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff76806a5a0 0x7ff76806aa20 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3514118613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff768159ec0 0x7ff7680af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3514118613 >> v2:172.21.15.161:3300/0 conn(0x7ff76815a290 0x7ff7680afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3514118613 >> 172.21.15.148:0/3514118613 conn(0x7ff7680d1880 msgr2=0x7ff7680cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3514118613 shutdown_connections 2023-10-10T20:26:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.111+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3514118613 wait complete. 2023-10-10T20:26:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.112+0000 7ff76ef7d700 1 Processor -- start 2023-10-10T20:26:04.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 -- start start 2023-10-10T20:26:04.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff76806a5a0 0x7ff7681572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff76815a290 0x7ff768151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768151490 con 0x7ff768159ec0 2023-10-10T20:26:04.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768156e10 con 0x7ff76815a290 2023-10-10T20:26:04.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.113+0000 7ff76ef7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b3e10 con 0x7ff76806a5a0 2023-10-10T20:26:04.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37460/0 (socket says 172.21.15.148:37460) 2023-10-10T20:26:04.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 -- 172.21.15.148:0/3260290402 learned_addr learned my addr 172.21.15.148:0/3260290402 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:04.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff76cd19700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff76806a5a0 0x7ff7681572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff76d51a700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff76815a290 0x7ff768151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 -- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff76806a5a0 msgr2=0x7ff7681572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:04.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff76806a5a0 0x7ff7681572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 -- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff76815a290 msgr2=0x7ff768151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff76815a290 0x7ff768151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 -- 172.21.15.148:0/3260290402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600094f0 con 0x7ff768159ec0 2023-10-10T20:26:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75ffff700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e930 tx=0x7ff7580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758013680 con 0x7ff768159ec0 2023-10-10T20:26:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7580137e0 con 0x7ff768159ec0 2023-10-10T20:26:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758013b80 con 0x7ff768159ec0 2023-10-10T20:26:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff768156200 con 0x7ff768159ec0 2023-10-10T20:26:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.114+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7681567c0 con 0x7ff768159ec0 2023-10-10T20:26:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.115+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff768156fa0 con 0x7ff768159ec0 2023-10-10T20:26:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.116+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff75800c3e0 con 0x7ff768159ec0 2023-10-10T20:26:04.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.116+0000 7ff75cff9700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:04.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.116+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff758084630 con 0x7ff768159ec0 2023-10-10T20:26:04.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.116+0000 7ff76cd19700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:04.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.116+0000 7ff76cd19700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 0x7ff7540693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff764009fd0 tx=0x7ff76400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:04.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.121+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff758051850 con 0x7ff768159ec0 2023-10-10T20:26:04.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.342+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff76804c3c0 con 0x7ff768159ec0 2023-10-10T20:26:04.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.978+0000 7ff75cff9700 1 -- 172.21.15.148:0/3260290402 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff758054e70 con 0x7ff768159ec0 2023-10-10T20:26:04.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.980+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 msgr2=0x7ff7540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:04.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.980+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 0x7ff7540693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff764009fd0 tx=0x7ff76400b040 comp rx=0 tx=0).stop 2023-10-10T20:26:04.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.980+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 msgr2=0x7ff768151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:04.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.980+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e930 tx=0x7ff7580091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 shutdown_connections 2023-10-10T20:26:04.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff76815a290 0x7ff768151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff76806a5a0 0x7ff7681572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 --2- 172.21.15.148:0/3260290402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff768159ec0 0x7ff768151600 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:04.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 >> 172.21.15.148:0/3260290402 conn(0x7ff7680d1880 msgr2=0x7ff7680c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:04.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 shutdown_connections 2023-10-10T20:26:04.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:04.981+0000 7ff76ef7d700 1 -- 172.21.15.148:0/3260290402 wait complete. 2023-10-10T20:26:04.996 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:26:05.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.114+0000 7efc4b6fc700 1 Processor -- start 2023-10-10T20:26:05.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.114+0000 7efc4b6fc700 1 -- start start 2023-10-10T20:26:05.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 0x7efc4415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc440b6a10 0x7efc440b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc440b1a60 con 0x7efc440b15e0 2023-10-10T20:26:05.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc440b1d40 con 0x7efc440b6a10 2023-10-10T20:26:05.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440b2020 con 0x7efc44159ef0 2023-10-10T20:26:05.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc440b6a10 0x7efc440b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc49c99700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efc440b15e0 0x7efc440b46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40096/0 (socket says 172.21.15.148:40096) 2023-10-10T20:26:05.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc440b6a10 0x7efc440b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37472/0 (socket says 172.21.15.148:37472) 2023-10-10T20:26:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc49c99700 1 -- 172.21.15.148:0/3581850989 learned_addr learned my addr 172.21.15.148:0/3581850989 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc49498700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 0x7efc4415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 -- 172.21.15.148:0/3581850989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 msgr2=0x7efc4415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 0x7efc4415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 -- 172.21.15.148:0/3581850989 >> v1:172.21.15.148:6789/0 conn(0x7efc440b15e0 legacy=0x7efc440b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc3bfff700 1 -- 172.21.15.148:0/3581850989 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086144943 0 0) 0x7efc440b1a60 con 0x7efc440b15e0 2023-10-10T20:26:05.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 -- 172.21.15.148:0/3581850989 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440b2300 con 0x7efc440b6a10 2023-10-10T20:26:05.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc49498700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 0x7efc4415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:05.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc48c97700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440b6a10 0x7efc440b10a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7efc340098e0 tx=0x7efc340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86b04fff282328a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc3bfff700 1 -- 172.21.15.148:0/3581850989 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34009e80 con 0x7efc440b6a10 2023-10-10T20:26:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc3bfff700 1 -- 172.21.15.148:0/3581850989 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400b860 con 0x7efc440b6a10 2023-10-10T20:26:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc3bfff700 1 -- 172.21.15.148:0/3581850989 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400bbe0 con 0x7efc440b6a10 2023-10-10T20:26:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 -- 172.21.15.148:0/3581850989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440b6a10 msgr2=0x7efc440b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.115+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440b6a10 0x7efc440b10a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7efc340098e0 tx=0x7efc340091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 -- 172.21.15.148:0/3581850989 shutdown_connections 2023-10-10T20:26:05.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc44159ef0 0x7efc4415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/3581850989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440b6a10 0x7efc440b10a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 -- 172.21.15.148:0/3581850989 >> 172.21.15.148:0/3581850989 conn(0x7efc4409a0a0 msgr2=0x7efc44067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:05.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 -- 172.21.15.148:0/3581850989 shutdown_connections 2023-10-10T20:26:05.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.117+0000 7efc4b6fc700 1 -- 172.21.15.148:0/3581850989 wait complete. 2023-10-10T20:26:05.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.118+0000 7efc4b6fc700 1 Processor -- start 2023-10-10T20:26:05.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.118+0000 7efc4b6fc700 1 -- start start 2023-10-10T20:26:05.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.118+0000 7efc4b6fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc440b6a10 0x7efc4414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44157910 0x7efc4409ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc44157db0 con 0x7efc44159ef0 2023-10-10T20:26:05.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4409c3c0 con 0x7efc44157910 2023-10-10T20:26:05.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4409b7b0 con 0x7efc440b6a10 2023-10-10T20:26:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37486/0 (socket says 172.21.15.148:37486) 2023-10-10T20:26:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 -- 172.21.15.148:0/566642734 learned_addr learned my addr 172.21.15.148:0/566642734 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc49498700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc440b6a10 0x7efc4414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc49c99700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44157910 0x7efc4409ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 -- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc440b6a10 msgr2=0x7efc4414e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc440b6a10 0x7efc4414e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 -- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44157910 msgr2=0x7efc4409ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44157910 0x7efc4409ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 -- 172.21.15.148:0/566642734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc34009420 con 0x7efc44159ef0 2023-10-10T20:26:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc48c97700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7efc44071f80 tx=0x7efc34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400bbe0 con 0x7efc44159ef0 2023-10-10T20:26:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc34011420 con 0x7efc44159ef0 2023-10-10T20:26:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4409ba90 con 0x7efc44159ef0 2023-10-10T20:26:05.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34011800 con 0x7efc44159ef0 2023-10-10T20:26:05.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.119+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4409c050 con 0x7efc44159ef0 2023-10-10T20:26:05.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.120+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4404c3c0 con 0x7efc44159ef0 2023-10-10T20:26:05.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.121+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc34011c30 con 0x7efc44159ef0 2023-10-10T20:26:05.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.121+0000 7efc397fa700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 0x7efc280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.121+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5791+0+0 (secure 0 0 0) 0x7efc34086090 con 0x7efc44159ef0 2023-10-10T20:26:05.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.121+0000 7efc49498700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 0x7efc280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.121+0000 7efc49498700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 0x7efc280693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d5f0 tx=0x7efc40007800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.125+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc340532b0 con 0x7efc44159ef0 2023-10-10T20:26:05.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.306+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc4409c500 con 0x7efc44159ef0 2023-10-10T20:26:05.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.538+0000 7efc397fa700 1 -- 172.21.15.148:0/566642734 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7efc340568d0 con 0x7efc44159ef0 2023-10-10T20:26:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 msgr2=0x7efc280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 0x7efc280693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d5f0 tx=0x7efc40007800 comp rx=0 tx=0).stop 2023-10-10T20:26:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 msgr2=0x7efc441573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7efc44071f80 tx=0x7efc34017000 comp rx=0 tx=0).stop 2023-10-10T20:26:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 shutdown_connections 2023-10-10T20:26:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efc28066f30 0x7efc280693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44157910 0x7efc4409ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc440b6a10 0x7efc4414e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 --2- 172.21.15.148:0/566642734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc44159ef0 0x7efc441573d0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 >> 172.21.15.148:0/566642734 conn(0x7efc4409a0a0 msgr2=0x7efc4414edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 shutdown_connections 2023-10-10T20:26:05.545 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.541+0000 7efc4b6fc700 1 -- 172.21.15.148:0/566642734 wait complete. 2023-10-10T20:26:05.546 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:26:05.554 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:26:05.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.673+0000 7f14d5694700 1 Processor -- start 2023-10-10T20:26:05.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.673+0000 7f14d5694700 1 -- start start 2023-10-10T20:26:05.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14d5694700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14d00c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14d5694700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14d00c6bd0 0x7f14d006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14d5694700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d006dc30 con 0x7f14d015bce0 2023-10-10T20:26:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14d5694700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d015c520 con 0x7f14d00c6bd0 2023-10-10T20:26:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14d5694700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d006df10 con 0x7f14d015c0b0 2023-10-10T20:26:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14ceffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f14d015bce0 0x7f14d00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40112/0 (socket says 172.21.15.148:40112) 2023-10-10T20:26:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14ceffd700 1 -- 172.21.15.148:0/1391619255 learned_addr learned my addr 172.21.15.148:0/1391619255 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14cf7fe700 1 --2- 172.21.15.148:0/1391619255 >> v2:172.21.15.161:3300/0 conn(0x7f14d00c6bd0 0x7f14d006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.674+0000 7f14ce7fc700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14d00c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518616557 0 0) 0x7f14d006dc30 con 0x7f14d015bce0 2023-10-10T20:26:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b8003570 con 0x7f14d015bce0 2023-10-10T20:26:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388482926 0 0) 0x7f14b8003570 con 0x7f14d015bce0 2023-10-10T20:26:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 msgr2=0x7f14d00c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14d00c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 >> v2:172.21.15.161:3300/0 conn(0x7f14d00c6bd0 msgr2=0x7f14d006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 --2- 172.21.15.148:0/1391619255 >> v2:172.21.15.161:3300/0 conn(0x7f14d00c6bd0 0x7f14d006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d006e1f0 con 0x7f14d015bce0 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f14c00017f0 con 0x7f14d015bce0 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 >> v1:172.21.15.148:6789/0 conn(0x7f14d015bce0 legacy=0x7f14d00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8003a60 0x7f14b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8004810 0x7f14b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d015c520 con 0x7f14b8003a60 2023-10-10T20:26:05.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d006df10 con 0x7f14b8004810 2023-10-10T20:26:05.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b800ba60 con 0x7f14d015c0b0 2023-10-10T20:26:05.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8003a60 0x7f14b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ce7fc700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8004810 0x7f14b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14cf7fe700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 -- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 msgr2=0x7f14b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d015c0b0 0x7f14b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 -- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8004810 msgr2=0x7f14b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8004810 0x7f14b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.675+0000 7f14ceffd700 1 -- 172.21.15.148:0/1391619255 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8003570 con 0x7f14b8003a60 2023-10-10T20:26:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14ceffd700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8003a60 0x7f14b8003e70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14c000a9a0 tx=0x7f14d00aa680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0b4b0eaeaf7d6d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00070d0 con 0x7f14b8003a60 2023-10-10T20:26:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0007230 con 0x7f14b8003a60 2023-10-10T20:26:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14cdffb700 1 -- 172.21.15.148:0/1391619255 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00075f0 con 0x7f14b8003a60 2023-10-10T20:26:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 -- 172.21.15.148:0/1391619255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8003a60 msgr2=0x7f14b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 --2- 172.21.15.148:0/1391619255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14b8003a60 0x7f14b8003e70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14c000a9a0 tx=0x7f14d00aa680 comp rx=0 tx=0).stop 2023-10-10T20:26:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14cf7fe700 1 -- 172.21.15.148:0/1391619255 reap_dead start 2023-10-10T20:26:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 -- 172.21.15.148:0/1391619255 shutdown_connections 2023-10-10T20:26:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 -- 172.21.15.148:0/1391619255 >> 172.21.15.148:0/1391619255 conn(0x7f14d00cd230 msgr2=0x7f14d00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 -- 172.21.15.148:0/1391619255 shutdown_connections 2023-10-10T20:26:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.676+0000 7f14d5694700 1 -- 172.21.15.148:0/1391619255 wait complete. 2023-10-10T20:26:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.677+0000 7f14d5694700 1 Processor -- start 2023-10-10T20:26:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.677+0000 7f14d5694700 1 -- start start 2023-10-10T20:26:05.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 0x7f14d015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 0x7f14d015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00c17c0 con 0x7f14d00c5780 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00c0bb0 con 0x7f14b8003a60 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14d5694700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00c0e90 con 0x7f14d0159d70 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14ce7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14ce7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37506/0 (socket says 172.21.15.148:37506) 2023-10-10T20:26:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14ce7fc700 1 -- 172.21.15.148:0/4227563367 learned_addr learned my addr 172.21.15.148:0/4227563367 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14ceffd700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 0x7f14d015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14cf7fe700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 0x7f14d015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.678+0000 7f14ce7fc700 1 -- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 msgr2=0x7f14d015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ce7fc700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 0x7f14d015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ce7fc700 1 -- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 msgr2=0x7f14d015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ce7fc700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 0x7f14d015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ce7fc700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00c1170 con 0x7f14d00c5780 2023-10-10T20:26:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ceffd700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 0x7f14d015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14cf7fe700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 0x7f14d015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14ce7fc700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011d40 tx=0x7f14c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400fac0 con 0x7f14d00c5780 2023-10-10T20:26:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c400fc20 con 0x7f14d00c5780 2023-10-10T20:26:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00c1450 con 0x7f14d00c5780 2023-10-10T20:26:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4014440 con 0x7f14d00c5780 2023-10-10T20:26:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.679+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d00c0840 con 0x7f14d00c5780 2023-10-10T20:26:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.680+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d00c1970 con 0x7f14d00c5780 2023-10-10T20:26:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.680+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14c4014a80 con 0x7f14d00c5780 2023-10-10T20:26:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.681+0000 7f14b6ffd700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 0x7f14bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.681+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f14c4018030 con 0x7f14d00c5780 2023-10-10T20:26:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.681+0000 7f14ceffd700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 0x7f14bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.681+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=137}) v3 -- 0x7f14bc06dcf0 con 0x7f14d00c5780 2023-10-10T20:26:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.681+0000 7f14ceffd700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 0x7f14bc0693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f14c000a940 tx=0x7f14c0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.685+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14c4022060 con 0x7f14d00c5780 2023-10-10T20:26:05.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:05.871+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14d004c3c0 con 0x7f14d00c5780 2023-10-10T20:26:06.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.742+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(137..137 src has 1..137) v4 ==== 288+0+0 (secure 0 0 0) 0x7f14c40568e0 con 0x7f14d00c5780 2023-10-10T20:26:06.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.742+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7f14bc06ca80 con 0x7f14d00c5780 2023-10-10T20:26:06.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.748+0000 7f14b6ffd700 1 -- 172.21.15.148:0/4227563367 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) v1 ==== 169+0+0 (secure 0 0 0) 0x7f14c405b030 con 0x7f14d00c5780 2023-10-10T20:26:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 msgr2=0x7f14bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 0x7f14bc0693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f14c000a940 tx=0x7f14c0016000 comp rx=0 tx=0).stop 2023-10-10T20:26:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 msgr2=0x7f14d00c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011d40 tx=0x7f14c40091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 shutdown_connections 2023-10-10T20:26:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f14bc066f30 0x7f14bc0693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14d0159d70 0x7f14d015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b8003a60 0x7f14d015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.755+0000 7f14d5694700 1 --2- 172.21.15.148:0/4227563367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14d00c5780 0x7f14d00c5bf0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.756+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 >> 172.21.15.148:0/4227563367 conn(0x7f14d00cd230 msgr2=0x7f14d006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.756+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 shutdown_connections 2023-10-10T20:26:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.758+0000 7f14d5694700 1 -- 172.21.15.148:0/4227563367 wait complete. 2023-10-10T20:26:06.760 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:26:06.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:26:06.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.890+0000 7f70939dc700 1 Processor -- start 2023-10-10T20:26:06.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.890+0000 7f70939dc700 1 -- start start 2023-10-10T20:26:06.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.891+0000 7f70939dc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 0x7f708c15db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.891+0000 7f70939dc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 0x7f708c15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.891+0000 7f70939dc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0cae70 con 0x7f708c0c9390 2023-10-10T20:26:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.891+0000 7f70939dc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0cb150 con 0x7f708c0cbf10 2023-10-10T20:26:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.891+0000 7f70939dc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0cb430 con 0x7f708c0678b0 2023-10-10T20:26:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7090f77700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 0x7f708c15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 0x7f708c15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091f79700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f708c0c9390 0x7f708c06bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43962/0 (socket says 172.21.15.148:43962) 2023-10-10T20:26:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091f79700 1 -- 172.21.15.148:0/285364416 learned_addr learned my addr 172.21.15.148:0/285364416 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 -- 172.21.15.148:0/285364416 >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 msgr2=0x7f708c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 --2- 172.21.15.148:0/285364416 >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 0x7f708c15e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 -- 172.21.15.148:0/285364416 >> v1:172.21.15.161:6789/0 conn(0x7f708c0c9390 legacy=0x7f708c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 -- 172.21.15.148:0/285364416 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0cb710 con 0x7f708c0678b0 2023-10-10T20:26:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7090f77700 1 --2- 172.21.15.148:0/285364416 >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 0x7f708c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7091778700 1 --2- 172.21.15.148:0/285364416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 0x7f708c15db80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70880098e0 tx=0x7f70880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=703fc690950d12a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7083fff700 1 -- 172.21.15.148:0/285364416 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800b640 con 0x7f708c0678b0 2023-10-10T20:26:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7083fff700 1 -- 172.21.15.148:0/285364416 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708800b7a0 con 0x7f708c0678b0 2023-10-10T20:26:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.892+0000 7f7083fff700 1 -- 172.21.15.148:0/285364416 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800bb40 con 0x7f708c0678b0 2023-10-10T20:26:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 -- 172.21.15.148:0/285364416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 msgr2=0x7f708c15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 --2- 172.21.15.148:0/285364416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 0x7f708c15db80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70880098e0 tx=0x7f70880091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 -- 172.21.15.148:0/285364416 shutdown_connections 2023-10-10T20:26:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 --2- 172.21.15.148:0/285364416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0678b0 0x7f708c15db80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 --2- 172.21.15.148:0/285364416 >> v2:172.21.15.161:3300/0 conn(0x7f708c0cbf10 0x7f708c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 -- 172.21.15.148:0/285364416 >> 172.21.15.148:0/285364416 conn(0x7f708c069870 msgr2=0x7f708c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 -- 172.21.15.148:0/285364416 shutdown_connections 2023-10-10T20:26:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.893+0000 7f70939dc700 1 -- 172.21.15.148:0/285364416 wait complete. 2023-10-10T20:26:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.894+0000 7f70939dc700 1 Processor -- start 2023-10-10T20:26:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.894+0000 7f70939dc700 1 -- start start 2023-10-10T20:26:06.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0c9390 0x7f708c16e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 0x7f708c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c15e100 con 0x7f708c0678b0 2023-10-10T20:26:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1503a0 con 0x7f708c0cbf10 2023-10-10T20:26:06.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f70939dc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c16dac0 con 0x7f708c0c9390 2023-10-10T20:26:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37512/0 (socket says 172.21.15.148:37512) 2023-10-10T20:26:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 -- 172.21.15.148:0/2469011928 learned_addr learned my addr 172.21.15.148:0/2469011928 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091f79700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 0x7f708c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7090f77700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0c9390 0x7f708c16e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 -- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0c9390 msgr2=0x7f708c16e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0c9390 0x7f708c16e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 -- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 msgr2=0x7f708c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.895+0000 7f7091778700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 0x7f708c15b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f7091778700 1 -- 172.21.15.148:0/2469011928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009420 con 0x7f708c0678b0 2023-10-10T20:26:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f7091f79700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 0x7f708c15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f7091778700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f708c0cb570 tx=0x7f7088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708801bcd0 con 0x7f708c0678b0 2023-10-10T20:26:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708801be30 con 0x7f708c0678b0 2023-10-10T20:26:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088011670 con 0x7f708c0678b0 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7088009420 con 0x7f708c0678b0 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.896+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c15ce20 con 0x7f708c0678b0 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.897+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c04c3c0 con 0x7f708c0678b0 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.897+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7088011a20 con 0x7f708c0678b0 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.898+0000 7f70817fa700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:06.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.898+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7088085820 con 0x7f708c0678b0 2023-10-10T20:26:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.898+0000 7f7090f77700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.898+0000 7f7090f77700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 0x7f70700693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f707c006f00 tx=0x7f707c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:06.902+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7088052d90 con 0x7f708c0678b0 2023-10-10T20:26:07.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.085+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f708c068da0 con 0x7f708c0678b0 2023-10-10T20:26:07.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.089+0000 7f70817fa700 1 -- 172.21.15.148:0/2469011928 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70880563b0 con 0x7f708c0678b0 2023-10-10T20:26:07.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 msgr2=0x7f70700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 0x7f70700693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f707c006f00 tx=0x7f707c008040 comp rx=0 tx=0).stop 2023-10-10T20:26:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 msgr2=0x7f708c14fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f708c0cb570 tx=0x7f7088017000 comp rx=0 tx=0).stop 2023-10-10T20:26:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 shutdown_connections 2023-10-10T20:26:07.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7070066f30 0x7f70700693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f708c0cbf10 0x7f708c15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f708c0678b0 0x7f708c14fe60 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 --2- 172.21.15.148:0/2469011928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f708c0c9390 0x7f708c16e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 >> 172.21.15.148:0/2469011928 conn(0x7f708c069870 msgr2=0x7f708c0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:07.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 shutdown_connections 2023-10-10T20:26:07.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.091+0000 7f70939dc700 1 -- 172.21.15.148:0/2469011928 wait complete. 2023-10-10T20:26:07.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-10T20:26:07.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.217+0000 7f2cc7115700 1 Processor -- start 2023-10-10T20:26:07.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.217+0000 7f2cc7115700 1 -- start start 2023-10-10T20:26:07.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc7115700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cc015a620 0x7f2cc015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc7115700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 0x7f2cc0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc7115700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc015cbb0 con 0x7f2cc0157870 2023-10-10T20:26:07.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc7115700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc015bdb0 con 0x7f2cc015a620 2023-10-10T20:26:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc7115700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc015bfa0 con 0x7f2cc015d580 2023-10-10T20:26:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cb7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cc015a620 0x7f2cc015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc4eb1700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2cc0157870 0x7f2cc015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40114/0 (socket says 172.21.15.148:40114) 2023-10-10T20:26:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc4eb1700 1 -- 172.21.15.148:0/1829243698 learned_addr learned my addr 172.21.15.148:0/1829243698 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:07.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cb7fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cc015a620 0x7f2cc015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37514/0 (socket says 172.21.15.148:37514) 2023-10-10T20:26:07.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cc56b2700 1 --2- 172.21.15.148:0/1829243698 >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 0x7f2cc0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1829243698 >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 msgr2=0x7f2cc0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:07.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.218+0000 7f2cb77fe700 1 -- 172.21.15.148:0/1829243698 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465991781 0 0) 0x7f2cc015cbb0 con 0x7f2cc0157870 2023-10-10T20:26:07.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb7fff700 1 --2- 172.21.15.148:0/1829243698 >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 0x7f2cc0158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1829243698 >> v1:172.21.15.148:6789/0 conn(0x7f2cc0157870 legacy=0x7f2cc015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1829243698 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc015c280 con 0x7f2cc015a620 2023-10-10T20:26:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc56b2700 1 --2- 172.21.15.148:0/1829243698 >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 0x7f2cc0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb7fff700 1 --2- 172.21.15.148:0/1829243698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc015d040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2cc006b4e0 tx=0x7f2cb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb434689702e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb77fe700 1 -- 172.21.15.148:0/1829243698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000b6b0 con 0x7f2cc015a620 2023-10-10T20:26:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb77fe700 1 -- 172.21.15.148:0/1829243698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb000b810 con 0x7f2cc015a620 2023-10-10T20:26:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cb77fe700 1 -- 172.21.15.148:0/1829243698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000bbb0 con 0x7f2cc015a620 2023-10-10T20:26:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 -- 172.21.15.148:0/1829243698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 msgr2=0x7f2cc015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1829243698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc015d040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2cc006b4e0 tx=0x7f2cb00091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 -- 172.21.15.148:0/1829243698 shutdown_connections 2023-10-10T20:26:07.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1829243698 >> v2:172.21.15.161:3300/0 conn(0x7f2cc015d580 0x7f2cc0158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1829243698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc015d040 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 -- 172.21.15.148:0/1829243698 >> 172.21.15.148:0/1829243698 conn(0x7f2cc0068ce0 msgr2=0x7f2cc00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:07.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 -- 172.21.15.148:0/1829243698 shutdown_connections 2023-10-10T20:26:07.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.219+0000 7f2cc7115700 1 -- 172.21.15.148:0/1829243698 wait complete. 2023-10-10T20:26:07.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.220+0000 7f2cc7115700 1 Processor -- start 2023-10-10T20:26:07.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 -- start start 2023-10-10T20:26:07.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 0x7f2cc0153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 0x7f2cc014a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc016b790 con 0x7f2cc015a620 2023-10-10T20:26:07.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0154f50 con 0x7f2cc015d580 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cc7115700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0154340 con 0x7f2cc0157870 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cb7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.221+0000 7f2cb7fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37530/0 (socket says 172.21.15.148:37530) 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1734543403 learned_addr learned my addr 172.21.15.148:0/1734543403 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc4eb1700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 0x7f2cc0153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc56b2700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 0x7f2cc014a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 msgr2=0x7f2cc0153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 0x7f2cc0153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 msgr2=0x7f2cc014a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 0x7f2cc014a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 -- 172.21.15.148:0/1734543403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009420 con 0x7f2cc015a620 2023-10-10T20:26:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc56b2700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 0x7f2cc014a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc4eb1700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 0x7f2cc0153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb7fff700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2cc006b530 tx=0x7f2cb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0011420 con 0x7f2cc015a620 2023-10-10T20:26:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0011580 con 0x7f2cc015a620 2023-10-10T20:26:07.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc0154620 con 0x7f2cc015a620 2023-10-10T20:26:07.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0011920 con 0x7f2cc015a620 2023-10-10T20:26:07.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.222+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc0154be0 con 0x7f2cc015a620 2023-10-10T20:26:07.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.223+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc004c3c0 con 0x7f2cc015a620 2023-10-10T20:26:07.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.223+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cb0011d10 con 0x7f2cc015a620 2023-10-10T20:26:07.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.224+0000 7f2cb4ff9700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 0x7f2cac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.224+0000 7f2cc4eb1700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 0x7f2cac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.224+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2cb0085820 con 0x7f2cc015a620 2023-10-10T20:26:07.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.224+0000 7f2cc4eb1700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 0x7f2cac0693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2cc01558f0 tx=0x7f2cbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.236 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.228+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cb0052e10 con 0x7f2cc015a620 2023-10-10T20:26:07.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.488+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f2cc0155090 con 0x7f2cc015a620 2023-10-10T20:26:07.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.758+0000 7f2cb4ff9700 1 -- 172.21.15.148:0/1734543403 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v350) v1 ==== 140+0+0 (secure 0 0 0) 0x7f2cb0056430 con 0x7f2cc015a620 2023-10-10T20:26:07.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 msgr2=0x7f2cac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 0x7f2cac0693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2cc01558f0 tx=0x7f2cbc009040 comp rx=0 tx=0).stop 2023-10-10T20:26:07.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 msgr2=0x7f2cc016b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2cc006b530 tx=0x7f2cb0017000 comp rx=0 tx=0).stop 2023-10-10T20:26:07.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 shutdown_connections 2023-10-10T20:26:07.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2cac066f30 0x7f2cac0693e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc015d580 0x7f2cc014a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc0157870 0x7f2cc0153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 --2- 172.21.15.148:0/1734543403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cc015a620 0x7f2cc016b250 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 >> 172.21.15.148:0/1734543403 conn(0x7f2cc0068ce0 msgr2=0x7f2cc0158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:07.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 shutdown_connections 2023-10-10T20:26:07.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.761+0000 7f2cc7115700 1 -- 172.21.15.148:0/1734543403 wait complete. 2023-10-10T20:26:07.778 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-10T20:26:07.779 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-10-10T20:26:07.779 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-10T20:26:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 Processor -- start 2023-10-10T20:26:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 -- start start 2023-10-10T20:26:07.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63cc15a2b0 0x7f63cc0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63cc0cfe20 0x7f63cc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc0c5d80 con 0x7f63cc0d1c30 2023-10-10T20:26:07.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc068470 con 0x7f63cc15a2b0 2023-10-10T20:26:07.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.894+0000 7f63d1463700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0c6060 con 0x7f63cc0cfe20 2023-10-10T20:26:07.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63cc15a2b0 0x7f63cc0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63c27fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f63cc0d1c30 0x7f63cc0cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40116/0 (socket says 172.21.15.148:40116) 2023-10-10T20:26:07.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63cc15a2b0 0x7f63cc0b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37538/0 (socket says 172.21.15.148:37538) 2023-10-10T20:26:07.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63c27fc700 1 -- 172.21.15.148:0/2104358359 learned_addr learned my addr 172.21.15.148:0/2104358359 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:07.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63cb7fe700 1 --2- 172.21.15.148:0/2104358359 >> v2:172.21.15.161:3300/0 conn(0x7f63cc0cfe20 0x7f63cc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 -- 172.21.15.148:0/2104358359 >> v2:172.21.15.161:3300/0 conn(0x7f63cc0cfe20 msgr2=0x7f63cc0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:07.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 --2- 172.21.15.148:0/2104358359 >> v2:172.21.15.161:3300/0 conn(0x7f63cc0cfe20 0x7f63cc0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 -- 172.21.15.148:0/2104358359 >> v1:172.21.15.148:6789/0 conn(0x7f63cc0d1c30 legacy=0x7f63cc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 -- 172.21.15.148:0/2104358359 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0c6340 con 0x7f63cc15a2b0 2023-10-10T20:26:07.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63ca7fc700 1 -- 172.21.15.148:0/2104358359 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102394923 0 0) 0x7f63cc0c5d80 con 0x7f63cc0d1c30 2023-10-10T20:26:07.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63caffd700 1 --2- 172.21.15.148:0/2104358359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc15a2b0 0x7f63cc0b3da0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f63cc064970 tx=0x7f63bc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e58649282be43 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63ca7fc700 1 -- 172.21.15.148:0/2104358359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc005c10 con 0x7f63cc15a2b0 2023-10-10T20:26:07.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63ca7fc700 1 -- 172.21.15.148:0/2104358359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc005d70 con 0x7f63cc15a2b0 2023-10-10T20:26:07.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.895+0000 7f63ca7fc700 1 -- 172.21.15.148:0/2104358359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc006260 con 0x7f63cc15a2b0 2023-10-10T20:26:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 -- 172.21.15.148:0/2104358359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc15a2b0 msgr2=0x7f63cc0b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 --2- 172.21.15.148:0/2104358359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc15a2b0 0x7f63cc0b3da0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f63cc064970 tx=0x7f63bc009120 comp rx=0 tx=0).stop 2023-10-10T20:26:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 -- 172.21.15.148:0/2104358359 shutdown_connections 2023-10-10T20:26:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 --2- 172.21.15.148:0/2104358359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc15a2b0 0x7f63cc0b3da0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 --2- 172.21.15.148:0/2104358359 >> v2:172.21.15.161:3300/0 conn(0x7f63cc0cfe20 0x7f63cc0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 -- 172.21.15.148:0/2104358359 >> 172.21.15.148:0/2104358359 conn(0x7f63cc067e40 msgr2=0x7f63cc0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 -- 172.21.15.148:0/2104358359 shutdown_connections 2023-10-10T20:26:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.896+0000 7f63d1463700 1 -- 172.21.15.148:0/2104358359 wait complete. 2023-10-10T20:26:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.897+0000 7f63d1463700 1 Processor -- start 2023-10-10T20:26:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.897+0000 7f63d1463700 1 -- start start 2023-10-10T20:26:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63cc0d1c30 0x7f63cc0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63cc15a2b0 0x7f63cc16a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0c9ec0 con 0x7f63cc0cfe20 2023-10-10T20:26:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0c92b0 con 0x7f63cc0d1c30 2023-10-10T20:26:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63d1463700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0c9590 con 0x7f63cc15a2b0 2023-10-10T20:26:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37548/0 (socket says 172.21.15.148:37548) 2023-10-10T20:26:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 -- 172.21.15.148:0/3289787266 learned_addr learned my addr 172.21.15.148:0/3289787266 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 -- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63cc15a2b0 msgr2=0x7f63cc16a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63cc15a2b0 0x7f63cc16a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63c27fc700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63cc0d1c30 0x7f63cc0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 -- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63cc0d1c30 msgr2=0x7f63cc0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63cc0d1c30 0x7f63cc0ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 -- 172.21.15.148:0/3289787266 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0093c0 con 0x7f63cc0cfe20 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.898+0000 7f63caffd700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00b0e0 tx=0x7f63bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.899+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00bd20 con 0x7f63cc0cfe20 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.899+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00be80 con 0x7f63cc0cfe20 2023-10-10T20:26:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.899+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc0c9870 con 0x7f63cc0cfe20 2023-10-10T20:26:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.899+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc026640 con 0x7f63cc0cfe20 2023-10-10T20:26:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.899+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc158110 con 0x7f63cc0cfe20 2023-10-10T20:26:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.900+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc04c3c0 con 0x7f63cc0cfe20 2023-10-10T20:26:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.900+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63bc01a420 con 0x7f63cc0cfe20 2023-10-10T20:26:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.900+0000 7f63c37fe700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 0x7f63ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.900+0000 7f63c27fc700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 0x7f63ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.901+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f63bc085e00 con 0x7f63cc0cfe20 2023-10-10T20:26:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.901+0000 7f63c27fc700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 0x7f63ac0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f63cc15eb20 tx=0x7f63b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:07.905+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63bc053370 con 0x7f63cc0cfe20 2023-10-10T20:26:08.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.164+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f63cc064a00 con 0x7f63cc0cfe20 2023-10-10T20:26:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.760+0000 7f63c37fe700 1 -- 172.21.15.148:0/3289787266 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v351) v1 ==== 135+0+0 (secure 0 0 0) 0x7f63bc02b0c0 con 0x7f63cc0cfe20 2023-10-10T20:26:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 msgr2=0x7f63ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 0x7f63ac0693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f63cc15eb20 tx=0x7f63b4009040 comp rx=0 tx=0).stop 2023-10-10T20:26:08.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 msgr2=0x7f63cc157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00b0e0 tx=0x7f63bc016000 comp rx=0 tx=0).stop 2023-10-10T20:26:08.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 shutdown_connections 2023-10-10T20:26:08.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f63ac066f30 0x7f63ac0693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63cc15a2b0 0x7f63cc16a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63cc0cfe20 0x7f63cc157bd0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 --2- 172.21.15.148:0/3289787266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63cc0d1c30 0x7f63cc0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 >> 172.21.15.148:0/3289787266 conn(0x7f63cc067e40 msgr2=0x7f63cc0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 shutdown_connections 2023-10-10T20:26:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.762+0000 7f63d1463700 1 -- 172.21.15.148:0/3289787266 wait complete. 2023-10-10T20:26:08.776 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:26:08.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.892+0000 7fa45659d700 1 Processor -- start 2023-10-10T20:26:08.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.892+0000 7fa45659d700 1 -- start start 2023-10-10T20:26:08.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa45659d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa4500af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa45659d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa45006acb0 0x7fa4500afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa45659d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500b0790 con 0x7fa45015a290 2023-10-10T20:26:08.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa45659d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500ce1f0 con 0x7fa45006acb0 2023-10-10T20:26:08.902 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa45659d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b0b20 con 0x7fa450159ec0 2023-10-10T20:26:08.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa454b3a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa45006acb0 0x7fa4500afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa44f7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa45015a290 0x7fa45006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40118/0 (socket says 172.21.15.148:40118) 2023-10-10T20:26:08.903 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.893+0000 7fa44f7fe700 1 -- 172.21.15.148:0/1122916308 learned_addr learned my addr 172.21.15.148:0/1122916308 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:08.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa44ffff700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa4500af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa454b3a700 1 -- 172.21.15.148:0/1122916308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 msgr2=0x7fa4500af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:08.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa454b3a700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa4500af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa454b3a700 1 -- 172.21.15.148:0/1122916308 >> v1:172.21.15.148:6789/0 conn(0x7fa45015a290 legacy=0x7fa45006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa454b3a700 1 -- 172.21.15.148:0/1122916308 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500b04b0 con 0x7fa45006acb0 2023-10-10T20:26:08.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa44effd700 1 -- 172.21.15.148:0/1122916308 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404063211 0 0) 0x7fa4500b0790 con 0x7fa45015a290 2023-10-10T20:26:08.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa454b3a700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45006acb0 0x7fa4500afe80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa448009e60 tx=0x7fa448009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da8610a1c5687bd server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:08.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa44effd700 1 -- 172.21.15.148:0/1122916308 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800e070 con 0x7fa45006acb0 2023-10-10T20:26:08.906 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa44effd700 1 -- 172.21.15.148:0/1122916308 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44800bc30 con 0x7fa45006acb0 2023-10-10T20:26:08.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa44effd700 1 -- 172.21.15.148:0/1122916308 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448012430 con 0x7fa45006acb0 2023-10-10T20:26:08.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 -- 172.21.15.148:0/1122916308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45006acb0 msgr2=0x7fa4500afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45006acb0 0x7fa4500afe80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa448009e60 tx=0x7fa448009780 comp rx=0 tx=0).stop 2023-10-10T20:26:08.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 -- 172.21.15.148:0/1122916308 shutdown_connections 2023-10-10T20:26:08.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45006acb0 0x7fa4500afe80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 --2- 172.21.15.148:0/1122916308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa4500af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 -- 172.21.15.148:0/1122916308 >> 172.21.15.148:0/1122916308 conn(0x7fa4500d1880 msgr2=0x7fa4500cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:08.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 -- 172.21.15.148:0/1122916308 shutdown_connections 2023-10-10T20:26:08.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.894+0000 7fa45659d700 1 -- 172.21.15.148:0/1122916308 wait complete. 2023-10-10T20:26:08.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.895+0000 7fa45659d700 1 Processor -- start 2023-10-10T20:26:08.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 -- start start 2023-10-10T20:26:08.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45006acb0 0x7fa4500c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa450156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.911 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450157980 con 0x7fa45015a290 2023-10-10T20:26:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450156ad0 con 0x7fa45006acb0 2023-10-10T20:26:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa45659d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa450156d70 con 0x7fa450159ec0 2023-10-10T20:26:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.896+0000 7fa454b3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37564/0 (socket says 172.21.15.148:37564) 2023-10-10T20:26:08.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 -- 172.21.15.148:0/1716894318 learned_addr learned my addr 172.21.15.148:0/1716894318 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:08.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa44ffff700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45006acb0 0x7fa4500c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa44f7fe700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa450156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 -- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 msgr2=0x7fa450156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa450156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 -- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45006acb0 msgr2=0x7fa4500c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:08.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45006acb0 0x7fa4500c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:08.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 -- 172.21.15.148:0/1716894318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448009a20 con 0x7fa45015a290 2023-10-10T20:26:08.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa44f7fe700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa450156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:08.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa454b3a700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa448013000 tx=0x7fa448019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:08.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448012e70 con 0x7fa45015a290 2023-10-10T20:26:08.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44801e630 con 0x7fa45015a290 2023-10-10T20:26:08.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450157050 con 0x7fa45015a290 2023-10-10T20:26:08.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44801ea10 con 0x7fa45015a290 2023-10-10T20:26:08.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.897+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450157610 con 0x7fa45015a290 2023-10-10T20:26:08.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.898+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450157af0 con 0x7fa45015a290 2023-10-10T20:26:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.899+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa448024070 con 0x7fa45015a290 2023-10-10T20:26:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.899+0000 7fa437fff700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.899+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa448086a10 con 0x7fa45015a290 2023-10-10T20:26:08.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.899+0000 7fa44ffff700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:08.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.899+0000 7fa44ffff700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa440006fd0 tx=0x7fa440008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:08.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:08.903+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa448054030 con 0x7fa45015a290 2023-10-10T20:26:09.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.084+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa45004c3c0 con 0x7fa45015a290 2023-10-10T20:26:09.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.088+0000 7fa437fff700 1 -- 172.21.15.148:0/1716894318 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa448057650 con 0x7fa45015a290 2023-10-10T20:26:09.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 msgr2=0x7fa43c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 0x7fa43c0693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa440006fd0 tx=0x7fa440008040 comp rx=0 tx=0).stop 2023-10-10T20:26:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 msgr2=0x7fa4500ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa448013000 tx=0x7fa448019000 comp rx=0 tx=0).stop 2023-10-10T20:26:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 shutdown_connections 2023-10-10T20:26:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa43c066f30 0x7fa43c0693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45015a290 0x7fa4500ca1a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa45006acb0 0x7fa4500c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 --2- 172.21.15.148:0/1716894318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa450159ec0 0x7fa450156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 >> 172.21.15.148:0/1716894318 conn(0x7fa4500d1880 msgr2=0x7fa4500afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:09.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 shutdown_connections 2023-10-10T20:26:09.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.090+0000 7fa45659d700 1 -- 172.21.15.148:0/1716894318 wait complete. 2023-10-10T20:26:09.104 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:26:09.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:26:09.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:26:09.197 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:26:09.197 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:26:09.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.311+0000 7fb55eb44700 1 Processor -- start 2023-10-10T20:26:09.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.311+0000 7fb55eb44700 1 -- start start 2023-10-10T20:26:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55eb44700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb558157870 0x7fb55815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55eb44700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55815cbb0 con 0x7fb558157c40 2023-10-10T20:26:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55eb44700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55815bfa0 con 0x7fb55815a620 2023-10-10T20:26:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55eb44700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55815c280 con 0x7fb558157870 2023-10-10T20:26:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb558157870 0x7fb55815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb558157870 0x7fb55815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37576/0 (socket says 172.21.15.148:37576) 2023-10-10T20:26:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55d0e1700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb558157c40 0x7fb558158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40132/0 (socket says 172.21.15.148:40132) 2023-10-10T20:26:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 -- 172.21.15.148:0/465889909 learned_addr learned my addr 172.21.15.148:0/465889909 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 -- 172.21.15.148:0/465889909 >> v1:172.21.15.161:6789/0 conn(0x7fb55815a620 legacy=0x7fb55815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 -- 172.21.15.148:0/465889909 >> v1:172.21.15.148:6789/0 conn(0x7fb558157c40 legacy=0x7fb558158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.312+0000 7fb55c8e0700 1 -- 172.21.15.148:0/465889909 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55815c560 con 0x7fb558157870 2023-10-10T20:26:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55c8e0700 1 --2- 172.21.15.148:0/465889909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157870 0x7fb55815b6a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb5540099b0 tx=0x7fb554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb33210b46b443a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb54f7fe700 1 -- 172.21.15.148:0/465889909 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400b820 con 0x7fb558157870 2023-10-10T20:26:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb54f7fe700 1 -- 172.21.15.148:0/465889909 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55400b980 con 0x7fb558157870 2023-10-10T20:26:09.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb54f7fe700 1 -- 172.21.15.148:0/465889909 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400bd20 con 0x7fb558157870 2023-10-10T20:26:09.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 -- 172.21.15.148:0/465889909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157870 msgr2=0x7fb55815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 --2- 172.21.15.148:0/465889909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157870 0x7fb55815b6a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb5540099b0 tx=0x7fb554009250 comp rx=0 tx=0).stop 2023-10-10T20:26:09.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 -- 172.21.15.148:0/465889909 shutdown_connections 2023-10-10T20:26:09.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 --2- 172.21.15.148:0/465889909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157870 0x7fb55815b6a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 -- 172.21.15.148:0/465889909 >> 172.21.15.148:0/465889909 conn(0x7fb558068ce0 msgr2=0x7fb5580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:09.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 -- 172.21.15.148:0/465889909 shutdown_connections 2023-10-10T20:26:09.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.313+0000 7fb55eb44700 1 -- 172.21.15.148:0/465889909 wait complete. 2023-10-10T20:26:09.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.314+0000 7fb55eb44700 1 Processor -- start 2023-10-10T20:26:09.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 -- start start 2023-10-10T20:26:09.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558157870 0x7fb5580b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb55815a620 0x7fb5580b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558150f50 con 0x7fb558157c40 2023-10-10T20:26:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5580b27f0 con 0x7fb558157870 2023-10-10T20:26:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb55eb44700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5580b1480 con 0x7fb55815a620 2023-10-10T20:26:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb54ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb54ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37590/0 (socket says 172.21.15.148:37590) 2023-10-10T20:26:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.315+0000 7fb54ffff700 1 -- 172.21.15.148:0/3930244754 learned_addr learned my addr 172.21.15.148:0/3930244754 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb55c8e0700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558157870 0x7fb5580b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb55d0e1700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb55815a620 0x7fb5580b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 -- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb55815a620 msgr2=0x7fb5580b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb55815a620 0x7fb5580b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 -- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558157870 msgr2=0x7fb5580b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558157870 0x7fb5580b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 -- 172.21.15.148:0/3930244754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540094f0 con 0x7fb558157c40 2023-10-10T20:26:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54ffff700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b5e0 tx=0x7fb5480090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800f510 con 0x7fb558157c40 2023-10-10T20:26:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54800f6b0 con 0x7fb558157c40 2023-10-10T20:26:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800fa70 con 0x7fb558157c40 2023-10-10T20:26:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55814ff30 con 0x7fb558157c40 2023-10-10T20:26:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.316+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5581504f0 con 0x7fb558157c40 2023-10-10T20:26:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.317+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558150710 con 0x7fb558157c40 2023-10-10T20:26:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.317+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb54801a400 con 0x7fb558157c40 2023-10-10T20:26:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.318+0000 7fb54cff9700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 0x7fb5440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.318+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb548083db0 con 0x7fb558157c40 2023-10-10T20:26:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.318+0000 7fb55c8e0700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 0x7fb5440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.318+0000 7fb55c8e0700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 0x7fb5440693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5540099b0 tx=0x7fb554016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.322+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5480513a0 con 0x7fb558157c40 2023-10-10T20:26:09.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.548+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb55804c3c0 con 0x7fb558157c40 2023-10-10T20:26:09.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.548+0000 7fb54cff9700 1 -- 172.21.15.148:0/3930244754 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb5480549c0 con 0x7fb558157c40 2023-10-10T20:26:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:09.551 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5930,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3624775339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3624775339},{"type":"v1","addr":"172.21.15.148:6837","nonce":3624775339}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":5936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/84311247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":84311247},{"type":"v1","addr":"172.21.15.161:6834","nonce":84311247}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6152,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1947946611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1947946611},{"type":"v1","addr":"172.21.15.161:6835","nonce":1947946611}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":7083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/564020702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":564020702},{"type":"v1","addr":"172.21.15.148:6836","nonce":564020702}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-10-10T20:26:09.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 msgr2=0x7fb5440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 0x7fb5440693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5540099b0 tx=0x7fb554016000 comp rx=0 tx=0).stop 2023-10-10T20:26:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 msgr2=0x7fb558150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b5e0 tx=0x7fb5480090f0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 shutdown_connections 2023-10-10T20:26:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb544066f30 0x7fb5440693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb55815a620 0x7fb5580b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb558157870 0x7fb5580b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 --2- 172.21.15.148:0/3930244754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb558157c40 0x7fb558150a10 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 >> 172.21.15.148:0/3930244754 conn(0x7fb558068ce0 msgr2=0x7fb5580b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.550+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 shutdown_connections 2023-10-10T20:26:09.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.551+0000 7fb55eb44700 1 -- 172.21.15.148:0/3930244754 wait complete. 2023-10-10T20:26:09.555 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 66 2023-10-10T20:26:09.564 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:26:09.564 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:26:09.565 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:26:09.565 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:26:09.565 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:26:09.566 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:26:09.567 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:26:09.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:26:09.567 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:26:09.567 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:26:09.568 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:26:09.568 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:26:09.568 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:26:09.568 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:26:09.569 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:26:09.569 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:26:09.574 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:26:09.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:26:09.575 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:26:09.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:26:09.579 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:26:09.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:26:09.580 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:26:09.581 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:26:09.617 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:26:09.618 DEBUG:teuthology.parallel:result is None 2023-10-10T20:26:09.619 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:26:09.620 DEBUG:teuthology.parallel:result is None 2023-10-10T20:26:09.623 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:26:09.623 DEBUG:teuthology.parallel:result is None 2023-10-10T20:26:09.624 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:26:09.625 DEBUG:teuthology.parallel:result is None 2023-10-10T20:26:09.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:26:09.691 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:26:09.691 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:26:09.704 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:26:09.705 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:26:09.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.746+0000 7faf6099f700 1 Processor -- start 2023-10-10T20:26:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.746+0000 7faf6099f700 1 -- start start 2023-10-10T20:26:09.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faf5c1577d0 0x7faf5c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf5c157ba0 0x7faf5c06a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c15b0d0 0x7faf5c06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15c360 con 0x7faf5c157ba0 2023-10-10T20:26:09.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15b790 con 0x7faf5c1577d0 2023-10-10T20:26:09.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf6099f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c15ba30 con 0x7faf5c15b0d0 2023-10-10T20:26:09.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.747+0000 7faf59d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf5c157ba0 0x7faf5c06a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf5c157ba0 0x7faf5c06a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37642/0 (socket says 172.21.15.148:37642) 2023-10-10T20:26:09.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 -- 172.21.15.148:0/2295782183 learned_addr learned my addr 172.21.15.148:0/2295782183 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:09.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf5a59c700 1 --2- 172.21.15.148:0/2295782183 >> v2:172.21.15.161:3300/0 conn(0x7faf5c1577d0 0x7faf5c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf5ad9d700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c15b0d0 0x7faf5c06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 -- 172.21.15.148:0/2295782183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c15b0d0 msgr2=0x7faf5c06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c15b0d0 0x7faf5c06aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 -- 172.21.15.148:0/2295782183 >> v2:172.21.15.161:3300/0 conn(0x7faf5c1577d0 msgr2=0x7faf5c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 --2- 172.21.15.148:0/2295782183 >> v2:172.21.15.161:3300/0 conn(0x7faf5c1577d0 0x7faf5c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 -- 172.21.15.148:0/2295782183 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c15bd10 con 0x7faf5c157ba0 2023-10-10T20:26:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf59d9b700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c157ba0 0x7faf5c06a520 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0b6dc0 tx=0x7faf50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d2904416b2d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf5959a700 1 -- 172.21.15.148:0/2295782183 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000b860 con 0x7faf5c157ba0 2023-10-10T20:26:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf5959a700 1 -- 172.21.15.148:0/2295782183 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000b9c0 con 0x7faf5c157ba0 2023-10-10T20:26:09.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.748+0000 7faf5959a700 1 -- 172.21.15.148:0/2295782183 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000bd60 con 0x7faf5c157ba0 2023-10-10T20:26:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 -- 172.21.15.148:0/2295782183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c157ba0 msgr2=0x7faf5c06a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c157ba0 0x7faf5c06a520 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0b6dc0 tx=0x7faf50009250 comp rx=0 tx=0).stop 2023-10-10T20:26:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 -- 172.21.15.148:0/2295782183 shutdown_connections 2023-10-10T20:26:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c15b0d0 0x7faf5c06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 --2- 172.21.15.148:0/2295782183 >> v2:172.21.15.161:3300/0 conn(0x7faf5c1577d0 0x7faf5c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 --2- 172.21.15.148:0/2295782183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c157ba0 0x7faf5c06a520 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 -- 172.21.15.148:0/2295782183 >> 172.21.15.148:0/2295782183 conn(0x7faf5c068ce0 msgr2=0x7faf5c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 -- 172.21.15.148:0/2295782183 shutdown_connections 2023-10-10T20:26:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.749+0000 7faf6099f700 1 -- 172.21.15.148:0/2295782183 wait complete. 2023-10-10T20:26:09.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.750+0000 7faf6099f700 1 Processor -- start 2023-10-10T20:26:09.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.750+0000 7faf6099f700 1 -- start start 2023-10-10T20:26:09.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c1577d0 0x7faf5c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf5c157ba0 0x7faf5c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c15f680 con 0x7faf5c15b0d0 2023-10-10T20:26:09.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b17b0 con 0x7faf5c157ba0 2023-10-10T20:26:09.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b0ba0 con 0x7faf5c1577d0 2023-10-10T20:26:09.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37652/0 (socket says 172.21.15.148:37652) 2023-10-10T20:26:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 -- 172.21.15.148:0/3310775122 learned_addr learned my addr 172.21.15.148:0/3310775122 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5a59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c1577d0 0x7faf5c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf59d9b700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf5c157ba0 0x7faf5c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 -- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c1577d0 msgr2=0x7faf5c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c1577d0 0x7faf5c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 -- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf5c157ba0 msgr2=0x7faf5c15f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf5c157ba0 0x7faf5c15f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 -- 172.21.15.148:0/3310775122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500094f0 con 0x7faf5c15b0d0 2023-10-10T20:26:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf5ad9d700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7faf54009fa0 tx=0x7faf54009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400eb60 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5400ecc0 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c0b0e80 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54014670 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.751+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c0b1440 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.752+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c04c3c0 con 0x7faf5c15b0d0 2023-10-10T20:26:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.753+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf540149e0 con 0x7faf5c15b0d0 2023-10-10T20:26:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.753+0000 7faf4a7fc700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.753+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faf54084870 con 0x7faf5c15b0d0 2023-10-10T20:26:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.753+0000 7faf5a59c700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:09.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.754+0000 7faf5a59c700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 0x7faf3c0693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faf4c006fd0 tx=0x7faf4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:09.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.757+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf54051de0 con 0x7faf5c15b0d0 2023-10-10T20:26:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:09.963+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf5c15bff0 con 0x7faf5c15b0d0 2023-10-10T20:26:10.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.776+0000 7faf4a7fc700 1 -- 172.21.15.148:0/3310775122 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v142) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf54055400 con 0x7faf5c15b0d0 2023-10-10T20:26:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 msgr2=0x7faf3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 0x7faf3c0693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faf4c006fd0 tx=0x7faf4c008040 comp rx=0 tx=0).stop 2023-10-10T20:26:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 msgr2=0x7faf5c155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7faf54009fa0 tx=0x7faf54009900 comp rx=0 tx=0).stop 2023-10-10T20:26:10.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 shutdown_connections 2023-10-10T20:26:10.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf5c15b0d0 0x7faf5c155600 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf5c1577d0 0x7faf5c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 --2- 172.21.15.148:0/3310775122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf5c157ba0 0x7faf5c15f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.778+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 >> 172.21.15.148:0/3310775122 conn(0x7faf5c068ce0 msgr2=0x7faf5c156820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:10.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.779+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 shutdown_connections 2023-10-10T20:26:10.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.779+0000 7faf6099f700 1 -- 172.21.15.148:0/3310775122 wait complete. 2023-10-10T20:26:10.785 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:26:10.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:26:10.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.906+0000 7f97f9433700 1 Processor -- start 2023-10-10T20:26:10.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.906+0000 7f97f9433700 1 -- start start 2023-10-10T20:26:10.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f9433700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97f4157870 0x7f97f415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:10.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f9433700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f415cbb0 con 0x7f97f4157c40 2023-10-10T20:26:10.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f9433700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f415bfa0 con 0x7f97f415a620 2023-10-10T20:26:10.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f9433700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f415c280 con 0x7f97f4157870 2023-10-10T20:26:10.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97f4157870 0x7f97f415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:10.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97f4157c40 0x7f97f4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40144/0 (socket says 172.21.15.148:40144) 2023-10-10T20:26:10.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f37fe700 1 -- 172.21.15.148:0/2249328321 learned_addr learned my addr 172.21.15.148:0/2249328321 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:10.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f97f4157870 0x7f97f415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37654/0 (socket says 172.21.15.148:37654) 2023-10-10T20:26:10.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 -- 172.21.15.148:0/2249328321 >> v1:172.21.15.161:6789/0 conn(0x7f97f415a620 legacy=0x7f97f415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:10.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 -- 172.21.15.148:0/2249328321 >> v1:172.21.15.148:6789/0 conn(0x7f97f4157c40 legacy=0x7f97f4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f1ffb700 1 -- 172.21.15.148:0/2249328321 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192526389 0 0) 0x7f97f415cbb0 con 0x7f97f4157c40 2023-10-10T20:26:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 -- 172.21.15.148:0/2249328321 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f415c560 con 0x7f97f4157870 2023-10-10T20:26:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f2ffd700 1 --2- 172.21.15.148:0/2249328321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f4157870 0x7f97f415b6a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009e60 tx=0x7f97ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8251fdcda1c0da server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f1ffb700 1 -- 172.21.15.148:0/2249328321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00e070 con 0x7f97f4157870 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f1ffb700 1 -- 172.21.15.148:0/2249328321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec00bbf0 con 0x7f97f4157870 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.907+0000 7f97f1ffb700 1 -- 172.21.15.148:0/2249328321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec012430 con 0x7f97f4157870 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 -- 172.21.15.148:0/2249328321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f4157870 msgr2=0x7f97f415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 --2- 172.21.15.148:0/2249328321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f4157870 0x7f97f415b6a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009e60 tx=0x7f97ec009780 comp rx=0 tx=0).stop 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 -- 172.21.15.148:0/2249328321 shutdown_connections 2023-10-10T20:26:10.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 --2- 172.21.15.148:0/2249328321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f4157870 0x7f97f415b6a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 -- 172.21.15.148:0/2249328321 >> 172.21.15.148:0/2249328321 conn(0x7f97f4068ce0 msgr2=0x7f97f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 -- 172.21.15.148:0/2249328321 shutdown_connections 2023-10-10T20:26:10.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.908+0000 7f97f9433700 1 -- 172.21.15.148:0/2249328321 wait complete. 2023-10-10T20:26:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.909+0000 7f97f9433700 1 Processor -- start 2023-10-10T20:26:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.909+0000 7f97f9433700 1 -- start start 2023-10-10T20:26:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97f4157870 0x7f97f415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:10.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4157c40 0x7f97f40d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4160360 con 0x7f97f415a620 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4160640 con 0x7f97f4157c40 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f9433700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4160920 con 0x7f97f4157870 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:10.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37664/0 (socket says 172.21.15.148:37664) 2023-10-10T20:26:10.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 -- 172.21.15.148:0/3054539670 learned_addr learned my addr 172.21.15.148:0/3054539670 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:10.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f27fc700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4157c40 0x7f97f40d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:10.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f2ffd700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97f4157870 0x7f97f415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:10.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 -- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97f4157870 msgr2=0x7f97f415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97f4157870 0x7f97f415fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 -- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4157c40 msgr2=0x7f97f40d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:10.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4157c40 0x7f97f40d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:10.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.910+0000 7f97f37fe700 1 -- 172.21.15.148:0/3054539670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009a20 con 0x7f97f415a620 2023-10-10T20:26:10.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97f37fe700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c5b0 tx=0x7f97e4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:10.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4005b30 con 0x7f97f415a620 2023-10-10T20:26:10.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4005c90 con 0x7f97f415a620 2023-10-10T20:26:10.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f4160c00 con 0x7f97f415a620 2023-10-10T20:26:10.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40072e0 con 0x7f97f415a620 2023-10-10T20:26:10.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.911+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f4160050 con 0x7f97f415a620 2023-10-10T20:26:10.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.912+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4161120 con 0x7f97f415a620 2023-10-10T20:26:10.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.912+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97e4007690 con 0x7f97f415a620 2023-10-10T20:26:10.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.913+0000 7f97eaffd700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 0x7f97d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:10.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.913+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f97e4083650 con 0x7f97f415a620 2023-10-10T20:26:10.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.913+0000 7f97f2ffd700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 0x7f97d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:10.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.913+0000 7f97f2ffd700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 0x7f97d40693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0099f0 tx=0x7f97ec018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:10.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:10.917+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97e404b5a0 con 0x7f97f415a620 2023-10-10T20:26:11.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.170+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97f404c3c0 con 0x7f97f415a620 2023-10-10T20:26:11.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.170+0000 7f97eaffd700 1 -- 172.21.15.148:0/3054539670 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97e4017090 con 0x7f97f415a620 2023-10-10T20:26:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:11.172 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:26:11.172 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:26:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:26:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:26:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.173 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:26:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:26:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:26:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:26:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.175 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:26:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:26:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.177 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:26:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:26:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.178 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:26:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:26:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.179 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:26:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:26:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.180 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.181 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:26:11.182 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:26:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.183 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:26:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:26:11.184 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:26:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:26:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.186 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:26:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:26:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.188 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.189 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:26:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:26:11.190 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.191 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.192 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:26:11.193 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:26:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.194 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:26:11.196 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:26:11.197 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:26:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:26:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.198 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:26:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:26:11.199 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:26:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:26:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.201 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:26:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:26:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:26:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:26:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:26:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:26:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:26:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:26:11.204 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:26:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:26:11.205 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:26:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:26:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:26:11.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:26:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:26:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:26:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:26:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:26:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:26:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:26:11.210 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:26:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:26:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:26:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.211 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:26:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:26:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:26:11.212 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:26:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:26:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:26:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:26:11.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:11.214 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:26:11.214 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:26:11.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 msgr2=0x7f97d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.217 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 0x7f97d40693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0099f0 tx=0x7f97ec018000 comp rx=0 tx=0).stop 2023-10-10T20:26:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 msgr2=0x7f97f41390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c5b0 tx=0x7f97e4007d30 comp rx=0 tx=0).stop 2023-10-10T20:26:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 shutdown_connections 2023-10-10T20:26:11.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f97d4066f30 0x7f97d40693e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.219 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97f415a620 0x7f97f41390f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f97f4157870 0x7f97f415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.220 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.172+0000 7f97f9433700 1 --2- 172.21.15.148:0/3054539670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4157c40 0x7f97f40d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.173+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 >> 172.21.15.148:0/3054539670 conn(0x7f97f4068ce0 msgr2=0x7f97f40b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:11.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.173+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 shutdown_connections 2023-10-10T20:26:11.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.173+0000 7f97f9433700 1 -- 172.21.15.148:0/3054539670 wait complete. 2023-10-10T20:26:11.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:11.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.336+0000 7f7ba808e700 1 Processor -- start 2023-10-10T20:26:11.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.336+0000 7f7ba808e700 1 -- start start 2023-10-10T20:26:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba808e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba808e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba808e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba0068a90 con 0x7f7ba015c4f0 2023-10-10T20:26:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba808e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00ca330 con 0x7f7ba00cb000 2023-10-10T20:26:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba808e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba015ca70 con 0x7f7ba015b340 2023-10-10T20:26:11.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5629700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:57212/0 (socket says 172.21.15.148:57212) 2023-10-10T20:26:11.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5629700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:59366/0 (socket says 172.21.15.148:59366) 2023-10-10T20:26:11.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 -- 172.21.15.148:0/3190914200 learned_addr learned my addr 172.21.15.148:0/3190914200 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:11.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba662b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7ba015c4f0 0x7f7ba0067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:44008/0 (socket says 172.21.15.148:44008) 2023-10-10T20:26:11.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 -- 172.21.15.148:0/3190914200 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 msgr2=0x7f7ba0070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:11.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 --2- 172.21.15.148:0/3190914200 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 -- 172.21.15.148:0/3190914200 >> v1:172.21.15.161:6789/0 conn(0x7f7ba015c4f0 legacy=0x7f7ba0067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5e2a700 1 -- 172.21.15.148:0/3190914200 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0068f20 con 0x7f7ba015b340 2023-10-10T20:26:11.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.337+0000 7f7ba5629700 1 --2- 172.21.15.148:0/3190914200 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:11.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba5e2a700 1 --2- 172.21.15.148:0/3190914200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099b0 tx=0x7f7b9c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd59077d4aa8f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba4e28700 1 -- 172.21.15.148:0/3190914200 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c009ea0 con 0x7f7ba015b340 2023-10-10T20:26:11.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba4e28700 1 -- 172.21.15.148:0/3190914200 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00b860 con 0x7f7ba015b340 2023-10-10T20:26:11.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba4e28700 1 -- 172.21.15.148:0/3190914200 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00bc00 con 0x7f7ba015b340 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 -- 172.21.15.148:0/3190914200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 msgr2=0x7f7ba0070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 --2- 172.21.15.148:0/3190914200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099b0 tx=0x7f7b9c009250 comp rx=0 tx=0).stop 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 -- 172.21.15.148:0/3190914200 shutdown_connections 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 --2- 172.21.15.148:0/3190914200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba015b340 0x7f7ba0070700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 --2- 172.21.15.148:0/3190914200 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00cb000 0x7f7ba0070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 -- 172.21.15.148:0/3190914200 >> 172.21.15.148:0/3190914200 conn(0x7f7ba00ca4f0 msgr2=0x7f7ba00ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:11.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 -- 172.21.15.148:0/3190914200 shutdown_connections 2023-10-10T20:26:11.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.338+0000 7f7ba808e700 1 -- 172.21.15.148:0/3190914200 wait complete. 2023-10-10T20:26:11.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.339+0000 7f7ba808e700 1 Processor -- start 2023-10-10T20:26:11.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.339+0000 7f7ba808e700 1 -- start start 2023-10-10T20:26:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00cb000 0x7f7ba00c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba015b340 0x7f7ba00b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c2df0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c30d0 con 0x7f7ba015b340 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba808e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c6d60 con 0x7f7ba00cb000 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba662b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba662b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37672/0 (socket says 172.21.15.148:37672) 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba662b700 1 -- 172.21.15.148:0/258649008 learned_addr learned my addr 172.21.15.148:0/258649008 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba5629700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba015b340 0x7f7ba00b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.340+0000 7f7ba5e2a700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00cb000 0x7f7ba00c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 -- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00cb000 msgr2=0x7f7ba00c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00cb000 0x7f7ba00c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 -- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba015b340 msgr2=0x7f7ba00b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba015b340 0x7f7ba00b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 -- 172.21.15.148:0/258649008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009420 con 0x7f7ba015c4f0 2023-10-10T20:26:11.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba662b700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800cca0 tx=0x7f7b98001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98005df0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800a5f0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0094f0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800a970 con 0x7f7ba015c4f0 2023-10-10T20:26:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.341+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba00c7320 con 0x7f7ba015c4f0 2023-10-10T20:26:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.342+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba004c3c0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.342+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b9800ad20 con 0x7f7ba015c4f0 2023-10-10T20:26:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.343+0000 7f7b95ffb700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.343+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7b980841a0 con 0x7f7ba015c4f0 2023-10-10T20:26:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.343+0000 7f7ba5e2a700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.343+0000 7f7ba5e2a700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099b0 tx=0x7f7b9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.347+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b98051760 con 0x7f7ba015c4f0 2023-10-10T20:26:11.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.533+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ba00c3210 con 0x7f7ba015c4f0 2023-10-10T20:26:11.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.534+0000 7f7b95ffb700 1 -- 172.21.15.148:0/258649008 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+12193 (secure 0 0 0) 0x7f7b98054d80 con 0x7f7ba015c4f0 2023-10-10T20:26:11.535 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:11.535 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":142,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:26:10.773499+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 msgr2=0x7f7b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0099b0 tx=0x7f7b9c017000 comp rx=0 tx=0).stop 2023-10-10T20:26:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 msgr2=0x7f7ba016d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800cca0 tx=0x7f7b98001a70 comp rx=0 tx=0).stop 2023-10-10T20:26:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 shutdown_connections 2023-10-10T20:26:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba015c4f0 0x7f7ba016d4a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00cb000 0x7f7ba00c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 --2- 172.21.15.148:0/258649008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba015b340 0x7f7ba00b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 >> 172.21.15.148:0/258649008 conn(0x7f7ba00ca4f0 msgr2=0x7f7ba006e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 shutdown_connections 2023-10-10T20:26:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.536+0000 7f7ba808e700 1 -- 172.21.15.148:0/258649008 wait complete. 2023-10-10T20:26:11.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:26:11.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:26:11.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.666+0000 7fa2310e9700 1 Processor -- start 2023-10-10T20:26:11.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.666+0000 7fa2310e9700 1 -- start start 2023-10-10T20:26:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa2310e9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa22c0cce30 0x7fa22c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa2310e9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa22c0b6580 0x7fa22c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa2310e9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22c06ac90 con 0x7fa22c06a830 2023-10-10T20:26:11.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa2310e9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c0b4d70 con 0x7fa22c0b6580 2023-10-10T20:26:11.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa2310e9700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c0cd8d0 con 0x7fa22c0cce30 2023-10-10T20:26:11.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa22c0b6580 0x7fa22c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa22c0b6580 0x7fa22c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37674/0 (socket says 172.21.15.148:37674) 2023-10-10T20:26:11.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 -- 172.21.15.148:0/1386946360 learned_addr learned my addr 172.21.15.148:0/1386946360 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:11.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22bfff700 1 --2- 172.21.15.148:0/1386946360 >> v2:172.21.15.161:3300/0 conn(0x7fa22c0cce30 0x7fa22c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 -- 172.21.15.148:0/1386946360 >> v2:172.21.15.161:3300/0 conn(0x7fa22c0cce30 msgr2=0x7fa22c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:11.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 --2- 172.21.15.148:0/1386946360 >> v2:172.21.15.161:3300/0 conn(0x7fa22c0cce30 0x7fa22c15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 -- 172.21.15.148:0/1386946360 >> v1:172.21.15.161:6789/0 conn(0x7fa22c06a830 legacy=0x7fa22c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.667+0000 7fa22b7fe700 1 -- 172.21.15.148:0/1386946360 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0b4440 con 0x7fa22c0b6580 2023-10-10T20:26:11.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa22b7fe700 1 --2- 172.21.15.148:0/1386946360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c0b6580 0x7fa22c06a2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b3770 tx=0x7fa220009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc252f896c6c2f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa22affd700 1 -- 172.21.15.148:0/1386946360 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220009e70 con 0x7fa22c0b6580 2023-10-10T20:26:11.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa22affd700 1 -- 172.21.15.148:0/1386946360 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22000b860 con 0x7fa22c0b6580 2023-10-10T20:26:11.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa22affd700 1 -- 172.21.15.148:0/1386946360 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000bc00 con 0x7fa22c0b6580 2023-10-10T20:26:11.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 -- 172.21.15.148:0/1386946360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c0b6580 msgr2=0x7fa22c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1386946360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c0b6580 0x7fa22c06a2f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b3770 tx=0x7fa220009250 comp rx=0 tx=0).stop 2023-10-10T20:26:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 -- 172.21.15.148:0/1386946360 shutdown_connections 2023-10-10T20:26:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1386946360 >> v2:172.21.15.161:3300/0 conn(0x7fa22c0cce30 0x7fa22c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1386946360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c0b6580 0x7fa22c06a2f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 -- 172.21.15.148:0/1386946360 >> 172.21.15.148:0/1386946360 conn(0x7fa22c068cb0 msgr2=0x7fa22c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 -- 172.21.15.148:0/1386946360 shutdown_connections 2023-10-10T20:26:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.668+0000 7fa2310e9700 1 -- 172.21.15.148:0/1386946360 wait complete. 2023-10-10T20:26:11.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.669+0000 7fa2310e9700 1 Processor -- start 2023-10-10T20:26:11.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 -- start start 2023-10-10T20:26:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0b6580 0x7fa22c152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 0x7fa22c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c156060 con 0x7fa22c06a830 2023-10-10T20:26:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c143c50 con 0x7fa22c0b6580 2023-10-10T20:26:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2310e9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c16d550 con 0x7fa22c0cce30 2023-10-10T20:26:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa22bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37676/0 (socket says 172.21.15.148:37676) 2023-10-10T20:26:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa22bfff700 1 -- 172.21.15.148:0/1366989389 learned_addr learned my addr 172.21.15.148:0/1366989389 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa22b7fe700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0b6580 0x7fa22c152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.670+0000 7fa2308e8700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 0x7fa22c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 -- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 msgr2=0x7fa22c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 0x7fa22c0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 -- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0b6580 msgr2=0x7fa22c152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0b6580 0x7fa22c152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 -- 172.21.15.148:0/1366989389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2200094f0 con 0x7fa22c06a830 2023-10-10T20:26:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa2308e8700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 0x7fa22c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa22bfff700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e930 tx=0x7fa21c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c013680 con 0x7fa22c06a830 2023-10-10T20:26:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c0137e0 con 0x7fa22c06a830 2023-10-10T20:26:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c1518a0 con 0x7fa22c06a830 2023-10-10T20:26:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c013b80 con 0x7fa22c06a830 2023-10-10T20:26:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.671+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c151e60 con 0x7fa22c06a830 2023-10-10T20:26:11.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.672+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c14ff90 con 0x7fa22c06a830 2023-10-10T20:26:11.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.673+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa21c00c460 con 0x7fa22c06a830 2023-10-10T20:26:11.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.673+0000 7fa213fff700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 0x7fa2180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.673+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa21c084b60 con 0x7fa22c06a830 2023-10-10T20:26:11.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.673+0000 7fa22b7fe700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 0x7fa2180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.674+0000 7fa22b7fe700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 0x7fa2180693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b3770 tx=0x7fa220017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:11.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.677+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa21c0521a0 con 0x7fa22c06a830 2023-10-10T20:26:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.863+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa22c0b4720 con 0x7fa22c06a830 2023-10-10T20:26:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.863+0000 7fa213fff700 1 -- 172.21.15.148:0/1366989389 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa21c0557c0 con 0x7fa22c06a830 2023-10-10T20:26:11.865 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:26:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 msgr2=0x7fa2180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 0x7fa2180693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b3770 tx=0x7fa220017000 comp rx=0 tx=0).stop 2023-10-10T20:26:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 msgr2=0x7fa22c143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e930 tx=0x7fa21c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 shutdown_connections 2023-10-10T20:26:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa218066f30 0x7fa2180693e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa22c0cce30 0x7fa22c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa22c06a830 0x7fa22c143710 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 --2- 172.21.15.148:0/1366989389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0b6580 0x7fa22c152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 >> 172.21.15.148:0/1366989389 conn(0x7fa22c068cb0 msgr2=0x7fa22c150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 shutdown_connections 2023-10-10T20:26:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.865+0000 7fa2310e9700 1 -- 172.21.15.148:0/1366989389 wait complete. 2023-10-10T20:26:11.879 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:26:11.993 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.992+0000 7fb8273f4700 1 Processor -- start 2023-10-10T20:26:11.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.992+0000 7fb8273f4700 1 -- start start 2023-10-10T20:26:11.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 0x7fb8200cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 0x7fb82015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb82015dda0 0x7fb82015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:11.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82015e850 con 0x7fb82015dda0 2023-10-10T20:26:11.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8200cd030 con 0x7fb8200cd6e0 2023-10-10T20:26:11.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8273f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200cb7c0 con 0x7fb8200cdab0 2023-10-10T20:26:11.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb82015dda0 0x7fb82015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:11.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb82015dda0 0x7fb82015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37692/0 (socket says 172.21.15.148:37692) 2023-10-10T20:26:11.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 -- 172.21.15.148:0/1641513737 learned_addr learned my addr 172.21.15.148:0/1641513737 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:11.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8263f2700 1 --2- 172.21.15.148:0/1641513737 >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 0x7fb8200cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 -- 172.21.15.148:0/1641513737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 msgr2=0x7fb82015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb825bf1700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 0x7fb82015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 0x7fb82015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 -- 172.21.15.148:0/1641513737 >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 msgr2=0x7fb8200cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 --2- 172.21.15.148:0/1641513737 >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 0x7fb8200cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb826bf3700 1 -- 172.21.15.148:0/1641513737 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82015e290 con 0x7fb82015dda0 2023-10-10T20:26:12.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb8263f2700 1 --2- 172.21.15.148:0/1641513737 >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 0x7fb8200cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:12.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.993+0000 7fb825bf1700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 0x7fb82015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:12.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb826bf3700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82015dda0 0x7fb82015cbd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb82006b730 tx=0x7fb818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6952916a3248faeb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8253f0700 1 -- 172.21.15.148:0/1641513737 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800b860 con 0x7fb82015dda0 2023-10-10T20:26:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8253f0700 1 -- 172.21.15.148:0/1641513737 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81800b9c0 con 0x7fb82015dda0 2023-10-10T20:26:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8253f0700 1 -- 172.21.15.148:0/1641513737 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800bd60 con 0x7fb82015dda0 2023-10-10T20:26:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 -- 172.21.15.148:0/1641513737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82015dda0 msgr2=0x7fb82015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82015dda0 0x7fb82015cbd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb82006b730 tx=0x7fb818009250 comp rx=0 tx=0).stop 2023-10-10T20:26:12.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 -- 172.21.15.148:0/1641513737 shutdown_connections 2023-10-10T20:26:12.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82015dda0 0x7fb82015cbd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1641513737 >> v2:172.21.15.161:3300/0 conn(0x7fb8200cd6e0 0x7fb8200cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1641513737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb8200cdab0 0x7fb82015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 -- 172.21.15.148:0/1641513737 >> 172.21.15.148:0/1641513737 conn(0x7fb820067660 msgr2=0x7fb820067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:12.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 -- 172.21.15.148:0/1641513737 shutdown_connections 2023-10-10T20:26:12.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.994+0000 7fb8273f4700 1 -- 172.21.15.148:0/1641513737 wait complete. 2023-10-10T20:26:12.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.995+0000 7fb8273f4700 1 Processor -- start 2023-10-10T20:26:12.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 -- start start 2023-10-10T20:26:12.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8200cdab0 0x7fb8200c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb82015dda0 0x7fb8201584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b6760 con 0x7fb8200cd6e0 2023-10-10T20:26:12.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b52f0 con 0x7fb8200cdab0 2023-10-10T20:26:12.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8273f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82015c300 con 0x7fb82015dda0 2023-10-10T20:26:12.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8263f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8263f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37704/0 (socket says 172.21.15.148:37704) 2023-10-10T20:26:12.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.996+0000 7fb8263f2700 1 -- 172.21.15.148:0/1239967401 learned_addr learned my addr 172.21.15.148:0/1239967401 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:12.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb825bf1700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8200cdab0 0x7fb8200c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb826bf3700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb82015dda0 0x7fb8201584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 -- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb82015dda0 msgr2=0x7fb8201584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb82015dda0 0x7fb8201584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 -- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8200cdab0 msgr2=0x7fb8200c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8200cdab0 0x7fb8200c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 -- 172.21.15.148:0/1239967401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180094f0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8263f2700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00cef0 tx=0x7fb81c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00ae20 con 0x7fb8200cd6e0 2023-10-10T20:26:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c016420 con 0x7fb8200cd6e0 2023-10-10T20:26:12.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0167e0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200b4ce0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.997+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8200b4f80 con 0x7fb8200cd6e0 2023-10-10T20:26:12.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.998+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb820089280 con 0x7fb8200cd6e0 2023-10-10T20:26:12.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.998+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb81c016bd0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.999+0000 7fb8167fc700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.999+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb81c012070 con 0x7fb8200cd6e0 2023-10-10T20:26:12.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.999+0000 7fb825bf1700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:11.999+0000 7fb825bf1700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 0x7fb8040693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8200d1350 tx=0x7fb810006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.003+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb81c04e5a0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.183+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb8200b55d0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.184+0000 7fb8167fc700 1 -- 172.21.15.148:0/1239967401 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb81c0520f0 con 0x7fb8200cd6e0 2023-10-10T20:26:12.185 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:12.186 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:26:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:26:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:26:12.186 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:26:12.187 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:26:12.187 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:26:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 msgr2=0x7fb8040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 0x7fb8040693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8200d1350 tx=0x7fb810006f90 comp rx=0 tx=0).stop 2023-10-10T20:26:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 msgr2=0x7fb8201553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00cef0 tx=0x7fb81c0077e0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 shutdown_connections 2023-10-10T20:26:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb82015dda0 0x7fb8201584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8200cd6e0 0x7fb8201553a0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 --2- 172.21.15.148:0/1239967401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8200cdab0 0x7fb8200c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 >> 172.21.15.148:0/1239967401 conn(0x7fb820067660 msgr2=0x7fb82015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:12.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 shutdown_connections 2023-10-10T20:26:12.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.186+0000 7fb8273f4700 1 -- 172.21.15.148:0/1239967401 wait complete. 2023-10-10T20:26:12.199 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:26:12.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:26:12.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.317+0000 7f84f9a11700 1 Processor -- start 2023-10-10T20:26:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.317+0000 7f84f9a11700 1 -- start start 2023-10-10T20:26:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f9a11700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84f4068cc0 0x7f84f4156390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f9a11700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f4068830 con 0x7f84f406a3d0 2023-10-10T20:26:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f9a11700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f4067c20 con 0x7f84f4067660 2023-10-10T20:26:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f9a11700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f4067f00 con 0x7f84f4068cc0 2023-10-10T20:26:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f84f406a3d0 0x7f84f406eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40158/0 (socket says 172.21.15.148:40158) 2023-10-10T20:26:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f2ffd700 1 -- 172.21.15.148:0/3863515317 learned_addr learned my addr 172.21.15.148:0/3863515317 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166970825 0 0) 0x7f84f4068830 con 0x7f84f406a3d0 2023-10-10T20:26:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84dc003570 con 0x7f84f406a3d0 2023-10-10T20:26:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f27fc700 1 --2- 172.21.15.148:0/3863515317 >> v2:172.21.15.161:3300/0 conn(0x7f84f4068cc0 0x7f84f4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965926379 0 0) 0x7f84dc003570 con 0x7f84f406a3d0 2023-10-10T20:26:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 >> v2:172.21.15.161:3300/0 conn(0x7f84f4068cc0 msgr2=0x7f84f4156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 --2- 172.21.15.148:0/3863515317 >> v2:172.21.15.161:3300/0 conn(0x7f84f4068cc0 0x7f84f4156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 >> v1:172.21.15.161:6789/0 conn(0x7f84f4067660 legacy=0x7f84f41568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f40681e0 con 0x7f84f406a3d0 2023-10-10T20:26:12.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f84e4001470 con 0x7f84f406a3d0 2023-10-10T20:26:12.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 >> v1:172.21.15.148:6789/0 conn(0x7f84f406a3d0 legacy=0x7f84f406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.318+0000 7f84f1ffb700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84dc003a60 0x7f84dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 0x7f84dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc0051c0 0x7f84dc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4067f00 con 0x7f84dc004810 2023-10-10T20:26:12.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc00c0d0 con 0x7f84dc003a60 2023-10-10T20:26:12.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc00c290 con 0x7f84dc0051c0 2023-10-10T20:26:12.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 0x7f84dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f2ffd700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84dc003a60 0x7f84dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f37fe700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc0051c0 0x7f84dc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 -- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc0051c0 msgr2=0x7f84dc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc0051c0 0x7f84dc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 -- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84dc003a60 msgr2=0x7f84dc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84dc003a60 0x7f84dc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f37fe700 1 -- 172.21.15.148:0/3863515317 reap_dead start 2023-10-10T20:26:12.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 -- 172.21.15.148:0/3863515317 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc003570 con 0x7f84dc004810 2023-10-10T20:26:12.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f27fc700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 0x7f84dc004c80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f84f4069120 tx=0x7f84e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea09ca6cab3748 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800e710 con 0x7f84dc004810 2023-10-10T20:26:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800e8b0 con 0x7f84dc004810 2023-10-10T20:26:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.319+0000 7f84f1ffb700 1 -- 172.21.15.148:0/3863515317 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800ec70 con 0x7f84dc004810 2023-10-10T20:26:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 -- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 msgr2=0x7f84dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 0x7f84dc004c80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f84f4069120 tx=0x7f84e80091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 -- 172.21.15.148:0/3863515317 shutdown_connections 2023-10-10T20:26:12.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 --2- 172.21.15.148:0/3863515317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84dc004810 0x7f84dc004c80 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 -- 172.21.15.148:0/3863515317 >> 172.21.15.148:0/3863515317 conn(0x7f84f40cd930 msgr2=0x7f84f40cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:12.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 -- 172.21.15.148:0/3863515317 shutdown_connections 2023-10-10T20:26:12.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.320+0000 7f84f9a11700 1 -- 172.21.15.148:0/3863515317 wait complete. 2023-10-10T20:26:12.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.321+0000 7f84f9a11700 1 Processor -- start 2023-10-10T20:26:12.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.321+0000 7f84f9a11700 1 -- start start 2023-10-10T20:26:12.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc004810 0x7f84f40c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 0x7f84f4151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f40c8ad0 con 0x7f84f41592e0 2023-10-10T20:26:12.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4157450 con 0x7f84f40b2aa0 2023-10-10T20:26:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f9a11700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4159c70 con 0x7f84dc004810 2023-10-10T20:26:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37722/0 (socket says 172.21.15.148:37722) 2023-10-10T20:26:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 -- 172.21.15.148:0/1755666207 learned_addr learned my addr 172.21.15.148:0/1755666207 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:12.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f2ffd700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc004810 0x7f84f40c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f27fc700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 0x7f84f4151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 -- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc004810 msgr2=0x7f84f40c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc004810 0x7f84f40c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 -- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 msgr2=0x7f84f4151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 0x7f84f4151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 -- 172.21.15.148:0/1755666207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f40b3210 con 0x7f84f41592e0 2023-10-10T20:26:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f27fc700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 0x7f84f4151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84f37fe700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f84f40cd500 tx=0x7f84ec001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.322+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec006e80 con 0x7f84f41592e0 2023-10-10T20:26:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.323+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00c540 con 0x7f84f41592e0 2023-10-10T20:26:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.323+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00c900 con 0x7f84f41592e0 2023-10-10T20:26:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.323+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f40b34f0 con 0x7f84f41592e0 2023-10-10T20:26:12.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.323+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f40b2f10 con 0x7f84f41592e0 2023-10-10T20:26:12.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.324+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f41575e0 con 0x7f84f41592e0 2023-10-10T20:26:12.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.324+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84ec00ccb0 con 0x7f84f41592e0 2023-10-10T20:26:12.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.324+0000 7f84daffd700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.324+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f84ec084020 con 0x7f84f41592e0 2023-10-10T20:26:12.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.324+0000 7f84f2ffd700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.325+0000 7f84f2ffd700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 0x7f84e00693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84f4066950 tx=0x7f84e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.329+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84ec0515e0 con 0x7f84f41592e0 2023-10-10T20:26:12.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.511+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f84f40cc5c0 con 0x7f84f41592e0 2023-10-10T20:26:12.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.795+0000 7f84daffd700 1 -- 172.21.15.148:0/1755666207 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v143) v1 ==== 138+0+0 (secure 0 0 0) 0x7f84ec054c00 con 0x7f84f41592e0 2023-10-10T20:26:12.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.798+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 msgr2=0x7f84e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.798+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 0x7f84e00693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84f4066950 tx=0x7f84e4009040 comp rx=0 tx=0).stop 2023-10-10T20:26:12.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.798+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 msgr2=0x7f84f4159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.798+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f84f40cd500 tx=0x7f84ec001640 comp rx=0 tx=0).stop 2023-10-10T20:26:12.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 shutdown_connections 2023-10-10T20:26:12.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f84e0066f30 0x7f84e00693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f41592e0 0x7f84f4159730 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84dc004810 0x7f84f40c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 --2- 172.21.15.148:0/1755666207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f40b2aa0 0x7f84f4151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 >> 172.21.15.148:0/1755666207 conn(0x7f84f40cd930 msgr2=0x7f84f40adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:12.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 shutdown_connections 2023-10-10T20:26:12.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.799+0000 7f84f9a11700 1 -- 172.21.15.148:0/1755666207 wait complete. 2023-10-10T20:26:12.802 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:26:12.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:26:12.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.935+0000 7fcf6eff8700 1 Processor -- start 2023-10-10T20:26:12.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.935+0000 7fcf6eff8700 1 -- start start 2023-10-10T20:26:12.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6eff8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf681542a0 0x7fcf680b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6eff8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf680cd630 0x7fcf6806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6eff8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6806ac90 con 0x7fcf6806a830 2023-10-10T20:26:12.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6eff8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6815b2e0 con 0x7fcf680cd630 2023-10-10T20:26:12.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6eff8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6806b0a0 con 0x7fcf681542a0 2023-10-10T20:26:12.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf680cd630 0x7fcf6806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6e7f7700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcf6806a830 0x7fcf6815d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57700/0 (socket says 172.21.15.148:57700) 2023-10-10T20:26:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6e7f7700 1 -- 172.21.15.148:0/3339242734 learned_addr learned my addr 172.21.15.148:0/3339242734 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6dff6700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf681542a0 0x7fcf680b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3339242734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf681542a0 msgr2=0x7fcf680b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf681542a0 0x7fcf680b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3339242734 >> v1:172.21.15.148:6789/0 conn(0x7fcf6806a830 legacy=0x7fcf6815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3339242734 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6815ad20 con 0x7fcf680cd630 2023-10-10T20:26:12.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6cff4700 1 -- 172.21.15.148:0/3339242734 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030035324 0 0) 0x7fcf60001490 con 0x7fcf6806a830 2023-10-10T20:26:12.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6d7f5700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf6806a2f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcf580098e0 tx=0x7fcf580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a53983a631153a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.936+0000 7fcf6cff4700 1 -- 172.21.15.148:0/3339242734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009e80 con 0x7fcf680cd630 2023-10-10T20:26:12.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6cff4700 1 -- 172.21.15.148:0/3339242734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800b860 con 0x7fcf680cd630 2023-10-10T20:26:12.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6cff4700 1 -- 172.21.15.148:0/3339242734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800bbe0 con 0x7fcf680cd630 2023-10-10T20:26:12.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3339242734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 msgr2=0x7fcf6806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf6806a2f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcf580098e0 tx=0x7fcf580091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3339242734 shutdown_connections 2023-10-10T20:26:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf681542a0 0x7fcf680b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3339242734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf6806a2f0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3339242734 >> 172.21.15.148:0/3339242734 conn(0x7fcf68068cb0 msgr2=0x7fcf680690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3339242734 shutdown_connections 2023-10-10T20:26:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.937+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3339242734 wait complete. 2023-10-10T20:26:12.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.938+0000 7fcf6eff8700 1 Processor -- start 2023-10-10T20:26:12.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.938+0000 7fcf6eff8700 1 -- start start 2023-10-10T20:26:12.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6806a830 0x7fcf68152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf681542a0 0x7fcf680cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68153210 con 0x7fcf680cd630 2023-10-10T20:26:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c82c0 con 0x7fcf681542a0 2023-10-10T20:26:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6eff8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c5d40 con 0x7fcf6806a830 2023-10-10T20:26:12.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46218/0 (socket says 172.21.15.148:46218) 2023-10-10T20:26:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3531918412 learned_addr learned my addr 172.21.15.148:0/3531918412 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6dff6700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6806a830 0x7fcf68152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6e7f7700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf681542a0 0x7fcf680cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6806a830 msgr2=0x7fcf68152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6806a830 0x7fcf68152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf681542a0 msgr2=0x7fcf680cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf681542a0 0x7fcf680cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.939+0000 7fcf6d7f5700 1 -- 172.21.15.148:0/3531918412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009420 con 0x7fcf680cd630 2023-10-10T20:26:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf6d7f5700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806b3a0 tx=0x7fcf58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800bbe0 con 0x7fcf680cd630 2023-10-10T20:26:12.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58011420 con 0x7fcf680cd630 2023-10-10T20:26:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680c6020 con 0x7fcf680cd630 2023-10-10T20:26:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf580117e0 con 0x7fcf680cd630 2023-10-10T20:26:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.940+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf680c65e0 con 0x7fcf680cd630 2023-10-10T20:26:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.941+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf68157220 con 0x7fcf680cd630 2023-10-10T20:26:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.941+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf58011bd0 con 0x7fcf680cd630 2023-10-10T20:26:12.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.942+0000 7fcf5dffb700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:12.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.942+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fcf58085d80 con 0x7fcf680cd630 2023-10-10T20:26:12.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.942+0000 7fcf6dff6700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:12.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.942+0000 7fcf6dff6700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006480 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:12.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:12.946+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf68157220 con 0x7fcf680cd630 2023-10-10T20:26:13.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.130+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcf68157220 con 0x7fcf680cd630 2023-10-10T20:26:13.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.804+0000 7fcf5dffb700 1 -- 172.21.15.148:0/3531918412 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v144) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcf68157220 con 0x7fcf680cd630 2023-10-10T20:26:13.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 msgr2=0x7fcf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:13.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006480 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2023-10-10T20:26:13.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 msgr2=0x7fcf681598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:13.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcf6806b3a0 tx=0x7fcf58017000 comp rx=0 tx=0).stop 2023-10-10T20:26:13.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 shutdown_connections 2023-10-10T20:26:13.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf681542a0 0x7fcf680cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6806a830 0x7fcf68152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 --2- 172.21.15.148:0/3531918412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf680cd630 0x7fcf681598a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 >> 172.21.15.148:0/3531918412 conn(0x7fcf68068cb0 msgr2=0x7fcf680b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 shutdown_connections 2023-10-10T20:26:13.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.814+0000 7fcf6eff8700 1 -- 172.21.15.148:0/3531918412 wait complete. 2023-10-10T20:26:13.818 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:26:13.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:26:13.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.946+0000 7fe2d0c29700 1 Processor -- start 2023-10-10T20:26:13.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.946+0000 7fe2d0c29700 1 -- start start 2023-10-10T20:26:13.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2d0c29700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2d0c29700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc06d370 0x7fe2cc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2d0c29700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc15c8d0 con 0x7fe2cc152300 2023-10-10T20:26:13.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2d0c29700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b1400 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2d0c29700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc15be20 con 0x7fe2cc06d370 2023-10-10T20:26:13.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46234/0 (socket says 172.21.15.148:46234) 2023-10-10T20:26:13.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 -- 172.21.15.148:0/552035558 learned_addr learned my addr 172.21.15.148:0/552035558 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:13.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2cad9d700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc06d370 0x7fe2cc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 -- 172.21.15.148:0/552035558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc06d370 msgr2=0x7fe2cc06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:13.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc06d370 0x7fe2cc06d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 -- 172.21.15.148:0/552035558 >> v1:172.21.15.161:6789/0 conn(0x7fe2cc152300 legacy=0x7fe2cc158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:13.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 -- 172.21.15.148:0/552035558 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc0c9370 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00099b0 tx=0x7fe2c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847f1717558fc519 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:13.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c959a700 1 -- 172.21.15.148:0/552035558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000b820 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c959a700 1 -- 172.21.15.148:0/552035558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c000b980 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.947+0000 7fe2c959a700 1 -- 172.21.15.148:0/552035558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000bd20 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 -- 172.21.15.148:0/552035558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc15b9a0 msgr2=0x7fe2cc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00099b0 tx=0x7fe2c0009250 comp rx=0 tx=0).stop 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 -- 172.21.15.148:0/552035558 shutdown_connections 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc06d370 0x7fe2cc06d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/552035558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc0caaf0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 -- 172.21.15.148:0/552035558 >> 172.21.15.148:0/552035558 conn(0x7fe2cc069870 msgr2=0x7fe2cc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:13.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 -- 172.21.15.148:0/552035558 shutdown_connections 2023-10-10T20:26:13.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.948+0000 7fe2d0c29700 1 -- 172.21.15.148:0/552035558 wait complete. 2023-10-10T20:26:13.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.949+0000 7fe2d0c29700 1 Processor -- start 2023-10-10T20:26:13.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.949+0000 7fe2d0c29700 1 -- start start 2023-10-10T20:26:13.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 0x7fe2cc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc153ac0 con 0x7fe2cc06d370 2023-10-10T20:26:13.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b2080 con 0x7fe2cc15b9a0 2023-10-10T20:26:13.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc153d10 con 0x7fe2cc152300 2023-10-10T20:26:13.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46244/0 (socket says 172.21.15.148:46244) 2023-10-10T20:26:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 -- 172.21.15.148:0/2131037989 learned_addr learned my addr 172.21.15.148:0/2131037989 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 0x7fe2cc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2cad9d700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 -- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 msgr2=0x7fe2cc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 0x7fe2cc154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 -- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2cc15b9a0 msgr2=0x7fe2cc154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 -- 172.21.15.148:0/2131037989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00094f0 con 0x7fe2cc06d370 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 0x7fe2cc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:13.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ca59c700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00c950 tx=0x7fe2bc002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:13.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00a7e0 con 0x7fe2cc06d370 2023-10-10T20:26:13.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2bc00a940 con 0x7fe2cc06d370 2023-10-10T20:26:13.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00ad00 con 0x7fe2cc06d370 2023-10-10T20:26:13.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc0b1a30 con 0x7fe2cc06d370 2023-10-10T20:26:13.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.950+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc152ba0 con 0x7fe2cc06d370 2023-10-10T20:26:13.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.951+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc153ed0 con 0x7fe2cc06d370 2023-10-10T20:26:13.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.952+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2bc012630 con 0x7fe2cc06d370 2023-10-10T20:26:13.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.952+0000 7fe2ba7fc700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:13.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.952+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fe2bc084a60 con 0x7fe2cc06d370 2023-10-10T20:26:13.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.952+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:13.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.952+0000 7fe2c9d9b700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00099b0 tx=0x7fe2c0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:13.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:13.957+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2cc153ed0 con 0x7fe2cc06d370 2023-10-10T20:26:14.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.179+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe2cc153ed0 con 0x7fe2cc06d370 2023-10-10T20:26:14.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.802+0000 7fe2ba7fc700 1 -- 172.21.15.148:0/2131037989 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe2cc153ed0 con 0x7fe2cc06d370 2023-10-10T20:26:14.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.808+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 msgr2=0x7fe2ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.808+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00099b0 tx=0x7fe2c0016000 comp rx=0 tx=0).stop 2023-10-10T20:26:14.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.808+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 msgr2=0x7fe2cc144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.808+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00c950 tx=0x7fe2bc002760 comp rx=0 tx=0).stop 2023-10-10T20:26:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 shutdown_connections 2023-10-10T20:26:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe2ac066f30 0x7fe2ac0693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2cc15b9a0 0x7fe2cc154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc06d370 0x7fe2cc144820 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 --2- 172.21.15.148:0/2131037989 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe2cc152300 0x7fe2cc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 >> 172.21.15.148:0/2131037989 conn(0x7fe2cc069870 msgr2=0x7fe2cc0cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:14.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 shutdown_connections 2023-10-10T20:26:14.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.809+0000 7fe2d0c29700 1 -- 172.21.15.148:0/2131037989 wait complete. 2023-10-10T20:26:14.814 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 18 and data pool 19 2023-10-10T20:26:14.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:14.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.945+0000 7f35f13a0700 1 Processor -- start 2023-10-10T20:26:14.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.945+0000 7f35f13a0700 1 -- start start 2023-10-10T20:26:14.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35f13a0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f35ec157c40 0x7f35ec158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:14.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35f13a0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec15cbb0 con 0x7f35ec157870 2023-10-10T20:26:14.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35f13a0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec15bfa0 con 0x7f35ec15a620 2023-10-10T20:26:14.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35f13a0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec15bdb0 con 0x7f35ec157c40 2023-10-10T20:26:14.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35eaffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f35ec157870 0x7f35ec15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57702/0 (socket says 172.21.15.148:57702) 2023-10-10T20:26:14.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35eaffd700 1 -- 172.21.15.148:0/2168607508 learned_addr learned my addr 172.21.15.148:0/2168607508 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:14.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.946+0000 7f35eb7fe700 1 --2- 172.21.15.148:0/2168607508 >> v2:172.21.15.161:3300/0 conn(0x7f35ec157c40 0x7f35ec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:14.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35ea7fc700 1 -- 172.21.15.148:0/2168607508 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918536138 0 0) 0x7f35ec15cbb0 con 0x7f35ec157870 2023-10-10T20:26:14.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35ea7fc700 1 -- 172.21.15.148:0/2168607508 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d8003570 con 0x7f35ec157870 2023-10-10T20:26:14.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35eb7fe700 1 -- 172.21.15.148:0/2168607508 >> v1:172.21.15.161:6789/0 conn(0x7f35ec15a620 legacy=0x7f35ec15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35eb7fe700 1 -- 172.21.15.148:0/2168607508 >> v1:172.21.15.148:6789/0 conn(0x7f35ec157870 legacy=0x7f35ec15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35eb7fe700 1 -- 172.21.15.148:0/2168607508 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec15c280 con 0x7f35ec157c40 2023-10-10T20:26:14.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35eb7fe700 1 --2- 172.21.15.148:0/2168607508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157c40 0x7f35ec158110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009e60 tx=0x7f35e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74c10a3e883849d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:14.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35ea7fc700 1 -- 172.21.15.148:0/2168607508 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400e070 con 0x7f35ec157c40 2023-10-10T20:26:14.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35ea7fc700 1 -- 172.21.15.148:0/2168607508 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e400bb60 con 0x7f35ec157c40 2023-10-10T20:26:14.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.947+0000 7f35ea7fc700 1 -- 172.21.15.148:0/2168607508 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4012430 con 0x7f35ec157c40 2023-10-10T20:26:14.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 -- 172.21.15.148:0/2168607508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157c40 msgr2=0x7f35ec158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 --2- 172.21.15.148:0/2168607508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157c40 0x7f35ec158110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009e60 tx=0x7f35e4009780 comp rx=0 tx=0).stop 2023-10-10T20:26:14.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 -- 172.21.15.148:0/2168607508 shutdown_connections 2023-10-10T20:26:14.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 --2- 172.21.15.148:0/2168607508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157c40 0x7f35ec158110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 -- 172.21.15.148:0/2168607508 >> 172.21.15.148:0/2168607508 conn(0x7f35ec068ce0 msgr2=0x7f35ec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:14.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 -- 172.21.15.148:0/2168607508 shutdown_connections 2023-10-10T20:26:14.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.948+0000 7f35f13a0700 1 -- 172.21.15.148:0/2168607508 wait complete. 2023-10-10T20:26:14.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.949+0000 7f35f13a0700 1 Processor -- start 2023-10-10T20:26:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.949+0000 7f35f13a0700 1 -- start start 2023-10-10T20:26:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 0x7f35ec16b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 0x7f35ec0a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0aa9e0 con 0x7f35ec157c40 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec16b5d0 con 0x7f35ec157870 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec156410 con 0x7f35ec15a620 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46252/0 (socket says 172.21.15.148:46252) 2023-10-10T20:26:14.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 -- 172.21.15.148:0/613530609 learned_addr learned my addr 172.21.15.148:0/613530609 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:14.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35eaffd700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 0x7f35ec16b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:14.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35eb7fe700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 0x7f35ec0a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:14.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 -- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 msgr2=0x7f35ec0a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 0x7f35ec0a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 -- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 msgr2=0x7f35ec16b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 0x7f35ec16b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:14.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 -- 172.21.15.148:0/613530609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009a20 con 0x7f35ec157c40 2023-10-10T20:26:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35eb7fe700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 0x7f35ec0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35eaffd700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 0x7f35ec16b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e27fc700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c4e0 tx=0x7f35d4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011590 con 0x7f35ec157c40 2023-10-10T20:26:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d40116f0 con 0x7f35ec157c40 2023-10-10T20:26:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec0a7b60 con 0x7f35ec157c40 2023-10-10T20:26:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011ab0 con 0x7f35ec157c40 2023-10-10T20:26:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.950+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec0a8400 con 0x7f35ec157c40 2023-10-10T20:26:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.951+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec04c3c0 con 0x7f35ec157c40 2023-10-10T20:26:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.952+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35d401b3b0 con 0x7f35ec157c40 2023-10-10T20:26:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.952+0000 7f35e37fe700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.952+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35d40839d0 con 0x7f35ec157c40 2023-10-10T20:26:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.952+0000 7f35eaffd700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.953+0000 7f35eaffd700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 0x7f35cc0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f35ec160610 tx=0x7f35dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:14.957+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35d4050c40 con 0x7f35ec157c40 2023-10-10T20:26:15.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.141+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35ec15c560 con 0x7f35ec157c40 2023-10-10T20:26:15.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.142+0000 7f35e37fe700 1 -- 172.21.15.148:0/613530609 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f35d4017070 con 0x7f35ec157c40 2023-10-10T20:26:15.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:15.144 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":145,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:26:14.180469+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:15.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 msgr2=0x7f35cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 0x7f35cc0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f35ec160610 tx=0x7f35dc009040 comp rx=0 tx=0).stop 2023-10-10T20:26:15.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 msgr2=0x7f35ec145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c4e0 tx=0x7f35d4007f00 comp rx=0 tx=0).stop 2023-10-10T20:26:15.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 shutdown_connections 2023-10-10T20:26:15.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f35cc066f30 0x7f35cc0693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35ec15a620 0x7f35ec0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ec157870 0x7f35ec16b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 --2- 172.21.15.148:0/613530609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec157c40 0x7f35ec145750 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 >> 172.21.15.148:0/613530609 conn(0x7f35ec068ce0 msgr2=0x7f35ec0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:15.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 shutdown_connections 2023-10-10T20:26:15.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.144+0000 7f35f13a0700 1 -- 172.21.15.148:0/613530609 wait complete. 2023-10-10T20:26:15.157 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:15.271 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.270+0000 7f50d2d1d700 1 Processor -- start 2023-10-10T20:26:15.272 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.270+0000 7f50d2d1d700 1 -- start start 2023-10-10T20:26:15.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d2d1d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50cc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d2d1d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f50cc15a620 0x7f50cc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d2d1d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50cc15cbb0 con 0x7f50cc15d580 2023-10-10T20:26:15.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d2d1d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50cc15bfa0 con 0x7f50cc15a620 2023-10-10T20:26:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d2d1d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc15c280 con 0x7f50cc157870 2023-10-10T20:26:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d12ba700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50cc15d580 0x7f50cc158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57708/0 (socket says 172.21.15.148:57708) 2023-10-10T20:26:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d12ba700 1 -- 172.21.15.148:0/4285284115 learned_addr learned my addr 172.21.15.148:0/4285284115 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cbfff700 1 --2- 172.21.15.148:0/4285284115 >> v2:172.21.15.161:3300/0 conn(0x7f50cc15a620 0x7f50cc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50cc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430699519 0 0) 0x7f50cc15cbb0 con 0x7f50cc15d580 2023-10-10T20:26:15.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc003570 con 0x7f50cc15d580 2023-10-10T20:26:15.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255852169 0 0) 0x7f50bc003570 con 0x7f50cc15d580 2023-10-10T20:26:15.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 msgr2=0x7f50cc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50cc15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 >> v2:172.21.15.161:3300/0 conn(0x7f50cc15a620 msgr2=0x7f50cc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 --2- 172.21.15.148:0/4285284115 >> v2:172.21.15.161:3300/0 conn(0x7f50cc15a620 0x7f50cc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc15c560 con 0x7f50cc15d580 2023-10-10T20:26:15.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f50c40017f0 con 0x7f50cc15d580 2023-10-10T20:26:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.271+0000 7f50cbfff700 1 --2- 172.21.15.148:0/4285284115 >> v2:172.21.15.161:3300/0 conn(0x7f50cc15a620 0x7f50cc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 >> v1:172.21.15.148:6789/0 conn(0x7f50cc15d580 legacy=0x7f50cc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc004d50 0x7f50bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc15bfa0 con 0x7f50bc003a60 2023-10-10T20:26:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc15c280 con 0x7f50bc004d50 2023-10-10T20:26:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc00bae0 con 0x7f50cc157870 2023-10-10T20:26:15.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cbfff700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d12ba700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc004d50 0x7f50bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 -- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 msgr2=0x7f50bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 -- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc004d50 msgr2=0x7f50bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50bc004d50 0x7f50bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 -- 172.21.15.148:0/4285284115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc003570 con 0x7f50bc003a60 2023-10-10T20:26:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cbfff700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc157870 0x7f50bc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50bc003e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009c00 tx=0x7f50c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bfec4d7768685c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000e710 con 0x7f50bc003a60 2023-10-10T20:26:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c000e870 con 0x7f50bc003a60 2023-10-10T20:26:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.272+0000 7f50cb7fe700 1 -- 172.21.15.148:0/4285284115 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000ec30 con 0x7f50bc003a60 2023-10-10T20:26:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 -- 172.21.15.148:0/4285284115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 msgr2=0x7f50bc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/4285284115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50bc003e70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009c00 tx=0x7f50c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d12ba700 1 -- 172.21.15.148:0/4285284115 reap_dead start 2023-10-10T20:26:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 -- 172.21.15.148:0/4285284115 shutdown_connections 2023-10-10T20:26:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 -- 172.21.15.148:0/4285284115 >> 172.21.15.148:0/4285284115 conn(0x7f50cc068ce0 msgr2=0x7f50cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 -- 172.21.15.148:0/4285284115 shutdown_connections 2023-10-10T20:26:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.273+0000 7f50d2d1d700 1 -- 172.21.15.148:0/4285284115 wait complete. 2023-10-10T20:26:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.274+0000 7f50d2d1d700 1 Processor -- start 2023-10-10T20:26:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.274+0000 7f50d2d1d700 1 -- start start 2023-10-10T20:26:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc0b2190 0x7f50cc0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc16a3e0 0x7f50cc16a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc0b2b40 con 0x7f50bc003a60 2023-10-10T20:26:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc0c7580 con 0x7f50cc0b2190 2023-10-10T20:26:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d2d1d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc16bd50 con 0x7f50cc16a3e0 2023-10-10T20:26:15.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46262/0 (socket says 172.21.15.148:46262) 2023-10-10T20:26:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 -- 172.21.15.148:0/2885645765 learned_addr learned my addr 172.21.15.148:0/2885645765 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 -- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc16a3e0 msgr2=0x7f50cc16a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc16a3e0 0x7f50cc16a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 -- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc0b2190 msgr2=0x7f50cc0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc0b2190 0x7f50cc0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 -- 172.21.15.148:0/2885645765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc16b140 con 0x7f50bc003a60 2023-10-10T20:26:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50d0ab9700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f50c0006eb0 tx=0x7f50c001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.275+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0022070 con 0x7f50bc003a60 2023-10-10T20:26:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.276+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c001c450 con 0x7f50bc003a60 2023-10-10T20:26:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.276+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc16b420 con 0x7f50bc003a60 2023-10-10T20:26:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.276+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c001c810 con 0x7f50bc003a60 2023-10-10T20:26:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.276+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc16b9e0 con 0x7f50bc003a60 2023-10-10T20:26:15.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.276+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50cc04c3c0 con 0x7f50bc003a60 2023-10-10T20:26:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.277+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50c001cc00 con 0x7f50bc003a60 2023-10-10T20:26:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.277+0000 7f50c8ff9700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.277+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50c00860c0 con 0x7f50bc003a60 2023-10-10T20:26:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.277+0000 7f50cbfff700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.278+0000 7f50cbfff700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006dc0 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.282+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50c0053330 con 0x7f50bc003a60 2023-10-10T20:26:15.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.464+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50cc15c840 con 0x7f50bc003a60 2023-10-10T20:26:15.466 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.465+0000 7f50c8ff9700 1 -- 172.21.15.148:0/2885645765 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f50c0056950 con 0x7f50bc003a60 2023-10-10T20:26:15.467 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:15.467 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":145,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:26:14.180469+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:15.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 msgr2=0x7f50b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 0x7f50b00693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006dc0 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2023-10-10T20:26:15.469 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 msgr2=0x7f50cc155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f50c0006eb0 tx=0x7f50c001a000 comp rx=0 tx=0).stop 2023-10-10T20:26:15.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 shutdown_connections 2023-10-10T20:26:15.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50b0066f30 0x7f50b00693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50cc16a3e0 0x7f50cc16a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50bc003a60 0x7f50cc155480 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 --2- 172.21.15.148:0/2885645765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc0b2190 0x7f50cc0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.467+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 >> 172.21.15.148:0/2885645765 conn(0x7f50cc068ce0 msgr2=0x7f50cc159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:15.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.468+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 shutdown_connections 2023-10-10T20:26:15.471 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.468+0000 7f50d2d1d700 1 -- 172.21.15.148:0/2885645765 wait complete. 2023-10-10T20:26:15.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:26:15.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.598+0000 7fde221da700 1 Processor -- start 2023-10-10T20:26:15.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.598+0000 7fde221da700 1 -- start start 2023-10-10T20:26:15.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde221da700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde1c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde221da700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde1c06e7d0 0x7fde1c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde221da700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c06b800 con 0x7fde1c06e400 2023-10-10T20:26:15.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde221da700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c15acd0 con 0x7fde1c06e7d0 2023-10-10T20:26:15.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde221da700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c157eb0 con 0x7fde1c0ca170 2023-10-10T20:26:15.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde211d8700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fde1c06e400 0x7fde1c0c9a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57724/0 (socket says 172.21.15.148:57724) 2023-10-10T20:26:15.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde211d8700 1 -- 172.21.15.148:0/110299614 learned_addr learned my addr 172.21.15.148:0/110299614 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:15.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168899878 0 0) 0x7fde1c06b800 con 0x7fde1c06e400 2023-10-10T20:26:15.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde08003570 con 0x7fde1c06e400 2023-10-10T20:26:15.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.599+0000 7fde209d7700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde1c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde219d9700 1 --2- 172.21.15.148:0/110299614 >> v2:172.21.15.161:3300/0 conn(0x7fde1c06e7d0 0x7fde1c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011424708 0 0) 0x7fde08003570 con 0x7fde1c06e400 2023-10-10T20:26:15.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 msgr2=0x7fde1c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde1c156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 >> v2:172.21.15.161:3300/0 conn(0x7fde1c06e7d0 msgr2=0x7fde1c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 --2- 172.21.15.148:0/110299614 >> v2:172.21.15.161:3300/0 conn(0x7fde1c06e7d0 0x7fde1c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c157bd0 con 0x7fde1c06e400 2023-10-10T20:26:15.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fde18001470 con 0x7fde1c06e400 2023-10-10T20:26:15.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 >> v1:172.21.15.148:6789/0 conn(0x7fde1c06e400 legacy=0x7fde1c0c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde08003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde08004d50 0x7fde080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c15acd0 con 0x7fde080040b0 2023-10-10T20:26:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c157eb0 con 0x7fde08004d50 2023-10-10T20:26:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0800dd70 con 0x7fde1c0ca170 2023-10-10T20:26:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde211d8700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde08003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde219d9700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde08004d50 0x7fde080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 -- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 msgr2=0x7fde08003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0ca170 0x7fde08003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 -- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde08004d50 msgr2=0x7fde080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:15.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde08004d50 0x7fde080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 -- 172.21.15.148:0/110299614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08003570 con 0x7fde080040b0 2023-10-10T20:26:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.600+0000 7fde209d7700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde08004810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fde0c006f00 tx=0x7fde0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61735ca033c05274 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00e710 con 0x7fde080040b0 2023-10-10T20:26:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c00e870 con 0x7fde080040b0 2023-10-10T20:26:15.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde13fff700 1 -- 172.21.15.148:0/110299614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00ec30 con 0x7fde080040b0 2023-10-10T20:26:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 -- 172.21.15.148:0/110299614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 msgr2=0x7fde08004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 --2- 172.21.15.148:0/110299614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde08004810 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fde0c006f00 tx=0x7fde0c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde219d9700 1 -- 172.21.15.148:0/110299614 reap_dead start 2023-10-10T20:26:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 -- 172.21.15.148:0/110299614 shutdown_connections 2023-10-10T20:26:15.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 -- 172.21.15.148:0/110299614 >> 172.21.15.148:0/110299614 conn(0x7fde1c067660 msgr2=0x7fde1c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 -- 172.21.15.148:0/110299614 shutdown_connections 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.601+0000 7fde221da700 1 -- 172.21.15.148:0/110299614 wait complete. 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.602+0000 7fde221da700 1 Processor -- start 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.602+0000 7fde221da700 1 -- start start 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 0x7fde1c0c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde1c0cbc90 0x7fde1c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0a9dc0 con 0x7fde080040b0 2023-10-10T20:26:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1555e0 con 0x7fde1c0cbc90 2023-10-10T20:26:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde221da700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0cc650 con 0x7fde1c0c5880 2023-10-10T20:26:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde211d8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde211d8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46280/0 (socket says 172.21.15.148:46280) 2023-10-10T20:26:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde211d8700 1 -- 172.21.15.148:0/3904994092 learned_addr learned my addr 172.21.15.148:0/3904994092 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde209d7700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 0x7fde1c0c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.603+0000 7fde219d9700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde1c0cbc90 0x7fde1c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 -- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 msgr2=0x7fde1c0c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 0x7fde1c0c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 -- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde1c0cbc90 msgr2=0x7fde1c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:15.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde1c0cbc90 0x7fde1c0cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:15.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 -- 172.21.15.148:0/3904994092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0c62d0 con 0x7fde080040b0 2023-10-10T20:26:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde209d7700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 0x7fde1c0c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde211d8700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fde18004b60 tx=0x7fde18001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18017050 con 0x7fde080040b0 2023-10-10T20:26:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18018040 con 0x7fde080040b0 2023-10-10T20:26:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0afe10 con 0x7fde080040b0 2023-10-10T20:26:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1801d460 con 0x7fde080040b0 2023-10-10T20:26:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.604+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0b03d0 con 0x7fde080040b0 2023-10-10T20:26:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.605+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c0ccba0 con 0x7fde080040b0 2023-10-10T20:26:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.605+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde1801c3c0 con 0x7fde080040b0 2023-10-10T20:26:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.606+0000 7fde117fa700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 0x7fde000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.606+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde1808c3c0 con 0x7fde080040b0 2023-10-10T20:26:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.606+0000 7fde209d7700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 0x7fde000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.606+0000 7fde209d7700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 0x7fde000693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0091e0 tx=0x7fde0c01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:15.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.610+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde180565a0 con 0x7fde080040b0 2023-10-10T20:26:15.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:15.844+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde1c067da0 con 0x7fde080040b0 2023-10-10T20:26:16.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.825+0000 7fde117fa700 1 -- 172.21.15.148:0/3904994092 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde18022030 con 0x7fde080040b0 2023-10-10T20:26:16.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.827+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 msgr2=0x7fde000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.827+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 0x7fde000693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0091e0 tx=0x7fde0c01a000 comp rx=0 tx=0).stop 2023-10-10T20:26:16.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.827+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 msgr2=0x7fde1c1550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.827+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fde18004b60 tx=0x7fde18001e50 comp rx=0 tx=0).stop 2023-10-10T20:26:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 shutdown_connections 2023-10-10T20:26:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fde00066f30 0x7fde000693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde1c0cbc90 0x7fde1c0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080040b0 0x7fde1c1550a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 --2- 172.21.15.148:0/3904994092 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde1c0c5880 0x7fde1c0c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 >> 172.21.15.148:0/3904994092 conn(0x7fde1c067660 msgr2=0x7fde1c0c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 shutdown_connections 2023-10-10T20:26:16.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.828+0000 7fde221da700 1 -- 172.21.15.148:0/3904994092 wait complete. 2023-10-10T20:26:16.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:26:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.965+0000 7f8ee60a9700 1 Processor -- start 2023-10-10T20:26:16.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.965+0000 7f8ee60a9700 1 -- start start 2023-10-10T20:26:16.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8ee60a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0152300 0x7f8ee015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8ee60a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ee0158df0 0x7f8ee00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8ee60a9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee015c8d0 con 0x7f8ee006d370 2023-10-10T20:26:16.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8ee60a9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee00b1400 con 0x7f8ee0158df0 2023-10-10T20:26:16.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8ee60a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00c9370 con 0x7f8ee0152300 2023-10-10T20:26:16.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8edeffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ee0158df0 0x7f8ee00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:16.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8edffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ee006d370 0x7f8ee006d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57728/0 (socket says 172.21.15.148:57728) 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8edffff700 1 -- 172.21.15.148:0/3071185971 learned_addr learned my addr 172.21.15.148:0/3071185971 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.966+0000 7f8edf7fe700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0152300 0x7f8ee015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8edeffd700 1 -- 172.21.15.148:0/3071185971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0152300 msgr2=0x7f8ee015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8edeffd700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0152300 0x7f8ee015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8edeffd700 1 -- 172.21.15.148:0/3071185971 >> v1:172.21.15.148:6789/0 conn(0x7f8ee006d370 legacy=0x7f8ee006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8edeffd700 1 -- 172.21.15.148:0/3071185971 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee015c030 con 0x7f8ee0158df0 2023-10-10T20:26:16.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8ede7fc700 1 -- 172.21.15.148:0/3071185971 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217535324 0 0) 0x7f8ee015c8d0 con 0x7f8ee006d370 2023-10-10T20:26:16.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8edeffd700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee0158df0 0x7f8ee00caaf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40098e0 tx=0x7f8ed40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beacf3af9eb451a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:16.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8ede7fc700 1 -- 172.21.15.148:0/3071185971 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400b640 con 0x7f8ee0158df0 2023-10-10T20:26:16.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8ede7fc700 1 -- 172.21.15.148:0/3071185971 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400b7a0 con 0x7f8ee0158df0 2023-10-10T20:26:16.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.967+0000 7f8ede7fc700 1 -- 172.21.15.148:0/3071185971 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400bb40 con 0x7f8ee0158df0 2023-10-10T20:26:16.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 -- 172.21.15.148:0/3071185971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee0158df0 msgr2=0x7f8ee00caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee0158df0 0x7f8ee00caaf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40098e0 tx=0x7f8ed40091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 -- 172.21.15.148:0/3071185971 shutdown_connections 2023-10-10T20:26:16.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0152300 0x7f8ee015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/3071185971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee0158df0 0x7f8ee00caaf0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 -- 172.21.15.148:0/3071185971 >> 172.21.15.148:0/3071185971 conn(0x7f8ee0069870 msgr2=0x7f8ee0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 -- 172.21.15.148:0/3071185971 shutdown_connections 2023-10-10T20:26:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.968+0000 7f8ee60a9700 1 -- 172.21.15.148:0/3071185971 wait complete. 2023-10-10T20:26:16.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.969+0000 7f8ee60a9700 1 Processor -- start 2023-10-10T20:26:16.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.969+0000 7f8ee60a9700 1 -- start start 2023-10-10T20:26:16.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee0152300 0x7f8ee01477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0158df0 0x7f8ee00c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0147d20 con 0x7f8ee006d370 2023-10-10T20:26:16.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee01581a0 con 0x7f8ee0152300 2023-10-10T20:26:16.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0157590 con 0x7f8ee0158df0 2023-10-10T20:26:16.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:16.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46306/0 (socket says 172.21.15.148:46306) 2023-10-10T20:26:16.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 -- 172.21.15.148:0/2146832110 learned_addr learned my addr 172.21.15.148:0/2146832110 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:16.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 -- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0158df0 msgr2=0x7f8ee00c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0158df0 0x7f8ee00c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 -- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee0152300 msgr2=0x7f8ee01477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:16.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee0152300 0x7f8ee01477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:16.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 -- 172.21.15.148:0/2146832110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4009420 con 0x7f8ee006d370 2023-10-10T20:26:16.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8edf7fe700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009ef0 tx=0x7f8ed000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:16.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000cc50 con 0x7f8ee006d370 2023-10-10T20:26:16.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000f040 con 0x7f8ee006d370 2023-10-10T20:26:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee0157870 con 0x7f8ee006d370 2023-10-10T20:26:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0014480 con 0x7f8ee006d370 2023-10-10T20:26:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.970+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee0157eb0 con 0x7f8ee006d370 2023-10-10T20:26:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.971+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee004c3c0 con 0x7f8ee006d370 2023-10-10T20:26:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.972+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ed00147f0 con 0x7f8ee006d370 2023-10-10T20:26:16.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.972+0000 7f8ec77fe700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 0x7f8ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:16.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.972+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ed0085010 con 0x7f8ee006d370 2023-10-10T20:26:16.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.972+0000 7f8edeffd700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 0x7f8ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:16.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.973+0000 7f8edeffd700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 0x7f8ecc0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ee015b380 tx=0x7f8ed4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:16.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:16.977+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ed0052280 con 0x7f8ee006d370 2023-10-10T20:26:17.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.199+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ee00cdf10 con 0x7f8ee006d370 2023-10-10T20:26:17.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.829+0000 7f8ec77fe700 1 -- 172.21.15.148:0/2146832110 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ed00558a0 con 0x7f8ee006d370 2023-10-10T20:26:17.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 msgr2=0x7f8ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 0x7f8ecc0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ee015b380 tx=0x7f8ed4017000 comp rx=0 tx=0).stop 2023-10-10T20:26:17.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 msgr2=0x7f8ee01472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009ef0 tx=0x7f8ed000b040 comp rx=0 tx=0).stop 2023-10-10T20:26:17.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 shutdown_connections 2023-10-10T20:26:17.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8ecc066f30 0x7f8ecc0693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ee0158df0 0x7f8ee00c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ee006d370 0x7f8ee01472a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.831+0000 7f8ee60a9700 1 --2- 172.21.15.148:0/2146832110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ee0152300 0x7f8ee01477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.832+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 >> 172.21.15.148:0/2146832110 conn(0x7f8ee0069870 msgr2=0x7f8ee00cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:17.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.832+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 shutdown_connections 2023-10-10T20:26:17.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.832+0000 7f8ee60a9700 1 -- 172.21.15.148:0/2146832110 wait complete. 2023-10-10T20:26:17.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:17.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.961+0000 7f2af2f19700 1 Processor -- start 2023-10-10T20:26:17.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.961+0000 7f2af2f19700 1 -- start start 2023-10-10T20:26:17.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2f19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 0x7f2aec15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:17.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2f19700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0cb7c0 con 0x7f2aec15dda0 2023-10-10T20:26:17.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2f19700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec15e290 con 0x7f2aec0cd6e0 2023-10-10T20:26:17.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2f19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec15e850 con 0x7f2aec0cdab0 2023-10-10T20:26:17.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2718700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2aec15dda0 0x7f2aec15cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57736/0 (socket says 172.21.15.148:57736) 2023-10-10T20:26:17.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af2718700 1 -- 172.21.15.148:0/3615881080 learned_addr learned my addr 172.21.15.148:0/3615881080 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:17.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af1716700 1 --2- 172.21.15.148:0/3615881080 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 0x7f2aec15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:17.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625506055 0 0) 0x7f2aec0cb7c0 con 0x7f2aec15dda0 2023-10-10T20:26:17.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad8003570 con 0x7f2aec15dda0 2023-10-10T20:26:17.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567177846 0 0) 0x7f2aec15e290 con 0x7f2aec0cd6e0 2023-10-10T20:26:17.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.962+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aec0cb7c0 con 0x7f2aec0cd6e0 2023-10-10T20:26:17.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af1716700 1 -- 172.21.15.148:0/3615881080 >> v1:172.21.15.161:6789/0 conn(0x7f2aec0cd6e0 legacy=0x7f2aec0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af1716700 1 -- 172.21.15.148:0/3615881080 >> v1:172.21.15.148:6789/0 conn(0x7f2aec15dda0 legacy=0x7f2aec15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af1716700 1 -- 172.21.15.148:0/3615881080 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec15e570 con 0x7f2aec0cdab0 2023-10-10T20:26:17.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af1716700 1 --2- 172.21.15.148:0/3615881080 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 0x7f2aec15d860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009910 tx=0x7f2adc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb36d54ecd043520 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:17.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc011630 con 0x7f2aec0cdab0 2023-10-10T20:26:17.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc011790 con 0x7f2aec0cdab0 2023-10-10T20:26:17.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af0f15700 1 -- 172.21.15.148:0/3615881080 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc011b30 con 0x7f2aec0cdab0 2023-10-10T20:26:17.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af2f19700 1 -- 172.21.15.148:0/3615881080 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 msgr2=0x7f2aec15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af2f19700 1 --2- 172.21.15.148:0/3615881080 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 0x7f2aec15d860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009910 tx=0x7f2adc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af2f19700 1 -- 172.21.15.148:0/3615881080 shutdown_connections 2023-10-10T20:26:17.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af2f19700 1 --2- 172.21.15.148:0/3615881080 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cdab0 0x7f2aec15d860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.963+0000 7f2af2f19700 1 -- 172.21.15.148:0/3615881080 >> 172.21.15.148:0/3615881080 conn(0x7f2aec067660 msgr2=0x7f2aec067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:17.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.964+0000 7f2af2f19700 1 -- 172.21.15.148:0/3615881080 shutdown_connections 2023-10-10T20:26:17.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.964+0000 7f2af2f19700 1 -- 172.21.15.148:0/3615881080 wait complete. 2023-10-10T20:26:17.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 Processor -- start 2023-10-10T20:26:17.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 -- start start 2023-10-10T20:26:17.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cd6e0 0x7f2aec16ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:17.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec0cdab0 0x7f2aec0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:17.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:17.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0cc360 con 0x7f2aec15dda0 2023-10-10T20:26:17.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0b68c0 con 0x7f2aec0cdab0 2023-10-10T20:26:17.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2f19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0b1760 con 0x7f2aec0cd6e0 2023-10-10T20:26:17.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2718700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2718700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46308/0 (socket says 172.21.15.148:46308) 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.965+0000 7f2af2718700 1 -- 172.21.15.148:0/296878631 learned_addr learned my addr 172.21.15.148:0/296878631 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af1f17700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cd6e0 0x7f2aec16ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af1716700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec0cdab0 0x7f2aec0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 -- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cd6e0 msgr2=0x7f2aec16ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cd6e0 0x7f2aec16ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 -- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec0cdab0 msgr2=0x7f2aec0b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:17.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec0cdab0 0x7f2aec0b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:17.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 -- 172.21.15.148:0/296878631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009420 con 0x7f2aec15dda0 2023-10-10T20:26:17.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2718700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400cbc0 tx=0x7f2ae4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:17.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400a7e0 con 0x7f2aec15dda0 2023-10-10T20:26:17.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae400a940 con 0x7f2aec15dda0 2023-10-10T20:26:17.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec0b1a00 con 0x7f2aec15dda0 2023-10-10T20:26:17.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400ad00 con 0x7f2aec15dda0 2023-10-10T20:26:17.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.966+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec0b22a0 con 0x7f2aec15dda0 2023-10-10T20:26:17.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.967+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec0cbb50 con 0x7f2aec15dda0 2023-10-10T20:26:17.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.968+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ae4007300 con 0x7f2aec15dda0 2023-10-10T20:26:17.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.968+0000 7f2ae1ffb700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 0x7f2ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:17.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.968+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ae40847d0 con 0x7f2aec15dda0 2023-10-10T20:26:17.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.968+0000 7f2af1f17700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 0x7f2ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:17.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.969+0000 7f2af1f17700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 0x7f2ad00693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007fd0 tx=0x7f2ae8007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:17.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:17.973+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ae4051ac0 con 0x7f2aec15dda0 2023-10-10T20:26:18.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.196+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2aec04bb80 con 0x7f2aec15dda0 2023-10-10T20:26:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.197+0000 7f2ae1ffb700 1 -- 172.21.15.148:0/296878631 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f2ae400ae60 con 0x7f2aec15dda0 2023-10-10T20:26:18.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:18.200 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:26:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.199+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 msgr2=0x7f2ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 0x7f2ad00693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007fd0 tx=0x7f2ae8007d50 comp rx=0 tx=0).stop 2023-10-10T20:26:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 msgr2=0x7f2aec16a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400cbc0 tx=0x7f2ae4001a20 comp rx=0 tx=0).stop 2023-10-10T20:26:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 shutdown_connections 2023-10-10T20:26:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2ad0066f30 0x7f2ad00693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2aec15dda0 0x7f2aec16a160 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2aec0cd6e0 0x7f2aec16ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 --2- 172.21.15.148:0/296878631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec0cdab0 0x7f2aec0b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 >> 172.21.15.148:0/296878631 conn(0x7f2aec067660 msgr2=0x7f2aec0c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 shutdown_connections 2023-10-10T20:26:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.200+0000 7f2af2f19700 1 -- 172.21.15.148:0/296878631 wait complete. 2023-10-10T20:26:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:26:18.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:26:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.327+0000 7ff04995f700 1 Processor -- start 2023-10-10T20:26:18.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.327+0000 7ff04995f700 1 -- start start 2023-10-10T20:26:18.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04995f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0440cd6e0 0x7ff0440cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04995f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff04415cbd0 0x7ff04415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04995f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0440cb7c0 con 0x7ff0440cdab0 2023-10-10T20:26:18.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04995f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04415e850 con 0x7ff0440cd6e0 2023-10-10T20:26:18.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04995f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0440cd030 con 0x7ff04415cbd0 2023-10-10T20:26:18.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04895d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0440cd6e0 0x7ff0440cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff043fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0440cdab0 0x7ff04415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57742/0 (socket says 172.21.15.148:57742) 2023-10-10T20:26:18.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff043fff700 1 -- 172.21.15.148:0/740251508 learned_addr learned my addr 172.21.15.148:0/740251508 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:18.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04895d700 1 -- 172.21.15.148:0/740251508 >> v2:172.21.15.161:3300/0 conn(0x7ff04415cbd0 msgr2=0x7ff04415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:18.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04915e700 1 --2- 172.21.15.148:0/740251508 >> v2:172.21.15.161:3300/0 conn(0x7ff04415cbd0 0x7ff04415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.328+0000 7ff04895d700 1 --2- 172.21.15.148:0/740251508 >> v2:172.21.15.161:3300/0 conn(0x7ff04415cbd0 0x7ff04415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04895d700 1 -- 172.21.15.148:0/740251508 >> v1:172.21.15.148:6789/0 conn(0x7ff0440cdab0 legacy=0x7ff04415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04895d700 1 -- 172.21.15.148:0/740251508 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04415e290 con 0x7ff0440cd6e0 2023-10-10T20:26:18.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff0437fe700 1 -- 172.21.15.148:0/740251508 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672940183 0 0) 0x7ff0440cb7c0 con 0x7ff0440cdab0 2023-10-10T20:26:18.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04895d700 1 --2- 172.21.15.148:0/740251508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff0440cbc50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff0440cb730 tx=0x7ff0380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff44cf8e003660f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff0437fe700 1 -- 172.21.15.148:0/740251508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800b6b0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff0437fe700 1 -- 172.21.15.148:0/740251508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800b810 con 0x7ff0440cd6e0 2023-10-10T20:26:18.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff0437fe700 1 -- 172.21.15.148:0/740251508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800bbb0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 -- 172.21.15.148:0/740251508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 msgr2=0x7ff0440cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 --2- 172.21.15.148:0/740251508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff0440cbc50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff0440cb730 tx=0x7ff0380091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 -- 172.21.15.148:0/740251508 shutdown_connections 2023-10-10T20:26:18.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 --2- 172.21.15.148:0/740251508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff0440cbc50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 --2- 172.21.15.148:0/740251508 >> v2:172.21.15.161:3300/0 conn(0x7ff04415cbd0 0x7ff04415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 -- 172.21.15.148:0/740251508 >> 172.21.15.148:0/740251508 conn(0x7ff044067660 msgr2=0x7ff044067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:18.337 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 -- 172.21.15.148:0/740251508 shutdown_connections 2023-10-10T20:26:18.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.329+0000 7ff04995f700 1 -- 172.21.15.148:0/740251508 wait complete. 2023-10-10T20:26:18.338 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.330+0000 7ff04995f700 1 Processor -- start 2023-10-10T20:26:18.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 -- start start 2023-10-10T20:26:18.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0440cdab0 0x7ff044157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 0x7ff0441578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440b57d0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04416e600 con 0x7ff04415cbd0 2023-10-10T20:26:18.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04995f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440c8410 con 0x7ff0440cdab0 2023-10-10T20:26:18.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04895d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04895d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46334/0 (socket says 172.21.15.148:46334) 2023-10-10T20:26:18.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.331+0000 7ff04895d700 1 -- 172.21.15.148:0/922714896 learned_addr learned my addr 172.21.15.148:0/922714896 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:18.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04915e700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 0x7ff0441578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff043fff700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0440cdab0 0x7ff044157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 -- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0440cdab0 msgr2=0x7ff044157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0440cdab0 0x7ff044157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 -- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 msgr2=0x7ff0441578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 0x7ff0441578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 -- 172.21.15.148:0/922714896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009420 con 0x7ff0440cd6e0 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04915e700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 0x7ff0441578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04895d700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b0e0 tx=0x7ff038017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011420 con 0x7ff0440cd6e0 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038011580 con 0x7ff0440cd6e0 2023-10-10T20:26:18.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04416e320 con 0x7ff0440cd6e0 2023-10-10T20:26:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011920 con 0x7ff0440cd6e0 2023-10-10T20:26:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.332+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0440c80a0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.333+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0440c8230 con 0x7ff0440cd6e0 2023-10-10T20:26:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.333+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff038011d10 con 0x7ff0440cd6e0 2023-10-10T20:26:18.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.334+0000 7ff040ff9700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 0x7ff0280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.334+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0380860b0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.334+0000 7ff043fff700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 0x7ff0280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.334+0000 7ff043fff700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 0x7ff0280693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff0440b4b10 tx=0x7ff030009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.338+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0380533e0 con 0x7ff0440cd6e0 2023-10-10T20:26:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.520+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff04415e570 con 0x7ff0440cd6e0 2023-10-10T20:26:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.521+0000 7ff040ff9700 1 -- 172.21.15.148:0/922714896 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+15413 (secure 0 0 0) 0x7ff038015090 con 0x7ff0440cd6e0 2023-10-10T20:26:18.523 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:18.524 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":146,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:26:15.799496+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:26:18.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 msgr2=0x7ff0280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 0x7ff0280693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff0440b4b10 tx=0x7ff030009040 comp rx=0 tx=0).stop 2023-10-10T20:26:18.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 msgr2=0x7ff04414c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b0e0 tx=0x7ff038017000 comp rx=0 tx=0).stop 2023-10-10T20:26:18.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 shutdown_connections 2023-10-10T20:26:18.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff028066f30 0x7ff0280693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff04415cbd0 0x7ff0441578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0440cd6e0 0x7ff04414c870 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 --2- 172.21.15.148:0/922714896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0440cdab0 0x7ff044157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 >> 172.21.15.148:0/922714896 conn(0x7ff044067660 msgr2=0x7ff0440d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:18.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 shutdown_connections 2023-10-10T20:26:18.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.523+0000 7ff04995f700 1 -- 172.21.15.148:0/922714896 wait complete. 2023-10-10T20:26:18.538 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:26:18.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:26:18.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.650+0000 7fb6bf26d700 1 Processor -- start 2023-10-10T20:26:18.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.651+0000 7fb6bf26d700 1 -- start start 2023-10-10T20:26:18.655 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.651+0000 7fb6bf26d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6b8152010 0x7fb6b815b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.651+0000 7fb6bf26d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b806c010 con 0x7fb6b8158e50 2023-10-10T20:26:18.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.651+0000 7fb6bf26d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b806bd10 con 0x7fb6b81523e0 2023-10-10T20:26:18.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.651+0000 7fb6bf26d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b815c060 con 0x7fb6b8152010 2023-10-10T20:26:18.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd80a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6b8158e50 0x7fb6b80cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57744/0 (socket says 172.21.15.148:57744) 2023-10-10T20:26:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd80a700 1 -- 172.21.15.148:0/2465561795 learned_addr learned my addr 172.21.15.148:0/2465561795 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd009700 1 --2- 172.21.15.148:0/2465561795 >> v2:172.21.15.161:3300/0 conn(0x7fb6b8152010 0x7fb6b815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098083787 0 0) 0x7fb6b806c010 con 0x7fb6b8158e50 2023-10-10T20:26:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a0003570 con 0x7fb6b8158e50 2023-10-10T20:26:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd009700 1 -- 172.21.15.148:0/2465561795 >> v1:172.21.15.161:6789/0 conn(0x7fb6b81523e0 legacy=0x7fb6b806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573187079 0 0) 0x7fb6b806bd10 con 0x7fb6b81523e0 2023-10-10T20:26:18.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd009700 1 -- 172.21.15.148:0/2465561795 >> v1:172.21.15.148:6789/0 conn(0x7fb6b8158e50 legacy=0x7fb6b80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd009700 1 -- 172.21.15.148:0/2465561795 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b815c340 con 0x7fb6b8152010 2023-10-10T20:26:18.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6bd009700 1 --2- 172.21.15.148:0/2465561795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b8152010 0x7fb6b815b810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098e0 tx=0x7fb6a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d7129e46b64fba server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.652+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800b640 con 0x7fb6b8152010 2023-10-10T20:26:18.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a800b7a0 con 0x7fb6b8152010 2023-10-10T20:26:18.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6b7fff700 1 -- 172.21.15.148:0/2465561795 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800bb20 con 0x7fb6b8152010 2023-10-10T20:26:18.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2465561795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b8152010 msgr2=0x7fb6b815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2465561795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b8152010 0x7fb6b815b810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098e0 tx=0x7fb6a80091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2465561795 shutdown_connections 2023-10-10T20:26:18.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2465561795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b8152010 0x7fb6b815b810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2465561795 >> 172.21.15.148:0/2465561795 conn(0x7fb6b8069870 msgr2=0x7fb6b8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:18.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2465561795 shutdown_connections 2023-10-10T20:26:18.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.653+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2465561795 wait complete. 2023-10-10T20:26:18.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.654+0000 7fb6bf26d700 1 Processor -- start 2023-10-10T20:26:18.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.654+0000 7fb6bf26d700 1 -- start start 2023-10-10T20:26:18.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 0x7fb6b815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 0x7fb6b815a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b20d0 con 0x7fb6b8152010 2023-10-10T20:26:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b816bf20 con 0x7fb6b81523e0 2023-10-10T20:26:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bf26d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b1a80 con 0x7fb6b8158e50 2023-10-10T20:26:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46336/0 (socket says 172.21.15.148:46336) 2023-10-10T20:26:18.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 -- 172.21.15.148:0/2032993234 learned_addr learned my addr 172.21.15.148:0/2032993234 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:18.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd80a700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 0x7fb6b815a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bc808700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 0x7fb6b815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 -- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 msgr2=0x7fb6b815a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 0x7fb6b815a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 -- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 msgr2=0x7fb6b815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.655+0000 7fb6bd009700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 0x7fb6b815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bd009700 1 -- 172.21.15.148:0/2032993234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8009420 con 0x7fb6b8152010 2023-10-10T20:26:18.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bc808700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 0x7fb6b815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:18.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bd80a700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 0x7fb6b815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:18.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bd009700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80c59e0 tx=0x7fb6a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a800bb20 con 0x7fb6b8152010 2023-10-10T20:26:18.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a801a430 con 0x7fb6b8152010 2023-10-10T20:26:18.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80b1d60 con 0x7fb6b8152010 2023-10-10T20:26:18.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a801a7f0 con 0x7fb6b8152010 2023-10-10T20:26:18.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.656+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b80b27a0 con 0x7fb6b8152010 2023-10-10T20:26:18.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.657+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80b18e0 con 0x7fb6b8152010 2023-10-10T20:26:18.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.657+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6a801abe0 con 0x7fb6b8152010 2023-10-10T20:26:18.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.658+0000 7fb6b57fa700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 0x7fb6a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:18.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.658+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6a8086270 con 0x7fb6b8152010 2023-10-10T20:26:18.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.658+0000 7fb6bc808700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 0x7fb6a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:18.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.658+0000 7fb6bc808700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 0x7fb6a40693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb6b806b3d0 tx=0x7fb6ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:18.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.662+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b80b18e0 con 0x7fb6b8152010 2023-10-10T20:26:18.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.844+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6b80b2210 con 0x7fb6a4066f30 2023-10-10T20:26:18.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.852+0000 7fb6b57fa700 1 -- 172.21.15.148:0/2032993234 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285577 (secure 0 0 0) 0x7fb6ac00e5d0 con 0x7fb6a4066f30 2023-10-10T20:26:18.858 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 msgr2=0x7fb6a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 0x7fb6a40693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb6b806b3d0 tx=0x7fb6ac009040 comp rx=0 tx=0).stop 2023-10-10T20:26:18.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 msgr2=0x7fb6b8145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80c59e0 tx=0x7fb6a8017000 comp rx=0 tx=0).stop 2023-10-10T20:26:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 shutdown_connections 2023-10-10T20:26:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb6a4066f30 0x7fb6a40693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6b8158e50 0x7fb6b815a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6b8152010 0x7fb6b8145870 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 --2- 172.21.15.148:0/2032993234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b81523e0 0x7fb6b815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 >> 172.21.15.148:0/2032993234 conn(0x7fb6b8069870 msgr2=0x7fb6b80c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 shutdown_connections 2023-10-10T20:26:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:18.855+0000 7fb6bf26d700 1 -- 172.21.15.148:0/2032993234 wait complete. 2023-10-10T20:26:18.863 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:26:18.869 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-10-10T20:26:17.412243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7455,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555044,"statfs":{"total":1600000000,"available":1592365290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.618196"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813192+0000","last_change":"2023-10-10T20:26:14.813080+0000","last_active":"2023-10-10T20:26:14.813192+0000","last_peered":"2023-10-10T20:26:14.813192+0000","last_clean":"2023-10-10T20:26:14.813192+0000","last_became_active":"2023-10-10T20:26:14.812823+0000","last_became_peered":"2023-10-10T20:26:14.812823+0000","last_unstale":"2023-10-10T20:26:14.813192+0000","last_undegraded":"2023-10-10T20:26:14.813192+0000","last_fullsized":"2023-10-10T20:26:14.813192+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:22.346445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816665+0000","last_change":"2023-10-10T20:26:14.816612+0000","last_active":"2023-10-10T20:26:14.816665+0000","last_peered":"2023-10-10T20:26:14.816665+0000","last_clean":"2023-10-10T20:26:14.816665+0000","last_became_active":"2023-10-10T20:26:14.816302+0000","last_became_peered":"2023-10-10T20:26:14.816302+0000","last_unstale":"2023-10-10T20:26:14.816665+0000","last_undegraded":"2023-10-10T20:26:14.816665+0000","last_fullsized":"2023-10-10T20:26:14.816665+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:35.431993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812943+0000","last_change":"2023-10-10T20:26:14.812288+0000","last_active":"2023-10-10T20:26:14.812943+0000","last_peered":"2023-10-10T20:26:14.812943+0000","last_clean":"2023-10-10T20:26:14.812943+0000","last_became_active":"2023-10-10T20:26:14.811433+0000","last_became_peered":"2023-10-10T20:26:14.811433+0000","last_unstale":"2023-10-10T20:26:14.812943+0000","last_undegraded":"2023-10-10T20:26:14.812943+0000","last_fullsized":"2023-10-10T20:26:14.812943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:09.386940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823098+0000","last_change":"2023-10-10T20:26:14.823010+0000","last_active":"2023-10-10T20:26:14.823098+0000","last_peered":"2023-10-10T20:26:14.823098+0000","last_clean":"2023-10-10T20:26:14.823098+0000","last_became_active":"2023-10-10T20:26:14.822777+0000","last_became_peered":"2023-10-10T20:26:14.822777+0000","last_unstale":"2023-10-10T20:26:14.823098+0000","last_undegraded":"2023-10-10T20:26:14.823098+0000","last_fullsized":"2023-10-10T20:26:14.823098+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:37.495784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818183+0000","last_change":"2023-10-10T20:26:14.816660+0000","last_active":"2023-10-10T20:26:14.818183+0000","last_peered":"2023-10-10T20:26:14.818183+0000","last_clean":"2023-10-10T20:26:14.818183+0000","last_became_active":"2023-10-10T20:26:14.816067+0000","last_became_peered":"2023-10-10T20:26:14.816067+0000","last_unstale":"2023-10-10T20:26:14.818183+0000","last_undegraded":"2023-10-10T20:26:14.818183+0000","last_fullsized":"2023-10-10T20:26:14.818183+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:32.634303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810090+0000","last_change":"2023-10-10T20:26:14.809822+0000","last_active":"2023-10-10T20:26:14.810090+0000","last_peered":"2023-10-10T20:26:14.810090+0000","last_clean":"2023-10-10T20:26:14.810090+0000","last_became_active":"2023-10-10T20:26:14.809569+0000","last_became_peered":"2023-10-10T20:26:14.809569+0000","last_unstale":"2023-10-10T20:26:14.810090+0000","last_undegraded":"2023-10-10T20:26:14.810090+0000","last_fullsized":"2023-10-10T20:26:14.810090+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:58.873883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810070+0000","last_change":"2023-10-10T20:26:14.809817+0000","last_active":"2023-10-10T20:26:14.810070+0000","last_peered":"2023-10-10T20:26:14.810070+0000","last_clean":"2023-10-10T20:26:14.810070+0000","last_became_active":"2023-10-10T20:26:14.809039+0000","last_became_peered":"2023-10-10T20:26:14.809039+0000","last_unstale":"2023-10-10T20:26:14.810070+0000","last_undegraded":"2023-10-10T20:26:14.810070+0000","last_fullsized":"2023-10-10T20:26:14.810070+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:36.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816416+0000","last_change":"2023-10-10T20:26:14.816310+0000","last_active":"2023-10-10T20:26:14.816416+0000","last_peered":"2023-10-10T20:26:14.816416+0000","last_clean":"2023-10-10T20:26:14.816416+0000","last_became_active":"2023-10-10T20:26:14.816074+0000","last_became_peered":"2023-10-10T20:26:14.816074+0000","last_unstale":"2023-10-10T20:26:14.816416+0000","last_undegraded":"2023-10-10T20:26:14.816416+0000","last_fullsized":"2023-10-10T20:26:14.816416+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:16:11.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805230+0000","last_change":"2023-10-10T20:26:14.805153+0000","last_active":"2023-10-10T20:26:14.805230+0000","last_peered":"2023-10-10T20:26:14.805230+0000","last_clean":"2023-10-10T20:26:14.805230+0000","last_became_active":"2023-10-10T20:26:14.804899+0000","last_became_peered":"2023-10-10T20:26:14.804899+0000","last_unstale":"2023-10-10T20:26:14.805230+0000","last_undegraded":"2023-10-10T20:26:14.805230+0000","last_fullsized":"2023-10-10T20:26:14.805230+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:48.579836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809144+0000","last_change":"2023-10-10T20:26:14.808823+0000","last_active":"2023-10-10T20:26:14.809144+0000","last_peered":"2023-10-10T20:26:14.809144+0000","last_clean":"2023-10-10T20:26:14.809144+0000","last_became_active":"2023-10-10T20:26:14.808602+0000","last_became_peered":"2023-10-10T20:26:14.808602+0000","last_unstale":"2023-10-10T20:26:14.809144+0000","last_undegraded":"2023-10-10T20:26:14.809144+0000","last_fullsized":"2023-10-10T20:26:14.809144+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:58.586866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.817749+0000","last_change":"2023-10-10T20:26:14.817633+0000","last_active":"2023-10-10T20:26:14.817749+0000","last_peered":"2023-10-10T20:26:14.817749+0000","last_clean":"2023-10-10T20:26:14.817749+0000","last_became_active":"2023-10-10T20:26:14.817446+0000","last_became_peered":"2023-10-10T20:26:14.817446+0000","last_unstale":"2023-10-10T20:26:14.817749+0000","last_undegraded":"2023-10-10T20:26:14.817749+0000","last_fullsized":"2023-10-10T20:26:14.817749+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:24.406069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.806247+0000","last_change":"2023-10-10T20:26:14.805724+0000","last_active":"2023-10-10T20:26:14.806247+0000","last_peered":"2023-10-10T20:26:14.806247+0000","last_clean":"2023-10-10T20:26:14.806247+0000","last_became_active":"2023-10-10T20:26:14.805182+0000","last_became_peered":"2023-10-10T20:26:14.805182+0000","last_unstale":"2023-10-10T20:26:14.806247+0000","last_undegraded":"2023-10-10T20:26:14.806247+0000","last_fullsized":"2023-10-10T20:26:14.806247+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:36.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815595+0000","last_change":"2023-10-10T20:26:14.814133+0000","last_active":"2023-10-10T20:26:14.815595+0000","last_peered":"2023-10-10T20:26:14.815595+0000","last_clean":"2023-10-10T20:26:14.815595+0000","last_became_active":"2023-10-10T20:26:14.813910+0000","last_became_peered":"2023-10-10T20:26:14.813910+0000","last_unstale":"2023-10-10T20:26:14.815595+0000","last_undegraded":"2023-10-10T20:26:14.815595+0000","last_fullsized":"2023-10-10T20:26:14.815595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:04.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819501+0000","last_change":"2023-10-10T20:26:14.819349+0000","last_active":"2023-10-10T20:26:14.819501+0000","last_peered":"2023-10-10T20:26:14.819501+0000","last_clean":"2023-10-10T20:26:14.819501+0000","last_became_active":"2023-10-10T20:26:14.819067+0000","last_became_peered":"2023-10-10T20:26:14.819067+0000","last_unstale":"2023-10-10T20:26:14.819501+0000","last_undegraded":"2023-10-10T20:26:14.819501+0000","last_fullsized":"2023-10-10T20:26:14.819501+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:12.720556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815626+0000","last_change":"2023-10-10T20:26:14.814642+0000","last_active":"2023-10-10T20:26:14.815626+0000","last_peered":"2023-10-10T20:26:14.815626+0000","last_clean":"2023-10-10T20:26:14.815626+0000","last_became_active":"2023-10-10T20:26:14.813109+0000","last_became_peered":"2023-10-10T20:26:14.813109+0000","last_unstale":"2023-10-10T20:26:14.815626+0000","last_undegraded":"2023-10-10T20:26:14.815626+0000","last_fullsized":"2023-10-10T20:26:14.815626+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:05.424456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816446+0000","last_change":"2023-10-10T20:26:14.816383+0000","last_active":"2023-10-10T20:26:14.816446+0000","last_peered":"2023-10-10T20:26:14.816446+0000","last_clean":"2023-10-10T20:26:14.816446+0000","last_became_active":"2023-10-10T20:26:14.815920+0000","last_became_peered":"2023-10-10T20:26:14.815920+0000","last_unstale":"2023-10-10T20:26:14.816446+0000","last_undegraded":"2023-10-10T20:26:14.816446+0000","last_fullsized":"2023-10-10T20:26:14.816446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:11:25.506245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818759+0000","last_change":"2023-10-10T20:26:14.818559+0000","last_active":"2023-10-10T20:26:14.818759+0000","last_peered":"2023-10-10T20:26:14.818759+0000","last_clean":"2023-10-10T20:26:14.818759+0000","last_became_active":"2023-10-10T20:26:14.818320+0000","last_became_peered":"2023-10-10T20:26:14.818320+0000","last_unstale":"2023-10-10T20:26:14.818759+0000","last_undegraded":"2023-10-10T20:26:14.818759+0000","last_fullsized":"2023-10-10T20:26:14.818759+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:50.826456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813013+0000","last_change":"2023-10-10T20:26:14.812936+0000","last_active":"2023-10-10T20:26:14.813013+0000","last_peered":"2023-10-10T20:26:14.813013+0000","last_clean":"2023-10-10T20:26:14.813013+0000","last_became_active":"2023-10-10T20:26:14.812392+0000","last_became_peered":"2023-10-10T20:26:14.812392+0000","last_unstale":"2023-10-10T20:26:14.813013+0000","last_undegraded":"2023-10-10T20:26:14.813013+0000","last_fullsized":"2023-10-10T20:26:14.813013+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:28.152057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816510+0000","last_change":"2023-10-10T20:26:14.814552+0000","last_active":"2023-10-10T20:26:14.816510+0000","last_peered":"2023-10-10T20:26:14.816510+0000","last_clean":"2023-10-10T20:26:14.816510+0000","last_became_active":"2023-10-10T20:26:14.814013+0000","last_became_peered":"2023-10-10T20:26:14.814013+0000","last_unstale":"2023-10-10T20:26:14.816510+0000","last_undegraded":"2023-10-10T20:26:14.816510+0000","last_fullsized":"2023-10-10T20:26:14.816510+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:55.273265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815432+0000","last_change":"2023-10-10T20:26:14.814992+0000","last_active":"2023-10-10T20:26:14.815432+0000","last_peered":"2023-10-10T20:26:14.815432+0000","last_clean":"2023-10-10T20:26:14.815432+0000","last_became_active":"2023-10-10T20:26:14.814777+0000","last_became_peered":"2023-10-10T20:26:14.814777+0000","last_unstale":"2023-10-10T20:26:14.815432+0000","last_undegraded":"2023-10-10T20:26:14.815432+0000","last_fullsized":"2023-10-10T20:26:14.815432+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:02.788705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815297+0000","last_change":"2023-10-10T20:26:14.814843+0000","last_active":"2023-10-10T20:26:14.815297+0000","last_peered":"2023-10-10T20:26:14.815297+0000","last_clean":"2023-10-10T20:26:14.815297+0000","last_became_active":"2023-10-10T20:26:14.814615+0000","last_became_peered":"2023-10-10T20:26:14.814615+0000","last_unstale":"2023-10-10T20:26:14.815297+0000","last_undegraded":"2023-10-10T20:26:14.815297+0000","last_fullsized":"2023-10-10T20:26:14.815297+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:47.218066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818886+0000","last_change":"2023-10-10T20:26:14.818823+0000","last_active":"2023-10-10T20:26:14.818886+0000","last_peered":"2023-10-10T20:26:14.818886+0000","last_clean":"2023-10-10T20:26:14.818886+0000","last_became_active":"2023-10-10T20:26:14.818599+0000","last_became_peered":"2023-10-10T20:26:14.818599+0000","last_unstale":"2023-10-10T20:26:14.818886+0000","last_undegraded":"2023-10-10T20:26:14.818886+0000","last_fullsized":"2023-10-10T20:26:14.818886+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:59.677714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805598+0000","last_change":"2023-10-10T20:26:14.805393+0000","last_active":"2023-10-10T20:26:14.805598+0000","last_peered":"2023-10-10T20:26:14.805598+0000","last_clean":"2023-10-10T20:26:14.805598+0000","last_became_active":"2023-10-10T20:26:14.805130+0000","last_became_peered":"2023-10-10T20:26:14.805130+0000","last_unstale":"2023-10-10T20:26:14.805598+0000","last_undegraded":"2023-10-10T20:26:14.805598+0000","last_fullsized":"2023-10-10T20:26:14.805598+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:12.602173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816489+0000","last_change":"2023-10-10T20:26:14.814238+0000","last_active":"2023-10-10T20:26:14.816489+0000","last_peered":"2023-10-10T20:26:14.816489+0000","last_clean":"2023-10-10T20:26:14.816489+0000","last_became_active":"2023-10-10T20:26:14.813706+0000","last_became_peered":"2023-10-10T20:26:14.813706+0000","last_unstale":"2023-10-10T20:26:14.816489+0000","last_undegraded":"2023-10-10T20:26:14.816489+0000","last_fullsized":"2023-10-10T20:26:14.816489+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:52.366102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822156+0000","last_change":"2023-10-10T20:26:14.821805+0000","last_active":"2023-10-10T20:26:14.822156+0000","last_peered":"2023-10-10T20:26:14.822156+0000","last_clean":"2023-10-10T20:26:14.822156+0000","last_became_active":"2023-10-10T20:26:14.817027+0000","last_became_peered":"2023-10-10T20:26:14.817027+0000","last_unstale":"2023-10-10T20:26:14.822156+0000","last_undegraded":"2023-10-10T20:26:14.822156+0000","last_fullsized":"2023-10-10T20:26:14.822156+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:46:57.561657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816308+0000","last_change":"2023-10-10T20:26:14.816247+0000","last_active":"2023-10-10T20:26:14.816308+0000","last_peered":"2023-10-10T20:26:14.816308+0000","last_clean":"2023-10-10T20:26:14.816308+0000","last_became_active":"2023-10-10T20:26:14.815644+0000","last_became_peered":"2023-10-10T20:26:14.815644+0000","last_unstale":"2023-10-10T20:26:14.816308+0000","last_undegraded":"2023-10-10T20:26:14.816308+0000","last_fullsized":"2023-10-10T20:26:14.816308+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:49.792681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813937+0000","last_change":"2023-10-10T20:26:14.813487+0000","last_active":"2023-10-10T20:26:14.813937+0000","last_peered":"2023-10-10T20:26:14.813937+0000","last_clean":"2023-10-10T20:26:14.813937+0000","last_became_active":"2023-10-10T20:26:14.813005+0000","last_became_peered":"2023-10-10T20:26:14.813005+0000","last_unstale":"2023-10-10T20:26:14.813937+0000","last_undegraded":"2023-10-10T20:26:14.813937+0000","last_fullsized":"2023-10-10T20:26:14.813937+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:39.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815580+0000","last_change":"2023-10-10T20:26:14.815511+0000","last_active":"2023-10-10T20:26:14.815580+0000","last_peered":"2023-10-10T20:26:14.815580+0000","last_clean":"2023-10-10T20:26:14.815580+0000","last_became_active":"2023-10-10T20:26:14.815220+0000","last_became_peered":"2023-10-10T20:26:14.815220+0000","last_unstale":"2023-10-10T20:26:14.815580+0000","last_undegraded":"2023-10-10T20:26:14.815580+0000","last_fullsized":"2023-10-10T20:26:14.815580+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:23:09.555039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822026+0000","last_change":"2023-10-10T20:26:14.821875+0000","last_active":"2023-10-10T20:26:14.822026+0000","last_peered":"2023-10-10T20:26:14.822026+0000","last_clean":"2023-10-10T20:26:14.822026+0000","last_became_active":"2023-10-10T20:26:14.821384+0000","last_became_peered":"2023-10-10T20:26:14.821384+0000","last_unstale":"2023-10-10T20:26:14.822026+0000","last_undegraded":"2023-10-10T20:26:14.822026+0000","last_fullsized":"2023-10-10T20:26:14.822026+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:27.143818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815612+0000","last_change":"2023-10-10T20:26:14.814513+0000","last_active":"2023-10-10T20:26:14.815612+0000","last_peered":"2023-10-10T20:26:14.815612+0000","last_clean":"2023-10-10T20:26:14.815612+0000","last_became_active":"2023-10-10T20:26:14.812918+0000","last_became_peered":"2023-10-10T20:26:14.812918+0000","last_unstale":"2023-10-10T20:26:14.815612+0000","last_undegraded":"2023-10-10T20:26:14.815612+0000","last_fullsized":"2023-10-10T20:26:14.815612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:05.146769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819030+0000","last_change":"2023-10-10T20:26:14.818305+0000","last_active":"2023-10-10T20:26:14.819030+0000","last_peered":"2023-10-10T20:26:14.819030+0000","last_clean":"2023-10-10T20:26:14.819030+0000","last_became_active":"2023-10-10T20:26:14.817679+0000","last_became_peered":"2023-10-10T20:26:14.817679+0000","last_unstale":"2023-10-10T20:26:14.819030+0000","last_undegraded":"2023-10-10T20:26:14.819030+0000","last_fullsized":"2023-10-10T20:26:14.819030+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:47.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812595+0000","last_change":"2023-10-10T20:26:14.812347+0000","last_active":"2023-10-10T20:26:14.812595+0000","last_peered":"2023-10-10T20:26:14.812595+0000","last_clean":"2023-10-10T20:26:14.812595+0000","last_became_active":"2023-10-10T20:26:14.811734+0000","last_became_peered":"2023-10-10T20:26:14.811734+0000","last_unstale":"2023-10-10T20:26:14.812595+0000","last_undegraded":"2023-10-10T20:26:14.812595+0000","last_fullsized":"2023-10-10T20:26:14.812595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:59.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.811924+0000","last_change":"2023-10-10T20:26:14.811640+0000","last_active":"2023-10-10T20:26:14.811924+0000","last_peered":"2023-10-10T20:26:14.811924+0000","last_clean":"2023-10-10T20:26:14.811924+0000","last_became_active":"2023-10-10T20:26:14.811355+0000","last_became_peered":"2023-10-10T20:26:14.811355+0000","last_unstale":"2023-10-10T20:26:14.811924+0000","last_undegraded":"2023-10-10T20:26:14.811924+0000","last_fullsized":"2023-10-10T20:26:14.811924+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:47.218066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"145'4","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820580+0000","last_change":"2023-10-10T20:26:13.813742+0000","last_active":"2023-10-10T20:26:14.820580+0000","last_peered":"2023-10-10T20:26:14.820580+0000","last_clean":"2023-10-10T20:26:14.820580+0000","last_became_active":"2023-10-10T20:26:13.813327+0000","last_became_peered":"2023-10-10T20:26:13.813327+0000","last_unstale":"2023-10-10T20:26:14.820580+0000","last_undegraded":"2023-10-10T20:26:14.820580+0000","last_fullsized":"2023-10-10T20:26:14.820580+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:49.817040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812922+0000","last_change":"2023-10-10T20:26:14.812139+0000","last_active":"2023-10-10T20:26:14.812922+0000","last_peered":"2023-10-10T20:26:14.812922+0000","last_clean":"2023-10-10T20:26:14.812922+0000","last_became_active":"2023-10-10T20:26:14.811097+0000","last_became_peered":"2023-10-10T20:26:14.811097+0000","last_unstale":"2023-10-10T20:26:14.812922+0000","last_undegraded":"2023-10-10T20:26:14.812922+0000","last_fullsized":"2023-10-10T20:26:14.812922+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:25.171860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819456+0000","last_change":"2023-10-10T20:26:13.806645+0000","last_active":"2023-10-10T20:26:14.819456+0000","last_peered":"2023-10-10T20:26:14.819456+0000","last_clean":"2023-10-10T20:26:14.819456+0000","last_became_active":"2023-10-10T20:26:13.806360+0000","last_became_peered":"2023-10-10T20:26:13.806360+0000","last_unstale":"2023-10-10T20:26:14.819456+0000","last_undegraded":"2023-10-10T20:26:14.819456+0000","last_fullsized":"2023-10-10T20:26:14.819456+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:21.337029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819048+0000","last_change":"2023-10-10T20:26:14.818461+0000","last_active":"2023-10-10T20:26:14.819048+0000","last_peered":"2023-10-10T20:26:14.819048+0000","last_clean":"2023-10-10T20:26:14.819048+0000","last_became_active":"2023-10-10T20:26:14.817998+0000","last_became_peered":"2023-10-10T20:26:14.817998+0000","last_unstale":"2023-10-10T20:26:14.819048+0000","last_undegraded":"2023-10-10T20:26:14.819048+0000","last_fullsized":"2023-10-10T20:26:14.819048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:28.152057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818696+0000","last_change":"2023-10-10T20:26:13.805620+0000","last_active":"2023-10-10T20:26:14.818696+0000","last_peered":"2023-10-10T20:26:14.818696+0000","last_clean":"2023-10-10T20:26:14.818696+0000","last_became_active":"2023-10-10T20:26:13.805404+0000","last_became_peered":"2023-10-10T20:26:13.805404+0000","last_unstale":"2023-10-10T20:26:14.818696+0000","last_undegraded":"2023-10-10T20:26:14.818696+0000","last_fullsized":"2023-10-10T20:26:14.818696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:16.583830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822234+0000","last_change":"2023-10-10T20:26:14.822169+0000","last_active":"2023-10-10T20:26:14.822234+0000","last_peered":"2023-10-10T20:26:14.822234+0000","last_clean":"2023-10-10T20:26:14.822234+0000","last_became_active":"2023-10-10T20:26:14.821958+0000","last_became_peered":"2023-10-10T20:26:14.821958+0000","last_unstale":"2023-10-10T20:26:14.822234+0000","last_undegraded":"2023-10-10T20:26:14.822234+0000","last_fullsized":"2023-10-10T20:26:14.822234+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:29.222891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819936+0000","last_change":"2023-10-10T20:26:13.804037+0000","last_active":"2023-10-10T20:26:14.819936+0000","last_peered":"2023-10-10T20:26:14.819936+0000","last_clean":"2023-10-10T20:26:14.819936+0000","last_became_active":"2023-10-10T20:26:13.803820+0000","last_became_peered":"2023-10-10T20:26:13.803820+0000","last_unstale":"2023-10-10T20:26:14.819936+0000","last_undegraded":"2023-10-10T20:26:14.819936+0000","last_fullsized":"2023-10-10T20:26:14.819936+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:33.311364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813336+0000","last_change":"2023-10-10T20:26:14.813080+0000","last_active":"2023-10-10T20:26:14.813336+0000","last_peered":"2023-10-10T20:26:14.813336+0000","last_clean":"2023-10-10T20:26:14.813336+0000","last_became_active":"2023-10-10T20:26:14.812681+0000","last_became_peered":"2023-10-10T20:26:14.812681+0000","last_unstale":"2023-10-10T20:26:14.813336+0000","last_undegraded":"2023-10-10T20:26:14.813336+0000","last_fullsized":"2023-10-10T20:26:14.813336+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:53:07.893203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822497+0000","last_change":"2023-10-10T20:26:13.803921+0000","last_active":"2023-10-10T20:26:14.822497+0000","last_peered":"2023-10-10T20:26:14.822497+0000","last_clean":"2023-10-10T20:26:14.822497+0000","last_became_active":"2023-10-10T20:26:13.803671+0000","last_became_peered":"2023-10-10T20:26:13.803671+0000","last_unstale":"2023-10-10T20:26:14.822497+0000","last_undegraded":"2023-10-10T20:26:14.822497+0000","last_fullsized":"2023-10-10T20:26:14.822497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:23.783432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812622+0000","last_change":"2023-10-10T20:26:14.812517+0000","last_active":"2023-10-10T20:26:14.812622+0000","last_peered":"2023-10-10T20:26:14.812622+0000","last_clean":"2023-10-10T20:26:14.812622+0000","last_became_active":"2023-10-10T20:26:14.812074+0000","last_became_peered":"2023-10-10T20:26:14.812074+0000","last_unstale":"2023-10-10T20:26:14.812622+0000","last_undegraded":"2023-10-10T20:26:14.812622+0000","last_fullsized":"2023-10-10T20:26:14.812622+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:38.344472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823191+0000","last_change":"2023-10-10T20:26:13.811443+0000","last_active":"2023-10-10T20:26:14.823191+0000","last_peered":"2023-10-10T20:26:14.823191+0000","last_clean":"2023-10-10T20:26:14.823191+0000","last_became_active":"2023-10-10T20:26:13.811075+0000","last_became_peered":"2023-10-10T20:26:13.811075+0000","last_unstale":"2023-10-10T20:26:14.823191+0000","last_undegraded":"2023-10-10T20:26:14.823191+0000","last_fullsized":"2023-10-10T20:26:14.823191+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:11.592757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808602+0000","last_change":"2023-10-10T20:26:14.808260+0000","last_active":"2023-10-10T20:26:14.808602+0000","last_peered":"2023-10-10T20:26:14.808602+0000","last_clean":"2023-10-10T20:26:14.808602+0000","last_became_active":"2023-10-10T20:26:14.807933+0000","last_became_peered":"2023-10-10T20:26:14.807933+0000","last_unstale":"2023-10-10T20:26:14.808602+0000","last_undegraded":"2023-10-10T20:26:14.808602+0000","last_fullsized":"2023-10-10T20:26:14.808602+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:25.205828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805930+0000","last_change":"2023-10-10T20:26:13.810585+0000","last_active":"2023-10-10T20:26:14.805930+0000","last_peered":"2023-10-10T20:26:14.805930+0000","last_clean":"2023-10-10T20:26:14.805930+0000","last_became_active":"2023-10-10T20:26:13.810319+0000","last_became_peered":"2023-10-10T20:26:13.810319+0000","last_unstale":"2023-10-10T20:26:14.805930+0000","last_undegraded":"2023-10-10T20:26:14.805930+0000","last_fullsized":"2023-10-10T20:26:14.805930+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:27.365697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816633+0000","last_change":"2023-10-10T20:26:14.816573+0000","last_active":"2023-10-10T20:26:14.816633+0000","last_peered":"2023-10-10T20:26:14.816633+0000","last_clean":"2023-10-10T20:26:14.816633+0000","last_became_active":"2023-10-10T20:26:14.816326+0000","last_became_peered":"2023-10-10T20:26:14.816326+0000","last_unstale":"2023-10-10T20:26:14.816633+0000","last_undegraded":"2023-10-10T20:26:14.816633+0000","last_fullsized":"2023-10-10T20:26:14.816633+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:40.212927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.803320+0000","last_change":"2023-10-10T20:26:13.810239+0000","last_active":"2023-10-10T20:26:14.803320+0000","last_peered":"2023-10-10T20:26:14.803320+0000","last_clean":"2023-10-10T20:26:14.803320+0000","last_became_active":"2023-10-10T20:26:13.808022+0000","last_became_peered":"2023-10-10T20:26:13.808022+0000","last_unstale":"2023-10-10T20:26:14.803320+0000","last_undegraded":"2023-10-10T20:26:14.803320+0000","last_fullsized":"2023-10-10T20:26:14.803320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.687349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813171+0000","last_change":"2023-10-10T20:26:14.812788+0000","last_active":"2023-10-10T20:26:14.813171+0000","last_peered":"2023-10-10T20:26:14.813171+0000","last_clean":"2023-10-10T20:26:14.813171+0000","last_became_active":"2023-10-10T20:26:14.812241+0000","last_became_peered":"2023-10-10T20:26:14.812241+0000","last_unstale":"2023-10-10T20:26:14.813171+0000","last_undegraded":"2023-10-10T20:26:14.813171+0000","last_fullsized":"2023-10-10T20:26:14.813171+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:18.472820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.802364+0000","last_change":"2023-10-10T20:26:13.804155+0000","last_active":"2023-10-10T20:26:14.802364+0000","last_peered":"2023-10-10T20:26:14.802364+0000","last_clean":"2023-10-10T20:26:14.802364+0000","last_became_active":"2023-10-10T20:26:13.803882+0000","last_became_peered":"2023-10-10T20:26:13.803882+0000","last_unstale":"2023-10-10T20:26:14.802364+0000","last_undegraded":"2023-10-10T20:26:14.802364+0000","last_fullsized":"2023-10-10T20:26:14.802364+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:26.134402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810835+0000","last_change":"2023-10-10T20:26:14.810762+0000","last_active":"2023-10-10T20:26:14.810835+0000","last_peered":"2023-10-10T20:26:14.810835+0000","last_clean":"2023-10-10T20:26:14.810835+0000","last_became_active":"2023-10-10T20:26:14.810504+0000","last_became_peered":"2023-10-10T20:26:14.810504+0000","last_unstale":"2023-10-10T20:26:14.810835+0000","last_undegraded":"2023-10-10T20:26:14.810835+0000","last_fullsized":"2023-10-10T20:26:14.810835+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:37.199366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808760+0000","last_change":"2023-10-10T20:26:13.810274+0000","last_active":"2023-10-10T20:26:14.808760+0000","last_peered":"2023-10-10T20:26:14.808760+0000","last_clean":"2023-10-10T20:26:14.808760+0000","last_became_active":"2023-10-10T20:26:13.810029+0000","last_became_peered":"2023-10-10T20:26:13.810029+0000","last_unstale":"2023-10-10T20:26:14.808760+0000","last_undegraded":"2023-10-10T20:26:14.808760+0000","last_fullsized":"2023-10-10T20:26:14.808760+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:57.864466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820066+0000","last_change":"2023-10-10T20:26:14.819892+0000","last_active":"2023-10-10T20:26:14.820066+0000","last_peered":"2023-10-10T20:26:14.820066+0000","last_clean":"2023-10-10T20:26:14.820066+0000","last_became_active":"2023-10-10T20:26:14.818432+0000","last_became_peered":"2023-10-10T20:26:14.818432+0000","last_unstale":"2023-10-10T20:26:14.820066+0000","last_undegraded":"2023-10-10T20:26:14.820066+0000","last_fullsized":"2023-10-10T20:26:14.820066+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:53:07.893203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"145'3","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819338+0000","last_change":"2023-10-10T20:26:13.809553+0000","last_active":"2023-10-10T20:26:14.819338+0000","last_peered":"2023-10-10T20:26:14.819338+0000","last_clean":"2023-10-10T20:26:14.819338+0000","last_became_active":"2023-10-10T20:26:13.809254+0000","last_became_peered":"2023-10-10T20:26:13.809254+0000","last_unstale":"2023-10-10T20:26:14.819338+0000","last_undegraded":"2023-10-10T20:26:14.819338+0000","last_fullsized":"2023-10-10T20:26:14.819338+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:23.783432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818239+0000","last_change":"2023-10-10T20:26:14.817395+0000","last_active":"2023-10-10T20:26:14.818239+0000","last_peered":"2023-10-10T20:26:14.818239+0000","last_clean":"2023-10-10T20:26:14.818239+0000","last_became_active":"2023-10-10T20:26:14.817171+0000","last_became_peered":"2023-10-10T20:26:14.817171+0000","last_unstale":"2023-10-10T20:26:14.818239+0000","last_undegraded":"2023-10-10T20:26:14.818239+0000","last_fullsized":"2023-10-10T20:26:14.818239+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:51.663471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"145'6","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.824066+0000","last_change":"2023-10-10T20:26:13.811194+0000","last_active":"2023-10-10T20:26:14.824066+0000","last_peered":"2023-10-10T20:26:14.824066+0000","last_clean":"2023-10-10T20:26:14.824066+0000","last_became_active":"2023-10-10T20:26:13.810989+0000","last_became_peered":"2023-10-10T20:26:13.810989+0000","last_unstale":"2023-10-10T20:26:14.824066+0000","last_undegraded":"2023-10-10T20:26:14.824066+0000","last_fullsized":"2023-10-10T20:26:14.824066+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:34.422577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810206+0000","last_change":"2023-10-10T20:26:14.809889+0000","last_active":"2023-10-10T20:26:14.810206+0000","last_peered":"2023-10-10T20:26:14.810206+0000","last_clean":"2023-10-10T20:26:14.810206+0000","last_became_active":"2023-10-10T20:26:14.809715+0000","last_became_peered":"2023-10-10T20:26:14.809715+0000","last_unstale":"2023-10-10T20:26:14.810206+0000","last_undegraded":"2023-10-10T20:26:14.810206+0000","last_fullsized":"2023-10-10T20:26:14.810206+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:25.394848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804361+0000","last_change":"2023-10-10T20:26:13.812264+0000","last_active":"2023-10-10T20:26:14.804361+0000","last_peered":"2023-10-10T20:26:14.804361+0000","last_clean":"2023-10-10T20:26:14.804361+0000","last_became_active":"2023-10-10T20:26:13.812018+0000","last_became_peered":"2023-10-10T20:26:13.812018+0000","last_unstale":"2023-10-10T20:26:14.804361+0000","last_undegraded":"2023-10-10T20:26:14.804361+0000","last_fullsized":"2023-10-10T20:26:14.804361+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:49:12.107043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808139+0000","last_change":"2023-10-10T20:26:14.808030+0000","last_active":"2023-10-10T20:26:14.808139+0000","last_peered":"2023-10-10T20:26:14.808139+0000","last_clean":"2023-10-10T20:26:14.808139+0000","last_became_active":"2023-10-10T20:26:14.807768+0000","last_became_peered":"2023-10-10T20:26:14.807768+0000","last_unstale":"2023-10-10T20:26:14.808139+0000","last_undegraded":"2023-10-10T20:26:14.808139+0000","last_fullsized":"2023-10-10T20:26:14.808139+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:09.165136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801234+0000","last_change":"2023-10-10T20:26:13.811090+0000","last_active":"2023-10-10T20:26:14.801234+0000","last_peered":"2023-10-10T20:26:14.801234+0000","last_clean":"2023-10-10T20:26:14.801234+0000","last_became_active":"2023-10-10T20:26:13.810474+0000","last_became_peered":"2023-10-10T20:26:13.810474+0000","last_unstale":"2023-10-10T20:26:14.801234+0000","last_undegraded":"2023-10-10T20:26:14.801234+0000","last_fullsized":"2023-10-10T20:26:14.801234+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:21.337029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804397+0000","last_change":"2023-10-10T20:26:14.804198+0000","last_active":"2023-10-10T20:26:14.804397+0000","last_peered":"2023-10-10T20:26:14.804397+0000","last_clean":"2023-10-10T20:26:14.804397+0000","last_became_active":"2023-10-10T20:26:14.803958+0000","last_became_peered":"2023-10-10T20:26:14.803958+0000","last_unstale":"2023-10-10T20:26:14.804397+0000","last_undegraded":"2023-10-10T20:26:14.804397+0000","last_fullsized":"2023-10-10T20:26:14.804397+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:22.908780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"145'8","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.821324+0000","last_change":"2023-10-10T20:26:13.810023+0000","last_active":"2023-10-10T20:26:14.821324+0000","last_peered":"2023-10-10T20:26:14.821324+0000","last_clean":"2023-10-10T20:26:14.821324+0000","last_became_active":"2023-10-10T20:26:13.809762+0000","last_became_peered":"2023-10-10T20:26:13.809762+0000","last_unstale":"2023-10-10T20:26:14.821324+0000","last_undegraded":"2023-10-10T20:26:14.821324+0000","last_fullsized":"2023-10-10T20:26:14.821324+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:51.070132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809883+0000","last_change":"2023-10-10T20:26:14.809806+0000","last_active":"2023-10-10T20:26:14.809883+0000","last_peered":"2023-10-10T20:26:14.809883+0000","last_clean":"2023-10-10T20:26:14.809883+0000","last_became_active":"2023-10-10T20:26:14.809576+0000","last_became_peered":"2023-10-10T20:26:14.809576+0000","last_unstale":"2023-10-10T20:26:14.809883+0000","last_undegraded":"2023-10-10T20:26:14.809883+0000","last_fullsized":"2023-10-10T20:26:14.809883+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:59.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820592+0000","last_change":"2023-10-10T20:26:13.809260+0000","last_active":"2023-10-10T20:26:14.820592+0000","last_peered":"2023-10-10T20:26:14.820592+0000","last_clean":"2023-10-10T20:26:14.820592+0000","last_became_active":"2023-10-10T20:26:13.808950+0000","last_became_peered":"2023-10-10T20:26:13.808950+0000","last_unstale":"2023-10-10T20:26:14.820592+0000","last_undegraded":"2023-10-10T20:26:14.820592+0000","last_fullsized":"2023-10-10T20:26:14.820592+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.687349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819721+0000","last_change":"2023-10-10T20:26:14.819646+0000","last_active":"2023-10-10T20:26:14.819721+0000","last_peered":"2023-10-10T20:26:14.819721+0000","last_clean":"2023-10-10T20:26:14.819721+0000","last_became_active":"2023-10-10T20:26:14.819063+0000","last_became_peered":"2023-10-10T20:26:14.819063+0000","last_unstale":"2023-10-10T20:26:14.819721+0000","last_undegraded":"2023-10-10T20:26:14.819721+0000","last_fullsized":"2023-10-10T20:26:14.819721+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:15.442789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818475+0000","last_change":"2023-10-10T20:26:13.806539+0000","last_active":"2023-10-10T20:26:14.818475+0000","last_peered":"2023-10-10T20:26:14.818475+0000","last_clean":"2023-10-10T20:26:14.818475+0000","last_became_active":"2023-10-10T20:26:13.806299+0000","last_became_peered":"2023-10-10T20:26:13.806299+0000","last_unstale":"2023-10-10T20:26:14.818475+0000","last_undegraded":"2023-10-10T20:26:14.818475+0000","last_fullsized":"2023-10-10T20:26:14.818475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:33.034655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809818+0000","last_change":"2023-10-10T20:26:14.809669+0000","last_active":"2023-10-10T20:26:14.809818+0000","last_peered":"2023-10-10T20:26:14.809818+0000","last_clean":"2023-10-10T20:26:14.809818+0000","last_became_active":"2023-10-10T20:26:14.809389+0000","last_became_peered":"2023-10-10T20:26:14.809389+0000","last_unstale":"2023-10-10T20:26:14.809818+0000","last_undegraded":"2023-10-10T20:26:14.809818+0000","last_fullsized":"2023-10-10T20:26:14.809818+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:47.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.817587+0000","last_change":"2023-10-10T20:26:13.810263+0000","last_active":"2023-10-10T20:26:14.817587+0000","last_peered":"2023-10-10T20:26:14.817587+0000","last_clean":"2023-10-10T20:26:14.817587+0000","last_became_active":"2023-10-10T20:26:13.809885+0000","last_became_peered":"2023-10-10T20:26:13.809885+0000","last_unstale":"2023-10-10T20:26:14.817587+0000","last_undegraded":"2023-10-10T20:26:14.817587+0000","last_fullsized":"2023-10-10T20:26:14.817587+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:11.711139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814091+0000","last_change":"2023-10-10T20:26:14.813086+0000","last_active":"2023-10-10T20:26:14.814091+0000","last_peered":"2023-10-10T20:26:14.814091+0000","last_clean":"2023-10-10T20:26:14.814091+0000","last_became_active":"2023-10-10T20:26:14.812187+0000","last_became_peered":"2023-10-10T20:26:14.812187+0000","last_unstale":"2023-10-10T20:26:14.814091+0000","last_undegraded":"2023-10-10T20:26:14.814091+0000","last_fullsized":"2023-10-10T20:26:14.814091+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:52.079548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801913+0000","last_change":"2023-10-10T20:26:13.804815+0000","last_active":"2023-10-10T20:26:14.801913+0000","last_peered":"2023-10-10T20:26:14.801913+0000","last_clean":"2023-10-10T20:26:14.801913+0000","last_became_active":"2023-10-10T20:26:13.804574+0000","last_became_peered":"2023-10-10T20:26:13.804574+0000","last_unstale":"2023-10-10T20:26:14.801913+0000","last_undegraded":"2023-10-10T20:26:14.801913+0000","last_fullsized":"2023-10-10T20:26:14.801913+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:38.929755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.821700+0000","last_change":"2023-10-10T20:26:14.821633+0000","last_active":"2023-10-10T20:26:14.821700+0000","last_peered":"2023-10-10T20:26:14.821700+0000","last_clean":"2023-10-10T20:26:14.821700+0000","last_became_active":"2023-10-10T20:26:14.821054+0000","last_became_peered":"2023-10-10T20:26:14.821054+0000","last_unstale":"2023-10-10T20:26:14.821700+0000","last_undegraded":"2023-10-10T20:26:14.821700+0000","last_fullsized":"2023-10-10T20:26:14.821700+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:42.782562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819546+0000","last_change":"2023-10-10T20:26:13.804323+0000","last_active":"2023-10-10T20:26:14.819546+0000","last_peered":"2023-10-10T20:26:14.819546+0000","last_clean":"2023-10-10T20:26:14.819546+0000","last_became_active":"2023-10-10T20:26:13.803498+0000","last_became_peered":"2023-10-10T20:26:13.803498+0000","last_unstale":"2023-10-10T20:26:14.819546+0000","last_undegraded":"2023-10-10T20:26:14.819546+0000","last_fullsized":"2023-10-10T20:26:14.819546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:41.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.807859+0000","last_change":"2023-10-10T20:26:14.807779+0000","last_active":"2023-10-10T20:26:14.807859+0000","last_peered":"2023-10-10T20:26:14.807859+0000","last_clean":"2023-10-10T20:26:14.807859+0000","last_became_active":"2023-10-10T20:26:14.807518+0000","last_became_peered":"2023-10-10T20:26:14.807518+0000","last_unstale":"2023-10-10T20:26:14.807859+0000","last_undegraded":"2023-10-10T20:26:14.807859+0000","last_fullsized":"2023-10-10T20:26:14.807859+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:39.824997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.806260+0000","last_change":"2023-10-10T20:26:13.809050+0000","last_active":"2023-10-10T20:26:14.806260+0000","last_peered":"2023-10-10T20:26:14.806260+0000","last_clean":"2023-10-10T20:26:14.806260+0000","last_became_active":"2023-10-10T20:26:13.808776+0000","last_became_peered":"2023-10-10T20:26:13.808776+0000","last_unstale":"2023-10-10T20:26:14.806260+0000","last_undegraded":"2023-10-10T20:26:14.806260+0000","last_fullsized":"2023-10-10T20:26:14.806260+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:58.314132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814109+0000","last_change":"2023-10-10T20:26:14.813239+0000","last_active":"2023-10-10T20:26:14.814109+0000","last_peered":"2023-10-10T20:26:14.814109+0000","last_clean":"2023-10-10T20:26:14.814109+0000","last_became_active":"2023-10-10T20:26:14.812862+0000","last_became_peered":"2023-10-10T20:26:14.812862+0000","last_unstale":"2023-10-10T20:26:14.814109+0000","last_undegraded":"2023-10-10T20:26:14.814109+0000","last_fullsized":"2023-10-10T20:26:14.814109+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:26.787393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805805+0000","last_change":"2023-10-10T20:26:13.809813+0000","last_active":"2023-10-10T20:26:14.805805+0000","last_peered":"2023-10-10T20:26:14.805805+0000","last_clean":"2023-10-10T20:26:14.805805+0000","last_became_active":"2023-10-10T20:26:13.809547+0000","last_became_peered":"2023-10-10T20:26:13.809547+0000","last_unstale":"2023-10-10T20:26:14.805805+0000","last_undegraded":"2023-10-10T20:26:14.805805+0000","last_fullsized":"2023-10-10T20:26:14.805805+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:37.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818208+0000","last_change":"2023-10-10T20:26:14.816799+0000","last_active":"2023-10-10T20:26:14.818208+0000","last_peered":"2023-10-10T20:26:14.818208+0000","last_clean":"2023-10-10T20:26:14.818208+0000","last_became_active":"2023-10-10T20:26:14.816388+0000","last_became_peered":"2023-10-10T20:26:14.816388+0000","last_unstale":"2023-10-10T20:26:14.818208+0000","last_undegraded":"2023-10-10T20:26:14.818208+0000","last_fullsized":"2023-10-10T20:26:14.818208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:46:06.212765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808977+0000","last_change":"2023-10-10T20:26:13.806628+0000","last_active":"2023-10-10T20:26:14.808977+0000","last_peered":"2023-10-10T20:26:14.808977+0000","last_clean":"2023-10-10T20:26:14.808977+0000","last_became_active":"2023-10-10T20:26:13.806422+0000","last_became_peered":"2023-10-10T20:26:13.806422+0000","last_unstale":"2023-10-10T20:26:14.808977+0000","last_undegraded":"2023-10-10T20:26:14.808977+0000","last_fullsized":"2023-10-10T20:26:14.808977+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:46.027821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810733+0000","last_change":"2023-10-10T20:26:14.810516+0000","last_active":"2023-10-10T20:26:14.810733+0000","last_peered":"2023-10-10T20:26:14.810733+0000","last_clean":"2023-10-10T20:26:14.810733+0000","last_became_active":"2023-10-10T20:26:14.810213+0000","last_became_peered":"2023-10-10T20:26:14.810213+0000","last_unstale":"2023-10-10T20:26:14.810733+0000","last_undegraded":"2023-10-10T20:26:14.810733+0000","last_fullsized":"2023-10-10T20:26:14.810733+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:29.789396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819595+0000","last_change":"2023-10-10T20:26:13.804807+0000","last_active":"2023-10-10T20:26:14.819595+0000","last_peered":"2023-10-10T20:26:14.819595+0000","last_clean":"2023-10-10T20:26:14.819595+0000","last_became_active":"2023-10-10T20:26:13.804594+0000","last_became_peered":"2023-10-10T20:26:13.804594+0000","last_unstale":"2023-10-10T20:26:14.819595+0000","last_undegraded":"2023-10-10T20:26:14.819595+0000","last_fullsized":"2023-10-10T20:26:14.819595+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:01.844525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813921+0000","last_change":"2023-10-10T20:26:14.813358+0000","last_active":"2023-10-10T20:26:14.813921+0000","last_peered":"2023-10-10T20:26:14.813921+0000","last_clean":"2023-10-10T20:26:14.813921+0000","last_became_active":"2023-10-10T20:26:14.812558+0000","last_became_peered":"2023-10-10T20:26:14.812558+0000","last_unstale":"2023-10-10T20:26:14.813921+0000","last_undegraded":"2023-10-10T20:26:14.813921+0000","last_fullsized":"2023-10-10T20:26:14.813921+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:50:35.923557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809407+0000","last_change":"2023-10-10T20:26:13.813589+0000","last_active":"2023-10-10T20:26:14.809407+0000","last_peered":"2023-10-10T20:26:14.809407+0000","last_clean":"2023-10-10T20:26:14.809407+0000","last_became_active":"2023-10-10T20:26:13.811357+0000","last_became_peered":"2023-10-10T20:26:13.811357+0000","last_unstale":"2023-10-10T20:26:14.809407+0000","last_undegraded":"2023-10-10T20:26:14.809407+0000","last_fullsized":"2023-10-10T20:26:14.809407+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:38.815580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822173+0000","last_change":"2023-10-10T20:26:14.821951+0000","last_active":"2023-10-10T20:26:14.822173+0000","last_peered":"2023-10-10T20:26:14.822173+0000","last_clean":"2023-10-10T20:26:14.822173+0000","last_became_active":"2023-10-10T20:26:14.817332+0000","last_became_peered":"2023-10-10T20:26:14.817332+0000","last_unstale":"2023-10-10T20:26:14.822173+0000","last_undegraded":"2023-10-10T20:26:14.822173+0000","last_fullsized":"2023-10-10T20:26:14.822173+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:28.375113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801894+0000","last_change":"2023-10-10T20:26:13.807314+0000","last_active":"2023-10-10T20:26:14.801894+0000","last_peered":"2023-10-10T20:26:14.801894+0000","last_clean":"2023-10-10T20:26:14.801894+0000","last_became_active":"2023-10-10T20:26:13.807065+0000","last_became_peered":"2023-10-10T20:26:13.807065+0000","last_unstale":"2023-10-10T20:26:14.801894+0000","last_undegraded":"2023-10-10T20:26:14.801894+0000","last_fullsized":"2023-10-10T20:26:14.801894+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:26.134402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819951+0000","last_change":"2023-10-10T20:26:14.819706+0000","last_active":"2023-10-10T20:26:14.819951+0000","last_peered":"2023-10-10T20:26:14.819951+0000","last_clean":"2023-10-10T20:26:14.819951+0000","last_became_active":"2023-10-10T20:26:14.819431+0000","last_became_peered":"2023-10-10T20:26:14.819431+0000","last_unstale":"2023-10-10T20:26:14.819951+0000","last_undegraded":"2023-10-10T20:26:14.819951+0000","last_fullsized":"2023-10-10T20:26:14.819951+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:25.171860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804803+0000","last_change":"2023-10-10T20:26:13.810975+0000","last_active":"2023-10-10T20:26:14.804803+0000","last_peered":"2023-10-10T20:26:14.804803+0000","last_clean":"2023-10-10T20:26:14.804803+0000","last_became_active":"2023-10-10T20:26:13.810748+0000","last_became_peered":"2023-10-10T20:26:13.810748+0000","last_unstale":"2023-10-10T20:26:14.804803+0000","last_undegraded":"2023-10-10T20:26:14.804803+0000","last_fullsized":"2023-10-10T20:26:14.804803+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:36.907520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814281+0000","last_change":"2023-10-10T20:26:14.813772+0000","last_active":"2023-10-10T20:26:14.814281+0000","last_peered":"2023-10-10T20:26:14.814281+0000","last_clean":"2023-10-10T20:26:14.814281+0000","last_became_active":"2023-10-10T20:26:14.813543+0000","last_became_peered":"2023-10-10T20:26:14.813543+0000","last_unstale":"2023-10-10T20:26:14.814281+0000","last_undegraded":"2023-10-10T20:26:14.814281+0000","last_fullsized":"2023-10-10T20:26:14.814281+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:49.792681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801600+0000","last_change":"2023-10-10T20:26:13.813643+0000","last_active":"2023-10-10T20:26:14.801600+0000","last_peered":"2023-10-10T20:26:14.801600+0000","last_clean":"2023-10-10T20:26:14.801600+0000","last_became_active":"2023-10-10T20:26:13.812956+0000","last_became_peered":"2023-10-10T20:26:13.812956+0000","last_unstale":"2023-10-10T20:26:14.801600+0000","last_undegraded":"2023-10-10T20:26:14.801600+0000","last_fullsized":"2023-10-10T20:26:14.801600+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:35.844291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.807278+0000","last_change":"2023-10-10T20:26:14.806934+0000","last_active":"2023-10-10T20:26:14.807278+0000","last_peered":"2023-10-10T20:26:14.807278+0000","last_clean":"2023-10-10T20:26:14.807278+0000","last_became_active":"2023-10-10T20:26:14.806709+0000","last_became_peered":"2023-10-10T20:26:14.806709+0000","last_unstale":"2023-10-10T20:26:14.807278+0000","last_undegraded":"2023-10-10T20:26:14.807278+0000","last_fullsized":"2023-10-10T20:26:14.807278+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:10:26.395044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809343+0000","last_change":"2023-10-10T20:26:13.811870+0000","last_active":"2023-10-10T20:26:14.809343+0000","last_peered":"2023-10-10T20:26:14.809343+0000","last_clean":"2023-10-10T20:26:14.809343+0000","last_became_active":"2023-10-10T20:26:13.811616+0000","last_became_peered":"2023-10-10T20:26:13.811616+0000","last_unstale":"2023-10-10T20:26:14.809343+0000","last_undegraded":"2023-10-10T20:26:14.809343+0000","last_fullsized":"2023-10-10T20:26:14.809343+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:35.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815448+0000","last_change":"2023-10-10T20:26:14.815402+0000","last_active":"2023-10-10T20:26:14.815448+0000","last_peered":"2023-10-10T20:26:14.815448+0000","last_clean":"2023-10-10T20:26:14.815448+0000","last_became_active":"2023-10-10T20:26:14.815144+0000","last_became_peered":"2023-10-10T20:26:14.815144+0000","last_unstale":"2023-10-10T20:26:14.815448+0000","last_undegraded":"2023-10-10T20:26:14.815448+0000","last_fullsized":"2023-10-10T20:26:14.815448+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:51.663471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804862+0000","last_change":"2023-10-10T20:26:13.811310+0000","last_active":"2023-10-10T20:26:14.804862+0000","last_peered":"2023-10-10T20:26:14.804862+0000","last_clean":"2023-10-10T20:26:14.804862+0000","last_became_active":"2023-10-10T20:26:13.810664+0000","last_became_peered":"2023-10-10T20:26:13.810664+0000","last_unstale":"2023-10-10T20:26:14.804862+0000","last_undegraded":"2023-10-10T20:26:14.804862+0000","last_fullsized":"2023-10-10T20:26:14.804862+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:24.162443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823052+0000","last_change":"2023-10-10T20:26:14.822989+0000","last_active":"2023-10-10T20:26:14.823052+0000","last_peered":"2023-10-10T20:26:14.823052+0000","last_clean":"2023-10-10T20:26:14.823052+0000","last_became_active":"2023-10-10T20:26:14.822714+0000","last_became_peered":"2023-10-10T20:26:14.822714+0000","last_unstale":"2023-10-10T20:26:14.823052+0000","last_undegraded":"2023-10-10T20:26:14.823052+0000","last_fullsized":"2023-10-10T20:26:14.823052+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:56.760999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804462+0000","last_change":"2023-10-10T20:26:13.813285+0000","last_active":"2023-10-10T20:26:14.804462+0000","last_peered":"2023-10-10T20:26:14.804462+0000","last_clean":"2023-10-10T20:26:14.804462+0000","last_became_active":"2023-10-10T20:26:13.813065+0000","last_became_peered":"2023-10-10T20:26:13.813065+0000","last_unstale":"2023-10-10T20:26:14.804462+0000","last_undegraded":"2023-10-10T20:26:14.804462+0000","last_fullsized":"2023-10-10T20:26:14.804462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:07.466438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820438+0000","last_change":"2023-10-10T20:26:14.820367+0000","last_active":"2023-10-10T20:26:14.820438+0000","last_peered":"2023-10-10T20:26:14.820438+0000","last_clean":"2023-10-10T20:26:14.820438+0000","last_became_active":"2023-10-10T20:26:14.820065+0000","last_became_peered":"2023-10-10T20:26:14.820065+0000","last_unstale":"2023-10-10T20:26:14.820438+0000","last_undegraded":"2023-10-10T20:26:14.820438+0000","last_fullsized":"2023-10-10T20:26:14.820438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:11:25.506245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820340+0000","last_change":"2023-10-10T20:26:13.812930+0000","last_active":"2023-10-10T20:26:14.820340+0000","last_peered":"2023-10-10T20:26:14.820340+0000","last_clean":"2023-10-10T20:26:14.820340+0000","last_became_active":"2023-10-10T20:26:13.812718+0000","last_became_peered":"2023-10-10T20:26:13.812718+0000","last_unstale":"2023-10-10T20:26:14.820340+0000","last_undegraded":"2023-10-10T20:26:14.820340+0000","last_fullsized":"2023-10-10T20:26:14.820340+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:37.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":275,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808011+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:26:14.808011+0000","last_peered":"2023-10-10T20:26:14.808011+0000","last_clean":"2023-10-10T20:26:14.808011+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:26:14.808011+0000","last_undegraded":"2023-10-10T20:26:14.808011+0000","last_fullsized":"2023-10-10T20:26:14.808011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1366,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193946,"statfs":{"total":200000000,"available":198601063,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70299999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.378},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.74199999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.626}]},{"osd":5,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.51000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199192041,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.53500000000000003},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":1.865,"15min":1.865},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64200000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72799999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69099999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71499999999999997,"5min":15.862,"15min":15.862},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45300000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.62,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.46200000000000002},"min":{"1min":0.28000000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.50700000000000001,"15min":1.6100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.38300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":1.647},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38700000000000001},"min":{"1min":0.24199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.627,"15min":1.524},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.58799999999999997,"15min":1.3959999999999999},"last":0.313}]},{"osd":3,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38800000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":1.4470000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63,"15min":1.673},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.90900000000000003,"15min":1.633},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.506},"last":0.42999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.696},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.9359999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.97499999999999998,"15min":1.7490000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.872,"15min":1.585},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.478},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.63},"min":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42299999999999999},"max":{"1min":0.75900000000000001,"5min":0.91000000000000003,"15min":1.722},"last":0.52900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1363,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.44500000000000001},"min":{"1min":0.216,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.439},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.435,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58099999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.47399999999999998},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52400000000000002,"5min":1.647,"15min":2.4830000000000001},"last":0.497},{"interface":"front","average":{"1min":0.375,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":5.891,"15min":5.891},"last":0.45400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.433},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.57999999999999996,"5min":1.462,"15min":1.462},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.92600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77800000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":5.758,"15min":5.758},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.79200000000000004,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.64600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194525,"statfs":{"total":200000000,"available":199194000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.38600000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.54200000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53100000000000003,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.22},"max":{"1min":0.42299999999999999,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.48899999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.379}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.50600000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.49399999999999999,"5min":11.815,"15min":11.815},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.56699999999999995,"15min":0.46300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.47899999999999998,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.29299999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.32300000000000001,"15min":0.28100000000000003},"max":{"1min":0.82899999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70199999999999996,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":6.399,"15min":6.399},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.66700000000000004,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.623},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75900000000000001,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.745}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.67300000000000004,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193417,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.30299999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.34200000000000003,"15min":0.315},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.44,"5min":0.57099999999999995,"15min":0.65600000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.55200000000000005,"15min":0.67900000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53700000000000003,"5min":0.63200000000000001,"15min":0.751},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.25,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.58199999999999996,"5min":0.621,"15min":0.74099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.63500000000000001,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.35599999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85099999999999998,"5min":0.871,"15min":0.871},"last":0.80600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.496},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.41299999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73999999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.49399999999999999},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.627}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"max":{"1min":0.92600000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.35399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.91200000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.79900000000000004},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.35399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.72599999999999998,"5min":0.83599999999999997,"15min":0.89000000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.78000000000000003,"15min":0.96299999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.36499999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.434},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.752,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.56299999999999994}]}]}],"pool_statfs":[]}} 2023-10-10T20:26:18.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:26:19.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.022+0000 7f383ec9f700 1 Processor -- start 2023-10-10T20:26:19.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.022+0000 7f383ec9f700 1 -- start start 2023-10-10T20:26:19.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383ec9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f383815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383ec9f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380cb7c0 con 0x7f38380cd6e0 2023-10-10T20:26:19.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383ec9f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383815e290 con 0x7f38380cdab0 2023-10-10T20:26:19.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383ec9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383815e850 con 0x7f383815cbd0 2023-10-10T20:26:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383dc9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f38380cd6e0 0x7f38380cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57754/0 (socket says 172.21.15.148:57754) 2023-10-10T20:26:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383dc9d700 1 -- 172.21.15.148:0/345926617 learned_addr learned my addr 172.21.15.148:0/345926617 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383e49e700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f383815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485217045 0 0) 0x7f38380cb7c0 con 0x7f38380cd6e0 2023-10-10T20:26:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3820003570 con 0x7f38380cd6e0 2023-10-10T20:26:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464437823 0 0) 0x7f3820003570 con 0x7f38380cd6e0 2023-10-10T20:26:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 msgr2=0x7f383815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f383815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 >> v1:172.21.15.161:6789/0 conn(0x7f38380cdab0 legacy=0x7f383815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383815e570 con 0x7f38380cd6e0 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f3830001470 con 0x7f38380cd6e0 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 >> v1:172.21.15.148:6789/0 conn(0x7f38380cd6e0 legacy=0x7f38380cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3820003a90 0x7f3820003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f3820004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3820004d50 0x7f38200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383815e850 con 0x7f3820003a90 2023-10-10T20:26:19.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382000bb10 con 0x7f3820004d50 2023-10-10T20:26:19.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.023+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382000bcd0 con 0x7f383815cbd0 2023-10-10T20:26:19.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3820003a90 0x7f3820003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383e49e700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3820004d50 0x7f38200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383d49c700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f3820004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 -- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 msgr2=0x7f3820004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f383815cbd0 0x7f3820004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 -- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3820004d50 msgr2=0x7f38200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3820004d50 0x7f38200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 -- 172.21.15.148:0/345926617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3820003570 con 0x7f3820003a90 2023-10-10T20:26:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383e49e700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3820004d50 0x7f38200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383dc9d700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3820003a90 0x7f3820003e60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f383000c3c0 tx=0x7f3830001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60263b265b5335da server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000cc90 con 0x7f3820003a90 2023-10-10T20:26:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000cdf0 con 0x7f3820003a90 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383cc9b700 1 -- 172.21.15.148:0/345926617 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830005b70 con 0x7f3820003a90 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383ec9f700 1 -- 172.21.15.148:0/345926617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3820003a90 msgr2=0x7f3820003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383ec9f700 1 --2- 172.21.15.148:0/345926617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3820003a90 0x7f3820003e60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f383000c3c0 tx=0x7f3830001e70 comp rx=0 tx=0).stop 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.024+0000 7f383e49e700 1 -- 172.21.15.148:0/345926617 reap_dead start 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.025+0000 7f383ec9f700 1 -- 172.21.15.148:0/345926617 shutdown_connections 2023-10-10T20:26:19.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.025+0000 7f383ec9f700 1 -- 172.21.15.148:0/345926617 >> 172.21.15.148:0/345926617 conn(0x7f3838067660 msgr2=0x7f3838067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:19.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.025+0000 7f383ec9f700 1 -- 172.21.15.148:0/345926617 shutdown_connections 2023-10-10T20:26:19.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.025+0000 7f383ec9f700 1 -- 172.21.15.148:0/345926617 wait complete. 2023-10-10T20:26:19.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 Processor -- start 2023-10-10T20:26:19.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 -- start start 2023-10-10T20:26:19.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3820003a90 0x7f383814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383816ad50 0x7f383816b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380cc360 con 0x7f38380b5f10 2023-10-10T20:26:19.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380b5a80 con 0x7f383816ad50 2023-10-10T20:26:19.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.026+0000 7f383ec9f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380b4b90 con 0x7f3820003a90 2023-10-10T20:26:19.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46352/0 (socket says 172.21.15.148:46352) 2023-10-10T20:26:19.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 -- 172.21.15.148:0/2773523241 learned_addr learned my addr 172.21.15.148:0/2773523241 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:19.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383d49c700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383816ad50 0x7f383816b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383dc9d700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3820003a90 0x7f383814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 -- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3820003a90 msgr2=0x7f383814c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3820003a90 0x7f383814c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 -- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383816ad50 msgr2=0x7f383816b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383816ad50 0x7f383816b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 -- 172.21.15.148:0/2773523241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380b4e70 con 0x7f38380b5f10 2023-10-10T20:26:19.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383e49e700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3838089ac0 tx=0x7f382800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382801b070 con 0x7f38380b5f10 2023-10-10T20:26:19.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800f040 con 0x7f38380b5f10 2023-10-10T20:26:19.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38380b5150 con 0x7f38380b5f10 2023-10-10T20:26:19.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800e050 con 0x7f38380b5f10 2023-10-10T20:26:19.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.027+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38380b5710 con 0x7f38380b5f10 2023-10-10T20:26:19.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.028+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38380b58d0 con 0x7f38380b5f10 2023-10-10T20:26:19.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.030+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3828003020 con 0x7f38380b5f10 2023-10-10T20:26:19.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.030+0000 7f3835ffb700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.030+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3828085650 con 0x7f38380b5f10 2023-10-10T20:26:19.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.030+0000 7f383dc9d700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.030+0000 7f383dc9d700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3830001eb0 tx=0x7f3830015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.033+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38280528a0 con 0x7f38380b5f10 2023-10-10T20:26:19.214 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.212+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38380b5c30 con 0x7f3824066f30 2023-10-10T20:26:19.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.220+0000 7f3835ffb700 1 -- 172.21.15.148:0/2773523241 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285577 (secure 0 0 0) 0x7f38380b5c30 con 0x7f3824066f30 2023-10-10T20:26:19.225 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 msgr2=0x7f38240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3830001eb0 tx=0x7f3830015000 comp rx=0 tx=0).stop 2023-10-10T20:26:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 msgr2=0x7f38380b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3838089ac0 tx=0x7f382800b040 comp rx=0 tx=0).stop 2023-10-10T20:26:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 shutdown_connections 2023-10-10T20:26:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3824066f30 0x7f38240693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38380b5f10 0x7f38380b6380 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3820003a90 0x7f383814c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 --2- 172.21.15.148:0/2773523241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f383816ad50 0x7f383816b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 >> 172.21.15.148:0/2773523241 conn(0x7f3838067660 msgr2=0x7f383814d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 shutdown_connections 2023-10-10T20:26:19.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.223+0000 7f383ec9f700 1 -- 172.21.15.148:0/2773523241 wait complete. 2023-10-10T20:26:19.231 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:26:19.237 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-10-10T20:26:17.412243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7455,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555044,"statfs":{"total":1600000000,"available":1592365290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.618196"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813192+0000","last_change":"2023-10-10T20:26:14.813080+0000","last_active":"2023-10-10T20:26:14.813192+0000","last_peered":"2023-10-10T20:26:14.813192+0000","last_clean":"2023-10-10T20:26:14.813192+0000","last_became_active":"2023-10-10T20:26:14.812823+0000","last_became_peered":"2023-10-10T20:26:14.812823+0000","last_unstale":"2023-10-10T20:26:14.813192+0000","last_undegraded":"2023-10-10T20:26:14.813192+0000","last_fullsized":"2023-10-10T20:26:14.813192+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:22.346445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816665+0000","last_change":"2023-10-10T20:26:14.816612+0000","last_active":"2023-10-10T20:26:14.816665+0000","last_peered":"2023-10-10T20:26:14.816665+0000","last_clean":"2023-10-10T20:26:14.816665+0000","last_became_active":"2023-10-10T20:26:14.816302+0000","last_became_peered":"2023-10-10T20:26:14.816302+0000","last_unstale":"2023-10-10T20:26:14.816665+0000","last_undegraded":"2023-10-10T20:26:14.816665+0000","last_fullsized":"2023-10-10T20:26:14.816665+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:35.431993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812943+0000","last_change":"2023-10-10T20:26:14.812288+0000","last_active":"2023-10-10T20:26:14.812943+0000","last_peered":"2023-10-10T20:26:14.812943+0000","last_clean":"2023-10-10T20:26:14.812943+0000","last_became_active":"2023-10-10T20:26:14.811433+0000","last_became_peered":"2023-10-10T20:26:14.811433+0000","last_unstale":"2023-10-10T20:26:14.812943+0000","last_undegraded":"2023-10-10T20:26:14.812943+0000","last_fullsized":"2023-10-10T20:26:14.812943+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:29:09.386940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823098+0000","last_change":"2023-10-10T20:26:14.823010+0000","last_active":"2023-10-10T20:26:14.823098+0000","last_peered":"2023-10-10T20:26:14.823098+0000","last_clean":"2023-10-10T20:26:14.823098+0000","last_became_active":"2023-10-10T20:26:14.822777+0000","last_became_peered":"2023-10-10T20:26:14.822777+0000","last_unstale":"2023-10-10T20:26:14.823098+0000","last_undegraded":"2023-10-10T20:26:14.823098+0000","last_fullsized":"2023-10-10T20:26:14.823098+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:32:37.495784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818183+0000","last_change":"2023-10-10T20:26:14.816660+0000","last_active":"2023-10-10T20:26:14.818183+0000","last_peered":"2023-10-10T20:26:14.818183+0000","last_clean":"2023-10-10T20:26:14.818183+0000","last_became_active":"2023-10-10T20:26:14.816067+0000","last_became_peered":"2023-10-10T20:26:14.816067+0000","last_unstale":"2023-10-10T20:26:14.818183+0000","last_undegraded":"2023-10-10T20:26:14.818183+0000","last_fullsized":"2023-10-10T20:26:14.818183+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:32.634303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810090+0000","last_change":"2023-10-10T20:26:14.809822+0000","last_active":"2023-10-10T20:26:14.810090+0000","last_peered":"2023-10-10T20:26:14.810090+0000","last_clean":"2023-10-10T20:26:14.810090+0000","last_became_active":"2023-10-10T20:26:14.809569+0000","last_became_peered":"2023-10-10T20:26:14.809569+0000","last_unstale":"2023-10-10T20:26:14.810090+0000","last_undegraded":"2023-10-10T20:26:14.810090+0000","last_fullsized":"2023-10-10T20:26:14.810090+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:58.873883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810070+0000","last_change":"2023-10-10T20:26:14.809817+0000","last_active":"2023-10-10T20:26:14.810070+0000","last_peered":"2023-10-10T20:26:14.810070+0000","last_clean":"2023-10-10T20:26:14.810070+0000","last_became_active":"2023-10-10T20:26:14.809039+0000","last_became_peered":"2023-10-10T20:26:14.809039+0000","last_unstale":"2023-10-10T20:26:14.810070+0000","last_undegraded":"2023-10-10T20:26:14.810070+0000","last_fullsized":"2023-10-10T20:26:14.810070+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:36.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816416+0000","last_change":"2023-10-10T20:26:14.816310+0000","last_active":"2023-10-10T20:26:14.816416+0000","last_peered":"2023-10-10T20:26:14.816416+0000","last_clean":"2023-10-10T20:26:14.816416+0000","last_became_active":"2023-10-10T20:26:14.816074+0000","last_became_peered":"2023-10-10T20:26:14.816074+0000","last_unstale":"2023-10-10T20:26:14.816416+0000","last_undegraded":"2023-10-10T20:26:14.816416+0000","last_fullsized":"2023-10-10T20:26:14.816416+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:16:11.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805230+0000","last_change":"2023-10-10T20:26:14.805153+0000","last_active":"2023-10-10T20:26:14.805230+0000","last_peered":"2023-10-10T20:26:14.805230+0000","last_clean":"2023-10-10T20:26:14.805230+0000","last_became_active":"2023-10-10T20:26:14.804899+0000","last_became_peered":"2023-10-10T20:26:14.804899+0000","last_unstale":"2023-10-10T20:26:14.805230+0000","last_undegraded":"2023-10-10T20:26:14.805230+0000","last_fullsized":"2023-10-10T20:26:14.805230+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:48.579836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809144+0000","last_change":"2023-10-10T20:26:14.808823+0000","last_active":"2023-10-10T20:26:14.809144+0000","last_peered":"2023-10-10T20:26:14.809144+0000","last_clean":"2023-10-10T20:26:14.809144+0000","last_became_active":"2023-10-10T20:26:14.808602+0000","last_became_peered":"2023-10-10T20:26:14.808602+0000","last_unstale":"2023-10-10T20:26:14.809144+0000","last_undegraded":"2023-10-10T20:26:14.809144+0000","last_fullsized":"2023-10-10T20:26:14.809144+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:40:58.586866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.817749+0000","last_change":"2023-10-10T20:26:14.817633+0000","last_active":"2023-10-10T20:26:14.817749+0000","last_peered":"2023-10-10T20:26:14.817749+0000","last_clean":"2023-10-10T20:26:14.817749+0000","last_became_active":"2023-10-10T20:26:14.817446+0000","last_became_peered":"2023-10-10T20:26:14.817446+0000","last_unstale":"2023-10-10T20:26:14.817749+0000","last_undegraded":"2023-10-10T20:26:14.817749+0000","last_fullsized":"2023-10-10T20:26:14.817749+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:24.406069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.806247+0000","last_change":"2023-10-10T20:26:14.805724+0000","last_active":"2023-10-10T20:26:14.806247+0000","last_peered":"2023-10-10T20:26:14.806247+0000","last_clean":"2023-10-10T20:26:14.806247+0000","last_became_active":"2023-10-10T20:26:14.805182+0000","last_became_peered":"2023-10-10T20:26:14.805182+0000","last_unstale":"2023-10-10T20:26:14.806247+0000","last_undegraded":"2023-10-10T20:26:14.806247+0000","last_fullsized":"2023-10-10T20:26:14.806247+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:36.261230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815595+0000","last_change":"2023-10-10T20:26:14.814133+0000","last_active":"2023-10-10T20:26:14.815595+0000","last_peered":"2023-10-10T20:26:14.815595+0000","last_clean":"2023-10-10T20:26:14.815595+0000","last_became_active":"2023-10-10T20:26:14.813910+0000","last_became_peered":"2023-10-10T20:26:14.813910+0000","last_unstale":"2023-10-10T20:26:14.815595+0000","last_undegraded":"2023-10-10T20:26:14.815595+0000","last_fullsized":"2023-10-10T20:26:14.815595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:04.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819501+0000","last_change":"2023-10-10T20:26:14.819349+0000","last_active":"2023-10-10T20:26:14.819501+0000","last_peered":"2023-10-10T20:26:14.819501+0000","last_clean":"2023-10-10T20:26:14.819501+0000","last_became_active":"2023-10-10T20:26:14.819067+0000","last_became_peered":"2023-10-10T20:26:14.819067+0000","last_unstale":"2023-10-10T20:26:14.819501+0000","last_undegraded":"2023-10-10T20:26:14.819501+0000","last_fullsized":"2023-10-10T20:26:14.819501+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:12.720556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815626+0000","last_change":"2023-10-10T20:26:14.814642+0000","last_active":"2023-10-10T20:26:14.815626+0000","last_peered":"2023-10-10T20:26:14.815626+0000","last_clean":"2023-10-10T20:26:14.815626+0000","last_became_active":"2023-10-10T20:26:14.813109+0000","last_became_peered":"2023-10-10T20:26:14.813109+0000","last_unstale":"2023-10-10T20:26:14.815626+0000","last_undegraded":"2023-10-10T20:26:14.815626+0000","last_fullsized":"2023-10-10T20:26:14.815626+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:05.424456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816446+0000","last_change":"2023-10-10T20:26:14.816383+0000","last_active":"2023-10-10T20:26:14.816446+0000","last_peered":"2023-10-10T20:26:14.816446+0000","last_clean":"2023-10-10T20:26:14.816446+0000","last_became_active":"2023-10-10T20:26:14.815920+0000","last_became_peered":"2023-10-10T20:26:14.815920+0000","last_unstale":"2023-10-10T20:26:14.816446+0000","last_undegraded":"2023-10-10T20:26:14.816446+0000","last_fullsized":"2023-10-10T20:26:14.816446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:11:25.506245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818759+0000","last_change":"2023-10-10T20:26:14.818559+0000","last_active":"2023-10-10T20:26:14.818759+0000","last_peered":"2023-10-10T20:26:14.818759+0000","last_clean":"2023-10-10T20:26:14.818759+0000","last_became_active":"2023-10-10T20:26:14.818320+0000","last_became_peered":"2023-10-10T20:26:14.818320+0000","last_unstale":"2023-10-10T20:26:14.818759+0000","last_undegraded":"2023-10-10T20:26:14.818759+0000","last_fullsized":"2023-10-10T20:26:14.818759+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:50.826456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813013+0000","last_change":"2023-10-10T20:26:14.812936+0000","last_active":"2023-10-10T20:26:14.813013+0000","last_peered":"2023-10-10T20:26:14.813013+0000","last_clean":"2023-10-10T20:26:14.813013+0000","last_became_active":"2023-10-10T20:26:14.812392+0000","last_became_peered":"2023-10-10T20:26:14.812392+0000","last_unstale":"2023-10-10T20:26:14.813013+0000","last_undegraded":"2023-10-10T20:26:14.813013+0000","last_fullsized":"2023-10-10T20:26:14.813013+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:28.152057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816510+0000","last_change":"2023-10-10T20:26:14.814552+0000","last_active":"2023-10-10T20:26:14.816510+0000","last_peered":"2023-10-10T20:26:14.816510+0000","last_clean":"2023-10-10T20:26:14.816510+0000","last_became_active":"2023-10-10T20:26:14.814013+0000","last_became_peered":"2023-10-10T20:26:14.814013+0000","last_unstale":"2023-10-10T20:26:14.816510+0000","last_undegraded":"2023-10-10T20:26:14.816510+0000","last_fullsized":"2023-10-10T20:26:14.816510+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:13:55.273265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815432+0000","last_change":"2023-10-10T20:26:14.814992+0000","last_active":"2023-10-10T20:26:14.815432+0000","last_peered":"2023-10-10T20:26:14.815432+0000","last_clean":"2023-10-10T20:26:14.815432+0000","last_became_active":"2023-10-10T20:26:14.814777+0000","last_became_peered":"2023-10-10T20:26:14.814777+0000","last_unstale":"2023-10-10T20:26:14.815432+0000","last_undegraded":"2023-10-10T20:26:14.815432+0000","last_fullsized":"2023-10-10T20:26:14.815432+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:59:02.788705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815297+0000","last_change":"2023-10-10T20:26:14.814843+0000","last_active":"2023-10-10T20:26:14.815297+0000","last_peered":"2023-10-10T20:26:14.815297+0000","last_clean":"2023-10-10T20:26:14.815297+0000","last_became_active":"2023-10-10T20:26:14.814615+0000","last_became_peered":"2023-10-10T20:26:14.814615+0000","last_unstale":"2023-10-10T20:26:14.815297+0000","last_undegraded":"2023-10-10T20:26:14.815297+0000","last_fullsized":"2023-10-10T20:26:14.815297+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:47.218066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818886+0000","last_change":"2023-10-10T20:26:14.818823+0000","last_active":"2023-10-10T20:26:14.818886+0000","last_peered":"2023-10-10T20:26:14.818886+0000","last_clean":"2023-10-10T20:26:14.818886+0000","last_became_active":"2023-10-10T20:26:14.818599+0000","last_became_peered":"2023-10-10T20:26:14.818599+0000","last_unstale":"2023-10-10T20:26:14.818886+0000","last_undegraded":"2023-10-10T20:26:14.818886+0000","last_fullsized":"2023-10-10T20:26:14.818886+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:10:59.677714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805598+0000","last_change":"2023-10-10T20:26:14.805393+0000","last_active":"2023-10-10T20:26:14.805598+0000","last_peered":"2023-10-10T20:26:14.805598+0000","last_clean":"2023-10-10T20:26:14.805598+0000","last_became_active":"2023-10-10T20:26:14.805130+0000","last_became_peered":"2023-10-10T20:26:14.805130+0000","last_unstale":"2023-10-10T20:26:14.805598+0000","last_undegraded":"2023-10-10T20:26:14.805598+0000","last_fullsized":"2023-10-10T20:26:14.805598+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:12.602173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816489+0000","last_change":"2023-10-10T20:26:14.814238+0000","last_active":"2023-10-10T20:26:14.816489+0000","last_peered":"2023-10-10T20:26:14.816489+0000","last_clean":"2023-10-10T20:26:14.816489+0000","last_became_active":"2023-10-10T20:26:14.813706+0000","last_became_peered":"2023-10-10T20:26:14.813706+0000","last_unstale":"2023-10-10T20:26:14.816489+0000","last_undegraded":"2023-10-10T20:26:14.816489+0000","last_fullsized":"2023-10-10T20:26:14.816489+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:52.366102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822156+0000","last_change":"2023-10-10T20:26:14.821805+0000","last_active":"2023-10-10T20:26:14.822156+0000","last_peered":"2023-10-10T20:26:14.822156+0000","last_clean":"2023-10-10T20:26:14.822156+0000","last_became_active":"2023-10-10T20:26:14.817027+0000","last_became_peered":"2023-10-10T20:26:14.817027+0000","last_unstale":"2023-10-10T20:26:14.822156+0000","last_undegraded":"2023-10-10T20:26:14.822156+0000","last_fullsized":"2023-10-10T20:26:14.822156+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:46:57.561657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816308+0000","last_change":"2023-10-10T20:26:14.816247+0000","last_active":"2023-10-10T20:26:14.816308+0000","last_peered":"2023-10-10T20:26:14.816308+0000","last_clean":"2023-10-10T20:26:14.816308+0000","last_became_active":"2023-10-10T20:26:14.815644+0000","last_became_peered":"2023-10-10T20:26:14.815644+0000","last_unstale":"2023-10-10T20:26:14.816308+0000","last_undegraded":"2023-10-10T20:26:14.816308+0000","last_fullsized":"2023-10-10T20:26:14.816308+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:49.792681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813937+0000","last_change":"2023-10-10T20:26:14.813487+0000","last_active":"2023-10-10T20:26:14.813937+0000","last_peered":"2023-10-10T20:26:14.813937+0000","last_clean":"2023-10-10T20:26:14.813937+0000","last_became_active":"2023-10-10T20:26:14.813005+0000","last_became_peered":"2023-10-10T20:26:14.813005+0000","last_unstale":"2023-10-10T20:26:14.813937+0000","last_undegraded":"2023-10-10T20:26:14.813937+0000","last_fullsized":"2023-10-10T20:26:14.813937+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:39.939171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815580+0000","last_change":"2023-10-10T20:26:14.815511+0000","last_active":"2023-10-10T20:26:14.815580+0000","last_peered":"2023-10-10T20:26:14.815580+0000","last_clean":"2023-10-10T20:26:14.815580+0000","last_became_active":"2023-10-10T20:26:14.815220+0000","last_became_peered":"2023-10-10T20:26:14.815220+0000","last_unstale":"2023-10-10T20:26:14.815580+0000","last_undegraded":"2023-10-10T20:26:14.815580+0000","last_fullsized":"2023-10-10T20:26:14.815580+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:23:09.555039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822026+0000","last_change":"2023-10-10T20:26:14.821875+0000","last_active":"2023-10-10T20:26:14.822026+0000","last_peered":"2023-10-10T20:26:14.822026+0000","last_clean":"2023-10-10T20:26:14.822026+0000","last_became_active":"2023-10-10T20:26:14.821384+0000","last_became_peered":"2023-10-10T20:26:14.821384+0000","last_unstale":"2023-10-10T20:26:14.822026+0000","last_undegraded":"2023-10-10T20:26:14.822026+0000","last_fullsized":"2023-10-10T20:26:14.822026+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:27.143818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815612+0000","last_change":"2023-10-10T20:26:14.814513+0000","last_active":"2023-10-10T20:26:14.815612+0000","last_peered":"2023-10-10T20:26:14.815612+0000","last_clean":"2023-10-10T20:26:14.815612+0000","last_became_active":"2023-10-10T20:26:14.812918+0000","last_became_peered":"2023-10-10T20:26:14.812918+0000","last_unstale":"2023-10-10T20:26:14.815612+0000","last_undegraded":"2023-10-10T20:26:14.815612+0000","last_fullsized":"2023-10-10T20:26:14.815612+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:15:05.146769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819030+0000","last_change":"2023-10-10T20:26:14.818305+0000","last_active":"2023-10-10T20:26:14.819030+0000","last_peered":"2023-10-10T20:26:14.819030+0000","last_clean":"2023-10-10T20:26:14.819030+0000","last_became_active":"2023-10-10T20:26:14.817679+0000","last_became_peered":"2023-10-10T20:26:14.817679+0000","last_unstale":"2023-10-10T20:26:14.819030+0000","last_undegraded":"2023-10-10T20:26:14.819030+0000","last_fullsized":"2023-10-10T20:26:14.819030+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:47.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812595+0000","last_change":"2023-10-10T20:26:14.812347+0000","last_active":"2023-10-10T20:26:14.812595+0000","last_peered":"2023-10-10T20:26:14.812595+0000","last_clean":"2023-10-10T20:26:14.812595+0000","last_became_active":"2023-10-10T20:26:14.811734+0000","last_became_peered":"2023-10-10T20:26:14.811734+0000","last_unstale":"2023-10-10T20:26:14.812595+0000","last_undegraded":"2023-10-10T20:26:14.812595+0000","last_fullsized":"2023-10-10T20:26:14.812595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:59.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.811924+0000","last_change":"2023-10-10T20:26:14.811640+0000","last_active":"2023-10-10T20:26:14.811924+0000","last_peered":"2023-10-10T20:26:14.811924+0000","last_clean":"2023-10-10T20:26:14.811924+0000","last_became_active":"2023-10-10T20:26:14.811355+0000","last_became_peered":"2023-10-10T20:26:14.811355+0000","last_unstale":"2023-10-10T20:26:14.811924+0000","last_undegraded":"2023-10-10T20:26:14.811924+0000","last_fullsized":"2023-10-10T20:26:14.811924+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:38:47.218066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"145'4","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820580+0000","last_change":"2023-10-10T20:26:13.813742+0000","last_active":"2023-10-10T20:26:14.820580+0000","last_peered":"2023-10-10T20:26:14.820580+0000","last_clean":"2023-10-10T20:26:14.820580+0000","last_became_active":"2023-10-10T20:26:13.813327+0000","last_became_peered":"2023-10-10T20:26:13.813327+0000","last_unstale":"2023-10-10T20:26:14.820580+0000","last_undegraded":"2023-10-10T20:26:14.820580+0000","last_fullsized":"2023-10-10T20:26:14.820580+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:10:49.817040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812922+0000","last_change":"2023-10-10T20:26:14.812139+0000","last_active":"2023-10-10T20:26:14.812922+0000","last_peered":"2023-10-10T20:26:14.812922+0000","last_clean":"2023-10-10T20:26:14.812922+0000","last_became_active":"2023-10-10T20:26:14.811097+0000","last_became_peered":"2023-10-10T20:26:14.811097+0000","last_unstale":"2023-10-10T20:26:14.812922+0000","last_undegraded":"2023-10-10T20:26:14.812922+0000","last_fullsized":"2023-10-10T20:26:14.812922+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:25.171860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819456+0000","last_change":"2023-10-10T20:26:13.806645+0000","last_active":"2023-10-10T20:26:14.819456+0000","last_peered":"2023-10-10T20:26:14.819456+0000","last_clean":"2023-10-10T20:26:14.819456+0000","last_became_active":"2023-10-10T20:26:13.806360+0000","last_became_peered":"2023-10-10T20:26:13.806360+0000","last_unstale":"2023-10-10T20:26:14.819456+0000","last_undegraded":"2023-10-10T20:26:14.819456+0000","last_fullsized":"2023-10-10T20:26:14.819456+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:21.337029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819048+0000","last_change":"2023-10-10T20:26:14.818461+0000","last_active":"2023-10-10T20:26:14.819048+0000","last_peered":"2023-10-10T20:26:14.819048+0000","last_clean":"2023-10-10T20:26:14.819048+0000","last_became_active":"2023-10-10T20:26:14.817998+0000","last_became_peered":"2023-10-10T20:26:14.817998+0000","last_unstale":"2023-10-10T20:26:14.819048+0000","last_undegraded":"2023-10-10T20:26:14.819048+0000","last_fullsized":"2023-10-10T20:26:14.819048+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:51:28.152057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818696+0000","last_change":"2023-10-10T20:26:13.805620+0000","last_active":"2023-10-10T20:26:14.818696+0000","last_peered":"2023-10-10T20:26:14.818696+0000","last_clean":"2023-10-10T20:26:14.818696+0000","last_became_active":"2023-10-10T20:26:13.805404+0000","last_became_peered":"2023-10-10T20:26:13.805404+0000","last_unstale":"2023-10-10T20:26:14.818696+0000","last_undegraded":"2023-10-10T20:26:14.818696+0000","last_fullsized":"2023-10-10T20:26:14.818696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:02:16.583830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822234+0000","last_change":"2023-10-10T20:26:14.822169+0000","last_active":"2023-10-10T20:26:14.822234+0000","last_peered":"2023-10-10T20:26:14.822234+0000","last_clean":"2023-10-10T20:26:14.822234+0000","last_became_active":"2023-10-10T20:26:14.821958+0000","last_became_peered":"2023-10-10T20:26:14.821958+0000","last_unstale":"2023-10-10T20:26:14.822234+0000","last_undegraded":"2023-10-10T20:26:14.822234+0000","last_fullsized":"2023-10-10T20:26:14.822234+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:58:29.222891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819936+0000","last_change":"2023-10-10T20:26:13.804037+0000","last_active":"2023-10-10T20:26:14.819936+0000","last_peered":"2023-10-10T20:26:14.819936+0000","last_clean":"2023-10-10T20:26:14.819936+0000","last_became_active":"2023-10-10T20:26:13.803820+0000","last_became_peered":"2023-10-10T20:26:13.803820+0000","last_unstale":"2023-10-10T20:26:14.819936+0000","last_undegraded":"2023-10-10T20:26:14.819936+0000","last_fullsized":"2023-10-10T20:26:14.819936+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:28:33.311364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813336+0000","last_change":"2023-10-10T20:26:14.813080+0000","last_active":"2023-10-10T20:26:14.813336+0000","last_peered":"2023-10-10T20:26:14.813336+0000","last_clean":"2023-10-10T20:26:14.813336+0000","last_became_active":"2023-10-10T20:26:14.812681+0000","last_became_peered":"2023-10-10T20:26:14.812681+0000","last_unstale":"2023-10-10T20:26:14.813336+0000","last_undegraded":"2023-10-10T20:26:14.813336+0000","last_fullsized":"2023-10-10T20:26:14.813336+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:53:07.893203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822497+0000","last_change":"2023-10-10T20:26:13.803921+0000","last_active":"2023-10-10T20:26:14.822497+0000","last_peered":"2023-10-10T20:26:14.822497+0000","last_clean":"2023-10-10T20:26:14.822497+0000","last_became_active":"2023-10-10T20:26:13.803671+0000","last_became_peered":"2023-10-10T20:26:13.803671+0000","last_unstale":"2023-10-10T20:26:14.822497+0000","last_undegraded":"2023-10-10T20:26:14.822497+0000","last_fullsized":"2023-10-10T20:26:14.822497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:23.783432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.812622+0000","last_change":"2023-10-10T20:26:14.812517+0000","last_active":"2023-10-10T20:26:14.812622+0000","last_peered":"2023-10-10T20:26:14.812622+0000","last_clean":"2023-10-10T20:26:14.812622+0000","last_became_active":"2023-10-10T20:26:14.812074+0000","last_became_peered":"2023-10-10T20:26:14.812074+0000","last_unstale":"2023-10-10T20:26:14.812622+0000","last_undegraded":"2023-10-10T20:26:14.812622+0000","last_fullsized":"2023-10-10T20:26:14.812622+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:38.344472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823191+0000","last_change":"2023-10-10T20:26:13.811443+0000","last_active":"2023-10-10T20:26:14.823191+0000","last_peered":"2023-10-10T20:26:14.823191+0000","last_clean":"2023-10-10T20:26:14.823191+0000","last_became_active":"2023-10-10T20:26:13.811075+0000","last_became_peered":"2023-10-10T20:26:13.811075+0000","last_unstale":"2023-10-10T20:26:14.823191+0000","last_undegraded":"2023-10-10T20:26:14.823191+0000","last_fullsized":"2023-10-10T20:26:14.823191+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:11.592757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808602+0000","last_change":"2023-10-10T20:26:14.808260+0000","last_active":"2023-10-10T20:26:14.808602+0000","last_peered":"2023-10-10T20:26:14.808602+0000","last_clean":"2023-10-10T20:26:14.808602+0000","last_became_active":"2023-10-10T20:26:14.807933+0000","last_became_peered":"2023-10-10T20:26:14.807933+0000","last_unstale":"2023-10-10T20:26:14.808602+0000","last_undegraded":"2023-10-10T20:26:14.808602+0000","last_fullsized":"2023-10-10T20:26:14.808602+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:25.205828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805930+0000","last_change":"2023-10-10T20:26:13.810585+0000","last_active":"2023-10-10T20:26:14.805930+0000","last_peered":"2023-10-10T20:26:14.805930+0000","last_clean":"2023-10-10T20:26:14.805930+0000","last_became_active":"2023-10-10T20:26:13.810319+0000","last_became_peered":"2023-10-10T20:26:13.810319+0000","last_unstale":"2023-10-10T20:26:14.805930+0000","last_undegraded":"2023-10-10T20:26:14.805930+0000","last_fullsized":"2023-10-10T20:26:14.805930+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:27.365697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.816633+0000","last_change":"2023-10-10T20:26:14.816573+0000","last_active":"2023-10-10T20:26:14.816633+0000","last_peered":"2023-10-10T20:26:14.816633+0000","last_clean":"2023-10-10T20:26:14.816633+0000","last_became_active":"2023-10-10T20:26:14.816326+0000","last_became_peered":"2023-10-10T20:26:14.816326+0000","last_unstale":"2023-10-10T20:26:14.816633+0000","last_undegraded":"2023-10-10T20:26:14.816633+0000","last_fullsized":"2023-10-10T20:26:14.816633+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:52:40.212927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.803320+0000","last_change":"2023-10-10T20:26:13.810239+0000","last_active":"2023-10-10T20:26:14.803320+0000","last_peered":"2023-10-10T20:26:14.803320+0000","last_clean":"2023-10-10T20:26:14.803320+0000","last_became_active":"2023-10-10T20:26:13.808022+0000","last_became_peered":"2023-10-10T20:26:13.808022+0000","last_unstale":"2023-10-10T20:26:14.803320+0000","last_undegraded":"2023-10-10T20:26:14.803320+0000","last_fullsized":"2023-10-10T20:26:14.803320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.687349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813171+0000","last_change":"2023-10-10T20:26:14.812788+0000","last_active":"2023-10-10T20:26:14.813171+0000","last_peered":"2023-10-10T20:26:14.813171+0000","last_clean":"2023-10-10T20:26:14.813171+0000","last_became_active":"2023-10-10T20:26:14.812241+0000","last_became_peered":"2023-10-10T20:26:14.812241+0000","last_unstale":"2023-10-10T20:26:14.813171+0000","last_undegraded":"2023-10-10T20:26:14.813171+0000","last_fullsized":"2023-10-10T20:26:14.813171+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:24:18.472820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.802364+0000","last_change":"2023-10-10T20:26:13.804155+0000","last_active":"2023-10-10T20:26:14.802364+0000","last_peered":"2023-10-10T20:26:14.802364+0000","last_clean":"2023-10-10T20:26:14.802364+0000","last_became_active":"2023-10-10T20:26:13.803882+0000","last_became_peered":"2023-10-10T20:26:13.803882+0000","last_unstale":"2023-10-10T20:26:14.802364+0000","last_undegraded":"2023-10-10T20:26:14.802364+0000","last_fullsized":"2023-10-10T20:26:14.802364+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:26.134402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810835+0000","last_change":"2023-10-10T20:26:14.810762+0000","last_active":"2023-10-10T20:26:14.810835+0000","last_peered":"2023-10-10T20:26:14.810835+0000","last_clean":"2023-10-10T20:26:14.810835+0000","last_became_active":"2023-10-10T20:26:14.810504+0000","last_became_peered":"2023-10-10T20:26:14.810504+0000","last_unstale":"2023-10-10T20:26:14.810835+0000","last_undegraded":"2023-10-10T20:26:14.810835+0000","last_fullsized":"2023-10-10T20:26:14.810835+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:54:37.199366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808760+0000","last_change":"2023-10-10T20:26:13.810274+0000","last_active":"2023-10-10T20:26:14.808760+0000","last_peered":"2023-10-10T20:26:14.808760+0000","last_clean":"2023-10-10T20:26:14.808760+0000","last_became_active":"2023-10-10T20:26:13.810029+0000","last_became_peered":"2023-10-10T20:26:13.810029+0000","last_unstale":"2023-10-10T20:26:14.808760+0000","last_undegraded":"2023-10-10T20:26:14.808760+0000","last_fullsized":"2023-10-10T20:26:14.808760+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:53:57.864466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820066+0000","last_change":"2023-10-10T20:26:14.819892+0000","last_active":"2023-10-10T20:26:14.820066+0000","last_peered":"2023-10-10T20:26:14.820066+0000","last_clean":"2023-10-10T20:26:14.820066+0000","last_became_active":"2023-10-10T20:26:14.818432+0000","last_became_peered":"2023-10-10T20:26:14.818432+0000","last_unstale":"2023-10-10T20:26:14.820066+0000","last_undegraded":"2023-10-10T20:26:14.820066+0000","last_fullsized":"2023-10-10T20:26:14.820066+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:53:07.893203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"145'3","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819338+0000","last_change":"2023-10-10T20:26:13.809553+0000","last_active":"2023-10-10T20:26:14.819338+0000","last_peered":"2023-10-10T20:26:14.819338+0000","last_clean":"2023-10-10T20:26:14.819338+0000","last_became_active":"2023-10-10T20:26:13.809254+0000","last_became_peered":"2023-10-10T20:26:13.809254+0000","last_unstale":"2023-10-10T20:26:14.819338+0000","last_undegraded":"2023-10-10T20:26:14.819338+0000","last_fullsized":"2023-10-10T20:26:14.819338+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:42:23.783432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818239+0000","last_change":"2023-10-10T20:26:14.817395+0000","last_active":"2023-10-10T20:26:14.818239+0000","last_peered":"2023-10-10T20:26:14.818239+0000","last_clean":"2023-10-10T20:26:14.818239+0000","last_became_active":"2023-10-10T20:26:14.817171+0000","last_became_peered":"2023-10-10T20:26:14.817171+0000","last_unstale":"2023-10-10T20:26:14.818239+0000","last_undegraded":"2023-10-10T20:26:14.818239+0000","last_fullsized":"2023-10-10T20:26:14.818239+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:51.663471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"145'6","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.824066+0000","last_change":"2023-10-10T20:26:13.811194+0000","last_active":"2023-10-10T20:26:14.824066+0000","last_peered":"2023-10-10T20:26:14.824066+0000","last_clean":"2023-10-10T20:26:14.824066+0000","last_became_active":"2023-10-10T20:26:13.810989+0000","last_became_peered":"2023-10-10T20:26:13.810989+0000","last_unstale":"2023-10-10T20:26:14.824066+0000","last_undegraded":"2023-10-10T20:26:14.824066+0000","last_fullsized":"2023-10-10T20:26:14.824066+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:35:34.422577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810206+0000","last_change":"2023-10-10T20:26:14.809889+0000","last_active":"2023-10-10T20:26:14.810206+0000","last_peered":"2023-10-10T20:26:14.810206+0000","last_clean":"2023-10-10T20:26:14.810206+0000","last_became_active":"2023-10-10T20:26:14.809715+0000","last_became_peered":"2023-10-10T20:26:14.809715+0000","last_unstale":"2023-10-10T20:26:14.810206+0000","last_undegraded":"2023-10-10T20:26:14.810206+0000","last_fullsized":"2023-10-10T20:26:14.810206+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:01:25.394848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804361+0000","last_change":"2023-10-10T20:26:13.812264+0000","last_active":"2023-10-10T20:26:14.804361+0000","last_peered":"2023-10-10T20:26:14.804361+0000","last_clean":"2023-10-10T20:26:14.804361+0000","last_became_active":"2023-10-10T20:26:13.812018+0000","last_became_peered":"2023-10-10T20:26:13.812018+0000","last_unstale":"2023-10-10T20:26:14.804361+0000","last_undegraded":"2023-10-10T20:26:14.804361+0000","last_fullsized":"2023-10-10T20:26:14.804361+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:49:12.107043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808139+0000","last_change":"2023-10-10T20:26:14.808030+0000","last_active":"2023-10-10T20:26:14.808139+0000","last_peered":"2023-10-10T20:26:14.808139+0000","last_clean":"2023-10-10T20:26:14.808139+0000","last_became_active":"2023-10-10T20:26:14.807768+0000","last_became_peered":"2023-10-10T20:26:14.807768+0000","last_unstale":"2023-10-10T20:26:14.808139+0000","last_undegraded":"2023-10-10T20:26:14.808139+0000","last_fullsized":"2023-10-10T20:26:14.808139+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:09.165136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801234+0000","last_change":"2023-10-10T20:26:13.811090+0000","last_active":"2023-10-10T20:26:14.801234+0000","last_peered":"2023-10-10T20:26:14.801234+0000","last_clean":"2023-10-10T20:26:14.801234+0000","last_became_active":"2023-10-10T20:26:13.810474+0000","last_became_peered":"2023-10-10T20:26:13.810474+0000","last_unstale":"2023-10-10T20:26:14.801234+0000","last_undegraded":"2023-10-10T20:26:14.801234+0000","last_fullsized":"2023-10-10T20:26:14.801234+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:55:21.337029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804397+0000","last_change":"2023-10-10T20:26:14.804198+0000","last_active":"2023-10-10T20:26:14.804397+0000","last_peered":"2023-10-10T20:26:14.804397+0000","last_clean":"2023-10-10T20:26:14.804397+0000","last_became_active":"2023-10-10T20:26:14.803958+0000","last_became_peered":"2023-10-10T20:26:14.803958+0000","last_unstale":"2023-10-10T20:26:14.804397+0000","last_undegraded":"2023-10-10T20:26:14.804397+0000","last_fullsized":"2023-10-10T20:26:14.804397+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:46:22.908780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"145'8","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.821324+0000","last_change":"2023-10-10T20:26:13.810023+0000","last_active":"2023-10-10T20:26:14.821324+0000","last_peered":"2023-10-10T20:26:14.821324+0000","last_clean":"2023-10-10T20:26:14.821324+0000","last_became_active":"2023-10-10T20:26:13.809762+0000","last_became_peered":"2023-10-10T20:26:13.809762+0000","last_unstale":"2023-10-10T20:26:14.821324+0000","last_undegraded":"2023-10-10T20:26:14.821324+0000","last_fullsized":"2023-10-10T20:26:14.821324+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:51.070132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809883+0000","last_change":"2023-10-10T20:26:14.809806+0000","last_active":"2023-10-10T20:26:14.809883+0000","last_peered":"2023-10-10T20:26:14.809883+0000","last_clean":"2023-10-10T20:26:14.809883+0000","last_became_active":"2023-10-10T20:26:14.809576+0000","last_became_peered":"2023-10-10T20:26:14.809576+0000","last_unstale":"2023-10-10T20:26:14.809883+0000","last_undegraded":"2023-10-10T20:26:14.809883+0000","last_fullsized":"2023-10-10T20:26:14.809883+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:59.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820592+0000","last_change":"2023-10-10T20:26:13.809260+0000","last_active":"2023-10-10T20:26:14.820592+0000","last_peered":"2023-10-10T20:26:14.820592+0000","last_clean":"2023-10-10T20:26:14.820592+0000","last_became_active":"2023-10-10T20:26:13.808950+0000","last_became_peered":"2023-10-10T20:26:13.808950+0000","last_unstale":"2023-10-10T20:26:14.820592+0000","last_undegraded":"2023-10-10T20:26:14.820592+0000","last_fullsized":"2023-10-10T20:26:14.820592+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:32:03.687349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819721+0000","last_change":"2023-10-10T20:26:14.819646+0000","last_active":"2023-10-10T20:26:14.819721+0000","last_peered":"2023-10-10T20:26:14.819721+0000","last_clean":"2023-10-10T20:26:14.819721+0000","last_became_active":"2023-10-10T20:26:14.819063+0000","last_became_peered":"2023-10-10T20:26:14.819063+0000","last_unstale":"2023-10-10T20:26:14.819721+0000","last_undegraded":"2023-10-10T20:26:14.819721+0000","last_fullsized":"2023-10-10T20:26:14.819721+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:13:15.442789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818475+0000","last_change":"2023-10-10T20:26:13.806539+0000","last_active":"2023-10-10T20:26:14.818475+0000","last_peered":"2023-10-10T20:26:14.818475+0000","last_clean":"2023-10-10T20:26:14.818475+0000","last_became_active":"2023-10-10T20:26:13.806299+0000","last_became_peered":"2023-10-10T20:26:13.806299+0000","last_unstale":"2023-10-10T20:26:14.818475+0000","last_undegraded":"2023-10-10T20:26:14.818475+0000","last_fullsized":"2023-10-10T20:26:14.818475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:21:33.034655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809818+0000","last_change":"2023-10-10T20:26:14.809669+0000","last_active":"2023-10-10T20:26:14.809818+0000","last_peered":"2023-10-10T20:26:14.809818+0000","last_clean":"2023-10-10T20:26:14.809818+0000","last_became_active":"2023-10-10T20:26:14.809389+0000","last_became_peered":"2023-10-10T20:26:14.809389+0000","last_unstale":"2023-10-10T20:26:14.809818+0000","last_undegraded":"2023-10-10T20:26:14.809818+0000","last_fullsized":"2023-10-10T20:26:14.809818+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:47.037238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"145'1","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.817587+0000","last_change":"2023-10-10T20:26:13.810263+0000","last_active":"2023-10-10T20:26:14.817587+0000","last_peered":"2023-10-10T20:26:14.817587+0000","last_clean":"2023-10-10T20:26:14.817587+0000","last_became_active":"2023-10-10T20:26:13.809885+0000","last_became_peered":"2023-10-10T20:26:13.809885+0000","last_unstale":"2023-10-10T20:26:14.817587+0000","last_undegraded":"2023-10-10T20:26:14.817587+0000","last_fullsized":"2023-10-10T20:26:14.817587+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:19:11.711139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814091+0000","last_change":"2023-10-10T20:26:14.813086+0000","last_active":"2023-10-10T20:26:14.814091+0000","last_peered":"2023-10-10T20:26:14.814091+0000","last_clean":"2023-10-10T20:26:14.814091+0000","last_became_active":"2023-10-10T20:26:14.812187+0000","last_became_peered":"2023-10-10T20:26:14.812187+0000","last_unstale":"2023-10-10T20:26:14.814091+0000","last_undegraded":"2023-10-10T20:26:14.814091+0000","last_fullsized":"2023-10-10T20:26:14.814091+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:58:52.079548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801913+0000","last_change":"2023-10-10T20:26:13.804815+0000","last_active":"2023-10-10T20:26:14.801913+0000","last_peered":"2023-10-10T20:26:14.801913+0000","last_clean":"2023-10-10T20:26:14.801913+0000","last_became_active":"2023-10-10T20:26:13.804574+0000","last_became_peered":"2023-10-10T20:26:13.804574+0000","last_unstale":"2023-10-10T20:26:14.801913+0000","last_undegraded":"2023-10-10T20:26:14.801913+0000","last_fullsized":"2023-10-10T20:26:14.801913+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:38.929755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.821700+0000","last_change":"2023-10-10T20:26:14.821633+0000","last_active":"2023-10-10T20:26:14.821700+0000","last_peered":"2023-10-10T20:26:14.821700+0000","last_clean":"2023-10-10T20:26:14.821700+0000","last_became_active":"2023-10-10T20:26:14.821054+0000","last_became_peered":"2023-10-10T20:26:14.821054+0000","last_unstale":"2023-10-10T20:26:14.821700+0000","last_undegraded":"2023-10-10T20:26:14.821700+0000","last_fullsized":"2023-10-10T20:26:14.821700+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:42.782562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819546+0000","last_change":"2023-10-10T20:26:13.804323+0000","last_active":"2023-10-10T20:26:14.819546+0000","last_peered":"2023-10-10T20:26:14.819546+0000","last_clean":"2023-10-10T20:26:14.819546+0000","last_became_active":"2023-10-10T20:26:13.803498+0000","last_became_peered":"2023-10-10T20:26:13.803498+0000","last_unstale":"2023-10-10T20:26:14.819546+0000","last_undegraded":"2023-10-10T20:26:14.819546+0000","last_fullsized":"2023-10-10T20:26:14.819546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:41.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.807859+0000","last_change":"2023-10-10T20:26:14.807779+0000","last_active":"2023-10-10T20:26:14.807859+0000","last_peered":"2023-10-10T20:26:14.807859+0000","last_clean":"2023-10-10T20:26:14.807859+0000","last_became_active":"2023-10-10T20:26:14.807518+0000","last_became_peered":"2023-10-10T20:26:14.807518+0000","last_unstale":"2023-10-10T20:26:14.807859+0000","last_undegraded":"2023-10-10T20:26:14.807859+0000","last_fullsized":"2023-10-10T20:26:14.807859+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:39.824997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.806260+0000","last_change":"2023-10-10T20:26:13.809050+0000","last_active":"2023-10-10T20:26:14.806260+0000","last_peered":"2023-10-10T20:26:14.806260+0000","last_clean":"2023-10-10T20:26:14.806260+0000","last_became_active":"2023-10-10T20:26:13.808776+0000","last_became_peered":"2023-10-10T20:26:13.808776+0000","last_unstale":"2023-10-10T20:26:14.806260+0000","last_undegraded":"2023-10-10T20:26:14.806260+0000","last_fullsized":"2023-10-10T20:26:14.806260+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:58.314132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814109+0000","last_change":"2023-10-10T20:26:14.813239+0000","last_active":"2023-10-10T20:26:14.814109+0000","last_peered":"2023-10-10T20:26:14.814109+0000","last_clean":"2023-10-10T20:26:14.814109+0000","last_became_active":"2023-10-10T20:26:14.812862+0000","last_became_peered":"2023-10-10T20:26:14.812862+0000","last_unstale":"2023-10-10T20:26:14.814109+0000","last_undegraded":"2023-10-10T20:26:14.814109+0000","last_fullsized":"2023-10-10T20:26:14.814109+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:24:26.787393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.805805+0000","last_change":"2023-10-10T20:26:13.809813+0000","last_active":"2023-10-10T20:26:14.805805+0000","last_peered":"2023-10-10T20:26:14.805805+0000","last_clean":"2023-10-10T20:26:14.805805+0000","last_became_active":"2023-10-10T20:26:13.809547+0000","last_became_peered":"2023-10-10T20:26:13.809547+0000","last_unstale":"2023-10-10T20:26:14.805805+0000","last_undegraded":"2023-10-10T20:26:14.805805+0000","last_fullsized":"2023-10-10T20:26:14.805805+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:37.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.818208+0000","last_change":"2023-10-10T20:26:14.816799+0000","last_active":"2023-10-10T20:26:14.818208+0000","last_peered":"2023-10-10T20:26:14.818208+0000","last_clean":"2023-10-10T20:26:14.818208+0000","last_became_active":"2023-10-10T20:26:14.816388+0000","last_became_peered":"2023-10-10T20:26:14.816388+0000","last_unstale":"2023-10-10T20:26:14.818208+0000","last_undegraded":"2023-10-10T20:26:14.818208+0000","last_fullsized":"2023-10-10T20:26:14.818208+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:46:06.212765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808977+0000","last_change":"2023-10-10T20:26:13.806628+0000","last_active":"2023-10-10T20:26:14.808977+0000","last_peered":"2023-10-10T20:26:14.808977+0000","last_clean":"2023-10-10T20:26:14.808977+0000","last_became_active":"2023-10-10T20:26:13.806422+0000","last_became_peered":"2023-10-10T20:26:13.806422+0000","last_unstale":"2023-10-10T20:26:14.808977+0000","last_undegraded":"2023-10-10T20:26:14.808977+0000","last_fullsized":"2023-10-10T20:26:14.808977+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:09:46.027821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.810733+0000","last_change":"2023-10-10T20:26:14.810516+0000","last_active":"2023-10-10T20:26:14.810733+0000","last_peered":"2023-10-10T20:26:14.810733+0000","last_clean":"2023-10-10T20:26:14.810733+0000","last_became_active":"2023-10-10T20:26:14.810213+0000","last_became_peered":"2023-10-10T20:26:14.810213+0000","last_unstale":"2023-10-10T20:26:14.810733+0000","last_undegraded":"2023-10-10T20:26:14.810733+0000","last_fullsized":"2023-10-10T20:26:14.810733+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:22:29.789396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819595+0000","last_change":"2023-10-10T20:26:13.804807+0000","last_active":"2023-10-10T20:26:14.819595+0000","last_peered":"2023-10-10T20:26:14.819595+0000","last_clean":"2023-10-10T20:26:14.819595+0000","last_became_active":"2023-10-10T20:26:13.804594+0000","last_became_peered":"2023-10-10T20:26:13.804594+0000","last_unstale":"2023-10-10T20:26:14.819595+0000","last_undegraded":"2023-10-10T20:26:14.819595+0000","last_fullsized":"2023-10-10T20:26:14.819595+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:01.844525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.813921+0000","last_change":"2023-10-10T20:26:14.813358+0000","last_active":"2023-10-10T20:26:14.813921+0000","last_peered":"2023-10-10T20:26:14.813921+0000","last_clean":"2023-10-10T20:26:14.813921+0000","last_became_active":"2023-10-10T20:26:14.812558+0000","last_became_peered":"2023-10-10T20:26:14.812558+0000","last_unstale":"2023-10-10T20:26:14.813921+0000","last_undegraded":"2023-10-10T20:26:14.813921+0000","last_fullsized":"2023-10-10T20:26:14.813921+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:50:35.923557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809407+0000","last_change":"2023-10-10T20:26:13.813589+0000","last_active":"2023-10-10T20:26:14.809407+0000","last_peered":"2023-10-10T20:26:14.809407+0000","last_clean":"2023-10-10T20:26:14.809407+0000","last_became_active":"2023-10-10T20:26:13.811357+0000","last_became_peered":"2023-10-10T20:26:13.811357+0000","last_unstale":"2023-10-10T20:26:14.809407+0000","last_undegraded":"2023-10-10T20:26:14.809407+0000","last_fullsized":"2023-10-10T20:26:14.809407+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:54:38.815580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.822173+0000","last_change":"2023-10-10T20:26:14.821951+0000","last_active":"2023-10-10T20:26:14.822173+0000","last_peered":"2023-10-10T20:26:14.822173+0000","last_clean":"2023-10-10T20:26:14.822173+0000","last_became_active":"2023-10-10T20:26:14.817332+0000","last_became_peered":"2023-10-10T20:26:14.817332+0000","last_unstale":"2023-10-10T20:26:14.822173+0000","last_undegraded":"2023-10-10T20:26:14.822173+0000","last_fullsized":"2023-10-10T20:26:14.822173+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:52:28.375113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801894+0000","last_change":"2023-10-10T20:26:13.807314+0000","last_active":"2023-10-10T20:26:14.801894+0000","last_peered":"2023-10-10T20:26:14.801894+0000","last_clean":"2023-10-10T20:26:14.801894+0000","last_became_active":"2023-10-10T20:26:13.807065+0000","last_became_peered":"2023-10-10T20:26:13.807065+0000","last_unstale":"2023-10-10T20:26:14.801894+0000","last_undegraded":"2023-10-10T20:26:14.801894+0000","last_fullsized":"2023-10-10T20:26:14.801894+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:33:26.134402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.819951+0000","last_change":"2023-10-10T20:26:14.819706+0000","last_active":"2023-10-10T20:26:14.819951+0000","last_peered":"2023-10-10T20:26:14.819951+0000","last_clean":"2023-10-10T20:26:14.819951+0000","last_became_active":"2023-10-10T20:26:14.819431+0000","last_became_peered":"2023-10-10T20:26:14.819431+0000","last_unstale":"2023-10-10T20:26:14.819951+0000","last_undegraded":"2023-10-10T20:26:14.819951+0000","last_fullsized":"2023-10-10T20:26:14.819951+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:25.171860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804803+0000","last_change":"2023-10-10T20:26:13.810975+0000","last_active":"2023-10-10T20:26:14.804803+0000","last_peered":"2023-10-10T20:26:14.804803+0000","last_clean":"2023-10-10T20:26:14.804803+0000","last_became_active":"2023-10-10T20:26:13.810748+0000","last_became_peered":"2023-10-10T20:26:13.810748+0000","last_unstale":"2023-10-10T20:26:14.804803+0000","last_undegraded":"2023-10-10T20:26:14.804803+0000","last_fullsized":"2023-10-10T20:26:14.804803+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:21:36.907520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.814281+0000","last_change":"2023-10-10T20:26:14.813772+0000","last_active":"2023-10-10T20:26:14.814281+0000","last_peered":"2023-10-10T20:26:14.814281+0000","last_clean":"2023-10-10T20:26:14.814281+0000","last_became_active":"2023-10-10T20:26:14.813543+0000","last_became_peered":"2023-10-10T20:26:14.813543+0000","last_unstale":"2023-10-10T20:26:14.814281+0000","last_undegraded":"2023-10-10T20:26:14.814281+0000","last_fullsized":"2023-10-10T20:26:14.814281+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:00:49.792681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.801600+0000","last_change":"2023-10-10T20:26:13.813643+0000","last_active":"2023-10-10T20:26:14.801600+0000","last_peered":"2023-10-10T20:26:14.801600+0000","last_clean":"2023-10-10T20:26:14.801600+0000","last_became_active":"2023-10-10T20:26:13.812956+0000","last_became_peered":"2023-10-10T20:26:13.812956+0000","last_unstale":"2023-10-10T20:26:14.801600+0000","last_undegraded":"2023-10-10T20:26:14.801600+0000","last_fullsized":"2023-10-10T20:26:14.801600+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:45:35.844291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.807278+0000","last_change":"2023-10-10T20:26:14.806934+0000","last_active":"2023-10-10T20:26:14.807278+0000","last_peered":"2023-10-10T20:26:14.807278+0000","last_clean":"2023-10-10T20:26:14.807278+0000","last_became_active":"2023-10-10T20:26:14.806709+0000","last_became_peered":"2023-10-10T20:26:14.806709+0000","last_unstale":"2023-10-10T20:26:14.807278+0000","last_undegraded":"2023-10-10T20:26:14.807278+0000","last_fullsized":"2023-10-10T20:26:14.807278+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:10:26.395044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.809343+0000","last_change":"2023-10-10T20:26:13.811870+0000","last_active":"2023-10-10T20:26:14.809343+0000","last_peered":"2023-10-10T20:26:14.809343+0000","last_clean":"2023-10-10T20:26:14.809343+0000","last_became_active":"2023-10-10T20:26:13.811616+0000","last_became_peered":"2023-10-10T20:26:13.811616+0000","last_unstale":"2023-10-10T20:26:14.809343+0000","last_undegraded":"2023-10-10T20:26:14.809343+0000","last_fullsized":"2023-10-10T20:26:14.809343+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:35.251813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.815448+0000","last_change":"2023-10-10T20:26:14.815402+0000","last_active":"2023-10-10T20:26:14.815448+0000","last_peered":"2023-10-10T20:26:14.815448+0000","last_clean":"2023-10-10T20:26:14.815448+0000","last_became_active":"2023-10-10T20:26:14.815144+0000","last_became_peered":"2023-10-10T20:26:14.815144+0000","last_unstale":"2023-10-10T20:26:14.815448+0000","last_undegraded":"2023-10-10T20:26:14.815448+0000","last_fullsized":"2023-10-10T20:26:14.815448+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:13:51.663471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804862+0000","last_change":"2023-10-10T20:26:13.811310+0000","last_active":"2023-10-10T20:26:14.804862+0000","last_peered":"2023-10-10T20:26:14.804862+0000","last_clean":"2023-10-10T20:26:14.804862+0000","last_became_active":"2023-10-10T20:26:13.810664+0000","last_became_peered":"2023-10-10T20:26:13.810664+0000","last_unstale":"2023-10-10T20:26:14.804862+0000","last_undegraded":"2023-10-10T20:26:14.804862+0000","last_fullsized":"2023-10-10T20:26:14.804862+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:24.162443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.823052+0000","last_change":"2023-10-10T20:26:14.822989+0000","last_active":"2023-10-10T20:26:14.823052+0000","last_peered":"2023-10-10T20:26:14.823052+0000","last_clean":"2023-10-10T20:26:14.823052+0000","last_became_active":"2023-10-10T20:26:14.822714+0000","last_became_peered":"2023-10-10T20:26:14.822714+0000","last_unstale":"2023-10-10T20:26:14.823052+0000","last_undegraded":"2023-10-10T20:26:14.823052+0000","last_fullsized":"2023-10-10T20:26:14.823052+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:56:56.760999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.804462+0000","last_change":"2023-10-10T20:26:13.813285+0000","last_active":"2023-10-10T20:26:14.804462+0000","last_peered":"2023-10-10T20:26:14.804462+0000","last_clean":"2023-10-10T20:26:14.804462+0000","last_became_active":"2023-10-10T20:26:13.813065+0000","last_became_peered":"2023-10-10T20:26:13.813065+0000","last_unstale":"2023-10-10T20:26:14.804462+0000","last_undegraded":"2023-10-10T20:26:14.804462+0000","last_fullsized":"2023-10-10T20:26:14.804462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:07.466438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820438+0000","last_change":"2023-10-10T20:26:14.820367+0000","last_active":"2023-10-10T20:26:14.820438+0000","last_peered":"2023-10-10T20:26:14.820438+0000","last_clean":"2023-10-10T20:26:14.820438+0000","last_became_active":"2023-10-10T20:26:14.820065+0000","last_became_peered":"2023-10-10T20:26:14.820065+0000","last_unstale":"2023-10-10T20:26:14.820438+0000","last_undegraded":"2023-10-10T20:26:14.820438+0000","last_fullsized":"2023-10-10T20:26:14.820438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:13.791369+0000","last_clean_scrub_stamp":"2023-10-10T20:26:13.791369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:11:25.506245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"145'2","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.820340+0000","last_change":"2023-10-10T20:26:13.812930+0000","last_active":"2023-10-10T20:26:14.820340+0000","last_peered":"2023-10-10T20:26:14.820340+0000","last_clean":"2023-10-10T20:26:14.820340+0000","last_became_active":"2023-10-10T20:26:13.812718+0000","last_became_peered":"2023-10-10T20:26:13.812718+0000","last_unstale":"2023-10-10T20:26:14.820340+0000","last_undegraded":"2023-10-10T20:26:14.820340+0000","last_fullsized":"2023-10-10T20:26:14.820340+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:26:12.781953+0000","last_clean_scrub_stamp":"2023-10-10T20:26:12.781953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:51:37.335055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":275,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-10T20:26:14.808011+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:26:14.808011+0000","last_peered":"2023-10-10T20:26:14.808011+0000","last_clean":"2023-10-10T20:26:14.808011+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:26:14.808011+0000","last_undegraded":"2023-10-10T20:26:14.808011+0000","last_fullsized":"2023-10-10T20:26:14.808011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1366,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193946,"statfs":{"total":200000000,"available":198601063,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70299999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.625,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.378},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.74199999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.52200000000000002},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.626}]},{"osd":5,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.28799999999999998},"max":{"1min":0.67600000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.51000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199192041,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72299999999999998,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.56000000000000005,"15min":0.53500000000000003},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":4.4219999999999997,"15min":4.4219999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":1.865,"15min":1.865},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68200000000000005,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64200000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72799999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.69099999999999995,"5min":4.0430000000000001,"15min":4.0430000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.67600000000000005,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71499999999999997,"5min":15.862,"15min":15.862},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45300000000000001,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.62,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.46200000000000002},"min":{"1min":0.28000000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.27300000000000002},"max":{"1min":0.69099999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32900000000000001,"15min":0.378},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20300000000000001},"max":{"1min":0.49399999999999999,"5min":0.50700000000000001,"15min":1.6100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.38300000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":1.647},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38700000000000001},"min":{"1min":0.24199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.627,"5min":0.627,"15min":1.524},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.58799999999999997,"15min":1.3959999999999999},"last":0.313}]},{"osd":3,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38800000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":1.4470000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63,"15min":1.673},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.90900000000000003,"15min":1.633},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":1.506},"last":0.42999999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":1.696},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.57999999999999996,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.9359999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":0.97499999999999998,"15min":1.7490000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.63400000000000001},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.872,"15min":1.585},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.478},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.63},"min":{"1min":0.438,"5min":0.42699999999999999,"15min":0.42299999999999999},"max":{"1min":0.75900000000000001,"5min":0.91000000000000003,"15min":1.722},"last":0.52900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1363,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.44500000000000001},"min":{"1min":0.216,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.439},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.435,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58099999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.376,"15min":0.47399999999999998},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52400000000000002,"5min":1.647,"15min":2.4830000000000001},"last":0.497},{"interface":"front","average":{"1min":0.375,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.26600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":5.891,"15min":5.891},"last":0.45400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23000000000000001},"max":{"1min":0.504,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.433},"min":{"1min":0.29399999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.57999999999999996,"5min":1.462,"15min":1.462},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80900000000000005,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.92600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77800000000000002,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":5.758,"15min":5.758},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39100000000000001},"max":{"1min":0.79200000000000004,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.64600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194525,"statfs":{"total":200000000,"available":199194000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35299999999999998,"15min":0.38600000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.54200000000000004,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53100000000000003,"5min":8.2629999999999999,"15min":8.2629999999999999},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.22},"max":{"1min":0.42299999999999999,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.48899999999999999,"15min":0.434},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58299999999999996,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.379}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.50600000000000001,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.49399999999999999,"5min":11.815,"15min":11.815},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.56699999999999995,"15min":0.46300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.47899999999999998,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.29299999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.32300000000000001,"15min":0.28100000000000003},"max":{"1min":0.82899999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.70199999999999996,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.73799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.38400000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":6.399,"15min":6.399},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68899999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.66700000000000004,"15min":0.60199999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.623},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75900000000000001,"5min":6.4969999999999999,"15min":6.4969999999999999},"last":0.745}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.67300000000000004,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":9.3450000000000006,"15min":9.3450000000000006},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.66000000000000003,"15min":0.63800000000000001},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193417,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32000000000000001,"15min":0.30299999999999999},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.34200000000000003,"15min":0.315},"min":{"1min":0.23000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.44,"5min":0.57099999999999995,"15min":0.65600000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35599999999999998,"15min":0.33600000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.55200000000000005,"15min":0.67900000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53700000000000003,"5min":0.63200000000000001,"15min":0.751},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40799999999999997,"15min":0.37},"min":{"1min":0.25,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.58199999999999996,"5min":0.621,"15min":0.74099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.41599999999999998,"15min":0.39200000000000002},"min":{"1min":0.29399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.56899999999999995,"5min":0.63500000000000001,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.36799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.72799999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.35599999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.55000000000000004},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.79800000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:25:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85099999999999998,"5min":0.871,"15min":0.871},"last":0.80600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.496},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.41299999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.52800000000000002},"min":{"1min":0.41199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73999999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.49399999999999999},"min":{"1min":0.33500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.627}]},{"osd":5,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.40699999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.504,"15min":0.47399999999999998},"min":{"1min":0.309,"5min":0.30499999999999999,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.32300000000000001},"max":{"1min":0.92600000000000005,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":787,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194524,"statfs":{"total":200000000,"available":199193465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.35399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.71299999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.77200000000000002,"5min":0.81599999999999995,"15min":0.91200000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.311,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.79900000000000004},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.35399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.72599999999999998,"5min":0.83599999999999997,"15min":0.89000000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.39900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66200000000000003,"5min":0.78000000000000003,"15min":0.96299999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.875,"15min":0.875},"last":0.36499999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.31,"15min":0.31},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.434},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.55800000000000005},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.752,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:25:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54500000000000004,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.56299999999999994}]}]}],"pool_statfs":[]}} 2023-10-10T20:26:19.244 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:26:19.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:26:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.397+0000 7efe814c4700 1 Processor -- start 2023-10-10T20:26:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.397+0000 7efe814c4700 1 -- start start 2023-10-10T20:26:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe814c4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe7c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe814c4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efe7c15b900 0x7efe7c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe814c4700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c0afa40 con 0x7efe7c15c2b0 2023-10-10T20:26:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe814c4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c064150 con 0x7efe7c15b900 2023-10-10T20:26:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe814c4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0c9410 con 0x7efe7c0af670 2023-10-10T20:26:19.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe80cc3700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe7c15c2b0 0x7efe7c1582c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57768/0 (socket says 172.21.15.148:57768) 2023-10-10T20:26:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.398+0000 7efe80cc3700 1 -- 172.21.15.148:0/2147793388 learned_addr learned my addr 172.21.15.148:0/2147793388 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe7c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/2147793388 >> v2:172.21.15.161:3300/0 conn(0x7efe7c15b900 0x7efe7c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350900285 0 0) 0x7efe7c0afa40 con 0x7efe7c15c2b0 2023-10-10T20:26:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe64003570 con 0x7efe7c15c2b0 2023-10-10T20:26:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600037111 0 0) 0x7efe64003570 con 0x7efe7c15c2b0 2023-10-10T20:26:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 msgr2=0x7efe7c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe7c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 >> v2:172.21.15.161:3300/0 conn(0x7efe7c15b900 msgr2=0x7efe7c15bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 --2- 172.21.15.148:0/2147793388 >> v2:172.21.15.161:3300/0 conn(0x7efe7c15b900 0x7efe7c15bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c15c710 con 0x7efe7c15c2b0 2023-10-10T20:26:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7efe740017f0 con 0x7efe7c15c2b0 2023-10-10T20:26:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 >> v1:172.21.15.148:6789/0 conn(0x7efe7c15c2b0 legacy=0x7efe7c1582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64004d50 0x7efe640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0c9410 con 0x7efe64003a60 2023-10-10T20:26:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6400ba60 con 0x7efe64004d50 2023-10-10T20:26:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6400bba0 con 0x7efe7c0af670 2023-10-10T20:26:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe80cc3700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64004d50 0x7efe640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 -- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 msgr2=0x7efe64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0af670 0x7efe64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 -- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64004d50 msgr2=0x7efe640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe64004d50 0x7efe640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.399+0000 7efe7bfff700 1 -- 172.21.15.148:0/2147793388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe64003570 con 0x7efe64003a60 2023-10-10T20:26:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe7bfff700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe64003e70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009c00 tx=0x7efe6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec334850853a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00e6b0 con 0x7efe64003a60 2023-10-10T20:26:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c00e810 con 0x7efe64003a60 2023-10-10T20:26:19.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe7affd700 1 -- 172.21.15.148:0/2147793388 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00ebd0 con 0x7efe64003a60 2023-10-10T20:26:19.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 -- 172.21.15.148:0/2147793388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 msgr2=0x7efe64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 --2- 172.21.15.148:0/2147793388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe64003e70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009c00 tx=0x7efe6c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe80cc3700 1 -- 172.21.15.148:0/2147793388 reap_dead start 2023-10-10T20:26:19.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 -- 172.21.15.148:0/2147793388 shutdown_connections 2023-10-10T20:26:19.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 -- 172.21.15.148:0/2147793388 >> 172.21.15.148:0/2147793388 conn(0x7efe7c0c9ae0 msgr2=0x7efe7c0c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:19.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 -- 172.21.15.148:0/2147793388 shutdown_connections 2023-10-10T20:26:19.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.400+0000 7efe814c4700 1 -- 172.21.15.148:0/2147793388 wait complete. 2023-10-10T20:26:19.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.401+0000 7efe814c4700 1 Processor -- start 2023-10-10T20:26:19.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 -- start start 2023-10-10T20:26:19.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 0x7efe7c0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0984c0 0x7efe7c098910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c06f520 con 0x7efe64003a60 2023-10-10T20:26:19.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c098e50 con 0x7efe7c155c90 2023-10-10T20:26:19.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe814c4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0c3180 con 0x7efe7c0984c0 2023-10-10T20:26:19.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe7bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe7bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46356/0 (socket says 172.21.15.148:46356) 2023-10-10T20:26:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe7bfff700 1 -- 172.21.15.148:0/3085700717 learned_addr learned my addr 172.21.15.148:0/3085700717 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.402+0000 7efe80cc3700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0984c0 0x7efe7c098910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 0x7efe7c0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 -- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0984c0 msgr2=0x7efe7c098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0984c0 0x7efe7c098910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 -- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 msgr2=0x7efe7c0c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 0x7efe7c0c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 -- 172.21.15.148:0/3085700717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c156100 con 0x7efe64003a60 2023-10-10T20:26:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 0x7efe7c0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe7bfff700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e4b0 tx=0x7efe6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c022070 con 0x7efe64003a60 2023-10-10T20:26:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c00e980 con 0x7efe64003a60 2023-10-10T20:26:19.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c06ebf0 con 0x7efe64003a60 2023-10-10T20:26:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c027470 con 0x7efe64003a60 2023-10-10T20:26:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.403+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c06f1b0 con 0x7efe64003a60 2023-10-10T20:26:19.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.404+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c06f800 con 0x7efe64003a60 2023-10-10T20:26:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.404+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe6c0278a0 con 0x7efe64003a60 2023-10-10T20:26:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.405+0000 7efe63fff700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 0x7efe680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.405+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe6c085fc0 con 0x7efe64003a60 2023-10-10T20:26:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.405+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 0x7efe680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.405+0000 7efe7b7fe700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 0x7efe680693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe70006dc0 tx=0x7efe70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.409+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe6c0b75a0 con 0x7efe64003a60 2023-10-10T20:26:19.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.669+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efe7c04bb80 con 0x7efe64003a60 2023-10-10T20:26:19.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.672+0000 7efe63fff700 1 -- 172.21.15.148:0/3085700717 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7efe6c0170e0 con 0x7efe64003a60 2023-10-10T20:26:19.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 msgr2=0x7efe680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 0x7efe680693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe70006dc0 tx=0x7efe70008040 comp rx=0 tx=0).stop 2023-10-10T20:26:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 msgr2=0x7efe7c09a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e4b0 tx=0x7efe6c01a000 comp rx=0 tx=0).stop 2023-10-10T20:26:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 shutdown_connections 2023-10-10T20:26:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe68066f30 0x7efe680693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe7c0984c0 0x7efe7c098910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe64003a60 0x7efe7c09a590 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 --2- 172.21.15.148:0/3085700717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe7c155c90 0x7efe7c0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 >> 172.21.15.148:0/3085700717 conn(0x7efe7c0c9ae0 msgr2=0x7efe7c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 shutdown_connections 2023-10-10T20:26:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.675+0000 7efe814c4700 1 -- 172.21.15.148:0/3085700717 wait complete. 2023-10-10T20:26:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:26:19.689 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.801+0000 7fa81a7ed700 1 Processor -- start 2023-10-10T20:26:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 -- start start 2023-10-10T20:26:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa8140caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8140adb20 0x7fa8140c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81406e650 con 0x7fa8140c8800 2023-10-10T20:26:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa814067cd0 con 0x7fa8140adb20 2023-10-10T20:26:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa81a7ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140c8c50 con 0x7fa8140ca630 2023-10-10T20:26:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.802+0000 7fa8137fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8140adb20 0x7fa8140c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8140adb20 0x7fa8140c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46372/0 (socket says 172.21.15.148:46372) 2023-10-10T20:26:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 -- 172.21.15.148:0/286864829 learned_addr learned my addr 172.21.15.148:0/286864829 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa813fff700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa8140caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 -- 172.21.15.148:0/286864829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 msgr2=0x7fa8140caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa8140caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 -- 172.21.15.148:0/286864829 >> v1:172.21.15.161:6789/0 conn(0x7fa8140c8800 legacy=0x7fa8141566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 -- 172.21.15.148:0/286864829 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140679a0 con 0x7fa8140adb20 2023-10-10T20:26:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa813fff700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa8140caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa8137fe700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140adb20 0x7fa8140c82c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa8080099b0 tx=0x7fa808009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae95c0b218e789b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa812ffd700 1 -- 172.21.15.148:0/286864829 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800b820 con 0x7fa8140adb20 2023-10-10T20:26:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa812ffd700 1 -- 172.21.15.148:0/286864829 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80800b980 con 0x7fa8140adb20 2023-10-10T20:26:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.803+0000 7fa812ffd700 1 -- 172.21.15.148:0/286864829 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800bd20 con 0x7fa8140adb20 2023-10-10T20:26:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 -- 172.21.15.148:0/286864829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140adb20 msgr2=0x7fa8140c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140adb20 0x7fa8140c82c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa8080099b0 tx=0x7fa808009250 comp rx=0 tx=0).stop 2023-10-10T20:26:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 -- 172.21.15.148:0/286864829 shutdown_connections 2023-10-10T20:26:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa8140caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/286864829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140adb20 0x7fa8140c82c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 -- 172.21.15.148:0/286864829 >> 172.21.15.148:0/286864829 conn(0x7fa81406a250 msgr2=0x7fa81406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 -- 172.21.15.148:0/286864829 shutdown_connections 2023-10-10T20:26:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.804+0000 7fa81a7ed700 1 -- 172.21.15.148:0/286864829 wait complete. 2023-10-10T20:26:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.805+0000 7fa81a7ed700 1 Processor -- start 2023-10-10T20:26:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.805+0000 7fa81a7ed700 1 -- start start 2023-10-10T20:26:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8140adb20 0x7fa8141520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa814155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814169de0 con 0x7fa8140c8800 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141691f0 con 0x7fa8140adb20 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa814169440 con 0x7fa8140ca630 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46384/0 (socket says 172.21.15.148:46384) 2023-10-10T20:26:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 -- 172.21.15.148:0/2215682294 learned_addr learned my addr 172.21.15.148:0/2215682294 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa813fff700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8140adb20 0x7fa8141520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa818d8a700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa814155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 -- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 msgr2=0x7fa814155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa814155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 -- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8140adb20 msgr2=0x7fa8141520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8140adb20 0x7fa8141520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 -- 172.21.15.148:0/2215682294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080094f0 con 0x7fa8140c8800 2023-10-10T20:26:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa8137fe700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa808009290 tx=0x7fa808016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808011c10 con 0x7fa8140c8800 2023-10-10T20:26:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808020070 con 0x7fa8140c8800 2023-10-10T20:26:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80801b450 con 0x7fa8140c8800 2023-10-10T20:26:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa814169720 con 0x7fa8140c8800 2023-10-10T20:26:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.806+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141525f0 con 0x7fa8140c8800 2023-10-10T20:26:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.807+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814169f20 con 0x7fa8140c8800 2023-10-10T20:26:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.808+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa808022030 con 0x7fa8140c8800 2023-10-10T20:26:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.808+0000 7fa7fbfff700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.808+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8080863c0 con 0x7fa8140c8800 2023-10-10T20:26:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.808+0000 7fa813fff700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.809+0000 7fa813fff700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 0x7fa8000693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa804006fd0 tx=0x7fa804008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:19.813+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa814169f20 con 0x7fa8140c8800 2023-10-10T20:26:20.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.035+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa814169bc0 con 0x7fa8140c8800 2023-10-10T20:26:20.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.036+0000 7fa7fbfff700 1 -- 172.21.15.148:0/2215682294 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fa808053630 con 0x7fa8140c8800 2023-10-10T20:26:20.038 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:20.039 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:26:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.037+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 msgr2=0x7fa8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.037+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 0x7fa8000693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa804006fd0 tx=0x7fa804008040 comp rx=0 tx=0).stop 2023-10-10T20:26:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 msgr2=0x7fa8141556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa808009290 tx=0x7fa808016000 comp rx=0 tx=0).stop 2023-10-10T20:26:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 shutdown_connections 2023-10-10T20:26:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa8140ca630 0x7fa814155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8140adb20 0x7fa8141520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 --2- 172.21.15.148:0/2215682294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140c8800 0x7fa8141556a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 >> 172.21.15.148:0/2215682294 conn(0x7fa81406a250 msgr2=0x7fa8140ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 shutdown_connections 2023-10-10T20:26:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.038+0000 7fa81a7ed700 1 -- 172.21.15.148:0/2215682294 wait complete. 2023-10-10T20:26:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:26:20.099 INFO:tasks.ceph.mds.a.smithi148.stderr:2023-10-10T20:26:20.097+0000 7f124f71e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:26:20.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:26:14.180789+0000', 'modified': '2023-10-10T20:26:17.824138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7470': {'gid': 7470, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/2641217056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 2641217056}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2641217056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:26:20.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:26:20.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-10T20:26:20.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.573+0000 7f2398d34700 1 Processor -- start 2023-10-10T20:26:20.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.573+0000 7f2398d34700 1 -- start start 2023-10-10T20:26:20.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2398d34700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f239415a170 0x7f2394068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2398d34700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940694a0 con 0x7f2394069040 2023-10-10T20:26:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2398d34700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239415ea70 con 0x7f2394159da0 2023-10-10T20:26:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2398d34700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23940b34b0 con 0x7f239415a170 2023-10-10T20:26:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2393fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2394069040 0x7f23940af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57770/0 (socket says 172.21.15.148:57770) 2023-10-10T20:26:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2393fff700 1 -- 172.21.15.148:0/459266706 learned_addr learned my addr 172.21.15.148:0/459266706 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f2392ffd700 1 --2- 172.21.15.148:0/459266706 >> v2:172.21.15.161:3300/0 conn(0x7f239415a170 0x7f2394068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577465637 0 0) 0x7f238c001110 con 0x7f2394069040 2023-10-10T20:26:20.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.574+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f237c003570 con 0x7f2394069040 2023-10-10T20:26:20.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473084421 0 0) 0x7f237c003570 con 0x7f2394069040 2023-10-10T20:26:20.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 >> v2:172.21.15.161:3300/0 conn(0x7f239415a170 msgr2=0x7f2394068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 --2- 172.21.15.148:0/459266706 >> v2:172.21.15.161:3300/0 conn(0x7f239415a170 0x7f2394068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 >> v1:172.21.15.161:6789/0 conn(0x7f2394159da0 legacy=0x7f23940b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940b3990 con 0x7f2394069040 2023-10-10T20:26:20.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f238c001600 con 0x7f2394069040 2023-10-10T20:26:20.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 >> v1:172.21.15.148:6789/0 conn(0x7f2394069040 legacy=0x7f23940af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f237c003a60 0x7f237c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237c004810 0x7f237c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f237c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940b34b0 con 0x7f237c0052e0 2023-10-10T20:26:20.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c00c2a0 con 0x7f237c003a60 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c00c460 con 0x7f237c004810 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f237c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23937fe700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f237c003a60 0x7f237c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2392ffd700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237c004810 0x7f237c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 -- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237c004810 msgr2=0x7f237c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237c004810 0x7f237c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 -- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f237c003a60 msgr2=0x7f237c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f237c003a60 0x7f237c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 -- 172.21.15.148:0/459266706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c003570 con 0x7f237c0052e0 2023-10-10T20:26:20.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 -- 172.21.15.148:0/459266706 reap_dead start 2023-10-10T20:26:20.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f2393fff700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f237c00ba60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c5c0 tx=0x7f238c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308561654765cce9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00cdc0 con 0x7f237c0052e0 2023-10-10T20:26:20.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c005920 con 0x7f237c0052e0 2023-10-10T20:26:20.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.575+0000 7f23927fc700 1 -- 172.21.15.148:0/459266706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c005ce0 con 0x7f237c0052e0 2023-10-10T20:26:20.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 -- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 msgr2=0x7f237c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f237c00ba60 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c5c0 tx=0x7f238c007a50 comp rx=0 tx=0).stop 2023-10-10T20:26:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 -- 172.21.15.148:0/459266706 shutdown_connections 2023-10-10T20:26:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 --2- 172.21.15.148:0/459266706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f237c00ba60 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 -- 172.21.15.148:0/459266706 >> 172.21.15.148:0/459266706 conn(0x7f23940d0bc0 msgr2=0x7f23940d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 -- 172.21.15.148:0/459266706 shutdown_connections 2023-10-10T20:26:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.576+0000 7f2398d34700 1 -- 172.21.15.148:0/459266706 wait complete. 2023-10-10T20:26:20.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.577+0000 7f2398d34700 1 Processor -- start 2023-10-10T20:26:20.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.577+0000 7f2398d34700 1 -- start start 2023-10-10T20:26:20.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 0x7f23940ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23940cee30 0x7f23940cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940cf7e0 con 0x7f237c0052e0 2023-10-10T20:26:20.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23940c8fe0 con 0x7f23940ca150 2023-10-10T20:26:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2398d34700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940c8b50 con 0x7f23940cee30 2023-10-10T20:26:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46400/0 (socket says 172.21.15.148:46400) 2023-10-10T20:26:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 -- 172.21.15.148:0/2300694474 learned_addr learned my addr 172.21.15.148:0/2300694474 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 0x7f23940ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2393fff700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23940cee30 0x7f23940cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 -- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23940cee30 msgr2=0x7f23940cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23940cee30 0x7f23940cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 -- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 msgr2=0x7f23940ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 0x7f23940ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 -- 172.21.15.148:0/2300694474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940c7f40 con 0x7f237c0052e0 2023-10-10T20:26:20.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 0x7f23940ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:20.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.578+0000 7f23937fe700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f238400ebc0 tx=0x7f2384001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.579+0000 7f237b7fe700 1 -- 172.21.15.148:0/2300694474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384006ba0 con 0x7f237c0052e0 2023-10-10T20:26:20.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.579+0000 7f237b7fe700 1 -- 172.21.15.148:0/2300694474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2384006d00 con 0x7f237c0052e0 2023-10-10T20:26:20.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.579+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23940c8220 con 0x7f237c0052e0 2023-10-10T20:26:20.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.579+0000 7f237b7fe700 1 -- 172.21.15.148:0/2300694474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400cab0 con 0x7f237c0052e0 2023-10-10T20:26:20.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.579+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23940c87e0 con 0x7f237c0052e0 2023-10-10T20:26:20.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.580+0000 7f237b7fe700 1 -- 172.21.15.148:0/2300694474 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2384015430 con 0x7f237c0052e0 2023-10-10T20:26:20.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.580+0000 7f237b7fe700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 0x7f23800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.580+0000 7f237b7fe700 1 -- 172.21.15.148:0/2300694474 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23840849e0 con 0x7f237c0052e0 2023-10-10T20:26:20.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.581+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 0x7f23800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.581+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 0x7f23800693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f23940cea90 tx=0x7f2388008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 Processor -- start 2023-10-10T20:26:20.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 -- start start 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4d0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 0x55ded4ce0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4c37a70 con 0x55ded4cde5f0 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4c35270 con 0x55ded4ce0b80 2023-10-10T20:26:20.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.583+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded4c31060 con 0x55ded4d2b480 2023-10-10T20:26:20.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46406/0 (socket says 172.21.15.148:46406) 2023-10-10T20:26:20.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 -- 172.21.15.148:0/2452723774 learned_addr learned my addr 172.21.15.148:0/2452723774 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2393fff700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 0x55ded4ce0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f23937fe700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4d0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 -- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 msgr2=0x55ded4d0a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4d0a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 -- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 msgr2=0x55ded4ce0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 0x55ded4ce0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 -- 172.21.15.148:0/2452723774 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded4acb270 con 0x55ded4cde5f0 2023-10-10T20:26:20.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2393fff700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 0x55ded4ce0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:26:20.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f2388014eb0 tx=0x7f2388019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8336f8e70f4479e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2378ff9700 1 -- 172.21.15.148:0/2452723774 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238801c040 con 0x55ded4cde5f0 2023-10-10T20:26:20.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2378ff9700 1 -- 172.21.15.148:0/2452723774 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388016db0 con 0x55ded4cde5f0 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.584+0000 7f2378ff9700 1 -- 172.21.15.148:0/2452723774 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388024670 con 0x55ded4cde5f0 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 -- 172.21.15.148:0/2452723774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 msgr2=0x55ded4d2b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f2388014eb0 tx=0x7f2388019000 comp rx=0 tx=0).stop 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 -- 172.21.15.148:0/2452723774 shutdown_connections 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4ce0b80 0x55ded4ce0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4d0a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2452723774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4cde5f0 0x55ded4d2b850 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 -- 172.21.15.148:0/2452723774 >> 172.21.15.148:0/2452723774 conn(0x55ded4ce20c0 msgr2=0x55ded4ce28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 -- 172.21.15.148:0/2452723774 shutdown_connections 2023-10-10T20:26:20.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.585+0000 7f23a328fb80 1 -- 172.21.15.148:0/2452723774 wait complete. 2023-10-10T20:26:20.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 -- start start 2023-10-10T20:26:20.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 Processor -- start 2023-10-10T20:26:20.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4cde5f0 0x55ded4d0c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4d2b480 0x55ded4d04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4bdc890 con 0x55ded4ce0b80 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4bf7ea0 con 0x55ded4d2b480 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded4ab3e30 con 0x55ded4cde5f0 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f23a328fb80 10 client.0 _ll_register_callbacks cb 0x55ded4ca42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.586+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46422/0 (socket says 172.21.15.148:46422) 2023-10-10T20:26:20.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 -- 172.21.15.148:0/2522451542 learned_addr learned my addr 172.21.15.148:0/2522451542 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f23937fe700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4cde5f0 0x55ded4d0c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2393fff700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4d2b480 0x55ded4d04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 -- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4cde5f0 msgr2=0x55ded4d0c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4cde5f0 0x55ded4d0c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 -- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4d2b480 msgr2=0x55ded4d04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4d2b480 0x55ded4d04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 -- 172.21.15.148:0/2522451542 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388014ab0 con 0x55ded4ce0b80 2023-10-10T20:26:20.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2388008010 tx=0x7f238801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab24357c7fd8e83a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:26:20.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388009ab0 con 0x55ded4ce0b80 2023-10-10T20:26:20.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2388005c10 con 0x55ded4ce0b80 2023-10-10T20:26:20.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ded4cd5cd0 con 0x55ded4ce0b80 2023-10-10T20:26:20.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2388005e10 con 0x55ded4ce0b80 2023-10-10T20:26:20.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f23a328fb80 10 client.7605 fetch_fsmap learned FSMap version 71 2023-10-10T20:26:20.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ded4a6dfe0 con 0x55ded4ce0b80 2023-10-10T20:26:20.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ded4cd5cd0 con 0x55ded4ce0b80 2023-10-10T20:26:20.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2388022040 con 0x55ded4ce0b80 2023-10-10T20:26:20.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.587+0000 7f23a328fb80 10 client.7605 fetch_fsmap finished waiting for FSMap version 71 2023-10-10T20:26:20.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 10 client.7605 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:26:20.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 20 client.7605 populate_metadata read hostname 'smithi148' 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d09340 0x55ded4d09710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 4 client.7605 mds_command: new command op to 7470 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 --> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ded4cb2aa0 con 0x55ded4d09340 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23937fe700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d09340 0x55ded4d09710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f2371ffb700 10 client.7605 ms_handle_connect on v2:172.21.15.148:6835/2641217056 2023-10-10T20:26:20.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23937fe700 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d09340 0x55ded4d09710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17396ca7195f93f server_cookie=d72ff087bbf48c88 in_seq=0 out_seq=0 2023-10-10T20:26:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f2371ffb700 1 -- 172.21.15.148:0/2522451542 <== mds.0 v2:172.21.15.148:6835/2641217056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ded4cb2aa0 con 0x55ded4d09340 2023-10-10T20:26:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f2371ffb700 10 client.7605 handle_command_reply: tid=0 2023-10-10T20:26:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.588+0000 7f23a328fb80 1 client.7605 shutdown 2023-10-10T20:26:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 msgr2=0x55ded4d0be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2388008010 tx=0x7f238801e000 comp rx=0 tx=0).stop 2023-10-10T20:26:20.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 shutdown_connections 2023-10-10T20:26:20.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d09340 0x55ded4d09710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4d2b480 0x55ded4d04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4cde5f0 0x55ded4d0c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f2371ffb700 0 client.7605 ms_handle_reset on v2:172.21.15.148:6835/2641217056 2023-10-10T20:26:20.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2522451542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4d0be10 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 >> 172.21.15.148:0/2522451542 conn(0x55ded4ce20c0 msgr2=0x55ded4ce4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 shutdown_connections 2023-10-10T20:26:20.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 1 -- 172.21.15.148:0/2522451542 wait complete. 2023-10-10T20:26:20.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.589+0000 7f23a328fb80 20 client.7605 trim_cache size 0 max 16384 2023-10-10T20:26:20.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.605+0000 7f23a328fb80 1 Processor -- start 2023-10-10T20:26:20.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.605+0000 7f23a328fb80 1 -- start start 2023-10-10T20:26:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4d0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4cdce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4cd7040 con 0x55ded4ce0b80 2023-10-10T20:26:20.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4d08da0 con 0x55ded4cde5f0 2023-10-10T20:26:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded4cb9410 con 0x55ded4d2b480 2023-10-10T20:26:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46434/0 (socket says 172.21.15.148:46434) 2023-10-10T20:26:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 -- 172.21.15.148:0/3346241605 learned_addr learned my addr 172.21.15.148:0/3346241605 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f23937fe700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4d0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2393fff700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4cdce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 -- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 msgr2=0x55ded4cdce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4cdce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 -- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 msgr2=0x55ded4d0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4d0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2392ffd700 1 -- 172.21.15.148:0/3346241605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ded4bff780 con 0x55ded4ce0b80 2023-10-10T20:26:20.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.606+0000 7f2393fff700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4cdce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:20.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f2392ffd700 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2388014d20 tx=0x7f2388032070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e554f4e2c3edb6e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23797fa700 1 -- 172.21.15.148:0/3346241605 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388006320 con 0x55ded4ce0b80 2023-10-10T20:26:20.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23797fa700 1 -- 172.21.15.148:0/3346241605 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388016aa0 con 0x55ded4ce0b80 2023-10-10T20:26:20.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23797fa700 1 -- 172.21.15.148:0/3346241605 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388016c10 con 0x55ded4ce0b80 2023-10-10T20:26:20.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 -- 172.21.15.148:0/3346241605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 msgr2=0x55ded4cd7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2388014d20 tx=0x7f2388032070 comp rx=0 tx=0).stop 2023-10-10T20:26:20.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 -- 172.21.15.148:0/3346241605 shutdown_connections 2023-10-10T20:26:20.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4d2b480 0x55ded4cdce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4d0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 --2- 172.21.15.148:0/3346241605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4ce0b80 0x55ded4cd7990 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 -- 172.21.15.148:0/3346241605 >> 172.21.15.148:0/3346241605 conn(0x55ded4ce20c0 msgr2=0x55ded4d06410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 -- 172.21.15.148:0/3346241605 shutdown_connections 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.607+0000 7f23a328fb80 1 -- 172.21.15.148:0/3346241605 wait complete. 2023-10-10T20:26:20.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.608+0000 7f23a328fb80 1 -- start start 2023-10-10T20:26:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.608+0000 7f23a328fb80 1 Processor -- start 2023-10-10T20:26:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4cdda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4ce0b80 0x55ded4cddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4d391f0 con 0x55ded4d2b480 2023-10-10T20:26:20.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ded4d8d100 con 0x55ded4cde5f0 2023-10-10T20:26:20.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ded4d8d350 con 0x55ded4ce0b80 2023-10-10T20:26:20.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 10 client.0 _ll_register_callbacks cb 0x55ded4ca42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:26:20.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46450/0 (socket says 172.21.15.148:46450) 2023-10-10T20:26:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 -- 172.21.15.148:0/2144831408 learned_addr learned my addr 172.21.15.148:0/2144831408 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23937fe700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4cdda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2392ffd700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4ce0b80 0x55ded4cddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 -- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4ce0b80 msgr2=0x55ded4cddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4ce0b80 0x55ded4cddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 -- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 msgr2=0x55ded4cdda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4cdda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 -- 172.21.15.148:0/2144831408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ded4a6dfe0 con 0x55ded4d2b480 2023-10-10T20:26:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2393fff700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f23940cb610 tx=0x7f238c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98998de1b0001167 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:26:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00ce40 con 0x55ded4d2b480 2023-10-10T20:26:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f238c000ed0 con 0x55ded4d2b480 2023-10-10T20:26:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2388016db0 con 0x55ded4d2b480 2023-10-10T20:26:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f238c005a90 con 0x55ded4d2b480 2023-10-10T20:26:20.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 10 client.7611 fetch_fsmap learned FSMap version 71 2023-10-10T20:26:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ded4be6470 con 0x55ded4d2b480 2023-10-10T20:26:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.609+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388016db0 con 0x55ded4d2b480 2023-10-10T20:26:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f238c005bf0 con 0x55ded4d2b480 2023-10-10T20:26:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 10 client.7611 fetch_fsmap finished waiting for FSMap version 71 2023-10-10T20:26:20.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 10 client.7611 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 20 client.7611 populate_metadata read hostname 'smithi148' 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d0b430 0x55ded4d0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 4 client.7611 mds_command: new command op to 7470 tid=0[{"prefix": "status"}] 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 --> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ded4bd3e00 con 0x55ded4d0b430 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23937fe700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d0b430 0x55ded4d0ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f2371ffb700 10 client.7611 ms_handle_connect on v2:172.21.15.148:6835/2641217056 2023-10-10T20:26:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23937fe700 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d0b430 0x55ded4d0ccb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566b7407f771018e server_cookie=d4a561089878d091 in_seq=0 out_seq=0 2023-10-10T20:26:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f2371ffb700 1 -- 172.21.15.148:0/2144831408 <== mds.0 v2:172.21.15.148:6835/2641217056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ded4bd3e00 con 0x55ded4d0b430 2023-10-10T20:26:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f2371ffb700 10 client.7611 handle_command_reply: tid=0 2023-10-10T20:26:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 client.7611 shutdown 2023-10-10T20:26:20.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 msgr2=0x55ded4d0ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f23940cb610 tx=0x7f238c013000 comp rx=0 tx=0).stop 2023-10-10T20:26:20.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 shutdown_connections 2023-10-10T20:26:20.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ded4d2b480 0x55ded4d0ad10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.148:6835/2641217056,v1:172.21.15.148:6837/2641217056] conn(0x55ded4d0b430 0x55ded4d0ccb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ded4cde5f0 0x55ded4cdda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 --2- 172.21.15.148:0/2144831408 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ded4ce0b80 0x55ded4cddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f2371ffb700 0 client.7611 ms_handle_reset on v2:172.21.15.148:6835/2641217056 2023-10-10T20:26:20.642 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 >> 172.21.15.148:0/2144831408 conn(0x55ded4ce20c0 msgr2=0x55ded4d0bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 shutdown_connections 2023-10-10T20:26:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.610+0000 7f23a328fb80 1 -- 172.21.15.148:0/2144831408 wait complete. 2023-10-10T20:26:20.643 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f23a328fb80 20 client.7611 trim_cache size 0 max 16384 2023-10-10T20:26:20.643 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:26:20.644 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:26:20.644 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:26:20.644 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 7470, 2023-10-10T20:26:20.644 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:26:20.644 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.8119846050000001, 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 71, 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 146, 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 146, 2023-10-10T20:26:20.645 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.919767088 2023-10-10T20:26:20.646 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:26:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 msgr2=0x7f23800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 0x7f23800693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f23940cea90 tx=0x7f2388008040 comp rx=0 tx=0).stop 2023-10-10T20:26:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 msgr2=0x7f2394157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f238400ebc0 tx=0x7f2384001670 comp rx=0 tx=0).stop 2023-10-10T20:26:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 shutdown_connections 2023-10-10T20:26:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2380066f30 0x7f23800693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.648 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23940cee30 0x7f23940cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237c0052e0 0x7f2394157620 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 --2- 172.21.15.148:0/2300694474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23940ca150 0x7f23940ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 >> 172.21.15.148:0/2300694474 conn(0x7f23940d0bc0 msgr2=0x7f23940d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 shutdown_connections 2023-10-10T20:26:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.611+0000 7f2398d34700 1 -- 172.21.15.148:0/2300694474 wait complete. 2023-10-10T20:26:20.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:26:20.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.762+0000 7f834b9c0700 1 Processor -- start 2023-10-10T20:26:20.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.762+0000 7f834b9c0700 1 -- start start 2023-10-10T20:26:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f834b9c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f834415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f834b9c0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834415cbb0 con 0x7f8344157c40 2023-10-10T20:26:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f834b9c0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834415bfa0 con 0x7f834415a620 2023-10-10T20:26:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f834b9c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834415c280 con 0x7f8344157870 2023-10-10T20:26:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f8349f5d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8344157c40 0x7f8344158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57774/0 (socket says 172.21.15.148:57774) 2023-10-10T20:26:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f8349f5d700 1 -- 172.21.15.148:0/959670 learned_addr learned my addr 172.21.15.148:0/959670 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377690301 0 0) 0x7f834415cbb0 con 0x7f8344157c40 2023-10-10T20:26:20.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8330003570 con 0x7f8344157c40 2023-10-10T20:26:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f834975c700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f834415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656239031 0 0) 0x7f8330003570 con 0x7f8344157c40 2023-10-10T20:26:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 msgr2=0x7f834415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f834415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 >> v1:172.21.15.161:6789/0 conn(0x7f834415a620 legacy=0x7f834415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834415c560 con 0x7f8344157c40 2023-10-10T20:26:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f833c001470 con 0x7f8344157c40 2023-10-10T20:26:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 >> v1:172.21.15.148:6789/0 conn(0x7f8344157c40 legacy=0x7f8344158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f8330003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f8330004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.763+0000 7f833bfff700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8330004d50 0x7f83300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834415c280 con 0x7f8330004010 2023-10-10T20:26:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833000bbf0 con 0x7f8330004d50 2023-10-10T20:26:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833000bdb0 con 0x7f8344157870 2023-10-10T20:26:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f8330004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f834975c700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f8330003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8349f5d700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8330004d50 0x7f83300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 -- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 msgr2=0x7f8330003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8344157870 0x7f8330003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 -- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8330004d50 msgr2=0x7f83300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:20.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8330004d50 0x7f83300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 -- 172.21.15.148:0/959670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8330003570 con 0x7f8330004010 2023-10-10T20:26:20.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f8348f5b700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f8330004810 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8334001880 tx=0x7f8334007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d2bfc8ff2c17 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334012420 con 0x7f8330004010 2023-10-10T20:26:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334012580 con 0x7f8330004010 2023-10-10T20:26:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.764+0000 7f833bfff700 1 -- 172.21.15.148:0/959670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334012920 con 0x7f8330004010 2023-10-10T20:26:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 -- 172.21.15.148:0/959670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 msgr2=0x7f8330004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 --2- 172.21.15.148:0/959670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f8330004810 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8334001880 tx=0x7f8334007d30 comp rx=0 tx=0).stop 2023-10-10T20:26:20.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f8349f5d700 1 -- 172.21.15.148:0/959670 reap_dead start 2023-10-10T20:26:20.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 -- 172.21.15.148:0/959670 shutdown_connections 2023-10-10T20:26:20.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 -- 172.21.15.148:0/959670 >> 172.21.15.148:0/959670 conn(0x7f8344068ce0 msgr2=0x7f83440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:20.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 -- 172.21.15.148:0/959670 shutdown_connections 2023-10-10T20:26:20.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.765+0000 7f834b9c0700 1 -- 172.21.15.148:0/959670 wait complete. 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.766+0000 7f834b9c0700 1 Processor -- start 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.766+0000 7f834b9c0700 1 -- start start 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 0x7f8344150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f834414c7f0 0x7f8344153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440b2c20 con 0x7f8330004010 2023-10-10T20:26:20.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440b2610 con 0x7f834414cf90 2023-10-10T20:26:20.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8344151ec0 con 0x7f834414c7f0 2023-10-10T20:26:20.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46472/0 (socket says 172.21.15.148:46472) 2023-10-10T20:26:20.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 -- 172.21.15.148:0/485157309 learned_addr learned my addr 172.21.15.148:0/485157309 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:20.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f8349f5d700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f834414c7f0 0x7f8344153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f8348f5b700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 0x7f8344150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 -- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f834414c7f0 msgr2=0x7f8344153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f834414c7f0 0x7f8344153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 -- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 msgr2=0x7f8344150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:20.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 0x7f8344150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:20.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 -- 172.21.15.148:0/485157309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344151900 con 0x7f8330004010 2023-10-10T20:26:20.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f8348f5b700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 0x7f8344150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:26:20.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834975c700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8340011f80 tx=0x7f834000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834000e040 con 0x7f8330004010 2023-10-10T20:26:20.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834000f040 con 0x7f8330004010 2023-10-10T20:26:20.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.767+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8344151be0 con 0x7f8330004010 2023-10-10T20:26:20.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.768+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340015430 con 0x7f8330004010 2023-10-10T20:26:20.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.768+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834414cc90 con 0x7f8330004010 2023-10-10T20:26:20.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.768+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83440b2470 con 0x7f8330004010 2023-10-10T20:26:20.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.769+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83400157e0 con 0x7f8330004010 2023-10-10T20:26:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.769+0000 7f83397fa700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 0x7f83280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.769+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83400852b0 con 0x7f8330004010 2023-10-10T20:26:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.769+0000 7f8348f5b700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 0x7f83280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:20.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.770+0000 7f8348f5b700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 0x7f83280693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f83440afa70 tx=0x7f8334015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:20.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:20.774+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83440b2470 con 0x7f8330004010 2023-10-10T20:26:21.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.001+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83440b2470 con 0x7f8330004010 2023-10-10T20:26:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.002+0000 7f83397fa700 1 -- 172.21.15.148:0/485157309 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f83440b2470 con 0x7f8330004010 2023-10-10T20:26:21.004 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:26:21.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 msgr2=0x7f83280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 0x7f83280693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f83440afa70 tx=0x7f8334015000 comp rx=0 tx=0).stop 2023-10-10T20:26:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 msgr2=0x7f83440b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8340011f80 tx=0x7f834000b040 comp rx=0 tx=0).stop 2023-10-10T20:26:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 shutdown_connections 2023-10-10T20:26:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8328066f30 0x7f83280693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f834414c7f0 0x7f8344153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8330004010 0x7f83440b30b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 --2- 172.21.15.148:0/485157309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f834414cf90 0x7f8344150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 >> 172.21.15.148:0/485157309 conn(0x7f8344068ce0 msgr2=0x7f83440b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 shutdown_connections 2023-10-10T20:26:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.004+0000 7f834b9c0700 1 -- 172.21.15.148:0/485157309 wait complete. 2023-10-10T20:26:21.018 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:26:21.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:26:21.019 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:26:21.019 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:26:21.019 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:26:21.020 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:26:21.020 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:26:21.020 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:26:21.021 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:26:21.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.022 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:26:21.037 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:26:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:26:21.037 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:26:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:26:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.038 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:21.038 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:26:21.039 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:21.039 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:26:21.039 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42130sec preferred_lft 42130sec 2023-10-10T20:26:21.040 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:26:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.041 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:26:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:26:21.041 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:26:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.042 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:26:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:26:21.042 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:26:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.043 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:26:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:26:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:26:21.043 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:21.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.044 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:26:21.100 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:26:21.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.102 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:26:21.157 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:26:21.157 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:26:21.158 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:26:21.158 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:26:21.158 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:26:21.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.159 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:26:21.159 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:26:21.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:26:21.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:26:21.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:26:21.161 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:26:21.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.259 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:26:21.274 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:26:21.275 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:26:21.275 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:26:21.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.278 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:26:21.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:26:21.340 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:26:21.395 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:26:21.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:26:21.396 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:26:21.450 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:26:21.450 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:26:21.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:21.529 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:21.530 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:21.548 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:21.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:21.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:26:21.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:26:21.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:26:21.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:26:21.550 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:26:21.550 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:26:21.570 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:21.571 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:21.571 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:26:21.572 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:26:21.572 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:26:21.572 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:26:21.572 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:26:21.573 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:26:21.573 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:26:21.573 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:26:21.573 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:26:21.574 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:26:21.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:21.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:26:21.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:26:21.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.792+0000 7f9529a32700 1 Processor -- start 2023-10-10T20:26:21.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.792+0000 7f9529a32700 1 -- start start 2023-10-10T20:26:21.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9529a32700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9524157870 0x7f952415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9529a32700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9524157c40 0x7f9524158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9529a32700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952415cbb0 con 0x7f952415a620 2023-10-10T20:26:21.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9529a32700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952415bfa0 con 0x7f9524157c40 2023-10-10T20:26:21.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9529a32700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952415c280 con 0x7f9524157870 2023-10-10T20:26:21.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f95237fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9524157c40 0x7f9524158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:21.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f951a7fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f952415a620 0x7f952415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57782/0 (socket says 172.21.15.148:57782) 2023-10-10T20:26:21.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f951a7fc700 1 -- 172.21.15.148:0/1416117789 learned_addr learned my addr 172.21.15.148:0/1416117789 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:21.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f9522ffd700 1 --2- 172.21.15.148:0/1416117789 >> v2:172.21.15.161:3300/0 conn(0x7f9524157870 0x7f952415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f95237fe700 1 -- 172.21.15.148:0/1416117789 >> v2:172.21.15.161:3300/0 conn(0x7f9524157870 msgr2=0x7f952415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f95237fe700 1 --2- 172.21.15.148:0/1416117789 >> v2:172.21.15.161:3300/0 conn(0x7f9524157870 0x7f952415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f95237fe700 1 -- 172.21.15.148:0/1416117789 >> v1:172.21.15.148:6789/0 conn(0x7f952415a620 legacy=0x7f952415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.793+0000 7f95237fe700 1 -- 172.21.15.148:0/1416117789 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952415c560 con 0x7f9524157c40 2023-10-10T20:26:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f95237fe700 1 --2- 172.21.15.148:0/1416117789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9524157c40 0x7f9524158110 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9524069d80 tx=0x7f951c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b944b1efb4ec0a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f95227fc700 1 -- 172.21.15.148:0/1416117789 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00e070 con 0x7f9524157c40 2023-10-10T20:26:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f95227fc700 1 -- 172.21.15.148:0/1416117789 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00bc30 con 0x7f9524157c40 2023-10-10T20:26:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f95227fc700 1 -- 172.21.15.148:0/1416117789 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c012430 con 0x7f9524157c40 2023-10-10T20:26:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 -- 172.21.15.148:0/1416117789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9524157c40 msgr2=0x7f9524158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 --2- 172.21.15.148:0/1416117789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9524157c40 0x7f9524158110 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9524069d80 tx=0x7f951c009780 comp rx=0 tx=0).stop 2023-10-10T20:26:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 -- 172.21.15.148:0/1416117789 shutdown_connections 2023-10-10T20:26:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 --2- 172.21.15.148:0/1416117789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9524157c40 0x7f9524158110 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 --2- 172.21.15.148:0/1416117789 >> v2:172.21.15.161:3300/0 conn(0x7f9524157870 0x7f952415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 -- 172.21.15.148:0/1416117789 >> 172.21.15.148:0/1416117789 conn(0x7f9524068ce0 msgr2=0x7f95240690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:21.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 -- 172.21.15.148:0/1416117789 shutdown_connections 2023-10-10T20:26:21.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.794+0000 7f9529a32700 1 -- 172.21.15.148:0/1416117789 wait complete. 2023-10-10T20:26:21.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.795+0000 7f9529a32700 1 Processor -- start 2023-10-10T20:26:21.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 -- start start 2023-10-10T20:26:21.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9524157870 0x7f952413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9524157c40 0x7f952414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240b1400 con 0x7f952415a620 2023-10-10T20:26:21.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240b07f0 con 0x7f9524157c40 2023-10-10T20:26:21.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f9529a32700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95240b0ad0 con 0x7f9524157870 2023-10-10T20:26:21.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f95237fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:21.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f95237fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46500/0 (socket says 172.21.15.148:46500) 2023-10-10T20:26:21.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.796+0000 7f95237fe700 1 -- 172.21.15.148:0/3628884770 learned_addr learned my addr 172.21.15.148:0/3628884770 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 -- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9524157870 msgr2=0x7f952413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9524157870 0x7f952413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 -- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9524157c40 msgr2=0x7f952414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9524157c40 0x7f952414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 -- 172.21.15.148:0/3628884770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009a20 con 0x7f952415a620 2023-10-10T20:26:21.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f95237fe700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f951c013000 tx=0x7f951c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:21.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c012e20 con 0x7f952415a620 2023-10-10T20:26:21.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c01e630 con 0x7f952415a620 2023-10-10T20:26:21.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95240b0db0 con 0x7f952415a620 2023-10-10T20:26:21.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c01e9d0 con 0x7f952415a620 2023-10-10T20:26:21.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.797+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952414ff20 con 0x7f952415a620 2023-10-10T20:26:21.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.798+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952404c3c0 con 0x7f952415a620 2023-10-10T20:26:21.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.798+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f951c024070 con 0x7f952415a620 2023-10-10T20:26:21.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.799+0000 7f951b7fe700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 0x7f95040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:21.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.799+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f951c086cc0 con 0x7f952415a620 2023-10-10T20:26:21.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.799+0000 7f9522ffd700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 0x7f95040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:21.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.799+0000 7f9522ffd700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 0x7f95040693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9514009a60 tx=0x7f9514008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:21.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:21.803+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f951c053fe0 con 0x7f952415a620 2023-10-10T20:26:22.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.031+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f95240b1540 con 0x7f952415a620 2023-10-10T20:26:22.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.032+0000 7f951b7fe700 1 -- 172.21.15.148:0/3628884770 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f951c057600 con 0x7f952415a620 2023-10-10T20:26:22.034 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:26:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 msgr2=0x7f95040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 0x7f95040693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9514009a60 tx=0x7f9514008040 comp rx=0 tx=0).stop 2023-10-10T20:26:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 msgr2=0x7f95240b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f951c013000 tx=0x7f951c019000 comp rx=0 tx=0).stop 2023-10-10T20:26:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 shutdown_connections 2023-10-10T20:26:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9504066f30 0x7f95040693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f952415a620 0x7f95240b1890 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9524157870 0x7f952413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 --2- 172.21.15.148:0/3628884770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9524157c40 0x7f952414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.033+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 >> 172.21.15.148:0/3628884770 conn(0x7f9524068ce0 msgr2=0x7f95240b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.034+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 shutdown_connections 2023-10-10T20:26:22.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.034+0000 7f9529a32700 1 -- 172.21.15.148:0/3628884770 wait complete. 2023-10-10T20:26:22.048 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:26:22.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:26:22.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:26:22.049 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:26:22.049 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:26:22.049 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:26:22.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:26:22.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:26:22.050 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:26:22.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.051 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:26:22.070 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:26:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:26:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:26:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:26:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.071 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:26:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:22.073 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:26:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:26:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42125sec preferred_lft 42125sec 2023-10-10T20:26:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:26:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.074 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:26:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:26:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:26:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:26:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.075 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:26:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:26:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:26:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.076 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:26:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:26:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:26:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:26:22.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.077 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:26:22.138 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:26:22.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.140 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:26:22.196 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:26:22.196 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:26:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:26:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:26:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:26:22.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.198 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:26:22.198 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:26:22.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:26:22.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:26:22.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:26:22.200 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:26:22.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.302 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:26:22.317 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:26:22.318 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:26:22.318 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:26:22.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:26:22.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:26:22.384 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:26:22.439 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:26:22.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:26:22.440 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:26:22.493 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:26:22.494 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:26:22.494 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:22.574 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:22.574 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:22.592 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:22.593 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:22.593 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:26:22.593 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:26:22.593 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:26:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:26:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:26:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:26:22.614 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:26:22.615 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:26:22.615 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:26:22.615 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:26:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:26:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:26:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:26:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:26:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:26:22.617 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:26:22.617 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:26:22.618 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:26:22.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:26:22.619 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:26:22.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:26:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.798+0000 7efcd2376700 1 Processor -- start 2023-10-10T20:26:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.798+0000 7efcd2376700 1 -- start start 2023-10-10T20:26:22.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.799+0000 7efcd2376700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efccc159d50 0x7efccc0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.799+0000 7efcd2376700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.799+0000 7efcd2376700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efccc069d90 con 0x7efccc0aee00 2023-10-10T20:26:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.799+0000 7efcd2376700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efccc0ccdd0 con 0x7efccc159d50 2023-10-10T20:26:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.799+0000 7efcd2376700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc069740 con 0x7efccc15a120 2023-10-10T20:26:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efcd0913700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efccc0aee00 0x7efccc0af210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40278/0 (socket says 172.21.15.148:40278) 2023-10-10T20:26:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efcd0913700 1 -- 172.21.15.148:0/1474733023 learned_addr learned my addr 172.21.15.148:0/1474733023 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 --2- 172.21.15.148:0/1474733023 >> v2:172.21.15.161:3300/0 conn(0x7efccc159d50 0x7efccc0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccb7fe700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc06daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccaffd700 1 -- 172.21.15.148:0/1474733023 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039504235 0 0) 0x7efccc069d90 con 0x7efccc0aee00 2023-10-10T20:26:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccaffd700 1 -- 172.21.15.148:0/1474733023 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb4003570 con 0x7efccc0aee00 2023-10-10T20:26:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 -- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 msgr2=0x7efccc06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc06daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 -- 172.21.15.148:0/1474733023 >> v1:172.21.15.148:6789/0 conn(0x7efccc0aee00 legacy=0x7efccc0af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 -- 172.21.15.148:0/1474733023 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc069a20 con 0x7efccc159d50 2023-10-10T20:26:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccb7fe700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccbfff700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc0cc7f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0098e0 tx=0x7efcbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b136706e156131a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccaffd700 1 -- 172.21.15.148:0/1474733023 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc011630 con 0x7efccc159d50 2023-10-10T20:26:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccaffd700 1 -- 172.21.15.148:0/1474733023 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc011790 con 0x7efccc159d50 2023-10-10T20:26:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.800+0000 7efccaffd700 1 -- 172.21.15.148:0/1474733023 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc011b30 con 0x7efccc159d50 2023-10-10T20:26:22.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 -- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 msgr2=0x7efccc0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc0cc7f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0098e0 tx=0x7efcbc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 -- 172.21.15.148:0/1474733023 shutdown_connections 2023-10-10T20:26:22.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc0cc7f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 --2- 172.21.15.148:0/1474733023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 -- 172.21.15.148:0/1474733023 >> 172.21.15.148:0/1474733023 conn(0x7efccc0cd200 msgr2=0x7efccc0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:22.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 -- 172.21.15.148:0/1474733023 shutdown_connections 2023-10-10T20:26:22.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.801+0000 7efcd2376700 1 -- 172.21.15.148:0/1474733023 wait complete. 2023-10-10T20:26:22.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.802+0000 7efcd2376700 1 Processor -- start 2023-10-10T20:26:22.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.802+0000 7efcd2376700 1 -- start start 2023-10-10T20:26:22.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc156840 con 0x7efccc0aee00 2023-10-10T20:26:22.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc098c70 con 0x7efccc159d50 2023-10-10T20:26:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd2376700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0b0d70 con 0x7efccc15a120 2023-10-10T20:26:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36012/0 (socket says 172.21.15.148:36012) 2023-10-10T20:26:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 -- 172.21.15.148:0/3139488851 learned_addr learned my addr 172.21.15.148:0/3139488851 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccb7fe700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efcd0913700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 -- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 msgr2=0x7efccc0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 -- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 msgr2=0x7efccc156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:22.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.803+0000 7efccbfff700 1 -- 172.21.15.148:0/3139488851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009420 con 0x7efccc0aee00 2023-10-10T20:26:22.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efccbfff700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efccc072050 tx=0x7efcbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:22.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc011900 con 0x7efccc0aee00 2023-10-10T20:26:22.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc011a60 con 0x7efccc0aee00 2023-10-10T20:26:22.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc025640 con 0x7efccc0aee00 2023-10-10T20:26:22.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc009420 con 0x7efccc0aee00 2023-10-10T20:26:22.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.804+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc0b1330 con 0x7efccc0aee00 2023-10-10T20:26:22.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.805+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc098a40 con 0x7efccc0aee00 2023-10-10T20:26:22.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.805+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcbc025a30 con 0x7efccc0aee00 2023-10-10T20:26:22.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.805+0000 7efcb3fff700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 0x7efcb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:22.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.805+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efcbc085cc0 con 0x7efccc0aee00 2023-10-10T20:26:22.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.806+0000 7efccb7fe700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 0x7efcb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:22.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.806+0000 7efccb7fe700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 0x7efcb80693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efccc063fb0 tx=0x7efcc00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:22.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:22.810+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efccc098a40 con 0x7efccc0aee00 2023-10-10T20:26:23.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.035+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efccc098a40 con 0x7efccc0aee00 2023-10-10T20:26:23.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.036+0000 7efcb3fff700 1 -- 172.21.15.148:0/3139488851 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7efccc098a40 con 0x7efccc0aee00 2023-10-10T20:26:23.039 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:26:23.039 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:26:23.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 msgr2=0x7efcb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 0x7efcb80693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efccc063fb0 tx=0x7efcc00091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 msgr2=0x7efccc15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efccc072050 tx=0x7efcbc017000 comp rx=0 tx=0).stop 2023-10-10T20:26:23.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 shutdown_connections 2023-10-10T20:26:23.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efcb8066f30 0x7efcb80693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efccc15a120 0x7efccc0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efccc0aee00 0x7efccc15a510 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 --2- 172.21.15.148:0/3139488851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efccc159d50 0x7efccc156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.037+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 >> 172.21.15.148:0/3139488851 conn(0x7efccc0cd200 msgr2=0x7efccc157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:23.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.038+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 shutdown_connections 2023-10-10T20:26:23.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.038+0000 7efcd2376700 1 -- 172.21.15.148:0/3139488851 wait complete. 2023-10-10T20:26:23.043 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:26:23.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-10T20:26:23.186 INFO:teuthology.orchestra.run.smithi148.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":7470,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3753480029999992,"mdsmap_epoch":71,"osdmap_epoch":146,"osdmap_epoch_barrier":146,"uptime":13.48313924} 2023-10-10T20:26:23.186 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 7470, "mdsmap_epoch": 71, "osdmap_epoch": 146, "osdmap_epoch_barrier": 146, "rank_uptime": 8.375348003, "state": "up:active", "uptime": 13.48313924, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:26:23.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-10T20:26:23.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:26:23.283 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:26:23.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.306+0000 7f8f9fee0700 1 Processor -- start 2023-10-10T20:26:23.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.306+0000 7f8f9fee0700 1 -- start start 2023-10-10T20:26:23.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9fee0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f98159e40 0x7f8f9806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9fee0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 0x7f8f9806f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9fee0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9806fb60 con 0x7f8f98070700 2023-10-10T20:26:23.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9fee0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98068880 con 0x7f8f98159e40 2023-10-10T20:26:23.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9fee0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980cd650 con 0x7f8f9815a210 2023-10-10T20:26:23.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9e47d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8f98070700 0x7f8f98067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:40288/0 (socket says 172.21.15.148:40288) 2023-10-10T20:26:23.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9e47d700 1 -- 172.21.15.148:0/1080857761 learned_addr learned my addr 172.21.15.148:0/1080857761 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:26:23.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/1080857761 >> v2:172.21.15.161:3300/0 conn(0x7f8f98159e40 0x7f8f9806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.307+0000 7f8f9d47b700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 0x7f8f9806f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9cc7a700 1 -- 172.21.15.148:0/1080857761 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234729378 0 0) 0x7f8f90001490 con 0x7f8f98070700 2023-10-10T20:26:23.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9cc7a700 1 -- 172.21.15.148:0/1080857761 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f84003570 con 0x7f8f98070700 2023-10-10T20:26:23.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9dc7c700 1 -- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 msgr2=0x7f8f9806f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:26:23.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 0x7f8f9806f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9dc7c700 1 -- 172.21.15.148:0/1080857761 >> v1:172.21.15.148:6789/0 conn(0x7f8f98070700 legacy=0x7f8f98067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9dc7c700 1 -- 172.21.15.148:0/1080857761 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980cd060 con 0x7f8f98159e40 2023-10-10T20:26:23.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9d47b700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 0x7f8f9806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:23.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f98159e40 0x7f8f9806ee30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098e0 tx=0x7f8f880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f8d89c40c6ac8ea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:23.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9cc7a700 1 -- 172.21.15.148:0/1080857761 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011630 con 0x7f8f98159e40 2023-10-10T20:26:23.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9cc7a700 1 -- 172.21.15.148:0/1080857761 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88011790 con 0x7f8f98159e40 2023-10-10T20:26:23.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.308+0000 7f8f9cc7a700 1 -- 172.21.15.148:0/1080857761 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011b30 con 0x7f8f98159e40 2023-10-10T20:26:23.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 -- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f98159e40 msgr2=0x7f8f9806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f98159e40 0x7f8f9806ee30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098e0 tx=0x7f8f880091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 -- 172.21.15.148:0/1080857761 shutdown_connections 2023-10-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f98159e40 0x7f8f9806ee30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/1080857761 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f9815a210 0x7f8f9806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 -- 172.21.15.148:0/1080857761 >> 172.21.15.148:0/1080857761 conn(0x7f8f980ce5c0 msgr2=0x7f8f980ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 -- 172.21.15.148:0/1080857761 shutdown_connections 2023-10-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.309+0000 7f8f9fee0700 1 -- 172.21.15.148:0/1080857761 wait complete. 2023-10-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.310+0000 7f8f9fee0700 1 Processor -- start 2023-10-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.310+0000 7f8f9fee0700 1 -- start start 2023-10-10T20:26:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f98070700 0x7f8f980c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 0x7f8f980b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815af40 con 0x7f8f98159e40 2023-10-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815b220 con 0x7f8f9815a210 2023-10-10T20:26:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9fee0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815b500 con 0x7f8f98070700 2023-10-10T20:26:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36020/0 (socket says 172.21.15.148:36020) 2023-10-10T20:26:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 -- 172.21.15.148:0/2606562594 learned_addr learned my addr 172.21.15.148:0/2606562594 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:26:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9e47d700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 0x7f8f980b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f98070700 0x7f8f980c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 -- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f98070700 msgr2=0x7f8f980c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f98070700 0x7f8f980c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 -- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 msgr2=0x7f8f980b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 0x7f8f980b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 -- 172.21.15.148:0/2606562594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94009420 con 0x7f8f98159e40 2023-10-10T20:26:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9e47d700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 0x7f8f980b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:26:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f9d47b700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400ec50 tx=0x7f8f940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.311+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400c850 con 0x7f8f98159e40 2023-10-10T20:26:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.312+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400c9b0 con 0x7f8f98159e40 2023-10-10T20:26:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.312+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400cd50 con 0x7f8f98159e40 2023-10-10T20:26:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.312+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f88009420 con 0x7f8f98159e40 2023-10-10T20:26:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.312+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f980ca4a0 con 0x7f8f98159e40 2023-10-10T20:26:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.313+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9815b8f0 con 0x7f8f98159e40 2023-10-10T20:26:23.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.313+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f94013680 con 0x7f8f98159e40 2023-10-10T20:26:23.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.314+0000 7f8f8dffb700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 0x7f8f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:26:23.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.314+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 0x7f8f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:26:23.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.314+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f94084830 con 0x7f8f98159e40 2023-10-10T20:26:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.314+0000 7f8f9dc7c700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 0x7f8f7c0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f880091e0 tx=0x7f8f88017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:26:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.318+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f94051aa0 con 0x7f8f98159e40 2023-10-10T20:26:23.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.501+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f8f9815b6c0 con 0x7f8f98159e40 2023-10-10T20:26:23.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.845+0000 7f8f8dffb700 1 -- 172.21.15.148:0/2606562594 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8f94084070 con 0x7f8f98159e40 2023-10-10T20:26:23.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 msgr2=0x7f8f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 0x7f8f7c0693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f880091e0 tx=0x7f8f88017000 comp rx=0 tx=0).stop 2023-10-10T20:26:23.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 msgr2=0x7f8f98157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:26:23.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400ec50 tx=0x7f8f940091a0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 shutdown_connections 2023-10-10T20:26:23.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f7c066f30 0x7f8f7c0693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f9815a210 0x7f8f980b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f98070700 0x7f8f980c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 --2- 172.21.15.148:0/2606562594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f98159e40 0x7f8f98157b20 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:26:23.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 >> 172.21.15.148:0/2606562594 conn(0x7f8f980ce5c0 msgr2=0x7f8f980c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:26:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 shutdown_connections 2023-10-10T20:26:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:26:23.847+0000 7f8f9fee0700 1 -- 172.21.15.148:0/2606562594 wait complete. 2023-10-10T20:26:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-10T20:26:23.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:26:23.993 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:26:23.994 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:26:23.995 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-10T20:26:24.117 INFO:teuthology.orchestra.run.smithi148.stdout:{"mon_tick_interval":"5"} 2023-10-10T20:26:24.117 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-10T20:26:24.118 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-10T20:26:24.118 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:26:24.119 DEBUG:teuthology.orchestra.run.smithi148:> import time 2023-10-10T20:26:24.119 DEBUG:teuthology.orchestra.run.smithi148:> import datetime 2023-10-10T20:26:24.120 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-10-10T20:26:24.120 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:26:24.120 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.121 DEBUG:teuthology.orchestra.run.smithi148:> # Write some buffered data through before going full, all should be well 2023-10-10T20:26:24.121 DEBUG:teuthology.orchestra.run.smithi148:> print("writing some data through which we expect to succeed") 2023-10-10T20:26:24.121 DEBUG:teuthology.orchestra.run.smithi148:> bytes = 0 2023-10-10T20:26:24.122 DEBUG:teuthology.orchestra.run.smithi148:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-10T20:26:24.122 DEBUG:teuthology.orchestra.run.smithi148:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-10T20:26:24.122 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-10T20:26:24.123 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:26:24.123 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-10T20:26:24.123 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.124 DEBUG:teuthology.orchestra.run.smithi148:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-10T20:26:24.124 DEBUG:teuthology.orchestra.run.smithi148:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-10T20:26:24.124 DEBUG:teuthology.orchestra.run.smithi148:> # buffered IO, the error should always only appear from fsync and not 2023-10-10T20:26:24.125 DEBUG:teuthology.orchestra.run.smithi148:> # from write 2023-10-10T20:26:24.125 DEBUG:teuthology.orchestra.run.smithi148:> full = False 2023-10-10T20:26:24.125 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.126 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(32 * 0.9)): 2023-10-10T20:26:24.126 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:26:24.126 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-10T20:26:24.127 DEBUG:teuthology.orchestra.run.smithi148:> print("done writing {0} bytes".format(bytes)) 2023-10-10T20:26:24.127 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.127 DEBUG:teuthology.orchestra.run.smithi148:> # OK, now we should sneak in under the full condition 2023-10-10T20:26:24.128 DEBUG:teuthology.orchestra.run.smithi148:> # due to the time it takes the OSDs to report to the 2023-10-10T20:26:24.128 DEBUG:teuthology.orchestra.run.smithi148:> # mons, and get a successful fsync on our full-making data 2023-10-10T20:26:24.128 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:26:24.129 DEBUG:teuthology.orchestra.run.smithi148:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-10T20:26:24.129 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.129 DEBUG:teuthology.orchestra.run.smithi148:> # buffered write, add more dirty data to the buffer 2023-10-10T20:26:24.130 DEBUG:teuthology.orchestra.run.smithi148:> print("starting buffered write") 2023-10-10T20:26:24.130 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:26:24.130 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(32 * 0.2)): 2023-10-10T20:26:24.131 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:26:24.131 DEBUG:teuthology.orchestra.run.smithi148:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-10T20:26:24.131 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(15.0) 2023-10-10T20:26:24.132 DEBUG:teuthology.orchestra.run.smithi148:> except OSError: 2023-10-10T20:26:24.132 DEBUG:teuthology.orchestra.run.smithi148:> pass; 2023-10-10T20:26:24.132 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.133 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-10T20:26:24.133 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.133 DEBUG:teuthology.orchestra.run.smithi148:> # Wait long enough for a background flush that should fail 2023-10-10T20:26:24.134 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(30) 2023-10-10T20:26:24.134 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.134 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2023-10-10T20:26:24.135 DEBUG:teuthology.orchestra.run.smithi148:> # ...and check that the failed background flush is reflected in fclose 2023-10-10T20:26:24.135 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:26:24.135 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:26:24.136 DEBUG:teuthology.orchestra.run.smithi148:> except OSError: 2023-10-10T20:26:24.136 DEBUG:teuthology.orchestra.run.smithi148:> print("close() returned an error as expected") 2023-10-10T20:26:24.136 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:26:24.137 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("close() failed to raise error") 2023-10-10T20:26:24.137 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:26:24.137 DEBUG:teuthology.orchestra.run.smithi148:> # The kernel cephfs client does not raise errors on fclose 2023-10-10T20:26:24.138 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:26:24.138 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:26:24.138 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-10T20:26:24.139 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:26:53.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:26:53.356 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:27:23.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:27:23.427 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:27:53.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:27:53.491 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:28:23.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:28:23.580 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:28:24.833 INFO:teuthology.orchestra.run.smithi148.stdout:writing some data through which we expect to succeed 2023-10-10T20:28:24.833 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-10T20:28:24.833 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-10T20:28:24.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-10T20:28:24.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-10T20:28:24.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-10T20:28:24.834 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-10T20:28:24.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-10T20:28:24.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-10T20:28:24.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-10T20:28:24.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-10T20:28:24.835 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-10T20:28:24.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-10T20:28:24.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-10T20:28:24.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-10T20:28:24.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-10T20:28:24.836 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-10T20:28:24.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-10T20:28:24.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-10T20:28:24.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-10T20:28:24.837 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-10T20:28:24.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-10T20:28:24.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-10T20:28:24.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-10T20:28:24.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-10T20:28:24.838 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-10T20:28:24.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-10T20:28:24.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-10T20:28:24.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-10T20:28:24.839 INFO:teuthology.orchestra.run.smithi148.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-10T20:28:24.839 INFO:teuthology.orchestra.run.smithi148.stdout:done writing 29884416 bytes 2023-10-10T20:28:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:successfully fsync'ed prior to getting full state reported 2023-10-10T20:28:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:starting buffered write 2023-10-10T20:28:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.840 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.841 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.841 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.841 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:28:24.841 INFO:teuthology.orchestra.run.smithi148.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-10T20:28:24.960 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:28:24.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:28:25.008 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:28:25.051 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:28:25.051 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:28:25.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:28:25.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:28:25.097 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:28:25.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:25.098 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:28:25.154 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:28:25.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:28:25.155 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:28:25.207 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:28:25.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:25.208 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:28:25.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:25.346 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.344+0000 7fc75366d700 1 Processor -- start 2023-10-10T20:28:25.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.344+0000 7fc75366d700 1 -- start start 2023-10-10T20:28:25.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc75366d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 0x7fc74c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc75366d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 0x7fc74c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc75366d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c15cbb0 con 0x7fc74c15a620 2023-10-10T20:28:25.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc75366d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c15bfa0 con 0x7fc74c157870 2023-10-10T20:28:25.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc75366d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c15c280 con 0x7fc74c157c40 2023-10-10T20:28:25.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc751c0a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 0x7fc74c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.350 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc751409700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 0x7fc74c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc750c08700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc74c15a620 0x7fc74c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:48558/0 (socket says 172.21.15.148:48558) 2023-10-10T20:28:25.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.345+0000 7fc750c08700 1 -- 172.21.15.148:0/2870217937 learned_addr learned my addr 172.21.15.148:0/2870217937 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:25.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751c0a700 1 -- 172.21.15.148:0/2870217937 >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 msgr2=0x7fc74c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:25.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751c0a700 1 --2- 172.21.15.148:0/2870217937 >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 0x7fc74c15b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.351 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751c0a700 1 -- 172.21.15.148:0/2870217937 >> v1:172.21.15.161:6789/0 conn(0x7fc74c15a620 legacy=0x7fc74c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751c0a700 1 -- 172.21.15.148:0/2870217937 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c15c560 con 0x7fc74c157c40 2023-10-10T20:28:25.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc743fff700 1 -- 172.21.15.148:0/2870217937 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031987135 0 0) 0x7fc74c15cbb0 con 0x7fc74c15a620 2023-10-10T20:28:25.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751409700 1 --2- 172.21.15.148:0/2870217937 >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 0x7fc74c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:25.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc751c0a700 1 --2- 172.21.15.148:0/2870217937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 0x7fc74c158110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7440098e0 tx=0x7fc7440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b102bf679529d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.352 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc743fff700 1 -- 172.21.15.148:0/2870217937 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400b640 con 0x7fc74c157c40 2023-10-10T20:28:25.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc743fff700 1 -- 172.21.15.148:0/2870217937 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400b7a0 con 0x7fc74c157c40 2023-10-10T20:28:25.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.346+0000 7fc743fff700 1 -- 172.21.15.148:0/2870217937 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400bb40 con 0x7fc74c157c40 2023-10-10T20:28:25.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 -- 172.21.15.148:0/2870217937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 msgr2=0x7fc74c158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 --2- 172.21.15.148:0/2870217937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 0x7fc74c158110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7440098e0 tx=0x7fc7440091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.353 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 -- 172.21.15.148:0/2870217937 shutdown_connections 2023-10-10T20:28:25.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 --2- 172.21.15.148:0/2870217937 >> v2:172.21.15.161:3300/0 conn(0x7fc74c157870 0x7fc74c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 --2- 172.21.15.148:0/2870217937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c157c40 0x7fc74c158110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 -- 172.21.15.148:0/2870217937 >> 172.21.15.148:0/2870217937 conn(0x7fc74c068ce0 msgr2=0x7fc74c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:25.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 -- 172.21.15.148:0/2870217937 shutdown_connections 2023-10-10T20:28:25.354 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.347+0000 7fc75366d700 1 -- 172.21.15.148:0/2870217937 wait complete. 2023-10-10T20:28:25.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.348+0000 7fc75366d700 1 Processor -- start 2023-10-10T20:28:25.357 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.348+0000 7fc75366d700 1 -- start start 2023-10-10T20:28:25.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc74c157c40 0x7fc74c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.358 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c15a620 0x7fc74c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0b25a0 con 0x7fc74c157870 2023-10-10T20:28:25.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0b2880 con 0x7fc74c157c40 2023-10-10T20:28:25.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc75366d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0b2b60 con 0x7fc74c15a620 2023-10-10T20:28:25.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45342/0 (socket says 172.21.15.148:45342) 2023-10-10T20:28:25.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 -- 172.21.15.148:0/1005116516 learned_addr learned my addr 172.21.15.148:0/1005116516 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:25.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc750c08700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc74c157c40 0x7fc74c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751c0a700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c15a620 0x7fc74c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 -- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c15a620 msgr2=0x7fc74c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c15a620 0x7fc74c15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 -- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc74c157c40 msgr2=0x7fc74c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc74c157c40 0x7fc74c1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc748009a20 con 0x7fc74c157870 2023-10-10T20:28:25.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc751409700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c170 tx=0x7fc74800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.349+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800ccb0 con 0x7fc74c157870 2023-10-10T20:28:25.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.350+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800f040 con 0x7fc74c157870 2023-10-10T20:28:25.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.350+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc744009420 con 0x7fc74c157870 2023-10-10T20:28:25.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.350+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748014480 con 0x7fc74c157870 2023-10-10T20:28:25.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.350+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c153ad0 con 0x7fc74c157870 2023-10-10T20:28:25.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.351+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c16a540 con 0x7fc74c157870 2023-10-10T20:28:25.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.351+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7480147f0 con 0x7fc74c157870 2023-10-10T20:28:25.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.351+0000 7fc7417fa700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 0x7fc7300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.351+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc7480852f0 con 0x7fc74c157870 2023-10-10T20:28:25.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.351+0000 7fc750c08700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 0x7fc7300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.352+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7fc73006eb10 con 0x7fc74c157870 2023-10-10T20:28:25.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.352+0000 7fc750c08700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 0x7fc7300693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc74c15f2f0 tx=0x7fc73c007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.356+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc74805b020 con 0x7fc74c157870 2023-10-10T20:28:25.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.580+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc74c15a2d0 con 0x7fc74c157870 2023-10-10T20:28:25.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.580+0000 7fc7417fa700 1 -- 172.21.15.148:0/1005116516 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fc74805b020 con 0x7fc74c157870 2023-10-10T20:28:25.583 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:25.583 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:28:25.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.582+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 msgr2=0x7fc7300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.582+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 0x7fc7300693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc74c15f2f0 tx=0x7fc73c007f50 comp rx=0 tx=0).stop 2023-10-10T20:28:25.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.582+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 msgr2=0x7fc74c153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.582+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c170 tx=0x7fc74800b040 comp rx=0 tx=0).stop 2023-10-10T20:28:25.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 shutdown_connections 2023-10-10T20:28:25.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc730066f30 0x7fc7300693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc74c15a620 0x7fc74c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc74c157870 0x7fc74c153590 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 --2- 172.21.15.148:0/1005116516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc74c157c40 0x7fc74c1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 >> 172.21.15.148:0/1005116516 conn(0x7fc74c068ce0 msgr2=0x7fc74c1580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:25.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 shutdown_connections 2023-10-10T20:28:25.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.583+0000 7fc75366d700 1 -- 172.21.15.148:0/1005116516 wait complete. 2023-10-10T20:28:25.588 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:28:25.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:25.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.710+0000 7f31c9757700 1 Processor -- start 2023-10-10T20:28:25.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.710+0000 7f31c9757700 1 -- start start 2023-10-10T20:28:25.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.711+0000 7f31c9757700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31c4157870 0x7f31c415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.711+0000 7f31c9757700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.711+0000 7f31c9757700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c415cbb0 con 0x7f31c415d580 2023-10-10T20:28:25.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.711+0000 7f31c9757700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c415bfa0 con 0x7f31c4157870 2023-10-10T20:28:25.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.711+0000 7f31c9757700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c415c280 con 0x7f31c415a620 2023-10-10T20:28:25.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31c4157870 0x7f31c415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f31c415d580 0x7f31c4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48340/0 (socket says 172.21.15.148:48340) 2023-10-10T20:28:25.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c37fe700 1 -- 172.21.15.148:0/813449397 learned_addr learned my addr 172.21.15.148:0/813449397 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:25.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31bbfff700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c27fc700 1 -- 172.21.15.148:0/813449397 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801639609 0 0) 0x7f31c415cbb0 con 0x7f31c415d580 2023-10-10T20:28:25.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 -- 172.21.15.148:0/813449397 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 msgr2=0x7f31c415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:25.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 -- 172.21.15.148:0/813449397 >> v1:172.21.15.148:6789/0 conn(0x7f31c415d580 legacy=0x7f31c4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 -- 172.21.15.148:0/813449397 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c415c560 con 0x7f31c4157870 2023-10-10T20:28:25.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31bbfff700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:25.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c2ffd700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c4157870 0x7f31c415b6a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f31b40098e0 tx=0x7f31b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5552859be472355 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c27fc700 1 -- 172.21.15.148:0/813449397 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4009e80 con 0x7f31c4157870 2023-10-10T20:28:25.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c27fc700 1 -- 172.21.15.148:0/813449397 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b400b860 con 0x7f31c4157870 2023-10-10T20:28:25.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.712+0000 7f31c27fc700 1 -- 172.21.15.148:0/813449397 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b400bbe0 con 0x7f31c4157870 2023-10-10T20:28:25.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 -- 172.21.15.148:0/813449397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c4157870 msgr2=0x7f31c415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c4157870 0x7f31c415b6a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f31b40098e0 tx=0x7f31b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 -- 172.21.15.148:0/813449397 shutdown_connections 2023-10-10T20:28:25.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c4157870 0x7f31c415b6a0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 --2- 172.21.15.148:0/813449397 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 -- 172.21.15.148:0/813449397 >> 172.21.15.148:0/813449397 conn(0x7f31c4068ce0 msgr2=0x7f31c40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:25.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 -- 172.21.15.148:0/813449397 shutdown_connections 2023-10-10T20:28:25.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.713+0000 7f31c9757700 1 -- 172.21.15.148:0/813449397 wait complete. 2023-10-10T20:28:25.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.714+0000 7f31c9757700 1 Processor -- start 2023-10-10T20:28:25.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.714+0000 7f31c9757700 1 -- start start 2023-10-10T20:28:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31c4157870 0x7f31c40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c40b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c416a430 con 0x7f31c415d580 2023-10-10T20:28:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40b2fa0 con 0x7f31c4157870 2023-10-10T20:28:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c9757700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40b3550 con 0x7f31c415a620 2023-10-10T20:28:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45360/0 (socket says 172.21.15.148:45360) 2023-10-10T20:28:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 -- 172.21.15.148:0/4051595785 learned_addr learned my addr 172.21.15.148:0/4051595785 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31bbfff700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c40b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c2ffd700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31c4157870 0x7f31c40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 -- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 msgr2=0x7f31c40b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c40b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 -- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31c4157870 msgr2=0x7f31c40c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.715+0000 7f31c37fe700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31c4157870 0x7f31c40c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31c37fe700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4009420 con 0x7f31c415d580 2023-10-10T20:28:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31c37fe700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00ccc0 tx=0x7f31bc0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00a880 con 0x7f31c415d580 2023-10-10T20:28:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc00aa00 con 0x7f31c415d580 2023-10-10T20:28:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00adc0 con 0x7f31c415d580 2023-10-10T20:28:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c40b2390 con 0x7f31c415d580 2023-10-10T20:28:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.716+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c40b2950 con 0x7f31c415d580 2023-10-10T20:28:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.717+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c404c3c0 con 0x7f31c415d580 2023-10-10T20:28:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.717+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31bc007300 con 0x7f31c415d580 2023-10-10T20:28:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.717+0000 7f31baffd700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 0x7f31a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.717+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f31bc084940 con 0x7f31c415d580 2023-10-10T20:28:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.718+0000 7f31c2ffd700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 0x7f31a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.718+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f31a406eb10 con 0x7f31c415d580 2023-10-10T20:28:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.718+0000 7f31c2ffd700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 0x7f31a40693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f31b40091e0 tx=0x7f31b4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:25.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.722+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31bc051b60 con 0x7f31c415d580 2023-10-10T20:28:25.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.912+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31c40b3c30 con 0x7f31c415d580 2023-10-10T20:28:25.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.913+0000 7f31baffd700 1 -- 172.21.15.148:0/4051595785 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+15537 (secure 0 0 0) 0x7f31bc055180 con 0x7f31c415d580 2023-10-10T20:28:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:25.917 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":150,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:27:45.558400+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.23","mappings":[{"from":2,"to":0}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 msgr2=0x7f31a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 0x7f31a40693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f31b40091e0 tx=0x7f31b4017000 comp rx=0 tx=0).stop 2023-10-10T20:28:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 msgr2=0x7f31c416d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00ccc0 tx=0x7f31bc0019e0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 shutdown_connections 2023-10-10T20:28:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31a4066f30 0x7f31a40693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c415d580 0x7f31c416d1b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31c4157870 0x7f31c40c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 --2- 172.21.15.148:0/4051595785 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31c415a620 0x7f31c40b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:25.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 >> 172.21.15.148:0/4051595785 conn(0x7f31c4068ce0 msgr2=0x7f31c4155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:25.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 shutdown_connections 2023-10-10T20:28:25.921 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:25.916+0000 7f31c9757700 1 -- 172.21.15.148:0/4051595785 wait complete. 2023-10-10T20:28:25.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:26:12.529204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:26:13.148325+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:28:25.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:28:26.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.046+0000 7f03b86ac700 1 Processor -- start 2023-10-10T20:28:26.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.046+0000 7f03b86ac700 1 -- start start 2023-10-10T20:28:26.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b86ac700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03b0159ec0 0x7f03b00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b86ac700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03b015a290 0x7f03b00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b86ac700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b00b0790 con 0x7f03b006a5a0 2023-10-10T20:28:26.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b86ac700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00ce1f0 con 0x7f03b0159ec0 2023-10-10T20:28:26.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b86ac700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00b0b20 con 0x7f03b015a290 2023-10-10T20:28:26.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b6448700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03b0159ec0 0x7f03b00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.047+0000 7f03b6c49700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f03b006a5a0 0x7f03b015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48344/0 (socket says 172.21.15.148:48344) 2023-10-10T20:28:26.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6c49700 1 -- 172.21.15.148:0/165470558 learned_addr learned my addr 172.21.15.148:0/165470558 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:26.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6448700 1 -- 172.21.15.148:0/165470558 >> v2:172.21.15.161:3300/0 conn(0x7f03b015a290 msgr2=0x7f03b00afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:26.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6448700 1 --2- 172.21.15.148:0/165470558 >> v2:172.21.15.161:3300/0 conn(0x7f03b015a290 0x7f03b00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6448700 1 -- 172.21.15.148:0/165470558 >> v1:172.21.15.148:6789/0 conn(0x7f03b006a5a0 legacy=0x7f03b015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b5446700 1 -- 172.21.15.148:0/165470558 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274015444 0 0) 0x7f03b00b0790 con 0x7f03b006a5a0 2023-10-10T20:28:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6448700 1 -- 172.21.15.148:0/165470558 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00b04b0 con 0x7f03b0159ec0 2023-10-10T20:28:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b6448700 1 --2- 172.21.15.148:0/165470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00af940 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f03b0089910 tx=0x7f03ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16d8c57787e6d0b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b5446700 1 -- 172.21.15.148:0/165470558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00e070 con 0x7f03b0159ec0 2023-10-10T20:28:26.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b5446700 1 -- 172.21.15.148:0/165470558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac00bc30 con 0x7f03b0159ec0 2023-10-10T20:28:26.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b5446700 1 -- 172.21.15.148:0/165470558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac012430 con 0x7f03b0159ec0 2023-10-10T20:28:26.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b86ac700 1 -- 172.21.15.148:0/165470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 msgr2=0x7f03b00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.048+0000 7f03b86ac700 1 --2- 172.21.15.148:0/165470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00af940 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f03b0089910 tx=0x7f03ac009780 comp rx=0 tx=0).stop 2023-10-10T20:28:26.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 -- 172.21.15.148:0/165470558 shutdown_connections 2023-10-10T20:28:26.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 --2- 172.21.15.148:0/165470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00af940 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 --2- 172.21.15.148:0/165470558 >> v2:172.21.15.161:3300/0 conn(0x7f03b015a290 0x7f03b00afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 -- 172.21.15.148:0/165470558 >> 172.21.15.148:0/165470558 conn(0x7f03b00d1880 msgr2=0x7f03b00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 -- 172.21.15.148:0/165470558 shutdown_connections 2023-10-10T20:28:26.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.049+0000 7f03b86ac700 1 -- 172.21.15.148:0/165470558 wait complete. 2023-10-10T20:28:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 Processor -- start 2023-10-10T20:28:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 -- start start 2023-10-10T20:28:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 0x7f03b0158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 0x7f03b0153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b016e6c0 con 0x7f03b0159ec0 2023-10-10T20:28:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b016d810 con 0x7f03b015a290 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.050+0000 7f03b86ac700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b016dab0 con 0x7f03b006a5a0 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45374/0 (socket says 172.21.15.148:45374) 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 -- 172.21.15.148:0/3822888738 learned_addr learned my addr 172.21.15.148:0/3822888738 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b6c49700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 0x7f03b0153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b6448700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 0x7f03b0158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 -- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 msgr2=0x7f03b0158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 0x7f03b0158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 -- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 msgr2=0x7f03b0153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 0x7f03b0153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac009a20 con 0x7f03b0159ec0 2023-10-10T20:28:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b6c49700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 0x7f03b0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b6448700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 0x7f03b0158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b5c47700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b810 tx=0x7f03a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000f570 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a000f6d0 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000fa90 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b016dd90 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.051+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b016e350 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.052+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b004c3c0 con 0x7f03b0159ec0 2023-10-10T20:28:26.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.052+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03a001a460 con 0x7f03b0159ec0 2023-10-10T20:28:26.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.053+0000 7f03a67fc700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 0x7f03940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.053+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f03a0084250 con 0x7f03b0159ec0 2023-10-10T20:28:26.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.053+0000 7f03b6448700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 0x7f03940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.053+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f039406eb10 con 0x7f03b0159ec0 2023-10-10T20:28:26.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.053+0000 7f03b6448700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 0x7f03940693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03b0089910 tx=0x7f03ac019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.057+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03a0051500 con 0x7f03b0159ec0 2023-10-10T20:28:26.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.238+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03b00ceab0 con 0x7f03b0159ec0 2023-10-10T20:28:26.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.239+0000 7f03a67fc700 1 -- 172.21.15.148:0/3822888738 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03a0054b20 con 0x7f03b0159ec0 2023-10-10T20:28:26.241 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:28:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 msgr2=0x7f03940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 0x7f03940693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03b0089910 tx=0x7f03ac019000 comp rx=0 tx=0).stop 2023-10-10T20:28:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 msgr2=0x7f03b00b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b810 tx=0x7f03a00091d0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 shutdown_connections 2023-10-10T20:28:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0394066f30 0x7f03940693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b015a290 0x7f03b0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03b006a5a0 0x7f03b0158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 --2- 172.21.15.148:0/3822888738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b0159ec0 0x7f03b00b5c10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 >> 172.21.15.148:0/3822888738 conn(0x7f03b00d1880 msgr2=0x7f03b00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 shutdown_connections 2023-10-10T20:28:26.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.241+0000 7f03b86ac700 1 -- 172.21.15.148:0/3822888738 wait complete. 2023-10-10T20:28:26.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:28:26.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.370+0000 7f4ba75f8700 1 Processor -- start 2023-10-10T20:28:26.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.370+0000 7f4ba75f8700 1 -- start start 2023-10-10T20:28:26.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba75f8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ba0068960 0x7f4ba00b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba75f8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154850 0x7f4ba00b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba75f8700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba015b410 con 0x7f4ba0154480 2023-10-10T20:28:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba75f8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba015a800 con 0x7f4ba0068960 2023-10-10T20:28:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba75f8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba015aae0 con 0x7f4ba0154850 2023-10-10T20:28:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba5394700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4ba0154480 0x7f4ba0157b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48352/0 (socket says 172.21.15.148:48352) 2023-10-10T20:28:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba5394700 1 -- 172.21.15.148:0/1565163546 learned_addr learned my addr 172.21.15.148:0/1565163546 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:26.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.371+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/1565163546 >> v2:172.21.15.148:3300/0 conn(0x7f4ba0068960 0x7f4ba00b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba5b95700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154850 0x7f4ba00b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4b97fff700 1 -- 172.21.15.148:0/1565163546 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794879587 0 0) 0x7f4ba015b410 con 0x7f4ba0154480 2023-10-10T20:28:26.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4b97fff700 1 -- 172.21.15.148:0/1565163546 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8c003570 con 0x7f4ba0154480 2023-10-10T20:28:26.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba4b93700 1 -- 172.21.15.148:0/1565163546 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154850 msgr2=0x7f4ba00b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154850 0x7f4ba00b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba4b93700 1 -- 172.21.15.148:0/1565163546 >> v1:172.21.15.148:6789/0 conn(0x7f4ba0154480 legacy=0x7f4ba0157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba4b93700 1 -- 172.21.15.148:0/1565163546 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba015adc0 con 0x7f4ba0068960 2023-10-10T20:28:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba00b1920 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4b900098e0 tx=0x7f4b900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58ee1413894baec server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4b97fff700 1 -- 172.21.15.148:0/1565163546 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90011630 con 0x7f4ba0068960 2023-10-10T20:28:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4b97fff700 1 -- 172.21.15.148:0/1565163546 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90011790 con 0x7f4ba0068960 2023-10-10T20:28:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.372+0000 7f4b97fff700 1 -- 172.21.15.148:0/1565163546 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90011b30 con 0x7f4ba0068960 2023-10-10T20:28:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 -- 172.21.15.148:0/1565163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 msgr2=0x7f4ba00b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba00b1920 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4b900098e0 tx=0x7f4b900091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 -- 172.21.15.148:0/1565163546 shutdown_connections 2023-10-10T20:28:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154850 0x7f4ba00b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/1565163546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba00b1920 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 -- 172.21.15.148:0/1565163546 >> 172.21.15.148:0/1565163546 conn(0x7f4ba00ce580 msgr2=0x7f4ba00af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 -- 172.21.15.148:0/1565163546 shutdown_connections 2023-10-10T20:28:26.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.373+0000 7f4ba75f8700 1 -- 172.21.15.148:0/1565163546 wait complete. 2023-10-10T20:28:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.374+0000 7f4ba75f8700 1 Processor -- start 2023-10-10T20:28:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.374+0000 7f4ba75f8700 1 -- start start 2023-10-10T20:28:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 0x7f4ba00aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba0154850 0x7f4ba0069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01692b0 con 0x7f4ba0068960 2023-10-10T20:28:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01686a0 con 0x7f4ba0154850 2023-10-10T20:28:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0168980 con 0x7f4ba0154480 2023-10-10T20:28:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45390/0 (socket says 172.21.15.148:45390) 2023-10-10T20:28:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 -- 172.21.15.148:0/2044490485 learned_addr learned my addr 172.21.15.148:0/2044490485 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 0x7f4ba00aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5b95700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba0154850 0x7f4ba0069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 -- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 msgr2=0x7f4ba00aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 0x7f4ba00aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 -- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba0154850 msgr2=0x7f4ba0069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba0154850 0x7f4ba0069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009420 con 0x7f4ba0068960 2023-10-10T20:28:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 0x7f4ba00aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba5394700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c7c0 tx=0x7f4b9c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c005b30 con 0x7f4ba0068960 2023-10-10T20:28:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c005c90 con 0x7f4ba0068960 2023-10-10T20:28:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba0168c60 con 0x7f4ba0068960 2023-10-10T20:28:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c007340 con 0x7f4ba0068960 2023-10-10T20:28:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.375+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba00af020 con 0x7f4ba0068960 2023-10-10T20:28:26.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.376+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba00684f0 con 0x7f4ba0068960 2023-10-10T20:28:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b9c01b400 con 0x7f4ba0068960 2023-10-10T20:28:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4b957fa700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 0x7f4b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4b9c084080 con 0x7f4ba0068960 2023-10-10T20:28:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 0x7f4b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f4b8406eb10 con 0x7f4ba0068960 2023-10-10T20:28:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.377+0000 7f4ba4b93700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 0x7f4b840693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b900091e0 tx=0x7f4b90017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.381+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b9c051330 con 0x7f4ba0068960 2023-10-10T20:28:26.563 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.561+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ba004c3c0 con 0x7f4ba0068960 2023-10-10T20:28:26.564 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.561+0000 7f4b957fa700 1 -- 172.21.15.148:0/2044490485 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b9c054950 con 0x7f4ba0068960 2023-10-10T20:28:26.564 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:28:26.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 msgr2=0x7f4b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 0x7f4b840693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b900091e0 tx=0x7f4b90017000 comp rx=0 tx=0).stop 2023-10-10T20:28:26.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 msgr2=0x7f4ba0151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c7c0 tx=0x7f4b9c007e00 comp rx=0 tx=0).stop 2023-10-10T20:28:26.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 shutdown_connections 2023-10-10T20:28:26.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4b84066f30 0x7f4b840693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba0154850 0x7f4ba0069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ba0068960 0x7f4ba0151350 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 --2- 172.21.15.148:0/2044490485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba0154480 0x7f4ba00aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 >> 172.21.15.148:0/2044490485 conn(0x7f4ba00ce580 msgr2=0x7f4ba01556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 shutdown_connections 2023-10-10T20:28:26.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.564+0000 7f4ba75f8700 1 -- 172.21.15.148:0/2044490485 wait complete. 2023-10-10T20:28:26.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:28:26.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.694+0000 7f6de7209700 1 Processor -- start 2023-10-10T20:28:26.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.694+0000 7f6de7209700 1 -- start start 2023-10-10T20:28:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de7209700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6de015a250 0x7f6de015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de7209700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de00b6b10 con 0x7f6de0159e80 2023-10-10T20:28:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de7209700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de00b0e70 con 0x7f6de00b4b60 2023-10-10T20:28:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de7209700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de00b2310 con 0x7f6de015a250 2023-10-10T20:28:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6de015a250 0x7f6de015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6de015a250 0x7f6de015a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45396/0 (socket says 172.21.15.148:45396) 2023-10-10T20:28:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6207700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6de0159e80 0x7f6de00aada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48358/0 (socket says 172.21.15.148:48358) 2023-10-10T20:28:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 -- 172.21.15.148:0/201213929 learned_addr learned my addr 172.21.15.148:0/201213929 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 -- 172.21.15.148:0/201213929 >> v1:172.21.15.161:6789/0 conn(0x7f6de00b4b60 legacy=0x7f6de00ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 -- 172.21.15.148:0/201213929 >> v1:172.21.15.148:6789/0 conn(0x7f6de0159e80 legacy=0x7f6de00aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.695+0000 7f6de6a08700 1 -- 172.21.15.148:0/201213929 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00b5000 con 0x7f6de015a250 2023-10-10T20:28:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de6a08700 1 --2- 172.21.15.148:0/201213929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de015a250 0x7f6de015a6a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80099b0 tx=0x7f6dd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8162f27c88b21dc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de5205700 1 -- 172.21.15.148:0/201213929 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800b820 con 0x7f6de015a250 2023-10-10T20:28:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de5205700 1 -- 172.21.15.148:0/201213929 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800b980 con 0x7f6de015a250 2023-10-10T20:28:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de5205700 1 -- 172.21.15.148:0/201213929 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800bd20 con 0x7f6de015a250 2023-10-10T20:28:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 -- 172.21.15.148:0/201213929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de015a250 msgr2=0x7f6de015a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 --2- 172.21.15.148:0/201213929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de015a250 0x7f6de015a6a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80099b0 tx=0x7f6dd8009250 comp rx=0 tx=0).stop 2023-10-10T20:28:26.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 -- 172.21.15.148:0/201213929 shutdown_connections 2023-10-10T20:28:26.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 --2- 172.21.15.148:0/201213929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de015a250 0x7f6de015a6a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 -- 172.21.15.148:0/201213929 >> 172.21.15.148:0/201213929 conn(0x7f6de0099f50 msgr2=0x7f6de0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 -- 172.21.15.148:0/201213929 shutdown_connections 2023-10-10T20:28:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.696+0000 7f6de7209700 1 -- 172.21.15.148:0/201213929 wait complete. 2023-10-10T20:28:26.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.697+0000 7f6de7209700 1 Processor -- start 2023-10-10T20:28:26.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.697+0000 7f6de7209700 1 -- start start 2023-10-10T20:28:26.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6de00b4b60 0x7f6de01570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 0x7f6de009caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00ce030 con 0x7f6de0159e80 2023-10-10T20:28:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00cee20 con 0x7f6de015a250 2023-10-10T20:28:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de7209700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00ce210 con 0x7f6de00b4b60 2023-10-10T20:28:26.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de5a06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de5a06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45400/0 (socket says 172.21.15.148:45400) 2023-10-10T20:28:26.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de5a06700 1 -- 172.21.15.148:0/2402049819 learned_addr learned my addr 172.21.15.148:0/2402049819 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:26.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de6207700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6de00b4b60 0x7f6de01570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.698+0000 7f6de6a08700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 0x7f6de009caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 -- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6de00b4b60 msgr2=0x7f6de01570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6de00b4b60 0x7f6de01570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 -- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 msgr2=0x7f6de009caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 0x7f6de009caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd80094f0 con 0x7f6de0159e80 2023-10-10T20:28:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de6a08700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 0x7f6de009caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de5a06700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b5e0 tx=0x7f6dd0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000f510 con 0x7f6de0159e80 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd000f670 con 0x7f6de0159e80 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00ce4f0 con 0x7f6de0159e80 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000fa30 con 0x7f6de0159e80 2023-10-10T20:28:26.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.699+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de00ceab0 con 0x7f6de0159e80 2023-10-10T20:28:26.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.700+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de0089110 con 0x7f6de0159e80 2023-10-10T20:28:26.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.700+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dd001a400 con 0x7f6de0159e80 2023-10-10T20:28:26.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.701+0000 7f6dd67fc700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:26.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.701+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6dd00840b0 con 0x7f6de0159e80 2023-10-10T20:28:26.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.701+0000 7f6de6207700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:26.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.701+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f6dc406eb10 con 0x7f6de0159e80 2023-10-10T20:28:26.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.701+0000 7f6de6207700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 0x7f6dc40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0020c0 tx=0x7f6ddc001ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:26.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.705+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dd00512c0 con 0x7f6de0159e80 2023-10-10T20:28:26.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.886+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6de00b0310 con 0x7f6de0159e80 2023-10-10T20:28:26.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.886+0000 7f6dd67fc700 1 -- 172.21.15.148:0/2402049819 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dd00548e0 con 0x7f6de0159e80 2023-10-10T20:28:26.889 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:28:26.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 msgr2=0x7f6dc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 0x7f6dc40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0020c0 tx=0x7f6ddc001ad0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 msgr2=0x7f6de009c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:26.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000b5e0 tx=0x7f6dd0009120 comp rx=0 tx=0).stop 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 shutdown_connections 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dc4066f30 0x7f6dc40693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6de015a250 0x7f6de009caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6de00b4b60 0x7f6de01570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 --2- 172.21.15.148:0/2402049819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de0159e80 0x7f6de009c560 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:26.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 >> 172.21.15.148:0/2402049819 conn(0x7f6de0099f50 msgr2=0x7f6de0158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:26.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 shutdown_connections 2023-10-10T20:28:26.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:26.888+0000 7f6de7209700 1 -- 172.21.15.148:0/2402049819 wait complete. 2023-10-10T20:28:26.902 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.014+0000 7f8d22ee9700 1 Processor -- start 2023-10-10T20:28:27.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.014+0000 7f8d22ee9700 1 -- start start 2023-10-10T20:28:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d22ee9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d1c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d22ee9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c06c010 con 0x7f8d1c1523e0 2023-10-10T20:28:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d22ee9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c06bd10 con 0x7f8d1c158e50 2023-10-10T20:28:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d22ee9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15c060 con 0x7f8d1c152010 2023-10-10T20:28:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d1bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8d1c1523e0 0x7f8d1c06c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48366/0 (socket says 172.21.15.148:48366) 2023-10-10T20:28:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.015+0000 7f8d1bfff700 1 -- 172.21.15.148:0/2866204675 learned_addr learned my addr 172.21.15.148:0/2866204675 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:27.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d20c85700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d1c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333883797 0 0) 0x7f8d1c06c010 con 0x7f8d1c1523e0 2023-10-10T20:28:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d04003570 con 0x7f8d1c1523e0 2023-10-10T20:28:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468615761 0 0) 0x7f8d1c06bd10 con 0x7f8d1c158e50 2023-10-10T20:28:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d1c06c010 con 0x7f8d1c158e50 2023-10-10T20:28:27.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212534441 0 0) 0x7f8d04003570 con 0x7f8d1c1523e0 2023-10-10T20:28:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 msgr2=0x7f8d1c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d1c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 >> v1:172.21.15.161:6789/0 conn(0x7f8d1c158e50 legacy=0x7f8d1c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c15c340 con 0x7f8d1c1523e0 2023-10-10T20:28:27.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f8d10001470 con 0x7f8d1c1523e0 2023-10-10T20:28:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 >> v1:172.21.15.148:6789/0 conn(0x7f8d1c1523e0 legacy=0x7f8d1c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04003c20 0x7f8d04004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d04004810 0x7f8d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d04005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15c060 con 0x7f8d04004810 2023-10-10T20:28:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0400bbf0 con 0x7f8d04003c20 2023-10-10T20:28:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0400bdb0 con 0x7f8d1c152010 2023-10-10T20:28:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d04004810 0x7f8d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d20c85700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04003c20 0x7f8d04004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d21486700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d04005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 -- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 msgr2=0x7f8d04005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c152010 0x7f8d04005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 -- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04003c20 msgr2=0x7f8d04004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04003c20 0x7f8d04004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.016+0000 7f8d1bfff700 1 -- 172.21.15.148:0/2866204675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04003570 con 0x7f8d04004810 2023-10-10T20:28:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d1bfff700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d04004810 0x7f8d04004c80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d1c06bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429772d8bf8d46fc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000ca80 con 0x7f8d04004810 2023-10-10T20:28:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000cbe0 con 0x7f8d04004810 2023-10-10T20:28:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d1b7fe700 1 -- 172.21.15.148:0/2866204675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10005920 con 0x7f8d04004810 2023-10-10T20:28:27.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2866204675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d04004810 msgr2=0x7f8d04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2866204675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d04004810 0x7f8d04004c80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d1c06bf80 comp rx=0 tx=0).stop 2023-10-10T20:28:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d21486700 1 -- 172.21.15.148:0/2866204675 reap_dead start 2023-10-10T20:28:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2866204675 shutdown_connections 2023-10-10T20:28:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2866204675 >> 172.21.15.148:0/2866204675 conn(0x7f8d1c069870 msgr2=0x7f8d1c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2866204675 shutdown_connections 2023-10-10T20:28:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.017+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2866204675 wait complete. 2023-10-10T20:28:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.018+0000 7f8d22ee9700 1 Processor -- start 2023-10-10T20:28:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.018+0000 7f8d22ee9700 1 -- start start 2023-10-10T20:28:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04004810 0x7f8d1c16a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c069430 0x7f8d1c159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c068fa0 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15ab00 con 0x7f8d04004810 2023-10-10T20:28:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d22ee9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15ada0 con 0x7f8d1c069430 2023-10-10T20:28:27.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d21486700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d21486700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45408/0 (socket says 172.21.15.148:45408) 2023-10-10T20:28:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d21486700 1 -- 172.21.15.148:0/2359768856 learned_addr learned my addr 172.21.15.148:0/2359768856 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d20c85700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04004810 0x7f8d1c16a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.019+0000 7f8d1bfff700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c069430 0x7f8d1c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 -- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c069430 msgr2=0x7f8d1c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c069430 0x7f8d1c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 -- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04004810 msgr2=0x7f8d1c16a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04004810 0x7f8d1c16a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c15b080 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d21486700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c159220 tx=0x7f8d14007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14002e30 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14006a30 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c16a660 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14006df0 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.020+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c14dd60 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.021+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c04c3c0 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.021+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d14007240 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.022+0000 7f8d18ff9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.022+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8d14084150 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.022+0000 7f8d20c85700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.022+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f8d0806eb10 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.022+0000 7f8d20c85700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0094d0 tx=0x7f8d0c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.026+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d14019060 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.247+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1c0690e0 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.248+0000 7f8d18ff9700 1 -- 172.21.15.148:0/2359768856 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f8d14054e10 con 0x7f8d1c14d7a0 2023-10-10T20:28:27.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:27.251 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:28:27.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 msgr2=0x7f8d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0094d0 tx=0x7f8d0c00b040 comp rx=0 tx=0).stop 2023-10-10T20:28:27.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 msgr2=0x7f8d1c0cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c159220 tx=0x7f8d14007d30 comp rx=0 tx=0).stop 2023-10-10T20:28:27.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 shutdown_connections 2023-10-10T20:28:27.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d1c14d7a0 0x7f8d1c0cb010 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d04004810 0x7f8d1c16a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 --2- 172.21.15.148:0/2359768856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d1c069430 0x7f8d1c159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 >> 172.21.15.148:0/2359768856 conn(0x7f8d1c069870 msgr2=0x7f8d1c0ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 shutdown_connections 2023-10-10T20:28:27.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.250+0000 7f8d22ee9700 1 -- 172.21.15.148:0/2359768856 wait complete. 2023-10-10T20:28:27.255 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:28:27.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:27.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.376+0000 7f0668ea0700 1 Processor -- start 2023-10-10T20:28:27.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.376+0000 7f0668ea0700 1 -- start start 2023-10-10T20:28:27.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 0x7f06640c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06640b1220 0x7f066406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 0x7f06640cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066415de60 con 0x7f06640b1220 2023-10-10T20:28:27.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066415e140 con 0x7f066406dad0 2023-10-10T20:28:27.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0668ea0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066415e420 con 0x7f06640b0e50 2023-10-10T20:28:27.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0659d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06640b1220 0x7f066406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0659d9b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06640b1220 0x7f066406d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45412/0 (socket says 172.21.15.148:45412) 2023-10-10T20:28:27.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f0659d9b700 1 -- 172.21.15.148:0/1377220930 learned_addr learned my addr 172.21.15.148:0/1377220930 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:27.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.377+0000 7f066259c700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 0x7f06640c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0662d9d700 1 --2- 172.21.15.148:0/1377220930 >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 0x7f06640cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 -- 172.21.15.148:0/1377220930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 msgr2=0x7f06640c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:27.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 0x7f06640c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 -- 172.21.15.148:0/1377220930 >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 msgr2=0x7f06640cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:27.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 --2- 172.21.15.148:0/1377220930 >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 0x7f06640cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 -- 172.21.15.148:0/1377220930 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066415e700 con 0x7f06640b1220 2023-10-10T20:28:27.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f066259c700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 0x7f06640c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:27.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0662d9d700 1 --2- 172.21.15.148:0/1377220930 >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 0x7f06640cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:27.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0659d9b700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b1220 0x7f066406d590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f064c0099b0 tx=0x7f064c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2b34df4c8c1403 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0661d9b700 1 -- 172.21.15.148:0/1377220930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00b820 con 0x7f06640b1220 2023-10-10T20:28:27.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0661d9b700 1 -- 172.21.15.148:0/1377220930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c00b980 con 0x7f06640b1220 2023-10-10T20:28:27.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0661d9b700 1 -- 172.21.15.148:0/1377220930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00bd40 con 0x7f06640b1220 2023-10-10T20:28:27.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 -- 172.21.15.148:0/1377220930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b1220 msgr2=0x7f066406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b1220 0x7f066406d590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f064c0099b0 tx=0x7f064c009250 comp rx=0 tx=0).stop 2023-10-10T20:28:27.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 -- 172.21.15.148:0/1377220930 shutdown_connections 2023-10-10T20:28:27.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 --2- 172.21.15.148:0/1377220930 >> v2:172.21.15.161:3300/0 conn(0x7f066406dad0 0x7f06640cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b0e50 0x7f06640c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 --2- 172.21.15.148:0/1377220930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b1220 0x7f066406d590 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.378+0000 7f0668ea0700 1 -- 172.21.15.148:0/1377220930 >> 172.21.15.148:0/1377220930 conn(0x7f0664069870 msgr2=0x7f0664069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.379+0000 7f0668ea0700 1 -- 172.21.15.148:0/1377220930 shutdown_connections 2023-10-10T20:28:27.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.379+0000 7f0668ea0700 1 -- 172.21.15.148:0/1377220930 wait complete. 2023-10-10T20:28:27.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.380+0000 7f0668ea0700 1 Processor -- start 2023-10-10T20:28:27.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.380+0000 7f0668ea0700 1 -- start start 2023-10-10T20:28:27.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.380+0000 7f0668ea0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f066406dad0 0x7f066415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.380+0000 7f0668ea0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b1220 0x7f0664153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640cb8d0 con 0x7f06640b0e50 2023-10-10T20:28:27.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640cacc0 con 0x7f066406dad0 2023-10-10T20:28:27.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06640cafa0 con 0x7f06640b1220 2023-10-10T20:28:27.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45428/0 (socket says 172.21.15.148:45428) 2023-10-10T20:28:27.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 -- 172.21.15.148:0/2126739131 learned_addr learned my addr 172.21.15.148:0/2126739131 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0662d9d700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b1220 0x7f0664153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f066259c700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f066406dad0 0x7f066415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 -- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b1220 msgr2=0x7f0664153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b1220 0x7f0664153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 -- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f066406dad0 msgr2=0x7f066415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f066406dad0 0x7f066415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0094f0 con 0x7f06640b0e50 2023-10-10T20:28:27.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0659d9b700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06640c59e0 tx=0x7f064c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c01f070 con 0x7f06640b0e50 2023-10-10T20:28:27.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c002f30 con 0x7f06640b0e50 2023-10-10T20:28:27.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06640cb280 con 0x7f06640b0e50 2023-10-10T20:28:27.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00b820 con 0x7f06640b0e50 2023-10-10T20:28:27.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.381+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066415b510 con 0x7f06640b0e50 2023-10-10T20:28:27.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.382+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066415b730 con 0x7f06640b0e50 2023-10-10T20:28:27.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f064c023070 con 0x7f06640b0e50 2023-10-10T20:28:27.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f065affd700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 0x7f06440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f064c085880 con 0x7f06640b0e50 2023-10-10T20:28:27.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f066259c700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 0x7f06440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f064406eb10 con 0x7f06640b0e50 2023-10-10T20:28:27.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.383+0000 7f066259c700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 0x7f06440693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f065400adc0 tx=0x7f0654006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.387+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f064c02a060 con 0x7f06640b0e50 2023-10-10T20:28:27.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.571+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f066404c3c0 con 0x7f06640b0e50 2023-10-10T20:28:27.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.572+0000 7f065affd700 1 -- 172.21.15.148:0/2126739131 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+15537 (secure 0 0 0) 0x7f064c015020 con 0x7f06640b0e50 2023-10-10T20:28:27.575 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:27.575 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":150,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:27:45.558400+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.23","mappings":[{"from":2,"to":0}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:27.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.574+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 msgr2=0x7f06440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.574+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 0x7f06440693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f065400adc0 tx=0x7f0654006f90 comp rx=0 tx=0).stop 2023-10-10T20:28:27.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.574+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 msgr2=0x7f0664153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.574+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06640c59e0 tx=0x7f064c017000 comp rx=0 tx=0).stop 2023-10-10T20:28:27.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 shutdown_connections 2023-10-10T20:28:27.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0644066f30 0x7f06440693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06640b1220 0x7f0664153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f066406dad0 0x7f066415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 --2- 172.21.15.148:0/2126739131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06640b0e50 0x7f0664153600 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 >> 172.21.15.148:0/2126739131 conn(0x7f0664069870 msgr2=0x7f06640680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 shutdown_connections 2023-10-10T20:28:27.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.575+0000 7f0668ea0700 1 -- 172.21.15.148:0/2126739131 wait complete. 2023-10-10T20:28:27.589 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:28:27.590 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:27.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.702+0000 7f38ab06b700 1 Processor -- start 2023-10-10T20:28:27.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.702+0000 7f38ab06b700 1 -- start start 2023-10-10T20:28:27.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38ab06b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38a4157c40 0x7f38a4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38ab06b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415cbb0 con 0x7f38a4157870 2023-10-10T20:28:27.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38ab06b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415bfa0 con 0x7f38a415a620 2023-10-10T20:28:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38ab06b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a415bdb0 con 0x7f38a4157c40 2023-10-10T20:28:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38a8e07700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f38a4157870 0x7f38a415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48374/0 (socket says 172.21.15.148:48374) 2023-10-10T20:28:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38a9608700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38a4157c40 0x7f38a4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f38a8e07700 1 -- 172.21.15.148:0/3346513098 learned_addr learned my addr 172.21.15.148:0/3346513098 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f389b7fe700 1 -- 172.21.15.148:0/3346513098 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140976679 0 0) 0x7f38a415cbb0 con 0x7f38a4157870 2023-10-10T20:28:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.703+0000 7f389b7fe700 1 -- 172.21.15.148:0/3346513098 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f388c003570 con 0x7f38a4157870 2023-10-10T20:28:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38a9608700 1 -- 172.21.15.148:0/3346513098 >> v1:172.21.15.161:6789/0 conn(0x7f38a415a620 legacy=0x7f38a415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38a9608700 1 -- 172.21.15.148:0/3346513098 >> v1:172.21.15.148:6789/0 conn(0x7f38a4157870 legacy=0x7f38a415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38a9608700 1 -- 172.21.15.148:0/3346513098 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a415c280 con 0x7f38a4157c40 2023-10-10T20:28:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38a9608700 1 --2- 172.21.15.148:0/3346513098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a4157c40 0x7f38a4158110 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e60 tx=0x7f389c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5a2fc02282b580 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f389b7fe700 1 -- 172.21.15.148:0/3346513098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00e070 con 0x7f38a4157c40 2023-10-10T20:28:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f389b7fe700 1 -- 172.21.15.148:0/3346513098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00bb60 con 0x7f38a4157c40 2023-10-10T20:28:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f389b7fe700 1 -- 172.21.15.148:0/3346513098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c012430 con 0x7f38a4157c40 2023-10-10T20:28:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38ab06b700 1 -- 172.21.15.148:0/3346513098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a4157c40 msgr2=0x7f38a4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38ab06b700 1 --2- 172.21.15.148:0/3346513098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a4157c40 0x7f38a4158110 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f389c009e60 tx=0x7f389c009780 comp rx=0 tx=0).stop 2023-10-10T20:28:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38ab06b700 1 -- 172.21.15.148:0/3346513098 shutdown_connections 2023-10-10T20:28:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38ab06b700 1 --2- 172.21.15.148:0/3346513098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a4157c40 0x7f38a4158110 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.704+0000 7f38ab06b700 1 -- 172.21.15.148:0/3346513098 >> 172.21.15.148:0/3346513098 conn(0x7f38a4068ce0 msgr2=0x7f38a40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.705+0000 7f38ab06b700 1 -- 172.21.15.148:0/3346513098 shutdown_connections 2023-10-10T20:28:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.705+0000 7f38ab06b700 1 -- 172.21.15.148:0/3346513098 wait complete. 2023-10-10T20:28:27.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 Processor -- start 2023-10-10T20:28:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 -- start start 2023-10-10T20:28:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38a4157870 0x7f38a4150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a4157c40 0x7f38a414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4150570 con 0x7f38a415a620 2023-10-10T20:28:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a414f9a0 con 0x7f38a4157c40 2023-10-10T20:28:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.706+0000 7f38ab06b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a414fc40 con 0x7f38a4157870 2023-10-10T20:28:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45452/0 (socket says 172.21.15.148:45452) 2023-10-10T20:28:27.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 -- 172.21.15.148:0/2384902737 learned_addr learned my addr 172.21.15.148:0/2384902737 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a8e07700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38a4157870 0x7f38a4150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f389bfff700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a4157c40 0x7f38a414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 -- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38a4157870 msgr2=0x7f38a4150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38a4157870 0x7f38a4150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 -- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a4157c40 msgr2=0x7f38a414bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a4157c40 0x7f38a414bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009a20 con 0x7f38a415a620 2023-10-10T20:28:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38a9608700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f389c013000 tx=0x7f389c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c012e00 con 0x7f38a415a620 2023-10-10T20:28:27.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c01d630 con 0x7f38a415a620 2023-10-10T20:28:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01d9f0 con 0x7f38a415a620 2023-10-10T20:28:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a414ff20 con 0x7f38a415a620 2023-10-10T20:28:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.707+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a4151410 con 0x7f38a415a620 2023-10-10T20:28:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.708+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4150720 con 0x7f38a415a620 2023-10-10T20:28:27.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389c024070 con 0x7f38a415a620 2023-10-10T20:28:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f3898ff9700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 0x7f38900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f389c086d50 con 0x7f38a415a620 2023-10-10T20:28:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f38a8e07700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 0x7f38900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f389006eb10 con 0x7f38a415a620 2023-10-10T20:28:27.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.709+0000 7f38a8e07700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 0x7f38900693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f38a00058d0 tx=0x7f38a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:27.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.713+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38a4150720 con 0x7f38a415a620 2023-10-10T20:28:27.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.940+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a4151600 con 0x7f38a415a620 2023-10-10T20:28:27.943 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.940+0000 7f3898ff9700 1 -- 172.21.15.148:0/2384902737 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f389c08a030 con 0x7f38a415a620 2023-10-10T20:28:27.943 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:27.943 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:28:27.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 msgr2=0x7f38900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 0x7f38900693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f38a00058d0 tx=0x7f38a0009040 comp rx=0 tx=0).stop 2023-10-10T20:28:27.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 msgr2=0x7f38a41517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:27.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f389c013000 tx=0x7f389c019000 comp rx=0 tx=0).stop 2023-10-10T20:28:27.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 shutdown_connections 2023-10-10T20:28:27.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3890066f30 0x7f38900693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38a415a620 0x7f38a41517b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38a4157870 0x7f38a4150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 --2- 172.21.15.148:0/2384902737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a4157c40 0x7f38a414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:27.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.942+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 >> 172.21.15.148:0/2384902737 conn(0x7f38a4068ce0 msgr2=0x7f38a40b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:27.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.943+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 shutdown_connections 2023-10-10T20:28:27.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:27.943+0000 7f38ab06b700 1 -- 172.21.15.148:0/2384902737 wait complete. 2023-10-10T20:28:27.948 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:28:27.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:28.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.069+0000 7f1114e79700 1 Processor -- start 2023-10-10T20:28:28.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.069+0000 7f1114e79700 1 -- start start 2023-10-10T20:28:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f1114e79700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1110157c40 0x7f1110158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f1114e79700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111015cbb0 con 0x7f1110157870 2023-10-10T20:28:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f1114e79700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111015bfa0 con 0x7f111015a620 2023-10-10T20:28:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f1114e79700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111015bdb0 con 0x7f1110157c40 2023-10-10T20:28:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f110ed9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1110157c40 0x7f1110158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f110e59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1110157870 0x7f111015b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48386/0 (socket says 172.21.15.148:48386) 2023-10-10T20:28:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f110e59c700 1 -- 172.21.15.148:0/384432349 learned_addr learned my addr 172.21.15.148:0/384432349 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.070+0000 7f110ed9d700 1 --2- 172.21.15.148:0/384432349 >> v2:172.21.15.148:3300/0 conn(0x7f1110157c40 0x7f1110158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45460/0 (socket says 172.21.15.148:45460) 2023-10-10T20:28:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110ed9d700 1 -- 172.21.15.148:0/384432349 >> v1:172.21.15.161:6789/0 conn(0x7f111015a620 legacy=0x7f111015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110ed9d700 1 -- 172.21.15.148:0/384432349 >> v1:172.21.15.148:6789/0 conn(0x7f1110157870 legacy=0x7f111015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110dd9b700 1 -- 172.21.15.148:0/384432349 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925051772 0 0) 0x7f111015cbb0 con 0x7f1110157870 2023-10-10T20:28:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110ed9d700 1 -- 172.21.15.148:0/384432349 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111015c280 con 0x7f1110157c40 2023-10-10T20:28:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110ed9d700 1 --2- 172.21.15.148:0/384432349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157c40 0x7f1110158110 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e60 tx=0x7f1108009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab6f60158c74495 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110dd9b700 1 -- 172.21.15.148:0/384432349 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800e070 con 0x7f1110157c40 2023-10-10T20:28:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110dd9b700 1 -- 172.21.15.148:0/384432349 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110800bc30 con 0x7f1110157c40 2023-10-10T20:28:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f110dd9b700 1 -- 172.21.15.148:0/384432349 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108012430 con 0x7f1110157c40 2023-10-10T20:28:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f1114e79700 1 -- 172.21.15.148:0/384432349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157c40 msgr2=0x7f1110158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f1114e79700 1 --2- 172.21.15.148:0/384432349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157c40 0x7f1110158110 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1108009e60 tx=0x7f1108009780 comp rx=0 tx=0).stop 2023-10-10T20:28:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f1114e79700 1 -- 172.21.15.148:0/384432349 shutdown_connections 2023-10-10T20:28:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f1114e79700 1 --2- 172.21.15.148:0/384432349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157c40 0x7f1110158110 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.071+0000 7f1114e79700 1 -- 172.21.15.148:0/384432349 >> 172.21.15.148:0/384432349 conn(0x7f1110068ce0 msgr2=0x7f11100690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:28.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.072+0000 7f1114e79700 1 -- 172.21.15.148:0/384432349 shutdown_connections 2023-10-10T20:28:28.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.072+0000 7f1114e79700 1 -- 172.21.15.148:0/384432349 wait complete. 2023-10-10T20:28:28.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 Processor -- start 2023-10-10T20:28:28.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 -- start start 2023-10-10T20:28:28.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1110157c40 0x7f111016a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f111015a620 0x7f111016aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111016af40 con 0x7f1110157870 2023-10-10T20:28:28.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100b1840 con 0x7f111015a620 2023-10-10T20:28:28.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.073+0000 7f1114e79700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111013f6e0 con 0x7f1110157c40 2023-10-10T20:28:28.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45462/0 (socket says 172.21.15.148:45462) 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 -- 172.21.15.148:0/2181486174 learned_addr learned my addr 172.21.15.148:0/2181486174 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110ed9d700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f111015a620 0x7f111016aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1105d9b700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1110157c40 0x7f111016a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 -- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1110157c40 msgr2=0x7f111016a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1110157c40 0x7f111016a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 -- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f111015a620 msgr2=0x7f111016aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f111015a620 0x7f111016aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108009a20 con 0x7f1110157870 2023-10-10T20:28:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f110e59c700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f110000c810 tx=0x7f1100007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100005b30 con 0x7f1110157870 2023-10-10T20:28:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100005c90 con 0x7f1110157870 2023-10-10T20:28:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1110153a80 con 0x7f1110157870 2023-10-10T20:28:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100007300 con 0x7f1110157870 2023-10-10T20:28:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.074+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1110154040 con 0x7f1110157870 2023-10-10T20:28:28.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.075+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111004c3c0 con 0x7f1110157870 2023-10-10T20:28:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.075+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f110001b3b0 con 0x7f1110157870 2023-10-10T20:28:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.076+0000 7f1106ffd700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 0x7f10f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.076+0000 7f1105d9b700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 0x7f10f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.076+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1100083a70 con 0x7f1110157870 2023-10-10T20:28:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.076+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f10f006eb10 con 0x7f1110157870 2023-10-10T20:28:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.076+0000 7f1105d9b700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 0x7f10f00693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1110156bd0 tx=0x7f10f8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.080+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1100017020 con 0x7f1110157870 2023-10-10T20:28:28.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.302+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111015c560 con 0x7f1110157870 2023-10-10T20:28:28.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.302+0000 7f1106ffd700 1 -- 172.21.15.148:0/2181486174 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f1100050ca0 con 0x7f1110157870 2023-10-10T20:28:28.305 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:28.306 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:26:14.180789+0000","modified":"2023-10-10T20:26:17.824138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7470":{"gid":7470,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/2641217056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2641217056},{"type":"v1","addr":"172.21.15.148:6837","nonce":2641217056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-10T20:28:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 msgr2=0x7f10f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 0x7f10f00693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1110156bd0 tx=0x7f10f8007ec0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 msgr2=0x7f111013f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f110000c810 tx=0x7f1100007df0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 shutdown_connections 2023-10-10T20:28:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f10f0066f30 0x7f10f00693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f111015a620 0x7f111016aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1110157870 0x7f111013f1a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 --2- 172.21.15.148:0/2181486174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1110157c40 0x7f111016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 >> 172.21.15.148:0/2181486174 conn(0x7f1110068ce0 msgr2=0x7f11100690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 shutdown_connections 2023-10-10T20:28:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.305+0000 7f1114e79700 1 -- 172.21.15.148:0/2181486174 wait complete. 2023-10-10T20:28:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 71 2023-10-10T20:28:28.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:28.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.432+0000 7fd7d4c85700 1 Processor -- start 2023-10-10T20:28:28.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.432+0000 7fd7d4c85700 1 -- start start 2023-10-10T20:28:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7d4c85700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 0x7fd7d00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7d4c85700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 0x7fd7d00cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7d4c85700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d015de60 con 0x7fd7d00b1220 2023-10-10T20:28:28.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7d4c85700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00cbed0 con 0x7fd7d00b0e50 2023-10-10T20:28:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7d4c85700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d015e140 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7c5d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd7d00b1220 0x7fd7d006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48402/0 (socket says 172.21.15.148:48402) 2023-10-10T20:28:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7c5d9b700 1 -- 172.21.15.148:0/3992072903 learned_addr learned my addr 172.21.15.148:0/3992072903 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7ce59c700 1 --2- 172.21.15.148:0/3992072903 >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 0x7fd7d00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.433+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3992072903 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 0x7fd7d00cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7cdd9b700 1 -- 172.21.15.148:0/3992072903 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700101847 0 0) 0x7fd7d015de60 con 0x7fd7d00b1220 2023-10-10T20:28:28.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7cdd9b700 1 -- 172.21.15.148:0/3992072903 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc003570 con 0x7fd7d00b1220 2023-10-10T20:28:28.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3992072903 >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 msgr2=0x7fd7d00c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:28.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3992072903 >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 0x7fd7d00c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3992072903 >> v1:172.21.15.148:6789/0 conn(0x7fd7d00b1220 legacy=0x7fd7d006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3992072903 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d015e420 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ce59c700 1 --2- 172.21.15.148:0/3992072903 >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 0x7fd7d00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3992072903 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 0x7fd7d00cb990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009e60 tx=0x7fd7c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cb318d9f180f4e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7cdd9b700 1 -- 172.21.15.148:0/3992072903 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800e070 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7cdd9b700 1 -- 172.21.15.148:0/3992072903 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c800bab0 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.434+0000 7fd7cdd9b700 1 -- 172.21.15.148:0/3992072903 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800be30 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3992072903 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 msgr2=0x7fd7d00cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3992072903 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 0x7fd7d00cb990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009e60 tx=0x7fd7c8009780 comp rx=0 tx=0).stop 2023-10-10T20:28:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3992072903 shutdown_connections 2023-10-10T20:28:28.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3992072903 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00cb4f0 0x7fd7d00cb990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3992072903 >> v2:172.21.15.161:3300/0 conn(0x7fd7d00b0e50 0x7fd7d00c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3992072903 >> 172.21.15.148:0/3992072903 conn(0x7fd7d0069870 msgr2=0x7fd7d0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3992072903 shutdown_connections 2023-10-10T20:28:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.435+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3992072903 wait complete. 2023-10-10T20:28:28.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.436+0000 7fd7d4c85700 1 Processor -- start 2023-10-10T20:28:28.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.436+0000 7fd7d4c85700 1 -- start start 2023-10-10T20:28:28.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.436+0000 7fd7d4c85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d00b0e50 0x7fd7d00c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00b1220 0x7fd7d00b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0155f80 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00b3840 con 0x7fd7d00b0e50 2023-10-10T20:28:28.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0155090 con 0x7fd7d00b1220 2023-10-10T20:28:28.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45466/0 (socket says 172.21.15.148:45466) 2023-10-10T20:28:28.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3882949740 learned_addr learned my addr 172.21.15.148:0/3882949740 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:28.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ce59c700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d00b0e50 0x7fd7d00c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7c5d9b700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00b1220 0x7fd7d00b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00b1220 msgr2=0x7fd7d00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00b1220 0x7fd7d00b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d00b0e50 msgr2=0x7fd7d00c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d00b0e50 0x7fd7d00c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0009420 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7ced9d700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b610 tx=0x7fd7c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8012d80 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80263f0 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8026790 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c8009a20 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.437+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d0155650 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.438+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d004c3c0 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7c8026b40 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7c6ffd700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd7c8086050 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7ce59c700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7fd7b006eb10 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.439+0000 7fd7ce59c700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 0x7fd7b00693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009ea0 tx=0x7fd7c00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.443+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7c8053280 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.626+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7d01560c0 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.627+0000 7fd7c6ffd700 1 -- 172.21.15.148:0/3882949740 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+15537 (secure 0 0 0) 0x7fd7c80568a0 con 0x7fd7d00cb4f0 2023-10-10T20:28:28.630 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:28.630 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":150,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:27:45.558400+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:26:12.529204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-10T20:26:13.148325+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.23","mappings":[{"from":2,"to":0}]},{"pgid":"19.32","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:28.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 msgr2=0x7fd7b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 0x7fd7b00693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009ea0 tx=0x7fd7c00091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 msgr2=0x7fd7d01406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800b610 tx=0x7fd7c8019000 comp rx=0 tx=0).stop 2023-10-10T20:28:28.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 shutdown_connections 2023-10-10T20:28:28.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd7b0066f30 0x7fd7b00693e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d00cb4f0 0x7fd7d01406a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7d00b0e50 0x7fd7d00c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 --2- 172.21.15.148:0/3882949740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7d00b1220 0x7fd7d00b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.629+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 >> 172.21.15.148:0/3882949740 conn(0x7fd7d0069870 msgr2=0x7fd7d00c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:28.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.630+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 shutdown_connections 2023-10-10T20:28:28.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.630+0000 7fd7d4c85700 1 -- 172.21.15.148:0/3882949740 wait complete. 2023-10-10T20:28:28.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:28:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.757+0000 7f0c2d948700 1 Processor -- start 2023-10-10T20:28:28.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.757+0000 7f0c2d948700 1 -- start start 2023-10-10T20:28:28.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c2d948700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c280b0890 0x7f0c280cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c2d948700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c28159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c2d948700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c280cb370 con 0x7f0c280cbd40 2023-10-10T20:28:28.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c2d948700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c280b0c60 con 0x7f0c280b0890 2023-10-10T20:28:28.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c2d948700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2806a1e0 con 0x7f0c28067d70 2023-10-10T20:28:28.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c26ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c280b0890 0x7f0c280cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c1e7fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0c280cbd40 0x7f0c28067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48406/0 (socket says 172.21.15.148:48406) 2023-10-10T20:28:28.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.758+0000 7f0c1e7fc700 1 -- 172.21.15.148:0/3696094872 learned_addr learned my addr 172.21.15.148:0/3696094872 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:28.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c277fe700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c28159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c26ffd700 1 -- 172.21.15.148:0/3696094872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 msgr2=0x7f0c28159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:28.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c26ffd700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c28159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c26ffd700 1 -- 172.21.15.148:0/3696094872 >> v1:172.21.15.148:6789/0 conn(0x7f0c280cbd40 legacy=0x7f0c28067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c26ffd700 1 -- 172.21.15.148:0/3696094872 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2806cb80 con 0x7f0c280b0890 2023-10-10T20:28:28.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c267fc700 1 -- 172.21.15.148:0/3696094872 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174883673 0 0) 0x7f0c280cb370 con 0x7f0c280cbd40 2023-10-10T20:28:28.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c26ffd700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280cb800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098e0 tx=0x7f0c180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cebe188d0e225 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c267fc700 1 -- 172.21.15.148:0/3696094872 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800b640 con 0x7f0c280b0890 2023-10-10T20:28:28.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c267fc700 1 -- 172.21.15.148:0/3696094872 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800b7a0 con 0x7f0c280b0890 2023-10-10T20:28:28.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.759+0000 7f0c267fc700 1 -- 172.21.15.148:0/3696094872 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800bb40 con 0x7f0c280b0890 2023-10-10T20:28:28.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 -- 172.21.15.148:0/3696094872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 msgr2=0x7f0c280cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280cb800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0c180098e0 tx=0x7f0c180091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 -- 172.21.15.148:0/3696094872 shutdown_connections 2023-10-10T20:28:28.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c28159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 --2- 172.21.15.148:0/3696094872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280cb800 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 -- 172.21.15.148:0/3696094872 >> 172.21.15.148:0/3696094872 conn(0x7f0c2806fe50 msgr2=0x7f0c28069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:28.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 -- 172.21.15.148:0/3696094872 shutdown_connections 2023-10-10T20:28:28.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.760+0000 7f0c2d948700 1 -- 172.21.15.148:0/3696094872 wait complete. 2023-10-10T20:28:28.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.761+0000 7f0c2d948700 1 Processor -- start 2023-10-10T20:28:28.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.761+0000 7f0c2d948700 1 -- start start 2023-10-10T20:28:28.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c280c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 0x7f0c280c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280c4840 con 0x7f0c280b0890 2023-10-10T20:28:28.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280c5210 con 0x7f0c280cbd40 2023-10-10T20:28:28.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280b2cc0 con 0x7f0c28067d70 2023-10-10T20:28:28.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45478/0 (socket says 172.21.15.148:45478) 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 -- 172.21.15.148:0/1189075073 learned_addr learned my addr 172.21.15.148:0/1189075073 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c26ffd700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c280c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c277fe700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 0x7f0c280c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 -- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 msgr2=0x7f0c280c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c280c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 -- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 msgr2=0x7f0c280c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 0x7f0c280c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009420 con 0x7f0c280b0890 2023-10-10T20:28:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c277fe700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 0x7f0c280c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1e7fc700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c7e0 tx=0x7f0c10007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10005b30 con 0x7f0c280b0890 2023-10-10T20:28:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10005c90 con 0x7f0c280b0890 2023-10-10T20:28:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c280b2fa0 con 0x7f0c280b0890 2023-10-10T20:28:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007300 con 0x7f0c280b0890 2023-10-10T20:28:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.762+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c280b3280 con 0x7f0c280b0890 2023-10-10T20:28:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.763+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c280c4690 con 0x7f0c280b0890 2023-10-10T20:28:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.764+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c100076b0 con 0x7f0c280b0890 2023-10-10T20:28:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.764+0000 7f0c1f7fe700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 0x7f0c080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.764+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f0c10083ae0 con 0x7f0c280b0890 2023-10-10T20:28:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.764+0000 7f0c26ffd700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 0x7f0c080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.764+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f0c0806eb10 con 0x7f0c280b0890 2023-10-10T20:28:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.765+0000 7f0c26ffd700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 0x7f0c080693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0c180091e0 tx=0x7f0c18017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.769+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c10017060 con 0x7f0c280b0890 2023-10-10T20:28:28.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.991+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0c280b4770 con 0x7f0c280b0890 2023-10-10T20:28:28.994 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:28:28.992+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:28:29.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.998+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(151..151 src has 1..151) v4 ==== 366+0+0 (secure 0 0 0) 0x7f0c100550d0 con 0x7f0c280b0890 2023-10-10T20:28:29.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:28.998+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7f0c0806f3f0 con 0x7f0c280b0890 2023-10-10T20:28:29.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.001+0000 7f0c1f7fe700 1 -- 172.21.15.148:0/1189075073 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0c100547a0 con 0x7f0c280b0890 2023-10-10T20:28:29.004 INFO:tasks.ceph.mds.a.smithi148.stderr: -8837> 2023-10-10T20:26:20.097+0000 7f124f71e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:28:29.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 msgr2=0x7f0c080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 0x7f0c080693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0c180091e0 tx=0x7f0c18017000 comp rx=0 tx=0).stop 2023-10-10T20:28:29.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 msgr2=0x7f0c280b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c7e0 tx=0x7f0c10007df0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 shutdown_connections 2023-10-10T20:28:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0c08066f30 0x7f0c080693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c280cbd40 0x7f0c280c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c28067d70 0x7f0c280c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 --2- 172.21.15.148:0/1189075073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c280b0890 0x7f0c280b25e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.003+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 >> 172.21.15.148:0/1189075073 conn(0x7f0c2806fe50 msgr2=0x7f0c280cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.004+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 shutdown_connections 2023-10-10T20:28:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.004+0000 7f0c2d948700 1 -- 172.21.15.148:0/1189075073 wait complete. 2023-10-10T20:28:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:28:29.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:28:29.035 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:28:29.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.133+0000 7f873880f700 1 Processor -- start 2023-10-10T20:28:29.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.133+0000 7f873880f700 1 -- start start 2023-10-10T20:28:29.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.134+0000 7f873880f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f873415a120 0x7f87340c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.134+0000 7f873880f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340c9000 con 0x7f87340cc960 2023-10-10T20:28:29.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.134+0000 7f873880f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340cced0 con 0x7f87340c26b0 2023-10-10T20:28:29.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.134+0000 7f873880f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87340c9a00 con 0x7f873415a120 2023-10-10T20:28:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f8731d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f87340cc960 0x7f87340c1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48414/0 (socket says 172.21.15.148:48414) 2023-10-10T20:28:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f8731d9b700 1 -- 172.21.15.148:0/1378884202 learned_addr learned my addr 172.21.15.148:0/1378884202 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873259c700 1 --2- 172.21.15.148:0/1378884202 >> v2:172.21.15.161:3300/0 conn(0x7f873415a120 0x7f87340c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120242709 0 0) 0x7f87340c9000 con 0x7f87340cc960 2023-10-10T20:28:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8720003570 con 0x7f87340cc960 2023-10-10T20:28:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846019811 0 0) 0x7f8720003570 con 0x7f87340cc960 2023-10-10T20:28:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 >> v2:172.21.15.161:3300/0 conn(0x7f873415a120 msgr2=0x7f87340c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 --2- 172.21.15.148:0/1378884202 >> v2:172.21.15.161:3300/0 conn(0x7f873415a120 0x7f87340c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 >> v1:172.21.15.161:6789/0 conn(0x7f87340c26b0 legacy=0x7f87340aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873406f750 con 0x7f87340cc960 2023-10-10T20:28:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f871c001470 con 0x7f87340cc960 2023-10-10T20:28:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 >> v1:172.21.15.148:6789/0 conn(0x7f87340cc960 legacy=0x7f87340c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720003a60 0x7f8720003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 0x7f872000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87340c9a00 con 0x7f87200052e0 2023-10-10T20:28:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872000c2a0 con 0x7f8720003a60 2023-10-10T20:28:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872000c460 con 0x7f8720004810 2023-10-10T20:28:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f8732d9d700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 0x7f872000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f873259c700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720003a60 0x7f8720003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.135+0000 7f8731d9b700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 -- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8720004810 msgr2=0x7f8720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8720004810 0x7f8720004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 -- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720003a60 msgr2=0x7f8720003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8720003a60 0x7f8720003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 -- 172.21.15.148:0/1378884202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720003570 con 0x7f87200052e0 2023-10-10T20:28:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 -- 172.21.15.148:0/1378884202 reap_dead start 2023-10-10T20:28:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f8732d9d700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 0x7f872000ba60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f872c0018b0 tx=0x7f872c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c472896567df77 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c012420 con 0x7f87200052e0 2023-10-10T20:28:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c0125a0 con 0x7f87200052e0 2023-10-10T20:28:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873159a700 1 -- 172.21.15.148:0/1378884202 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c012960 con 0x7f87200052e0 2023-10-10T20:28:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 -- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 msgr2=0x7f872000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 0x7f872000ba60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f872c0018b0 tx=0x7f872c007d30 comp rx=0 tx=0).stop 2023-10-10T20:28:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 -- 172.21.15.148:0/1378884202 shutdown_connections 2023-10-10T20:28:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 --2- 172.21.15.148:0/1378884202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87200052e0 0x7f872000ba60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 -- 172.21.15.148:0/1378884202 >> 172.21.15.148:0/1378884202 conn(0x7f87340680f0 msgr2=0x7f87340cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 -- 172.21.15.148:0/1378884202 shutdown_connections 2023-10-10T20:28:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.136+0000 7f873880f700 1 -- 172.21.15.148:0/1378884202 wait complete. 2023-10-10T20:28:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.137+0000 7f873880f700 1 Processor -- start 2023-10-10T20:28:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 -- start start 2023-10-10T20:28:29.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 0x7f87341588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8734148fc0 0x7f873416d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873416df90 con 0x7f873416daf0 2023-10-10T20:28:29.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734146b40 con 0x7f87200052e0 2023-10-10T20:28:29.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873880f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8734146e20 con 0x7f8734148fc0 2023-10-10T20:28:29.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f8732d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f8732d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45536/0 (socket says 172.21.15.148:45536) 2023-10-10T20:28:29.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f873259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 0x7f87341588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f8732d9d700 1 -- 172.21.15.148:0/3896007950 learned_addr learned my addr 172.21.15.148:0/3896007950 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.138+0000 7f8731d9b700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8734148fc0 0x7f873416d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 -- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8734148fc0 msgr2=0x7f873416d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8734148fc0 0x7f873416d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 -- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 msgr2=0x7f87341588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 0x7f87341588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734147100 con 0x7f873416daf0 2023-10-10T20:28:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f873259c700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 0x7f87341588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f8732d9d700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f87340640c0 tx=0x7f872c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c012420 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c001200 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87341473e0 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c012940 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.139+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8734158e20 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.140+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873404c3c0 con 0x7f873416daf0 2023-10-10T20:28:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.140+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f872c019070 con 0x7f873416daf0 2023-10-10T20:28:29.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.140+0000 7f872a7fc700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:29.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.141+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f872c0858b0 con 0x7f873416daf0 2023-10-10T20:28:29.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.141+0000 7f873259c700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:29.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.141+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7f871406ebb0 con 0x7f873416daf0 2023-10-10T20:28:29.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.141+0000 7f873259c700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 0x7f87140693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8724009fd0 tx=0x7f872400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:29.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.145+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f872c017030 con 0x7f873416daf0 2023-10-10T20:28:29.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:29.370+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f873406fa30 con 0x7f873416daf0 2023-10-10T20:28:30.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.005+0000 7f872a7fc700 1 -- 172.21.15.148:0/3896007950 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f872c017020 con 0x7f873416daf0 2023-10-10T20:28:30.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 msgr2=0x7f87140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 0x7f87140693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8724009fd0 tx=0x7f872400b040 comp rx=0 tx=0).stop 2023-10-10T20:28:30.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 msgr2=0x7f8734145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f87340640c0 tx=0x7f872c015000 comp rx=0 tx=0).stop 2023-10-10T20:28:30.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 shutdown_connections 2023-10-10T20:28:30.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8714066f30 0x7f87140693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f873416daf0 0x7f8734145dd0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87200052e0 0x7f87341588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 --2- 172.21.15.148:0/3896007950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8734148fc0 0x7f873416d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 >> 172.21.15.148:0/3896007950 conn(0x7f87340680f0 msgr2=0x7f873406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:30.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 shutdown_connections 2023-10-10T20:28:30.015 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.008+0000 7f873880f700 1 -- 172.21.15.148:0/3896007950 wait complete. 2023-10-10T20:28:30.023 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:28:30.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.180+0000 7f3dad6dd700 1 Processor -- start 2023-10-10T20:28:30.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.180+0000 7f3dad6dd700 1 -- start start 2023-10-10T20:28:30.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.180+0000 7f3dad6dd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 0x7f3da815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3dad6dd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3da80b46f0 0x7f3da80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3dad6dd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80b17b0 con 0x7f3da80b6a10 2023-10-10T20:28:30.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3dad6dd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da80b5020 con 0x7f3da80b46f0 2023-10-10T20:28:30.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3dad6dd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da80b1a60 con 0x7f3da8159ef0 2023-10-10T20:28:30.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3da80b46f0 0x7f3da80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3d9e7fc700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3da80b6a10 0x7f3da80b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48422/0 (socket says 172.21.15.148:48422) 2023-10-10T20:28:30.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3d9e7fc700 1 -- 172.21.15.148:0/2467475319 learned_addr learned my addr 172.21.15.148:0/2467475319 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:30.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da6ffd700 1 --2- 172.21.15.148:0/2467475319 >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 0x7f3da815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 -- 172.21.15.148:0/2467475319 >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 msgr2=0x7f3da815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:30.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 --2- 172.21.15.148:0/2467475319 >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 0x7f3da815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 -- 172.21.15.148:0/2467475319 >> v1:172.21.15.148:6789/0 conn(0x7f3da80b6a10 legacy=0x7f3da80b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 -- 172.21.15.148:0/2467475319 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80b1d40 con 0x7f3da80b46f0 2023-10-10T20:28:30.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da67fc700 1 -- 172.21.15.148:0/2467475319 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940101991 0 0) 0x7f3da80b17b0 con 0x7f3da80b6a10 2023-10-10T20:28:30.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da6ffd700 1 --2- 172.21.15.148:0/2467475319 >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 0x7f3da815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:30.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da77fe700 1 --2- 172.21.15.148:0/2467475319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da80b46f0 0x7f3da80b4ae0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f3da808a710 tx=0x7f3da0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0928709c5788d53 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da67fc700 1 -- 172.21.15.148:0/2467475319 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000e070 con 0x7f3da80b46f0 2023-10-10T20:28:30.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da67fc700 1 -- 172.21.15.148:0/2467475319 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da000bc30 con 0x7f3da80b46f0 2023-10-10T20:28:30.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.181+0000 7f3da67fc700 1 -- 172.21.15.148:0/2467475319 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0012430 con 0x7f3da80b46f0 2023-10-10T20:28:30.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 -- 172.21.15.148:0/2467475319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da80b46f0 msgr2=0x7f3da80b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/2467475319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da80b46f0 0x7f3da80b4ae0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f3da808a710 tx=0x7f3da0009780 comp rx=0 tx=0).stop 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 -- 172.21.15.148:0/2467475319 shutdown_connections 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/2467475319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da80b46f0 0x7f3da80b4ae0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/2467475319 >> v2:172.21.15.161:3300/0 conn(0x7f3da8159ef0 0x7f3da815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 -- 172.21.15.148:0/2467475319 >> 172.21.15.148:0/2467475319 conn(0x7f3da809a0a0 msgr2=0x7f3da8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:30.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 -- 172.21.15.148:0/2467475319 shutdown_connections 2023-10-10T20:28:30.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.182+0000 7f3dad6dd700 1 -- 172.21.15.148:0/2467475319 wait complete. 2023-10-10T20:28:30.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.183+0000 7f3dad6dd700 1 Processor -- start 2023-10-10T20:28:30.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.183+0000 7f3dad6dd700 1 -- start start 2023-10-10T20:28:30.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3da80b46f0 0x7f3da80ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da80b6a10 0x7f3da809d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80af030 con 0x7f3da8159ef0 2023-10-10T20:28:30.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80af310 con 0x7f3da80b6a10 2023-10-10T20:28:30.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3dad6dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80af5f0 con 0x7f3da80b46f0 2023-10-10T20:28:30.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45564/0 (socket says 172.21.15.148:45564) 2023-10-10T20:28:30.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 -- 172.21.15.148:0/187276821 learned_addr learned my addr 172.21.15.148:0/187276821 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:30.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3d9e7fc700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da80b6a10 0x7f3da809d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da6ffd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3da80b46f0 0x7f3da80ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 -- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3da80b46f0 msgr2=0x7f3da80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3da80b46f0 0x7f3da80ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 -- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da80b6a10 msgr2=0x7f3da809d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da80b6a10 0x7f3da809d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da0009a20 con 0x7f3da8159ef0 2023-10-10T20:28:30.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3da77fe700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3da8071f80 tx=0x7f3da0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da001d630 con 0x7f3da8159ef0 2023-10-10T20:28:30.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da001d790 con 0x7f3da8159ef0 2023-10-10T20:28:30.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.184+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da001db50 con 0x7f3da8159ef0 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.185+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da80af8d0 con 0x7f3da8159ef0 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.185+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da8158080 con 0x7f3da8159ef0 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da804c3c0 con 0x7f3da8159ef0 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3da0024070 con 0x7f3da8159ef0 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3d9f7fe700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 0x7f3d880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f3da0086ed0 con 0x7f3da8159ef0 2023-10-10T20:28:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3da6ffd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 0x7f3d880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.186+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7f3d8806ebb0 con 0x7f3da8159ef0 2023-10-10T20:28:30.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.187+0000 7f3da6ffd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 0x7f3d880693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800aaf0 tx=0x7f3d98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.191+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3da00540b0 con 0x7f3da8159ef0 2023-10-10T20:28:30.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.373+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3da80abd30 con 0x7f3da8159ef0 2023-10-10T20:28:30.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.569+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(152..152 src has 1..152) v4 ==== 679+0+0 (secure 0 0 0) 0x7f3da0058470 con 0x7f3da8159ef0 2023-10-10T20:28:30.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.570+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7f3d8806cfe0 con 0x7f3da8159ef0 2023-10-10T20:28:30.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.572+0000 7f3d9f7fe700 1 -- 172.21.15.148:0/187276821 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v152) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3da002c020 con 0x7f3da8159ef0 2023-10-10T20:28:30.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 msgr2=0x7f3d880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 0x7f3d880693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800aaf0 tx=0x7f3d98008040 comp rx=0 tx=0).stop 2023-10-10T20:28:30.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 msgr2=0x7f3da809def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3da8071f80 tx=0x7f3da0019000 comp rx=0 tx=0).stop 2023-10-10T20:28:30.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 shutdown_connections 2023-10-10T20:28:30.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.578+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3d88066f30 0x7f3d880693e0 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800aaf0 tx=0x7f3d98008040 comp rx=0 tx=0).stop 2023-10-10T20:28:30.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3da8159ef0 0x7f3da809def0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3da80b46f0 0x7f3da80ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 --2- 172.21.15.148:0/187276821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3da80b6a10 0x7f3da809d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 >> 172.21.15.148:0/187276821 conn(0x7f3da809a0a0 msgr2=0x7f3da815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:30.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 shutdown_connections 2023-10-10T20:28:30.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.579+0000 7f3dad6dd700 1 -- 172.21.15.148:0/187276821 wait complete. 2023-10-10T20:28:30.584 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:28:30.594 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:28:30.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.712+0000 7f5958ed9700 1 Processor -- start 2023-10-10T20:28:30.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.712+0000 7f5958ed9700 1 -- start start 2023-10-10T20:28:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5958ed9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f595406f2c0 0x7f5954070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5958ed9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5954068250 0x7f5954070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5958ed9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5954067dc0 con 0x7f5954159e50 2023-10-10T20:28:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5958ed9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540cb9e0 con 0x7f5954068250 2023-10-10T20:28:30.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5958ed9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595406fa70 con 0x7f595406f2c0 2023-10-10T20:28:30.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5952d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5954068250 0x7f5954070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5952d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5954068250 0x7f5954070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45576/0 (socket says 172.21.15.148:45576) 2023-10-10T20:28:30.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.713+0000 7f5952d9d700 1 -- 172.21.15.148:0/386515243 learned_addr learned my addr 172.21.15.148:0/386515243 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:30.719 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5949d9b700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f595406f2c0 0x7f5954070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5952d9d700 1 -- 172.21.15.148:0/386515243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f595406f2c0 msgr2=0x7f5954070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5952d9d700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f595406f2c0 0x7f5954070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5952d9d700 1 -- 172.21.15.148:0/386515243 >> v1:172.21.15.161:6789/0 conn(0x7f5954159e50 legacy=0x7f595406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5952d9d700 1 -- 172.21.15.148:0/386515243 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595406f790 con 0x7f5954068250 2023-10-10T20:28:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5952d9d700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954068250 0x7f5954070c40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f20 tx=0x7f594c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b5a11c43f49b35 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5951d9b700 1 -- 172.21.15.148:0/386515243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00e070 con 0x7f5954068250 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5951d9b700 1 -- 172.21.15.148:0/386515243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00ba80 con 0x7f5954068250 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5951d9b700 1 -- 172.21.15.148:0/386515243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0092e0 con 0x7f5954068250 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 -- 172.21.15.148:0/386515243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954068250 msgr2=0x7f5954070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954068250 0x7f5954070c40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f20 tx=0x7f594c009840 comp rx=0 tx=0).stop 2023-10-10T20:28:30.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 -- 172.21.15.148:0/386515243 shutdown_connections 2023-10-10T20:28:30.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5954068250 0x7f5954070c40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 --2- 172.21.15.148:0/386515243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f595406f2c0 0x7f5954070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.714+0000 7f5958ed9700 1 -- 172.21.15.148:0/386515243 >> 172.21.15.148:0/386515243 conn(0x7f59540cdae0 msgr2=0x7f59540cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:30.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.715+0000 7f5958ed9700 1 -- 172.21.15.148:0/386515243 shutdown_connections 2023-10-10T20:28:30.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.715+0000 7f5958ed9700 1 -- 172.21.15.148:0/386515243 wait complete. 2023-10-10T20:28:30.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 Processor -- start 2023-10-10T20:28:30.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 -- start start 2023-10-10T20:28:30.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5954068250 0x7f59540b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5954159e50 0x7f59540aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540c61c0 con 0x7f595406f2c0 2023-10-10T20:28:30.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540c4b90 con 0x7f5954159e50 2023-10-10T20:28:30.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.716+0000 7f5958ed9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595416b100 con 0x7f5954068250 2023-10-10T20:28:30.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45580/0 (socket says 172.21.15.148:45580) 2023-10-10T20:28:30.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 -- 172.21.15.148:0/2452065663 learned_addr learned my addr 172.21.15.148:0/2452065663 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:30.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f595259c700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5954068250 0x7f59540b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5952d9d700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5954159e50 0x7f59540aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 -- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5954068250 msgr2=0x7f59540b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5954068250 0x7f59540b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 -- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5954159e50 msgr2=0x7f59540aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:30.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5954159e50 0x7f59540aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:30.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 -- 172.21.15.148:0/2452065663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009ae0 con 0x7f595406f2c0 2023-10-10T20:28:30.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5949d9b700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d890 tx=0x7f593c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0125f0 con 0x7f595406f2c0 2023-10-10T20:28:30.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c012750 con 0x7f595406f2c0 2023-10-10T20:28:30.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c012af0 con 0x7f595406f2c0 2023-10-10T20:28:30.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540b0d80 con 0x7f595406f2c0 2023-10-10T20:28:30.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.717+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540b1340 con 0x7f595406f2c0 2023-10-10T20:28:30.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.718+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595404c3c0 con 0x7f595406f2c0 2023-10-10T20:28:30.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.719+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f593c01e460 con 0x7f595406f2c0 2023-10-10T20:28:30.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.719+0000 7f594affd700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 0x7f59340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:30.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.719+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f593c0849a0 con 0x7f595406f2c0 2023-10-10T20:28:30.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.719+0000 7f595259c700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 0x7f59340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:30.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.719+0000 7f595259c700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 0x7f59340693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f59440039e0 tx=0x7f5944007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:30.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.723+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f593c0557c0 con 0x7f595406f2c0 2023-10-10T20:28:30.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:30.913+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5954067770 con 0x7f595406f2c0 2023-10-10T20:28:31.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.590+0000 7f594affd700 1 -- 172.21.15.148:0/2452065663 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v153) v1 ==== 169+0+0 (secure 0 0 0) 0x7f593c055350 con 0x7f595406f2c0 2023-10-10T20:28:31.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 msgr2=0x7f59340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 0x7f59340693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f59440039e0 tx=0x7f5944007ec0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 msgr2=0x7f595416abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d890 tx=0x7f593c008040 comp rx=0 tx=0).stop 2023-10-10T20:28:31.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 shutdown_connections 2023-10-10T20:28:31.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5934066f30 0x7f59340693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5954159e50 0x7f59540aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5954068250 0x7f59540b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 --2- 172.21.15.148:0/2452065663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f595406f2c0 0x7f595416abc0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.596+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 >> 172.21.15.148:0/2452065663 conn(0x7f59540cdae0 msgr2=0x7f5954068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:31.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.597+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 shutdown_connections 2023-10-10T20:28:31.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.598+0000 7f5958ed9700 1 -- 172.21.15.148:0/2452065663 wait complete. 2023-10-10T20:28:31.602 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:28:31.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:28:31.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.728+0000 7f0eef287700 1 Processor -- start 2023-10-10T20:28:31.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.728+0000 7f0eef287700 1 -- start start 2023-10-10T20:28:31.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eef287700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ee80cce30 0x7f0ee815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eef287700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0ee806aa00 0x7f0ee80b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eef287700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee80b4d70 con 0x7f0ee80b6580 2023-10-10T20:28:31.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eef287700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee80cd8d0 con 0x7f0ee80cce30 2023-10-10T20:28:31.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eef287700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee80b4440 con 0x7f0ee806aa00 2023-10-10T20:28:31.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eee285700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ee80cce30 0x7f0ee815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eeda84700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ee80b6580 0x7f0ee806a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48432/0 (socket says 172.21.15.148:48432) 2023-10-10T20:28:31.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.729+0000 7f0eeda84700 1 -- 172.21.15.148:0/529789952 learned_addr learned my addr 172.21.15.148:0/529789952 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:31.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eeea86700 1 --2- 172.21.15.148:0/529789952 >> v2:172.21.15.161:3300/0 conn(0x7f0ee806aa00 0x7f0ee80b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eee285700 1 -- 172.21.15.148:0/529789952 >> v2:172.21.15.161:3300/0 conn(0x7f0ee806aa00 msgr2=0x7f0ee80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:31.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eee285700 1 --2- 172.21.15.148:0/529789952 >> v2:172.21.15.161:3300/0 conn(0x7f0ee806aa00 0x7f0ee80b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eed283700 1 -- 172.21.15.148:0/529789952 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848668333 0 0) 0x7f0ee80b4d70 con 0x7f0ee80b6580 2023-10-10T20:28:31.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eee285700 1 -- 172.21.15.148:0/529789952 >> v1:172.21.15.148:6789/0 conn(0x7f0ee80b6580 legacy=0x7f0ee806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eee285700 1 -- 172.21.15.148:0/529789952 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee80b4720 con 0x7f0ee80cce30 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eee285700 1 --2- 172.21.15.148:0/529789952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee815aca0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b4ce0 tx=0x7f0ee4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c3fef8562197c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eed283700 1 -- 172.21.15.148:0/529789952 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400e070 con 0x7f0ee80cce30 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eed283700 1 -- 172.21.15.148:0/529789952 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400bc30 con 0x7f0ee80cce30 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eed283700 1 -- 172.21.15.148:0/529789952 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4012430 con 0x7f0ee80cce30 2023-10-10T20:28:31.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eef287700 1 -- 172.21.15.148:0/529789952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 msgr2=0x7f0ee815aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.730+0000 7f0eef287700 1 --2- 172.21.15.148:0/529789952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee815aca0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b4ce0 tx=0x7f0ee4009780 comp rx=0 tx=0).stop 2023-10-10T20:28:31.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 -- 172.21.15.148:0/529789952 shutdown_connections 2023-10-10T20:28:31.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 --2- 172.21.15.148:0/529789952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee815aca0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 --2- 172.21.15.148:0/529789952 >> v2:172.21.15.161:3300/0 conn(0x7f0ee806aa00 0x7f0ee80b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 -- 172.21.15.148:0/529789952 >> 172.21.15.148:0/529789952 conn(0x7f0ee8068cb0 msgr2=0x7f0ee80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:31.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 -- 172.21.15.148:0/529789952 shutdown_connections 2023-10-10T20:28:31.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.731+0000 7f0eef287700 1 -- 172.21.15.148:0/529789952 wait complete. 2023-10-10T20:28:31.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 Processor -- start 2023-10-10T20:28:31.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 -- start start 2023-10-10T20:28:31.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee806aa00 0x7f0ee81534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ee80b6580 0x7f0ee814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8154ee0 con 0x7f0ee80cce30 2023-10-10T20:28:31.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8153a20 con 0x7f0ee806aa00 2023-10-10T20:28:31.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.732+0000 7f0eef287700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8153cf0 con 0x7f0ee80b6580 2023-10-10T20:28:31.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45600/0 (socket says 172.21.15.148:45600) 2023-10-10T20:28:31.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 -- 172.21.15.148:0/1762220720 learned_addr learned my addr 172.21.15.148:0/1762220720 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:31.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eee285700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee806aa00 0x7f0ee81534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeda84700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ee80b6580 0x7f0ee814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 -- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ee80b6580 msgr2=0x7f0ee814f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ee80b6580 0x7f0ee814f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 -- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee806aa00 msgr2=0x7f0ee81534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee806aa00 0x7f0ee81534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 -- 172.21.15.148:0/1762220720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009a20 con 0x7f0ee80cce30 2023-10-10T20:28:31.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eeea86700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d890 tx=0x7f0ee0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:31.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00125f0 con 0x7f0ee80cce30 2023-10-10T20:28:31.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee0012750 con 0x7f0ee80cce30 2023-10-10T20:28:31.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8153fd0 con 0x7f0ee80cce30 2023-10-10T20:28:31.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0012b10 con 0x7f0ee80cce30 2023-10-10T20:28:31.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.733+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee8154b70 con 0x7f0ee80cce30 2023-10-10T20:28:31.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.734+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8155090 con 0x7f0ee80cce30 2023-10-10T20:28:31.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.734+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ee0013480 con 0x7f0ee80cce30 2023-10-10T20:28:31.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.735+0000 7f0ede7fc700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:31.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.735+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0ee0084560 con 0x7f0ee80cce30 2023-10-10T20:28:31.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.735+0000 7f0eee285700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:31.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.735+0000 7f0eee285700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 0x7f0ecc0693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b4ce0 tx=0x7f0ee4019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:31.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.739+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ee8155090 con 0x7f0ee80cce30 2023-10-10T20:28:31.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.921+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0ee8155020 con 0x7f0ee80cce30 2023-10-10T20:28:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.924+0000 7f0ede7fc700 1 -- 172.21.15.148:0/1762220720 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ee0051ad0 con 0x7f0ee80cce30 2023-10-10T20:28:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 msgr2=0x7f0ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 0x7f0ecc0693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b4ce0 tx=0x7f0ee4019000 comp rx=0 tx=0).stop 2023-10-10T20:28:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 msgr2=0x7f0ee816d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000d890 tx=0x7f0ee0008040 comp rx=0 tx=0).stop 2023-10-10T20:28:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 shutdown_connections 2023-10-10T20:28:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ecc066f30 0x7f0ecc0693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ee80cce30 0x7f0ee816d660 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.926+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee806aa00 0x7f0ee81534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.927+0000 7f0eef287700 1 --2- 172.21.15.148:0/1762220720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ee80b6580 0x7f0ee814f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.927+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 >> 172.21.15.148:0/1762220720 conn(0x7f0ee8068cb0 msgr2=0x7f0ee81503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.927+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 shutdown_connections 2023-10-10T20:28:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:31.927+0000 7f0eef287700 1 -- 172.21.15.148:0/1762220720 wait complete. 2023-10-10T20:28:31.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-10T20:28:32.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 Processor -- start 2023-10-10T20:28:32.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 -- start start 2023-10-10T20:28:32.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ac80b0d30 0x7f9ac80b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac815e860 con 0x7f9ac80c9180 2023-10-10T20:28:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac815da40 con 0x7f9ac80b0d30 2023-10-10T20:28:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9acd2c9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815dc50 con 0x7f9ac80cbce0 2023-10-10T20:28:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.055+0000 7f9ac6ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ac80b0d30 0x7f9ac80b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ac80b0d30 0x7f9ac80b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45610/0 (socket says 172.21.15.148:45610) 2023-10-10T20:28:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 -- 172.21.15.148:0/1660734361 learned_addr learned my addr 172.21.15.148:0/1660734361 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac77fe700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 -- 172.21.15.148:0/1660734361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 msgr2=0x7f9ac80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 -- 172.21.15.148:0/1660734361 >> v1:172.21.15.161:6789/0 conn(0x7f9ac80c9180 legacy=0x7f9ac80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 -- 172.21.15.148:0/1660734361 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac815df30 con 0x7f9ac80b0d30 2023-10-10T20:28:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac6ffd700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80b0d30 0x7f9ac80b1100 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009f20 tx=0x7f9ab8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c82509f0ced5bea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac5ffb700 1 -- 172.21.15.148:0/1660734361 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800e070 con 0x7f9ac80b0d30 2023-10-10T20:28:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac5ffb700 1 -- 172.21.15.148:0/1660734361 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab800ba80 con 0x7f9ac80b0d30 2023-10-10T20:28:32.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.056+0000 7f9ac5ffb700 1 -- 172.21.15.148:0/1660734361 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80092e0 con 0x7f9ac80b0d30 2023-10-10T20:28:32.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 -- 172.21.15.148:0/1660734361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80b0d30 msgr2=0x7f9ac80b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80b0d30 0x7f9ac80b1100 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009f20 tx=0x7f9ab8009840 comp rx=0 tx=0).stop 2023-10-10T20:28:32.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 -- 172.21.15.148:0/1660734361 shutdown_connections 2023-10-10T20:28:32.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80b0d30 0x7f9ac80b1100 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/1660734361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 -- 172.21.15.148:0/1660734361 >> 172.21.15.148:0/1660734361 conn(0x7f9ac8069870 msgr2=0x7f9ac8069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:32.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 -- 172.21.15.148:0/1660734361 shutdown_connections 2023-10-10T20:28:32.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.057+0000 7f9acd2c9700 1 -- 172.21.15.148:0/1660734361 wait complete. 2023-10-10T20:28:32.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.058+0000 7f9acd2c9700 1 Processor -- start 2023-10-10T20:28:32.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.058+0000 7f9acd2c9700 1 -- start start 2023-10-10T20:28:32.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ac80b0d30 0x7f9ac815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b4990 con 0x7f9ac80c9180 2023-10-10T20:28:32.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac813d290 con 0x7f9ac80b0d30 2023-10-10T20:28:32.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b20d0 con 0x7f9ac80cbce0 2023-10-10T20:28:32.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45622/0 (socket says 172.21.15.148:45622) 2023-10-10T20:28:32.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 -- 172.21.15.148:0/789660667 learned_addr learned my addr 172.21.15.148:0/789660667 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:32.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac77fe700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac6ffd700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ac80b0d30 0x7f9ac815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 -- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 msgr2=0x7f9ac80b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 -- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ac80b0d30 msgr2=0x7f9ac815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ac80b0d30 0x7f9ac815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 -- 172.21.15.148:0/789660667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009ae0 con 0x7f9ac80c9180 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9ac67fc700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c4e0 tx=0x7f9abc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc011590 con 0x7f9ac80c9180 2023-10-10T20:28:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc011710 con 0x7f9ac80c9180 2023-10-10T20:28:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac815cc80 con 0x7f9ac80c9180 2023-10-10T20:28:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc011ad0 con 0x7f9ac80c9180 2023-10-10T20:28:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.059+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80b1d60 con 0x7f9ac80c9180 2023-10-10T20:28:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.060+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac804c3c0 con 0x7f9ac80c9180 2023-10-10T20:28:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.061+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9abc01b3b0 con 0x7f9ac80c9180 2023-10-10T20:28:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.061+0000 7f9aaeffd700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 0x7f9ab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.061+0000 7f9ac6ffd700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 0x7f9ab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.061+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9abc083630 con 0x7f9ac80c9180 2023-10-10T20:28:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.062+0000 7f9ac6ffd700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 0x7f9ab40693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009880 tx=0x7f9ab8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.066+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9abc050ba0 con 0x7f9ac80c9180 2023-10-10T20:28:32.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.325+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f9ac80b2210 con 0x7f9ac80c9180 2023-10-10T20:28:32.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.590+0000 7f9aaeffd700 1 -- 172.21.15.148:0/789660667 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v437) v1 ==== 132+0+0 (secure 0 0 0) 0x7f9abc017020 con 0x7f9ac80c9180 2023-10-10T20:28:32.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 msgr2=0x7f9ab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 0x7f9ab40693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009880 tx=0x7f9ab8018000 comp rx=0 tx=0).stop 2023-10-10T20:28:32.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 msgr2=0x7f9ac815b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c4e0 tx=0x7f9abc00c7f0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 shutdown_connections 2023-10-10T20:28:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9ab4066f30 0x7f9ab40693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ac80cbce0 0x7f9ac80b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ac80b0d30 0x7f9ac815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 --2- 172.21.15.148:0/789660667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ac80c9180 0x7f9ac815b3e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.592+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 >> 172.21.15.148:0/789660667 conn(0x7f9ac8069870 msgr2=0x7f9ac8158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:32.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.593+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 shutdown_connections 2023-10-10T20:28:32.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.593+0000 7f9acd2c9700 1 -- 172.21.15.148:0/789660667 wait complete. 2023-10-10T20:28:32.607 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-10T20:28:32.608 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-10-10T20:28:32.608 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-10-10T20:28:32.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-10T20:28:32.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 Processor -- start 2023-10-10T20:28:32.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 -- start start 2023-10-10T20:28:32.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f951c157870 0x7f951c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c15cbb0 con 0x7f951c15a620 2023-10-10T20:28:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c15bfa0 con 0x7f951c157870 2023-10-10T20:28:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.721+0000 7f9521304700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15c280 con 0x7f951c157c40 2023-10-10T20:28:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f951c157870 0x7f951c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f951c157870 0x7f951c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35950/0 (socket says 172.21.15.148:35950) 2023-10-10T20:28:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 -- 172.21.15.148:0/2290359610 learned_addr learned my addr 172.21.15.148:0/2290359610 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951b7fe700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 -- 172.21.15.148:0/2290359610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 msgr2=0x7f951c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 -- 172.21.15.148:0/2290359610 >> v1:172.21.15.161:6789/0 conn(0x7f951c15a620 legacy=0x7f951c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 -- 172.21.15.148:0/2290359610 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c15c560 con 0x7f951c157870 2023-10-10T20:28:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951affd700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c157870 0x7f951c15b6a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f950c0099b0 tx=0x7f950c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1299e55920d92b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951a7fc700 1 -- 172.21.15.148:0/2290359610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00b860 con 0x7f951c157870 2023-10-10T20:28:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951a7fc700 1 -- 172.21.15.148:0/2290359610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c00b9c0 con 0x7f951c157870 2023-10-10T20:28:32.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.722+0000 7f951a7fc700 1 -- 172.21.15.148:0/2290359610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00bd60 con 0x7f951c157870 2023-10-10T20:28:32.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 -- 172.21.15.148:0/2290359610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c157870 msgr2=0x7f951c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c157870 0x7f951c15b6a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f950c0099b0 tx=0x7f950c009250 comp rx=0 tx=0).stop 2023-10-10T20:28:32.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 -- 172.21.15.148:0/2290359610 shutdown_connections 2023-10-10T20:28:32.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c157870 0x7f951c15b6a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 --2- 172.21.15.148:0/2290359610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 -- 172.21.15.148:0/2290359610 >> 172.21.15.148:0/2290359610 conn(0x7f951c068ce0 msgr2=0x7f951c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 -- 172.21.15.148:0/2290359610 shutdown_connections 2023-10-10T20:28:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.723+0000 7f9521304700 1 -- 172.21.15.148:0/2290359610 wait complete. 2023-10-10T20:28:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.724+0000 7f9521304700 1 Processor -- start 2023-10-10T20:28:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.724+0000 7f9521304700 1 -- start start 2023-10-10T20:28:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f951c157870 0x7f951c148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c0c6b90 con 0x7f951c15a620 2023-10-10T20:28:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15d640 con 0x7f951c157870 2023-10-10T20:28:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c148e80 con 0x7f951c157c40 2023-10-10T20:28:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35956/0 (socket says 172.21.15.148:35956) 2023-10-10T20:28:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 -- 172.21.15.148:0/3795255147 learned_addr learned my addr 172.21.15.148:0/3795255147 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951affd700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f951c157870 0x7f951c148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f95127fc700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 -- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 msgr2=0x7f951c0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 -- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f951c157870 msgr2=0x7f951c148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f951c157870 0x7f951c148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 -- 172.21.15.148:0/3795255147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c0094f0 con 0x7f951c15a620 2023-10-10T20:28:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f951b7fe700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9514007f60 tx=0x7f95140098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400ec40 con 0x7f951c15a620 2023-10-10T20:28:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951400eda0 con 0x7f951c15a620 2023-10-10T20:28:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0c6050 con 0x7f951c15a620 2023-10-10T20:28:32.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.725+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514014670 con 0x7f951c15a620 2023-10-10T20:28:32.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.726+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0c65d0 con 0x7f951c15a620 2023-10-10T20:28:32.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.727+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c15d7f0 con 0x7f951c15a620 2023-10-10T20:28:32.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.727+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9514014a60 con 0x7f951c15a620 2023-10-10T20:28:32.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.727+0000 7f95137fe700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 0x7f94fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:32.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.727+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9514084870 con 0x7f951c15a620 2023-10-10T20:28:32.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.727+0000 7f951affd700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 0x7f94fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:32.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.728+0000 7f951affd700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 0x7f94fc0693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f950c0099b0 tx=0x7f950c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:32.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.732+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f951c15d7f0 con 0x7f951c15a620 2023-10-10T20:28:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:32.989+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f951c15d7f0 con 0x7f951c15a620 2023-10-10T20:28:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.595+0000 7f95137fe700 1 -- 172.21.15.148:0/3795255147 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v438) v1 ==== 134+0+0 (secure 0 0 0) 0x7f951c15d7f0 con 0x7f951c15a620 2023-10-10T20:28:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 msgr2=0x7f94fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 0x7f94fc0693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f950c0099b0 tx=0x7f950c017000 comp rx=0 tx=0).stop 2023-10-10T20:28:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 msgr2=0x7f951c0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9514007f60 tx=0x7f95140098e0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 shutdown_connections 2023-10-10T20:28:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f94fc066f30 0x7f94fc0693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951c15a620 0x7f951c0c7320 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f951c157870 0x7f951c148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 --2- 172.21.15.148:0/3795255147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f951c157c40 0x7f951c0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 >> 172.21.15.148:0/3795255147 conn(0x7f951c068ce0 msgr2=0x7f951c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:33.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 shutdown_connections 2023-10-10T20:28:33.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.597+0000 7f9521304700 1 -- 172.21.15.148:0/3795255147 wait complete. 2023-10-10T20:28:33.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:28:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.730+0000 7fd0426f1700 1 Processor -- start 2023-10-10T20:28:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.730+0000 7fd0426f1700 1 -- start start 2023-10-10T20:28:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0426f1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0426f1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 0x7fd03c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0426f1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c0b6b10 con 0x7fd03c159e80 2023-10-10T20:28:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0426f1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0b0e70 con 0x7fd03c0b4b60 2023-10-10T20:28:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0426f1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0b5000 con 0x7fd03c0b1300 2023-10-10T20:28:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35966/0 (socket says 172.21.15.148:35966) 2023-10-10T20:28:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd0416ef700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd03c159e80 0x7fd03c0aada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57018/0 (socket says 172.21.15.148:57018) 2023-10-10T20:28:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 -- 172.21.15.148:0/2466275699 learned_addr learned my addr 172.21.15.148:0/2466275699 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd041ef0700 1 --2- 172.21.15.148:0/2466275699 >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 0x7fd03c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 -- 172.21.15.148:0/2466275699 >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 msgr2=0x7fd03c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 --2- 172.21.15.148:0/2466275699 >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 0x7fd03c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 -- 172.21.15.148:0/2466275699 >> v1:172.21.15.148:6789/0 conn(0x7fd03c159e80 legacy=0x7fd03c0aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd040eee700 1 -- 172.21.15.148:0/2466275699 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0b52c0 con 0x7fd03c0b4b60 2023-10-10T20:28:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.731+0000 7fd041ef0700 1 --2- 172.21.15.148:0/2466275699 >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 0x7fd03c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd040eee700 1 --2- 172.21.15.148:0/2466275699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08aec0 tx=0x7fd02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d28a1abf87b8858 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd033fff700 1 -- 172.21.15.148:0/2466275699 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00b860 con 0x7fd03c0b4b60 2023-10-10T20:28:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd033fff700 1 -- 172.21.15.148:0/2466275699 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c00b9c0 con 0x7fd03c0b4b60 2023-10-10T20:28:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd033fff700 1 -- 172.21.15.148:0/2466275699 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00bd60 con 0x7fd03c0b4b60 2023-10-10T20:28:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 -- 172.21.15.148:0/2466275699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b4b60 msgr2=0x7fd03c0ab4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 --2- 172.21.15.148:0/2466275699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08aec0 tx=0x7fd02c009250 comp rx=0 tx=0).stop 2023-10-10T20:28:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 -- 172.21.15.148:0/2466275699 shutdown_connections 2023-10-10T20:28:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 --2- 172.21.15.148:0/2466275699 >> v2:172.21.15.161:3300/0 conn(0x7fd03c0b1300 0x7fd03c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 --2- 172.21.15.148:0/2466275699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b4b60 0x7fd03c0ab4b0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 -- 172.21.15.148:0/2466275699 >> 172.21.15.148:0/2466275699 conn(0x7fd03c099f50 msgr2=0x7fd03c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 -- 172.21.15.148:0/2466275699 shutdown_connections 2023-10-10T20:28:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.732+0000 7fd0426f1700 1 -- 172.21.15.148:0/2466275699 wait complete. 2023-10-10T20:28:33.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.733+0000 7fd0426f1700 1 Processor -- start 2023-10-10T20:28:33.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.733+0000 7fd0426f1700 1 -- start start 2023-10-10T20:28:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 0x7fd03c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 0x7fd03c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b01c0 con 0x7fd03c0b1300 2023-10-10T20:28:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c09bf20 con 0x7fd03c159e80 2023-10-10T20:28:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0426f1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c09c960 con 0x7fd03c0b4b60 2023-10-10T20:28:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0416ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0416ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35974/0 (socket says 172.21.15.148:35974) 2023-10-10T20:28:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.734+0000 7fd0416ef700 1 -- 172.21.15.148:0/1783691243 learned_addr learned my addr 172.21.15.148:0/1783691243 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:33.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd041ef0700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 0x7fd03c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd040eee700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 0x7fd03c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 -- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 msgr2=0x7fd03c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 0x7fd03c15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 -- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 msgr2=0x7fd03c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 0x7fd03c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 -- 172.21.15.148:0/1783691243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0094f0 con 0x7fd03c0b1300 2023-10-10T20:28:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd041ef0700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 0x7fd03c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd040eee700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 0x7fd03c15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0416ef700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c7c0 tx=0x7fd038007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:33.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038005a70 con 0x7fd03c0b1300 2023-10-10T20:28:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038005bf0 con 0x7fd03c0b1300 2023-10-10T20:28:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380070f0 con 0x7fd03c0b1300 2023-10-10T20:28:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c09bc40 con 0x7fd03c0b1300 2023-10-10T20:28:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.735+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c09c5f0 con 0x7fd03c0b1300 2023-10-10T20:28:33.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.736+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c089110 con 0x7fd03c0b1300 2023-10-10T20:28:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.736+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd038007460 con 0x7fd03c0b1300 2023-10-10T20:28:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.737+0000 7fd0317fa700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 0x7fd020069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.737+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd038083d30 con 0x7fd03c0b1300 2023-10-10T20:28:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.737+0000 7fd040eee700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 0x7fd020069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:33.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.737+0000 7fd040eee700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 0x7fd020069460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08aec0 tx=0x7fd02c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:33.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.741+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd038089020 con 0x7fd03c0b1300 2023-10-10T20:28:33.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.924+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd03c0b0310 con 0x7fd03c0b1300 2023-10-10T20:28:33.929 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.927+0000 7fd0317fa700 1 -- 172.21.15.148:0/1783691243 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd038051280 con 0x7fd03c0b1300 2023-10-10T20:28:33.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.929+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 msgr2=0x7fd020069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.932 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.929+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 0x7fd020069460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08aec0 tx=0x7fd02c017000 comp rx=0 tx=0).stop 2023-10-10T20:28:33.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 msgr2=0x7fd03c09b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:33.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c7c0 tx=0x7fd038007f70 comp rx=0 tx=0).stop 2023-10-10T20:28:33.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 shutdown_connections 2023-10-10T20:28:33.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd020066fb0 0x7fd020069460 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.933 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd03c159e80 0x7fd03c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd03c0b1300 0x7fd03c09b270 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 --2- 172.21.15.148:0/1783691243 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd03c0b4b60 0x7fd03c15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:33.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 >> 172.21.15.148:0/1783691243 conn(0x7fd03c099f50 msgr2=0x7fd03c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:33.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 shutdown_connections 2023-10-10T20:28:33.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:33.930+0000 7fd0426f1700 1 -- 172.21.15.148:0/1783691243 wait complete. 2023-10-10T20:28:33.945 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:28:33.945 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:28:33.981 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:28:34.035 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:28:34.035 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:28:34.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.148+0000 7fb91e704700 1 Processor -- start 2023-10-10T20:28:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.148+0000 7fb91e704700 1 -- start start 2023-10-10T20:28:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91e704700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9180cd6e0 0x7fb9180cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91e704700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb91815cbd0 0x7fb91815d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91e704700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9180cb7c0 con 0x7fb9180cdab0 2023-10-10T20:28:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91e704700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91815e850 con 0x7fb91815cbd0 2023-10-10T20:28:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91e704700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180cd030 con 0x7fb9180cd6e0 2023-10-10T20:28:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91df03700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb91815cbd0 0x7fb91815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91df03700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb91815cbd0 0x7fb91815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35976/0 (socket says 172.21.15.148:35976) 2023-10-10T20:28:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91df03700 1 -- 172.21.15.148:0/237274025 learned_addr learned my addr 172.21.15.148:0/237274025 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.149+0000 7fb91d702700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9180cd6e0 0x7fb9180cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91df03700 1 -- 172.21.15.148:0/237274025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9180cd6e0 msgr2=0x7fb9180cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91df03700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9180cd6e0 0x7fb9180cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91df03700 1 -- 172.21.15.148:0/237274025 >> v1:172.21.15.161:6789/0 conn(0x7fb9180cdab0 legacy=0x7fb91815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91df03700 1 -- 172.21.15.148:0/237274025 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91815e290 con 0x7fb91815cbd0 2023-10-10T20:28:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91df03700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91815cbd0 0x7fb91815d020 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb910009f20 tx=0x7fb910009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8204c8880836723 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb90ffff700 1 -- 172.21.15.148:0/237274025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000e070 con 0x7fb91815cbd0 2023-10-10T20:28:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb90ffff700 1 -- 172.21.15.148:0/237274025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91000b9d0 con 0x7fb91815cbd0 2023-10-10T20:28:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb90ffff700 1 -- 172.21.15.148:0/237274025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100092e0 con 0x7fb91815cbd0 2023-10-10T20:28:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91e704700 1 -- 172.21.15.148:0/237274025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91815cbd0 msgr2=0x7fb91815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.150+0000 7fb91e704700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91815cbd0 0x7fb91815d020 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb910009f20 tx=0x7fb910009840 comp rx=0 tx=0).stop 2023-10-10T20:28:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 -- 172.21.15.148:0/237274025 shutdown_connections 2023-10-10T20:28:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91815cbd0 0x7fb91815d020 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 --2- 172.21.15.148:0/237274025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb9180cd6e0 0x7fb9180cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 -- 172.21.15.148:0/237274025 >> 172.21.15.148:0/237274025 conn(0x7fb918067660 msgr2=0x7fb918067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 -- 172.21.15.148:0/237274025 shutdown_connections 2023-10-10T20:28:34.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.151+0000 7fb91e704700 1 -- 172.21.15.148:0/237274025 wait complete. 2023-10-10T20:28:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.152+0000 7fb91e704700 1 Processor -- start 2023-10-10T20:28:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.152+0000 7fb91e704700 1 -- start start 2023-10-10T20:28:34.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 0x7fb918158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb91815cbd0 0x7fb91815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180c8d00 con 0x7fb9180cd6e0 2023-10-10T20:28:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180b1df0 con 0x7fb9180cdab0 2023-10-10T20:28:34.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb918155980 con 0x7fb91815cbd0 2023-10-10T20:28:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35992/0 (socket says 172.21.15.148:35992) 2023-10-10T20:28:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 -- 172.21.15.148:0/758130556 learned_addr learned my addr 172.21.15.148:0/758130556 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91df03700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb91815cbd0 0x7fb91815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91cf01700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 0x7fb918158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 -- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb91815cbd0 msgr2=0x7fb91815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb91815cbd0 0x7fb91815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 -- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 msgr2=0x7fb918158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 0x7fb918158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 -- 172.21.15.148:0/758130556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb910009ae0 con 0x7fb9180cd6e0 2023-10-10T20:28:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91cf01700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 0x7fb918158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91d702700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ecb0 tx=0x7fb9140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91400c920 con 0x7fb9180cd6e0 2023-10-10T20:28:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91400ca80 con 0x7fb9180cd6e0 2023-10-10T20:28:34.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9180b0f00 con 0x7fb9180cd6e0 2023-10-10T20:28:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91400ce20 con 0x7fb9180cd6e0 2023-10-10T20:28:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.153+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9180b14c0 con 0x7fb9180cd6e0 2023-10-10T20:28:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.154+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb918089280 con 0x7fb9180cd6e0 2023-10-10T20:28:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.155+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb914013680 con 0x7fb9180cd6e0 2023-10-10T20:28:34.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.155+0000 7fb90d7fa700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.155+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb914084580 con 0x7fb9180cd6e0 2023-10-10T20:28:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.155+0000 7fb91cf01700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.156+0000 7fb91cf01700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 0x7fb8fc0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.159+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb914051af0 con 0x7fb9180cd6e0 2023-10-10T20:28:34.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.383+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb91815e570 con 0x7fb9180cd6e0 2023-10-10T20:28:34.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.383+0000 7fb90d7fa700 1 -- 172.21.15.148:0/758130556 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fb914017df0 con 0x7fb9180cd6e0 2023-10-10T20:28:34.386 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:34.386 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2372109441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2372109441},{"type":"v1","addr":"172.21.15.161:6835","nonce":2372109441}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/4086278446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4086278446},{"type":"v1","addr":"172.21.15.161:6834","nonce":4086278446}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1886244088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1886244088},{"type":"v1","addr":"172.21.15.148:6836","nonce":1886244088}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/792875374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":792875374},{"type":"v1","addr":"172.21.15.148:6837","nonce":792875374}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2023-10-10T20:28:34.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 msgr2=0x7fb8fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 0x7fb8fc0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908009040 comp rx=0 tx=0).stop 2023-10-10T20:28:34.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 msgr2=0x7fb9180b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ecb0 tx=0x7fb9140091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 shutdown_connections 2023-10-10T20:28:34.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb8fc066f30 0x7fb8fc0693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb91815cbd0 0x7fb91815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9180cd6e0 0x7fb9180b2280 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 --2- 172.21.15.148:0/758130556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9180cdab0 0x7fb918158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.385+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 >> 172.21.15.148:0/758130556 conn(0x7fb918067660 msgr2=0x7fb9180b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:34.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.386+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 shutdown_connections 2023-10-10T20:28:34.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.386+0000 7fb91e704700 1 -- 172.21.15.148:0/758130556 wait complete. 2023-10-10T20:28:34.391 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 73 2023-10-10T20:28:34.400 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:28:34.400 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:28:34.401 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:28:34.401 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:28:34.401 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:28:34.401 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:28:34.402 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:28:34.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:28:34.402 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:28:34.403 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:28:34.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:28:34.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:28:34.404 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:28:34.404 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:28:34.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:28:34.405 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:28:34.409 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:28:34.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:28:34.411 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:28:34.411 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:28:34.414 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:28:34.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:28:34.416 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:28:34.416 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:28:34.452 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:28:34.453 DEBUG:teuthology.parallel:result is None 2023-10-10T20:28:34.454 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:28:34.454 DEBUG:teuthology.parallel:result is None 2023-10-10T20:28:34.457 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:28:34.458 DEBUG:teuthology.parallel:result is None 2023-10-10T20:28:34.458 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:28:34.459 DEBUG:teuthology.parallel:result is None 2023-10-10T20:28:34.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:28:34.526 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:28:34.526 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:28:34.538 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:28:34.539 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:28:34.539 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-10-10T20:28:34.538+0000 7f501ac38700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:28:34.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.579+0000 7ffb1011b700 1 Processor -- start 2023-10-10T20:28:34.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.580+0000 7ffb1011b700 1 -- start start 2023-10-10T20:28:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.580+0000 7ffb1011b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb08159d50 0x7ffb080cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.580+0000 7ffb1011b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffb0806daa0 0x7ffb080af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb1011b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb08069d90 con 0x7ffb0815a120 2023-10-10T20:28:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb1011b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb080ccdd0 con 0x7ffb0806daa0 2023-10-10T20:28:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb1011b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08069740 con 0x7ffb08159d50 2023-10-10T20:28:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0d6b6700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffb0815a120 0x7ffb080aee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57028/0 (socket says 172.21.15.148:57028) 2023-10-10T20:28:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0d6b6700 1 -- 172.21.15.148:0/1451557240 learned_addr learned my addr 172.21.15.148:0/1451557240 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1451557240 >> v2:172.21.15.161:3300/0 conn(0x7ffb0806daa0 0x7ffb080af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0deb7700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb08159d50 0x7ffb080cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0ceb5700 1 -- 172.21.15.148:0/1451557240 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513203330 0 0) 0x7ffb08069d90 con 0x7ffb0815a120 2023-10-10T20:28:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0ceb5700 1 -- 172.21.15.148:0/1451557240 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf4003570 con 0x7ffb0815a120 2023-10-10T20:28:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb08159d50 msgr2=0x7ffb080cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb08159d50 0x7ffb080cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1451557240 >> v1:172.21.15.148:6789/0 conn(0x7ffb0815a120 legacy=0x7ffb080aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1451557240 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08069a20 con 0x7ffb0806daa0 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb0806daa0 0x7ffb080af510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009e60 tx=0x7ffb00009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10d4c767bfb808e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0ceb5700 1 -- 172.21.15.148:0/1451557240 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000e070 con 0x7ffb0806daa0 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0ceb5700 1 -- 172.21.15.148:0/1451557240 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0000bab0 con 0x7ffb0806daa0 2023-10-10T20:28:34.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.581+0000 7ffb0ceb5700 1 -- 172.21.15.148:0/1451557240 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000be70 con 0x7ffb0806daa0 2023-10-10T20:28:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 -- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb0806daa0 msgr2=0x7ffb080af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb0806daa0 0x7ffb080af510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009e60 tx=0x7ffb00009780 comp rx=0 tx=0).stop 2023-10-10T20:28:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 -- 172.21.15.148:0/1451557240 shutdown_connections 2023-10-10T20:28:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb08159d50 0x7ffb080cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1451557240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb0806daa0 0x7ffb080af510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 -- 172.21.15.148:0/1451557240 >> 172.21.15.148:0/1451557240 conn(0x7ffb080cd200 msgr2=0x7ffb080cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 -- 172.21.15.148:0/1451557240 shutdown_connections 2023-10-10T20:28:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.582+0000 7ffb1011b700 1 -- 172.21.15.148:0/1451557240 wait complete. 2023-10-10T20:28:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.583+0000 7ffb1011b700 1 Processor -- start 2023-10-10T20:28:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.583+0000 7ffb1011b700 1 -- start start 2023-10-10T20:28:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 0x7ffb0815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 0x7ffb0815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080b0370 con 0x7ffb0815a120 2023-10-10T20:28:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0815a7e0 con 0x7ffb08159d50 2023-10-10T20:28:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb1011b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080b0040 con 0x7ffb0806daa0 2023-10-10T20:28:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36036/0 (socket says 172.21.15.148:36036) 2023-10-10T20:28:34.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1557990630 learned_addr learned my addr 172.21.15.148:0/1557990630 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:34.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0d6b6700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 0x7ffb0815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0deb7700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 0x7ffb0815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 msgr2=0x7ffb0815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 0x7ffb0815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 msgr2=0x7ffb0815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:34.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 0x7ffb0815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:34.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 -- 172.21.15.148:0/1557990630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00009a20 con 0x7ffb0815a120 2023-10-10T20:28:34.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0d6b6700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 0x7ffb0815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0deb7700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 0x7ffb0815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.584+0000 7ffb0e6b8700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb08072050 tx=0x7ffb00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.585+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00027470 con 0x7ffb0815a120 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.585+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000275f0 con 0x7ffb0815a120 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.585+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0815cd00 con 0x7ffb0815a120 2023-10-10T20:28:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.585+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000279b0 con 0x7ffb0815a120 2023-10-10T20:28:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.585+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0815d2c0 con 0x7ffb0815a120 2023-10-10T20:28:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.586+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb080b0520 con 0x7ffb0815a120 2023-10-10T20:28:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.586+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb00024070 con 0x7ffb0815a120 2023-10-10T20:28:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.587+0000 7ffaf9ffb700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.587+0000 7ffb0deb7700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.587+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffb00086b50 con 0x7ffb0815a120 2023-10-10T20:28:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.587+0000 7ffb0deb7700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 0x7ffaec0693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb08160850 tx=0x7ffafc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.591+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb080b0520 con 0x7ffb0815a120 2023-10-10T20:28:34.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:34.791+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb0804c3c0 con 0x7ffb0815a120 2023-10-10T20:28:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.610+0000 7ffaf9ffb700 1 -- 172.21.15.148:0/1557990630 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v158) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb000540c0 con 0x7ffb0815a120 2023-10-10T20:28:35.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 msgr2=0x7ffaec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 0x7ffaec0693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb08160850 tx=0x7ffafc008040 comp rx=0 tx=0).stop 2023-10-10T20:28:35.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 msgr2=0x7ffb0815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb08072050 tx=0x7ffb00019000 comp rx=0 tx=0).stop 2023-10-10T20:28:35.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 shutdown_connections 2023-10-10T20:28:35.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb0815a120 0x7ffb0815c6d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb0806daa0 0x7ffb0815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 --2- 172.21.15.148:0/1557990630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb08159d50 0x7ffb0815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 >> 172.21.15.148:0/1557990630 conn(0x7ffb080cd200 msgr2=0x7ffb080cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:35.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 shutdown_connections 2023-10-10T20:28:35.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.612+0000 7ffb1011b700 1 -- 172.21.15.148:0/1557990630 wait complete. 2023-10-10T20:28:35.618 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:28:35.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:28:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.741+0000 7fa5d1244700 1 Processor -- start 2023-10-10T20:28:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.741+0000 7fa5d1244700 1 -- start start 2023-10-10T20:28:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5d1244700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5cc15a620 0x7fa5cc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5d1244700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5cc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5d1244700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc15cbb0 con 0x7fa5cc157870 2023-10-10T20:28:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5d1244700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc15bdb0 con 0x7fa5cc15a620 2023-10-10T20:28:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5d1244700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc15bfa0 con 0x7fa5cc15d580 2023-10-10T20:28:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5cad9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa5cc157870 0x7fa5cc15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57040/0 (socket says 172.21.15.148:57040) 2023-10-10T20:28:35.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5cad9d700 1 -- 172.21.15.148:0/1326413563 learned_addr learned my addr 172.21.15.148:0/1326413563 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795481056 0 0) 0x7fa5cc15cbb0 con 0x7fa5cc157870 2023-10-10T20:28:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8003570 con 0x7fa5cc157870 2023-10-10T20:28:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5c259c700 1 --2- 172.21.15.148:0/1326413563 >> v2:172.21.15.161:3300/0 conn(0x7fa5cc15a620 0x7fa5cc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.742+0000 7fa5cb59e700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5cc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.754 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407634099 0 0) 0x7fa5b8003570 con 0x7fa5cc157870 2023-10-10T20:28:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 msgr2=0x7fa5cc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5cc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 >> v2:172.21.15.161:3300/0 conn(0x7fa5cc15a620 msgr2=0x7fa5cc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 --2- 172.21.15.148:0/1326413563 >> v2:172.21.15.161:3300/0 conn(0x7fa5cc15a620 0x7fa5cc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.755 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc15c280 con 0x7fa5cc157870 2023-10-10T20:28:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa5bc001470 con 0x7fa5cc157870 2023-10-10T20:28:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 >> v1:172.21.15.148:6789/0 conn(0x7fa5cc157870 legacy=0x7fa5cc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8003a60 0x7fa5b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc15bfa0 con 0x7fa5b8003a60 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800ba60 con 0x7fa5b8004810 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800bba0 con 0x7fa5cc15d580 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8003a60 0x7fa5b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5c259c700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cb59e700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 -- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 msgr2=0x7fa5b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc15d580 0x7fa5b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 -- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8004810 msgr2=0x7fa5b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8004810 0x7fa5b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 -- 172.21.15.148:0/1326413563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8003570 con 0x7fa5b8003a60 2023-10-10T20:28:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8003a60 0x7fa5b8003e70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c3f0 tx=0x7fa5bc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9813263dc22084f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cc80 con 0x7fa5b8003a60 2023-10-10T20:28:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00ce00 con 0x7fa5b8003a60 2023-10-10T20:28:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.743+0000 7fa5ca59c700 1 -- 172.21.15.148:0/1326413563 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc005b70 con 0x7fa5b8003a60 2023-10-10T20:28:35.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 -- 172.21.15.148:0/1326413563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8003a60 msgr2=0x7fa5b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1326413563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8003a60 0x7fa5b8003e70 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c3f0 tx=0x7fa5bc001e90 comp rx=0 tx=0).stop 2023-10-10T20:28:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5cb59e700 1 -- 172.21.15.148:0/1326413563 reap_dead start 2023-10-10T20:28:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 -- 172.21.15.148:0/1326413563 shutdown_connections 2023-10-10T20:28:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 -- 172.21.15.148:0/1326413563 >> 172.21.15.148:0/1326413563 conn(0x7fa5cc068ce0 msgr2=0x7fa5cc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:35.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 -- 172.21.15.148:0/1326413563 shutdown_connections 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.744+0000 7fa5d1244700 1 -- 172.21.15.148:0/1326413563 wait complete. 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.745+0000 7fa5d1244700 1 Processor -- start 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.745+0000 7fa5d1244700 1 -- start start 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8003a60 0x7fa5cc14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc14c060 0x7fa5cc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc16eba0 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc14b080 con 0x7fa5b8003a60 2023-10-10T20:28:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5d1244700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc16df90 con 0x7fa5cc14c060 2023-10-10T20:28:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36058/0 (socket says 172.21.15.148:36058) 2023-10-10T20:28:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 -- 172.21.15.148:0/1742914522 learned_addr learned my addr 172.21.15.148:0/1742914522 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5cb59e700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc14c060 0x7fa5cc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 -- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc14c060 msgr2=0x7fa5cc1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8003a60 0x7fa5cc14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc14c060 0x7fa5cc1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 -- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8003a60 msgr2=0x7fa5cc14ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8003a60 0x7fa5cc14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 -- 172.21.15.148:0/1742914522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc16e270 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.746+0000 7fa5c259c700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011ac0 tx=0x7fa5b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:35.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.747+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400fa10 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.747+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b400fb70 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.747+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc16e550 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.747+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4014440 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.747+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc0b0830 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.748+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc16e9f0 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.748+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5b4014870 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.748+0000 7fa5c37fe700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 0x7fa5ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.749+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 0x7fa5ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.749+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa5b4018030 con 0x7fa5cc14bbf0 2023-10-10T20:28:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.749+0000 7fa5cad9d700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 0x7fa5ac0693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c390 tx=0x7fa5bc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:35.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:35.753+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b40521f0 con 0x7fa5cc14bbf0 2023-10-10T20:28:36.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.011+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa5cc04c3c0 con 0x7fa5cc14bbf0 2023-10-10T20:28:36.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.012+0000 7fa5c37fe700 1 -- 172.21.15.148:0/1742914522 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa5b4022090 con 0x7fa5cc14bbf0 2023-10-10T20:28:36.015 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:36.015 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:28:36.015 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:28:36.015 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.016 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:28:36.016 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:28:36.016 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.016 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.016 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:28:36.017 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:28:36.017 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.017 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.017 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.017 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:28:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:28:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:28:36.019 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:28:36.019 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.019 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.019 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:28:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:28:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.020 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.021 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:28:36.022 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:28:36.022 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.022 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.022 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:28:36.022 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.023 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:28:36.024 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:28:36.024 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.024 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.024 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.024 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.025 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.025 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.025 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.025 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:28:36.025 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:28:36.026 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.026 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.026 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.026 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.026 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.027 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.027 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.027 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:28:36.027 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:28:36.027 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:28:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:28:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.029 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.030 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.030 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.030 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.030 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.030 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.031 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.032 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.032 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.032 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:28:36.032 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:28:36.032 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.033 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.033 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.033 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.033 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.033 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.034 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.034 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:28:36.034 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:28:36.034 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.034 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.035 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:28:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:28:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:28:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:28:36.036 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:28:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.039 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.039 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:28:36.039 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:28:36.039 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.039 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.040 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:28:36.040 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:28:36.040 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:28:36.040 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.041 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.041 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.041 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:28:36.041 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:28:36.041 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:28:36.042 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.043 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.043 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.043 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:28:36.043 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:28:36.043 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:28:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:28:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:28:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:28:36.044 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.045 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:28:36.046 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:28:36.046 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:28:36.046 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.046 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.046 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.047 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.047 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:28:36.047 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:28:36.047 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.047 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:28:36.048 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:28:36.048 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.048 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.048 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:28:36.048 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:36.049 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:28:36.049 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:28:36.049 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:28:36.049 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:28:36.049 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:28:36.050 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:28:36.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.050 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:36.050 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:28:36.050 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:28:36.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 msgr2=0x7fa5ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 0x7fa5ac0693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c390 tx=0x7fa5bc015000 comp rx=0 tx=0).stop 2023-10-10T20:28:36.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 msgr2=0x7fa5cc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011ac0 tx=0x7fa5b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 shutdown_connections 2023-10-10T20:28:36.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5ac066f30 0x7fa5ac0693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5cc14c060 0x7fa5cc1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5b8003a60 0x7fa5cc14ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 --2- 172.21.15.148:0/1742914522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5cc14bbf0 0x7fa5cc154b70 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 >> 172.21.15.148:0/1742914522 conn(0x7fa5cc068ce0 msgr2=0x7fa5cc14c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 shutdown_connections 2023-10-10T20:28:36.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.014+0000 7fa5d1244700 1 -- 172.21.15.148:0/1742914522 wait complete. 2023-10-10T20:28:36.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:36.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.169+0000 7f77c3f0d700 1 Processor -- start 2023-10-10T20:28:36.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.169+0000 7f77c3f0d700 1 -- start start 2023-10-10T20:28:36.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c3f0d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77bc152010 0x7f77bc15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c3f0d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc06c010 con 0x7f77bc158e50 2023-10-10T20:28:36.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c3f0d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc06bd10 con 0x7f77bc1523e0 2023-10-10T20:28:36.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c3f0d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc15c060 con 0x7f77bc152010 2023-10-10T20:28:36.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c1ca9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77bc152010 0x7f77bc15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c24aa700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f77bc158e50 0x7f77bc0cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57056/0 (socket says 172.21.15.148:57056) 2023-10-10T20:28:36.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.170+0000 7f77c24aa700 1 -- 172.21.15.148:0/3947110463 learned_addr learned my addr 172.21.15.148:0/3947110463 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:36.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c1ca9700 1 -- 172.21.15.148:0/3947110463 >> v1:172.21.15.161:6789/0 conn(0x7f77bc1523e0 legacy=0x7f77bc06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:36.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c1ca9700 1 -- 172.21.15.148:0/3947110463 >> v1:172.21.15.148:6789/0 conn(0x7f77bc158e50 legacy=0x7f77bc0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c1ca9700 1 -- 172.21.15.148:0/3947110463 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc15c340 con 0x7f77bc152010 2023-10-10T20:28:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c0ca7700 1 -- 172.21.15.148:0/3947110463 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351820910 0 0) 0x7f77bc06c010 con 0x7f77bc158e50 2023-10-10T20:28:36.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c1ca9700 1 --2- 172.21.15.148:0/3947110463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc152010 0x7f77bc15b810 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098e0 tx=0x7f77b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f05800f684558e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c0ca7700 1 -- 172.21.15.148:0/3947110463 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8009e80 con 0x7f77bc152010 2023-10-10T20:28:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c0ca7700 1 -- 172.21.15.148:0/3947110463 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b800b820 con 0x7f77bc152010 2023-10-10T20:28:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c0ca7700 1 -- 172.21.15.148:0/3947110463 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800bba0 con 0x7f77bc152010 2023-10-10T20:28:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c3f0d700 1 -- 172.21.15.148:0/3947110463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc152010 msgr2=0x7f77bc15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.171+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/3947110463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc152010 0x7f77bc15b810 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098e0 tx=0x7f77b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.172+0000 7f77c3f0d700 1 -- 172.21.15.148:0/3947110463 shutdown_connections 2023-10-10T20:28:36.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.172+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/3947110463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc152010 0x7f77bc15b810 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.172+0000 7f77c3f0d700 1 -- 172.21.15.148:0/3947110463 >> 172.21.15.148:0/3947110463 conn(0x7f77bc069870 msgr2=0x7f77bc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.172+0000 7f77c3f0d700 1 -- 172.21.15.148:0/3947110463 shutdown_connections 2023-10-10T20:28:36.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.172+0000 7f77c3f0d700 1 -- 172.21.15.148:0/3947110463 wait complete. 2023-10-10T20:28:36.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 Processor -- start 2023-10-10T20:28:36.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 -- start start 2023-10-10T20:28:36.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77bc152010 0x7f77bc165f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 0x7f77bc168be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc16a4a0 con 0x7f77bc158e50 2023-10-10T20:28:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc169890 con 0x7f77bc1523e0 2023-10-10T20:28:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.173+0000 7f77c3f0d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc169b70 con 0x7f77bc152010 2023-10-10T20:28:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36076/0 (socket says 172.21.15.148:36076) 2023-10-10T20:28:36.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 -- 172.21.15.148:0/192541676 learned_addr learned my addr 172.21.15.148:0/192541676 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:36.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c14a8700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 0x7f77bc168be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 -- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77bc152010 msgr2=0x7f77bc165f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c1ca9700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77bc152010 0x7f77bc165f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.199 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77bc152010 0x7f77bc165f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 -- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 msgr2=0x7f77bc168be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 0x7f77bc168be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 -- 172.21.15.148:0/192541676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009420 con 0x7f77bc158e50 2023-10-10T20:28:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c14a8700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 0x7f77bc168be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:36.200 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c24aa700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f77b400cbf0 tx=0x7f77b40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b400a9c0 con 0x7f77bc158e50 2023-10-10T20:28:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b400ab20 con 0x7f77bc158e50 2023-10-10T20:28:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40070f0 con 0x7f77bc158e50 2023-10-10T20:28:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc169e50 con 0x7f77bc158e50 2023-10-10T20:28:36.201 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.174+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc0c8720 con 0x7f77bc158e50 2023-10-10T20:28:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.175+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc16a650 con 0x7f77bc158e50 2023-10-10T20:28:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.175+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77b4007520 con 0x7f77bc158e50 2023-10-10T20:28:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.176+0000 7f77b1ffb700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 0x7f77a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.176+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f77b40845e0 con 0x7f77bc158e50 2023-10-10T20:28:36.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.176+0000 7f77c1ca9700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 0x7f77a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.176+0000 7f77c1ca9700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 0x7f77a00693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f77b80091e0 tx=0x7f77b8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.180+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77bc16a650 con 0x7f77bc158e50 2023-10-10T20:28:36.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.364+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77bc04c3c0 con 0x7f77bc158e50 2023-10-10T20:28:36.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.364+0000 7f77b1ffb700 1 -- 172.21.15.148:0/192541676 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+12193 (secure 0 0 0) 0x7f77b4051c20 con 0x7f77bc158e50 2023-10-10T20:28:36.367 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:36.368 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":158,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:28:35.606460+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:36.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 msgr2=0x7f77a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.369 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 0x7f77a00693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f77b80091e0 tx=0x7f77b8016000 comp rx=0 tx=0).stop 2023-10-10T20:28:36.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 msgr2=0x7f77bc156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f77b400cbf0 tx=0x7f77b40019b0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 shutdown_connections 2023-10-10T20:28:36.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f77a0066f30 0x7f77a00693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77bc158e50 0x7f77bc156f30 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77bc152010 0x7f77bc165f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 --2- 172.21.15.148:0/192541676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77bc1523e0 0x7f77bc168be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 >> 172.21.15.148:0/192541676 conn(0x7f77bc069870 msgr2=0x7f77bc158280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 shutdown_connections 2023-10-10T20:28:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.367+0000 7f77c3f0d700 1 -- 172.21.15.148:0/192541676 wait complete. 2023-10-10T20:28:36.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:28:36.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:28:36.500 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.498+0000 7fd4d5ee3700 1 Processor -- start 2023-10-10T20:28:36.501 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.498+0000 7fd4d5ee3700 1 -- start start 2023-10-10T20:28:36.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00cce30 0x7fd4d015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4d00b6580 0x7fd4d006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4d006a830 0x7fd4d015d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00b4d70 con 0x7fd4d00b6580 2023-10-10T20:28:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d006ac70 con 0x7fd4d006a830 2023-10-10T20:28:36.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d5ee3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00cd8d0 con 0x7fd4d00cce30 2023-10-10T20:28:36.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4d00b6580 0x7fd4d006a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4d00b6580 0x7fd4d006a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36082/0 (socket says 172.21.15.148:36082) 2023-10-10T20:28:36.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 -- 172.21.15.148:0/1098288029 learned_addr learned my addr 172.21.15.148:0/1098288029 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:36.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d4ee1700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00cce30 0x7fd4d015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4d56e2700 1 --2- 172.21.15.148:0/1098288029 >> v2:172.21.15.161:3300/0 conn(0x7fd4d006a830 0x7fd4d015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 -- 172.21.15.148:0/1098288029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00cce30 msgr2=0x7fd4d015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00cce30 0x7fd4d015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 -- 172.21.15.148:0/1098288029 >> v2:172.21.15.161:3300/0 conn(0x7fd4d006a830 msgr2=0x7fd4d015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 --2- 172.21.15.148:0/1098288029 >> v2:172.21.15.161:3300/0 conn(0x7fd4d006a830 0x7fd4d015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 -- 172.21.15.148:0/1098288029 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00b4440 con 0x7fd4d00b6580 2023-10-10T20:28:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.499+0000 7fd4cffff700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00b6580 0x7fd4d006a2f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4d015a280 tx=0x7fd4c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735543697826378b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4cf7fe700 1 -- 172.21.15.148:0/1098288029 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400b860 con 0x7fd4d00b6580 2023-10-10T20:28:36.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4cf7fe700 1 -- 172.21.15.148:0/1098288029 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c400b9c0 con 0x7fd4d00b6580 2023-10-10T20:28:36.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4cf7fe700 1 -- 172.21.15.148:0/1098288029 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400bd60 con 0x7fd4d00b6580 2023-10-10T20:28:36.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/1098288029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00b6580 msgr2=0x7fd4d006a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00b6580 0x7fd4d006a2f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4d015a280 tx=0x7fd4c4009250 comp rx=0 tx=0).stop 2023-10-10T20:28:36.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/1098288029 shutdown_connections 2023-10-10T20:28:36.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/1098288029 >> v2:172.21.15.161:3300/0 conn(0x7fd4d006a830 0x7fd4d015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00cce30 0x7fd4d015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/1098288029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00b6580 0x7fd4d006a2f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/1098288029 >> 172.21.15.148:0/1098288029 conn(0x7fd4d0068cb0 msgr2=0x7fd4d00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/1098288029 shutdown_connections 2023-10-10T20:28:36.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.500+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/1098288029 wait complete. 2023-10-10T20:28:36.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.501+0000 7fd4d5ee3700 1 Processor -- start 2023-10-10T20:28:36.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.501+0000 7fd4d5ee3700 1 -- start start 2023-10-10T20:28:36.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d006a830 0x7fd4d01424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00b6580 0x7fd4d00b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0169f60 con 0x7fd4d00cce30 2023-10-10T20:28:36.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b2960 con 0x7fd4d006a830 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d5ee3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00b2c40 con 0x7fd4d00b6580 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d56e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d56e2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36084/0 (socket says 172.21.15.148:36084) 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.502+0000 7fd4d56e2700 1 -- 172.21.15.148:0/938490062 learned_addr learned my addr 172.21.15.148:0/938490062 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 -- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00b6580 msgr2=0x7fd4d00b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00b6580 0x7fd4d00b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 -- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d006a830 msgr2=0x7fd4d01424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d4ee1700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d006a830 0x7fd4d01424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d006a830 0x7fd4d01424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 -- 172.21.15.148:0/938490062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40094f0 con 0x7fd4d00cce30 2023-10-10T20:28:36.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d56e2700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d890 tx=0x7fd4c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80125f0 con 0x7fd4d00cce30 2023-10-10T20:28:36.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8012770 con 0x7fd4d00cce30 2023-10-10T20:28:36.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8012b30 con 0x7fd4d00cce30 2023-10-10T20:28:36.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d00b2f20 con 0x7fd4d00cce30 2023-10-10T20:28:36.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.503+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01429f0 con 0x7fd4d00cce30 2023-10-10T20:28:36.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.504+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d0142be0 con 0x7fd4d00cce30 2023-10-10T20:28:36.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.504+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4c801e460 con 0x7fd4d00cce30 2023-10-10T20:28:36.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.504+0000 7fd4ccff9700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 0x7fd4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.505+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd4c8084850 con 0x7fd4d00cce30 2023-10-10T20:28:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.505+0000 7fd4d4ee1700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 0x7fd4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.505+0000 7fd4d4ee1700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 0x7fd4bc0693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.509+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4d0142be0 con 0x7fd4d00cce30 2023-10-10T20:28:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.692+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4d016a1b0 con 0x7fd4d00cce30 2023-10-10T20:28:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.693+0000 7fd4ccff9700 1 -- 172.21.15.148:0/938490062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4d016a1b0 con 0x7fd4d00cce30 2023-10-10T20:28:36.696 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:28:36.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.694+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 msgr2=0x7fd4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 0x7fd4bc0693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c000b040 comp rx=0 tx=0).stop 2023-10-10T20:28:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 msgr2=0x7fd4d00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800d890 tx=0x7fd4c8008040 comp rx=0 tx=0).stop 2023-10-10T20:28:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 shutdown_connections 2023-10-10T20:28:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd4bc066f30 0x7fd4bc0693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4d00cce30 0x7fd4d00cf2d0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4d006a830 0x7fd4d01424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 --2- 172.21.15.148:0/938490062 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4d00b6580 0x7fd4d00b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 >> 172.21.15.148:0/938490062 conn(0x7fd4d0068cb0 msgr2=0x7fd4d015c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 shutdown_connections 2023-10-10T20:28:36.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.695+0000 7fd4d5ee3700 1 -- 172.21.15.148:0/938490062 wait complete. 2023-10-10T20:28:36.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:28:36.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.821+0000 7f3dd0d72700 1 Processor -- start 2023-10-10T20:28:36.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.821+0000 7f3dd0d72700 1 -- start start 2023-10-10T20:28:36.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dd0d72700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3dcc157870 0x7f3dcc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dd0d72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc15a620 0x7f3dcc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dd0d72700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc15cbb0 con 0x7f3dcc15d580 2023-10-10T20:28:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dd0d72700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc15bfa0 con 0x7f3dcc157870 2023-10-10T20:28:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dd0d72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15c280 con 0x7f3dcc15a620 2023-10-10T20:28:36.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dca59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3dcc157870 0x7f3dcc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dcad9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3dcc15d580 0x7f3dcc158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57072/0 (socket says 172.21.15.148:57072) 2023-10-10T20:28:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dcad9d700 1 -- 172.21.15.148:0/1473022981 learned_addr learned my addr 172.21.15.148:0/1473022981 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dc1d9b700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc15a620 0x7f3dcc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dca59c700 1 -- 172.21.15.148:0/1473022981 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc15a620 msgr2=0x7f3dcc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dca59c700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc15a620 0x7f3dcc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dca59c700 1 -- 172.21.15.148:0/1473022981 >> v1:172.21.15.148:6789/0 conn(0x7f3dcc15d580 legacy=0x7f3dcc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dca59c700 1 -- 172.21.15.148:0/1473022981 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc15c560 con 0x7f3dcc157870 2023-10-10T20:28:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.822+0000 7f3dc9d9b700 1 -- 172.21.15.148:0/1473022981 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952337140 0 0) 0x7f3dcc15cbb0 con 0x7f3dcc15d580 2023-10-10T20:28:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dca59c700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc157870 0x7f3dcc15b6a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098e0 tx=0x7f3dbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77292a9182b08508 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dc9d9b700 1 -- 172.21.15.148:0/1473022981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00b640 con 0x7f3dcc157870 2023-10-10T20:28:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dc9d9b700 1 -- 172.21.15.148:0/1473022981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc00b7a0 con 0x7f3dcc157870 2023-10-10T20:28:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dc9d9b700 1 -- 172.21.15.148:0/1473022981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00bb40 con 0x7f3dcc157870 2023-10-10T20:28:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1473022981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc157870 msgr2=0x7f3dcc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc157870 0x7f3dcc15b6a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098e0 tx=0x7f3dbc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1473022981 shutdown_connections 2023-10-10T20:28:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc157870 0x7f3dcc15b6a0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1473022981 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc15a620 0x7f3dcc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1473022981 >> 172.21.15.148:0/1473022981 conn(0x7f3dcc068ce0 msgr2=0x7f3dcc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1473022981 shutdown_connections 2023-10-10T20:28:36.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.823+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1473022981 wait complete. 2023-10-10T20:28:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 Processor -- start 2023-10-10T20:28:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 -- start start 2023-10-10T20:28:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc157870 0x7f3dcc0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15a620 0x7f3dcc16e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0b62b0 con 0x7f3dcc15d580 2023-10-10T20:28:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0b56a0 con 0x7f3dcc15a620 2023-10-10T20:28:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dd0d72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0b5980 con 0x7f3dcc157870 2023-10-10T20:28:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dcad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dcad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36090/0 (socket says 172.21.15.148:36090) 2023-10-10T20:28:36.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.825+0000 7f3dcad9d700 1 -- 172.21.15.148:0/1868216398 learned_addr learned my addr 172.21.15.148:0/1868216398 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dca59c700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc157870 0x7f3dcc0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dc1d9b700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15a620 0x7f3dcc16e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 -- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc157870 msgr2=0x7f3dcc0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc157870 0x7f3dcc0b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 -- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15a620 msgr2=0x7f3dcc16e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15a620 0x7f3dcc16e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 -- 172.21.15.148:0/1868216398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009420 con 0x7f3dcc15d580 2023-10-10T20:28:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dcad9d700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400cbc0 tx=0x7f3dc4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400a7e0 con 0x7f3dcc15d580 2023-10-10T20:28:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400a940 con 0x7f3dcc15d580 2023-10-10T20:28:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400ad00 con 0x7f3dcc15d580 2023-10-10T20:28:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc0b5c60 con 0x7f3dcc15d580 2023-10-10T20:28:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.826+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc14ade0 con 0x7f3dcc15d580 2023-10-10T20:28:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.827+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc16ec90 con 0x7f3dcc15d580 2023-10-10T20:28:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.827+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3dc4007300 con 0x7f3dcc15d580 2023-10-10T20:28:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.828+0000 7f3dc2ffd700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 0x7f3dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.828+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3dc40844a0 con 0x7f3dcc15d580 2023-10-10T20:28:36.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.828+0000 7f3dca59c700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 0x7f3dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:36.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.828+0000 7f3dca59c700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 0x7f3dac0693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0093f0 tx=0x7f3dbc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:36.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:36.832+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dcc16ec90 con 0x7f3dcc15d580 2023-10-10T20:28:37.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.015+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3dcc16ec90 con 0x7f3dcc15d580 2023-10-10T20:28:37.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.016+0000 7f3dc2ffd700 1 -- 172.21.15.148:0/1868216398 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3dcc16ec90 con 0x7f3dcc15d580 2023-10-10T20:28:37.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:37.019 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:28:37.019 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:28:37.019 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:28:37.020 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:28:37.020 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:28:37.020 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:28:37.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 msgr2=0x7f3dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 0x7f3dac0693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0093f0 tx=0x7f3dbc017000 comp rx=0 tx=0).stop 2023-10-10T20:28:37.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 msgr2=0x7f3dcc14a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400cbc0 tx=0x7f3dc4001980 comp rx=0 tx=0).stop 2023-10-10T20:28:37.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 shutdown_connections 2023-10-10T20:28:37.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3dac066f30 0x7f3dac0693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dcc15d580 0x7f3dcc14a8a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3dcc157870 0x7f3dcc0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 --2- 172.21.15.148:0/1868216398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15a620 0x7f3dcc16e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 >> 172.21.15.148:0/1868216398 conn(0x7f3dcc068ce0 msgr2=0x7f3dcc1593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:37.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 shutdown_connections 2023-10-10T20:28:37.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.018+0000 7f3dd0d72700 1 -- 172.21.15.148:0/1868216398 wait complete. 2023-10-10T20:28:37.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:28:37.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:28:37.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.149+0000 7fcb18f18700 1 Processor -- start 2023-10-10T20:28:37.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.149+0000 7fcb18f18700 1 -- start start 2023-10-10T20:28:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb18f18700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 0x7fcb1415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb18f18700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140b17b0 con 0x7fcb140b46f0 2023-10-10T20:28:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb18f18700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140b1a60 con 0x7fcb140b6a10 2023-10-10T20:28:37.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb18f18700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140b1d40 con 0x7fcb14159ef0 2023-10-10T20:28:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb12d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb140b46f0 0x7fcb140cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57078/0 (socket says 172.21.15.148:57078) 2023-10-10T20:28:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb12d9d700 1 -- 172.21.15.148:0/996676870 learned_addr learned my addr 172.21.15.148:0/996676870 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb1259c700 1 --2- 172.21.15.148:0/996676870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 0x7fcb1415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875139379 0 0) 0x7fcb140b17b0 con 0x7fcb140b46f0 2023-10-10T20:28:37.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003570 con 0x7fcb140b46f0 2023-10-10T20:28:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647978702 0 0) 0x7fcb140b1a60 con 0x7fcb140b6a10 2023-10-10T20:28:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb140b17b0 con 0x7fcb140b6a10 2023-10-10T20:28:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb1259c700 1 -- 172.21.15.148:0/996676870 >> v1:172.21.15.161:6789/0 conn(0x7fcb140b6a10 legacy=0x7fcb140b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb1259c700 1 -- 172.21.15.148:0/996676870 >> v1:172.21.15.148:6789/0 conn(0x7fcb140b46f0 legacy=0x7fcb140cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb1259c700 1 -- 172.21.15.148:0/996676870 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140b2020 con 0x7fcb14159ef0 2023-10-10T20:28:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.150+0000 7fcb1259c700 1 --2- 172.21.15.148:0/996676870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 0x7fcb1415a2c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009900 tx=0x7fcb04009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6b0545f8a412c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005c10 con 0x7fcb14159ef0 2023-10-10T20:28:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04005d70 con 0x7fcb14159ef0 2023-10-10T20:28:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb11d9b700 1 -- 172.21.15.148:0/996676870 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04006260 con 0x7fcb14159ef0 2023-10-10T20:28:37.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 -- 172.21.15.148:0/996676870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 msgr2=0x7fcb1415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 --2- 172.21.15.148:0/996676870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 0x7fcb1415a2c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009900 tx=0x7fcb04009120 comp rx=0 tx=0).stop 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 -- 172.21.15.148:0/996676870 shutdown_connections 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 --2- 172.21.15.148:0/996676870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb14159ef0 0x7fcb1415a2c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 -- 172.21.15.148:0/996676870 >> 172.21.15.148:0/996676870 conn(0x7fcb1409a0a0 msgr2=0x7fcb14067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 -- 172.21.15.148:0/996676870 shutdown_connections 2023-10-10T20:28:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.151+0000 7fcb18f18700 1 -- 172.21.15.148:0/996676870 wait complete. 2023-10-10T20:28:37.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.152+0000 7fcb18f18700 1 Processor -- start 2023-10-10T20:28:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.152+0000 7fcb18f18700 1 -- start start 2023-10-10T20:28:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb140b6a10 0x7fcb141429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb14159ef0 0x7fcb140af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14143f00 con 0x7fcb140b46f0 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141432f0 con 0x7fcb14159ef0 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb18f18700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141435d0 con 0x7fcb140b6a10 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36106/0 (socket says 172.21.15.148:36106) 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 -- 172.21.15.148:0/131715216 learned_addr learned my addr 172.21.15.148:0/131715216 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:37.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb0bfff700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb140b6a10 0x7fcb141429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb12d9d700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb14159ef0 0x7fcb140af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 -- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb140b6a10 msgr2=0x7fcb141429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb140b6a10 0x7fcb141429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 -- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb14159ef0 msgr2=0x7fcb140af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb14159ef0 0x7fcb140af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 -- 172.21.15.148:0/131715216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040093c0 con 0x7fcb140b46f0 2023-10-10T20:28:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.153+0000 7fcb1259c700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009160 tx=0x7fcb04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.154+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0401d780 con 0x7fcb140b46f0 2023-10-10T20:28:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.154+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0401d8e0 con 0x7fcb140b46f0 2023-10-10T20:28:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.154+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb141438b0 con 0x7fcb140b46f0 2023-10-10T20:28:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.154+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0401dca0 con 0x7fcb140b46f0 2023-10-10T20:28:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.154+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb14145a20 con 0x7fcb140b46f0 2023-10-10T20:28:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.155+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1404c3c0 con 0x7fcb140b46f0 2023-10-10T20:28:37.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.155+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb0401b690 con 0x7fcb140b46f0 2023-10-10T20:28:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.155+0000 7fcb0a7fc700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.156+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcb04085dd0 con 0x7fcb140b46f0 2023-10-10T20:28:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.156+0000 7fcb0bfff700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.167 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.156+0000 7fcb0bfff700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb140abf10 tx=0x7fcafc007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.160+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb04053390 con 0x7fcb140b46f0 2023-10-10T20:28:37.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.341+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb140ab430 con 0x7fcb140b46f0 2023-10-10T20:28:37.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.615+0000 7fcb0a7fc700 1 -- 172.21.15.148:0/131715216 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v159) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcb0402a0b0 con 0x7fcb140b46f0 2023-10-10T20:28:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 msgr2=0x7fcaf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb140abf10 tx=0x7fcafc007d50 comp rx=0 tx=0).stop 2023-10-10T20:28:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 msgr2=0x7fcb141454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009160 tx=0x7fcb04016000 comp rx=0 tx=0).stop 2023-10-10T20:28:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 shutdown_connections 2023-10-10T20:28:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb14159ef0 0x7fcb140af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb140b46f0 0x7fcb141454e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.619+0000 7fcb18f18700 1 --2- 172.21.15.148:0/131715216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb140b6a10 0x7fcb141429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.620+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 >> 172.21.15.148:0/131715216 conn(0x7fcb1409a0a0 msgr2=0x7fcb140b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:37.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.620+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 shutdown_connections 2023-10-10T20:28:37.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.620+0000 7fcb18f18700 1 -- 172.21.15.148:0/131715216 wait complete. 2023-10-10T20:28:37.624 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:28:37.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:28:37.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.754+0000 7f8186b69700 1 Processor -- start 2023-10-10T20:28:37.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.754+0000 7f8186b69700 1 -- start start 2023-10-10T20:28:37.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8186b69700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81800b4860 0x7f81800b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8186b69700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f81800b4c30 0x7f8180159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8186b69700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800b14f0 con 0x7f8180159fd0 2023-10-10T20:28:37.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8186b69700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800b17d0 con 0x7f81800b4860 2023-10-10T20:28:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8186b69700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800b1ab0 con 0x7f81800b4c30 2023-10-10T20:28:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81800b4860 0x7f81800b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f81800b4860 0x7f81800b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36112/0 (socket says 172.21.15.148:36112) 2023-10-10T20:28:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 -- 172.21.15.148:0/3544518710 learned_addr learned my addr 172.21.15.148:0/3544518710 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f817ffff700 1 --2- 172.21.15.148:0/3544518710 >> v2:172.21.15.161:3300/0 conn(0x7f81800b4c30 0x7f8180159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 -- 172.21.15.148:0/3544518710 >> v2:172.21.15.161:3300/0 conn(0x7f81800b4c30 msgr2=0x7f8180159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 --2- 172.21.15.148:0/3544518710 >> v2:172.21.15.161:3300/0 conn(0x7f81800b4c30 0x7f8180159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 -- 172.21.15.148:0/3544518710 >> v1:172.21.15.161:6789/0 conn(0x7f8180159fd0 legacy=0x7f81800cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 -- 172.21.15.148:0/3544518710 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800b1d90 con 0x7f81800b4860 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.755+0000 7f8184905700 1 --2- 172.21.15.148:0/3544518710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81800b6700 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f81800899c0 tx=0x7f8174009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f055a2a14eb1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f817f7fe700 1 -- 172.21.15.148:0/3544518710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174005c10 con 0x7f81800b4860 2023-10-10T20:28:37.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f817f7fe700 1 -- 172.21.15.148:0/3544518710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174005d70 con 0x7f81800b4860 2023-10-10T20:28:37.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f817f7fe700 1 -- 172.21.15.148:0/3544518710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174006260 con 0x7f81800b4860 2023-10-10T20:28:37.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 -- 172.21.15.148:0/3544518710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 msgr2=0x7f81800b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 --2- 172.21.15.148:0/3544518710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81800b6700 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f81800899c0 tx=0x7f8174009220 comp rx=0 tx=0).stop 2023-10-10T20:28:37.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 -- 172.21.15.148:0/3544518710 shutdown_connections 2023-10-10T20:28:37.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 --2- 172.21.15.148:0/3544518710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81800b6700 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 --2- 172.21.15.148:0/3544518710 >> v2:172.21.15.161:3300/0 conn(0x7f81800b4c30 0x7f8180159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 -- 172.21.15.148:0/3544518710 >> 172.21.15.148:0/3544518710 conn(0x7f818009a0f0 msgr2=0x7f8180067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:37.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 -- 172.21.15.148:0/3544518710 shutdown_connections 2023-10-10T20:28:37.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.756+0000 7f8186b69700 1 -- 172.21.15.148:0/3544518710 wait complete. 2023-10-10T20:28:37.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.757+0000 7f8186b69700 1 Processor -- start 2023-10-10T20:28:37.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.757+0000 7f8186b69700 1 -- start start 2023-10-10T20:28:37.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 0x7f818006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 0x7f818006a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818016bf70 con 0x7f81800b4860 2023-10-10T20:28:37.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818016b3a0 con 0x7f81800b4c30 2023-10-10T20:28:37.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8186b69700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818009b5c0 con 0x7f8180159fd0 2023-10-10T20:28:37.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8184905700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8184905700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36128/0 (socket says 172.21.15.148:36128) 2023-10-10T20:28:37.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8184905700 1 -- 172.21.15.148:0/3926909392 learned_addr learned my addr 172.21.15.148:0/3926909392 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f817ffff700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 0x7f818006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.758+0000 7f8185106700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 0x7f818006a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 -- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 msgr2=0x7f818006a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 0x7f818006a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 -- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 msgr2=0x7f818006a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:37.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 0x7f818006a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:37.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 -- 172.21.15.148:0/3926909392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009480 con 0x7f81800b4860 2023-10-10T20:28:37.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8185106700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 0x7f818006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:37.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f817ffff700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 0x7f818006a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:37.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8184905700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f817400b0e0 tx=0x7f8174016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400b990 con 0x7f81800b4860 2023-10-10T20:28:37.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174020070 con 0x7f81800b4860 2023-10-10T20:28:37.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400bd20 con 0x7f81800b4860 2023-10-10T20:28:37.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818016b640 con 0x7f81800b4860 2023-10-10T20:28:37.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.759+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818016bc00 con 0x7f81800b4860 2023-10-10T20:28:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.760+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818004c3c0 con 0x7f81800b4860 2023-10-10T20:28:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.760+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8174022030 con 0x7f81800b4860 2023-10-10T20:28:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.761+0000 7f817cff9700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 0x7f81640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.761+0000 7f817ffff700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 0x7f81640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.761+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8174085f50 con 0x7f81800b4860 2023-10-10T20:28:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.761+0000 7f817ffff700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 0x7f81640693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f818016b2b0 tx=0x7f816c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.765+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81740533a0 con 0x7f81800b4860 2023-10-10T20:28:37.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:37.949+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f81800b2070 con 0x7f81800b4860 2023-10-10T20:28:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.618+0000 7f817cff9700 1 -- 172.21.15.148:0/3926909392 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v160) v1 ==== 170+0+0 (secure 0 0 0) 0x7f81740569c0 con 0x7f81800b4860 2023-10-10T20:28:38.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.625+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 msgr2=0x7f81640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.625+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 0x7f81640693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f818016b2b0 tx=0x7f816c008040 comp rx=0 tx=0).stop 2023-10-10T20:28:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.625+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 msgr2=0x7f81801465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.625+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f817400b0e0 tx=0x7f8174016000 comp rx=0 tx=0).stop 2023-10-10T20:28:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 shutdown_connections 2023-10-10T20:28:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8164066f30 0x7f81640693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8180159fd0 0x7f818006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81800b4860 0x7f81801465d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 --2- 172.21.15.148:0/3926909392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81800b4c30 0x7f818006a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.626+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 >> 172.21.15.148:0/3926909392 conn(0x7f818009a0f0 msgr2=0x7f8180158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.628+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 shutdown_connections 2023-10-10T20:28:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.629+0000 7f8186b69700 1 -- 172.21.15.148:0/3926909392 wait complete. 2023-10-10T20:28:38.633 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:28:38.646 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:28:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.761+0000 7fab20a09700 1 Processor -- start 2023-10-10T20:28:38.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.761+0000 7fab20a09700 1 -- start start 2023-10-10T20:28:38.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fab1c0cce30 0x7fab1c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0b6580 0x7fab1c06a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c06a830 0x7fab1c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c06ac90 con 0x7fab1c0b6580 2023-10-10T20:28:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c0b4d70 con 0x7fab1c0cce30 2023-10-10T20:28:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab20a09700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cd8d0 con 0x7fab1c06a830 2023-10-10T20:28:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0b6580 0x7fab1c06a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0b6580 0x7fab1c06a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36142/0 (socket says 172.21.15.148:36142) 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 -- 172.21.15.148:0/1859981311 learned_addr learned my addr 172.21.15.148:0/1859981311 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab1bfff700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c06a830 0x7fab1c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab1b7fe700 1 --2- 172.21.15.148:0/1859981311 >> v2:172.21.15.161:3300/0 conn(0x7fab1c0cce30 0x7fab1c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 -- 172.21.15.148:0/1859981311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c06a830 msgr2=0x7fab1c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c06a830 0x7fab1c15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 -- 172.21.15.148:0/1859981311 >> v2:172.21.15.161:3300/0 conn(0x7fab1c0cce30 msgr2=0x7fab1c15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 --2- 172.21.15.148:0/1859981311 >> v2:172.21.15.161:3300/0 conn(0x7fab1c0cce30 0x7fab1c15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 -- 172.21.15.148:0/1859981311 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0b4440 con 0x7fab1c0b6580 2023-10-10T20:28:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.762+0000 7fab12ffd700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0b6580 0x7fab1c06a2f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0b3770 tx=0x7fab04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a7261d92d2cb0d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab1affd700 1 -- 172.21.15.148:0/1859981311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0400b640 con 0x7fab1c0b6580 2023-10-10T20:28:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab1affd700 1 -- 172.21.15.148:0/1859981311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0400b7a0 con 0x7fab1c0b6580 2023-10-10T20:28:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab1affd700 1 -- 172.21.15.148:0/1859981311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0400bb60 con 0x7fab1c0b6580 2023-10-10T20:28:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 -- 172.21.15.148:0/1859981311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0b6580 msgr2=0x7fab1c06a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0b6580 0x7fab1c06a2f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0b3770 tx=0x7fab04009250 comp rx=0 tx=0).stop 2023-10-10T20:28:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 -- 172.21.15.148:0/1859981311 shutdown_connections 2023-10-10T20:28:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c06a830 0x7fab1c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 --2- 172.21.15.148:0/1859981311 >> v2:172.21.15.161:3300/0 conn(0x7fab1c0cce30 0x7fab1c15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 --2- 172.21.15.148:0/1859981311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0b6580 0x7fab1c06a2f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 -- 172.21.15.148:0/1859981311 >> 172.21.15.148:0/1859981311 conn(0x7fab1c068cb0 msgr2=0x7fab1c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 -- 172.21.15.148:0/1859981311 shutdown_connections 2023-10-10T20:28:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.763+0000 7fab20a09700 1 -- 172.21.15.148:0/1859981311 wait complete. 2023-10-10T20:28:38.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.764+0000 7fab20a09700 1 Processor -- start 2023-10-10T20:28:38.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.764+0000 7fab20a09700 1 -- start start 2023-10-10T20:28:38.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c06a830 0x7fab1c16ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c0b6580 0x7fab1c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c150040 con 0x7fab1c0cce30 2023-10-10T20:28:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0c6ce0 con 0x7fab1c06a830 2023-10-10T20:28:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab20a09700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c16bf40 con 0x7fab1c0b6580 2023-10-10T20:28:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36148/0 (socket says 172.21.15.148:36148) 2023-10-10T20:28:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab1bfff700 1 -- 172.21.15.148:0/1080466314 learned_addr learned my addr 172.21.15.148:0/1080466314 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab12ffd700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c0b6580 0x7fab1c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.765+0000 7fab1b7fe700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c06a830 0x7fab1c16ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 -- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c0b6580 msgr2=0x7fab1c14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:38.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c0b6580 0x7fab1c14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 -- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c06a830 msgr2=0x7fab1c16ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:38.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c06a830 0x7fab1c16ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:38.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 -- 172.21.15.148:0/1080466314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040094f0 con 0x7fab1c0cce30 2023-10-10T20:28:38.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab1bfff700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e90 tx=0x7fab1400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:38.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cb70 con 0x7fab1c0cce30 2023-10-10T20:28:38.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1400f040 con 0x7fab1c0cce30 2023-10-10T20:28:38.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0c63f0 con 0x7fab1c0cce30 2023-10-10T20:28:38.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14014480 con 0x7fab1c0cce30 2023-10-10T20:28:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.766+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c0c6970 con 0x7fab1c0cce30 2023-10-10T20:28:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.767+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0c6e50 con 0x7fab1c0cce30 2023-10-10T20:28:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.767+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab140147f0 con 0x7fab1c0cce30 2023-10-10T20:28:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.768+0000 7fab13fff700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.768+0000 7fab1b7fe700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.768+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fab14085260 con 0x7fab1c0cce30 2023-10-10T20:28:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.768+0000 7fab1b7fe700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 0x7faafc0693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009d10 tx=0x7fab0c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.772+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab14052490 con 0x7fab1c0cce30 2023-10-10T20:28:39.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:38.998+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fab1c0b4720 con 0x7fab1c0cce30 2023-10-10T20:28:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.651+0000 7fab13fff700 1 -- 172.21.15.148:0/1080466314 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7fab14055ab0 con 0x7fab1c0cce30 2023-10-10T20:28:39.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.655+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 msgr2=0x7faafc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.655+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 0x7faafc0693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009d10 tx=0x7fab0c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.655+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 msgr2=0x7fab1c0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.655+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e90 tx=0x7fab1400b040 comp rx=0 tx=0).stop 2023-10-10T20:28:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 shutdown_connections 2023-10-10T20:28:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0cce30 0x7fab1c0c7f40 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab1c06a830 0x7fab1c16ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 --2- 172.21.15.148:0/1080466314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab1c0b6580 0x7fab1c14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 >> 172.21.15.148:0/1080466314 conn(0x7fab1c068cb0 msgr2=0x7fab1c150470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 shutdown_connections 2023-10-10T20:28:39.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.657+0000 7fab20a09700 1 -- 172.21.15.148:0/1080466314 wait complete. 2023-10-10T20:28:39.662 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 20 and data pool 21 2023-10-10T20:28:39.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:39.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.787+0000 7f614f883700 1 Processor -- start 2023-10-10T20:28:39.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.787+0000 7f614f883700 1 -- start start 2023-10-10T20:28:39.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614f883700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f614815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614f883700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f614815d580 0x7f6148158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614f883700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614815cbb0 con 0x7f6148157870 2023-10-10T20:28:39.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614f883700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614815bdb0 con 0x7f614815d580 2023-10-10T20:28:39.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614f883700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614815bfa0 con 0x7f614815a620 2023-10-10T20:28:39.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614d61f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6148157870 0x7f614815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57092/0 (socket says 172.21.15.148:57092) 2023-10-10T20:28:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614d61f700 1 -- 172.21.15.148:0/3761098505 learned_addr learned my addr 172.21.15.148:0/3761098505 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614de20700 1 --2- 172.21.15.148:0/3761098505 >> v2:172.21.15.161:3300/0 conn(0x7f614815d580 0x7f6148158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614ce1e700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f614815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801258814 0 0) 0x7f614815cbb0 con 0x7f6148157870 2023-10-10T20:28:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6134003570 con 0x7f6148157870 2023-10-10T20:28:39.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89261421 0 0) 0x7f6134003570 con 0x7f6148157870 2023-10-10T20:28:39.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 msgr2=0x7f614815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:39.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f614815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 >> v2:172.21.15.161:3300/0 conn(0x7f614815d580 msgr2=0x7f6148158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:39.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 --2- 172.21.15.148:0/3761098505 >> v2:172.21.15.161:3300/0 conn(0x7f614815d580 0x7f6148158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614815c280 con 0x7f6148157870 2023-10-10T20:28:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f6144001470 con 0x7f6148157870 2023-10-10T20:28:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.788+0000 7f614de20700 1 --2- 172.21.15.148:0/3761098505 >> v2:172.21.15.161:3300/0 conn(0x7f614815d580 0x7f6148158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 >> v1:172.21.15.148:6789/0 conn(0x7f6148157870 legacy=0x7f614815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f6134003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f6134004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6134004d50 0x7f61340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614815bfa0 con 0x7f6134003a60 2023-10-10T20:28:39.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613400ba60 con 0x7f6134004d50 2023-10-10T20:28:39.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613400bba0 con 0x7f614815a620 2023-10-10T20:28:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f6134003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614de20700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6134004d50 0x7f61340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614ce1e700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f6134004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 -- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 msgr2=0x7f6134004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815a620 0x7f6134004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 -- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6134004d50 msgr2=0x7f61340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6134004d50 0x7f61340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 -- 172.21.15.148:0/3761098505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134003570 con 0x7f6134003a60 2023-10-10T20:28:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f614d61f700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f6134003e70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f614400c3c0 tx=0x7f6144001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51606ba4dcf5e6c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400cc30 con 0x7f6134003a60 2023-10-10T20:28:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614400cdd0 con 0x7f6134003a60 2023-10-10T20:28:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.789+0000 7f613ffff700 1 -- 172.21.15.148:0/3761098505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144005b70 con 0x7f6134003a60 2023-10-10T20:28:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 -- 172.21.15.148:0/3761098505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 msgr2=0x7f6134003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 --2- 172.21.15.148:0/3761098505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f6134003e70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f614400c3c0 tx=0x7f6144001e90 comp rx=0 tx=0).stop 2023-10-10T20:28:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614de20700 1 -- 172.21.15.148:0/3761098505 reap_dead start 2023-10-10T20:28:39.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 -- 172.21.15.148:0/3761098505 shutdown_connections 2023-10-10T20:28:39.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 -- 172.21.15.148:0/3761098505 >> 172.21.15.148:0/3761098505 conn(0x7f6148068ce0 msgr2=0x7f61480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:39.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 -- 172.21.15.148:0/3761098505 shutdown_connections 2023-10-10T20:28:39.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.790+0000 7f614f883700 1 -- 172.21.15.148:0/3761098505 wait complete. 2023-10-10T20:28:39.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.791+0000 7f614f883700 1 Processor -- start 2023-10-10T20:28:39.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.791+0000 7f614f883700 1 -- start start 2023-10-10T20:28:39.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 0x7f6148159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 0x7f6148159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480b06c0 con 0x7f6134003a60 2023-10-10T20:28:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480b09a0 con 0x7f6148159430 2023-10-10T20:28:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614f883700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614815f170 con 0x7f614815de90 2023-10-10T20:28:39.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36152/0 (socket says 172.21.15.148:36152) 2023-10-10T20:28:39.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 -- 172.21.15.148:0/1359228763 learned_addr learned my addr 172.21.15.148:0/1359228763 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:39.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614ce1e700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 0x7f6148159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614de20700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 0x7f6148159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 -- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 msgr2=0x7f6148159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 0x7f6148159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 -- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 msgr2=0x7f6148159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 0x7f6148159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.808 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 -- 172.21.15.148:0/1359228763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614815f450 con 0x7f6134003a60 2023-10-10T20:28:39.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614de20700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 0x7f6148159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:39.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614ce1e700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 0x7f6148159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:39.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.792+0000 7f614d61f700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6144011000 tx=0x7f6144015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:39.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400cc30 con 0x7f6134003a60 2023-10-10T20:28:39.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61440072e0 con 0x7f6134003a60 2023-10-10T20:28:39.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144007680 con 0x7f6134003a60 2023-10-10T20:28:39.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61480b06c0 con 0x7f6134003a60 2023-10-10T20:28:39.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614814abd0 con 0x7f6134003a60 2023-10-10T20:28:39.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.793+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614804c3c0 con 0x7f6134003a60 2023-10-10T20:28:39.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.794+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6144019030 con 0x7f6134003a60 2023-10-10T20:28:39.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.794+0000 7f613d7fa700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 0x7f612c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:39.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.794+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6144085670 con 0x7f6134003a60 2023-10-10T20:28:39.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.794+0000 7f614ce1e700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 0x7f612c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:39.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.795+0000 7f614ce1e700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 0x7f612c0693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f61380093f0 tx=0x7f6138008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:39.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.799+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61440528e0 con 0x7f6134003a60 2023-10-10T20:28:39.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.983+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f614815c560 con 0x7f6134003a60 2023-10-10T20:28:39.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.984+0000 7f613d7fa700 1 -- 172.21.15.148:0/1359228763 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f6144055f00 con 0x7f6134003a60 2023-10-10T20:28:39.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:39.986 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":161,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:28:38.999940+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:39.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 msgr2=0x7f612c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 0x7f612c0693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f61380093f0 tx=0x7f6138008040 comp rx=0 tx=0).stop 2023-10-10T20:28:39.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 msgr2=0x7f614814a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6144011000 tx=0x7f6144015000 comp rx=0 tx=0).stop 2023-10-10T20:28:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 shutdown_connections 2023-10-10T20:28:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f612c066f30 0x7f612c0693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f614815de90 0x7f6148159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6134003a60 0x7f614814a690 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 --2- 172.21.15.148:0/1359228763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6148159430 0x7f6148159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:39.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 >> 172.21.15.148:0/1359228763 conn(0x7f6148068ce0 msgr2=0x7f61480cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:39.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 shutdown_connections 2023-10-10T20:28:39.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:39.986+0000 7f614f883700 1 -- 172.21.15.148:0/1359228763 wait complete. 2023-10-10T20:28:40.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.113+0000 7f3bcb141700 1 Processor -- start 2023-10-10T20:28:40.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.113+0000 7f3bcb141700 1 -- start start 2023-10-10T20:28:40.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bcb141700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bc415a330 0x7f3bc4156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bcb141700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bcb141700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc4157c30 con 0x7f3bc4156570 2023-10-10T20:28:40.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bcb141700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc41584f0 con 0x7f3bc415a330 2023-10-10T20:28:40.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bcb141700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc41573c0 con 0x7f3bc406da90 2023-10-10T20:28:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bbbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bc415a330 0x7f3bc4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bbbfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3bc415a330 0x7f3bc4156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36162/0 (socket says 172.21.15.148:36162) 2023-10-10T20:28:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.114+0000 7f3bbbfff700 1 -- 172.21.15.148:0/3953511512 learned_addr learned my addr 172.21.15.148:0/3953511512 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bc96de700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbbfff700 1 -- 172.21.15.148:0/3953511512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 msgr2=0x7f3bc4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:40.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbbfff700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbbfff700 1 -- 172.21.15.148:0/3953511512 >> v1:172.21.15.161:6789/0 conn(0x7f3bc4156570 legacy=0x7f3bc406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbbfff700 1 -- 172.21.15.148:0/3953511512 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4157670 con 0x7f3bc415a330 2023-10-10T20:28:40.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbbfff700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc415a330 0x7f3bc4156940 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40099b0 tx=0x7f3bb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246b52c4f261649a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbb7fe700 1 -- 172.21.15.148:0/3953511512 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4009ea0 con 0x7f3bc415a330 2023-10-10T20:28:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbb7fe700 1 -- 172.21.15.148:0/3953511512 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400b860 con 0x7f3bc415a330 2023-10-10T20:28:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.115+0000 7f3bbb7fe700 1 -- 172.21.15.148:0/3953511512 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400bc20 con 0x7f3bc415a330 2023-10-10T20:28:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 -- 172.21.15.148:0/3953511512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc415a330 msgr2=0x7f3bc4156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc415a330 0x7f3bc4156940 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40099b0 tx=0x7f3bb4009250 comp rx=0 tx=0).stop 2023-10-10T20:28:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 -- 172.21.15.148:0/3953511512 shutdown_connections 2023-10-10T20:28:40.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 --2- 172.21.15.148:0/3953511512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc415a330 0x7f3bc4156940 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 -- 172.21.15.148:0/3953511512 >> 172.21.15.148:0/3953511512 conn(0x7f3bc4067660 msgr2=0x7f3bc4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:40.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 -- 172.21.15.148:0/3953511512 shutdown_connections 2023-10-10T20:28:40.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.116+0000 7f3bcb141700 1 -- 172.21.15.148:0/3953511512 wait complete. 2023-10-10T20:28:40.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.117+0000 7f3bcb141700 1 Processor -- start 2023-10-10T20:28:40.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.117+0000 7f3bcb141700 1 -- start start 2023-10-10T20:28:40.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc40c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc415a330 0x7f3bc40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c5fd0 con 0x7f3bc4156570 2023-10-10T20:28:40.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c0e00 con 0x7f3bc415a330 2023-10-10T20:28:40.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c0510 con 0x7f3bc406da90 2023-10-10T20:28:40.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36174/0 (socket says 172.21.15.148:36174) 2023-10-10T20:28:40.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 -- 172.21.15.148:0/2283441280 learned_addr learned my addr 172.21.15.148:0/2283441280 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:40.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bc8edd700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc40c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bc96de700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc415a330 0x7f3bc40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 -- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 msgr2=0x7f3bc40c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc40c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 -- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc415a330 msgr2=0x7f3bc40c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc415a330 0x7f3bc40c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 -- 172.21.15.148:0/2283441280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40094f0 con 0x7f3bc4156570 2023-10-10T20:28:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bbbfff700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b3c0 tx=0x7f3bb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401f070 con 0x7f3bc4156570 2023-10-10T20:28:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400b9d0 con 0x7f3bc4156570 2023-10-10T20:28:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc40c07b0 con 0x7f3bc4156570 2023-10-10T20:28:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4011420 con 0x7f3bc4156570 2023-10-10T20:28:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.118+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc40c5cc0 con 0x7f3bc4156570 2023-10-10T20:28:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.119+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40c0f90 con 0x7f3bc4156570 2023-10-10T20:28:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.120+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bb4011810 con 0x7f3bc4156570 2023-10-10T20:28:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.120+0000 7f3bb8ff9700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 0x7f3bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.120+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3bb4085890 con 0x7f3bc4156570 2023-10-10T20:28:40.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.120+0000 7f3bc8edd700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 0x7f3bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.121+0000 7f3bc8edd700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 0x7f3bb00693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0007fd0 tx=0x7f3bc00016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.125+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bb4052b80 con 0x7f3bc4156570 2023-10-10T20:28:40.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.304+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc404c3c0 con 0x7f3bc4156570 2023-10-10T20:28:40.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.305+0000 7f3bb8ff9700 1 -- 172.21.15.148:0/2283441280 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f3bb4015090 con 0x7f3bc4156570 2023-10-10T20:28:40.308 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:40.308 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":161,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:28:38.999940+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:40.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 msgr2=0x7f3bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 0x7f3bb00693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0007fd0 tx=0x7f3bc00016f0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 msgr2=0x7f3bc41544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b3c0 tx=0x7f3bb4017000 comp rx=0 tx=0).stop 2023-10-10T20:28:40.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 shutdown_connections 2023-10-10T20:28:40.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3bb0066f30 0x7f3bb00693e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc415a330 0x7f3bc40c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bc406da90 0x7f3bc40c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 --2- 172.21.15.148:0/2283441280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4156570 0x7f3bc41544a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 >> 172.21.15.148:0/2283441280 conn(0x7f3bc4067660 msgr2=0x7f3bc40c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:40.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 shutdown_connections 2023-10-10T20:28:40.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.307+0000 7f3bcb141700 1 -- 172.21.15.148:0/2283441280 wait complete. 2023-10-10T20:28:40.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:28:40.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.437+0000 7fee0355b700 1 Processor -- start 2023-10-10T20:28:40.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.437+0000 7fee0355b700 1 -- start start 2023-10-10T20:28:40.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee0355b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fedfc0ce990 0x7fedfc15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee0355b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fedfc06f2a0 0x7fedfc06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee0355b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedfc15b6a0 con 0x7fedfc0ce5c0 2023-10-10T20:28:40.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee0355b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc15c630 con 0x7fedfc0ce990 2023-10-10T20:28:40.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee0355b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc15bc60 con 0x7fedfc06f2a0 2023-10-10T20:28:40.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fedfc0ce990 0x7fedfc15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee02559700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fedfc0ce5c0 0x7fedfc1580c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57108/0 (socket says 172.21.15.148:57108) 2023-10-10T20:28:40.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee02559700 1 -- 172.21.15.148:0/971175535 learned_addr learned my addr 172.21.15.148:0/971175535 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:40.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 -- 172.21.15.148:0/971175535 >> v2:172.21.15.161:3300/0 conn(0x7fedfc06f2a0 msgr2=0x7fedfc06f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:40.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee02d5a700 1 --2- 172.21.15.148:0/971175535 >> v2:172.21.15.161:3300/0 conn(0x7fedfc06f2a0 0x7fedfc06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 --2- 172.21.15.148:0/971175535 >> v2:172.21.15.161:3300/0 conn(0x7fedfc06f2a0 0x7fedfc06f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01557700 1 -- 172.21.15.148:0/971175535 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465043365 0 0) 0x7fedfc15b6a0 con 0x7fedfc0ce5c0 2023-10-10T20:28:40.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 -- 172.21.15.148:0/971175535 >> v1:172.21.15.148:6789/0 conn(0x7fedfc0ce5c0 legacy=0x7fedfc1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 -- 172.21.15.148:0/971175535 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc06fc00 con 0x7fedfc0ce990 2023-10-10T20:28:40.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.438+0000 7fee01d58700 1 --2- 172.21.15.148:0/971175535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc0ce990 0x7fedfc15c0f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0ae3c0 tx=0x7fedec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc36cd551309f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee01557700 1 -- 172.21.15.148:0/971175535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00b6b0 con 0x7fedfc0ce990 2023-10-10T20:28:40.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee01557700 1 -- 172.21.15.148:0/971175535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec00b810 con 0x7fedfc0ce990 2023-10-10T20:28:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee01557700 1 -- 172.21.15.148:0/971175535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00bbb0 con 0x7fedfc0ce990 2023-10-10T20:28:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 -- 172.21.15.148:0/971175535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc0ce990 msgr2=0x7fedfc15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 --2- 172.21.15.148:0/971175535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc0ce990 0x7fedfc15c0f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0ae3c0 tx=0x7fedec0091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 -- 172.21.15.148:0/971175535 shutdown_connections 2023-10-10T20:28:40.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 --2- 172.21.15.148:0/971175535 >> v2:172.21.15.161:3300/0 conn(0x7fedfc06f2a0 0x7fedfc06f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 --2- 172.21.15.148:0/971175535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc0ce990 0x7fedfc15c0f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 -- 172.21.15.148:0/971175535 >> 172.21.15.148:0/971175535 conn(0x7fedfc06bc70 msgr2=0x7fedfc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:40.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 -- 172.21.15.148:0/971175535 shutdown_connections 2023-10-10T20:28:40.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.439+0000 7fee0355b700 1 -- 172.21.15.148:0/971175535 wait complete. 2023-10-10T20:28:40.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.440+0000 7fee0355b700 1 Processor -- start 2023-10-10T20:28:40.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.440+0000 7fee0355b700 1 -- start start 2023-10-10T20:28:40.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedfc0ce5c0 0x7fedfc0a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedfc0ce990 0x7fedfc16a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc1561b0 con 0x7fedfc06f2a0 2023-10-10T20:28:40.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0a6af0 con 0x7fedfc0ce990 2023-10-10T20:28:40.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee0355b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0c7d50 con 0x7fedfc0ce5c0 2023-10-10T20:28:40.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36192/0 (socket says 172.21.15.148:36192) 2023-10-10T20:28:40.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 -- 172.21.15.148:0/2230603169 learned_addr learned my addr 172.21.15.148:0/2230603169 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:40.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 -- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedfc0ce5c0 msgr2=0x7fedfc0a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedfc0ce5c0 0x7fedfc0a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 -- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedfc0ce990 msgr2=0x7fedfc16a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:40.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedfc0ce990 0x7fedfc16a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:40.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 -- 172.21.15.148:0/2230603169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009420 con 0x7fedfc06f2a0 2023-10-10T20:28:40.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.441+0000 7fee02559700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c810 tx=0x7fedf8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.457 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8005b30 con 0x7fedfc06f2a0 2023-10-10T20:28:40.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8005c90 con 0x7fedfc06f2a0 2023-10-10T20:28:40.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc068650 con 0x7fedfc06f2a0 2023-10-10T20:28:40.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8007300 con 0x7fedfc06f2a0 2023-10-10T20:28:40.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc068c10 con 0x7fedfc06f2a0 2023-10-10T20:28:40.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.442+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc0ae100 con 0x7fedfc06f2a0 2023-10-10T20:28:40.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.443+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedf801b400 con 0x7fedfc06f2a0 2023-10-10T20:28:40.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.443+0000 7fedf27fc700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 0x7fede00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.443+0000 7fee01d58700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 0x7fede00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.444+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fedf80840b0 con 0x7fedfc06f2a0 2023-10-10T20:28:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.444+0000 7fee01d58700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 0x7fede00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0cedf0 tx=0x7fedec017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.448+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedf80513a0 con 0x7fedfc06f2a0 2023-10-10T20:28:40.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:40.674+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fedfc04bb80 con 0x7fedfc06f2a0 2023-10-10T20:28:41.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.647+0000 7fedf27fc700 1 -- 172.21.15.148:0/2230603169 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7fedf80549c0 con 0x7fedfc06f2a0 2023-10-10T20:28:41.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.649+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 msgr2=0x7fede00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:41.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.649+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 0x7fede00693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0cedf0 tx=0x7fedec017000 comp rx=0 tx=0).stop 2023-10-10T20:28:41.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.649+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 msgr2=0x7fedfc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:41.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.649+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c810 tx=0x7fedf8007df0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 shutdown_connections 2023-10-10T20:28:41.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fede0066f30 0x7fede00693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedfc0ce990 0x7fedfc16a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedfc06f2a0 0x7fedfc068040 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 --2- 172.21.15.148:0/2230603169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedfc0ce5c0 0x7fedfc0a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 >> 172.21.15.148:0/2230603169 conn(0x7fedfc06bc70 msgr2=0x7fedfc156990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:41.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 shutdown_connections 2023-10-10T20:28:41.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.650+0000 7fee0355b700 1 -- 172.21.15.148:0/2230603169 wait complete. 2023-10-10T20:28:41.665 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:28:41.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.783+0000 7f45b63dd700 1 Processor -- start 2023-10-10T20:28:41.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.783+0000 7f45b63dd700 1 -- start start 2023-10-10T20:28:41.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f45b0159ec0 0x7f45b00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45b015a290 0x7f45b00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b006a5a0 0x7f45b006aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b006af60 con 0x7f45b015a290 2023-10-10T20:28:41.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00b0790 con 0x7f45b0159ec0 2023-10-10T20:28:41.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b63dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00ce1f0 con 0x7f45b006a5a0 2023-10-10T20:28:41.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45af7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45b015a290 0x7f45b00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45af7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45b015a290 0x7f45b00afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36208/0 (socket says 172.21.15.148:36208) 2023-10-10T20:28:41.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45af7fe700 1 -- 172.21.15.148:0/3929891622 learned_addr learned my addr 172.21.15.148:0/3929891622 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:41.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45b497a700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b006a5a0 0x7f45b006aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45affff700 1 --2- 172.21.15.148:0/3929891622 >> v2:172.21.15.161:3300/0 conn(0x7f45b0159ec0 0x7f45b00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45af7fe700 1 -- 172.21.15.148:0/3929891622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b006a5a0 msgr2=0x7f45b006aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:41.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.784+0000 7f45af7fe700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b006a5a0 0x7f45b006aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45af7fe700 1 -- 172.21.15.148:0/3929891622 >> v2:172.21.15.161:3300/0 conn(0x7f45b0159ec0 msgr2=0x7f45b00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45af7fe700 1 --2- 172.21.15.148:0/3929891622 >> v2:172.21.15.161:3300/0 conn(0x7f45b0159ec0 0x7f45b00af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45af7fe700 1 -- 172.21.15.148:0/3929891622 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00b0b20 con 0x7f45b015a290 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45af7fe700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b015a290 0x7f45b00afe80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f45b00898e0 tx=0x7f45a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1e1815910a0e22 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45aeffd700 1 -- 172.21.15.148:0/3929891622 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400b640 con 0x7f45b015a290 2023-10-10T20:28:41.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45aeffd700 1 -- 172.21.15.148:0/3929891622 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a400b7a0 con 0x7f45b015a290 2023-10-10T20:28:41.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45aeffd700 1 -- 172.21.15.148:0/3929891622 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400bb60 con 0x7f45b015a290 2023-10-10T20:28:41.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 -- 172.21.15.148:0/3929891622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b015a290 msgr2=0x7f45b00afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:41.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b015a290 0x7f45b00afe80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f45b00898e0 tx=0x7f45a4009250 comp rx=0 tx=0).stop 2023-10-10T20:28:41.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 -- 172.21.15.148:0/3929891622 shutdown_connections 2023-10-10T20:28:41.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b006a5a0 0x7f45b006aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 --2- 172.21.15.148:0/3929891622 >> v2:172.21.15.161:3300/0 conn(0x7f45b0159ec0 0x7f45b00af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 --2- 172.21.15.148:0/3929891622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b015a290 0x7f45b00afe80 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 -- 172.21.15.148:0/3929891622 >> 172.21.15.148:0/3929891622 conn(0x7f45b00d1880 msgr2=0x7f45b00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 -- 172.21.15.148:0/3929891622 shutdown_connections 2023-10-10T20:28:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.785+0000 7f45b63dd700 1 -- 172.21.15.148:0/3929891622 wait complete. 2023-10-10T20:28:41.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.786+0000 7f45b63dd700 1 Processor -- start 2023-10-10T20:28:41.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 -- start start 2023-10-10T20:28:41.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 0x7f45b015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 0x7f45b016d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00b6a70 con 0x7f45b006a5a0 2023-10-10T20:28:41.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b015e8d0 con 0x7f45b015a290 2023-10-10T20:28:41.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45b63dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b015dcc0 con 0x7f45b0159ec0 2023-10-10T20:28:41.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.787+0000 7f45affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36214/0 (socket says 172.21.15.148:36214) 2023-10-10T20:28:41.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 -- 172.21.15.148:0/4222072608 learned_addr learned my addr 172.21.15.148:0/4222072608 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:41.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45af7fe700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 0x7f45b015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45b497a700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 0x7f45b016d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 -- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 msgr2=0x7f45b015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 0x7f45b015cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 -- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 msgr2=0x7f45b016d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 0x7f45b016d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 -- 172.21.15.148:0/4222072608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40094f0 con 0x7f45b006a5a0 2023-10-10T20:28:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45af7fe700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 0x7f45b015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45b497a700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 0x7f45b016d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45affff700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ec80 tx=0x7f45a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000c9d0 con 0x7f45b006a5a0 2023-10-10T20:28:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000cb30 con 0x7f45b006a5a0 2023-10-10T20:28:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0013470 con 0x7f45b006a5a0 2023-10-10T20:28:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b015dfa0 con 0x7f45b006a5a0 2023-10-10T20:28:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.788+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b015e560 con 0x7f45b006a5a0 2023-10-10T20:28:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.789+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b00afe00 con 0x7f45b006a5a0 2023-10-10T20:28:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.790+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45a0013820 con 0x7f45b006a5a0 2023-10-10T20:28:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.790+0000 7f4597fff700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.790+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f45a0084920 con 0x7f45b006a5a0 2023-10-10T20:28:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.790+0000 7f45af7fe700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:41.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.790+0000 7f45af7fe700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f45b00898e0 tx=0x7f45a4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:41.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:41.794+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b00afe00 con 0x7f45b006a5a0 2023-10-10T20:28:42.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.015+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f45b00afe00 con 0x7f45b006a5a0 2023-10-10T20:28:42.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.651+0000 7f4597fff700 1 -- 172.21.15.148:0/4222072608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45b00afe00 con 0x7f45b006a5a0 2023-10-10T20:28:42.656 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 msgr2=0x7f459c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f45b00898e0 tx=0x7f45a4017000 comp rx=0 tx=0).stop 2023-10-10T20:28:42.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 msgr2=0x7f45b015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ec80 tx=0x7f45a00091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 shutdown_connections 2023-10-10T20:28:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b015a290 0x7f45b016d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b006a5a0 0x7f45b015ed60 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 --2- 172.21.15.148:0/4222072608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45b0159ec0 0x7f45b015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 >> 172.21.15.148:0/4222072608 conn(0x7f45b00d1880 msgr2=0x7f45b00cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:42.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 shutdown_connections 2023-10-10T20:28:42.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.654+0000 7f45b63dd700 1 -- 172.21.15.148:0/4222072608 wait complete. 2023-10-10T20:28:42.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.783+0000 7f288ad0c700 1 Processor -- start 2023-10-10T20:28:42.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.783+0000 7f288ad0c700 1 -- start start 2023-10-10T20:28:42.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f288ad0c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2884068280 0x7f28840adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:42.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f288ad0c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840c87d0 con 0x7f2884156e20 2023-10-10T20:28:42.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f288ad0c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2884068750 con 0x7f2884156a50 2023-10-10T20:28:42.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f288ad0c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2884068d10 con 0x7f2884068280 2023-10-10T20:28:42.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f28892a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2884068280 0x7f28840adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:42.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f28892a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2884068280 0x7f28840adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58500/0 (socket says 172.21.15.148:58500) 2023-10-10T20:28:42.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f287bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2884156e20 0x7f2884067b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43448/0 (socket says 172.21.15.148:43448) 2023-10-10T20:28:42.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.784+0000 7f28892a9700 1 -- 172.21.15.148:0/830272351 learned_addr learned my addr 172.21.15.148:0/830272351 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:42.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f28892a9700 1 -- 172.21.15.148:0/830272351 >> v1:172.21.15.161:6789/0 conn(0x7f2884156a50 legacy=0x7f28840c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f28892a9700 1 -- 172.21.15.148:0/830272351 >> v1:172.21.15.148:6789/0 conn(0x7f2884156e20 legacy=0x7f2884067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f287b7fe700 1 -- 172.21.15.148:0/830272351 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495189207 0 0) 0x7f28840c87d0 con 0x7f2884156e20 2023-10-10T20:28:42.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f28892a9700 1 -- 172.21.15.148:0/830272351 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884068a30 con 0x7f2884068280 2023-10-10T20:28:42.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f28892a9700 1 --2- 172.21.15.148:0/830272351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f28840adab0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f287c0099b0 tx=0x7f287c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc62dad64a94c6f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:42.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f287b7fe700 1 -- 172.21.15.148:0/830272351 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c00b860 con 0x7f2884068280 2023-10-10T20:28:42.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f287b7fe700 1 -- 172.21.15.148:0/830272351 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c00b9c0 con 0x7f2884068280 2023-10-10T20:28:42.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.785+0000 7f287b7fe700 1 -- 172.21.15.148:0/830272351 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c00bd60 con 0x7f2884068280 2023-10-10T20:28:42.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 -- 172.21.15.148:0/830272351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 msgr2=0x7f28840adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 --2- 172.21.15.148:0/830272351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f28840adab0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f287c0099b0 tx=0x7f287c009250 comp rx=0 tx=0).stop 2023-10-10T20:28:42.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 -- 172.21.15.148:0/830272351 shutdown_connections 2023-10-10T20:28:42.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 --2- 172.21.15.148:0/830272351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f28840adab0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 -- 172.21.15.148:0/830272351 >> 172.21.15.148:0/830272351 conn(0x7f288406a250 msgr2=0x7f288406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:42.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 -- 172.21.15.148:0/830272351 shutdown_connections 2023-10-10T20:28:42.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.786+0000 7f288ad0c700 1 -- 172.21.15.148:0/830272351 wait complete. 2023-10-10T20:28:42.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.787+0000 7f288ad0c700 1 Processor -- start 2023-10-10T20:28:42.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.787+0000 7f288ad0c700 1 -- start start 2023-10-10T20:28:42.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:42.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2884156a50 0x7f28840aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:42.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2884156e20 0x7f28840b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:42.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288416a910 con 0x7f2884068280 2023-10-10T20:28:42.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ab180 con 0x7f2884156e20 2023-10-10T20:28:42.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288416a250 con 0x7f2884156a50 2023-10-10T20:28:42.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:42.797 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58508/0 (socket says 172.21.15.148:58508) 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 -- 172.21.15.148:0/889712198 learned_addr learned my addr 172.21.15.148:0/889712198 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f28892a9700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2884156e20 0x7f28840b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f287bfff700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2884156a50 0x7f28840aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 -- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2884156a50 msgr2=0x7f28840aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2884156a50 0x7f28840aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 -- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2884156e20 msgr2=0x7f28840b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:42.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2884156e20 0x7f28840b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:42.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 -- 172.21.15.148:0/889712198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0094f0 con 0x7f2884068280 2023-10-10T20:28:42.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2888aa8700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f288000c900 tx=0x7f2880001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:42.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880012630 con 0x7f2884068280 2023-10-10T20:28:42.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800127b0 con 0x7f2884068280 2023-10-10T20:28:42.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288416a530 con 0x7f2884068280 2023-10-10T20:28:42.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880012b70 con 0x7f2884068280 2023-10-10T20:28:42.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.788+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28840aadc0 con 0x7f2884068280 2023-10-10T20:28:42.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.789+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288416ab80 con 0x7f2884068280 2023-10-10T20:28:42.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.790+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2880003030 con 0x7f2884068280 2023-10-10T20:28:42.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.790+0000 7f2878ff9700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:42.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.790+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28800844d0 con 0x7f2884068280 2023-10-10T20:28:42.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.790+0000 7f287bfff700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:42.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.791+0000 7f287bfff700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 0x7f28700693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28740011e0 tx=0x7f2874007bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:42.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:42.794+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2880051740 con 0x7f2884068280 2023-10-10T20:28:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.018+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f288404c3c0 con 0x7f2884068280 2023-10-10T20:28:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.018+0000 7f2878ff9700 1 -- 172.21.15.148:0/889712198 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f2880054d60 con 0x7f2884068280 2023-10-10T20:28:43.021 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:43.021 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:28:43.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 msgr2=0x7f28700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 0x7f28700693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28740011e0 tx=0x7f2874007bd0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 msgr2=0x7f288416ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f288000c900 tx=0x7f2880001740 comp rx=0 tx=0).stop 2023-10-10T20:28:43.023 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 shutdown_connections 2023-10-10T20:28:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2870066f30 0x7f28700693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2884156e20 0x7f28840b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2884068280 0x7f288416ad10 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 --2- 172.21.15.148:0/889712198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2884156a50 0x7f28840aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 >> 172.21.15.148:0/889712198 conn(0x7f288406a250 msgr2=0x7f28840afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 shutdown_connections 2023-10-10T20:28:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.020+0000 7f288ad0c700 1 -- 172.21.15.148:0/889712198 wait complete. 2023-10-10T20:28:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:28:43.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:28:43.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.147+0000 7fad32859700 1 Processor -- start 2023-10-10T20:28:43.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.147+0000 7fad32859700 1 -- start start 2023-10-10T20:28:43.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.147+0000 7fad32859700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad2c157870 0x7fad2c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.147+0000 7fad32859700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad2c15a620 0x7fad2c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad32859700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c15cbb0 con 0x7fad2c15d580 2023-10-10T20:28:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad32859700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c15bfa0 con 0x7fad2c15a620 2023-10-10T20:28:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad32859700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c15c280 con 0x7fad2c157870 2023-10-10T20:28:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad2c15a620 0x7fad2c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad30df6700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fad2c15d580 0x7fad2c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43456/0 (socket says 172.21.15.148:43456) 2023-10-10T20:28:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad30df6700 1 -- 172.21.15.148:0/1943058033 learned_addr learned my addr 172.21.15.148:0/1943058033 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:43.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2bfff700 1 --2- 172.21.15.148:0/1943058033 >> v2:172.21.15.161:3300/0 conn(0x7fad2c157870 0x7fad2c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 -- 172.21.15.148:0/1943058033 >> v2:172.21.15.161:3300/0 conn(0x7fad2c157870 msgr2=0x7fad2c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:43.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 --2- 172.21.15.148:0/1943058033 >> v2:172.21.15.161:3300/0 conn(0x7fad2c157870 0x7fad2c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2affd700 1 -- 172.21.15.148:0/1943058033 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314676047 0 0) 0x7fad2c15cbb0 con 0x7fad2c15d580 2023-10-10T20:28:43.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 -- 172.21.15.148:0/1943058033 >> v1:172.21.15.148:6789/0 conn(0x7fad2c15d580 legacy=0x7fad2c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 -- 172.21.15.148:0/1943058033 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c15c560 con 0x7fad2c15a620 2023-10-10T20:28:43.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2b7fe700 1 --2- 172.21.15.148:0/1943058033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15d040 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fad2c069d80 tx=0x7fad200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6c974ad4d09071 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2affd700 1 -- 172.21.15.148:0/1943058033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000b6b0 con 0x7fad2c15a620 2023-10-10T20:28:43.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2affd700 1 -- 172.21.15.148:0/1943058033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2000b810 con 0x7fad2c15a620 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.148+0000 7fad2affd700 1 -- 172.21.15.148:0/1943058033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000bbb0 con 0x7fad2c15a620 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 -- 172.21.15.148:0/1943058033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 msgr2=0x7fad2c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 --2- 172.21.15.148:0/1943058033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15d040 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fad2c069d80 tx=0x7fad200091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 -- 172.21.15.148:0/1943058033 shutdown_connections 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 --2- 172.21.15.148:0/1943058033 >> v2:172.21.15.161:3300/0 conn(0x7fad2c157870 0x7fad2c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 --2- 172.21.15.148:0/1943058033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15d040 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 -- 172.21.15.148:0/1943058033 >> 172.21.15.148:0/1943058033 conn(0x7fad2c068ce0 msgr2=0x7fad2c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 -- 172.21.15.148:0/1943058033 shutdown_connections 2023-10-10T20:28:43.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.149+0000 7fad32859700 1 -- 172.21.15.148:0/1943058033 wait complete. 2023-10-10T20:28:43.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.150+0000 7fad32859700 1 Processor -- start 2023-10-10T20:28:43.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.150+0000 7fad32859700 1 -- start start 2023-10-10T20:28:43.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c157870 0x7fad2c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.160 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c15d580 0x7fad2c15f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0b1460 con 0x7fad2c15a620 2023-10-10T20:28:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c15fea0 con 0x7fad2c15d580 2023-10-10T20:28:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad32859700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0b0850 con 0x7fad2c157870 2023-10-10T20:28:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58518/0 (socket says 172.21.15.148:58518) 2023-10-10T20:28:43.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 -- 172.21.15.148:0/3724880563 learned_addr learned my addr 172.21.15.148:0/3724880563 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:43.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad30df6700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c15d580 0x7fad2c15f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2bfff700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c157870 0x7fad2c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 -- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c157870 msgr2=0x7fad2c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c157870 0x7fad2c1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 -- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c15d580 msgr2=0x7fad2c15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c15d580 0x7fad2c15f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.151+0000 7fad2b7fe700 1 -- 172.21.15.148:0/3724880563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009420 con 0x7fad2c15a620 2023-10-10T20:28:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad2b7fe700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fad20013000 tx=0x7fad20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20011420 con 0x7fad2c15a620 2023-10-10T20:28:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20011580 con 0x7fad2c15a620 2023-10-10T20:28:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c0b0b30 con 0x7fad2c15a620 2023-10-10T20:28:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20011920 con 0x7fad2c15a620 2023-10-10T20:28:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.152+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0b10f0 con 0x7fad2c15a620 2023-10-10T20:28:43.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.153+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c0b15d0 con 0x7fad2c15a620 2023-10-10T20:28:43.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.153+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad20011d10 con 0x7fad2c15a620 2023-10-10T20:28:43.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.153+0000 7fad13fff700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 0x7fad180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.154+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fad20085c50 con 0x7fad2c15a620 2023-10-10T20:28:43.165 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.154+0000 7fad2bfff700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 0x7fad180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.154+0000 7fad2bfff700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 0x7fad180693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fad2c15d350 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.158+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad2008b020 con 0x7fad2c15a620 2023-10-10T20:28:43.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.341+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad2c04c3c0 con 0x7fad2c15a620 2023-10-10T20:28:43.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.342+0000 7fad13fff700 1 -- 172.21.15.148:0/3724880563 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+15413 (secure 0 0 0) 0x7fad20052f40 con 0x7fad2c15a620 2023-10-10T20:28:43.345 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:43.345 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":162,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:28:40.632809+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:28:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 msgr2=0x7fad180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 0x7fad180693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fad2c15d350 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2023-10-10T20:28:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 msgr2=0x7fad2c15f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fad20013000 tx=0x7fad20017000 comp rx=0 tx=0).stop 2023-10-10T20:28:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 shutdown_connections 2023-10-10T20:28:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad18066f30 0x7fad180693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2c15d580 0x7fad2c15f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2c157870 0x7fad2c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 --2- 172.21.15.148:0/3724880563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad2c15a620 0x7fad2c15f420 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 >> 172.21.15.148:0/3724880563 conn(0x7fad2c068ce0 msgr2=0x7fad2c159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.344+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 shutdown_connections 2023-10-10T20:28:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.345+0000 7fad32859700 1 -- 172.21.15.148:0/3724880563 wait complete. 2023-10-10T20:28:43.359 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:28:43.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:28:43.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.473+0000 7fed87ac7700 1 Processor -- start 2023-10-10T20:28:43.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.473+0000 7fed87ac7700 1 -- start start 2023-10-10T20:28:43.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed87ac7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fed800b0c50 0x7fed80158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed87ac7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed801594f0 con 0x7fed800c9310 2023-10-10T20:28:43.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed87ac7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8006be10 con 0x7fed80069f80 2023-10-10T20:28:43.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed87ac7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8006c0f0 con 0x7fed800b0c50 2023-10-10T20:28:43.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed85863700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fed800c9310 0x7fed801579a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43470/0 (socket says 172.21.15.148:43470) 2023-10-10T20:28:43.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed85863700 1 -- 172.21.15.148:0/1725134967 learned_addr learned my addr 172.21.15.148:0/1725134967 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:43.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.474+0000 7fed86064700 1 --2- 172.21.15.148:0/1725134967 >> v2:172.21.15.148:3300/0 conn(0x7fed800b0c50 0x7fed80158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed86064700 1 -- 172.21.15.148:0/1725134967 >> v1:172.21.15.161:6789/0 conn(0x7fed80069f80 legacy=0x7fed800ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:43.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed86064700 1 -- 172.21.15.148:0/1725134967 >> v1:172.21.15.148:6789/0 conn(0x7fed800c9310 legacy=0x7fed801579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed84861700 1 -- 172.21.15.148:0/1725134967 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772129775 0 0) 0x7fed801594f0 con 0x7fed800c9310 2023-10-10T20:28:43.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed86064700 1 -- 172.21.15.148:0/1725134967 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8006c3d0 con 0x7fed800b0c50 2023-10-10T20:28:43.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed86064700 1 --2- 172.21.15.148:0/1725134967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed800b0c50 0x7fed80158fb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fed78009e60 tx=0x7fed78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a0364714165941 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed84861700 1 -- 172.21.15.148:0/1725134967 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800e070 con 0x7fed800b0c50 2023-10-10T20:28:43.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed84861700 1 -- 172.21.15.148:0/1725134967 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7800bbf0 con 0x7fed800b0c50 2023-10-10T20:28:43.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed84861700 1 -- 172.21.15.148:0/1725134967 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78012430 con 0x7fed800b0c50 2023-10-10T20:28:43.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed87ac7700 1 -- 172.21.15.148:0/1725134967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed800b0c50 msgr2=0x7fed80158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed87ac7700 1 --2- 172.21.15.148:0/1725134967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed800b0c50 0x7fed80158fb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fed78009e60 tx=0x7fed78009780 comp rx=0 tx=0).stop 2023-10-10T20:28:43.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed87ac7700 1 -- 172.21.15.148:0/1725134967 shutdown_connections 2023-10-10T20:28:43.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed87ac7700 1 --2- 172.21.15.148:0/1725134967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed800b0c50 0x7fed80158fb0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.475+0000 7fed87ac7700 1 -- 172.21.15.148:0/1725134967 >> 172.21.15.148:0/1725134967 conn(0x7fed8006fe50 msgr2=0x7fed80069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.476+0000 7fed87ac7700 1 -- 172.21.15.148:0/1725134967 shutdown_connections 2023-10-10T20:28:43.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.476+0000 7fed87ac7700 1 -- 172.21.15.148:0/1725134967 wait complete. 2023-10-10T20:28:43.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 Processor -- start 2023-10-10T20:28:43.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 -- start start 2023-10-10T20:28:43.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 0x7fed801685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 0x7fed80168b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed800b4110 con 0x7fed80069f80 2023-10-10T20:28:43.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed800b3820 con 0x7fed800b0c50 2023-10-10T20:28:43.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed87ac7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed800a73c0 con 0x7fed800c9310 2023-10-10T20:28:43.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.477+0000 7fed85863700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58530/0 (socket says 172.21.15.148:58530) 2023-10-10T20:28:43.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 -- 172.21.15.148:0/3896282004 learned_addr learned my addr 172.21.15.148:0/3896282004 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:43.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed86064700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 0x7fed80168b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85062700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 0x7fed801685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 -- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 msgr2=0x7fed80168b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 0x7fed80168b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 -- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 msgr2=0x7fed801685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 0x7fed801685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 -- 172.21.15.148:0/3896282004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78009a20 con 0x7fed80069f80 2023-10-10T20:28:43.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed86064700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 0x7fed80168b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:43.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85062700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 0x7fed801685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:43.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed85863700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c5b0 tx=0x7fed7c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c005b30 con 0x7fed80069f80 2023-10-10T20:28:43.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c005c90 con 0x7fed80069f80 2023-10-10T20:28:43.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed800b3ac0 con 0x7fed80069f80 2023-10-10T20:28:43.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c007300 con 0x7fed80069f80 2023-10-10T20:28:43.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.478+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed800b2a80 con 0x7fed80069f80 2023-10-10T20:28:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.479+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8004c3c0 con 0x7fed80069f80 2023-10-10T20:28:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.480+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed7c01b3b0 con 0x7fed80069f80 2023-10-10T20:28:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.480+0000 7fed75ffb700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.480+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed7c083a60 con 0x7fed80069f80 2023-10-10T20:28:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.480+0000 7fed85062700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.481+0000 7fed85062700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 0x7fed640693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed70007fd0 tx=0x7fed70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.485+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed7c050cd0 con 0x7fed80069f80 2023-10-10T20:28:43.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.662+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed800b4250 con 0x7fed64066f30 2023-10-10T20:28:43.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.670+0000 7fed75ffb700 1 -- 172.21.15.148:0/3896282004 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285588 (secure 0 0 0) 0x7fed70006820 con 0x7fed64066f30 2023-10-10T20:28:43.675 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:43.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 msgr2=0x7fed640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 0x7fed640693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed70007fd0 tx=0x7fed70009040 comp rx=0 tx=0).stop 2023-10-10T20:28:43.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 msgr2=0x7fed800b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c5b0 tx=0x7fed7c001070 comp rx=0 tx=0).stop 2023-10-10T20:28:43.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 shutdown_connections 2023-10-10T20:28:43.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fed64066f30 0x7fed640693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fed800c9310 0x7fed80168b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed80069f80 0x7fed800b2540 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 --2- 172.21.15.148:0/3896282004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed800b0c50 0x7fed801685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 >> 172.21.15.148:0/3896282004 conn(0x7fed8006fe50 msgr2=0x7fed800cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 shutdown_connections 2023-10-10T20:28:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.673+0000 7fed87ac7700 1 -- 172.21.15.148:0/3896282004 wait complete. 2023-10-10T20:28:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:28:43.687 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2023-10-10T20:28:43.448171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554313,"statfs":{"total":1600000000,"available":1591617050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.832740"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649121+0000","last_change":"2023-10-10T20:28:39.649081+0000","last_active":"2023-10-10T20:28:39.649121+0000","last_peered":"2023-10-10T20:28:39.649121+0000","last_clean":"2023-10-10T20:28:39.649121+0000","last_became_active":"2023-10-10T20:28:39.648924+0000","last_became_peered":"2023-10-10T20:28:39.648924+0000","last_unstale":"2023-10-10T20:28:39.649121+0000","last_undegraded":"2023-10-10T20:28:39.649121+0000","last_fullsized":"2023-10-10T20:28:39.649121+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:54.819407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642237+0000","last_change":"2023-10-10T20:28:39.642136+0000","last_active":"2023-10-10T20:28:39.642237+0000","last_peered":"2023-10-10T20:28:39.642237+0000","last_clean":"2023-10-10T20:28:39.642237+0000","last_became_active":"2023-10-10T20:28:39.641913+0000","last_became_peered":"2023-10-10T20:28:39.641913+0000","last_unstale":"2023-10-10T20:28:39.642237+0000","last_undegraded":"2023-10-10T20:28:39.642237+0000","last_fullsized":"2023-10-10T20:28:39.642237+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:52.174175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646980+0000","last_change":"2023-10-10T20:28:39.646698+0000","last_active":"2023-10-10T20:28:39.646980+0000","last_peered":"2023-10-10T20:28:39.646980+0000","last_clean":"2023-10-10T20:28:39.646980+0000","last_became_active":"2023-10-10T20:28:39.646085+0000","last_became_peered":"2023-10-10T20:28:39.646085+0000","last_unstale":"2023-10-10T20:28:39.646980+0000","last_undegraded":"2023-10-10T20:28:39.646980+0000","last_fullsized":"2023-10-10T20:28:39.646980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:03.432687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639363+0000","last_change":"2023-10-10T20:28:39.639297+0000","last_active":"2023-10-10T20:28:39.639363+0000","last_peered":"2023-10-10T20:28:39.639363+0000","last_clean":"2023-10-10T20:28:39.639363+0000","last_became_active":"2023-10-10T20:28:39.639073+0000","last_became_peered":"2023-10-10T20:28:39.639073+0000","last_unstale":"2023-10-10T20:28:39.639363+0000","last_undegraded":"2023-10-10T20:28:39.639363+0000","last_fullsized":"2023-10-10T20:28:39.639363+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:03.120000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648937+0000","last_change":"2023-10-10T20:28:39.648870+0000","last_active":"2023-10-10T20:28:39.648937+0000","last_peered":"2023-10-10T20:28:39.648937+0000","last_clean":"2023-10-10T20:28:39.648937+0000","last_became_active":"2023-10-10T20:28:39.648634+0000","last_became_peered":"2023-10-10T20:28:39.648634+0000","last_unstale":"2023-10-10T20:28:39.648937+0000","last_undegraded":"2023-10-10T20:28:39.648937+0000","last_fullsized":"2023-10-10T20:28:39.648937+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:27.810043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646523+0000","last_change":"2023-10-10T20:28:39.646421+0000","last_active":"2023-10-10T20:28:39.646523+0000","last_peered":"2023-10-10T20:28:39.646523+0000","last_clean":"2023-10-10T20:28:39.646523+0000","last_became_active":"2023-10-10T20:28:39.645235+0000","last_became_peered":"2023-10-10T20:28:39.645235+0000","last_unstale":"2023-10-10T20:28:39.646523+0000","last_undegraded":"2023-10-10T20:28:39.646523+0000","last_fullsized":"2023-10-10T20:28:39.646523+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:57.093229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648527+0000","last_change":"2023-10-10T20:28:39.648485+0000","last_active":"2023-10-10T20:28:39.648527+0000","last_peered":"2023-10-10T20:28:39.648527+0000","last_clean":"2023-10-10T20:28:39.648527+0000","last_became_active":"2023-10-10T20:28:39.648327+0000","last_became_peered":"2023-10-10T20:28:39.648327+0000","last_unstale":"2023-10-10T20:28:39.648527+0000","last_undegraded":"2023-10-10T20:28:39.648527+0000","last_fullsized":"2023-10-10T20:28:39.648527+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:28.934427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644349+0000","last_change":"2023-10-10T20:28:39.643796+0000","last_active":"2023-10-10T20:28:39.644349+0000","last_peered":"2023-10-10T20:28:39.644349+0000","last_clean":"2023-10-10T20:28:39.644349+0000","last_became_active":"2023-10-10T20:28:39.643595+0000","last_became_peered":"2023-10-10T20:28:39.643595+0000","last_unstale":"2023-10-10T20:28:39.644349+0000","last_undegraded":"2023-10-10T20:28:39.644349+0000","last_fullsized":"2023-10-10T20:28:39.644349+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:26:01.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642254+0000","last_change":"2023-10-10T20:28:39.642203+0000","last_active":"2023-10-10T20:28:39.642254+0000","last_peered":"2023-10-10T20:28:39.642254+0000","last_clean":"2023-10-10T20:28:39.642254+0000","last_became_active":"2023-10-10T20:28:39.641635+0000","last_became_peered":"2023-10-10T20:28:39.641635+0000","last_unstale":"2023-10-10T20:28:39.642254+0000","last_undegraded":"2023-10-10T20:28:39.642254+0000","last_fullsized":"2023-10-10T20:28:39.642254+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:59:33.550658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.650997+0000","last_change":"2023-10-10T20:28:39.650435+0000","last_active":"2023-10-10T20:28:39.650997+0000","last_peered":"2023-10-10T20:28:39.650997+0000","last_clean":"2023-10-10T20:28:39.650997+0000","last_became_active":"2023-10-10T20:28:39.649737+0000","last_became_peered":"2023-10-10T20:28:39.649737+0000","last_unstale":"2023-10-10T20:28:39.650997+0000","last_undegraded":"2023-10-10T20:28:39.650997+0000","last_fullsized":"2023-10-10T20:28:39.650997+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:34:13.077873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645895+0000","last_change":"2023-10-10T20:28:39.645679+0000","last_active":"2023-10-10T20:28:39.645895+0000","last_peered":"2023-10-10T20:28:39.645895+0000","last_clean":"2023-10-10T20:28:39.645895+0000","last_became_active":"2023-10-10T20:28:39.644432+0000","last_became_peered":"2023-10-10T20:28:39.644432+0000","last_unstale":"2023-10-10T20:28:39.645895+0000","last_undegraded":"2023-10-10T20:28:39.645895+0000","last_fullsized":"2023-10-10T20:28:39.645895+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:23.036296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647348+0000","last_change":"2023-10-10T20:28:39.646815+0000","last_active":"2023-10-10T20:28:39.647348+0000","last_peered":"2023-10-10T20:28:39.647348+0000","last_clean":"2023-10-10T20:28:39.647348+0000","last_became_active":"2023-10-10T20:28:39.646425+0000","last_became_peered":"2023-10-10T20:28:39.646425+0000","last_unstale":"2023-10-10T20:28:39.647348+0000","last_undegraded":"2023-10-10T20:28:39.647348+0000","last_fullsized":"2023-10-10T20:28:39.647348+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:59:33.550658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648246+0000","last_change":"2023-10-10T20:28:39.648138+0000","last_active":"2023-10-10T20:28:39.648246+0000","last_peered":"2023-10-10T20:28:39.648246+0000","last_clean":"2023-10-10T20:28:39.648246+0000","last_became_active":"2023-10-10T20:28:39.647580+0000","last_became_peered":"2023-10-10T20:28:39.647580+0000","last_unstale":"2023-10-10T20:28:39.648246+0000","last_undegraded":"2023-10-10T20:28:39.648246+0000","last_fullsized":"2023-10-10T20:28:39.648246+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:58:39.589233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647325+0000","last_change":"2023-10-10T20:28:39.647234+0000","last_active":"2023-10-10T20:28:39.647325+0000","last_peered":"2023-10-10T20:28:39.647325+0000","last_clean":"2023-10-10T20:28:39.647325+0000","last_became_active":"2023-10-10T20:28:39.646984+0000","last_became_peered":"2023-10-10T20:28:39.646984+0000","last_unstale":"2023-10-10T20:28:39.647325+0000","last_undegraded":"2023-10-10T20:28:39.647325+0000","last_fullsized":"2023-10-10T20:28:39.647325+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:23.036296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649295+0000","last_change":"2023-10-10T20:28:39.649247+0000","last_active":"2023-10-10T20:28:39.649295+0000","last_peered":"2023-10-10T20:28:39.649295+0000","last_clean":"2023-10-10T20:28:39.649295+0000","last_became_active":"2023-10-10T20:28:39.648882+0000","last_became_peered":"2023-10-10T20:28:39.648882+0000","last_unstale":"2023-10-10T20:28:39.649295+0000","last_undegraded":"2023-10-10T20:28:39.649295+0000","last_fullsized":"2023-10-10T20:28:39.649295+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:07.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646935+0000","last_change":"2023-10-10T20:28:39.646877+0000","last_active":"2023-10-10T20:28:39.646935+0000","last_peered":"2023-10-10T20:28:39.646935+0000","last_clean":"2023-10-10T20:28:39.646935+0000","last_became_active":"2023-10-10T20:28:39.646673+0000","last_became_peered":"2023-10-10T20:28:39.646673+0000","last_unstale":"2023-10-10T20:28:39.646935+0000","last_undegraded":"2023-10-10T20:28:39.646935+0000","last_fullsized":"2023-10-10T20:28:39.646935+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:13.161511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647325+0000","last_change":"2023-10-10T20:28:39.646661+0000","last_active":"2023-10-10T20:28:39.647325+0000","last_peered":"2023-10-10T20:28:39.647325+0000","last_clean":"2023-10-10T20:28:39.647325+0000","last_became_active":"2023-10-10T20:28:39.646082+0000","last_became_peered":"2023-10-10T20:28:39.646082+0000","last_unstale":"2023-10-10T20:28:39.647325+0000","last_undegraded":"2023-10-10T20:28:39.647325+0000","last_fullsized":"2023-10-10T20:28:39.647325+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:22.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646961+0000","last_change":"2023-10-10T20:28:39.646571+0000","last_active":"2023-10-10T20:28:39.646961+0000","last_peered":"2023-10-10T20:28:39.646961+0000","last_clean":"2023-10-10T20:28:39.646961+0000","last_became_active":"2023-10-10T20:28:39.645796+0000","last_became_peered":"2023-10-10T20:28:39.645796+0000","last_unstale":"2023-10-10T20:28:39.646961+0000","last_undegraded":"2023-10-10T20:28:39.646961+0000","last_fullsized":"2023-10-10T20:28:39.646961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:05.707813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644391+0000","last_change":"2023-10-10T20:28:39.644336+0000","last_active":"2023-10-10T20:28:39.644391+0000","last_peered":"2023-10-10T20:28:39.644391+0000","last_clean":"2023-10-10T20:28:39.644391+0000","last_became_active":"2023-10-10T20:28:39.643959+0000","last_became_peered":"2023-10-10T20:28:39.643959+0000","last_unstale":"2023-10-10T20:28:39.644391+0000","last_undegraded":"2023-10-10T20:28:39.644391+0000","last_fullsized":"2023-10-10T20:28:39.644391+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:28.441866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643485+0000","last_change":"2023-10-10T20:28:39.643420+0000","last_active":"2023-10-10T20:28:39.643485+0000","last_peered":"2023-10-10T20:28:39.643485+0000","last_clean":"2023-10-10T20:28:39.643485+0000","last_became_active":"2023-10-10T20:28:39.643174+0000","last_became_peered":"2023-10-10T20:28:39.643174+0000","last_unstale":"2023-10-10T20:28:39.643485+0000","last_undegraded":"2023-10-10T20:28:39.643485+0000","last_fullsized":"2023-10-10T20:28:39.643485+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:11.227327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648585+0000","last_change":"2023-10-10T20:28:39.648232+0000","last_active":"2023-10-10T20:28:39.648585+0000","last_peered":"2023-10-10T20:28:39.648585+0000","last_clean":"2023-10-10T20:28:39.648585+0000","last_became_active":"2023-10-10T20:28:39.647720+0000","last_became_peered":"2023-10-10T20:28:39.647720+0000","last_unstale":"2023-10-10T20:28:39.648585+0000","last_undegraded":"2023-10-10T20:28:39.648585+0000","last_fullsized":"2023-10-10T20:28:39.648585+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:53.439232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.636844+0000","last_change":"2023-10-10T20:28:39.636787+0000","last_active":"2023-10-10T20:28:39.636844+0000","last_peered":"2023-10-10T20:28:39.636844+0000","last_clean":"2023-10-10T20:28:39.636844+0000","last_became_active":"2023-10-10T20:28:39.636537+0000","last_became_peered":"2023-10-10T20:28:39.636537+0000","last_unstale":"2023-10-10T20:28:39.636844+0000","last_undegraded":"2023-10-10T20:28:39.636844+0000","last_fullsized":"2023-10-10T20:28:39.636844+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:28.573704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644285+0000","last_change":"2023-10-10T20:28:39.644176+0000","last_active":"2023-10-10T20:28:39.644285+0000","last_peered":"2023-10-10T20:28:39.644285+0000","last_clean":"2023-10-10T20:28:39.644285+0000","last_became_active":"2023-10-10T20:28:39.643945+0000","last_became_peered":"2023-10-10T20:28:39.643945+0000","last_unstale":"2023-10-10T20:28:39.644285+0000","last_undegraded":"2023-10-10T20:28:39.644285+0000","last_fullsized":"2023-10-10T20:28:39.644285+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:56.119191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.650040+0000","last_change":"2023-10-10T20:28:39.649981+0000","last_active":"2023-10-10T20:28:39.650040+0000","last_peered":"2023-10-10T20:28:39.650040+0000","last_clean":"2023-10-10T20:28:39.650040+0000","last_became_active":"2023-10-10T20:28:39.649730+0000","last_became_peered":"2023-10-10T20:28:39.649730+0000","last_unstale":"2023-10-10T20:28:39.650040+0000","last_undegraded":"2023-10-10T20:28:39.650040+0000","last_fullsized":"2023-10-10T20:28:39.650040+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:28.934427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642690+0000","last_change":"2023-10-10T20:28:39.642518+0000","last_active":"2023-10-10T20:28:39.642690+0000","last_peered":"2023-10-10T20:28:39.642690+0000","last_clean":"2023-10-10T20:28:39.642690+0000","last_became_active":"2023-10-10T20:28:39.642257+0000","last_became_peered":"2023-10-10T20:28:39.642257+0000","last_unstale":"2023-10-10T20:28:39.642690+0000","last_undegraded":"2023-10-10T20:28:39.642690+0000","last_fullsized":"2023-10-10T20:28:39.642690+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:42:49.780770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644741+0000","last_change":"2023-10-10T20:28:39.644678+0000","last_active":"2023-10-10T20:28:39.644741+0000","last_peered":"2023-10-10T20:28:39.644741+0000","last_clean":"2023-10-10T20:28:39.644741+0000","last_became_active":"2023-10-10T20:28:39.643784+0000","last_became_peered":"2023-10-10T20:28:39.643784+0000","last_unstale":"2023-10-10T20:28:39.644741+0000","last_undegraded":"2023-10-10T20:28:39.644741+0000","last_fullsized":"2023-10-10T20:28:39.644741+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:05.707813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641768+0000","last_change":"2023-10-10T20:28:39.641709+0000","last_active":"2023-10-10T20:28:39.641768+0000","last_peered":"2023-10-10T20:28:39.641768+0000","last_clean":"2023-10-10T20:28:39.641768+0000","last_became_active":"2023-10-10T20:28:39.641514+0000","last_became_peered":"2023-10-10T20:28:39.641514+0000","last_unstale":"2023-10-10T20:28:39.641768+0000","last_undegraded":"2023-10-10T20:28:39.641768+0000","last_fullsized":"2023-10-10T20:28:39.641768+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:47:19.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642660+0000","last_change":"2023-10-10T20:28:39.642603+0000","last_active":"2023-10-10T20:28:39.642660+0000","last_peered":"2023-10-10T20:28:39.642660+0000","last_clean":"2023-10-10T20:28:39.642660+0000","last_became_active":"2023-10-10T20:28:39.642366+0000","last_became_peered":"2023-10-10T20:28:39.642366+0000","last_unstale":"2023-10-10T20:28:39.642660+0000","last_undegraded":"2023-10-10T20:28:39.642660+0000","last_fullsized":"2023-10-10T20:28:39.642660+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:57.498975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644799+0000","last_change":"2023-10-10T20:28:39.644754+0000","last_active":"2023-10-10T20:28:39.644799+0000","last_peered":"2023-10-10T20:28:39.644799+0000","last_clean":"2023-10-10T20:28:39.644799+0000","last_became_active":"2023-10-10T20:28:39.644559+0000","last_became_peered":"2023-10-10T20:28:39.644559+0000","last_unstale":"2023-10-10T20:28:39.644799+0000","last_undegraded":"2023-10-10T20:28:39.644799+0000","last_fullsized":"2023-10-10T20:28:39.644799+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:09.644327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641996+0000","last_change":"2023-10-10T20:28:39.641907+0000","last_active":"2023-10-10T20:28:39.641996+0000","last_peered":"2023-10-10T20:28:39.641996+0000","last_clean":"2023-10-10T20:28:39.641996+0000","last_became_active":"2023-10-10T20:28:39.641667+0000","last_became_peered":"2023-10-10T20:28:39.641667+0000","last_unstale":"2023-10-10T20:28:39.641996+0000","last_undegraded":"2023-10-10T20:28:39.641996+0000","last_fullsized":"2023-10-10T20:28:39.641996+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:09.644327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641872+0000","last_change":"2023-10-10T20:28:39.641811+0000","last_active":"2023-10-10T20:28:39.641872+0000","last_peered":"2023-10-10T20:28:39.641872+0000","last_clean":"2023-10-10T20:28:39.641872+0000","last_became_active":"2023-10-10T20:28:39.641434+0000","last_became_peered":"2023-10-10T20:28:39.641434+0000","last_unstale":"2023-10-10T20:28:39.641872+0000","last_undegraded":"2023-10-10T20:28:39.641872+0000","last_fullsized":"2023-10-10T20:28:39.641872+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:01.342211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642399+0000","last_change":"2023-10-10T20:28:39.642336+0000","last_active":"2023-10-10T20:28:39.642399+0000","last_peered":"2023-10-10T20:28:39.642399+0000","last_clean":"2023-10-10T20:28:39.642399+0000","last_became_active":"2023-10-10T20:28:39.642019+0000","last_became_peered":"2023-10-10T20:28:39.642019+0000","last_unstale":"2023-10-10T20:28:39.642399+0000","last_undegraded":"2023-10-10T20:28:39.642399+0000","last_fullsized":"2023-10-10T20:28:39.642399+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:48:41.771961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648620+0000","last_change":"2023-10-10T20:28:39.648497+0000","last_active":"2023-10-10T20:28:39.648620+0000","last_peered":"2023-10-10T20:28:39.648620+0000","last_clean":"2023-10-10T20:28:39.648620+0000","last_became_active":"2023-10-10T20:28:39.648140+0000","last_became_peered":"2023-10-10T20:28:39.648140+0000","last_unstale":"2023-10-10T20:28:39.648620+0000","last_undegraded":"2023-10-10T20:28:39.648620+0000","last_fullsized":"2023-10-10T20:28:39.648620+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:16.913282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"161'4","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657366+0000","last_change":"2023-10-10T20:28:38.628272+0000","last_active":"2023-10-10T20:28:39.657366+0000","last_peered":"2023-10-10T20:28:39.657366+0000","last_clean":"2023-10-10T20:28:39.657366+0000","last_became_active":"2023-10-10T20:28:38.627823+0000","last_became_peered":"2023-10-10T20:28:38.627823+0000","last_unstale":"2023-10-10T20:28:39.657366+0000","last_undegraded":"2023-10-10T20:28:39.657366+0000","last_fullsized":"2023-10-10T20:28:39.657366+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:09.183516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639932+0000","last_change":"2023-10-10T20:28:39.639709+0000","last_active":"2023-10-10T20:28:39.639932+0000","last_peered":"2023-10-10T20:28:39.639932+0000","last_clean":"2023-10-10T20:28:39.639932+0000","last_became_active":"2023-10-10T20:28:39.639512+0000","last_became_peered":"2023-10-10T20:28:39.639512+0000","last_unstale":"2023-10-10T20:28:39.639932+0000","last_undegraded":"2023-10-10T20:28:39.639932+0000","last_fullsized":"2023-10-10T20:28:39.639932+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:34.074110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657760+0000","last_change":"2023-10-10T20:28:38.627254+0000","last_active":"2023-10-10T20:28:39.657760+0000","last_peered":"2023-10-10T20:28:39.657760+0000","last_clean":"2023-10-10T20:28:39.657760+0000","last_became_active":"2023-10-10T20:28:38.626818+0000","last_became_peered":"2023-10-10T20:28:38.626818+0000","last_unstale":"2023-10-10T20:28:39.657760+0000","last_undegraded":"2023-10-10T20:28:39.657760+0000","last_fullsized":"2023-10-10T20:28:39.657760+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:46.437313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645539+0000","last_change":"2023-10-10T20:28:39.645476+0000","last_active":"2023-10-10T20:28:39.645539+0000","last_peered":"2023-10-10T20:28:39.645539+0000","last_clean":"2023-10-10T20:28:39.645539+0000","last_became_active":"2023-10-10T20:28:39.644121+0000","last_became_peered":"2023-10-10T20:28:39.644121+0000","last_unstale":"2023-10-10T20:28:39.645539+0000","last_undegraded":"2023-10-10T20:28:39.645539+0000","last_fullsized":"2023-10-10T20:28:39.645539+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:10.187288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658281+0000","last_change":"2023-10-10T20:28:38.626245+0000","last_active":"2023-10-10T20:28:39.658281+0000","last_peered":"2023-10-10T20:28:39.658281+0000","last_clean":"2023-10-10T20:28:39.658281+0000","last_became_active":"2023-10-10T20:28:38.625947+0000","last_became_peered":"2023-10-10T20:28:38.625947+0000","last_unstale":"2023-10-10T20:28:39.658281+0000","last_undegraded":"2023-10-10T20:28:39.658281+0000","last_fullsized":"2023-10-10T20:28:39.658281+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:24.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.651016+0000","last_change":"2023-10-10T20:28:39.650680+0000","last_active":"2023-10-10T20:28:39.651016+0000","last_peered":"2023-10-10T20:28:39.651016+0000","last_clean":"2023-10-10T20:28:39.651016+0000","last_became_active":"2023-10-10T20:28:39.650075+0000","last_became_peered":"2023-10-10T20:28:39.650075+0000","last_unstale":"2023-10-10T20:28:39.651016+0000","last_undegraded":"2023-10-10T20:28:39.651016+0000","last_fullsized":"2023-10-10T20:28:39.651016+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:58.435248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659118+0000","last_change":"2023-10-10T20:28:38.630687+0000","last_active":"2023-10-10T20:28:39.659118+0000","last_peered":"2023-10-10T20:28:39.659118+0000","last_clean":"2023-10-10T20:28:39.659118+0000","last_became_active":"2023-10-10T20:28:38.630435+0000","last_became_peered":"2023-10-10T20:28:38.630435+0000","last_unstale":"2023-10-10T20:28:39.659118+0000","last_undegraded":"2023-10-10T20:28:39.659118+0000","last_fullsized":"2023-10-10T20:28:39.659118+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:23:02.137460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642037+0000","last_change":"2023-10-10T20:28:39.641968+0000","last_active":"2023-10-10T20:28:39.642037+0000","last_peered":"2023-10-10T20:28:39.642037+0000","last_clean":"2023-10-10T20:28:39.642037+0000","last_became_active":"2023-10-10T20:28:39.641380+0000","last_became_peered":"2023-10-10T20:28:39.641380+0000","last_unstale":"2023-10-10T20:28:39.642037+0000","last_undegraded":"2023-10-10T20:28:39.642037+0000","last_fullsized":"2023-10-10T20:28:39.642037+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:04:51.168883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657248+0000","last_change":"2023-10-10T20:28:38.628526+0000","last_active":"2023-10-10T20:28:39.657248+0000","last_peered":"2023-10-10T20:28:39.657248+0000","last_clean":"2023-10-10T20:28:39.657248+0000","last_became_active":"2023-10-10T20:28:38.628279+0000","last_became_peered":"2023-10-10T20:28:38.628279+0000","last_unstale":"2023-10-10T20:28:39.657248+0000","last_undegraded":"2023-10-10T20:28:39.657248+0000","last_fullsized":"2023-10-10T20:28:39.657248+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:23.092555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642632+0000","last_change":"2023-10-10T20:28:39.642305+0000","last_active":"2023-10-10T20:28:39.642632+0000","last_peered":"2023-10-10T20:28:39.642632+0000","last_clean":"2023-10-10T20:28:39.642632+0000","last_became_active":"2023-10-10T20:28:39.641940+0000","last_became_peered":"2023-10-10T20:28:39.641940+0000","last_unstale":"2023-10-10T20:28:39.642632+0000","last_undegraded":"2023-10-10T20:28:39.642632+0000","last_fullsized":"2023-10-10T20:28:39.642632+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:41:29.495073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656465+0000","last_change":"2023-10-10T20:28:38.627118+0000","last_active":"2023-10-10T20:28:39.656465+0000","last_peered":"2023-10-10T20:28:39.656465+0000","last_clean":"2023-10-10T20:28:39.656465+0000","last_became_active":"2023-10-10T20:28:38.626948+0000","last_became_peered":"2023-10-10T20:28:38.626948+0000","last_unstale":"2023-10-10T20:28:39.656465+0000","last_undegraded":"2023-10-10T20:28:39.656465+0000","last_fullsized":"2023-10-10T20:28:39.656465+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:17.416780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639560+0000","last_change":"2023-10-10T20:28:39.636255+0000","last_active":"2023-10-10T20:28:39.639560+0000","last_peered":"2023-10-10T20:28:39.639560+0000","last_clean":"2023-10-10T20:28:39.639560+0000","last_became_active":"2023-10-10T20:28:39.636027+0000","last_became_peered":"2023-10-10T20:28:39.636027+0000","last_unstale":"2023-10-10T20:28:39.639560+0000","last_undegraded":"2023-10-10T20:28:39.639560+0000","last_fullsized":"2023-10-10T20:28:39.639560+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:11:56.503437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.638968+0000","last_change":"2023-10-10T20:28:38.627400+0000","last_active":"2023-10-10T20:28:39.638968+0000","last_peered":"2023-10-10T20:28:39.638968+0000","last_clean":"2023-10-10T20:28:39.638968+0000","last_became_active":"2023-10-10T20:28:38.627015+0000","last_became_peered":"2023-10-10T20:28:38.627015+0000","last_unstale":"2023-10-10T20:28:39.638968+0000","last_undegraded":"2023-10-10T20:28:39.638968+0000","last_fullsized":"2023-10-10T20:28:39.638968+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:11.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642660+0000","last_change":"2023-10-10T20:28:39.641920+0000","last_active":"2023-10-10T20:28:39.642660+0000","last_peered":"2023-10-10T20:28:39.642660+0000","last_clean":"2023-10-10T20:28:39.642660+0000","last_became_active":"2023-10-10T20:28:39.641638+0000","last_became_peered":"2023-10-10T20:28:39.641638+0000","last_unstale":"2023-10-10T20:28:39.642660+0000","last_undegraded":"2023-10-10T20:28:39.642660+0000","last_fullsized":"2023-10-10T20:28:39.642660+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:46.519220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.636104+0000","last_change":"2023-10-10T20:28:38.629597+0000","last_active":"2023-10-10T20:28:39.636104+0000","last_peered":"2023-10-10T20:28:39.636104+0000","last_clean":"2023-10-10T20:28:39.636104+0000","last_became_active":"2023-10-10T20:28:38.629337+0000","last_became_peered":"2023-10-10T20:28:38.629337+0000","last_unstale":"2023-10-10T20:28:39.636104+0000","last_undegraded":"2023-10-10T20:28:39.636104+0000","last_fullsized":"2023-10-10T20:28:39.636104+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:33.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643573+0000","last_change":"2023-10-10T20:28:39.643460+0000","last_active":"2023-10-10T20:28:39.643573+0000","last_peered":"2023-10-10T20:28:39.643573+0000","last_clean":"2023-10-10T20:28:39.643573+0000","last_became_active":"2023-10-10T20:28:39.642997+0000","last_became_peered":"2023-10-10T20:28:39.642997+0000","last_unstale":"2023-10-10T20:28:39.643573+0000","last_undegraded":"2023-10-10T20:28:39.643573+0000","last_fullsized":"2023-10-10T20:28:39.643573+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:58.465941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633815+0000","last_change":"2023-10-10T20:28:38.630837+0000","last_active":"2023-10-10T20:28:39.633815+0000","last_peered":"2023-10-10T20:28:39.633815+0000","last_clean":"2023-10-10T20:28:39.633815+0000","last_became_active":"2023-10-10T20:28:38.630662+0000","last_became_peered":"2023-10-10T20:28:38.630662+0000","last_unstale":"2023-10-10T20:28:39.633815+0000","last_undegraded":"2023-10-10T20:28:39.633815+0000","last_fullsized":"2023-10-10T20:28:39.633815+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:53.815634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647368+0000","last_change":"2023-10-10T20:28:39.647074+0000","last_active":"2023-10-10T20:28:39.647368+0000","last_peered":"2023-10-10T20:28:39.647368+0000","last_clean":"2023-10-10T20:28:39.647368+0000","last_became_active":"2023-10-10T20:28:39.645697+0000","last_became_peered":"2023-10-10T20:28:39.645697+0000","last_unstale":"2023-10-10T20:28:39.647368+0000","last_undegraded":"2023-10-10T20:28:39.647368+0000","last_fullsized":"2023-10-10T20:28:39.647368+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:38.975918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640538+0000","last_change":"2023-10-10T20:28:38.630120+0000","last_active":"2023-10-10T20:28:39.640538+0000","last_peered":"2023-10-10T20:28:39.640538+0000","last_clean":"2023-10-10T20:28:39.640538+0000","last_became_active":"2023-10-10T20:28:38.629401+0000","last_became_peered":"2023-10-10T20:28:38.629401+0000","last_unstale":"2023-10-10T20:28:39.640538+0000","last_undegraded":"2023-10-10T20:28:39.640538+0000","last_fullsized":"2023-10-10T20:28:39.640538+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:46.437313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640189+0000","last_change":"2023-10-10T20:28:39.640055+0000","last_active":"2023-10-10T20:28:39.640189+0000","last_peered":"2023-10-10T20:28:39.640189+0000","last_clean":"2023-10-10T20:28:39.640189+0000","last_became_active":"2023-10-10T20:28:39.639841+0000","last_became_peered":"2023-10-10T20:28:39.639841+0000","last_unstale":"2023-10-10T20:28:39.640189+0000","last_undegraded":"2023-10-10T20:28:39.640189+0000","last_fullsized":"2023-10-10T20:28:39.640189+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:53.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"161'6","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659732+0000","last_change":"2023-10-10T20:28:38.625781+0000","last_active":"2023-10-10T20:28:39.659732+0000","last_peered":"2023-10-10T20:28:39.659732+0000","last_clean":"2023-10-10T20:28:39.659732+0000","last_became_active":"2023-10-10T20:28:38.625602+0000","last_became_peered":"2023-10-10T20:28:38.625602+0000","last_unstale":"2023-10-10T20:28:39.659732+0000","last_undegraded":"2023-10-10T20:28:39.659732+0000","last_fullsized":"2023-10-10T20:28:39.659732+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:51.170402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642211+0000","last_change":"2023-10-10T20:28:39.642134+0000","last_active":"2023-10-10T20:28:39.642211+0000","last_peered":"2023-10-10T20:28:39.642211+0000","last_clean":"2023-10-10T20:28:39.642211+0000","last_became_active":"2023-10-10T20:28:39.641118+0000","last_became_peered":"2023-10-10T20:28:39.641118+0000","last_unstale":"2023-10-10T20:28:39.642211+0000","last_undegraded":"2023-10-10T20:28:39.642211+0000","last_fullsized":"2023-10-10T20:28:39.642211+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658086+0000","last_change":"2023-10-10T20:28:38.628263+0000","last_active":"2023-10-10T20:28:39.658086+0000","last_peered":"2023-10-10T20:28:39.658086+0000","last_clean":"2023-10-10T20:28:39.658086+0000","last_became_active":"2023-10-10T20:28:38.628034+0000","last_became_peered":"2023-10-10T20:28:38.628034+0000","last_unstale":"2023-10-10T20:28:39.658086+0000","last_undegraded":"2023-10-10T20:28:39.658086+0000","last_fullsized":"2023-10-10T20:28:39.658086+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:05:54.825961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642777+0000","last_change":"2023-10-10T20:28:39.642722+0000","last_active":"2023-10-10T20:28:39.642777+0000","last_peered":"2023-10-10T20:28:39.642777+0000","last_clean":"2023-10-10T20:28:39.642777+0000","last_became_active":"2023-10-10T20:28:39.641971+0000","last_became_peered":"2023-10-10T20:28:39.641971+0000","last_unstale":"2023-10-10T20:28:39.642777+0000","last_undegraded":"2023-10-10T20:28:39.642777+0000","last_fullsized":"2023-10-10T20:28:39.642777+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:38.975918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639276+0000","last_change":"2023-10-10T20:28:38.626293+0000","last_active":"2023-10-10T20:28:39.639276+0000","last_peered":"2023-10-10T20:28:39.639276+0000","last_clean":"2023-10-10T20:28:39.639276+0000","last_became_active":"2023-10-10T20:28:38.626078+0000","last_became_peered":"2023-10-10T20:28:38.626078+0000","last_unstale":"2023-10-10T20:28:39.639276+0000","last_undegraded":"2023-10-10T20:28:39.639276+0000","last_fullsized":"2023-10-10T20:28:39.639276+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:37.087990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648228+0000","last_change":"2023-10-10T20:28:39.647989+0000","last_active":"2023-10-10T20:28:39.648228+0000","last_peered":"2023-10-10T20:28:39.648228+0000","last_clean":"2023-10-10T20:28:39.648228+0000","last_became_active":"2023-10-10T20:28:39.647295+0000","last_became_peered":"2023-10-10T20:28:39.647295+0000","last_unstale":"2023-10-10T20:28:39.648228+0000","last_undegraded":"2023-10-10T20:28:39.648228+0000","last_fullsized":"2023-10-10T20:28:39.648228+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:32.156438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639886+0000","last_change":"2023-10-10T20:28:38.624641+0000","last_active":"2023-10-10T20:28:39.639886+0000","last_peered":"2023-10-10T20:28:39.639886+0000","last_clean":"2023-10-10T20:28:39.639886+0000","last_became_active":"2023-10-10T20:28:38.624386+0000","last_became_peered":"2023-10-10T20:28:38.624386+0000","last_unstale":"2023-10-10T20:28:39.639886+0000","last_undegraded":"2023-10-10T20:28:39.639886+0000","last_fullsized":"2023-10-10T20:28:39.639886+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:23:02.137460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642961+0000","last_change":"2023-10-10T20:28:39.642668+0000","last_active":"2023-10-10T20:28:39.642961+0000","last_peered":"2023-10-10T20:28:39.642961+0000","last_clean":"2023-10-10T20:28:39.642961+0000","last_became_active":"2023-10-10T20:28:39.642223+0000","last_became_peered":"2023-10-10T20:28:39.642223+0000","last_unstale":"2023-10-10T20:28:39.642961+0000","last_undegraded":"2023-10-10T20:28:39.642961+0000","last_fullsized":"2023-10-10T20:28:39.642961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:47.441086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"161'8","reported_seq":18,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659687+0000","last_change":"2023-10-10T20:28:38.629479+0000","last_active":"2023-10-10T20:28:39.659687+0000","last_peered":"2023-10-10T20:28:39.659687+0000","last_clean":"2023-10-10T20:28:39.659687+0000","last_became_active":"2023-10-10T20:28:38.629244+0000","last_became_peered":"2023-10-10T20:28:38.629244+0000","last_unstale":"2023-10-10T20:28:39.659687+0000","last_undegraded":"2023-10-10T20:28:39.659687+0000","last_fullsized":"2023-10-10T20:28:39.659687+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.329793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649417+0000","last_change":"2023-10-10T20:28:39.649358+0000","last_active":"2023-10-10T20:28:39.649417+0000","last_peered":"2023-10-10T20:28:39.649417+0000","last_clean":"2023-10-10T20:28:39.649417+0000","last_became_active":"2023-10-10T20:28:39.649098+0000","last_became_peered":"2023-10-10T20:28:39.649098+0000","last_unstale":"2023-10-10T20:28:39.649417+0000","last_undegraded":"2023-10-10T20:28:39.649417+0000","last_fullsized":"2023-10-10T20:28:39.649417+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:46:40.931252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656063+0000","last_change":"2023-10-10T20:28:38.626852+0000","last_active":"2023-10-10T20:28:39.656063+0000","last_peered":"2023-10-10T20:28:39.656063+0000","last_clean":"2023-10-10T20:28:39.656063+0000","last_became_active":"2023-10-10T20:28:38.626628+0000","last_became_peered":"2023-10-10T20:28:38.626628+0000","last_unstale":"2023-10-10T20:28:39.656063+0000","last_undegraded":"2023-10-10T20:28:39.656063+0000","last_fullsized":"2023-10-10T20:28:39.656063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:10.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646883+0000","last_change":"2023-10-10T20:28:39.646831+0000","last_active":"2023-10-10T20:28:39.646883+0000","last_peered":"2023-10-10T20:28:39.646883+0000","last_clean":"2023-10-10T20:28:39.646883+0000","last_became_active":"2023-10-10T20:28:39.646454+0000","last_became_peered":"2023-10-10T20:28:39.646454+0000","last_unstale":"2023-10-10T20:28:39.646883+0000","last_undegraded":"2023-10-10T20:28:39.646883+0000","last_fullsized":"2023-10-10T20:28:39.646883+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:46.519220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656979+0000","last_change":"2023-10-10T20:28:38.629927+0000","last_active":"2023-10-10T20:28:39.656979+0000","last_peered":"2023-10-10T20:28:39.656979+0000","last_clean":"2023-10-10T20:28:39.656979+0000","last_became_active":"2023-10-10T20:28:38.629351+0000","last_became_peered":"2023-10-10T20:28:38.629351+0000","last_unstale":"2023-10-10T20:28:39.656979+0000","last_undegraded":"2023-10-10T20:28:39.656979+0000","last_fullsized":"2023-10-10T20:28:39.656979+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:02.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643959+0000","last_change":"2023-10-10T20:28:39.643664+0000","last_active":"2023-10-10T20:28:39.643959+0000","last_peered":"2023-10-10T20:28:39.643959+0000","last_clean":"2023-10-10T20:28:39.643959+0000","last_became_active":"2023-10-10T20:28:39.643048+0000","last_became_peered":"2023-10-10T20:28:39.643048+0000","last_unstale":"2023-10-10T20:28:39.643959+0000","last_undegraded":"2023-10-10T20:28:39.643959+0000","last_fullsized":"2023-10-10T20:28:39.643959+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:54.819407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657095+0000","last_change":"2023-10-10T20:28:38.626323+0000","last_active":"2023-10-10T20:28:39.657095+0000","last_peered":"2023-10-10T20:28:39.657095+0000","last_clean":"2023-10-10T20:28:39.657095+0000","last_became_active":"2023-10-10T20:28:38.626108+0000","last_became_peered":"2023-10-10T20:28:38.626108+0000","last_unstale":"2023-10-10T20:28:39.657095+0000","last_undegraded":"2023-10-10T20:28:39.657095+0000","last_fullsized":"2023-10-10T20:28:39.657095+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:00:47.967475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646346+0000","last_change":"2023-10-10T20:28:39.645874+0000","last_active":"2023-10-10T20:28:39.646346+0000","last_peered":"2023-10-10T20:28:39.646346+0000","last_clean":"2023-10-10T20:28:39.646346+0000","last_became_active":"2023-10-10T20:28:39.644927+0000","last_became_peered":"2023-10-10T20:28:39.644927+0000","last_unstale":"2023-10-10T20:28:39.646346+0000","last_undegraded":"2023-10-10T20:28:39.646346+0000","last_fullsized":"2023-10-10T20:28:39.646346+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633752+0000","last_change":"2023-10-10T20:28:38.628197+0000","last_active":"2023-10-10T20:28:39.633752+0000","last_peered":"2023-10-10T20:28:39.633752+0000","last_clean":"2023-10-10T20:28:39.633752+0000","last_became_active":"2023-10-10T20:28:38.627992+0000","last_became_peered":"2023-10-10T20:28:38.627992+0000","last_unstale":"2023-10-10T20:28:39.633752+0000","last_undegraded":"2023-10-10T20:28:39.633752+0000","last_fullsized":"2023-10-10T20:28:39.633752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:09.183516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643529+0000","last_change":"2023-10-10T20:28:39.643235+0000","last_active":"2023-10-10T20:28:39.643529+0000","last_peered":"2023-10-10T20:28:39.643529+0000","last_clean":"2023-10-10T20:28:39.643529+0000","last_became_active":"2023-10-10T20:28:39.642415+0000","last_became_peered":"2023-10-10T20:28:39.642415+0000","last_unstale":"2023-10-10T20:28:39.643529+0000","last_undegraded":"2023-10-10T20:28:39.643529+0000","last_fullsized":"2023-10-10T20:28:39.643529+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:45.995788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657550+0000","last_change":"2023-10-10T20:28:38.628787+0000","last_active":"2023-10-10T20:28:39.657550+0000","last_peered":"2023-10-10T20:28:39.657550+0000","last_clean":"2023-10-10T20:28:39.657550+0000","last_became_active":"2023-10-10T20:28:38.628570+0000","last_became_peered":"2023-10-10T20:28:38.628570+0000","last_unstale":"2023-10-10T20:28:39.657550+0000","last_undegraded":"2023-10-10T20:28:39.657550+0000","last_fullsized":"2023-10-10T20:28:39.657550+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:33.070338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645963+0000","last_change":"2023-10-10T20:28:39.645888+0000","last_active":"2023-10-10T20:28:39.645963+0000","last_peered":"2023-10-10T20:28:39.645963+0000","last_clean":"2023-10-10T20:28:39.645963+0000","last_became_active":"2023-10-10T20:28:39.645582+0000","last_became_peered":"2023-10-10T20:28:39.645582+0000","last_unstale":"2023-10-10T20:28:39.645963+0000","last_undegraded":"2023-10-10T20:28:39.645963+0000","last_fullsized":"2023-10-10T20:28:39.645963+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:58.435248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641599+0000","last_change":"2023-10-10T20:28:38.632175+0000","last_active":"2023-10-10T20:28:39.641599+0000","last_peered":"2023-10-10T20:28:39.641599+0000","last_clean":"2023-10-10T20:28:39.641599+0000","last_became_active":"2023-10-10T20:28:38.631956+0000","last_became_peered":"2023-10-10T20:28:38.631956+0000","last_unstale":"2023-10-10T20:28:39.641599+0000","last_undegraded":"2023-10-10T20:28:39.641599+0000","last_fullsized":"2023-10-10T20:28:39.641599+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:02.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644036+0000","last_change":"2023-10-10T20:28:39.643988+0000","last_active":"2023-10-10T20:28:39.644036+0000","last_peered":"2023-10-10T20:28:39.644036+0000","last_clean":"2023-10-10T20:28:39.644036+0000","last_became_active":"2023-10-10T20:28:39.643839+0000","last_became_peered":"2023-10-10T20:28:39.643839+0000","last_unstale":"2023-10-10T20:28:39.644036+0000","last_undegraded":"2023-10-10T20:28:39.644036+0000","last_fullsized":"2023-10-10T20:28:39.644036+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647261+0000","last_change":"2023-10-10T20:28:38.631602+0000","last_active":"2023-10-10T20:28:39.647261+0000","last_peered":"2023-10-10T20:28:39.647261+0000","last_clean":"2023-10-10T20:28:39.647261+0000","last_became_active":"2023-10-10T20:28:38.631357+0000","last_became_peered":"2023-10-10T20:28:38.631357+0000","last_unstale":"2023-10-10T20:28:39.647261+0000","last_undegraded":"2023-10-10T20:28:39.647261+0000","last_fullsized":"2023-10-10T20:28:39.647261+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:04.704041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646154+0000","last_change":"2023-10-10T20:28:39.646054+0000","last_active":"2023-10-10T20:28:39.646154+0000","last_peered":"2023-10-10T20:28:39.646154+0000","last_clean":"2023-10-10T20:28:39.646154+0000","last_became_active":"2023-10-10T20:28:39.645857+0000","last_became_peered":"2023-10-10T20:28:39.645857+0000","last_unstale":"2023-10-10T20:28:39.646154+0000","last_undegraded":"2023-10-10T20:28:39.646154+0000","last_fullsized":"2023-10-10T20:28:39.646154+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:07.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639379+0000","last_change":"2023-10-10T20:28:38.628342+0000","last_active":"2023-10-10T20:28:39.639379+0000","last_peered":"2023-10-10T20:28:39.639379+0000","last_clean":"2023-10-10T20:28:39.639379+0000","last_became_active":"2023-10-10T20:28:38.628099+0000","last_became_peered":"2023-10-10T20:28:38.628099+0000","last_unstale":"2023-10-10T20:28:39.639379+0000","last_undegraded":"2023-10-10T20:28:39.639379+0000","last_fullsized":"2023-10-10T20:28:39.639379+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:27.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648196+0000","last_change":"2023-10-10T20:28:39.647838+0000","last_active":"2023-10-10T20:28:39.648196+0000","last_peered":"2023-10-10T20:28:39.648196+0000","last_clean":"2023-10-10T20:28:39.648196+0000","last_became_active":"2023-10-10T20:28:39.646971+0000","last_became_peered":"2023-10-10T20:28:39.646971+0000","last_unstale":"2023-10-10T20:28:39.648196+0000","last_undegraded":"2023-10-10T20:28:39.648196+0000","last_fullsized":"2023-10-10T20:28:39.648196+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:48.738597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657158+0000","last_change":"2023-10-10T20:28:38.630237+0000","last_active":"2023-10-10T20:28:39.657158+0000","last_peered":"2023-10-10T20:28:39.657158+0000","last_clean":"2023-10-10T20:28:39.657158+0000","last_became_active":"2023-10-10T20:28:38.629683+0000","last_became_peered":"2023-10-10T20:28:38.629683+0000","last_unstale":"2023-10-10T20:28:39.657158+0000","last_undegraded":"2023-10-10T20:28:39.657158+0000","last_fullsized":"2023-10-10T20:28:39.657158+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:21.398248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643980+0000","last_change":"2023-10-10T20:28:39.643807+0000","last_active":"2023-10-10T20:28:39.643980+0000","last_peered":"2023-10-10T20:28:39.643980+0000","last_clean":"2023-10-10T20:28:39.643980+0000","last_became_active":"2023-10-10T20:28:39.643336+0000","last_became_peered":"2023-10-10T20:28:39.643336+0000","last_unstale":"2023-10-10T20:28:39.643980+0000","last_undegraded":"2023-10-10T20:28:39.643980+0000","last_fullsized":"2023-10-10T20:28:39.643980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:57.093229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639719+0000","last_change":"2023-10-10T20:28:38.626780+0000","last_active":"2023-10-10T20:28:39.639719+0000","last_peered":"2023-10-10T20:28:39.639719+0000","last_clean":"2023-10-10T20:28:39.639719+0000","last_became_active":"2023-10-10T20:28:38.626562+0000","last_became_peered":"2023-10-10T20:28:38.626562+0000","last_unstale":"2023-10-10T20:28:39.639719+0000","last_undegraded":"2023-10-10T20:28:39.639719+0000","last_fullsized":"2023-10-10T20:28:39.639719+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:33.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642629+0000","last_change":"2023-10-10T20:28:39.642563+0000","last_active":"2023-10-10T20:28:39.642629+0000","last_peered":"2023-10-10T20:28:39.642629+0000","last_clean":"2023-10-10T20:28:39.642629+0000","last_became_active":"2023-10-10T20:28:39.641625+0000","last_became_peered":"2023-10-10T20:28:39.641625+0000","last_unstale":"2023-10-10T20:28:39.642629+0000","last_undegraded":"2023-10-10T20:28:39.642629+0000","last_fullsized":"2023-10-10T20:28:39.642629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:44.127672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640509+0000","last_change":"2023-10-10T20:28:38.628816+0000","last_active":"2023-10-10T20:28:39.640509+0000","last_peered":"2023-10-10T20:28:39.640509+0000","last_clean":"2023-10-10T20:28:39.640509+0000","last_became_active":"2023-10-10T20:28:38.628606+0000","last_became_peered":"2023-10-10T20:28:38.628606+0000","last_unstale":"2023-10-10T20:28:39.640509+0000","last_undegraded":"2023-10-10T20:28:39.640509+0000","last_fullsized":"2023-10-10T20:28:39.640509+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:53.306026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641163+0000","last_change":"2023-10-10T20:28:39.641104+0000","last_active":"2023-10-10T20:28:39.641163+0000","last_peered":"2023-10-10T20:28:39.641163+0000","last_clean":"2023-10-10T20:28:39.641163+0000","last_became_active":"2023-10-10T20:28:39.640900+0000","last_became_peered":"2023-10-10T20:28:39.640900+0000","last_unstale":"2023-10-10T20:28:39.641163+0000","last_undegraded":"2023-10-10T20:28:39.641163+0000","last_fullsized":"2023-10-10T20:28:39.641163+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:25.416937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639939+0000","last_change":"2023-10-10T20:28:38.626629+0000","last_active":"2023-10-10T20:28:39.639939+0000","last_peered":"2023-10-10T20:28:39.639939+0000","last_clean":"2023-10-10T20:28:39.639939+0000","last_became_active":"2023-10-10T20:28:38.626382+0000","last_became_peered":"2023-10-10T20:28:38.626382+0000","last_unstale":"2023-10-10T20:28:39.639939+0000","last_undegraded":"2023-10-10T20:28:39.639939+0000","last_fullsized":"2023-10-10T20:28:39.639939+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:24.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643379+0000","last_change":"2023-10-10T20:28:39.643321+0000","last_active":"2023-10-10T20:28:39.643379+0000","last_peered":"2023-10-10T20:28:39.643379+0000","last_clean":"2023-10-10T20:28:39.643379+0000","last_became_active":"2023-10-10T20:28:39.643004+0000","last_became_peered":"2023-10-10T20:28:39.643004+0000","last_unstale":"2023-10-10T20:28:39.643379+0000","last_undegraded":"2023-10-10T20:28:39.643379+0000","last_fullsized":"2023-10-10T20:28:39.643379+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:00.521089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.638596+0000","last_change":"2023-10-10T20:28:38.624362+0000","last_active":"2023-10-10T20:28:39.638596+0000","last_peered":"2023-10-10T20:28:39.638596+0000","last_clean":"2023-10-10T20:28:39.638596+0000","last_became_active":"2023-10-10T20:28:38.624091+0000","last_became_peered":"2023-10-10T20:28:38.624091+0000","last_unstale":"2023-10-10T20:28:39.638596+0000","last_undegraded":"2023-10-10T20:28:39.638596+0000","last_fullsized":"2023-10-10T20:28:39.638596+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:00.338439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642919+0000","last_change":"2023-10-10T20:28:39.642751+0000","last_active":"2023-10-10T20:28:39.642919+0000","last_peered":"2023-10-10T20:28:39.642919+0000","last_clean":"2023-10-10T20:28:39.642919+0000","last_became_active":"2023-10-10T20:28:39.642315+0000","last_became_peered":"2023-10-10T20:28:39.642315+0000","last_unstale":"2023-10-10T20:28:39.642919+0000","last_undegraded":"2023-10-10T20:28:39.642919+0000","last_fullsized":"2023-10-10T20:28:39.642919+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:13.161511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640732+0000","last_change":"2023-10-10T20:28:38.630477+0000","last_active":"2023-10-10T20:28:39.640732+0000","last_peered":"2023-10-10T20:28:39.640732+0000","last_clean":"2023-10-10T20:28:39.640732+0000","last_became_active":"2023-10-10T20:28:38.629967+0000","last_became_peered":"2023-10-10T20:28:38.629967+0000","last_unstale":"2023-10-10T20:28:39.640732+0000","last_undegraded":"2023-10-10T20:28:39.640732+0000","last_fullsized":"2023-10-10T20:28:39.640732+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:33:47.542816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648604+0000","last_change":"2023-10-10T20:28:39.648398+0000","last_active":"2023-10-10T20:28:39.648604+0000","last_peered":"2023-10-10T20:28:39.648604+0000","last_clean":"2023-10-10T20:28:39.648604+0000","last_became_active":"2023-10-10T20:28:39.647867+0000","last_became_peered":"2023-10-10T20:28:39.647867+0000","last_unstale":"2023-10-10T20:28:39.648604+0000","last_undegraded":"2023-10-10T20:28:39.648604+0000","last_fullsized":"2023-10-10T20:28:39.648604+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:57.498975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639932+0000","last_change":"2023-10-10T20:28:38.628713+0000","last_active":"2023-10-10T20:28:39.639932+0000","last_peered":"2023-10-10T20:28:39.639932+0000","last_clean":"2023-10-10T20:28:39.639932+0000","last_became_active":"2023-10-10T20:28:38.627675+0000","last_became_peered":"2023-10-10T20:28:38.627675+0000","last_unstale":"2023-10-10T20:28:39.639932+0000","last_undegraded":"2023-10-10T20:28:39.639932+0000","last_fullsized":"2023-10-10T20:28:39.639932+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:24.413165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643557+0000","last_change":"2023-10-10T20:28:39.643421+0000","last_active":"2023-10-10T20:28:39.643557+0000","last_peered":"2023-10-10T20:28:39.643557+0000","last_clean":"2023-10-10T20:28:39.643557+0000","last_became_active":"2023-10-10T20:28:39.642543+0000","last_became_peered":"2023-10-10T20:28:39.642543+0000","last_unstale":"2023-10-10T20:28:39.643557+0000","last_undegraded":"2023-10-10T20:28:39.643557+0000","last_fullsized":"2023-10-10T20:28:39.643557+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:47:19.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633756+0000","last_change":"2023-10-10T20:28:38.627364+0000","last_active":"2023-10-10T20:28:39.633756+0000","last_peered":"2023-10-10T20:28:39.633756+0000","last_clean":"2023-10-10T20:28:39.633756+0000","last_became_active":"2023-10-10T20:28:38.627054+0000","last_became_peered":"2023-10-10T20:28:38.627054+0000","last_unstale":"2023-10-10T20:28:39.633756+0000","last_undegraded":"2023-10-10T20:28:39.633756+0000","last_fullsized":"2023-10-10T20:28:39.633756+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:04.704041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645801+0000","last_change":"2023-10-10T20:28:39.645736+0000","last_active":"2023-10-10T20:28:39.645801+0000","last_peered":"2023-10-10T20:28:39.645801+0000","last_clean":"2023-10-10T20:28:39.645801+0000","last_became_active":"2023-10-10T20:28:39.645509+0000","last_became_peered":"2023-10-10T20:28:39.645509+0000","last_unstale":"2023-10-10T20:28:39.645801+0000","last_undegraded":"2023-10-10T20:28:39.645801+0000","last_fullsized":"2023-10-10T20:28:39.645801+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:22.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"161'3","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658225+0000","last_change":"2023-10-10T20:28:38.629145+0000","last_active":"2023-10-10T20:28:39.658225+0000","last_peered":"2023-10-10T20:28:39.658225+0000","last_clean":"2023-10-10T20:28:39.658225+0000","last_became_active":"2023-10-10T20:28:38.628371+0000","last_became_peered":"2023-10-10T20:28:38.628371+0000","last_unstale":"2023-10-10T20:28:39.658225+0000","last_undegraded":"2023-10-10T20:28:39.658225+0000","last_fullsized":"2023-10-10T20:28:39.658225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:13:15.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":288,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641690+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:28:39.641690+0000","last_peered":"2023-10-10T20:28:39.641690+0000","last_clean":"2023-10-10T20:28:39.641690+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:28:39.641690+0000","last_undegraded":"2023-10-10T20:28:39.641690+0000","last_fullsized":"2023-10-10T20:28:39.641690+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193856,"statfs":{"total":200000000,"available":198509055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53100000000000003,"15min":0.49399999999999999},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.76600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.39900000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.754,"15min":0.754},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.505},"min":{"1min":0.45100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.76500000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72099999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.54200000000000004},"min":{"1min":0.48399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69399999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.73799999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.75900000000000001,"15min":0.79600000000000004},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.32400000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.503},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.376,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":0.746,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47799999999999998,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.69699999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.33100000000000002},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.316,"15min":0.29599999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":0.80800000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099901,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.72199999999999998,"5min":0.877,"15min":1.3140000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.27000000000000002},"max":{"1min":0.60299999999999998,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64000000000000001,"5min":0.92100000000000004,"15min":1.865},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":4.7709999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.497},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":0.65900000000000003,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":15.862},"last":0.42299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.442},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.222},"max":{"1min":0.75,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.76000000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.66000000000000003,"5min":0.82699999999999996,"15min":1.0840000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":1.02,"15min":1.0409999999999999},"last":0.33300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72999999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.93200000000000005,"15min":0.97199999999999998},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099493,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55000000000000004,"5min":0.56699999999999995,"15min":1.647},"last":0.27300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53100000000000003,"5min":0.627,"15min":1.524},"last":0.316},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.55000000000000004,"5min":0.64100000000000001,"15min":1.4470000000000001},"last":0.377},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.54100000000000004,"5min":0.627,"15min":1.673},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":1.633},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.5,"5min":0.36399999999999999,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88300000000000001,"15min":1.506},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.696},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.64700000000000002},"min":{"1min":0.45200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84999999999999998,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.629,"15min":0.629},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.877,"15min":1.7490000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.53900000000000003,"5min":0.45200000000000001,"15min":0.39400000000000002},"max":{"1min":0.88300000000000001,"5min":0.98899999999999999,"15min":1.585},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.478},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.624,"15min":0.629},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93500000000000005,"15min":1.722},"last":0.497}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1455,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193856,"statfs":{"total":200000000,"available":198509120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.41699999999999998},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.505,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.44400000000000001,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53800000000000003,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.44400000000000001},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":1.647,"15min":2.4830000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.42099999999999999,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.50700000000000001,"5min":5.891,"15min":5.891},"last":0.255}]},{"osd":3,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.47299999999999998,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.313},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.56499999999999995,"5min":1.462,"15min":1.462},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65300000000000002,"15min":0.624},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.82999999999999996},"min":{"1min":0.443,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84999999999999998,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.60599999999999998},"min":{"1min":0.47599999999999998,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64900000000000002,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":5.758,"15min":5.758},"last":0.504},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.40000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.375},"min":{"1min":0.23799999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":2.4409999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":0.53600000000000003,"5min":0.56399999999999995,"15min":8.2629999999999999},"last":0.379}]},{"osd":1,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.223,"15min":0.22},"max":{"1min":0.48499999999999999,"5min":0.60399999999999998,"15min":6.5229999999999997},"last":0.308},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.44},"min":{"1min":0.314,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":9.2949999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.44600000000000001},"min":{"1min":0.29499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":11.815},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.45900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":18.399999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.46400000000000002,"5min":0.376,"15min":0.28100000000000003},"max":{"1min":0.85699999999999998,"5min":1.004,"15min":2.1389999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.48799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89800000000000002,"5min":0.995,"15min":9.3740000000000006},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":6.399},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":1.0429999999999999,"15min":2.1040000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":1.0189999999999999,"15min":9.4019999999999992},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.98399999999999999,"15min":6.4969999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.33200000000000002},"max":{"1min":0.79300000000000004,"5min":0.92500000000000004,"15min":9.3450000000000006},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0309999999999999,"15min":6.4630000000000001},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.31},"min":{"1min":0.252,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.22},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.32500000000000001},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.21299999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.65600000000000003},"last":0.27700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.371,"15min":0.34599999999999997},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.255},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"min":{"1min":0.34200000000000003,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.751},"last":0.307}]},{"osd":2,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.32300000000000001,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.54600000000000004,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.437,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.55400000000000005},"min":{"1min":0.52800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.745,"5min":0.875,"15min":0.875},"last":0.435}]},{"osd":5,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38200000000000001},"max":{"1min":0.76000000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.55600000000000005},"min":{"1min":0.46400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88800000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.57499999999999996},"min":{"1min":0.52900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.871,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.82999999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.58799999999999997},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199100047,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.50900000000000001},"min":{"1min":0.34200000000000003,"5min":0.22700000000000001,"15min":0.22600000000000001},"max":{"1min":0.628,"5min":0.76400000000000001,"15min":0.85399999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.318,"15min":0.29999999999999999},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.41199999999999998,"5min":0.252,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.52600000000000002},"min":{"1min":0.42299999999999999,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.46000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.51200000000000001},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68799999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.629},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26100000000000001},"max":{"1min":0.66500000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53300000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.85399999999999998,"15min":0.88700000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199100025,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.78500000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.77200000000000002,"15min":0.91200000000000003},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.40999999999999998,"5min":0.311,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.53500000000000003},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.46000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.48899999999999999},"min":{"1min":0.28799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58399999999999996,"5min":0.76900000000000002,"15min":0.79900000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.80800000000000005,"15min":0.875},"last":0.33300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.752,"15min":0.78200000000000003},"last":0.37}]},{"osd":7,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.34799999999999998},"max":{"1min":0.751,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.38300000000000001}]}]}],"pool_statfs":[]}} 2023-10-10T20:28:43.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:28:43.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.839+0000 7f4d0e3ef700 1 Processor -- start 2023-10-10T20:28:43.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.839+0000 7f4d0e3ef700 1 -- start start 2023-10-10T20:28:43.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0e3ef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 0x7f4d0815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0e3ef700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 0x7f4d0815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0e3ef700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0815cbb0 con 0x7f4d0815d580 2023-10-10T20:28:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0e3ef700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0815bfa0 con 0x7f4d0815a620 2023-10-10T20:28:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0e3ef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0815c280 con 0x7f4d08157870 2023-10-10T20:28:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0c98c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4d0815d580 0x7f4d08158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43482/0 (socket says 172.21.15.148:43482) 2023-10-10T20:28:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d0c98c700 1 -- 172.21.15.148:0/3153396174 learned_addr learned my addr 172.21.15.148:0/3153396174 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:43.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3153396174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 0x7f4d0815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d077fe700 1 --2- 172.21.15.148:0/3153396174 >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 0x7f4d0815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.840+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494941355 0 0) 0x7f4d0815cbb0 con 0x7f4d0815d580 2023-10-10T20:28:43.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf0003570 con 0x7f4d0815d580 2023-10-10T20:28:43.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d07fff700 1 -- 172.21.15.148:0/3153396174 >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 msgr2=0x7f4d0815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:43.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3153396174 >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 0x7f4d0815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1311928069 0 0) 0x7f4cf0003570 con 0x7f4d0815d580 2023-10-10T20:28:43.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d07fff700 1 -- 172.21.15.148:0/3153396174 >> v1:172.21.15.148:6789/0 conn(0x7f4d0815d580 legacy=0x7f4d08158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d07fff700 1 -- 172.21.15.148:0/3153396174 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0815c560 con 0x7f4d08157870 2023-10-10T20:28:43.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d077fe700 1 --2- 172.21.15.148:0/3153396174 >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 0x7f4d0815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:43.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3153396174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 0x7f4d0815b6a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80098e0 tx=0x7f4cf80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9234363b7ada8136 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000017f0 con 0x7f4d08157870 2023-10-10T20:28:43.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf8011420 con 0x7f4d08157870 2023-10-10T20:28:43.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.841+0000 7f4d06ffd700 1 -- 172.21.15.148:0/3153396174 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80117a0 con 0x7f4d08157870 2023-10-10T20:28:43.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3153396174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 msgr2=0x7f4d0815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3153396174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 0x7f4d0815b6a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80098e0 tx=0x7f4cf80091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3153396174 shutdown_connections 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3153396174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d08157870 0x7f4d0815b6a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3153396174 >> v2:172.21.15.161:3300/0 conn(0x7f4d0815a620 0x7f4d0815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3153396174 >> 172.21.15.148:0/3153396174 conn(0x7f4d08068ce0 msgr2=0x7f4d080690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:43.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3153396174 shutdown_connections 2023-10-10T20:28:43.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.842+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3153396174 wait complete. 2023-10-10T20:28:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.843+0000 7f4d0e3ef700 1 Processor -- start 2023-10-10T20:28:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.843+0000 7f4d0e3ef700 1 -- start start 2023-10-10T20:28:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d0815a620 0x7f4d080b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d0815d580 0x7f4d0816a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0816b250 con 0x7f4d08157870 2023-10-10T20:28:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0816b530 con 0x7f4d0815d580 2023-10-10T20:28:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d080b07b0 con 0x7f4d0815a620 2023-10-10T20:28:43.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58540/0 (socket says 172.21.15.148:58540) 2023-10-10T20:28:43.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 -- 172.21.15.148:0/3286129152 learned_addr learned my addr 172.21.15.148:0/3286129152 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:43.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d077fe700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d0815a620 0x7f4d080b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0c98c700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d0815d580 0x7f4d0816a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 -- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d0815a620 msgr2=0x7f4d080b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d0815a620 0x7f4d080b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 -- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d0815d580 msgr2=0x7f4d0816a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d0815d580 0x7f4d0816a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:43.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 -- 172.21.15.148:0/3286129152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009420 con 0x7f4d08157870 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d07fff700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4d0815c3c0 tx=0x7f4cf8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801cba0 con 0x7f4d08157870 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf801cd00 con 0x7f4d08157870 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf8009420 con 0x7f4d08157870 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801a6a0 con 0x7f4d08157870 2023-10-10T20:28:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.844+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0816bea0 con 0x7f4d08157870 2023-10-10T20:28:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.845+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d080b0610 con 0x7f4d08157870 2023-10-10T20:28:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.846+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cf801aa50 con 0x7f4d08157870 2023-10-10T20:28:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.846+0000 7f4ceffff700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.846+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4cf8085f10 con 0x7f4d08157870 2023-10-10T20:28:43.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.846+0000 7f4d077fe700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:43.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.847+0000 7f4d077fe700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 0x7f4cf40693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0093f0 tx=0x7f4cfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:43.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:43.850+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d080b0610 con 0x7f4d08157870 2023-10-10T20:28:44.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.027+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d080b0610 con 0x7f4cf4066f30 2023-10-10T20:28:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.035+0000 7f4ceffff700 1 -- 172.21.15.148:0/3286129152 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285588 (secure 0 0 0) 0x7f4d080b0610 con 0x7f4cf4066f30 2023-10-10T20:28:44.041 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:44.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 msgr2=0x7f4cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 0x7f4cf40693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0093f0 tx=0x7f4cfc008040 comp rx=0 tx=0).stop 2023-10-10T20:28:44.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 msgr2=0x7f4d0816b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4d0815c3c0 tx=0x7f4cf8016000 comp rx=0 tx=0).stop 2023-10-10T20:28:44.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 shutdown_connections 2023-10-10T20:28:44.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4cf4066f30 0x7f4cf40693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d0815d580 0x7f4d0816a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d08157870 0x7f4d0816b960 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 --2- 172.21.15.148:0/3286129152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4d0815a620 0x7f4d080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 >> 172.21.15.148:0/3286129152 conn(0x7f4d08068ce0 msgr2=0x7f4d0815a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 shutdown_connections 2023-10-10T20:28:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.038+0000 7f4d0e3ef700 1 -- 172.21.15.148:0/3286129152 wait complete. 2023-10-10T20:28:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:28:44.052 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2023-10-10T20:28:43.448171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554313,"statfs":{"total":1600000000,"available":1591617050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.832740"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649121+0000","last_change":"2023-10-10T20:28:39.649081+0000","last_active":"2023-10-10T20:28:39.649121+0000","last_peered":"2023-10-10T20:28:39.649121+0000","last_clean":"2023-10-10T20:28:39.649121+0000","last_became_active":"2023-10-10T20:28:39.648924+0000","last_became_peered":"2023-10-10T20:28:39.648924+0000","last_unstale":"2023-10-10T20:28:39.649121+0000","last_undegraded":"2023-10-10T20:28:39.649121+0000","last_fullsized":"2023-10-10T20:28:39.649121+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:54.819407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642237+0000","last_change":"2023-10-10T20:28:39.642136+0000","last_active":"2023-10-10T20:28:39.642237+0000","last_peered":"2023-10-10T20:28:39.642237+0000","last_clean":"2023-10-10T20:28:39.642237+0000","last_became_active":"2023-10-10T20:28:39.641913+0000","last_became_peered":"2023-10-10T20:28:39.641913+0000","last_unstale":"2023-10-10T20:28:39.642237+0000","last_undegraded":"2023-10-10T20:28:39.642237+0000","last_fullsized":"2023-10-10T20:28:39.642237+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:52.174175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646980+0000","last_change":"2023-10-10T20:28:39.646698+0000","last_active":"2023-10-10T20:28:39.646980+0000","last_peered":"2023-10-10T20:28:39.646980+0000","last_clean":"2023-10-10T20:28:39.646980+0000","last_became_active":"2023-10-10T20:28:39.646085+0000","last_became_peered":"2023-10-10T20:28:39.646085+0000","last_unstale":"2023-10-10T20:28:39.646980+0000","last_undegraded":"2023-10-10T20:28:39.646980+0000","last_fullsized":"2023-10-10T20:28:39.646980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:03.432687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639363+0000","last_change":"2023-10-10T20:28:39.639297+0000","last_active":"2023-10-10T20:28:39.639363+0000","last_peered":"2023-10-10T20:28:39.639363+0000","last_clean":"2023-10-10T20:28:39.639363+0000","last_became_active":"2023-10-10T20:28:39.639073+0000","last_became_peered":"2023-10-10T20:28:39.639073+0000","last_unstale":"2023-10-10T20:28:39.639363+0000","last_undegraded":"2023-10-10T20:28:39.639363+0000","last_fullsized":"2023-10-10T20:28:39.639363+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:30:03.120000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648937+0000","last_change":"2023-10-10T20:28:39.648870+0000","last_active":"2023-10-10T20:28:39.648937+0000","last_peered":"2023-10-10T20:28:39.648937+0000","last_clean":"2023-10-10T20:28:39.648937+0000","last_became_active":"2023-10-10T20:28:39.648634+0000","last_became_peered":"2023-10-10T20:28:39.648634+0000","last_unstale":"2023-10-10T20:28:39.648937+0000","last_undegraded":"2023-10-10T20:28:39.648937+0000","last_fullsized":"2023-10-10T20:28:39.648937+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:13:27.810043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646523+0000","last_change":"2023-10-10T20:28:39.646421+0000","last_active":"2023-10-10T20:28:39.646523+0000","last_peered":"2023-10-10T20:28:39.646523+0000","last_clean":"2023-10-10T20:28:39.646523+0000","last_became_active":"2023-10-10T20:28:39.645235+0000","last_became_peered":"2023-10-10T20:28:39.645235+0000","last_unstale":"2023-10-10T20:28:39.646523+0000","last_undegraded":"2023-10-10T20:28:39.646523+0000","last_fullsized":"2023-10-10T20:28:39.646523+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:57.093229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648527+0000","last_change":"2023-10-10T20:28:39.648485+0000","last_active":"2023-10-10T20:28:39.648527+0000","last_peered":"2023-10-10T20:28:39.648527+0000","last_clean":"2023-10-10T20:28:39.648527+0000","last_became_active":"2023-10-10T20:28:39.648327+0000","last_became_peered":"2023-10-10T20:28:39.648327+0000","last_unstale":"2023-10-10T20:28:39.648527+0000","last_undegraded":"2023-10-10T20:28:39.648527+0000","last_fullsized":"2023-10-10T20:28:39.648527+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:28.934427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644349+0000","last_change":"2023-10-10T20:28:39.643796+0000","last_active":"2023-10-10T20:28:39.644349+0000","last_peered":"2023-10-10T20:28:39.644349+0000","last_clean":"2023-10-10T20:28:39.644349+0000","last_became_active":"2023-10-10T20:28:39.643595+0000","last_became_peered":"2023-10-10T20:28:39.643595+0000","last_unstale":"2023-10-10T20:28:39.644349+0000","last_undegraded":"2023-10-10T20:28:39.644349+0000","last_fullsized":"2023-10-10T20:28:39.644349+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:26:01.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642254+0000","last_change":"2023-10-10T20:28:39.642203+0000","last_active":"2023-10-10T20:28:39.642254+0000","last_peered":"2023-10-10T20:28:39.642254+0000","last_clean":"2023-10-10T20:28:39.642254+0000","last_became_active":"2023-10-10T20:28:39.641635+0000","last_became_peered":"2023-10-10T20:28:39.641635+0000","last_unstale":"2023-10-10T20:28:39.642254+0000","last_undegraded":"2023-10-10T20:28:39.642254+0000","last_fullsized":"2023-10-10T20:28:39.642254+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:59:33.550658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.650997+0000","last_change":"2023-10-10T20:28:39.650435+0000","last_active":"2023-10-10T20:28:39.650997+0000","last_peered":"2023-10-10T20:28:39.650997+0000","last_clean":"2023-10-10T20:28:39.650997+0000","last_became_active":"2023-10-10T20:28:39.649737+0000","last_became_peered":"2023-10-10T20:28:39.649737+0000","last_unstale":"2023-10-10T20:28:39.650997+0000","last_undegraded":"2023-10-10T20:28:39.650997+0000","last_fullsized":"2023-10-10T20:28:39.650997+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:34:13.077873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645895+0000","last_change":"2023-10-10T20:28:39.645679+0000","last_active":"2023-10-10T20:28:39.645895+0000","last_peered":"2023-10-10T20:28:39.645895+0000","last_clean":"2023-10-10T20:28:39.645895+0000","last_became_active":"2023-10-10T20:28:39.644432+0000","last_became_peered":"2023-10-10T20:28:39.644432+0000","last_unstale":"2023-10-10T20:28:39.645895+0000","last_undegraded":"2023-10-10T20:28:39.645895+0000","last_fullsized":"2023-10-10T20:28:39.645895+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:23.036296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647348+0000","last_change":"2023-10-10T20:28:39.646815+0000","last_active":"2023-10-10T20:28:39.647348+0000","last_peered":"2023-10-10T20:28:39.647348+0000","last_clean":"2023-10-10T20:28:39.647348+0000","last_became_active":"2023-10-10T20:28:39.646425+0000","last_became_peered":"2023-10-10T20:28:39.646425+0000","last_unstale":"2023-10-10T20:28:39.647348+0000","last_undegraded":"2023-10-10T20:28:39.647348+0000","last_fullsized":"2023-10-10T20:28:39.647348+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:59:33.550658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648246+0000","last_change":"2023-10-10T20:28:39.648138+0000","last_active":"2023-10-10T20:28:39.648246+0000","last_peered":"2023-10-10T20:28:39.648246+0000","last_clean":"2023-10-10T20:28:39.648246+0000","last_became_active":"2023-10-10T20:28:39.647580+0000","last_became_peered":"2023-10-10T20:28:39.647580+0000","last_unstale":"2023-10-10T20:28:39.648246+0000","last_undegraded":"2023-10-10T20:28:39.648246+0000","last_fullsized":"2023-10-10T20:28:39.648246+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:58:39.589233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647325+0000","last_change":"2023-10-10T20:28:39.647234+0000","last_active":"2023-10-10T20:28:39.647325+0000","last_peered":"2023-10-10T20:28:39.647325+0000","last_clean":"2023-10-10T20:28:39.647325+0000","last_became_active":"2023-10-10T20:28:39.646984+0000","last_became_peered":"2023-10-10T20:28:39.646984+0000","last_unstale":"2023-10-10T20:28:39.647325+0000","last_undegraded":"2023-10-10T20:28:39.647325+0000","last_fullsized":"2023-10-10T20:28:39.647325+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:36:23.036296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649295+0000","last_change":"2023-10-10T20:28:39.649247+0000","last_active":"2023-10-10T20:28:39.649295+0000","last_peered":"2023-10-10T20:28:39.649295+0000","last_clean":"2023-10-10T20:28:39.649295+0000","last_became_active":"2023-10-10T20:28:39.648882+0000","last_became_peered":"2023-10-10T20:28:39.648882+0000","last_unstale":"2023-10-10T20:28:39.649295+0000","last_undegraded":"2023-10-10T20:28:39.649295+0000","last_fullsized":"2023-10-10T20:28:39.649295+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:07.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646935+0000","last_change":"2023-10-10T20:28:39.646877+0000","last_active":"2023-10-10T20:28:39.646935+0000","last_peered":"2023-10-10T20:28:39.646935+0000","last_clean":"2023-10-10T20:28:39.646935+0000","last_became_active":"2023-10-10T20:28:39.646673+0000","last_became_peered":"2023-10-10T20:28:39.646673+0000","last_unstale":"2023-10-10T20:28:39.646935+0000","last_undegraded":"2023-10-10T20:28:39.646935+0000","last_fullsized":"2023-10-10T20:28:39.646935+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:13.161511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647325+0000","last_change":"2023-10-10T20:28:39.646661+0000","last_active":"2023-10-10T20:28:39.647325+0000","last_peered":"2023-10-10T20:28:39.647325+0000","last_clean":"2023-10-10T20:28:39.647325+0000","last_became_active":"2023-10-10T20:28:39.646082+0000","last_became_peered":"2023-10-10T20:28:39.646082+0000","last_unstale":"2023-10-10T20:28:39.647325+0000","last_undegraded":"2023-10-10T20:28:39.647325+0000","last_fullsized":"2023-10-10T20:28:39.647325+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:22.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646961+0000","last_change":"2023-10-10T20:28:39.646571+0000","last_active":"2023-10-10T20:28:39.646961+0000","last_peered":"2023-10-10T20:28:39.646961+0000","last_clean":"2023-10-10T20:28:39.646961+0000","last_became_active":"2023-10-10T20:28:39.645796+0000","last_became_peered":"2023-10-10T20:28:39.645796+0000","last_unstale":"2023-10-10T20:28:39.646961+0000","last_undegraded":"2023-10-10T20:28:39.646961+0000","last_fullsized":"2023-10-10T20:28:39.646961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:05.707813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644391+0000","last_change":"2023-10-10T20:28:39.644336+0000","last_active":"2023-10-10T20:28:39.644391+0000","last_peered":"2023-10-10T20:28:39.644391+0000","last_clean":"2023-10-10T20:28:39.644391+0000","last_became_active":"2023-10-10T20:28:39.643959+0000","last_became_peered":"2023-10-10T20:28:39.643959+0000","last_unstale":"2023-10-10T20:28:39.644391+0000","last_undegraded":"2023-10-10T20:28:39.644391+0000","last_fullsized":"2023-10-10T20:28:39.644391+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:28.441866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643485+0000","last_change":"2023-10-10T20:28:39.643420+0000","last_active":"2023-10-10T20:28:39.643485+0000","last_peered":"2023-10-10T20:28:39.643485+0000","last_clean":"2023-10-10T20:28:39.643485+0000","last_became_active":"2023-10-10T20:28:39.643174+0000","last_became_peered":"2023-10-10T20:28:39.643174+0000","last_unstale":"2023-10-10T20:28:39.643485+0000","last_undegraded":"2023-10-10T20:28:39.643485+0000","last_fullsized":"2023-10-10T20:28:39.643485+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:49:11.227327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648585+0000","last_change":"2023-10-10T20:28:39.648232+0000","last_active":"2023-10-10T20:28:39.648585+0000","last_peered":"2023-10-10T20:28:39.648585+0000","last_clean":"2023-10-10T20:28:39.648585+0000","last_became_active":"2023-10-10T20:28:39.647720+0000","last_became_peered":"2023-10-10T20:28:39.647720+0000","last_unstale":"2023-10-10T20:28:39.648585+0000","last_undegraded":"2023-10-10T20:28:39.648585+0000","last_fullsized":"2023-10-10T20:28:39.648585+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:53.439232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.636844+0000","last_change":"2023-10-10T20:28:39.636787+0000","last_active":"2023-10-10T20:28:39.636844+0000","last_peered":"2023-10-10T20:28:39.636844+0000","last_clean":"2023-10-10T20:28:39.636844+0000","last_became_active":"2023-10-10T20:28:39.636537+0000","last_became_peered":"2023-10-10T20:28:39.636537+0000","last_unstale":"2023-10-10T20:28:39.636844+0000","last_undegraded":"2023-10-10T20:28:39.636844+0000","last_fullsized":"2023-10-10T20:28:39.636844+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:07:28.573704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644285+0000","last_change":"2023-10-10T20:28:39.644176+0000","last_active":"2023-10-10T20:28:39.644285+0000","last_peered":"2023-10-10T20:28:39.644285+0000","last_clean":"2023-10-10T20:28:39.644285+0000","last_became_active":"2023-10-10T20:28:39.643945+0000","last_became_peered":"2023-10-10T20:28:39.643945+0000","last_unstale":"2023-10-10T20:28:39.644285+0000","last_undegraded":"2023-10-10T20:28:39.644285+0000","last_fullsized":"2023-10-10T20:28:39.644285+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:16:56.119191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.650040+0000","last_change":"2023-10-10T20:28:39.649981+0000","last_active":"2023-10-10T20:28:39.650040+0000","last_peered":"2023-10-10T20:28:39.650040+0000","last_clean":"2023-10-10T20:28:39.650040+0000","last_became_active":"2023-10-10T20:28:39.649730+0000","last_became_peered":"2023-10-10T20:28:39.649730+0000","last_unstale":"2023-10-10T20:28:39.650040+0000","last_undegraded":"2023-10-10T20:28:39.650040+0000","last_fullsized":"2023-10-10T20:28:39.650040+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:28.934427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642690+0000","last_change":"2023-10-10T20:28:39.642518+0000","last_active":"2023-10-10T20:28:39.642690+0000","last_peered":"2023-10-10T20:28:39.642690+0000","last_clean":"2023-10-10T20:28:39.642690+0000","last_became_active":"2023-10-10T20:28:39.642257+0000","last_became_peered":"2023-10-10T20:28:39.642257+0000","last_unstale":"2023-10-10T20:28:39.642690+0000","last_undegraded":"2023-10-10T20:28:39.642690+0000","last_fullsized":"2023-10-10T20:28:39.642690+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:42:49.780770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644741+0000","last_change":"2023-10-10T20:28:39.644678+0000","last_active":"2023-10-10T20:28:39.644741+0000","last_peered":"2023-10-10T20:28:39.644741+0000","last_clean":"2023-10-10T20:28:39.644741+0000","last_became_active":"2023-10-10T20:28:39.643784+0000","last_became_peered":"2023-10-10T20:28:39.643784+0000","last_unstale":"2023-10-10T20:28:39.644741+0000","last_undegraded":"2023-10-10T20:28:39.644741+0000","last_fullsized":"2023-10-10T20:28:39.644741+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:05.707813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641768+0000","last_change":"2023-10-10T20:28:39.641709+0000","last_active":"2023-10-10T20:28:39.641768+0000","last_peered":"2023-10-10T20:28:39.641768+0000","last_clean":"2023-10-10T20:28:39.641768+0000","last_became_active":"2023-10-10T20:28:39.641514+0000","last_became_peered":"2023-10-10T20:28:39.641514+0000","last_unstale":"2023-10-10T20:28:39.641768+0000","last_undegraded":"2023-10-10T20:28:39.641768+0000","last_fullsized":"2023-10-10T20:28:39.641768+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:47:19.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642660+0000","last_change":"2023-10-10T20:28:39.642603+0000","last_active":"2023-10-10T20:28:39.642660+0000","last_peered":"2023-10-10T20:28:39.642660+0000","last_clean":"2023-10-10T20:28:39.642660+0000","last_became_active":"2023-10-10T20:28:39.642366+0000","last_became_peered":"2023-10-10T20:28:39.642366+0000","last_unstale":"2023-10-10T20:28:39.642660+0000","last_undegraded":"2023-10-10T20:28:39.642660+0000","last_fullsized":"2023-10-10T20:28:39.642660+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:57.498975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644799+0000","last_change":"2023-10-10T20:28:39.644754+0000","last_active":"2023-10-10T20:28:39.644799+0000","last_peered":"2023-10-10T20:28:39.644799+0000","last_clean":"2023-10-10T20:28:39.644799+0000","last_became_active":"2023-10-10T20:28:39.644559+0000","last_became_peered":"2023-10-10T20:28:39.644559+0000","last_unstale":"2023-10-10T20:28:39.644799+0000","last_undegraded":"2023-10-10T20:28:39.644799+0000","last_fullsized":"2023-10-10T20:28:39.644799+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:09.644327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641996+0000","last_change":"2023-10-10T20:28:39.641907+0000","last_active":"2023-10-10T20:28:39.641996+0000","last_peered":"2023-10-10T20:28:39.641996+0000","last_clean":"2023-10-10T20:28:39.641996+0000","last_became_active":"2023-10-10T20:28:39.641667+0000","last_became_peered":"2023-10-10T20:28:39.641667+0000","last_unstale":"2023-10-10T20:28:39.641996+0000","last_undegraded":"2023-10-10T20:28:39.641996+0000","last_fullsized":"2023-10-10T20:28:39.641996+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:45:09.644327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641872+0000","last_change":"2023-10-10T20:28:39.641811+0000","last_active":"2023-10-10T20:28:39.641872+0000","last_peered":"2023-10-10T20:28:39.641872+0000","last_clean":"2023-10-10T20:28:39.641872+0000","last_became_active":"2023-10-10T20:28:39.641434+0000","last_became_peered":"2023-10-10T20:28:39.641434+0000","last_unstale":"2023-10-10T20:28:39.641872+0000","last_undegraded":"2023-10-10T20:28:39.641872+0000","last_fullsized":"2023-10-10T20:28:39.641872+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:01.342211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642399+0000","last_change":"2023-10-10T20:28:39.642336+0000","last_active":"2023-10-10T20:28:39.642399+0000","last_peered":"2023-10-10T20:28:39.642399+0000","last_clean":"2023-10-10T20:28:39.642399+0000","last_became_active":"2023-10-10T20:28:39.642019+0000","last_became_peered":"2023-10-10T20:28:39.642019+0000","last_unstale":"2023-10-10T20:28:39.642399+0000","last_undegraded":"2023-10-10T20:28:39.642399+0000","last_fullsized":"2023-10-10T20:28:39.642399+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:48:41.771961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648620+0000","last_change":"2023-10-10T20:28:39.648497+0000","last_active":"2023-10-10T20:28:39.648620+0000","last_peered":"2023-10-10T20:28:39.648620+0000","last_clean":"2023-10-10T20:28:39.648620+0000","last_became_active":"2023-10-10T20:28:39.648140+0000","last_became_peered":"2023-10-10T20:28:39.648140+0000","last_unstale":"2023-10-10T20:28:39.648620+0000","last_undegraded":"2023-10-10T20:28:39.648620+0000","last_fullsized":"2023-10-10T20:28:39.648620+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:53:16.913282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"161'4","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657366+0000","last_change":"2023-10-10T20:28:38.628272+0000","last_active":"2023-10-10T20:28:39.657366+0000","last_peered":"2023-10-10T20:28:39.657366+0000","last_clean":"2023-10-10T20:28:39.657366+0000","last_became_active":"2023-10-10T20:28:38.627823+0000","last_became_peered":"2023-10-10T20:28:38.627823+0000","last_unstale":"2023-10-10T20:28:39.657366+0000","last_undegraded":"2023-10-10T20:28:39.657366+0000","last_fullsized":"2023-10-10T20:28:39.657366+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:09.183516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639932+0000","last_change":"2023-10-10T20:28:39.639709+0000","last_active":"2023-10-10T20:28:39.639932+0000","last_peered":"2023-10-10T20:28:39.639932+0000","last_clean":"2023-10-10T20:28:39.639932+0000","last_became_active":"2023-10-10T20:28:39.639512+0000","last_became_peered":"2023-10-10T20:28:39.639512+0000","last_unstale":"2023-10-10T20:28:39.639932+0000","last_undegraded":"2023-10-10T20:28:39.639932+0000","last_fullsized":"2023-10-10T20:28:39.639932+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:34.074110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657760+0000","last_change":"2023-10-10T20:28:38.627254+0000","last_active":"2023-10-10T20:28:39.657760+0000","last_peered":"2023-10-10T20:28:39.657760+0000","last_clean":"2023-10-10T20:28:39.657760+0000","last_became_active":"2023-10-10T20:28:38.626818+0000","last_became_peered":"2023-10-10T20:28:38.626818+0000","last_unstale":"2023-10-10T20:28:39.657760+0000","last_undegraded":"2023-10-10T20:28:39.657760+0000","last_fullsized":"2023-10-10T20:28:39.657760+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:46.437313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645539+0000","last_change":"2023-10-10T20:28:39.645476+0000","last_active":"2023-10-10T20:28:39.645539+0000","last_peered":"2023-10-10T20:28:39.645539+0000","last_clean":"2023-10-10T20:28:39.645539+0000","last_became_active":"2023-10-10T20:28:39.644121+0000","last_became_peered":"2023-10-10T20:28:39.644121+0000","last_unstale":"2023-10-10T20:28:39.645539+0000","last_undegraded":"2023-10-10T20:28:39.645539+0000","last_fullsized":"2023-10-10T20:28:39.645539+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:10.187288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658281+0000","last_change":"2023-10-10T20:28:38.626245+0000","last_active":"2023-10-10T20:28:39.658281+0000","last_peered":"2023-10-10T20:28:39.658281+0000","last_clean":"2023-10-10T20:28:39.658281+0000","last_became_active":"2023-10-10T20:28:38.625947+0000","last_became_peered":"2023-10-10T20:28:38.625947+0000","last_unstale":"2023-10-10T20:28:39.658281+0000","last_undegraded":"2023-10-10T20:28:39.658281+0000","last_fullsized":"2023-10-10T20:28:39.658281+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:24.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.651016+0000","last_change":"2023-10-10T20:28:39.650680+0000","last_active":"2023-10-10T20:28:39.651016+0000","last_peered":"2023-10-10T20:28:39.651016+0000","last_clean":"2023-10-10T20:28:39.651016+0000","last_became_active":"2023-10-10T20:28:39.650075+0000","last_became_peered":"2023-10-10T20:28:39.650075+0000","last_unstale":"2023-10-10T20:28:39.651016+0000","last_undegraded":"2023-10-10T20:28:39.651016+0000","last_fullsized":"2023-10-10T20:28:39.651016+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:58.435248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659118+0000","last_change":"2023-10-10T20:28:38.630687+0000","last_active":"2023-10-10T20:28:39.659118+0000","last_peered":"2023-10-10T20:28:39.659118+0000","last_clean":"2023-10-10T20:28:39.659118+0000","last_became_active":"2023-10-10T20:28:38.630435+0000","last_became_peered":"2023-10-10T20:28:38.630435+0000","last_unstale":"2023-10-10T20:28:39.659118+0000","last_undegraded":"2023-10-10T20:28:39.659118+0000","last_fullsized":"2023-10-10T20:28:39.659118+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:23:02.137460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642037+0000","last_change":"2023-10-10T20:28:39.641968+0000","last_active":"2023-10-10T20:28:39.642037+0000","last_peered":"2023-10-10T20:28:39.642037+0000","last_clean":"2023-10-10T20:28:39.642037+0000","last_became_active":"2023-10-10T20:28:39.641380+0000","last_became_peered":"2023-10-10T20:28:39.641380+0000","last_unstale":"2023-10-10T20:28:39.642037+0000","last_undegraded":"2023-10-10T20:28:39.642037+0000","last_fullsized":"2023-10-10T20:28:39.642037+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:04:51.168883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657248+0000","last_change":"2023-10-10T20:28:38.628526+0000","last_active":"2023-10-10T20:28:39.657248+0000","last_peered":"2023-10-10T20:28:39.657248+0000","last_clean":"2023-10-10T20:28:39.657248+0000","last_became_active":"2023-10-10T20:28:38.628279+0000","last_became_peered":"2023-10-10T20:28:38.628279+0000","last_unstale":"2023-10-10T20:28:39.657248+0000","last_undegraded":"2023-10-10T20:28:39.657248+0000","last_fullsized":"2023-10-10T20:28:39.657248+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:23.092555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642632+0000","last_change":"2023-10-10T20:28:39.642305+0000","last_active":"2023-10-10T20:28:39.642632+0000","last_peered":"2023-10-10T20:28:39.642632+0000","last_clean":"2023-10-10T20:28:39.642632+0000","last_became_active":"2023-10-10T20:28:39.641940+0000","last_became_peered":"2023-10-10T20:28:39.641940+0000","last_unstale":"2023-10-10T20:28:39.642632+0000","last_undegraded":"2023-10-10T20:28:39.642632+0000","last_fullsized":"2023-10-10T20:28:39.642632+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:41:29.495073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656465+0000","last_change":"2023-10-10T20:28:38.627118+0000","last_active":"2023-10-10T20:28:39.656465+0000","last_peered":"2023-10-10T20:28:39.656465+0000","last_clean":"2023-10-10T20:28:39.656465+0000","last_became_active":"2023-10-10T20:28:38.626948+0000","last_became_peered":"2023-10-10T20:28:38.626948+0000","last_unstale":"2023-10-10T20:28:39.656465+0000","last_undegraded":"2023-10-10T20:28:39.656465+0000","last_fullsized":"2023-10-10T20:28:39.656465+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:57:17.416780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639560+0000","last_change":"2023-10-10T20:28:39.636255+0000","last_active":"2023-10-10T20:28:39.639560+0000","last_peered":"2023-10-10T20:28:39.639560+0000","last_clean":"2023-10-10T20:28:39.639560+0000","last_became_active":"2023-10-10T20:28:39.636027+0000","last_became_peered":"2023-10-10T20:28:39.636027+0000","last_unstale":"2023-10-10T20:28:39.639560+0000","last_undegraded":"2023-10-10T20:28:39.639560+0000","last_fullsized":"2023-10-10T20:28:39.639560+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:11:56.503437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.638968+0000","last_change":"2023-10-10T20:28:38.627400+0000","last_active":"2023-10-10T20:28:39.638968+0000","last_peered":"2023-10-10T20:28:39.638968+0000","last_clean":"2023-10-10T20:28:39.638968+0000","last_became_active":"2023-10-10T20:28:38.627015+0000","last_became_peered":"2023-10-10T20:28:38.627015+0000","last_unstale":"2023-10-10T20:28:39.638968+0000","last_undegraded":"2023-10-10T20:28:39.638968+0000","last_fullsized":"2023-10-10T20:28:39.638968+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:51:11.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642660+0000","last_change":"2023-10-10T20:28:39.641920+0000","last_active":"2023-10-10T20:28:39.642660+0000","last_peered":"2023-10-10T20:28:39.642660+0000","last_clean":"2023-10-10T20:28:39.642660+0000","last_became_active":"2023-10-10T20:28:39.641638+0000","last_became_peered":"2023-10-10T20:28:39.641638+0000","last_unstale":"2023-10-10T20:28:39.642660+0000","last_undegraded":"2023-10-10T20:28:39.642660+0000","last_fullsized":"2023-10-10T20:28:39.642660+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:46.519220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.636104+0000","last_change":"2023-10-10T20:28:38.629597+0000","last_active":"2023-10-10T20:28:39.636104+0000","last_peered":"2023-10-10T20:28:39.636104+0000","last_clean":"2023-10-10T20:28:39.636104+0000","last_became_active":"2023-10-10T20:28:38.629337+0000","last_became_peered":"2023-10-10T20:28:38.629337+0000","last_unstale":"2023-10-10T20:28:39.636104+0000","last_undegraded":"2023-10-10T20:28:39.636104+0000","last_fullsized":"2023-10-10T20:28:39.636104+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:33.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643573+0000","last_change":"2023-10-10T20:28:39.643460+0000","last_active":"2023-10-10T20:28:39.643573+0000","last_peered":"2023-10-10T20:28:39.643573+0000","last_clean":"2023-10-10T20:28:39.643573+0000","last_became_active":"2023-10-10T20:28:39.642997+0000","last_became_peered":"2023-10-10T20:28:39.642997+0000","last_unstale":"2023-10-10T20:28:39.643573+0000","last_undegraded":"2023-10-10T20:28:39.643573+0000","last_fullsized":"2023-10-10T20:28:39.643573+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:24:58.465941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633815+0000","last_change":"2023-10-10T20:28:38.630837+0000","last_active":"2023-10-10T20:28:39.633815+0000","last_peered":"2023-10-10T20:28:39.633815+0000","last_clean":"2023-10-10T20:28:39.633815+0000","last_became_active":"2023-10-10T20:28:38.630662+0000","last_became_peered":"2023-10-10T20:28:38.630662+0000","last_unstale":"2023-10-10T20:28:39.633815+0000","last_undegraded":"2023-10-10T20:28:39.633815+0000","last_fullsized":"2023-10-10T20:28:39.633815+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:53.815634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647368+0000","last_change":"2023-10-10T20:28:39.647074+0000","last_active":"2023-10-10T20:28:39.647368+0000","last_peered":"2023-10-10T20:28:39.647368+0000","last_clean":"2023-10-10T20:28:39.647368+0000","last_became_active":"2023-10-10T20:28:39.645697+0000","last_became_peered":"2023-10-10T20:28:39.645697+0000","last_unstale":"2023-10-10T20:28:39.647368+0000","last_undegraded":"2023-10-10T20:28:39.647368+0000","last_fullsized":"2023-10-10T20:28:39.647368+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:38.975918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640538+0000","last_change":"2023-10-10T20:28:38.630120+0000","last_active":"2023-10-10T20:28:39.640538+0000","last_peered":"2023-10-10T20:28:39.640538+0000","last_clean":"2023-10-10T20:28:39.640538+0000","last_became_active":"2023-10-10T20:28:38.629401+0000","last_became_peered":"2023-10-10T20:28:38.629401+0000","last_unstale":"2023-10-10T20:28:39.640538+0000","last_undegraded":"2023-10-10T20:28:39.640538+0000","last_fullsized":"2023-10-10T20:28:39.640538+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:46.437313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640189+0000","last_change":"2023-10-10T20:28:39.640055+0000","last_active":"2023-10-10T20:28:39.640189+0000","last_peered":"2023-10-10T20:28:39.640189+0000","last_clean":"2023-10-10T20:28:39.640189+0000","last_became_active":"2023-10-10T20:28:39.639841+0000","last_became_peered":"2023-10-10T20:28:39.639841+0000","last_unstale":"2023-10-10T20:28:39.640189+0000","last_undegraded":"2023-10-10T20:28:39.640189+0000","last_fullsized":"2023-10-10T20:28:39.640189+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:15:53.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"161'6","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659732+0000","last_change":"2023-10-10T20:28:38.625781+0000","last_active":"2023-10-10T20:28:39.659732+0000","last_peered":"2023-10-10T20:28:39.659732+0000","last_clean":"2023-10-10T20:28:39.659732+0000","last_became_active":"2023-10-10T20:28:38.625602+0000","last_became_peered":"2023-10-10T20:28:38.625602+0000","last_unstale":"2023-10-10T20:28:39.659732+0000","last_undegraded":"2023-10-10T20:28:39.659732+0000","last_fullsized":"2023-10-10T20:28:39.659732+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:51.170402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642211+0000","last_change":"2023-10-10T20:28:39.642134+0000","last_active":"2023-10-10T20:28:39.642211+0000","last_peered":"2023-10-10T20:28:39.642211+0000","last_clean":"2023-10-10T20:28:39.642211+0000","last_became_active":"2023-10-10T20:28:39.641118+0000","last_became_peered":"2023-10-10T20:28:39.641118+0000","last_unstale":"2023-10-10T20:28:39.642211+0000","last_undegraded":"2023-10-10T20:28:39.642211+0000","last_fullsized":"2023-10-10T20:28:39.642211+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658086+0000","last_change":"2023-10-10T20:28:38.628263+0000","last_active":"2023-10-10T20:28:39.658086+0000","last_peered":"2023-10-10T20:28:39.658086+0000","last_clean":"2023-10-10T20:28:39.658086+0000","last_became_active":"2023-10-10T20:28:38.628034+0000","last_became_peered":"2023-10-10T20:28:38.628034+0000","last_unstale":"2023-10-10T20:28:39.658086+0000","last_undegraded":"2023-10-10T20:28:39.658086+0000","last_fullsized":"2023-10-10T20:28:39.658086+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:05:54.825961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642777+0000","last_change":"2023-10-10T20:28:39.642722+0000","last_active":"2023-10-10T20:28:39.642777+0000","last_peered":"2023-10-10T20:28:39.642777+0000","last_clean":"2023-10-10T20:28:39.642777+0000","last_became_active":"2023-10-10T20:28:39.641971+0000","last_became_peered":"2023-10-10T20:28:39.641971+0000","last_unstale":"2023-10-10T20:28:39.642777+0000","last_undegraded":"2023-10-10T20:28:39.642777+0000","last_fullsized":"2023-10-10T20:28:39.642777+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:40:38.975918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639276+0000","last_change":"2023-10-10T20:28:38.626293+0000","last_active":"2023-10-10T20:28:39.639276+0000","last_peered":"2023-10-10T20:28:39.639276+0000","last_clean":"2023-10-10T20:28:39.639276+0000","last_became_active":"2023-10-10T20:28:38.626078+0000","last_became_peered":"2023-10-10T20:28:38.626078+0000","last_unstale":"2023-10-10T20:28:39.639276+0000","last_undegraded":"2023-10-10T20:28:39.639276+0000","last_fullsized":"2023-10-10T20:28:39.639276+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:31:37.087990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648228+0000","last_change":"2023-10-10T20:28:39.647989+0000","last_active":"2023-10-10T20:28:39.648228+0000","last_peered":"2023-10-10T20:28:39.648228+0000","last_clean":"2023-10-10T20:28:39.648228+0000","last_became_active":"2023-10-10T20:28:39.647295+0000","last_became_peered":"2023-10-10T20:28:39.647295+0000","last_unstale":"2023-10-10T20:28:39.648228+0000","last_undegraded":"2023-10-10T20:28:39.648228+0000","last_fullsized":"2023-10-10T20:28:39.648228+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:32.156438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639886+0000","last_change":"2023-10-10T20:28:38.624641+0000","last_active":"2023-10-10T20:28:39.639886+0000","last_peered":"2023-10-10T20:28:39.639886+0000","last_clean":"2023-10-10T20:28:39.639886+0000","last_became_active":"2023-10-10T20:28:38.624386+0000","last_became_peered":"2023-10-10T20:28:38.624386+0000","last_unstale":"2023-10-10T20:28:39.639886+0000","last_undegraded":"2023-10-10T20:28:39.639886+0000","last_fullsized":"2023-10-10T20:28:39.639886+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:23:02.137460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642961+0000","last_change":"2023-10-10T20:28:39.642668+0000","last_active":"2023-10-10T20:28:39.642961+0000","last_peered":"2023-10-10T20:28:39.642961+0000","last_clean":"2023-10-10T20:28:39.642961+0000","last_became_active":"2023-10-10T20:28:39.642223+0000","last_became_peered":"2023-10-10T20:28:39.642223+0000","last_unstale":"2023-10-10T20:28:39.642961+0000","last_undegraded":"2023-10-10T20:28:39.642961+0000","last_fullsized":"2023-10-10T20:28:39.642961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:44:47.441086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"161'8","reported_seq":18,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.659687+0000","last_change":"2023-10-10T20:28:38.629479+0000","last_active":"2023-10-10T20:28:39.659687+0000","last_peered":"2023-10-10T20:28:39.659687+0000","last_clean":"2023-10-10T20:28:39.659687+0000","last_became_active":"2023-10-10T20:28:38.629244+0000","last_became_peered":"2023-10-10T20:28:38.629244+0000","last_unstale":"2023-10-10T20:28:39.659687+0000","last_undegraded":"2023-10-10T20:28:39.659687+0000","last_fullsized":"2023-10-10T20:28:39.659687+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:12:11.329793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.649417+0000","last_change":"2023-10-10T20:28:39.649358+0000","last_active":"2023-10-10T20:28:39.649417+0000","last_peered":"2023-10-10T20:28:39.649417+0000","last_clean":"2023-10-10T20:28:39.649417+0000","last_became_active":"2023-10-10T20:28:39.649098+0000","last_became_peered":"2023-10-10T20:28:39.649098+0000","last_unstale":"2023-10-10T20:28:39.649417+0000","last_undegraded":"2023-10-10T20:28:39.649417+0000","last_fullsized":"2023-10-10T20:28:39.649417+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:46:40.931252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656063+0000","last_change":"2023-10-10T20:28:38.626852+0000","last_active":"2023-10-10T20:28:39.656063+0000","last_peered":"2023-10-10T20:28:39.656063+0000","last_clean":"2023-10-10T20:28:39.656063+0000","last_became_active":"2023-10-10T20:28:38.626628+0000","last_became_peered":"2023-10-10T20:28:38.626628+0000","last_unstale":"2023-10-10T20:28:39.656063+0000","last_undegraded":"2023-10-10T20:28:39.656063+0000","last_fullsized":"2023-10-10T20:28:39.656063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:55:10.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646883+0000","last_change":"2023-10-10T20:28:39.646831+0000","last_active":"2023-10-10T20:28:39.646883+0000","last_peered":"2023-10-10T20:28:39.646883+0000","last_clean":"2023-10-10T20:28:39.646883+0000","last_became_active":"2023-10-10T20:28:39.646454+0000","last_became_peered":"2023-10-10T20:28:39.646454+0000","last_unstale":"2023-10-10T20:28:39.646883+0000","last_undegraded":"2023-10-10T20:28:39.646883+0000","last_fullsized":"2023-10-10T20:28:39.646883+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:12:46.519220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.656979+0000","last_change":"2023-10-10T20:28:38.629927+0000","last_active":"2023-10-10T20:28:39.656979+0000","last_peered":"2023-10-10T20:28:39.656979+0000","last_clean":"2023-10-10T20:28:39.656979+0000","last_became_active":"2023-10-10T20:28:38.629351+0000","last_became_peered":"2023-10-10T20:28:38.629351+0000","last_unstale":"2023-10-10T20:28:39.656979+0000","last_undegraded":"2023-10-10T20:28:39.656979+0000","last_fullsized":"2023-10-10T20:28:39.656979+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:02.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643959+0000","last_change":"2023-10-10T20:28:39.643664+0000","last_active":"2023-10-10T20:28:39.643959+0000","last_peered":"2023-10-10T20:28:39.643959+0000","last_clean":"2023-10-10T20:28:39.643959+0000","last_became_active":"2023-10-10T20:28:39.643048+0000","last_became_peered":"2023-10-10T20:28:39.643048+0000","last_unstale":"2023-10-10T20:28:39.643959+0000","last_undegraded":"2023-10-10T20:28:39.643959+0000","last_fullsized":"2023-10-10T20:28:39.643959+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:28:54.819407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"161'1","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657095+0000","last_change":"2023-10-10T20:28:38.626323+0000","last_active":"2023-10-10T20:28:39.657095+0000","last_peered":"2023-10-10T20:28:39.657095+0000","last_clean":"2023-10-10T20:28:39.657095+0000","last_became_active":"2023-10-10T20:28:38.626108+0000","last_became_peered":"2023-10-10T20:28:38.626108+0000","last_unstale":"2023-10-10T20:28:39.657095+0000","last_undegraded":"2023-10-10T20:28:39.657095+0000","last_fullsized":"2023-10-10T20:28:39.657095+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:00:47.967475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646346+0000","last_change":"2023-10-10T20:28:39.645874+0000","last_active":"2023-10-10T20:28:39.646346+0000","last_peered":"2023-10-10T20:28:39.646346+0000","last_clean":"2023-10-10T20:28:39.646346+0000","last_became_active":"2023-10-10T20:28:39.644927+0000","last_became_peered":"2023-10-10T20:28:39.644927+0000","last_unstale":"2023-10-10T20:28:39.646346+0000","last_undegraded":"2023-10-10T20:28:39.646346+0000","last_fullsized":"2023-10-10T20:28:39.646346+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633752+0000","last_change":"2023-10-10T20:28:38.628197+0000","last_active":"2023-10-10T20:28:39.633752+0000","last_peered":"2023-10-10T20:28:39.633752+0000","last_clean":"2023-10-10T20:28:39.633752+0000","last_became_active":"2023-10-10T20:28:38.627992+0000","last_became_peered":"2023-10-10T20:28:38.627992+0000","last_unstale":"2023-10-10T20:28:39.633752+0000","last_undegraded":"2023-10-10T20:28:39.633752+0000","last_fullsized":"2023-10-10T20:28:39.633752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:09:09.183516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643529+0000","last_change":"2023-10-10T20:28:39.643235+0000","last_active":"2023-10-10T20:28:39.643529+0000","last_peered":"2023-10-10T20:28:39.643529+0000","last_clean":"2023-10-10T20:28:39.643529+0000","last_became_active":"2023-10-10T20:28:39.642415+0000","last_became_peered":"2023-10-10T20:28:39.642415+0000","last_unstale":"2023-10-10T20:28:39.643529+0000","last_undegraded":"2023-10-10T20:28:39.643529+0000","last_fullsized":"2023-10-10T20:28:39.643529+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:15:45.995788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657550+0000","last_change":"2023-10-10T20:28:38.628787+0000","last_active":"2023-10-10T20:28:39.657550+0000","last_peered":"2023-10-10T20:28:39.657550+0000","last_clean":"2023-10-10T20:28:39.657550+0000","last_became_active":"2023-10-10T20:28:38.628570+0000","last_became_peered":"2023-10-10T20:28:38.628570+0000","last_unstale":"2023-10-10T20:28:39.657550+0000","last_undegraded":"2023-10-10T20:28:39.657550+0000","last_fullsized":"2023-10-10T20:28:39.657550+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:56:33.070338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645963+0000","last_change":"2023-10-10T20:28:39.645888+0000","last_active":"2023-10-10T20:28:39.645963+0000","last_peered":"2023-10-10T20:28:39.645963+0000","last_clean":"2023-10-10T20:28:39.645963+0000","last_became_active":"2023-10-10T20:28:39.645582+0000","last_became_peered":"2023-10-10T20:28:39.645582+0000","last_unstale":"2023-10-10T20:28:39.645963+0000","last_undegraded":"2023-10-10T20:28:39.645963+0000","last_fullsized":"2023-10-10T20:28:39.645963+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:34:58.435248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641599+0000","last_change":"2023-10-10T20:28:38.632175+0000","last_active":"2023-10-10T20:28:39.641599+0000","last_peered":"2023-10-10T20:28:39.641599+0000","last_clean":"2023-10-10T20:28:39.641599+0000","last_became_active":"2023-10-10T20:28:38.631956+0000","last_became_peered":"2023-10-10T20:28:38.631956+0000","last_unstale":"2023-10-10T20:28:39.641599+0000","last_undegraded":"2023-10-10T20:28:39.641599+0000","last_fullsized":"2023-10-10T20:28:39.641599+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:39:02.428914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.644036+0000","last_change":"2023-10-10T20:28:39.643988+0000","last_active":"2023-10-10T20:28:39.644036+0000","last_peered":"2023-10-10T20:28:39.644036+0000","last_clean":"2023-10-10T20:28:39.644036+0000","last_became_active":"2023-10-10T20:28:39.643839+0000","last_became_peered":"2023-10-10T20:28:39.643839+0000","last_unstale":"2023-10-10T20:28:39.644036+0000","last_undegraded":"2023-10-10T20:28:39.644036+0000","last_fullsized":"2023-10-10T20:28:39.644036+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:52.314868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.647261+0000","last_change":"2023-10-10T20:28:38.631602+0000","last_active":"2023-10-10T20:28:39.647261+0000","last_peered":"2023-10-10T20:28:39.647261+0000","last_clean":"2023-10-10T20:28:39.647261+0000","last_became_active":"2023-10-10T20:28:38.631357+0000","last_became_peered":"2023-10-10T20:28:38.631357+0000","last_unstale":"2023-10-10T20:28:39.647261+0000","last_undegraded":"2023-10-10T20:28:39.647261+0000","last_fullsized":"2023-10-10T20:28:39.647261+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:04.704041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.646154+0000","last_change":"2023-10-10T20:28:39.646054+0000","last_active":"2023-10-10T20:28:39.646154+0000","last_peered":"2023-10-10T20:28:39.646154+0000","last_clean":"2023-10-10T20:28:39.646154+0000","last_became_active":"2023-10-10T20:28:39.645857+0000","last_became_peered":"2023-10-10T20:28:39.645857+0000","last_unstale":"2023-10-10T20:28:39.646154+0000","last_undegraded":"2023-10-10T20:28:39.646154+0000","last_fullsized":"2023-10-10T20:28:39.646154+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:07:07.138886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639379+0000","last_change":"2023-10-10T20:28:38.628342+0000","last_active":"2023-10-10T20:28:39.639379+0000","last_peered":"2023-10-10T20:28:39.639379+0000","last_clean":"2023-10-10T20:28:39.639379+0000","last_became_active":"2023-10-10T20:28:38.628099+0000","last_became_peered":"2023-10-10T20:28:38.628099+0000","last_unstale":"2023-10-10T20:28:39.639379+0000","last_undegraded":"2023-10-10T20:28:39.639379+0000","last_fullsized":"2023-10-10T20:28:39.639379+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:26:27.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648196+0000","last_change":"2023-10-10T20:28:39.647838+0000","last_active":"2023-10-10T20:28:39.648196+0000","last_peered":"2023-10-10T20:28:39.648196+0000","last_clean":"2023-10-10T20:28:39.648196+0000","last_became_active":"2023-10-10T20:28:39.646971+0000","last_became_peered":"2023-10-10T20:28:39.646971+0000","last_unstale":"2023-10-10T20:28:39.648196+0000","last_undegraded":"2023-10-10T20:28:39.648196+0000","last_fullsized":"2023-10-10T20:28:39.648196+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:29:48.738597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"161'2","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.657158+0000","last_change":"2023-10-10T20:28:38.630237+0000","last_active":"2023-10-10T20:28:39.657158+0000","last_peered":"2023-10-10T20:28:39.657158+0000","last_clean":"2023-10-10T20:28:39.657158+0000","last_became_active":"2023-10-10T20:28:38.629683+0000","last_became_peered":"2023-10-10T20:28:38.629683+0000","last_unstale":"2023-10-10T20:28:39.657158+0000","last_undegraded":"2023-10-10T20:28:39.657158+0000","last_fullsized":"2023-10-10T20:28:39.657158+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:21:21.398248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643980+0000","last_change":"2023-10-10T20:28:39.643807+0000","last_active":"2023-10-10T20:28:39.643980+0000","last_peered":"2023-10-10T20:28:39.643980+0000","last_clean":"2023-10-10T20:28:39.643980+0000","last_became_active":"2023-10-10T20:28:39.643336+0000","last_became_peered":"2023-10-10T20:28:39.643336+0000","last_unstale":"2023-10-10T20:28:39.643980+0000","last_undegraded":"2023-10-10T20:28:39.643980+0000","last_fullsized":"2023-10-10T20:28:39.643980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:46:57.093229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639719+0000","last_change":"2023-10-10T20:28:38.626780+0000","last_active":"2023-10-10T20:28:39.639719+0000","last_peered":"2023-10-10T20:28:39.639719+0000","last_clean":"2023-10-10T20:28:39.639719+0000","last_became_active":"2023-10-10T20:28:38.626562+0000","last_became_peered":"2023-10-10T20:28:38.626562+0000","last_unstale":"2023-10-10T20:28:39.639719+0000","last_undegraded":"2023-10-10T20:28:39.639719+0000","last_fullsized":"2023-10-10T20:28:39.639719+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:39:33.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642629+0000","last_change":"2023-10-10T20:28:39.642563+0000","last_active":"2023-10-10T20:28:39.642629+0000","last_peered":"2023-10-10T20:28:39.642629+0000","last_clean":"2023-10-10T20:28:39.642629+0000","last_became_active":"2023-10-10T20:28:39.641625+0000","last_became_peered":"2023-10-10T20:28:39.641625+0000","last_unstale":"2023-10-10T20:28:39.642629+0000","last_undegraded":"2023-10-10T20:28:39.642629+0000","last_fullsized":"2023-10-10T20:28:39.642629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:29:44.127672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640509+0000","last_change":"2023-10-10T20:28:38.628816+0000","last_active":"2023-10-10T20:28:39.640509+0000","last_peered":"2023-10-10T20:28:39.640509+0000","last_clean":"2023-10-10T20:28:39.640509+0000","last_became_active":"2023-10-10T20:28:38.628606+0000","last_became_peered":"2023-10-10T20:28:38.628606+0000","last_unstale":"2023-10-10T20:28:39.640509+0000","last_undegraded":"2023-10-10T20:28:39.640509+0000","last_fullsized":"2023-10-10T20:28:39.640509+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:10:53.306026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641163+0000","last_change":"2023-10-10T20:28:39.641104+0000","last_active":"2023-10-10T20:28:39.641163+0000","last_peered":"2023-10-10T20:28:39.641163+0000","last_clean":"2023-10-10T20:28:39.641163+0000","last_became_active":"2023-10-10T20:28:39.640900+0000","last_became_peered":"2023-10-10T20:28:39.640900+0000","last_unstale":"2023-10-10T20:28:39.641163+0000","last_undegraded":"2023-10-10T20:28:39.641163+0000","last_fullsized":"2023-10-10T20:28:39.641163+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:25.416937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639939+0000","last_change":"2023-10-10T20:28:38.626629+0000","last_active":"2023-10-10T20:28:39.639939+0000","last_peered":"2023-10-10T20:28:39.639939+0000","last_clean":"2023-10-10T20:28:39.639939+0000","last_became_active":"2023-10-10T20:28:38.626382+0000","last_became_peered":"2023-10-10T20:28:38.626382+0000","last_unstale":"2023-10-10T20:28:39.639939+0000","last_undegraded":"2023-10-10T20:28:39.639939+0000","last_fullsized":"2023-10-10T20:28:39.639939+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:24:24.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643379+0000","last_change":"2023-10-10T20:28:39.643321+0000","last_active":"2023-10-10T20:28:39.643379+0000","last_peered":"2023-10-10T20:28:39.643379+0000","last_clean":"2023-10-10T20:28:39.643379+0000","last_became_active":"2023-10-10T20:28:39.643004+0000","last_became_peered":"2023-10-10T20:28:39.643004+0000","last_unstale":"2023-10-10T20:28:39.643379+0000","last_undegraded":"2023-10-10T20:28:39.643379+0000","last_fullsized":"2023-10-10T20:28:39.643379+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:47:00.521089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.638596+0000","last_change":"2023-10-10T20:28:38.624362+0000","last_active":"2023-10-10T20:28:39.638596+0000","last_peered":"2023-10-10T20:28:39.638596+0000","last_clean":"2023-10-10T20:28:39.638596+0000","last_became_active":"2023-10-10T20:28:38.624091+0000","last_became_peered":"2023-10-10T20:28:38.624091+0000","last_unstale":"2023-10-10T20:28:39.638596+0000","last_undegraded":"2023-10-10T20:28:39.638596+0000","last_fullsized":"2023-10-10T20:28:39.638596+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:07:00.338439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.642919+0000","last_change":"2023-10-10T20:28:39.642751+0000","last_active":"2023-10-10T20:28:39.642919+0000","last_peered":"2023-10-10T20:28:39.642919+0000","last_clean":"2023-10-10T20:28:39.642919+0000","last_became_active":"2023-10-10T20:28:39.642315+0000","last_became_peered":"2023-10-10T20:28:39.642315+0000","last_unstale":"2023-10-10T20:28:39.642919+0000","last_undegraded":"2023-10-10T20:28:39.642919+0000","last_fullsized":"2023-10-10T20:28:39.642919+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:51:13.161511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.640732+0000","last_change":"2023-10-10T20:28:38.630477+0000","last_active":"2023-10-10T20:28:39.640732+0000","last_peered":"2023-10-10T20:28:39.640732+0000","last_clean":"2023-10-10T20:28:39.640732+0000","last_became_active":"2023-10-10T20:28:38.629967+0000","last_became_peered":"2023-10-10T20:28:38.629967+0000","last_unstale":"2023-10-10T20:28:39.640732+0000","last_undegraded":"2023-10-10T20:28:39.640732+0000","last_fullsized":"2023-10-10T20:28:39.640732+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:33:47.542816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.648604+0000","last_change":"2023-10-10T20:28:39.648398+0000","last_active":"2023-10-10T20:28:39.648604+0000","last_peered":"2023-10-10T20:28:39.648604+0000","last_clean":"2023-10-10T20:28:39.648604+0000","last_became_active":"2023-10-10T20:28:39.647867+0000","last_became_peered":"2023-10-10T20:28:39.647867+0000","last_unstale":"2023-10-10T20:28:39.648604+0000","last_undegraded":"2023-10-10T20:28:39.648604+0000","last_fullsized":"2023-10-10T20:28:39.648604+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:41:57.498975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.639932+0000","last_change":"2023-10-10T20:28:38.628713+0000","last_active":"2023-10-10T20:28:39.639932+0000","last_peered":"2023-10-10T20:28:39.639932+0000","last_clean":"2023-10-10T20:28:39.639932+0000","last_became_active":"2023-10-10T20:28:38.627675+0000","last_became_peered":"2023-10-10T20:28:38.627675+0000","last_unstale":"2023-10-10T20:28:39.639932+0000","last_undegraded":"2023-10-10T20:28:39.639932+0000","last_fullsized":"2023-10-10T20:28:39.639932+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:47:24.413165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.643557+0000","last_change":"2023-10-10T20:28:39.643421+0000","last_active":"2023-10-10T20:28:39.643557+0000","last_peered":"2023-10-10T20:28:39.643557+0000","last_clean":"2023-10-10T20:28:39.643557+0000","last_became_active":"2023-10-10T20:28:39.642543+0000","last_became_peered":"2023-10-10T20:28:39.642543+0000","last_unstale":"2023-10-10T20:28:39.643557+0000","last_undegraded":"2023-10-10T20:28:39.643557+0000","last_fullsized":"2023-10-10T20:28:39.643557+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:47:19.817675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.633756+0000","last_change":"2023-10-10T20:28:38.627364+0000","last_active":"2023-10-10T20:28:39.633756+0000","last_peered":"2023-10-10T20:28:39.633756+0000","last_clean":"2023-10-10T20:28:39.633756+0000","last_became_active":"2023-10-10T20:28:38.627054+0000","last_became_peered":"2023-10-10T20:28:38.627054+0000","last_unstale":"2023-10-10T20:28:39.633756+0000","last_undegraded":"2023-10-10T20:28:39.633756+0000","last_fullsized":"2023-10-10T20:28:39.633756+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:59:04.704041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.645801+0000","last_change":"2023-10-10T20:28:39.645736+0000","last_active":"2023-10-10T20:28:39.645801+0000","last_peered":"2023-10-10T20:28:39.645801+0000","last_clean":"2023-10-10T20:28:39.645801+0000","last_became_active":"2023-10-10T20:28:39.645509+0000","last_became_peered":"2023-10-10T20:28:39.645509+0000","last_unstale":"2023-10-10T20:28:39.645801+0000","last_undegraded":"2023-10-10T20:28:39.645801+0000","last_fullsized":"2023-10-10T20:28:39.645801+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:38.615195+0000","last_clean_scrub_stamp":"2023-10-10T20:28:38.615195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:21:22.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"161'3","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.658225+0000","last_change":"2023-10-10T20:28:38.629145+0000","last_active":"2023-10-10T20:28:39.658225+0000","last_peered":"2023-10-10T20:28:39.658225+0000","last_clean":"2023-10-10T20:28:39.658225+0000","last_became_active":"2023-10-10T20:28:38.628371+0000","last_became_peered":"2023-10-10T20:28:38.628371+0000","last_unstale":"2023-10-10T20:28:39.658225+0000","last_undegraded":"2023-10-10T20:28:39.658225+0000","last_fullsized":"2023-10-10T20:28:39.658225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:28:37.611422+0000","last_clean_scrub_stamp":"2023-10-10T20:28:37.611422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:13:15.182096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":288,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-10T20:28:39.641690+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:28:39.641690+0000","last_peered":"2023-10-10T20:28:39.641690+0000","last_clean":"2023-10-10T20:28:39.641690+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:28:39.641690+0000","last_undegraded":"2023-10-10T20:28:39.641690+0000","last_fullsized":"2023-10-10T20:28:39.641690+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193856,"statfs":{"total":200000000,"available":198509055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53100000000000003,"15min":0.49399999999999999},"min":{"1min":0.38400000000000001,"5min":0.32300000000000001,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.76600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.39900000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70799999999999996,"5min":0.754,"15min":0.754},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.505},"min":{"1min":0.45100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.76500000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72099999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.54200000000000004},"min":{"1min":0.48399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69399999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.32500000000000001},"max":{"1min":0.65500000000000003,"5min":0.72699999999999998,"15min":0.73799999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68100000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.75900000000000001,"15min":0.79600000000000004},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.46600000000000003},"min":{"1min":0.32400000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.69299999999999995,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.503},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.376,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":0.746,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47799999999999998,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.69699999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.33100000000000002},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.316,"15min":0.29599999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":0.80800000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55300000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.77100000000000002,"15min":0.79200000000000004},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099901,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.377,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.72199999999999998,"5min":0.877,"15min":1.3140000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.27000000000000002},"max":{"1min":0.60299999999999998,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.40500000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64000000000000001,"5min":0.92100000000000004,"15min":1.865},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":4.7709999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69199999999999995,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.497},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":0.65900000000000003,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.73299999999999998,"5min":0.77300000000000002,"15min":15.862},"last":0.42299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45600000000000002,"15min":0.442},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.222},"max":{"1min":0.75,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.76000000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.66000000000000003,"5min":0.82699999999999996,"15min":1.0840000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.316,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":1.02,"15min":1.0409999999999999},"last":0.33300000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72999999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.70699999999999996,"5min":0.93200000000000005,"15min":0.97199999999999998},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099493,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55000000000000004,"5min":0.56699999999999995,"15min":1.647},"last":0.27300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.53100000000000003,"5min":0.627,"15min":1.524},"last":0.316},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23200000000000001,"15min":0.223},"max":{"1min":0.52900000000000003,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.55000000000000004,"5min":0.64100000000000001,"15min":1.4470000000000001},"last":0.377},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.54100000000000004,"5min":0.627,"15min":1.673},"last":0.34599999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82199999999999995,"5min":0.84599999999999997,"15min":1.633},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.5,"5min":0.36399999999999999,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88300000000000001,"15min":1.506},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.696},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.64700000000000002},"min":{"1min":0.45200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84999999999999998,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.629,"15min":0.629},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.877,"15min":1.7490000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.53900000000000003,"5min":0.45200000000000001,"15min":0.39400000000000002},"max":{"1min":0.88300000000000001,"5min":0.98899999999999999,"15min":1.585},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80700000000000005,"5min":1.0009999999999999,"15min":1.478},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.624,"15min":0.629},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93500000000000005,"15min":1.722},"last":0.497}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1455,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193856,"statfs":{"total":200000000,"available":198509120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35799999999999998,"15min":0.41699999999999998},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.505,"5min":1.3260000000000001,"15min":1.8740000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.44400000000000001,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53800000000000003,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.44400000000000001},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":1.647,"15min":2.4830000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.42099999999999999,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.50700000000000001,"5min":5.891,"15min":5.891},"last":0.255}]},{"osd":3,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.47299999999999998,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":0.313},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.56499999999999995,"5min":1.462,"15min":1.462},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65300000000000002,"15min":0.624},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.36399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.82999999999999996},"min":{"1min":0.443,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84999999999999998,"5min":4.0990000000000002,"15min":9.7260000000000009},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83899999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":0.84199999999999997,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63300000000000001,"15min":0.60599999999999998},"min":{"1min":0.47599999999999998,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":5.6379999999999999,"15min":5.6379999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:28:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64900000000000002,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.39400000000000002,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":5.758,"15min":5.758},"last":0.504},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":4.0720000000000001,"15min":4.0720000000000001},"last":0.40000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.375},"min":{"1min":0.23799999999999999,"5min":0.189,"15min":0.189},"max":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":2.4409999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":0.53600000000000003,"5min":0.56399999999999995,"15min":8.2629999999999999},"last":0.379}]},{"osd":1,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.223,"15min":0.22},"max":{"1min":0.48499999999999999,"5min":0.60399999999999998,"15min":6.5229999999999997},"last":0.308},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.41499999999999998,"15min":0.44},"min":{"1min":0.314,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":9.2949999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.44600000000000001},"min":{"1min":0.29499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":11.815},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.45900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":18.399999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.46400000000000002,"5min":0.376,"15min":0.28100000000000003},"max":{"1min":0.85699999999999998,"5min":1.004,"15min":2.1389999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.48799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89800000000000002,"5min":0.995,"15min":9.3740000000000006},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.91900000000000004,"5min":0.97799999999999998,"15min":6.399},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79200000000000004,"5min":1.0429999999999999,"15min":2.1040000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":1.0189999999999999,"15min":9.4019999999999992},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.98399999999999999,"15min":6.4969999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.33200000000000002},"max":{"1min":0.79300000000000004,"5min":0.92500000000000004,"15min":9.3450000000000006},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.47499999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.88400000000000001,"5min":1.0309999999999999,"15min":6.4630000000000001},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.31},"min":{"1min":0.252,"5min":0.20200000000000001,"15min":0.19800000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.22},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.32500000000000001},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.21299999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.65600000000000003},"last":0.27700000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.371,"15min":0.34599999999999997},"min":{"1min":0.254,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.255},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.41099999999999998,"15min":0.38400000000000001},"min":{"1min":0.34200000000000003,"5min":0.26800000000000002,"15min":0.26100000000000001},"max":{"1min":0.622,"5min":0.622,"15min":0.751},"last":0.307}]},{"osd":2,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.32300000000000001,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.54600000000000004,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.437,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.55400000000000005},"min":{"1min":0.52800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58299999999999996,"15min":0.54800000000000004},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.745,"5min":0.875,"15min":0.875},"last":0.435}]},{"osd":5,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58599999999999997,"15min":0.55700000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.38200000000000001},"max":{"1min":0.76000000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.55600000000000005},"min":{"1min":0.46400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88800000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":0.79100000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.57499999999999996},"min":{"1min":0.52900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.871,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.47499999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.82999999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.58799999999999997},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199100047,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54600000000000004,"15min":0.50900000000000001},"min":{"1min":0.34200000000000003,"5min":0.22700000000000001,"15min":0.22600000000000001},"max":{"1min":0.628,"5min":0.76400000000000001,"15min":0.85399999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.318,"15min":0.29999999999999999},"max":{"1min":0.68700000000000006,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.41199999999999998,"5min":0.252,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.52800000000000002},"min":{"1min":0.40699999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73099999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.88900000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.52600000000000002},"min":{"1min":0.42299999999999999,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.46000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83299999999999996}]},{"osd":3,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.51200000000000001},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68799999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.81299999999999994}]},{"osd":5,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.629},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57199999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.26100000000000001},"max":{"1min":0.66500000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53300000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.85399999999999998,"15min":0.88700000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32300000000000001},"max":{"1min":0.70999999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":878,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199100025,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.78100000000000003,"15min":0.78500000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.77200000000000002,"15min":0.91200000000000003},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.40999999999999998,"5min":0.311,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57699999999999996,"15min":0.53500000000000003},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.32500000000000001},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.56499999999999995},"min":{"1min":0.46000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.70599999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.48899999999999999},"min":{"1min":0.28799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58399999999999996,"5min":0.76900000000000002,"15min":0.79900000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.501,"15min":0.52500000000000002},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.80800000000000005,"15min":0.875},"last":0.33300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.752,"15min":0.78200000000000003},"last":0.37}]},{"osd":7,"last update":"Tue Oct 10 20:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56499999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.34799999999999998},"max":{"1min":0.751,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.38300000000000001}]}]}],"pool_statfs":[]}} 2023-10-10T20:28:44.058 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:28:44.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:28:44.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.213+0000 7ff0eb014700 1 Processor -- start 2023-10-10T20:28:44.215 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.213+0000 7ff0eb014700 1 -- start start 2023-10-10T20:28:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0eb014700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0e40b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0eb014700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff0e40cd630 0x7ff0e415c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0eb014700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e406b0a0 con 0x7ff0e41542a0 2023-10-10T20:28:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0eb014700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e415b2e0 con 0x7ff0e40cd630 2023-10-10T20:28:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0eb014700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415ad20 con 0x7ff0e406aa00 2023-10-10T20:28:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0ea012700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0e41542a0 0x7ff0e406a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43498/0 (socket says 172.21.15.148:43498) 2023-10-10T20:28:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0ea012700 1 -- 172.21.15.148:0/1676387015 learned_addr learned my addr 172.21.15.148:0/1676387015 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0ea813700 1 --2- 172.21.15.148:0/1676387015 >> v2:172.21.15.161:3300/0 conn(0x7ff0e40cd630 0x7ff0e415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596854149 0 0) 0x7ff0e406b0a0 con 0x7ff0e41542a0 2023-10-10T20:28:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9811700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0e40b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003570 con 0x7ff0e41542a0 2023-10-10T20:28:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714241484 0 0) 0x7ff0d0003570 con 0x7ff0e41542a0 2023-10-10T20:28:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 msgr2=0x7ff0e40b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0e40b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 >> v2:172.21.15.161:3300/0 conn(0x7ff0e40cd630 msgr2=0x7ff0e415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:44.224 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 --2- 172.21.15.148:0/1676387015 >> v2:172.21.15.161:3300/0 conn(0x7ff0e40cd630 0x7ff0e415c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e415b000 con 0x7ff0e41542a0 2023-10-10T20:28:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7ff0e00015f0 con 0x7ff0e41542a0 2023-10-10T20:28:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 >> v1:172.21.15.148:6789/0 conn(0x7ff0e41542a0 legacy=0x7ff0e406a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d0003a60 0x7ff0d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415b2e0 con 0x7ff0d0004810 2023-10-10T20:28:44.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e415ad20 con 0x7ff0d0003a60 2023-10-10T20:28:44.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.214+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d000ba60 con 0x7ff0e406aa00 2023-10-10T20:28:44.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0ea012700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d0003a60 0x7ff0d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0ea813700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 -- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 msgr2=0x7ff0d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:44.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e406aa00 0x7ff0d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 -- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d0003a60 msgr2=0x7ff0d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:44.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d0003a60 0x7ff0d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 -- 172.21.15.148:0/1676387015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0003570 con 0x7ff0d0004810 2023-10-10T20:28:44.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9811700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0d0004c80 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009c00 tx=0x7ff0d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96a4fa41f7f507f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400e710 con 0x7ff0d0004810 2023-10-10T20:28:44.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400e870 con 0x7ff0d0004810 2023-10-10T20:28:44.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.215+0000 7ff0e9010700 1 -- 172.21.15.148:0/1676387015 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400ec30 con 0x7ff0d0004810 2023-10-10T20:28:44.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 -- 172.21.15.148:0/1676387015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 msgr2=0x7ff0d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 --2- 172.21.15.148:0/1676387015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0d0004c80 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009c00 tx=0x7ff0d40091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0ea813700 1 -- 172.21.15.148:0/1676387015 reap_dead start 2023-10-10T20:28:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 -- 172.21.15.148:0/1676387015 shutdown_connections 2023-10-10T20:28:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 -- 172.21.15.148:0/1676387015 >> 172.21.15.148:0/1676387015 conn(0x7ff0e4068cb0 msgr2=0x7ff0e40690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 -- 172.21.15.148:0/1676387015 shutdown_connections 2023-10-10T20:28:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.216+0000 7ff0eb014700 1 -- 172.21.15.148:0/1676387015 wait complete. 2023-10-10T20:28:44.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 Processor -- start 2023-10-10T20:28:44.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 -- start start 2023-10-10T20:28:44.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 0x7ff0e40c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e40c63c0 0x7ff0e40c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40c6830 con 0x7ff0d0004810 2023-10-10T20:28:44.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4158c70 con 0x7ff0e40c63c0 2023-10-10T20:28:44.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.217+0000 7ff0eb014700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4158340 con 0x7ff0e41537f0 2023-10-10T20:28:44.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58562/0 (socket says 172.21.15.148:58562) 2023-10-10T20:28:44.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 -- 172.21.15.148:0/174878948 learned_addr learned my addr 172.21.15.148:0/174878948 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:44.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea813700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e40c63c0 0x7ff0e40c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0e9811700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 0x7ff0e40c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 -- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 msgr2=0x7ff0e40c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.233 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 0x7ff0e40c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 -- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e40c63c0 msgr2=0x7ff0e40c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e40c63c0 0x7ff0e40c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 -- 172.21.15.148:0/174878948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4158620 con 0x7ff0d0004810 2023-10-10T20:28:44.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0e9811700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 0x7ff0e40c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:28:44.234 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0ea012700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000fc30 tx=0x7ff0e0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00072e0 con 0x7ff0d0004810 2023-10-10T20:28:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0007460 con 0x7ff0d0004810 2023-10-10T20:28:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e4158900 con 0x7ff0d0004810 2023-10-10T20:28:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00057f0 con 0x7ff0d0004810 2023-10-10T20:28:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.218+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e4158010 con 0x7ff0d0004810 2023-10-10T20:28:44.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.219+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4158a90 con 0x7ff0d0004810 2023-10-10T20:28:44.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.219+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0e0005ba0 con 0x7ff0d0004810 2023-10-10T20:28:44.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.220+0000 7ff0da7fc700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.220+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0e00847d0 con 0x7ff0d0004810 2023-10-10T20:28:44.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.220+0000 7ff0e9811700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.220+0000 7ff0e9811700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 0x7ff0c80693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40091e0 tx=0x7ff0d401a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.224+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0e0051a40 con 0x7ff0d0004810 2023-10-10T20:28:44.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.479+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff0e415d610 con 0x7ff0d0004810 2023-10-10T20:28:44.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.483+0000 7ff0da7fc700 1 -- 172.21.15.148:0/174878948 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff0e0055060 con 0x7ff0d0004810 2023-10-10T20:28:44.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 msgr2=0x7ff0c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 0x7ff0c80693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40091e0 tx=0x7ff0d401a000 comp rx=0 tx=0).stop 2023-10-10T20:28:44.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 msgr2=0x7ff0e4159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000fc30 tx=0x7ff0e0009040 comp rx=0 tx=0).stop 2023-10-10T20:28:44.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 shutdown_connections 2023-10-10T20:28:44.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff0c8066f30 0x7ff0c80693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0e40c63c0 0x7ff0e40c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0d0004810 0x7ff0e4159c40 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 --2- 172.21.15.148:0/174878948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0e41537f0 0x7ff0e40c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 >> 172.21.15.148:0/174878948 conn(0x7ff0e4068cb0 msgr2=0x7ff0e40b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:44.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 shutdown_connections 2023-10-10T20:28:44.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.485+0000 7ff0eb014700 1 -- 172.21.15.148:0/174878948 wait complete. 2023-10-10T20:28:44.490 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:28:44.499 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:44.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 Processor -- start 2023-10-10T20:28:44.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 -- start start 2023-10-10T20:28:44.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55500cffa0 0x7f555006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f555015a230 0x7f555006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55500b3d40 con 0x7f5550159e60 2023-10-10T20:28:44.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5550069d70 con 0x7f55500cffa0 2023-10-10T20:28:44.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.613+0000 7f55562a3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55500b4300 con 0x7f555015a230 2023-10-10T20:28:44.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55500cffa0 0x7f555006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55500cffa0 0x7f555006a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58566/0 (socket says 172.21.15.148:58566) 2023-10-10T20:28:44.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554ffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5550159e60 0x7f55500b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43514/0 (socket says 172.21.15.148:43514) 2023-10-10T20:28:44.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 -- 172.21.15.148:0/3272963259 learned_addr learned my addr 172.21.15.148:0/3272963259 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:44.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f5554840700 1 --2- 172.21.15.148:0/3272963259 >> v2:172.21.15.161:3300/0 conn(0x7f555015a230 0x7f555006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 -- 172.21.15.148:0/3272963259 >> v2:172.21.15.161:3300/0 conn(0x7f555015a230 msgr2=0x7f555006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 --2- 172.21.15.148:0/3272963259 >> v2:172.21.15.161:3300/0 conn(0x7f555015a230 0x7f555006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 -- 172.21.15.148:0/3272963259 >> v1:172.21.15.148:6789/0 conn(0x7f5550159e60 legacy=0x7f55500b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 -- 172.21.15.148:0/3272963259 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500d0480 con 0x7f55500cffa0 2023-10-10T20:28:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554effd700 1 -- 172.21.15.148:0/3272963259 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972179969 0 0) 0x7f55500b3d40 con 0x7f5550159e60 2023-10-10T20:28:44.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554f7fe700 1 --2- 172.21.15.148:0/3272963259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555006a6a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5550071520 tx=0x7f5544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5ba035d379499c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554effd700 1 -- 172.21.15.148:0/3272963259 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400b860 con 0x7f55500cffa0 2023-10-10T20:28:44.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554effd700 1 -- 172.21.15.148:0/3272963259 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400b9c0 con 0x7f55500cffa0 2023-10-10T20:28:44.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.614+0000 7f554effd700 1 -- 172.21.15.148:0/3272963259 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400bd60 con 0x7f55500cffa0 2023-10-10T20:28:44.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 -- 172.21.15.148:0/3272963259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 msgr2=0x7f555006a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 --2- 172.21.15.148:0/3272963259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555006a6a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5550071520 tx=0x7f5544009250 comp rx=0 tx=0).stop 2023-10-10T20:28:44.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 -- 172.21.15.148:0/3272963259 shutdown_connections 2023-10-10T20:28:44.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 --2- 172.21.15.148:0/3272963259 >> v2:172.21.15.161:3300/0 conn(0x7f555015a230 0x7f555006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 --2- 172.21.15.148:0/3272963259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555006a6a0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 -- 172.21.15.148:0/3272963259 >> 172.21.15.148:0/3272963259 conn(0x7f55500d1a50 msgr2=0x7f5550068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 -- 172.21.15.148:0/3272963259 shutdown_connections 2023-10-10T20:28:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.615+0000 7f55562a3700 1 -- 172.21.15.148:0/3272963259 wait complete. 2023-10-10T20:28:44.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.616+0000 7f55562a3700 1 Processor -- start 2023-10-10T20:28:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.616+0000 7f55562a3700 1 -- start start 2023-10-10T20:28:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5550159e60 0x7f5550158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555015a230 0x7f555015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500b15f0 con 0x7f55500cffa0 2023-10-10T20:28:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500b0970 con 0x7f555015a230 2023-10-10T20:28:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55500b0c50 con 0x7f5550159e60 2023-10-10T20:28:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58578/0 (socket says 172.21.15.148:58578) 2023-10-10T20:28:44.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 -- 172.21.15.148:0/3826680832 learned_addr learned my addr 172.21.15.148:0/3826680832 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 -- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5550159e60 msgr2=0x7f5550158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554f7fe700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5550159e60 0x7f5550158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f5554840700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555015a230 0x7f555015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5550159e60 0x7f5550158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 -- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555015a230 msgr2=0x7f555015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555015a230 0x7f555015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 -- 172.21.15.148:0/3826680832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440094f0 con 0x7f55500cffa0 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f554ffff700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f55400011e0 tx=0x7f5540007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540007300 con 0x7f55500cffa0 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5540007460 con 0x7f55500cffa0 2023-10-10T20:28:44.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55500b0f30 con 0x7f55500cffa0 2023-10-10T20:28:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540004830 con 0x7f55500cffa0 2023-10-10T20:28:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.617+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5550158cc0 con 0x7f55500cffa0 2023-10-10T20:28:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.618+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555004c3c0 con 0x7f55500cffa0 2023-10-10T20:28:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.619+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5540004be0 con 0x7f55500cffa0 2023-10-10T20:28:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.619+0000 7f5537fff700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 0x7f553c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:44.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.619+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5540083a70 con 0x7f55500cffa0 2023-10-10T20:28:44.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.619+0000 7f554f7fe700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 0x7f553c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:44.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.620+0000 7f554f7fe700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 0x7f553c0693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5550071520 tx=0x7f5544017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:44.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.624+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5540050d60 con 0x7f55500cffa0 2023-10-10T20:28:44.852 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.850+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55500d0760 con 0x7f55500cffa0 2023-10-10T20:28:44.853 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.850+0000 7f5537fff700 1 -- 172.21.15.148:0/3826680832 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f5540018020 con 0x7f55500cffa0 2023-10-10T20:28:44.853 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:44.853 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:28:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 msgr2=0x7f553c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 0x7f553c0693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5550071520 tx=0x7f5544017000 comp rx=0 tx=0).stop 2023-10-10T20:28:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 msgr2=0x7f555013f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:44.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f55400011e0 tx=0x7f5540007e10 comp rx=0 tx=0).stop 2023-10-10T20:28:44.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 shutdown_connections 2023-10-10T20:28:44.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f553c066f30 0x7f553c0693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555015a230 0x7f555015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500cffa0 0x7f555013f1b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 --2- 172.21.15.148:0/3826680832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5550159e60 0x7f5550158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:44.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 >> 172.21.15.148:0/3826680832 conn(0x7f55500d1a50 msgr2=0x7f55500b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:44.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 shutdown_connections 2023-10-10T20:28:44.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:44.852+0000 7f55562a3700 1 -- 172.21.15.148:0/3826680832 wait complete. 2023-10-10T20:28:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:28:44.936 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-10-10T20:28:44.935+0000 7f5014c2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:28:45.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:28:39.000301+0000', 'modified': '2023-10-10T20:28:42.647073+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7809}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7809': {'gid': 7809, 'name': 'b', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/1416347797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 1416347797}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1416347797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:28:45.270 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:28:45.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:28:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.384+0000 7fef5b59e700 1 Processor -- start 2023-10-10T20:28:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.384+0000 7fef5b59e700 1 -- start start 2023-10-10T20:28:45.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5b59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fef5c159da0 0x7fef5c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5b59e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef5c069210 0x7fef5c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5b59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5c15ea70 con 0x7fef5c15a170 2023-10-10T20:28:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5b59e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5c0b3990 con 0x7fef5c069210 2023-10-10T20:28:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5b59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5c0b0b70 con 0x7fef5c159da0 2023-10-10T20:28:45.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef5c069210 0x7fef5c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef5c069210 0x7fef5c0af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58588/0 (socket says 172.21.15.148:58588) 2023-10-10T20:28:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef59d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef5c15a170 0x7fef5c068b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43524/0 (socket says 172.21.15.148:43524) 2023-10-10T20:28:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 -- 172.21.15.148:0/3864308805 learned_addr learned my addr 172.21.15.148:0/3864308805 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5a59c700 1 --2- 172.21.15.148:0/3864308805 >> v2:172.21.15.161:3300/0 conn(0x7fef5c159da0 0x7fef5c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 -- 172.21.15.148:0/3864308805 >> v2:172.21.15.161:3300/0 conn(0x7fef5c159da0 msgr2=0x7fef5c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/3864308805 >> v2:172.21.15.161:3300/0 conn(0x7fef5c159da0 0x7fef5c0b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 -- 172.21.15.148:0/3864308805 >> v1:172.21.15.148:6789/0 conn(0x7fef5c15a170 legacy=0x7fef5c068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5959a700 1 -- 172.21.15.148:0/3864308805 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003168569 0 0) 0x7fef5c15ea70 con 0x7fef5c15a170 2023-10-10T20:28:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 -- 172.21.15.148:0/3864308805 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c15def0 con 0x7fef5c069210 2023-10-10T20:28:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.385+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/3864308805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c069210 0x7fef5c0af4a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fef5c06b290 tx=0x7fef4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17db65582a7280e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5959a700 1 -- 172.21.15.148:0/3864308805 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00b860 con 0x7fef5c069210 2023-10-10T20:28:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5959a700 1 -- 172.21.15.148:0/3864308805 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c00b9c0 con 0x7fef5c069210 2023-10-10T20:28:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5959a700 1 -- 172.21.15.148:0/3864308805 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00bd80 con 0x7fef5c069210 2023-10-10T20:28:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 -- 172.21.15.148:0/3864308805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c069210 msgr2=0x7fef5c0af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 --2- 172.21.15.148:0/3864308805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c069210 0x7fef5c0af4a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fef5c06b290 tx=0x7fef4c009250 comp rx=0 tx=0).stop 2023-10-10T20:28:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 -- 172.21.15.148:0/3864308805 shutdown_connections 2023-10-10T20:28:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 --2- 172.21.15.148:0/3864308805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c069210 0x7fef5c0af4a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 --2- 172.21.15.148:0/3864308805 >> v2:172.21.15.161:3300/0 conn(0x7fef5c159da0 0x7fef5c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 -- 172.21.15.148:0/3864308805 >> 172.21.15.148:0/3864308805 conn(0x7fef5c0d0bc0 msgr2=0x7fef5c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 -- 172.21.15.148:0/3864308805 shutdown_connections 2023-10-10T20:28:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.386+0000 7fef5b59e700 1 -- 172.21.15.148:0/3864308805 wait complete. 2023-10-10T20:28:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.387+0000 7fef5b59e700 1 Processor -- start 2023-10-10T20:28:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.387+0000 7fef5b59e700 1 -- start start 2023-10-10T20:28:45.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 0x7fef5c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef5c159da0 0x7fef5c0c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0c5980 con 0x7fef5c15a170 2023-10-10T20:28:45.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0c4b60 con 0x7fef5c069210 2023-10-10T20:28:45.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c0c4e00 con 0x7fef5c159da0 2023-10-10T20:28:45.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58592/0 (socket says 172.21.15.148:58592) 2023-10-10T20:28:45.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 -- 172.21.15.148:0/2854110985 learned_addr learned my addr 172.21.15.148:0/2854110985 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 0x7fef5c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 -- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef5c159da0 msgr2=0x7fef5c0c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef5c159da0 0x7fef5c0c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 -- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 msgr2=0x7fef5c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 0x7fef5c0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5ad9d700 1 -- 172.21.15.148:0/2854110985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0094f0 con 0x7fef5c15a170 2023-10-10T20:28:45.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.388+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 0x7fef5c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:45.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fef5c06b5c0 tx=0x7fef4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef527fc700 1 -- 172.21.15.148:0/2854110985 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c01f070 con 0x7fef5c15a170 2023-10-10T20:28:45.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef527fc700 1 -- 172.21.15.148:0/2854110985 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c006b10 con 0x7fef5c15a170 2023-10-10T20:28:45.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c0c50e0 con 0x7fef5c15a170 2023-10-10T20:28:45.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef527fc700 1 -- 172.21.15.148:0/2854110985 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00b860 con 0x7fef5c15a170 2023-10-10T20:28:45.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.389+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0c56a0 con 0x7fef5c15a170 2023-10-10T20:28:45.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.390+0000 7fef527fc700 1 -- 172.21.15.148:0/2854110985 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef4c01aaf0 con 0x7fef5c15a170 2023-10-10T20:28:45.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.390+0000 7fef527fc700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 0x7fef3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.391+0000 7fef527fc700 1 -- 172.21.15.148:0/2854110985 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef4c085c90 con 0x7fef5c15a170 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.391+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 0x7fef3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.391+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 0x7fef3c0693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef54007540 tx=0x7fef5400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 Processor -- start 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 -- start start 2023-10-10T20:28:45.408 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1835f0 0x55ca0c1d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c0dca70 con 0x55ca0c185b80 2023-10-10T20:28:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c0da270 con 0x55ca0c1d0480 2023-10-10T20:28:45.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.393+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c0d6060 con 0x55ca0c1835f0 2023-10-10T20:28:45.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58598/0 (socket says 172.21.15.148:58598) 2023-10-10T20:28:45.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 -- 172.21.15.148:0/1553532658 learned_addr learned my addr 172.21.15.148:0/1553532658 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.410 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5a59c700 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef59d9b700 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1835f0 0x55ca0c1d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 -- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1835f0 msgr2=0x55ca0c1d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1835f0 0x55ca0c1d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 -- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 msgr2=0x55ca0c1af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 -- 172.21.15.148:0/1553532658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0bf70270 con 0x55ca0c185b80 2023-10-10T20:28:45.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0841e0 tx=0x7fef4c084110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e08eeb8e0be5eff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef3b7fe700 1 -- 172.21.15.148:0/1553532658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c056c40 con 0x55ca0c185b80 2023-10-10T20:28:45.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef3b7fe700 1 -- 172.21.15.148:0/1553532658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c059e90 con 0x55ca0c185b80 2023-10-10T20:28:45.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.394+0000 7fef3b7fe700 1 -- 172.21.15.148:0/1553532658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c05dbe0 con 0x55ca0c185b80 2023-10-10T20:28:45.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- 172.21.15.148:0/1553532658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 msgr2=0x55ca0c185ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0841e0 tx=0x7fef4c084110 comp rx=0 tx=0).stop 2023-10-10T20:28:45.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- 172.21.15.148:0/1553532658 shutdown_connections 2023-10-10T20:28:45.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c185b80 0x55ca0c185ff0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1553532658 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1835f0 0x55ca0c1d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- 172.21.15.148:0/1553532658 >> 172.21.15.148:0/1553532658 conn(0x55ca0c1870c0 msgr2=0x55ca0c1878f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- 172.21.15.148:0/1553532658 shutdown_connections 2023-10-10T20:28:45.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- 172.21.15.148:0/1553532658 wait complete. 2023-10-10T20:28:45.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.395+0000 7fef69b39b80 1 -- start start 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 Processor -- start 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c081890 con 0x55ca0c1d0480 2023-10-10T20:28:45.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c09cea0 con 0x55ca0c1835f0 2023-10-10T20:28:45.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca0bf58e30 con 0x55ca0c185b80 2023-10-10T20:28:45.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.396+0000 7fef69b39b80 10 client.0 _ll_register_callbacks cb 0x55ca0c1492c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:28:45.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef59d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5a59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:58520/0 (socket says 172.21.15.148:58520) 2023-10-10T20:28:45.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58612/0 (socket says 172.21.15.148:58612) 2023-10-10T20:28:45.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5a59c700 1 -- 172.21.15.148:0/594719453 learned_addr learned my addr 172.21.15.148:0/594719453 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 -- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 msgr2=0x55ca0c1a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 -- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 msgr2=0x55ca0c1a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 -- 172.21.15.148:0/594719453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0c56a0 con 0x55ca0c1d0480 2023-10-10T20:28:45.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5a59c700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:45.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef59d9b700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:28:45.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fef4c053140 tx=0x7fef4c06b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e3c681386d1405 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:28:45.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c061070 con 0x55ca0c1d0480 2023-10-10T20:28:45.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef4c006d10 con 0x55ca0c1d0480 2023-10-10T20:28:45.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca0c17acd0 con 0x55ca0c1d0480 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c056130 con 0x55ca0c1d0480 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef69b39b80 10 client.7956 fetch_fsmap learned FSMap version 78 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55ca0bf12fe0 con 0x55ca0c1d0480 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.397+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0c17acd0 con 0x55ca0c1d0480 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef4c05abd0 con 0x55ca0c1d0480 2023-10-10T20:28:45.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 10 client.7956 fetch_fsmap finished waiting for FSMap version 78 2023-10-10T20:28:45.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 10 client.7956 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:28:45.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 20 client.7956 populate_metadata read hostname 'smithi148' 2023-10-10T20:28:45.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c1cce80 0x55ca0c1b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 4 client.7956 mds_command: new command op to 7809 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:28:45.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 --> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca0c157aa0 con 0x55ca0c1cce80 2023-10-10T20:28:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef5a59c700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c1cce80 0x55ca0c1b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef38ff9700 10 client.7956 ms_handle_connect on v2:172.21.15.161:6833/1416347797 2023-10-10T20:28:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.398+0000 7fef5a59c700 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c1cce80 0x55ca0c1b0050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe5e58c1cec04f5 server_cookie=49cf64f4769d9e7c in_seq=0 out_seq=0 2023-10-10T20:28:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef38ff9700 1 -- 172.21.15.148:0/594719453 <== mds.0 v2:172.21.15.161:6833/1416347797 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ca0c157aa0 con 0x55ca0c1cce80 2023-10-10T20:28:45.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef38ff9700 10 client.7956 handle_command_reply: tid=0 2023-10-10T20:28:45.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 client.7956 shutdown 2023-10-10T20:28:45.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 msgr2=0x55ca0c185fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fef4c053140 tx=0x7fef4c06b000 comp rx=0 tx=0).stop 2023-10-10T20:28:45.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 shutdown_connections 2023-10-10T20:28:45.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c1cce80 0x55ca0c1b0050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c185fb0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef38ff9700 0 client.7956 ms_handle_reset on v2:172.21.15.161:6833/1416347797 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1835f0 0x55ca0c1a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 --2- 172.21.15.148:0/594719453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 >> 172.21.15.148:0/594719453 conn(0x55ca0c1870c0 msgr2=0x55ca0c189560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 shutdown_connections 2023-10-10T20:28:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.399+0000 7fef69b39b80 1 -- 172.21.15.148:0/594719453 wait complete. 2023-10-10T20:28:45.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.400+0000 7fef69b39b80 20 client.7956 trim_cache size 0 max 16384 2023-10-10T20:28:45.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.416+0000 7fef69b39b80 1 Processor -- start 2023-10-10T20:28:45.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.416+0000 7fef69b39b80 1 -- start start 2023-10-10T20:28:45.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c1aedc0 con 0x55ca0c1835f0 2023-10-10T20:28:45.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c183420 con 0x55ca0c1d0480 2023-10-10T20:28:45.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c1a7470 con 0x55ca0c185b80 2023-10-10T20:28:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58628/0 (socket says 172.21.15.148:58628) 2023-10-10T20:28:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 -- 172.21.15.148:0/2837350777 learned_addr learned my addr 172.21.15.148:0/2837350777 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 -- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 msgr2=0x55ca0c1a7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 -- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 msgr2=0x55ca0c1add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 -- 172.21.15.148:0/2837350777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c0a4780 con 0x55ca0c1835f0 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.417+0000 7fef5a59c700 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef54007c90 tx=0x7fef5401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da648bba949443ea server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef3bfff700 1 -- 172.21.15.148:0/2837350777 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5402a070 con 0x55ca0c1835f0 2023-10-10T20:28:45.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef3bfff700 1 -- 172.21.15.148:0/2837350777 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5401f040 con 0x55ca0c1835f0 2023-10-10T20:28:45.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef3bfff700 1 -- 172.21.15.148:0/2837350777 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5401e040 con 0x55ca0c1835f0 2023-10-10T20:28:45.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 -- 172.21.15.148:0/2837350777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 msgr2=0x55ca0c181e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef54007c90 tx=0x7fef5401b040 comp rx=0 tx=0).stop 2023-10-10T20:28:45.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 -- 172.21.15.148:0/2837350777 shutdown_connections 2023-10-10T20:28:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c1d0480 0x55ca0c1add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1835f0 0x55ca0c181e60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 --2- 172.21.15.148:0/2837350777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c185b80 0x55ca0c1a7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 -- 172.21.15.148:0/2837350777 >> 172.21.15.148:0/2837350777 conn(0x55ca0c1870c0 msgr2=0x55ca0c1b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 -- 172.21.15.148:0/2837350777 shutdown_connections 2023-10-10T20:28:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.418+0000 7fef69b39b80 1 -- 172.21.15.148:0/2837350777 wait complete. 2023-10-10T20:28:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.419+0000 7fef69b39b80 1 -- start start 2023-10-10T20:28:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.419+0000 7fef69b39b80 1 Processor -- start 2023-10-10T20:28:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c185b80 0x55ca0c2239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1881c0 0x55ca0c229cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c232a80 con 0x55ca0c1d0480 2023-10-10T20:28:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c1ae7b0 con 0x55ca0c185b80 2023-10-10T20:28:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c187490 con 0x55ca0c1881c0 2023-10-10T20:28:45.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 10 client.0 _ll_register_callbacks cb 0x55ca0c1492c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:28:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58632/0 (socket says 172.21.15.148:58632) 2023-10-10T20:28:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 -- 172.21.15.148:0/1230122627 learned_addr learned my addr 172.21.15.148:0/1230122627 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef5ad9d700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1881c0 0x55ca0c229cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef5a59c700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c185b80 0x55ca0c2239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 -- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1881c0 msgr2=0x55ca0c229cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1881c0 0x55ca0c229cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 -- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c185b80 msgr2=0x55ca0c2239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c185b80 0x55ca0c2239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 -- 172.21.15.148:0/1230122627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef54019750 con 0x55ca0c1d0480 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef59d9b700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0b1e50 tx=0x7fef44007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb1aa301f74813b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44012e80 con 0x55ca0c1d0480 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef440145b0 con 0x55ca0c1d0480 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef4c059e90 con 0x55ca0c1d0480 2023-10-10T20:28:45.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef44014980 con 0x55ca0c1d0480 2023-10-10T20:28:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 10 client.7962 fetch_fsmap learned FSMap version 78 2023-10-10T20:28:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.420+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55ca0c08b470 con 0x55ca0c1d0480 2023-10-10T20:28:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c059e90 con 0x55ca0c1d0480 2023-10-10T20:28:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef44014ae0 con 0x55ca0c1d0480 2023-10-10T20:28:45.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 10 client.7962 fetch_fsmap finished waiting for FSMap version 78 2023-10-10T20:28:45.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 10 client.7962 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:28:45.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 20 client.7962 populate_metadata read hostname 'smithi148' 2023-10-10T20:28:45.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c232000 0x55ca0c1e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 4 client.7962 mds_command: new command op to 7809 tid=0[{"prefix": "status"}] 2023-10-10T20:28:45.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 --> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca0c078e00 con 0x55ca0c232000 2023-10-10T20:28:45.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef5a59c700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c232000 0x55ca0c1e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef38ff9700 10 client.7962 ms_handle_connect on v2:172.21.15.161:6833/1416347797 2023-10-10T20:28:45.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef5a59c700 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c232000 0x55ca0c1e56b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65dcf12a0147e4df server_cookie=7c9d28d671555992 in_seq=0 out_seq=0 2023-10-10T20:28:45.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef38ff9700 1 -- 172.21.15.148:0/1230122627 <== mds.0 v2:172.21.15.161:6833/1416347797 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55ca0c078e00 con 0x55ca0c232000 2023-10-10T20:28:45.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef38ff9700 10 client.7962 handle_command_reply: tid=0 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.421+0000 7fef69b39b80 1 client.7962 shutdown 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 msgr2=0x55ca0c229790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0b1e50 tx=0x7fef44007ef0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 shutdown_connections 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:6833/1416347797,v1:172.21.15.161:6835/1416347797] conn(0x55ca0c232000 0x55ca0c1e56b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55ca0c1881c0 0x55ca0c229cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca0c185b80 0x55ca0c2239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 --2- 172.21.15.148:0/1230122627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca0c1d0480 0x55ca0c229790 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef38ff9700 0 client.7962 ms_handle_reset on v2:172.21.15.161:6833/1416347797 2023-10-10T20:28:45.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 >> 172.21.15.148:0/1230122627 conn(0x55ca0c1870c0 msgr2=0x55ca0c181e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 shutdown_connections 2023-10-10T20:28:45.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 1 -- 172.21.15.148:0/1230122627 wait complete. 2023-10-10T20:28:45.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef69b39b80 20 client.7962 trim_cache size 0 max 16384 2023-10-10T20:28:45.447 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:28:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:28:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:28:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 7809, 2023-10-10T20:28:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:28:45.448 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:28:45.449 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:28:45.449 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.7724467429999997, 2023-10-10T20:28:45.449 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 78, 2023-10-10T20:28:45.449 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 162, 2023-10-10T20:28:45.449 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 162, 2023-10-10T20:28:45.450 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.884012113000001 2023-10-10T20:28:45.450 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:28:45.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 msgr2=0x7fef3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 0x7fef3c0693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef54007540 tx=0x7fef5400b040 comp rx=0 tx=0).stop 2023-10-10T20:28:45.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 msgr2=0x7fef5c1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fef5c06b5c0 tx=0x7fef4c017000 comp rx=0 tx=0).stop 2023-10-10T20:28:45.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 shutdown_connections 2023-10-10T20:28:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fef3c066f30 0x7fef3c0693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef5c15a170 0x7fef5c1571c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef5c069210 0x7fef5c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 --2- 172.21.15.148:0/2854110985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fef5c159da0 0x7fef5c0c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 >> 172.21.15.148:0/2854110985 conn(0x7fef5c0d0bc0 msgr2=0x7fef5c0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 shutdown_connections 2023-10-10T20:28:45.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.422+0000 7fef5b59e700 1 -- 172.21.15.148:0/2854110985 wait complete. 2023-10-10T20:28:45.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:28:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.566+0000 7fd5f63fd700 1 Processor -- start 2023-10-10T20:28:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.566+0000 7fd5f63fd700 1 -- start start 2023-10-10T20:28:45.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f63fd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd5f0157870 0x7fd5f015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f63fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f63fd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f015cbb0 con 0x7fd5f015d580 2023-10-10T20:28:45.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f63fd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f015bfa0 con 0x7fd5f0157870 2023-10-10T20:28:45.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f63fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f015c280 con 0x7fd5f015a620 2023-10-10T20:28:45.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f499a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd5f015d580 0x7fd5f0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43538/0 (socket says 172.21.15.148:43538) 2023-10-10T20:28:45.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5f499a700 1 -- 172.21.15.148:0/1730619971 learned_addr learned my addr 172.21.15.148:0/1730619971 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:45.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5effff700 1 --2- 172.21.15.148:0/1730619971 >> v2:172.21.15.161:3300/0 conn(0x7fd5f0157870 0x7fd5f015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.567+0000 7fd5ef7fe700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5eeffd700 1 -- 172.21.15.148:0/1730619971 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075858626 0 0) 0x7fd5f015cbb0 con 0x7fd5f015d580 2023-10-10T20:28:45.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5eeffd700 1 -- 172.21.15.148:0/1730619971 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d8003570 con 0x7fd5f015d580 2023-10-10T20:28:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5effff700 1 -- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 msgr2=0x7fd5f015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5effff700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5effff700 1 -- 172.21.15.148:0/1730619971 >> v1:172.21.15.148:6789/0 conn(0x7fd5f015d580 legacy=0x7fd5f0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5effff700 1 -- 172.21.15.148:0/1730619971 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f015c560 con 0x7fd5f0157870 2023-10-10T20:28:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5effff700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f0157870 0x7fd5f015b6a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00098e0 tx=0x7fd5e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ba8859c7b3eb81 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5eeffd700 1 -- 172.21.15.148:0/1730619971 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011630 con 0x7fd5f0157870 2023-10-10T20:28:45.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5eeffd700 1 -- 172.21.15.148:0/1730619971 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0011790 con 0x7fd5f0157870 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.568+0000 7fd5eeffd700 1 -- 172.21.15.148:0/1730619971 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011b10 con 0x7fd5f0157870 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 -- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f0157870 msgr2=0x7fd5f015b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f0157870 0x7fd5f015b6a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00098e0 tx=0x7fd5e00091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 -- 172.21.15.148:0/1730619971 shutdown_connections 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f0157870 0x7fd5f015b6a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/1730619971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 -- 172.21.15.148:0/1730619971 >> 172.21.15.148:0/1730619971 conn(0x7fd5f0068ce0 msgr2=0x7fd5f00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 -- 172.21.15.148:0/1730619971 shutdown_connections 2023-10-10T20:28:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.569+0000 7fd5f63fd700 1 -- 172.21.15.148:0/1730619971 wait complete. 2023-10-10T20:28:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.570+0000 7fd5f63fd700 1 Processor -- start 2023-10-10T20:28:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.570+0000 7fd5f63fd700 1 -- start start 2023-10-10T20:28:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f0154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f015d580 0x7fd5f0154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00cfc80 con 0x7fd5f0157870 2023-10-10T20:28:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00d1510 con 0x7fd5f015d580 2023-10-10T20:28:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f63fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00cf350 con 0x7fd5f015a620 2023-10-10T20:28:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58642/0 (socket says 172.21.15.148:58642) 2023-10-10T20:28:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 -- 172.21.15.148:0/2734613014 learned_addr learned my addr 172.21.15.148:0/2734613014 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5f499a700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f015d580 0x7fd5f0154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5ef7fe700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f0154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 -- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 msgr2=0x7fd5f0154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f0154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 -- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f015d580 msgr2=0x7fd5f0154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f015d580 0x7fd5f0154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 -- 172.21.15.148:0/2734613014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009420 con 0x7fd5f0157870 2023-10-10T20:28:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.571+0000 7fd5effff700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd5f006b530 tx=0x7fd5e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011900 con 0x7fd5f0157870 2023-10-10T20:28:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0011a60 con 0x7fd5f0157870 2023-10-10T20:28:45.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f00cf630 con 0x7fd5f0157870 2023-10-10T20:28:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e001a680 con 0x7fd5f0157870 2023-10-10T20:28:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0149450 con 0x7fd5f0157870 2023-10-10T20:28:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.572+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f004c3c0 con 0x7fd5f0157870 2023-10-10T20:28:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.573+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5e001aa30 con 0x7fd5f0157870 2023-10-10T20:28:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.573+0000 7fd5d7fff700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.573+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd5e0086090 con 0x7fd5f0157870 2023-10-10T20:28:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.573+0000 7fd5ef7fe700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.574+0000 7fd5ef7fe700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 0x7fd5dc0693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd5f014e6c0 tx=0x7fd5e40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.578+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5e0053300 con 0x7fd5f0157870 2023-10-10T20:28:45.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.807+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd5f015c840 con 0x7fd5f0157870 2023-10-10T20:28:45.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.808+0000 7fd5d7fff700 1 -- 172.21.15.148:0/2734613014 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd5e0056920 con 0x7fd5f0157870 2023-10-10T20:28:45.810 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:28:45.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.809+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 msgr2=0x7fd5dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.809+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 0x7fd5dc0693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd5f014e6c0 tx=0x7fd5e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.809+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 msgr2=0x7fd5f0148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:45.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.809+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd5f006b530 tx=0x7fd5e0017000 comp rx=0 tx=0).stop 2023-10-10T20:28:45.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 shutdown_connections 2023-10-10T20:28:45.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5f015d580 0x7fd5f0154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f0157870 0x7fd5f0148f10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 --2- 172.21.15.148:0/2734613014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5f015a620 0x7fd5f0154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:45.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 >> 172.21.15.148:0/2734613014 conn(0x7fd5f0068ce0 msgr2=0x7fd5f0159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:45.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 shutdown_connections 2023-10-10T20:28:45.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:45.810+0000 7fd5f63fd700 1 -- 172.21.15.148:0/2734613014 wait complete. 2023-10-10T20:28:45.824 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:28:45.825 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:28:45.825 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:28:45.825 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:28:45.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:28:45.826 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:28:45.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:28:45.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:28:45.827 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:28:45.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:45.828 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:28:45.843 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:28:45.844 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:28:45.844 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:28:45.844 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.844 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:28:45.845 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.845 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:45.845 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:45.845 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:28:45.845 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:45.846 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:45.846 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:28:45.846 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:45.846 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:45.846 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:28:45.847 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:45.847 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:28:45.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 41985sec preferred_lft 41985sec 2023-10-10T20:28:45.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:28:45.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.848 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:28:45.849 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:45.849 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:28:45.849 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.849 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:28:45.849 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.850 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:28:45.850 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:28:45.850 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:28:45.850 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.850 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:28:45.851 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:28:45.851 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:28:45.851 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:45.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:45.852 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:28:45.907 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:28:45.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:45.908 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:28:45.964 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:28:45.964 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:28:45.964 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:28:45.965 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:28:45.965 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:28:45.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:45.966 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:28:45.966 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:28:45.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:28:45.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:28:45.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:28:45.968 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:28:46.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.064 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:28:46.079 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:28:46.079 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:28:46.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:28:46.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:28:46.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:28:46.144 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:28:46.199 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:28:46.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:28:46.201 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:28:46.254 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:28:46.255 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:28:46.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:46.334 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:46.334 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:46.352 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:46.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:46.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:28:46.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:28:46.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:28:46.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:28:46.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:28:46.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:28:46.375 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:46.375 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:46.375 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:28:46.375 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:28:46.376 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:28:46.376 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:28:46.376 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:28:46.376 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:28:46.376 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:28:46.377 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:28:46.377 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:28:46.378 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:28:46.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:28:46.443 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:28:46.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.595+0000 7f113c5a3700 1 Processor -- start 2023-10-10T20:28:46.598 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.595+0000 7f113c5a3700 1 -- start start 2023-10-10T20:28:46.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f113c5a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1134152010 0x7f113415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f113c5a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f11340cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f113c5a3700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113406c010 con 0x7f11341523e0 2023-10-10T20:28:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f113c5a3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113406bd10 con 0x7f1134152010 2023-10-10T20:28:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f113c5a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113415c060 con 0x7f1134158e50 2023-10-10T20:28:46.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f1139b3e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f11341523e0 0x7f113406c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43542/0 (socket says 172.21.15.148:43542) 2023-10-10T20:28:46.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.596+0000 7f1139b3e700 1 -- 172.21.15.148:0/2515919753 learned_addr learned my addr 172.21.15.148:0/2515919753 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:28:46.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113a33f700 1 --2- 172.21.15.148:0/2515919753 >> v2:172.21.15.161:3300/0 conn(0x7f1134152010 0x7f113415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f11340cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167261516 0 0) 0x7f113406c010 con 0x7f11341523e0 2023-10-10T20:28:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1120003570 con 0x7f11341523e0 2023-10-10T20:28:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034896993 0 0) 0x7f1120003570 con 0x7f11341523e0 2023-10-10T20:28:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 msgr2=0x7f11340cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f11340cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.610 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 >> v2:172.21.15.161:3300/0 conn(0x7f1134152010 msgr2=0x7f113415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 --2- 172.21.15.148:0/2515919753 >> v2:172.21.15.161:3300/0 conn(0x7f1134152010 0x7f113415b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113415c340 con 0x7f11341523e0 2023-10-10T20:28:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f1124001470 con 0x7f11341523e0 2023-10-10T20:28:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 >> v1:172.21.15.148:6789/0 conn(0x7f11341523e0 legacy=0x7f113406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f1120003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120003fe0 0x7f1120004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1120004d50 0x7f11200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113406bd10 con 0x7f1120004d50 2023-10-10T20:28:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113415c060 con 0x7f1120003fe0 2023-10-10T20:28:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112000ba90 con 0x7f1134158e50 2023-10-10T20:28:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1120004d50 0x7f11200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113a33f700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f1120003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f1139b3e700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120003fe0 0x7f1120004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 -- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 msgr2=0x7f1120003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1134158e50 0x7f1120003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 -- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120003fe0 msgr2=0x7f1120004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120003fe0 0x7f1120004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.597+0000 7f113ab40700 1 -- 172.21.15.148:0/2515919753 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120003570 con 0x7f1120004d50 2023-10-10T20:28:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113ab40700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1120004d50 0x7f11200051c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f112c00e210 tx=0x7f112c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ae1d6abb65b0ae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ea60 con 0x7f1120004d50 2023-10-10T20:28:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00f040 con 0x7f1120004d50 2023-10-10T20:28:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113933d700 1 -- 172.21.15.148:0/2515919753 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ee10 con 0x7f1120004d50 2023-10-10T20:28:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 -- 172.21.15.148:0/2515919753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1120004d50 msgr2=0x7f11200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 --2- 172.21.15.148:0/2515919753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1120004d50 0x7f11200051c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f112c00e210 tx=0x7f112c00b040 comp rx=0 tx=0).stop 2023-10-10T20:28:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113ab40700 1 -- 172.21.15.148:0/2515919753 reap_dead start 2023-10-10T20:28:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 -- 172.21.15.148:0/2515919753 shutdown_connections 2023-10-10T20:28:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 -- 172.21.15.148:0/2515919753 >> 172.21.15.148:0/2515919753 conn(0x7f1134069870 msgr2=0x7f1134069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 -- 172.21.15.148:0/2515919753 shutdown_connections 2023-10-10T20:28:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.598+0000 7f113c5a3700 1 -- 172.21.15.148:0/2515919753 wait complete. 2023-10-10T20:28:46.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.599+0000 7f113c5a3700 1 Processor -- start 2023-10-10T20:28:46.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.599+0000 7f113c5a3700 1 -- start start 2023-10-10T20:28:46.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120004d50 0x7f11340b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f113414e530 0x7f1134149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340cb0f0 con 0x7f11340ca740 2023-10-10T20:28:46.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340c63c0 con 0x7f1120004d50 2023-10-10T20:28:46.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113c5a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340b2130 con 0x7f113414e530 2023-10-10T20:28:46.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113ab40700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113ab40700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58652/0 (socket says 172.21.15.148:58652) 2023-10-10T20:28:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113ab40700 1 -- 172.21.15.148:0/677740370 learned_addr learned my addr 172.21.15.148:0/677740370 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f1139b3e700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f113414e530 0x7f1134149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.600+0000 7f113a33f700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120004d50 0x7f11340b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 -- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f113414e530 msgr2=0x7f1134149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f113414e530 0x7f1134149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 -- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120004d50 msgr2=0x7f11340b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120004d50 0x7f11340b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 -- 172.21.15.148:0/677740370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340b2410 con 0x7f11340ca740 2023-10-10T20:28:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113ab40700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f112c016000 tx=0x7f112c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00eee0 con 0x7f11340ca740 2023-10-10T20:28:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c01d630 con 0x7f11340ca740 2023-10-10T20:28:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11340b26f0 con 0x7f11340ca740 2023-10-10T20:28:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c01d9f0 con 0x7f11340ca740 2023-10-10T20:28:46.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.601+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340b1e20 con 0x7f11340ca740 2023-10-10T20:28:46.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.602+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113404c3c0 con 0x7f11340ca740 2023-10-10T20:28:46.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.602+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f112c003040 con 0x7f11340ca740 2023-10-10T20:28:46.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.603+0000 7f112a7fc700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:46.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.603+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f112c086540 con 0x7f11340ca740 2023-10-10T20:28:46.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.603+0000 7f113a33f700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:46.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.603+0000 7f113a33f700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1130006f00 tx=0x7f11300091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:46.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.608+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f112c053870 con 0x7f11340ca740 2023-10-10T20:28:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.833+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11340c6190 con 0x7f11340ca740 2023-10-10T20:28:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.834+0000 7f112a7fc700 1 -- 172.21.15.148:0/677740370 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f112c018060 con 0x7f11340ca740 2023-10-10T20:28:46.837 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:28:46.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 msgr2=0x7f11180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1130006f00 tx=0x7f11300091a0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 msgr2=0x7f11340cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f112c016000 tx=0x7f112c01a000 comp rx=0 tx=0).stop 2023-10-10T20:28:46.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 shutdown_connections 2023-10-10T20:28:46.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1118066f30 0x7f11180693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11340ca740 0x7f11340cabb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1120004d50 0x7f11340b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 --2- 172.21.15.148:0/677740370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f113414e530 0x7f1134149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 >> 172.21.15.148:0/677740370 conn(0x7f1134069870 msgr2=0x7f11340c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 shutdown_connections 2023-10-10T20:28:46.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:46.836+0000 7f113c5a3700 1 -- 172.21.15.148:0/677740370 wait complete. 2023-10-10T20:28:46.851 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:28:46.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:28:46.852 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:28:46.853 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:28:46.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:28:46.854 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:28:46.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:28:46.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:28:46.856 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:28:46.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.857 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:28:46.876 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:28:46.877 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:28:46.877 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:28:46.878 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.878 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:28:46.879 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.879 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:46.879 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:46.880 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:28:46.880 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:46.880 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:46.881 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:28:46.881 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:28:46.882 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:46.882 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:28:46.882 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:46.883 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:28:46.883 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41981sec preferred_lft 41981sec 2023-10-10T20:28:46.883 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:28:46.884 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.884 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:28:46.884 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:28:46.884 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:28:46.885 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.885 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:28:46.885 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.886 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:28:46.886 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:28:46.886 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:28:46.886 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.887 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:28:46.887 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:28:46.887 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:28:46.888 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:28:46.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.889 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:28:46.943 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:28:46.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:46.946 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:28:47.003 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:28:47.004 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:28:47.004 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:28:47.004 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:28:47.005 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:28:47.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:47.006 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:28:47.010 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:28:47.010 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:28:47.011 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:28:47.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:28:47.012 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:28:47.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:47.108 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:28:47.123 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:28:47.123 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:28:47.124 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:28:47.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:47.125 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:28:47.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:28:47.188 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:28:47.243 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:28:47.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:28:47.244 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:28:47.297 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:28:47.298 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:28:47.298 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:47.376 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:47.377 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:47.395 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:47.396 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:47.396 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:28:47.396 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:28:47.396 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:28:47.397 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:28:47.397 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:28:47.397 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:28:47.419 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:28:47.420 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:28:47.420 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:28:47.420 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:28:47.420 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:28:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:28:47.422 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:28:47.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:28:47.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:28:47.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:28:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.599+0000 7f28d3a37700 1 Processor -- start 2023-10-10T20:28:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.599+0000 7f28d3a37700 1 -- start start 2023-10-10T20:28:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28cc0b0e50 0x7f28cc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc0b1220 0x7f28cc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f28cc06dad0 0x7f28cc0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc15de60 con 0x7f28cc0b0e50 2023-10-10T20:28:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc15e140 con 0x7f28cc06dad0 2023-10-10T20:28:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d3a37700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15e420 con 0x7f28cc0b1220 2023-10-10T20:28:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d17d3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28cc0b0e50 0x7f28cc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d17d3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28cc0b0e50 0x7f28cc0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58660/0 (socket says 172.21.15.148:58660) 2023-10-10T20:28:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.600+0000 7f28d17d3700 1 -- 172.21.15.148:0/1398179523 learned_addr learned my addr 172.21.15.148:0/1398179523 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 -- 172.21.15.148:0/1398179523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc0b1220 msgr2=0x7f28cc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d1fd4700 1 --2- 172.21.15.148:0/1398179523 >> v2:172.21.15.161:3300/0 conn(0x7f28cc06dad0 0x7f28cc0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d0fd2700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc0b1220 0x7f28cc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc0b1220 0x7f28cc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 -- 172.21.15.148:0/1398179523 >> v2:172.21.15.161:3300/0 conn(0x7f28cc06dad0 msgr2=0x7f28cc0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 --2- 172.21.15.148:0/1398179523 >> v2:172.21.15.161:3300/0 conn(0x7f28cc06dad0 0x7f28cc0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 -- 172.21.15.148:0/1398179523 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc15e700 con 0x7f28cc0b0e50 2023-10-10T20:28:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d17d3700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0c9180 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009f20 tx=0x7f28c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b571e0a7b91801 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28c3fff700 1 -- 172.21.15.148:0/1398179523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800e070 con 0x7f28cc0b0e50 2023-10-10T20:28:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28c3fff700 1 -- 172.21.15.148:0/1398179523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c800ba40 con 0x7f28cc0b0e50 2023-10-10T20:28:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28c3fff700 1 -- 172.21.15.148:0/1398179523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c80092e0 con 0x7f28cc0b0e50 2023-10-10T20:28:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d3a37700 1 -- 172.21.15.148:0/1398179523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b0e50 msgr2=0x7f28cc0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.601+0000 7f28d3a37700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0c9180 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009f20 tx=0x7f28c8009840 comp rx=0 tx=0).stop 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 -- 172.21.15.148:0/1398179523 shutdown_connections 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 --2- 172.21.15.148:0/1398179523 >> v2:172.21.15.161:3300/0 conn(0x7f28cc06dad0 0x7f28cc0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0c9180 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 --2- 172.21.15.148:0/1398179523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc0b1220 0x7f28cc06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 -- 172.21.15.148:0/1398179523 >> 172.21.15.148:0/1398179523 conn(0x7f28cc069870 msgr2=0x7f28cc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 -- 172.21.15.148:0/1398179523 shutdown_connections 2023-10-10T20:28:47.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.602+0000 7f28d3a37700 1 -- 172.21.15.148:0/1398179523 wait complete. 2023-10-10T20:28:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.603+0000 7f28d3a37700 1 Processor -- start 2023-10-10T20:28:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.603+0000 7f28d3a37700 1 -- start start 2023-10-10T20:28:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 0x7f28cc154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0c5bf0 con 0x7f28cc0b1220 2023-10-10T20:28:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ca320 con 0x7f28cc0b0e50 2023-10-10T20:28:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0b2310 con 0x7f28cc06dad0 2023-10-10T20:28:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d17d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 0x7f28cc154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d17d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 0x7f28cc154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:35648/0 (socket says 172.21.15.148:35648) 2023-10-10T20:28:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d17d3700 1 -- 172.21.15.148:0/825575105 learned_addr learned my addr 172.21.15.148:0/825575105 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58672/0 (socket says 172.21.15.148:58672) 2023-10-10T20:28:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d0fd2700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 -- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 msgr2=0x7f28cc154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 0x7f28cc154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 -- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28cc0b0e50 msgr2=0x7f28cc0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 -- 172.21.15.148:0/825575105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8009ae0 con 0x7f28cc0b1220 2023-10-10T20:28:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d1fd4700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ded0 tx=0x7f28c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011600 con 0x7f28cc0b1220 2023-10-10T20:28:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4011760 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc0b25f0 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011b20 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.604+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc155300 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.605+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc0ca4d0 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.606+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28c4018360 con 0x7f28cc0b1220 2023-10-10T20:28:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.606+0000 7f28c17fa700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 0x7f28b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.606+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28c4083f80 con 0x7f28cc0b1220 2023-10-10T20:28:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.606+0000 7f28d17d3700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 0x7f28b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.606+0000 7f28d17d3700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 0x7f28b00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009ab0 tx=0x7f28c8018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.610+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28cc0ca4d0 con 0x7f28cc0b1220 2023-10-10T20:28:47.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.840+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28cc0ca490 con 0x7f28cc0b1220 2023-10-10T20:28:47.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.840+0000 7f28c17fa700 1 -- 172.21.15.148:0/825575105 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f28c40511f0 con 0x7f28cc0b1220 2023-10-10T20:28:47.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:28:47.842 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:28:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 msgr2=0x7f28b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 0x7f28b00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009ab0 tx=0x7f28c8018000 comp rx=0 tx=0).stop 2023-10-10T20:28:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 msgr2=0x7f28cc0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ded0 tx=0x7f28c4008040 comp rx=0 tx=0).stop 2023-10-10T20:28:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 shutdown_connections 2023-10-10T20:28:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f28b0066f30 0x7f28b00693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28cc0b1220 0x7f28cc0b3dc0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28cc06dad0 0x7f28cc154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 --2- 172.21.15.148:0/825575105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28cc0b0e50 0x7f28cc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 >> 172.21.15.148:0/825575105 conn(0x7f28cc069870 msgr2=0x7f28cc06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:47.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 shutdown_connections 2023-10-10T20:28:47.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:47.842+0000 7f28d3a37700 1 -- 172.21.15.148:0/825575105 wait complete. 2023-10-10T20:28:47.846 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:28:47.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:28:47.990 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":7809,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3291171199999994,"mdsmap_epoch":78,"osdmap_epoch":162,"osdmap_epoch_barrier":162,"uptime":13.440683787999999} 2023-10-10T20:28:47.990 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 7809, "mdsmap_epoch": 78, "osdmap_epoch": 162, "osdmap_epoch_barrier": 162, "rank_uptime": 8.32911712, "state": "up:active", "uptime": 13.440683788, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:28:47.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-10T20:28:48.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.108+0000 7f7326b5e700 1 Processor -- start 2023-10-10T20:28:48.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 -- start start 2023-10-10T20:28:48.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7320070950 0x7f732006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 0x7f7320143650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200c8640 con 0x7f7320070d20 2023-10-10T20:28:48.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200c8e80 con 0x7f7320070950 2023-10-10T20:28:48.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.109+0000 7f7326b5e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732006f630 con 0x7f73201431f0 2023-10-10T20:28:48.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7320070950 0x7f732006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:48.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7320070950 0x7f732006ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58680/0 (socket says 172.21.15.148:58680) 2023-10-10T20:28:48.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f7317fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7320070d20 0x7f7320067e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43556/0 (socket says 172.21.15.148:43556) 2023-10-10T20:28:48.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 -- 172.21.15.148:0/3640923506 learned_addr learned my addr 172.21.15.148:0/3640923506 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:48.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73250fb700 1 --2- 172.21.15.148:0/3640923506 >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 0x7f7320143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:48.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 -- 172.21.15.148:0/3640923506 >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 msgr2=0x7f7320143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:28:48.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 --2- 172.21.15.148:0/3640923506 >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 0x7f7320143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 -- 172.21.15.148:0/3640923506 >> v1:172.21.15.148:6789/0 conn(0x7f7320070d20 legacy=0x7f7320067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 -- 172.21.15.148:0/3640923506 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200c8870 con 0x7f7320070950 2023-10-10T20:28:48.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73177fe700 1 -- 172.21.15.148:0/3640923506 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554465113 0 0) 0x7f73200c8640 con 0x7f7320070d20 2023-10-10T20:28:48.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73250fb700 1 --2- 172.21.15.148:0/3640923506 >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 0x7f7320143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73248fa700 1 --2- 172.21.15.148:0/3640923506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070950 0x7f732006ec70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f732015a570 tx=0x7f731c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a226e936c8a3011 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73177fe700 1 -- 172.21.15.148:0/3640923506 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00e070 con 0x7f7320070950 2023-10-10T20:28:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73177fe700 1 -- 172.21.15.148:0/3640923506 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00ba50 con 0x7f7320070950 2023-10-10T20:28:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.110+0000 7f73177fe700 1 -- 172.21.15.148:0/3640923506 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0092e0 con 0x7f7320070950 2023-10-10T20:28:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 -- 172.21.15.148:0/3640923506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070950 msgr2=0x7f732006ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3640923506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070950 0x7f732006ec70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f732015a570 tx=0x7f731c009840 comp rx=0 tx=0).stop 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 -- 172.21.15.148:0/3640923506 shutdown_connections 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3640923506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070950 0x7f732006ec70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3640923506 >> v2:172.21.15.161:3300/0 conn(0x7f73201431f0 0x7f7320143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 -- 172.21.15.148:0/3640923506 >> 172.21.15.148:0/3640923506 conn(0x7f73200ce570 msgr2=0x7f73200ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:48.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 -- 172.21.15.148:0/3640923506 shutdown_connections 2023-10-10T20:28:48.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.111+0000 7f7326b5e700 1 -- 172.21.15.148:0/3640923506 wait complete. 2023-10-10T20:28:48.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.112+0000 7f7326b5e700 1 Processor -- start 2023-10-10T20:28:48.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.112+0000 7f7326b5e700 1 -- start start 2023-10-10T20:28:48.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 0x7f732015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73201431f0 0x7f732013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732015c470 con 0x7f7320070d20 2023-10-10T20:28:48.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732015e8e0 con 0x7f7320070950 2023-10-10T20:28:48.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b16d0 con 0x7f73201431f0 2023-10-10T20:28:48.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:48.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58688/0 (socket says 172.21.15.148:58688) 2023-10-10T20:28:48.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 -- 172.21.15.148:0/3852064732 learned_addr learned my addr 172.21.15.148:0/3852064732 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f73248fa700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 0x7f732015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 -- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73201431f0 msgr2=0x7f732013e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73201431f0 0x7f732013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 -- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 msgr2=0x7f732015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 0x7f732015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 -- 172.21.15.148:0/3852064732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c009ae0 con 0x7f7320070d20 2023-10-10T20:28:48.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f73248fa700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 0x7f732015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:28:48.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7317fff700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f73100011e0 tx=0x7f7310007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:48.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310007300 con 0x7f7320070d20 2023-10-10T20:28:48.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310007460 con 0x7f7320070d20 2023-10-10T20:28:48.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732015be20 con 0x7f7320070d20 2023-10-10T20:28:48.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310004830 con 0x7f7320070d20 2023-10-10T20:28:48.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.113+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73200b0f90 con 0x7f7320070d20 2023-10-10T20:28:48.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.114+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732015c620 con 0x7f7320070d20 2023-10-10T20:28:48.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.115+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7310004be0 con 0x7f7320070d20 2023-10-10T20:28:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.115+0000 7f7314ff9700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 0x7f730c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:28:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.115+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7310083a40 con 0x7f7320070d20 2023-10-10T20:28:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.115+0000 7f73248fa700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 0x7f730c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:28:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.116+0000 7f73248fa700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 0x7f730c0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f731c009ab0 tx=0x7f731c018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:28:48.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.119+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f732015c620 con 0x7f7320070d20 2023-10-10T20:28:48.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.303+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f732015bc50 con 0x7f7320070d20 2023-10-10T20:28:48.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.678+0000 7f7314ff9700 1 -- 172.21.15.148:0/3852064732 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v163) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7310018020 con 0x7f7320070d20 2023-10-10T20:28:48.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 msgr2=0x7f730c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 0x7f730c0693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f731c009ab0 tx=0x7f731c018000 comp rx=0 tx=0).stop 2023-10-10T20:28:48.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 msgr2=0x7f73200b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:28:48.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f73100011e0 tx=0x7f7310007e10 comp rx=0 tx=0).stop 2023-10-10T20:28:48.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 shutdown_connections 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f730c066f30 0x7f730c0693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73201431f0 0x7f732013e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7320070950 0x7f732015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 --2- 172.21.15.148:0/3852064732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7320070d20 0x7f73200b0a50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 >> 172.21.15.148:0/3852064732 conn(0x7f73200ce570 msgr2=0x7f73201426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:28:48.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 shutdown_connections 2023-10-10T20:28:48.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:28:48.680+0000 7f7326b5e700 1 -- 172.21.15.148:0/3852064732 wait complete. 2023-10-10T20:28:48.684 INFO:teuthology.orchestra.run.smithi148.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-10T20:28:48.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:28:48.824 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:28:48.825 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:28:48.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-10T20:28:48.950 INFO:teuthology.orchestra.run.smithi148.stdout:{"mon_tick_interval":"5"} 2023-10-10T20:28:48.950 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-10T20:28:48.950 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-10T20:28:48.951 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-10T20:28:48.951 DEBUG:teuthology.orchestra.run.smithi148:> import time 2023-10-10T20:28:48.952 DEBUG:teuthology.orchestra.run.smithi148:> import datetime 2023-10-10T20:28:48.952 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2023-10-10T20:28:48.952 DEBUG:teuthology.orchestra.run.smithi148:> import os 2023-10-10T20:28:48.953 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.953 DEBUG:teuthology.orchestra.run.smithi148:> # Write some buffered data through before going full, all should be well 2023-10-10T20:28:48.953 DEBUG:teuthology.orchestra.run.smithi148:> print("writing some data through which we expect to succeed") 2023-10-10T20:28:48.954 DEBUG:teuthology.orchestra.run.smithi148:> bytes = 0 2023-10-10T20:28:48.954 DEBUG:teuthology.orchestra.run.smithi148:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-10T20:28:48.954 DEBUG:teuthology.orchestra.run.smithi148:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-10T20:28:48.955 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-10T20:28:48.955 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:28:48.955 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-10T20:28:48.956 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.956 DEBUG:teuthology.orchestra.run.smithi148:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-10T20:28:48.956 DEBUG:teuthology.orchestra.run.smithi148:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-10T20:28:48.957 DEBUG:teuthology.orchestra.run.smithi148:> # buffered IO, the error should always only appear from fsync and not 2023-10-10T20:28:48.957 DEBUG:teuthology.orchestra.run.smithi148:> # from write 2023-10-10T20:28:48.957 DEBUG:teuthology.orchestra.run.smithi148:> full = False 2023-10-10T20:28:48.958 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.958 DEBUG:teuthology.orchestra.run.smithi148:> for n in range(0, int(32 * 1.1)): 2023-10-10T20:28:48.958 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:28:48.959 DEBUG:teuthology.orchestra.run.smithi148:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-10T20:28:48.959 DEBUG:teuthology.orchestra.run.smithi148:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-10T20:28:48.959 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2023-10-10T20:28:48.960 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2023-10-10T20:28:48.960 DEBUG:teuthology.orchestra.run.smithi148:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-10T20:28:48.960 DEBUG:teuthology.orchestra.run.smithi148:> raise 2023-10-10T20:28:48.961 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:28:48.961 DEBUG:teuthology.orchestra.run.smithi148:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:28:48.961 DEBUG:teuthology.orchestra.run.smithi148:> full = True 2023-10-10T20:28:48.962 DEBUG:teuthology.orchestra.run.smithi148:> break 2023-10-10T20:28:48.962 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.962 DEBUG:teuthology.orchestra.run.smithi148:> try: 2023-10-10T20:28:48.963 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f) 2023-10-10T20:28:48.963 DEBUG:teuthology.orchestra.run.smithi148:> print("fsync'"'"'ed successfully") 2023-10-10T20:28:48.963 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2023-10-10T20:28:48.964 DEBUG:teuthology.orchestra.run.smithi148:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:28:48.964 DEBUG:teuthology.orchestra.run.smithi148:> full = True 2023-10-10T20:28:48.964 DEBUG:teuthology.orchestra.run.smithi148:> break 2023-10-10T20:28:48.965 DEBUG:teuthology.orchestra.run.smithi148:> else: 2023-10-10T20:28:48.965 DEBUG:teuthology.orchestra.run.smithi148:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-10T20:28:48.965 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.965 DEBUG:teuthology.orchestra.run.smithi148:> if n > 32 * 0.9: 2023-10-10T20:28:48.966 DEBUG:teuthology.orchestra.run.smithi148:> # Be cautious in the last region where we expect to hit 2023-10-10T20:28:48.966 DEBUG:teuthology.orchestra.run.smithi148:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-10T20:28:48.967 DEBUG:teuthology.orchestra.run.smithi148:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-10T20:28:48.967 DEBUG:teuthology.orchestra.run.smithi148:> time.sleep(15.0) 2023-10-10T20:28:48.967 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.967 DEBUG:teuthology.orchestra.run.smithi148:> if not full: 2023-10-10T20:28:48.968 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-10T20:28:48.968 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.968 DEBUG:teuthology.orchestra.run.smithi148:> # close() should not raise an error because we already caught it in 2023-10-10T20:28:48.969 DEBUG:teuthology.orchestra.run.smithi148:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-10T20:28:48.969 DEBUG:teuthology.orchestra.run.smithi148:> # since then because all IOs got cancelled on the full flag. 2023-10-10T20:28:48.969 DEBUG:teuthology.orchestra.run.smithi148:> print("calling close") 2023-10-10T20:28:48.970 DEBUG:teuthology.orchestra.run.smithi148:> os.close(f) 2023-10-10T20:28:48.970 DEBUG:teuthology.orchestra.run.smithi148:> print("close() did not raise error") 2023-10-10T20:28:48.970 DEBUG:teuthology.orchestra.run.smithi148:> 2023-10-10T20:28:48.971 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-10T20:28:48.971 DEBUG:teuthology.orchestra.run.smithi148:> ' 2023-10-10T20:28:53.655 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:28:53.659 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:29:23.735 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:29:23.738 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:29:49.696 INFO:teuthology.orchestra.run.smithi148.stdout:writing some data through which we expect to succeed 2023-10-10T20:29:49.697 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-10T20:29:49.697 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.697 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.697 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 1.00 MB 2023-10-10T20:29:49.698 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.698 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.698 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 2.00 MB 2023-10-10T20:29:49.698 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.698 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.699 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 3.00 MB 2023-10-10T20:29:49.699 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.699 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.699 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 4.00 MB 2023-10-10T20:29:49.699 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 5.00 MB 2023-10-10T20:29:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.700 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 6.00 MB 2023-10-10T20:29:49.701 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.701 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.701 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 7.00 MB 2023-10-10T20:29:49.701 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.701 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 8.00 MB 2023-10-10T20:29:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 9.00 MB 2023-10-10T20:29:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 10.00 MB 2023-10-10T20:29:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 11.00 MB 2023-10-10T20:29:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 12.00 MB 2023-10-10T20:29:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 13.00 MB 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 14.00 MB 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.705 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.706 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 15.00 MB 2023-10-10T20:29:49.706 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.706 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.706 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 16.00 MB 2023-10-10T20:29:49.706 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.707 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.707 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 17.00 MB 2023-10-10T20:29:49.707 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.707 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.707 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 18.00 MB 2023-10-10T20:29:49.708 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.708 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.708 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 19.00 MB 2023-10-10T20:29:49.708 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.708 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.709 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 20.00 MB 2023-10-10T20:29:49.709 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.709 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.709 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 21.00 MB 2023-10-10T20:29:49.709 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 22.00 MB 2023-10-10T20:29:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.710 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 23.00 MB 2023-10-10T20:29:49.711 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.711 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.711 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 24.00 MB 2023-10-10T20:29:49.711 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.711 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 25.00 MB 2023-10-10T20:29:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.712 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 26.00 MB 2023-10-10T20:29:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 27.00 MB 2023-10-10T20:29:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.713 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.714 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 28.00 MB 2023-10-10T20:29:49.714 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.714 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.714 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 29.00 MB 2023-10-10T20:29:49.714 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.715 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.715 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 30.00 MB 2023-10-10T20:29:49.715 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:29:49.715 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.715 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 31.00 MB 2023-10-10T20:29:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:29:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 32.00 MB 2023-10-10T20:29:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:29:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-10T20:29:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:fsync'ed successfully 2023-10-10T20:29:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:Not full yet after 33.00 MB 2023-10-10T20:29:49.717 INFO:teuthology.orchestra.run.smithi148.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-10T20:29:49.718 INFO:teuthology.orchestra.run.smithi148.stdout:Reached fullness after 33.00 MB 2023-10-10T20:29:49.718 INFO:teuthology.orchestra.run.smithi148.stdout:calling close 2023-10-10T20:29:49.718 INFO:teuthology.orchestra.run.smithi148.stdout:close() did not raise error 2023-10-10T20:29:49.719 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:29:49.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:29:49.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:29:49.819 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:29:49.819 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:29:49.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:29:49.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:29:49.868 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:29:49.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:29:49.869 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:29:49.885 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:29:49.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:29:49.886 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:29:49.933 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:29:49.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:29:49.934 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:29:49.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:29:50.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.069+0000 7f5d20a77700 1 Processor -- start 2023-10-10T20:29:50.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.069+0000 7f5d20a77700 1 -- start start 2023-10-10T20:29:50.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d20a77700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d1c0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d20a77700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c0c42b0 con 0x7f5d1c156ea0 2023-10-10T20:29:50.078 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d20a77700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c0c36a0 con 0x7f5d1c06e2b0 2023-10-10T20:29:50.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d20a77700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c06fac0 con 0x7f5d1c157270 2023-10-10T20:29:50.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d1a59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5d1c156ea0 0x7f5d1c06f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54976/0 (socket says 172.21.15.148:54976) 2023-10-10T20:29:50.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d1a59c700 1 -- 172.21.15.148:0/3944058336 learned_addr learned my addr 172.21.15.148:0/3944058336 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:50.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.070+0000 7f5d11d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d1c0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216284707 0 0) 0x7f5d1c0c42b0 con 0x7f5d1c156ea0 2023-10-10T20:29:50.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d08003570 con 0x7f5d1c156ea0 2023-10-10T20:29:50.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 321884448 0 0) 0x7f5d08003570 con 0x7f5d1c156ea0 2023-10-10T20:29:50.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 msgr2=0x7f5d1c0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d1c0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 >> v1:172.21.15.161:6789/0 conn(0x7f5d1c06e2b0 legacy=0x7f5d1c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0c3980 con 0x7f5d1c156ea0 2023-10-10T20:29:50.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f5d0c001470 con 0x7f5d1c156ea0 2023-10-10T20:29:50.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 >> v1:172.21.15.148:6789/0 conn(0x7f5d1c156ea0 legacy=0x7f5d1c06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d08003a90 0x7f5d08003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d08004d50 0x7f5d080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c06fac0 con 0x7f5d08004d50 2023-10-10T20:29:50.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800bb10 con 0x7f5d08003a90 2023-10-10T20:29:50.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800dee0 con 0x7f5d1c157270 2023-10-10T20:29:50.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d08004d50 0x7f5d080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d11d9b700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1a59c700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d08003a90 0x7f5d08003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 msgr2=0x7f5d08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c157270 0x7f5d08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d08003a90 msgr2=0x7f5d08003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d08003a90 0x7f5d08003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/3944058336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08003570 con 0x7f5d08004d50 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.071+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d08004d50 0x7f5d080051c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5d140020a0 tx=0x7f5d140016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9d06ddb5e8a8af server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140019e0 con 0x7f5d08004d50 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1400c8b0 con 0x7f5d08004d50 2023-10-10T20:29:50.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d19d9b700 1 -- 172.21.15.148:0/3944058336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1400cc50 con 0x7f5d08004d50 2023-10-10T20:29:50.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 -- 172.21.15.148:0/3944058336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d08004d50 msgr2=0x7f5d080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 --2- 172.21.15.148:0/3944058336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d08004d50 0x7f5d080051c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5d140020a0 tx=0x7f5d140016d0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/3944058336 reap_dead start 2023-10-10T20:29:50.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 -- 172.21.15.148:0/3944058336 shutdown_connections 2023-10-10T20:29:50.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 -- 172.21.15.148:0/3944058336 >> 172.21.15.148:0/3944058336 conn(0x7f5d1c0ca4f0 msgr2=0x7f5d1c0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 -- 172.21.15.148:0/3944058336 shutdown_connections 2023-10-10T20:29:50.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.072+0000 7f5d20a77700 1 -- 172.21.15.148:0/3944058336 wait complete. 2023-10-10T20:29:50.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.073+0000 7f5d20a77700 1 Processor -- start 2023-10-10T20:29:50.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.073+0000 7f5d20a77700 1 -- start start 2023-10-10T20:29:50.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c14abd0 0x7f5d1c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d1c158e10 0x7f5d1c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b2ff0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b2100 con 0x7f5d1c158e10 2023-10-10T20:29:50.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d20a77700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b23e0 con 0x7f5d1c14abd0 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d1ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d1ad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47210/0 (socket says 172.21.15.148:47210) 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/1373533687 learned_addr learned my addr 172.21.15.148:0/1373533687 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d1a59c700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c14abd0 0x7f5d1c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.074+0000 7f5d11d9b700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d1c158e10 0x7f5d1c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c14abd0 msgr2=0x7f5d1c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c14abd0 0x7f5d1c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d1c158e10 msgr2=0x7f5d1c067e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d1c158e10 0x7f5d1c067e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0b26c0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d1ad9d700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5d14012000 tx=0x7f5d14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140019e0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1401e640 con 0x7f5d1c0b3480 2023-10-10T20:29:50.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1401e9e0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c0b29a0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.075+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c14afa0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.076+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c0b2e40 con 0x7f5d1c0b3480 2023-10-10T20:29:50.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.076+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d14029460 con 0x7f5d1c0b3480 2023-10-10T20:29:50.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.077+0000 7f5d12ffd700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 0x7f5cfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.077+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f5d14086050 con 0x7f5d1c0b3480 2023-10-10T20:29:50.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.077+0000 7f5d1a59c700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 0x7f5cfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.077+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f5cfc06eab0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.077+0000 7f5d1a59c700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 0x7f5cfc0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c001ed0 tx=0x7f5d1c0c4220 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.081+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d140532a0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.306+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d1c04c3c0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.306+0000 7f5d12ffd700 1 -- 172.21.15.148:0/1373533687 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f5d1401a3b0 con 0x7f5d1c0b3480 2023-10-10T20:29:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:50.309 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:29:50.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 msgr2=0x7f5cfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 0x7f5cfc0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c001ed0 tx=0x7f5d1c0c4220 comp rx=0 tx=0).stop 2023-10-10T20:29:50.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 msgr2=0x7f5d1c068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5d14012000 tx=0x7f5d14016000 comp rx=0 tx=0).stop 2023-10-10T20:29:50.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 shutdown_connections 2023-10-10T20:29:50.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5cfc066f30 0x7f5cfc0693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1c0b3480 0x7f5d1c068370 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d1c14abd0 0x7f5d1c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 --2- 172.21.15.148:0/1373533687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d1c158e10 0x7f5d1c067e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 >> 172.21.15.148:0/1373533687 conn(0x7f5d1c0ca4f0 msgr2=0x7f5d1c153340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 shutdown_connections 2023-10-10T20:29:50.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.308+0000 7f5d20a77700 1 -- 172.21.15.148:0/1373533687 wait complete. 2023-10-10T20:29:50.313 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:29:50.323 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:29:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.436+0000 7f83c01c8700 1 Processor -- start 2023-10-10T20:29:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.436+0000 7f83c01c8700 1 -- start start 2023-10-10T20:29:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83c01c8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83b8157870 0x7f83b815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83c01c8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f83b8157c40 0x7f83b8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83c01c8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b815cbb0 con 0x7f83b815a620 2023-10-10T20:29:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83c01c8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b815bfa0 con 0x7f83b8157870 2023-10-10T20:29:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83c01c8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b815c280 con 0x7f83b8157c40 2023-10-10T20:29:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83bdf64700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83b8157870 0x7f83b815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83bdf64700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83b8157870 0x7f83b815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47222/0 (socket says 172.21.15.148:47222) 2023-10-10T20:29:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83bdf64700 1 -- 172.21.15.148:0/1040226294 learned_addr learned my addr 172.21.15.148:0/1040226294 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.437+0000 7f83be765700 1 --2- 172.21.15.148:0/1040226294 >> v2:172.21.15.161:3300/0 conn(0x7f83b8157c40 0x7f83b8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bdf64700 1 -- 172.21.15.148:0/1040226294 >> v2:172.21.15.161:3300/0 conn(0x7f83b8157c40 msgr2=0x7f83b8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bdf64700 1 --2- 172.21.15.148:0/1040226294 >> v2:172.21.15.161:3300/0 conn(0x7f83b8157c40 0x7f83b8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bdf64700 1 -- 172.21.15.148:0/1040226294 >> v1:172.21.15.161:6789/0 conn(0x7f83b815a620 legacy=0x7f83b815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bdf64700 1 -- 172.21.15.148:0/1040226294 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b815c560 con 0x7f83b8157870 2023-10-10T20:29:50.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bdf64700 1 --2- 172.21.15.148:0/1040226294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b815b6a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f83b806b4e0 tx=0x7f83b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8bba8facdf0e4a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bcf62700 1 -- 172.21.15.148:0/1040226294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400e070 con 0x7f83b8157870 2023-10-10T20:29:50.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bcf62700 1 -- 172.21.15.148:0/1040226294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400ba10 con 0x7f83b8157870 2023-10-10T20:29:50.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83bcf62700 1 -- 172.21.15.148:0/1040226294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40092e0 con 0x7f83b8157870 2023-10-10T20:29:50.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83c01c8700 1 -- 172.21.15.148:0/1040226294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 msgr2=0x7f83b815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83c01c8700 1 --2- 172.21.15.148:0/1040226294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b815b6a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f83b806b4e0 tx=0x7f83b4009840 comp rx=0 tx=0).stop 2023-10-10T20:29:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83c01c8700 1 -- 172.21.15.148:0/1040226294 shutdown_connections 2023-10-10T20:29:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83c01c8700 1 --2- 172.21.15.148:0/1040226294 >> v2:172.21.15.161:3300/0 conn(0x7f83b8157c40 0x7f83b8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.438+0000 7f83c01c8700 1 --2- 172.21.15.148:0/1040226294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b815b6a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.439+0000 7f83c01c8700 1 -- 172.21.15.148:0/1040226294 >> 172.21.15.148:0/1040226294 conn(0x7f83b8068ce0 msgr2=0x7f83b80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.439+0000 7f83c01c8700 1 -- 172.21.15.148:0/1040226294 shutdown_connections 2023-10-10T20:29:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.439+0000 7f83c01c8700 1 -- 172.21.15.148:0/1040226294 wait complete. 2023-10-10T20:29:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 Processor -- start 2023-10-10T20:29:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 -- start start 2023-10-10T20:29:50.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 0x7f83b81510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 0x7f83b8151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b814edb0 con 0x7f83b8157870 2023-10-10T20:29:50.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80b3000 con 0x7f83b8157c40 2023-10-10T20:29:50.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83c01c8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80b2d20 con 0x7f83b815a620 2023-10-10T20:29:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83bdf64700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83bdf64700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47236/0 (socket says 172.21.15.148:47236) 2023-10-10T20:29:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.440+0000 7f83bdf64700 1 -- 172.21.15.148:0/4020829126 learned_addr learned my addr 172.21.15.148:0/4020829126 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83be765700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 0x7f83b8151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bd763700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 0x7f83b81510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 -- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 msgr2=0x7f83b8151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 0x7f83b8151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 -- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 msgr2=0x7f83b81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 0x7f83b81510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.453 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009ae0 con 0x7f83b8157870 2023-10-10T20:29:50.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83be765700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 0x7f83b8151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:29:50.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bd763700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 0x7f83b81510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:29:50.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83bdf64700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009fd0 tx=0x7f83b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.454 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4012950 con 0x7f83b8157870 2023-10-10T20:29:50.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4012ab0 con 0x7f83b8157870 2023-10-10T20:29:50.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b814e760 con 0x7f83b8157870 2023-10-10T20:29:50.455 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4012e70 con 0x7f83b8157870 2023-10-10T20:29:50.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.441+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b813ba40 con 0x7f83b8157870 2023-10-10T20:29:50.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.442+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b804c3c0 con 0x7f83b8157870 2023-10-10T20:29:50.456 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83b401d630 con 0x7f83b8157870 2023-10-10T20:29:50.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83adffb700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 0x7f839c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f83b4085e30 con 0x7f83b8157870 2023-10-10T20:29:50.458 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83bd763700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 0x7f839c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f839c06eab0 con 0x7f83b8157870 2023-10-10T20:29:50.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.443+0000 7f83bd763700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 0x7f839c0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006fd0 tx=0x7f83a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.459 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.447+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83b4053080 con 0x7f83b8157870 2023-10-10T20:29:50.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.633+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83b815c840 con 0x7f83b8157870 2023-10-10T20:29:50.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.634+0000 7f83adffb700 1 -- 172.21.15.148:0/4020829126 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+15488 (secure 0 0 0) 0x7f83b402a030 con 0x7f83b8157870 2023-10-10T20:29:50.637 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:50.637 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":166,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:29:25.577507+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.4","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:29:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 msgr2=0x7f839c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 0x7f839c0693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006fd0 tx=0x7f83a8008040 comp rx=0 tx=0).stop 2023-10-10T20:29:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 msgr2=0x7f83b813b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009fd0 tx=0x7f83b4017000 comp rx=0 tx=0).stop 2023-10-10T20:29:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 shutdown_connections 2023-10-10T20:29:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f839c066f30 0x7f839c0693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83b815a620 0x7f83b8151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83b8157870 0x7f83b813b500 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 --2- 172.21.15.148:0/4020829126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8157c40 0x7f83b81510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 >> 172.21.15.148:0/4020829126 conn(0x7f83b8068ce0 msgr2=0x7f83b80b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 shutdown_connections 2023-10-10T20:29:50.641 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.636+0000 7f83c01c8700 1 -- 172.21.15.148:0/4020829126 wait complete. 2023-10-10T20:29:50.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:28:37.360189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:28:37.967832+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:29:50.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:29:50.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.771+0000 7f296a0e8700 1 Processor -- start 2023-10-10T20:29:50.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.771+0000 7f296a0e8700 1 -- start start 2023-10-10T20:29:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f296a0e8700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2964159ec0 0x7f29640af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f296a0e8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 0x7f29640afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f296a0e8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640b0790 con 0x7f296406a5a0 2023-10-10T20:29:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f296a0e8700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640ce1f0 con 0x7f2964159ec0 2023-10-10T20:29:50.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f296a0e8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640b0b20 con 0x7f296415a290 2023-10-10T20:29:50.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f29637fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2964159ec0 0x7f29640af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f29637fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2964159ec0 0x7f29640af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47242/0 (socket says 172.21.15.148:47242) 2023-10-10T20:29:50.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.772+0000 7f29637fe700 1 -- 172.21.15.148:0/2015703861 learned_addr learned my addr 172.21.15.148:0/2015703861 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f2962ffd700 1 --2- 172.21.15.148:0/2015703861 >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 0x7f29640afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29637fe700 1 -- 172.21.15.148:0/2015703861 >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 msgr2=0x7f29640afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29637fe700 1 --2- 172.21.15.148:0/2015703861 >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 0x7f29640afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29637fe700 1 -- 172.21.15.148:0/2015703861 >> v1:172.21.15.161:6789/0 conn(0x7f296406a5a0 legacy=0x7f296415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:50.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29637fe700 1 -- 172.21.15.148:0/2015703861 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640b04b0 con 0x7f2964159ec0 2023-10-10T20:29:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f2962ffd700 1 --2- 172.21.15.148:0/2015703861 >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 0x7f29640afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29637fe700 1 --2- 172.21.15.148:0/2015703861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f29640af940 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2964089910 tx=0x7f2954009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5beb27abe77a23c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29627fc700 1 -- 172.21.15.148:0/2015703861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295400e070 con 0x7f2964159ec0 2023-10-10T20:29:50.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29627fc700 1 -- 172.21.15.148:0/2015703861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295400ba10 con 0x7f2964159ec0 2023-10-10T20:29:50.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f29627fc700 1 -- 172.21.15.148:0/2015703861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540092e0 con 0x7f2964159ec0 2023-10-10T20:29:50.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 -- 172.21.15.148:0/2015703861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 msgr2=0x7f29640af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 --2- 172.21.15.148:0/2015703861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f29640af940 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2964089910 tx=0x7f2954009840 comp rx=0 tx=0).stop 2023-10-10T20:29:50.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 -- 172.21.15.148:0/2015703861 shutdown_connections 2023-10-10T20:29:50.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 --2- 172.21.15.148:0/2015703861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f29640af940 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 --2- 172.21.15.148:0/2015703861 >> v2:172.21.15.161:3300/0 conn(0x7f296415a290 0x7f29640afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.787 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 -- 172.21.15.148:0/2015703861 >> 172.21.15.148:0/2015703861 conn(0x7f29640d1880 msgr2=0x7f29640cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 -- 172.21.15.148:0/2015703861 shutdown_connections 2023-10-10T20:29:50.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.773+0000 7f296a0e8700 1 -- 172.21.15.148:0/2015703861 wait complete. 2023-10-10T20:29:50.788 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 Processor -- start 2023-10-10T20:29:50.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 -- start start 2023-10-10T20:29:50.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f296406a5a0 0x7f29640c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.789 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f296415a290 0x7f296416b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964155e70 con 0x7f2964159ec0 2023-10-10T20:29:50.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640c99e0 con 0x7f296415a290 2023-10-10T20:29:50.790 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.775+0000 7f296a0e8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296415ac20 con 0x7f296406a5a0 2023-10-10T20:29:50.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47258/0 (socket says 172.21.15.148:47258) 2023-10-10T20:29:50.791 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 -- 172.21.15.148:0/1636818398 learned_addr learned my addr 172.21.15.148:0/1636818398 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f29637fe700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f296406a5a0 0x7f29640c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2963fff700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f296415a290 0x7f296416b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 -- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f296406a5a0 msgr2=0x7f29640c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f296406a5a0 0x7f29640c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 -- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f296415a290 msgr2=0x7f296416b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f296415a290 0x7f296416b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009ae0 con 0x7f2964159ec0 2023-10-10T20:29:50.793 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f2962ffd700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f295800c850 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958011600 con 0x7f2964159ec0 2023-10-10T20:29:50.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2958011760 con 0x7f2964159ec0 2023-10-10T20:29:50.794 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29640c9c80 con 0x7f2964159ec0 2023-10-10T20:29:50.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958011b00 con 0x7f2964159ec0 2023-10-10T20:29:50.795 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.776+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29640ca240 con 0x7f2964159ec0 2023-10-10T20:29:50.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.777+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296404c3c0 con 0x7f2964159ec0 2023-10-10T20:29:50.796 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.778+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2958018350 con 0x7f2964159ec0 2023-10-10T20:29:50.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.778+0000 7f294b7fe700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:50.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.778+0000 7f29637fe700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:50.798 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.778+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f29580843f0 con 0x7f2964159ec0 2023-10-10T20:29:50.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.778+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f295006eab0 con 0x7f2964159ec0 2023-10-10T20:29:50.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.779+0000 7f29637fe700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 0x7f29500693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2954009880 tx=0x7f2954017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:50.799 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.783+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29580550d0 con 0x7f2964159ec0 2023-10-10T20:29:50.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.964+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29640ceab0 con 0x7f2964159ec0 2023-10-10T20:29:50.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.965+0000 7f294b7fe700 1 -- 172.21.15.148:0/1636818398 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) v1 ==== 93+0+10 (secure 0 0 0) 0x7f295801e0a0 con 0x7f2964159ec0 2023-10-10T20:29:50.968 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:29:50.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 msgr2=0x7f29500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 0x7f29500693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2954009880 tx=0x7f2954017000 comp rx=0 tx=0).stop 2023-10-10T20:29:50.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 msgr2=0x7f296416b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:50.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f295800c850 tx=0x7f2958008040 comp rx=0 tx=0).stop 2023-10-10T20:29:50.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 shutdown_connections 2023-10-10T20:29:50.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f296415a290 0x7f296416b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f296406a5a0 0x7f29640c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 --2- 172.21.15.148:0/1636818398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2964159ec0 0x7f296416b270 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:50.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 >> 172.21.15.148:0/1636818398 conn(0x7f29640d1880 msgr2=0x7f29640b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:50.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 shutdown_connections 2023-10-10T20:29:50.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:50.967+0000 7f296a0e8700 1 -- 172.21.15.148:0/1636818398 wait complete. 2023-10-10T20:29:50.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:29:51.101 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.098+0000 7f87c46fd700 1 Processor -- start 2023-10-10T20:29:51.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.099+0000 7f87c46fd700 1 -- start start 2023-10-10T20:29:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c46fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 0x7f87bc06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c46fd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 0x7f87bc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c46fd700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0b0790 con 0x7f87bc159ec0 2023-10-10T20:29:51.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c46fd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc06af60 con 0x7f87bc06aae0 2023-10-10T20:29:51.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c46fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0ce1f0 con 0x7f87bc15a290 2023-10-10T20:29:51.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c2499700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f87bc159ec0 0x7f87bc0af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54984/0 (socket says 172.21.15.148:54984) 2023-10-10T20:29:51.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c2499700 1 -- 172.21.15.148:0/371340815 learned_addr learned my addr 172.21.15.148:0/371340815 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:51.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 --2- 172.21.15.148:0/371340815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 0x7f87bc06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c2c9a700 1 --2- 172.21.15.148:0/371340815 >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 0x7f87bc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502312893 0 0) 0x7f87bc0b0790 con 0x7f87bc159ec0 2023-10-10T20:29:51.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a8003570 con 0x7f87bc159ec0 2023-10-10T20:29:51.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 -- 172.21.15.148:0/371340815 >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 msgr2=0x7f87bc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:51.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 --2- 172.21.15.148:0/371340815 >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 0x7f87bc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223941182 0 0) 0x7f87a8003570 con 0x7f87bc159ec0 2023-10-10T20:29:51.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 -- 172.21.15.148:0/371340815 >> v1:172.21.15.148:6789/0 conn(0x7f87bc159ec0 legacy=0x7f87bc0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 -- 172.21.15.148:0/371340815 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0b0b20 con 0x7f87bc15a290 2023-10-10T20:29:51.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c2c9a700 1 --2- 172.21.15.148:0/371340815 >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 0x7f87bc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:29:51.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.100+0000 7f87c1c98700 1 --2- 172.21.15.148:0/371340815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 0x7f87bc06a5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0098e0 tx=0x7f87ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310f01cfdc44abbf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8001470 con 0x7f87bc15a290 2023-10-10T20:29:51.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac011420 con 0x7f87bc15a290 2023-10-10T20:29:51.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c1497700 1 -- 172.21.15.148:0/371340815 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0117c0 con 0x7f87bc15a290 2023-10-10T20:29:51.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 -- 172.21.15.148:0/371340815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 msgr2=0x7f87bc06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.120 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 --2- 172.21.15.148:0/371340815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 0x7f87bc06a5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0098e0 tx=0x7f87ac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 -- 172.21.15.148:0/371340815 shutdown_connections 2023-10-10T20:29:51.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 --2- 172.21.15.148:0/371340815 >> v2:172.21.15.161:3300/0 conn(0x7f87bc06aae0 0x7f87bc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.121 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 --2- 172.21.15.148:0/371340815 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc15a290 0x7f87bc06a5a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 -- 172.21.15.148:0/371340815 >> 172.21.15.148:0/371340815 conn(0x7f87bc0d1880 msgr2=0x7f87bc0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 -- 172.21.15.148:0/371340815 shutdown_connections 2023-10-10T20:29:51.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.101+0000 7f87c46fd700 1 -- 172.21.15.148:0/371340815 wait complete. 2023-10-10T20:29:51.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.102+0000 7f87c46fd700 1 Processor -- start 2023-10-10T20:29:51.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.102+0000 7f87c46fd700 1 -- start start 2023-10-10T20:29:51.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc06aae0 0x7f87bc0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87bc15a290 0x7f87bc0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cb340 con 0x7f87bc159ec0 2023-10-10T20:29:51.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0ca730 con 0x7f87bc15a290 2023-10-10T20:29:51.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c46fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0caa10 con 0x7f87bc06aae0 2023-10-10T20:29:51.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47270/0 (socket says 172.21.15.148:47270) 2023-10-10T20:29:51.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 -- 172.21.15.148:0/1253629479 learned_addr learned my addr 172.21.15.148:0/1253629479 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:51.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c2c9a700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87bc15a290 0x7f87bc0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c2499700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc06aae0 0x7f87bc0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 -- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc06aae0 msgr2=0x7f87bc0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc06aae0 0x7f87bc0c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 -- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87bc15a290 msgr2=0x7f87bc0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87bc15a290 0x7f87bc0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009a20 con 0x7f87bc159ec0 2023-10-10T20:29:51.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.103+0000 7f87c1c98700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0030c0 tx=0x7f87ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.104+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01cc90 con 0x7f87bc159ec0 2023-10-10T20:29:51.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.104+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac01cdf0 con 0x7f87bc159ec0 2023-10-10T20:29:51.131 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.104+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01a6a0 con 0x7f87bc159ec0 2023-10-10T20:29:51.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.104+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac009420 con 0x7f87bc159ec0 2023-10-10T20:29:51.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.104+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc0cafd0 con 0x7f87bc159ec0 2023-10-10T20:29:51.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.105+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc0cb4f0 con 0x7f87bc159ec0 2023-10-10T20:29:51.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.105+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87ac01aad0 con 0x7f87bc159ec0 2023-10-10T20:29:51.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.105+0000 7f87b27fc700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 0x7f87a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.105+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f87ac085e50 con 0x7f87bc159ec0 2023-10-10T20:29:51.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.106+0000 7f87c2499700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 0x7f87a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.106+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f87a006eab0 con 0x7f87bc159ec0 2023-10-10T20:29:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.106+0000 7f87c2499700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 0x7f87a00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f87b80011e0 tx=0x7f87b8001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.110+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87bc0cb4f0 con 0x7f87bc159ec0 2023-10-10T20:29:51.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.289+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87bc0cb4f0 con 0x7f87bc159ec0 2023-10-10T20:29:51.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.290+0000 7f87b27fc700 1 -- 172.21.15.148:0/1253629479 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v166) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87bc0cb4f0 con 0x7f87bc159ec0 2023-10-10T20:29:51.293 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:29:51.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 msgr2=0x7f87a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 0x7f87a00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f87b80011e0 tx=0x7f87b8001520 comp rx=0 tx=0).stop 2023-10-10T20:29:51.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 msgr2=0x7f87bc0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0030c0 tx=0x7f87ac016000 comp rx=0 tx=0).stop 2023-10-10T20:29:51.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 shutdown_connections 2023-10-10T20:29:51.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f87a0066f30 0x7f87a00693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87bc15a290 0x7f87bc0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87bc06aae0 0x7f87bc0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 --2- 172.21.15.148:0/1253629479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87bc159ec0 0x7f87bc0c7e60 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 >> 172.21.15.148:0/1253629479 conn(0x7f87bc0d1880 msgr2=0x7f87bc1557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 shutdown_connections 2023-10-10T20:29:51.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.292+0000 7f87c46fd700 1 -- 172.21.15.148:0/1253629479 wait complete. 2023-10-10T20:29:51.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:29:51.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.421+0000 7f1c42112700 1 Processor -- start 2023-10-10T20:29:51.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.421+0000 7f1c42112700 1 -- start start 2023-10-10T20:29:51.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c42112700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c3c159ec0 0x7f1c3c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c42112700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1c3c06acb0 0x7f1c3c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c42112700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3c0b0790 con 0x7f1c3c15a290 2023-10-10T20:29:51.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c42112700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c0ce1f0 con 0x7f1c3c159ec0 2023-10-10T20:29:51.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c42112700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c0b0b20 con 0x7f1c3c06acb0 2023-10-10T20:29:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c3b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c3c159ec0 0x7f1c3c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c3b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c3c159ec0 0x7f1c3c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47286/0 (socket says 172.21.15.148:47286) 2023-10-10T20:29:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/2036391876 learned_addr learned my addr 172.21.15.148:0/2036391876 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.422+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/2036391876 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c06acb0 0x7f1c3c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/2036391876 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c06acb0 msgr2=0x7f1c3c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:51.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/2036391876 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c06acb0 0x7f1c3c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/2036391876 >> v1:172.21.15.161:6789/0 conn(0x7f1c3c15a290 legacy=0x7f1c3c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3b7fe700 1 -- 172.21.15.148:0/2036391876 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0b04b0 con 0x7f1c3c159ec0 2023-10-10T20:29:51.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/2036391876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159ec0 0x7f1c3c0af940 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0711f0 tx=0x7f1c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d06d5bba359be3e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/2036391876 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00b860 con 0x7f1c3c159ec0 2023-10-10T20:29:51.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/2036391876 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2c00b9c0 con 0x7f1c3c159ec0 2023-10-10T20:29:51.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c3a7fc700 1 -- 172.21.15.148:0/2036391876 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00bd60 con 0x7f1c3c159ec0 2023-10-10T20:29:51.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 -- 172.21.15.148:0/2036391876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159ec0 msgr2=0x7f1c3c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 --2- 172.21.15.148:0/2036391876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159ec0 0x7f1c3c0af940 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0711f0 tx=0x7f1c2c009250 comp rx=0 tx=0).stop 2023-10-10T20:29:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 -- 172.21.15.148:0/2036391876 shutdown_connections 2023-10-10T20:29:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 --2- 172.21.15.148:0/2036391876 >> v2:172.21.15.161:3300/0 conn(0x7f1c3c06acb0 0x7f1c3c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 --2- 172.21.15.148:0/2036391876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c159ec0 0x7f1c3c0af940 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 -- 172.21.15.148:0/2036391876 >> 172.21.15.148:0/2036391876 conn(0x7f1c3c0d1880 msgr2=0x7f1c3c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 -- 172.21.15.148:0/2036391876 shutdown_connections 2023-10-10T20:29:51.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.423+0000 7f1c42112700 1 -- 172.21.15.148:0/2036391876 wait complete. 2023-10-10T20:29:51.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 Processor -- start 2023-10-10T20:29:51.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 -- start start 2023-10-10T20:29:51.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c06acb0 0x7f1c3c16a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c159ec0 0x7f1c3c0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c16df50 con 0x7f1c3c15a290 2023-10-10T20:29:51.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c16ae50 con 0x7f1c3c06acb0 2023-10-10T20:29:51.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.425+0000 7f1c42112700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c16b130 con 0x7f1c3c159ec0 2023-10-10T20:29:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47294/0 (socket says 172.21.15.148:47294) 2023-10-10T20:29:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 -- 172.21.15.148:0/2724447242 learned_addr learned my addr 172.21.15.148:0/2724447242 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c06acb0 0x7f1c3c16a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3affd700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c159ec0 0x7f1c3c0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 -- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c159ec0 msgr2=0x7f1c3c0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c159ec0 0x7f1c3c0ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 -- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c06acb0 msgr2=0x7f1c3c16a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c06acb0 0x7f1c3c16a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0094f0 con 0x7f1c3c15a290 2023-10-10T20:29:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c3bfff700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007f20 tx=0x7f1c340098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400eb90 con 0x7f1c3c15a290 2023-10-10T20:29:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400ed10 con 0x7f1c3c15a290 2023-10-10T20:29:51.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c16b410 con 0x7f1c3c15a290 2023-10-10T20:29:51.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34014670 con 0x7f1c3c15a290 2023-10-10T20:29:51.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.426+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c16b6f0 con 0x7f1c3c15a290 2023-10-10T20:29:51.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.427+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c0b6a20 con 0x7f1c3c15a290 2023-10-10T20:29:51.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c34014a60 con 0x7f1c3c15a290 2023-10-10T20:29:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c237fe700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 0x7f1c280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f1c34084b40 con 0x7f1c3c15a290 2023-10-10T20:29:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 0x7f1c280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f1c2806eab0 con 0x7f1c3c15a290 2023-10-10T20:29:51.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.428+0000 7f1c3b7fe700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 0x7f1c280693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0711f0 tx=0x7f1c2c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.432+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c34051d90 con 0x7f1c3c15a290 2023-10-10T20:29:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.613+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c3c04c3c0 con 0x7f1c3c15a290 2023-10-10T20:29:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.614+0000 7f1c237fe700 1 -- 172.21.15.148:0/2724447242 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v166) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c340553b0 con 0x7f1c3c15a290 2023-10-10T20:29:51.616 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:29:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 msgr2=0x7f1c280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 0x7f1c280693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0711f0 tx=0x7f1c2c017000 comp rx=0 tx=0).stop 2023-10-10T20:29:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 msgr2=0x7f1c3c154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007f20 tx=0x7f1c340098a0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 shutdown_connections 2023-10-10T20:29:51.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1c28066f30 0x7f1c280693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c3c15a290 0x7f1c3c154400 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c3c06acb0 0x7f1c3c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 --2- 172.21.15.148:0/2724447242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c3c159ec0 0x7f1c3c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 >> 172.21.15.148:0/2724447242 conn(0x7f1c3c0d1880 msgr2=0x7f1c3c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 shutdown_connections 2023-10-10T20:29:51.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.616+0000 7f1c42112700 1 -- 172.21.15.148:0/2724447242 wait complete. 2023-10-10T20:29:51.630 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:29:51.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.743+0000 7f44646e4700 1 Processor -- start 2023-10-10T20:29:51.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.743+0000 7f44646e4700 1 -- start start 2023-10-10T20:29:51.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.744+0000 7f44646e4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f445c152010 0x7f445c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.744+0000 7f44646e4700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445c1523e0 0x7f445c06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.744+0000 7f44646e4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c06c010 con 0x7f445c158e50 2023-10-10T20:29:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.744+0000 7f44646e4700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c06bd10 con 0x7f445c1523e0 2023-10-10T20:29:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.744+0000 7f44646e4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c15c060 con 0x7f445c152010 2023-10-10T20:29:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445c1523e0 0x7f445c06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445c1523e0 0x7f445c06c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47306/0 (socket says 172.21.15.148:47306) 2023-10-10T20:29:51.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 -- 172.21.15.148:0/3816289312 learned_addr learned my addr 172.21.15.148:0/3816289312 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:51.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4462480700 1 --2- 172.21.15.148:0/3816289312 >> v2:172.21.15.161:3300/0 conn(0x7f445c152010 0x7f445c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 -- 172.21.15.148:0/3816289312 >> v2:172.21.15.161:3300/0 conn(0x7f445c152010 msgr2=0x7f445c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:51.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 --2- 172.21.15.148:0/3816289312 >> v2:172.21.15.161:3300/0 conn(0x7f445c152010 0x7f445c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 -- 172.21.15.148:0/3816289312 >> v1:172.21.15.161:6789/0 conn(0x7f445c158e50 legacy=0x7f445c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 -- 172.21.15.148:0/3816289312 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c15c340 con 0x7f445c1523e0 2023-10-10T20:29:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f4461c7f700 1 --2- 172.21.15.148:0/3816289312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c1523e0 0x7f445c06c4a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc680 tx=0x7f444c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651967a6b963d993 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f446147e700 1 -- 172.21.15.148:0/3816289312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00b820 con 0x7f445c1523e0 2023-10-10T20:29:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f446147e700 1 -- 172.21.15.148:0/3816289312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c00b980 con 0x7f445c1523e0 2023-10-10T20:29:51.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.745+0000 7f446147e700 1 -- 172.21.15.148:0/3816289312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00bd20 con 0x7f445c1523e0 2023-10-10T20:29:51.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 -- 172.21.15.148:0/3816289312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c1523e0 msgr2=0x7f445c06c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 --2- 172.21.15.148:0/3816289312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c1523e0 0x7f445c06c4a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc680 tx=0x7f444c009250 comp rx=0 tx=0).stop 2023-10-10T20:29:51.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 -- 172.21.15.148:0/3816289312 shutdown_connections 2023-10-10T20:29:51.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 --2- 172.21.15.148:0/3816289312 >> v2:172.21.15.161:3300/0 conn(0x7f445c152010 0x7f445c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 --2- 172.21.15.148:0/3816289312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c1523e0 0x7f445c06c4a0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 -- 172.21.15.148:0/3816289312 >> 172.21.15.148:0/3816289312 conn(0x7f445c069870 msgr2=0x7f445c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 -- 172.21.15.148:0/3816289312 shutdown_connections 2023-10-10T20:29:51.753 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.746+0000 7f44646e4700 1 -- 172.21.15.148:0/3816289312 wait complete. 2023-10-10T20:29:51.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.747+0000 7f44646e4700 1 Processor -- start 2023-10-10T20:29:51.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.747+0000 7f44646e4700 1 -- start start 2023-10-10T20:29:51.756 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1523e0 0x7f445c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f445c158e50 0x7f445c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c167f00 con 0x7f445c152010 2023-10-10T20:29:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c141ad0 con 0x7f445c1523e0 2023-10-10T20:29:51.757 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c69b0 con 0x7f445c158e50 2023-10-10T20:29:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47314/0 (socket says 172.21.15.148:47314) 2023-10-10T20:29:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 -- 172.21.15.148:0/2492799878 learned_addr learned my addr 172.21.15.148:0/2492799878 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462c81700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f445c158e50 0x7f445c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 -- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f445c158e50 msgr2=0x7f445c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f445c158e50 0x7f445c0c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4461c7f700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1523e0 0x7f445c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 -- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1523e0 msgr2=0x7f445c0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1523e0 0x7f445c0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0094f0 con 0x7f445c152010 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f4462480700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f445800d860 tx=0x7f4458008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580125f0 con 0x7f445c152010 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458012750 con 0x7f445c152010 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458012b10 con 0x7f445c152010 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c15a790 con 0x7f445c152010 2023-10-10T20:29:51.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.748+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c15ad50 con 0x7f445c152010 2023-10-10T20:29:51.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.749+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c167d00 con 0x7f445c152010 2023-10-10T20:29:51.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f445801e460 con 0x7f445c152010 2023-10-10T20:29:51.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f44527fc700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 0x7f44400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:51.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f4458084c10 con 0x7f445c152010 2023-10-10T20:29:51.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f4461c7f700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 0x7f44400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:51.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f444006eab0 con 0x7f445c152010 2023-10-10T20:29:51.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.750+0000 7f4461c7f700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 0x7f44400693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc680 tx=0x7f444c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:51.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.754+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4458051e60 con 0x7f445c152010 2023-10-10T20:29:51.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.979+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f445c158c10 con 0x7f445c152010 2023-10-10T20:29:51.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.980+0000 7f44527fc700 1 -- 172.21.15.148:0/2492799878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f4458055480 con 0x7f445c152010 2023-10-10T20:29:51.983 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:51.983 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:29:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 msgr2=0x7f44400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 0x7f44400693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc680 tx=0x7f444c016000 comp rx=0 tx=0).stop 2023-10-10T20:29:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 msgr2=0x7f445c141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:51.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f445800d860 tx=0x7f4458008040 comp rx=0 tx=0).stop 2023-10-10T20:29:51.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 shutdown_connections 2023-10-10T20:29:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4440066f30 0x7f44400693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f445c158e50 0x7f445c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445c152010 0x7f445c141590 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 --2- 172.21.15.148:0/2492799878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1523e0 0x7f445c0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:51.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 >> 172.21.15.148:0/2492799878 conn(0x7f445c069870 msgr2=0x7f445c150860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:51.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 shutdown_connections 2023-10-10T20:29:51.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:51.982+0000 7f44646e4700 1 -- 172.21.15.148:0/2492799878 wait complete. 2023-10-10T20:29:51.990 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:29:51.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:29:52.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.109+0000 7fad2759e700 1 Processor -- start 2023-10-10T20:29:52.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.109+0000 7fad2759e700 1 -- start start 2023-10-10T20:29:52.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fad280cdab0 0x7fad2815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad2815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad280cb7c0 con 0x7fad280cd6e0 2023-10-10T20:29:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2759e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2815e850 con 0x7fad280cdab0 2023-10-10T20:29:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cc360 con 0x7fad2815dda0 2023-10-10T20:29:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2659c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fad280cd6e0 0x7fad280cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54992/0 (socket says 172.21.15.148:54992) 2023-10-10T20:29:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad2659c700 1 -- 172.21.15.148:0/421586688 learned_addr learned my addr 172.21.15.148:0/421586688 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:52.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/421586688 >> v2:172.21.15.148:3300/0 conn(0x7fad280cdab0 0x7fad2815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad26d9d700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad2815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 -- 172.21.15.148:0/421586688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 msgr2=0x7fad2815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad2815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 -- 172.21.15.148:0/421586688 >> v1:172.21.15.148:6789/0 conn(0x7fad280cd6e0 legacy=0x7fad280cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad25d9b700 1 -- 172.21.15.148:0/421586688 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383283851 0 0) 0x7fad280cb7c0 con 0x7fad280cd6e0 2023-10-10T20:29:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 -- 172.21.15.148:0/421586688 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2815e290 con 0x7fad280cdab0 2023-10-10T20:29:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815d860 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fad100098e0 tx=0x7fad100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914f6882a3715dd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad25d9b700 1 -- 172.21.15.148:0/421586688 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10009e80 con 0x7fad280cdab0 2023-10-10T20:29:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad25d9b700 1 -- 172.21.15.148:0/421586688 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000b860 con 0x7fad280cdab0 2023-10-10T20:29:52.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.110+0000 7fad25d9b700 1 -- 172.21.15.148:0/421586688 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000bbe0 con 0x7fad280cdab0 2023-10-10T20:29:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 -- 172.21.15.148:0/421586688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 msgr2=0x7fad2815d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815d860 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fad100098e0 tx=0x7fad100091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 -- 172.21.15.148:0/421586688 shutdown_connections 2023-10-10T20:29:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad2815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 --2- 172.21.15.148:0/421586688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815d860 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 -- 172.21.15.148:0/421586688 >> 172.21.15.148:0/421586688 conn(0x7fad28067660 msgr2=0x7fad28067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 -- 172.21.15.148:0/421586688 shutdown_connections 2023-10-10T20:29:52.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.111+0000 7fad2759e700 1 -- 172.21.15.148:0/421586688 wait complete. 2023-10-10T20:29:52.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.112+0000 7fad2759e700 1 Processor -- start 2023-10-10T20:29:52.122 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.112+0000 7fad2759e700 1 -- start start 2023-10-10T20:29:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 0x7fad28155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad280b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c8ad0 con 0x7fad280cdab0 2023-10-10T20:29:52.123 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c7ec0 con 0x7fad280cd6e0 2023-10-10T20:29:52.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c81a0 con 0x7fad2815dda0 2023-10-10T20:29:52.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47334/0 (socket says 172.21.15.148:47334) 2023-10-10T20:29:52.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 -- 172.21.15.148:0/4048644741 learned_addr learned my addr 172.21.15.148:0/4048644741 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:52.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2659c700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 0x7fad28155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad26d9d700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad280b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 -- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 msgr2=0x7fad280b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad280b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 -- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 msgr2=0x7fad28155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 0x7fad28155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad1dd9b700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10009420 con 0x7fad280cdab0 2023-10-10T20:29:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad2659c700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 0x7fad28155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.113+0000 7fad26d9d700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad280b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad1dd9b700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fad280c7b10 tx=0x7fad10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.126 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000bbe0 con 0x7fad280cdab0 2023-10-10T20:29:52.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10011420 con 0x7fad280cdab0 2023-10-10T20:29:52.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad280c8480 con 0x7fad280cdab0 2023-10-10T20:29:52.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100117e0 con 0x7fad280cdab0 2023-10-10T20:29:52.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.114+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281560f0 con 0x7fad280cdab0 2023-10-10T20:29:52.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.115+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad28089280 con 0x7fad280cdab0 2023-10-10T20:29:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.115+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad10011bd0 con 0x7fad280cdab0 2023-10-10T20:29:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.115+0000 7fad1effd700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.116+0000 7fad2659c700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.116+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fad10085fc0 con 0x7fad280cdab0 2023-10-10T20:29:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.116+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7fad0806eab0 con 0x7fad280cdab0 2023-10-10T20:29:52.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.116+0000 7fad2659c700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad18001810 tx=0x7fad18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.120+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad100b75a0 con 0x7fad280cdab0 2023-10-10T20:29:52.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.302+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad280b34a0 con 0x7fad280cdab0 2023-10-10T20:29:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.303+0000 7fad1effd700 1 -- 172.21.15.148:0/4048644741 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+15488 (secure 0 0 0) 0x7fad10015030 con 0x7fad280cdab0 2023-10-10T20:29:52.305 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:52.305 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":166,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:29:25.577507+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.4","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:29:52.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.304+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 msgr2=0x7fad080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.304+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad18001810 tx=0x7fad18009040 comp rx=0 tx=0).stop 2023-10-10T20:29:52.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.304+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 msgr2=0x7fad2815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.304+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fad280c7b10 tx=0x7fad10017000 comp rx=0 tx=0).stop 2023-10-10T20:29:52.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 shutdown_connections 2023-10-10T20:29:52.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad2815dda0 0x7fad280b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad280cd6e0 0x7fad28155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 --2- 172.21.15.148:0/4048644741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad280cdab0 0x7fad2815be00 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 >> 172.21.15.148:0/4048644741 conn(0x7fad28067660 msgr2=0x7fad280c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 shutdown_connections 2023-10-10T20:29:52.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.305+0000 7fad2759e700 1 -- 172.21.15.148:0/4048644741 wait complete. 2023-10-10T20:29:52.319 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:29:52.319 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:29:52.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.431+0000 7f1521b3e700 1 Processor -- start 2023-10-10T20:29:52.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.431+0000 7f1521b3e700 1 -- start start 2023-10-10T20:29:52.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f1521b3e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c152010 0x7f151c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f1521b3e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f151c1523e0 0x7f151c06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f1521b3e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c06c010 con 0x7f151c158e50 2023-10-10T20:29:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f1521b3e700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c06bd10 con 0x7f151c1523e0 2023-10-10T20:29:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f1521b3e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15c060 con 0x7f151c152010 2023-10-10T20:29:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f151affd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f151c1523e0 0x7f151c06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f151bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f151c158e50 0x7f151c0cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55000/0 (socket says 172.21.15.148:55000) 2023-10-10T20:29:52.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f151bfff700 1 -- 172.21.15.148:0/1749007629 learned_addr learned my addr 172.21.15.148:0/1749007629 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:52.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.432+0000 7f151affd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f151c1523e0 0x7f151c06c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47342/0 (socket says 172.21.15.148:47342) 2023-10-10T20:29:52.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151b7fe700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c152010 0x7f151c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151affd700 1 -- 172.21.15.148:0/1749007629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c152010 msgr2=0x7f151c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:52.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151affd700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c152010 0x7f151c15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151affd700 1 -- 172.21.15.148:0/1749007629 >> v1:172.21.15.148:6789/0 conn(0x7f151c158e50 legacy=0x7f151c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151affd700 1 -- 172.21.15.148:0/1749007629 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c15c340 con 0x7f151c1523e0 2023-10-10T20:29:52.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151a7fc700 1 -- 172.21.15.148:0/1749007629 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880921660 0 0) 0x7f151c06c010 con 0x7f151c158e50 2023-10-10T20:29:52.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151affd700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c1523e0 0x7f151c06c4a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f15100098e0 tx=0x7f15100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fab9dbd3960636 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151a7fc700 1 -- 172.21.15.148:0/1749007629 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510009e80 con 0x7f151c1523e0 2023-10-10T20:29:52.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151a7fc700 1 -- 172.21.15.148:0/1749007629 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151000b860 con 0x7f151c1523e0 2023-10-10T20:29:52.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.433+0000 7f151a7fc700 1 -- 172.21.15.148:0/1749007629 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151000bbe0 con 0x7f151c1523e0 2023-10-10T20:29:52.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 -- 172.21.15.148:0/1749007629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c1523e0 msgr2=0x7f151c06c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c1523e0 0x7f151c06c4a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f15100098e0 tx=0x7f15100091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 -- 172.21.15.148:0/1749007629 shutdown_connections 2023-10-10T20:29:52.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c152010 0x7f151c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1749007629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c1523e0 0x7f151c06c4a0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 -- 172.21.15.148:0/1749007629 >> 172.21.15.148:0/1749007629 conn(0x7f151c069870 msgr2=0x7f151c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:52.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 -- 172.21.15.148:0/1749007629 shutdown_connections 2023-10-10T20:29:52.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.434+0000 7f1521b3e700 1 -- 172.21.15.148:0/1749007629 wait complete. 2023-10-10T20:29:52.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.435+0000 7f1521b3e700 1 Processor -- start 2023-10-10T20:29:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.435+0000 7f1521b3e700 1 -- start start 2023-10-10T20:29:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f151c152010 0x7f151c14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c1523e0 0x7f151c147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.446 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c14e940 con 0x7f151c158e50 2023-10-10T20:29:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c14db40 con 0x7f151c152010 2023-10-10T20:29:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c1484b0 con 0x7f151c1523e0 2023-10-10T20:29:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47348/0 (socket says 172.21.15.148:47348) 2023-10-10T20:29:52.447 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 -- 172.21.15.148:0/1070486693 learned_addr learned my addr 172.21.15.148:0/1070486693 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151affd700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c1523e0 0x7f151c147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151b7fe700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f151c152010 0x7f151c14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 -- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c1523e0 msgr2=0x7f151c147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c1523e0 0x7f151c147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.448 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 -- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f151c152010 msgr2=0x7f151c14d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f151c152010 0x7f151c14d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009420 con 0x7f151c158e50 2023-10-10T20:29:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f151bfff700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f151400c810 tx=0x7f1514007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514005b30 con 0x7f151c158e50 2023-10-10T20:29:52.449 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514005c90 con 0x7f151c158e50 2023-10-10T20:29:52.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514007320 con 0x7f151c158e50 2023-10-10T20:29:52.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c14ddc0 con 0x7f151c158e50 2023-10-10T20:29:52.450 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.436+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c14e380 con 0x7f151c158e50 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.437+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c04c3c0 con 0x7f151c158e50 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.438+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f151401b3b0 con 0x7f151c158e50 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.438+0000 7f15037fe700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.438+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f15140839b0 con 0x7f151c158e50 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.438+0000 7f151b7fe700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.451 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.438+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f150806eab0 con 0x7f151c158e50 2023-10-10T20:29:52.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.439+0000 7f151b7fe700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 0x7f15080693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f150c0075a0 tx=0x7f150c0097e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.452 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.443+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1514017020 con 0x7f151c158e50 2023-10-10T20:29:52.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.667+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f151c06d1b0 con 0x7f151c158e50 2023-10-10T20:29:52.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.668+0000 7f15037fe700 1 -- 172.21.15.148:0/1070486693 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f1514050c00 con 0x7f151c158e50 2023-10-10T20:29:52.671 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:52.671 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:29:52.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 msgr2=0x7f15080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 0x7f15080693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f150c0075a0 tx=0x7f150c0097e0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 msgr2=0x7f151c14eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f151400c810 tx=0x7f1514007d90 comp rx=0 tx=0).stop 2023-10-10T20:29:52.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 shutdown_connections 2023-10-10T20:29:52.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1508066f30 0x7f15080693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f151c158e50 0x7f151c14eda0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f151c152010 0x7f151c14d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 --2- 172.21.15.148:0/1070486693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f151c1523e0 0x7f151c147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 >> 172.21.15.148:0/1070486693 conn(0x7f151c069870 msgr2=0x7f151c1514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:52.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 shutdown_connections 2023-10-10T20:29:52.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.670+0000 7f1521b3e700 1 -- 172.21.15.148:0/1070486693 wait complete. 2023-10-10T20:29:52.676 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:29:52.684 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:29:52.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.797+0000 7f0ebde84700 1 Processor -- start 2023-10-10T20:29:52.800 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.797+0000 7f0ebde84700 1 -- start start 2023-10-10T20:29:52.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.798+0000 7f0ebde84700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 0x7f0eb8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.798+0000 7f0ebde84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 0x7f0eb806a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.798+0000 7f0ebde84700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb8067ba0 con 0x7f0eb8156ca0 2023-10-10T20:29:52.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.798+0000 7f0ebde84700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb815aa90 con 0x7f0eb81568d0 2023-10-10T20:29:52.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.798+0000 7f0ebde84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb8067870 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb77fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 0x7f0eb8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.803 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 0x7f0eb806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb77fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 0x7f0eb8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:56914/0 (socket says 172.21.15.148:56914) 2023-10-10T20:29:52.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb77fe700 1 -- 172.21.15.148:0/3249257584 learned_addr learned my addr 172.21.15.148:0/3249257584 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:52.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/3249257584 >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 msgr2=0x7f0eb8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:52.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 --2- 172.21.15.148:0/3249257584 >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 0x7f0eb8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/3249257584 >> v1:172.21.15.161:6789/0 conn(0x7f0eb8156ca0 legacy=0x7f0eb80c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/3249257584 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80c8ca0 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb6ffd700 1 --2- 172.21.15.148:0/3249257584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 0x7f0eb806a360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00bb90 tx=0x7f0eac00b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=effcd7d4418be93b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb67fc700 1 -- 172.21.15.148:0/3249257584 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00db40 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb67fc700 1 -- 172.21.15.148:0/3249257584 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00dca0 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.799+0000 7f0eb67fc700 1 -- 172.21.15.148:0/3249257584 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0196b0 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 -- 172.21.15.148:0/3249257584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 msgr2=0x7f0eb806a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 --2- 172.21.15.148:0/3249257584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 0x7f0eb806a360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00bb90 tx=0x7f0eac00b430 comp rx=0 tx=0).stop 2023-10-10T20:29:52.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 -- 172.21.15.148:0/3249257584 shutdown_connections 2023-10-10T20:29:52.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 --2- 172.21.15.148:0/3249257584 >> v2:172.21.15.161:3300/0 conn(0x7f0eb81568d0 0x7f0eb8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 --2- 172.21.15.148:0/3249257584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb80ad9b0 0x7f0eb806a360 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 -- 172.21.15.148:0/3249257584 >> 172.21.15.148:0/3249257584 conn(0x7f0eb80cda50 msgr2=0x7f0eb80cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:52.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 -- 172.21.15.148:0/3249257584 shutdown_connections 2023-10-10T20:29:52.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.800+0000 7f0ebde84700 1 -- 172.21.15.148:0/3249257584 wait complete. 2023-10-10T20:29:52.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.801+0000 7f0ebde84700 1 Processor -- start 2023-10-10T20:29:52.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.801+0000 7f0ebde84700 1 -- start start 2023-10-10T20:29:52.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb80ad9b0 0x7f0eb814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb8156ca0 0x7f0eb8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80c2d60 con 0x7f0eb81568d0 2023-10-10T20:29:52.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81502c0 con 0x7f0eb80ad9b0 2023-10-10T20:29:52.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0ebde84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81505a0 con 0x7f0eb8156ca0 2023-10-10T20:29:52.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35266/0 (socket says 172.21.15.148:35266) 2023-10-10T20:29:52.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/2403771455 learned_addr learned my addr 172.21.15.148:0/2403771455 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:52.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb77fe700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb80ad9b0 0x7f0eb814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb7fff700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb8156ca0 0x7f0eb8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb8156ca0 msgr2=0x7f0eb8070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb8156ca0 0x7f0eb8070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb80ad9b0 msgr2=0x7f0eb814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:52.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb80ad9b0 0x7f0eb814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:52.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.802+0000 7f0eb6ffd700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac00b6d0 con 0x7f0eb81568d0 2023-10-10T20:29:52.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0eb6ffd700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00b6a0 tx=0x7f0eac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac006e50 con 0x7f0eb81568d0 2023-10-10T20:29:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac010680 con 0x7f0eb81568d0 2023-10-10T20:29:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb8150880 con 0x7f0eb81568d0 2023-10-10T20:29:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac010a40 con 0x7f0eb81568d0 2023-10-10T20:29:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.803+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb8150e40 con 0x7f0eb81568d0 2023-10-10T20:29:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.804+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb8151030 con 0x7f0eb81568d0 2023-10-10T20:29:52.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.804+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0eac023070 con 0x7f0eb81568d0 2023-10-10T20:29:52.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.804+0000 7f0e9f7fe700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:52.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.804+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f0eac014070 con 0x7f0eb81568d0 2023-10-10T20:29:52.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.805+0000 7f0eb77fe700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:52.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.805+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f0ea406eab0 con 0x7f0eb81568d0 2023-10-10T20:29:52.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.805+0000 7f0eb77fe700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 0x7f0ea40693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800f570 tx=0x7f0ea800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:52.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:52.809+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eac055740 con 0x7f0eb81568d0 2023-10-10T20:29:53.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.033+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0eb804c3c0 con 0x7f0eb81568d0 2023-10-10T20:29:53.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.034+0000 7f0e9f7fe700 1 -- 172.21.15.148:0/2403771455 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f0eac02a070 con 0x7f0eb81568d0 2023-10-10T20:29:53.036 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:53.037 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:28:39.000301+0000","modified":"2023-10-10T20:28:42.647073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"b","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1416347797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1416347797},{"type":"v1","addr":"172.21.15.161:6835","nonce":1416347797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-10T20:29:53.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 msgr2=0x7f0ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 0x7f0ea40693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800f570 tx=0x7f0ea800b040 comp rx=0 tx=0).stop 2023-10-10T20:29:53.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 msgr2=0x7f0eb814f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00b6a0 tx=0x7f0eac01b000 comp rx=0 tx=0).stop 2023-10-10T20:29:53.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 shutdown_connections 2023-10-10T20:29:53.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0ea4066f30 0x7f0ea40693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0eb8156ca0 0x7f0eb8070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0eb80ad9b0 0x7f0eb814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 --2- 172.21.15.148:0/2403771455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0eb81568d0 0x7f0eb814f9c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 >> 172.21.15.148:0/2403771455 conn(0x7f0eb80cda50 msgr2=0x7f0eb80cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 shutdown_connections 2023-10-10T20:29:53.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.036+0000 7f0ebde84700 1 -- 172.21.15.148:0/2403771455 wait complete. 2023-10-10T20:29:53.042 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2023-10-10T20:29:53.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:29:53.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.163+0000 7f04fa1b3700 1 Processor -- start 2023-10-10T20:29:53.166 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.163+0000 7f04fa1b3700 1 -- start start 2023-10-10T20:29:53.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04fa1b3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04f415a620 0x7f04f415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04fa1b3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f04f415d580 0x7f04f4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04fa1b3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f415cbb0 con 0x7f04f4157870 2023-10-10T20:29:53.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04fa1b3700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f415bdb0 con 0x7f04f415a620 2023-10-10T20:29:53.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04fa1b3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f415bfa0 con 0x7f04f415d580 2023-10-10T20:29:53.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04f2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04f415a620 0x7f04f415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04f2ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f04f415a620 0x7f04f415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35280/0 (socket says 172.21.15.148:35280) 2023-10-10T20:29:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.164+0000 7f04f2ffd700 1 -- 172.21.15.148:0/418705477 learned_addr learned my addr 172.21.15.148:0/418705477 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f3fff700 1 --2- 172.21.15.148:0/418705477 >> v2:172.21.15.161:3300/0 conn(0x7f04f415d580 0x7f04f4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f2ffd700 1 -- 172.21.15.148:0/418705477 >> v2:172.21.15.161:3300/0 conn(0x7f04f415d580 msgr2=0x7f04f4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f2ffd700 1 --2- 172.21.15.148:0/418705477 >> v2:172.21.15.161:3300/0 conn(0x7f04f415d580 0x7f04f4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f2ffd700 1 -- 172.21.15.148:0/418705477 >> v1:172.21.15.161:6789/0 conn(0x7f04f4157870 legacy=0x7f04f415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:53.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f2ffd700 1 -- 172.21.15.148:0/418705477 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f415c280 con 0x7f04f415a620 2023-10-10T20:29:53.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f2ffd700 1 --2- 172.21.15.148:0/418705477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415a620 0x7f04f415d040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f04f406b4e0 tx=0x7f04e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e64dbf865dd90e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f27fc700 1 -- 172.21.15.148:0/418705477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800b820 con 0x7f04f415a620 2023-10-10T20:29:53.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f27fc700 1 -- 172.21.15.148:0/418705477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e800b980 con 0x7f04f415a620 2023-10-10T20:29:53.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04f27fc700 1 -- 172.21.15.148:0/418705477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800bd20 con 0x7f04f415a620 2023-10-10T20:29:53.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 -- 172.21.15.148:0/418705477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415a620 msgr2=0x7f04f415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/418705477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415a620 0x7f04f415d040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f04f406b4e0 tx=0x7f04e8009250 comp rx=0 tx=0).stop 2023-10-10T20:29:53.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 -- 172.21.15.148:0/418705477 shutdown_connections 2023-10-10T20:29:53.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/418705477 >> v2:172.21.15.161:3300/0 conn(0x7f04f415d580 0x7f04f4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/418705477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415a620 0x7f04f415d040 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.165+0000 7f04fa1b3700 1 -- 172.21.15.148:0/418705477 >> 172.21.15.148:0/418705477 conn(0x7f04f4068ce0 msgr2=0x7f04f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.166+0000 7f04fa1b3700 1 -- 172.21.15.148:0/418705477 shutdown_connections 2023-10-10T20:29:53.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.166+0000 7f04fa1b3700 1 -- 172.21.15.148:0/418705477 wait complete. 2023-10-10T20:29:53.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 Processor -- start 2023-10-10T20:29:53.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 -- start start 2023-10-10T20:29:53.176 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04f4157870 0x7f04f415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f415a620 0x7f04f414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f415ec30 con 0x7f04f415d580 2023-10-10T20:29:53.177 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f415f1f0 con 0x7f04f415a620 2023-10-10T20:29:53.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.167+0000 7f04fa1b3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f415fbc0 con 0x7f04f4157870 2023-10-10T20:29:53.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35292/0 (socket says 172.21.15.148:35292) 2023-10-10T20:29:53.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 -- 172.21.15.148:0/2737903290 learned_addr learned my addr 172.21.15.148:0/2737903290 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:53.178 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f2ffd700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f415a620 0x7f04f414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f37fe700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04f4157870 0x7f04f415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 -- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04f4157870 msgr2=0x7f04f415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04f4157870 0x7f04f415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 -- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f415a620 msgr2=0x7f04f414a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f415a620 0x7f04f414a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e80094f0 con 0x7f04f415d580 2023-10-10T20:29:53.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04f3fff700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec007f20 tx=0x7f04ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00eb30 con 0x7f04f415d580 2023-10-10T20:29:53.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec00ec90 con 0x7f04f415d580 2023-10-10T20:29:53.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec014420 con 0x7f04f415d580 2023-10-10T20:29:53.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f414bac0 con 0x7f04f415d580 2023-10-10T20:29:53.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.168+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f415ee80 con 0x7f04f415d580 2023-10-10T20:29:53.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.169+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f415f380 con 0x7f04f415d580 2023-10-10T20:29:53.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04ec014810 con 0x7f04f415d580 2023-10-10T20:29:53.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04db7fe700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f04ec084b10 con 0x7f04f415d580 2023-10-10T20:29:53.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04f37fe700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f04e006eab0 con 0x7f04f415d580 2023-10-10T20:29:53.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.170+0000 7f04f37fe700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003a30 tx=0x7f04e4007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.174+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04ec051d60 con 0x7f04f415d580 2023-10-10T20:29:53.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.357+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04f404c3c0 con 0x7f04f415d580 2023-10-10T20:29:53.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.358+0000 7f04db7fe700 1 -- 172.21.15.148:0/2737903290 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+15488 (secure 0 0 0) 0x7f04ec055380 con 0x7f04f415d580 2023-10-10T20:29:53.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:53.361 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":166,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:29:25.577507+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:28:37.360189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-10T20:28:37.967832+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.4","mappings":[{"from":4,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:29:53.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 msgr2=0x7f04e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003a30 tx=0x7f04e4007ea0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 msgr2=0x7f04f414b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec007f20 tx=0x7f04ec009840 comp rx=0 tx=0).stop 2023-10-10T20:29:53.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 shutdown_connections 2023-10-10T20:29:53.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04f415d580 0x7f04f414b000 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f04f4157870 0x7f04f415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 --2- 172.21.15.148:0/2737903290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04f415a620 0x7f04f414a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 >> 172.21.15.148:0/2737903290 conn(0x7f04f4068ce0 msgr2=0x7f04f414fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 shutdown_connections 2023-10-10T20:29:53.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.360+0000 7f04fa1b3700 1 -- 172.21.15.148:0/2737903290 wait complete. 2023-10-10T20:29:53.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:29:53.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.488+0000 7f73824bb700 1 Processor -- start 2023-10-10T20:29:53.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.488+0000 7f73824bb700 1 -- start start 2023-10-10T20:29:53.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f73824bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f73824bb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f73824bb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737c15cbb0 con 0x7f737c15d580 2023-10-10T20:29:53.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f73824bb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15bfa0 con 0x7f737c15a620 2023-10-10T20:29:53.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f73824bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c15c280 con 0x7f737c157870 2023-10-10T20:29:53.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f737b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f737bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.489+0000 7f7380a58700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f737c15d580 0x7f737c158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:45914/0 (socket says 172.21.15.148:45914) 2023-10-10T20:29:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:58868/0 (socket says 172.21.15.148:58868) 2023-10-10T20:29:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f7380a58700 1 -- 172.21.15.148:0/1719998698 learned_addr learned my addr 172.21.15.148:0/1719998698 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:53.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737b7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:56944/0 (socket says 172.21.15.148:56944) 2023-10-10T20:29:53.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737affd700 1 -- 172.21.15.148:0/1719998698 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323240442 0 0) 0x7f737c15cbb0 con 0x7f737c15d580 2023-10-10T20:29:53.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737affd700 1 -- 172.21.15.148:0/1719998698 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7364003570 con 0x7f737c15d580 2023-10-10T20:29:53.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 -- 172.21.15.148:0/1719998698 >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 msgr2=0x7f737c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:53.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 --2- 172.21.15.148:0/1719998698 >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.496 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 -- 172.21.15.148:0/1719998698 >> v1:172.21.15.161:6789/0 conn(0x7f737c15d580 legacy=0x7f737c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 -- 172.21.15.148:0/1719998698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c15c560 con 0x7f737c157870 2023-10-10T20:29:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737b7fe700 1 --2- 172.21.15.148:0/1719998698 >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:29:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737bfff700 1 --2- 172.21.15.148:0/1719998698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098e0 tx=0x7f736c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571890c03f2db915 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737affd700 1 -- 172.21.15.148:0/1719998698 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c011630 con 0x7f737c157870 2023-10-10T20:29:53.497 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737affd700 1 -- 172.21.15.148:0/1719998698 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c011790 con 0x7f737c157870 2023-10-10T20:29:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.490+0000 7f737affd700 1 -- 172.21.15.148:0/1719998698 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c011b30 con 0x7f737c157870 2023-10-10T20:29:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 -- 172.21.15.148:0/1719998698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 msgr2=0x7f737c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 --2- 172.21.15.148:0/1719998698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098e0 tx=0x7f736c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 -- 172.21.15.148:0/1719998698 shutdown_connections 2023-10-10T20:29:53.498 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 --2- 172.21.15.148:0/1719998698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c15b6a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 --2- 172.21.15.148:0/1719998698 >> v2:172.21.15.161:3300/0 conn(0x7f737c15a620 0x7f737c15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 -- 172.21.15.148:0/1719998698 >> 172.21.15.148:0/1719998698 conn(0x7f737c068ce0 msgr2=0x7f737c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 -- 172.21.15.148:0/1719998698 shutdown_connections 2023-10-10T20:29:53.499 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.491+0000 7f73824bb700 1 -- 172.21.15.148:0/1719998698 wait complete. 2023-10-10T20:29:53.502 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.492+0000 7f73824bb700 1 Processor -- start 2023-10-10T20:29:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.492+0000 7f73824bb700 1 -- start start 2023-10-10T20:29:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f737c15d580 0x7f737c16a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0b1660 con 0x7f737c15a620 2023-10-10T20:29:53.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0b1940 con 0x7f737c15d580 2023-10-10T20:29:53.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f73824bb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1539e0 con 0x7f737c157870 2023-10-10T20:29:53.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35308/0 (socket says 172.21.15.148:35308) 2023-10-10T20:29:53.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 -- 172.21.15.148:0/3482622188 learned_addr learned my addr 172.21.15.148:0/3482622188 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:53.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f7380a58700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f737c15d580 0x7f737c16a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737bfff700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 -- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 msgr2=0x7f737c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 -- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f737c15d580 msgr2=0x7f737c16a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f737c15d580 0x7f737c16a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370009420 con 0x7f737c15a620 2023-10-10T20:29:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.493+0000 7f737b7fe700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f737000ecb0 tx=0x7f73700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737000c920 con 0x7f737c15a620 2023-10-10T20:29:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737000ca80 con 0x7f737c15a620 2023-10-10T20:29:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c009420 con 0x7f737c15a620 2023-10-10T20:29:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737000ce00 con 0x7f737c15a620 2023-10-10T20:29:53.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c0d0f10 con 0x7f737c15a620 2023-10-10T20:29:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.494+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c04c3c0 con 0x7f737c15a620 2023-10-10T20:29:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.495+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7370013680 con 0x7f737c15a620 2023-10-10T20:29:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.495+0000 7f7363fff700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.495+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f73700848f0 con 0x7f737c15a620 2023-10-10T20:29:53.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.495+0000 7f737bfff700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.496+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f736806eab0 con 0x7f737c15a620 2023-10-10T20:29:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.496+0000 7f737bfff700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 0x7f73680693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f736c0091e0 tx=0x7f736c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.500+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7370017060 con 0x7f737c15a620 2023-10-10T20:29:53.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.727+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f737c15c840 con 0x7f737c15a620 2023-10-10T20:29:53.730 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:29:53.727+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:29:53.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.731+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(167..167 src has 1..167) v4 ==== 366+0+0 (secure 0 0 0) 0x7f7370055800 con 0x7f737c15a620 2023-10-10T20:29:53.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.731+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=168}) v3 -- 0x7f736806f3f0 con 0x7f737c15a620 2023-10-10T20:29:53.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.734+0000 7f7363fff700 1 -- 172.21.15.148:0/3482622188 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7370021020 con 0x7f737c15a620 2023-10-10T20:29:53.737 INFO:tasks.ceph.mds.b.smithi161.stderr: -9548> 2023-10-10T20:28:34.538+0000 7f501ac38700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-10T20:29:53.739 INFO:tasks.ceph.mds.b.smithi161.stderr: -8350> 2023-10-10T20:28:44.935+0000 7f5014c2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:29:53.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 msgr2=0x7f73680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 0x7f73680693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f736c0091e0 tx=0x7f736c017000 comp rx=0 tx=0).stop 2023-10-10T20:29:53.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 msgr2=0x7f737c16a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f737000ecb0 tx=0x7f73700091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 shutdown_connections 2023-10-10T20:29:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f737c15d580 0x7f737c16a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f737c157870 0x7f737c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 --2- 172.21.15.148:0/3482622188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f737c15a620 0x7f737c16a330 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 >> 172.21.15.148:0/3482622188 conn(0x7f737c068ce0 msgr2=0x7f737c1556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 shutdown_connections 2023-10-10T20:29:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.736+0000 7f73824bb700 1 -- 172.21.15.148:0/3482622188 wait complete. 2023-10-10T20:29:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:29:53.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:29:53.768 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:29:53.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:29:53.831 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:29:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.871+0000 7f2cb286c700 1 Processor -- start 2023-10-10T20:29:53.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 -- start start 2023-10-10T20:29:53.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cac159ec0 0x7f2cac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2cac15a290 0x7f2cac0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0b0790 con 0x7f2cac06a5a0 2023-10-10T20:29:53.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0ce1f0 con 0x7f2cac159ec0 2023-10-10T20:29:53.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cb286c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0b0b20 con 0x7f2cac15a290 2023-10-10T20:29:53.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cabfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cac159ec0 0x7f2cac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cabfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2cac159ec0 0x7f2cac0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35318/0 (socket says 172.21.15.148:35318) 2023-10-10T20:29:53.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.872+0000 7f2cabfff700 1 -- 172.21.15.148:0/3514459901 learned_addr learned my addr 172.21.15.148:0/3514459901 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:53.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cab7fe700 1 --2- 172.21.15.148:0/3514459901 >> v2:172.21.15.161:3300/0 conn(0x7f2cac15a290 0x7f2cac0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cabfff700 1 -- 172.21.15.148:0/3514459901 >> v2:172.21.15.161:3300/0 conn(0x7f2cac15a290 msgr2=0x7f2cac0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:53.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cabfff700 1 --2- 172.21.15.148:0/3514459901 >> v2:172.21.15.161:3300/0 conn(0x7f2cac15a290 0x7f2cac0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cabfff700 1 -- 172.21.15.148:0/3514459901 >> v1:172.21.15.161:6789/0 conn(0x7f2cac06a5a0 legacy=0x7f2cac15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cabfff700 1 -- 172.21.15.148:0/3514459901 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0b04b0 con 0x7f2cac159ec0 2023-10-10T20:29:53.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2cabfff700 1 --2- 172.21.15.148:0/3514459901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac159ec0 0x7f2cac0af940 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2cac089910 tx=0x7f2c9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dde5e0f6c6b8c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2caaffd700 1 -- 172.21.15.148:0/3514459901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00b860 con 0x7f2cac159ec0 2023-10-10T20:29:53.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2caaffd700 1 -- 172.21.15.148:0/3514459901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c00b9c0 con 0x7f2cac159ec0 2023-10-10T20:29:53.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.873+0000 7f2caaffd700 1 -- 172.21.15.148:0/3514459901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00bd60 con 0x7f2cac159ec0 2023-10-10T20:29:53.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 -- 172.21.15.148:0/3514459901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac159ec0 msgr2=0x7f2cac0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 --2- 172.21.15.148:0/3514459901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac159ec0 0x7f2cac0af940 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2cac089910 tx=0x7f2c9c009250 comp rx=0 tx=0).stop 2023-10-10T20:29:53.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 -- 172.21.15.148:0/3514459901 shutdown_connections 2023-10-10T20:29:53.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 --2- 172.21.15.148:0/3514459901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac159ec0 0x7f2cac0af940 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 --2- 172.21.15.148:0/3514459901 >> v2:172.21.15.161:3300/0 conn(0x7f2cac15a290 0x7f2cac0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 -- 172.21.15.148:0/3514459901 >> 172.21.15.148:0/3514459901 conn(0x7f2cac0d1880 msgr2=0x7f2cac0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:53.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 -- 172.21.15.148:0/3514459901 shutdown_connections 2023-10-10T20:29:53.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.874+0000 7f2cb286c700 1 -- 172.21.15.148:0/3514459901 wait complete. 2023-10-10T20:29:53.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.875+0000 7f2cb286c700 1 Processor -- start 2023-10-10T20:29:53.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.875+0000 7f2cb286c700 1 -- start start 2023-10-10T20:29:53.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cac06a5a0 0x7f2cac15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cac159ec0 0x7f2cac15abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac155c40 con 0x7f2cac15a290 2023-10-10T20:29:53.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0b19d0 con 0x7f2cac159ec0 2023-10-10T20:29:53.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac15dc90 con 0x7f2cac06a5a0 2023-10-10T20:29:53.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35320/0 (socket says 172.21.15.148:35320) 2023-10-10T20:29:53.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 -- 172.21.15.148:0/1604050948 learned_addr learned my addr 172.21.15.148:0/1604050948 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:53.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cab7fe700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cac159ec0 0x7f2cac15abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cabfff700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cac06a5a0 0x7f2cac15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 -- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cac06a5a0 msgr2=0x7f2cac15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cac06a5a0 0x7f2cac15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 -- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cac159ec0 msgr2=0x7f2cac15abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:53.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cac159ec0 0x7f2cac15abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0094f0 con 0x7f2cac15a290 2023-10-10T20:29:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb0e09700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c8e0 tx=0x7f2ca4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4012630 con 0x7f2cac15a290 2023-10-10T20:29:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca4012790 con 0x7f2cac15a290 2023-10-10T20:29:53.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac15df70 con 0x7f2cac15a290 2023-10-10T20:29:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4012b50 con 0x7f2cac15a290 2023-10-10T20:29:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.876+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac15e530 con 0x7f2cac15a290 2023-10-10T20:29:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.877+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac04c3c0 con 0x7f2cac15a290 2023-10-10T20:29:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ca401a3a0 con 0x7f2cac15a290 2023-10-10T20:29:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2c93fff700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 0x7f2c980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f2ca4084710 con 0x7f2cac15a290 2023-10-10T20:29:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2cabfff700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 0x7f2c980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=168}) v3 -- 0x7f2c9806eb50 con 0x7f2cac15a290 2023-10-10T20:29:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.878+0000 7f2cabfff700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 0x7f2c980693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2cac089910 tx=0x7f2c9c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:53.882+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ca4051910 con 0x7f2cac15a290 2023-10-10T20:29:54.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.105+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2cac0ceab0 con 0x7f2cac15a290 2023-10-10T20:29:54.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.731+0000 7f2c93fff700 1 -- 172.21.15.148:0/1604050948 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ca4054f30 con 0x7f2cac15a290 2023-10-10T20:29:54.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 msgr2=0x7f2c980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 0x7f2c980693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2cac089910 tx=0x7f2c9c017000 comp rx=0 tx=0).stop 2023-10-10T20:29:54.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 msgr2=0x7f2cac15d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400c8e0 tx=0x7f2ca4001710 comp rx=0 tx=0).stop 2023-10-10T20:29:54.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 shutdown_connections 2023-10-10T20:29:54.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2c98066f30 0x7f2c980693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cac15a290 0x7f2cac15d080 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cac06a5a0 0x7f2cac15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 --2- 172.21.15.148:0/1604050948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cac159ec0 0x7f2cac15abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 >> 172.21.15.148:0/1604050948 conn(0x7f2cac0d1880 msgr2=0x7f2cac0c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:54.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 shutdown_connections 2023-10-10T20:29:54.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.734+0000 7f2cb286c700 1 -- 172.21.15.148:0/1604050948 wait complete. 2023-10-10T20:29:54.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:29:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.906+0000 7f8232a1b700 1 Processor -- start 2023-10-10T20:29:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.906+0000 7f8232a1b700 1 -- start start 2023-10-10T20:29:54.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f8232a1b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f822c15daf0 0x7f822c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f8232a1b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 0x7f822c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f8232a1b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822c067660 con 0x7f822c0b0f80 2023-10-10T20:29:54.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f8232a1b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c15df90 con 0x7f822c15daf0 2023-10-10T20:29:54.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f8232a1b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c06bfa0 con 0x7f822c0c55e0 2023-10-10T20:29:54.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f822bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f822c0b0f80 0x7f822c06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38358/0 (socket says 172.21.15.148:38358) 2023-10-10T20:29:54.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f822bfff700 1 -- 172.21.15.148:0/3490394427 learned_addr learned my addr 172.21.15.148:0/3490394427 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:54.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.907+0000 7f822b7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f822c15daf0 0x7f822c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f8230fb8700 1 --2- 172.21.15.148:0/3490394427 >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 0x7f822c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822affd700 1 -- 172.21.15.148:0/3490394427 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061578962 0 0) 0x7f822c067660 con 0x7f822c0b0f80 2023-10-10T20:29:54.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822affd700 1 -- 172.21.15.148:0/3490394427 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8214003570 con 0x7f822c0b0f80 2023-10-10T20:29:54.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822b7fe700 1 -- 172.21.15.148:0/3490394427 >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 msgr2=0x7f822c15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:54.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822b7fe700 1 --2- 172.21.15.148:0/3490394427 >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 0x7f822c15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822b7fe700 1 -- 172.21.15.148:0/3490394427 >> v1:172.21.15.148:6789/0 conn(0x7f822c0b0f80 legacy=0x7f822c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822b7fe700 1 -- 172.21.15.148:0/3490394427 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0684c0 con 0x7f822c15daf0 2023-10-10T20:29:54.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f8230fb8700 1 --2- 172.21.15.148:0/3490394427 >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 0x7f822c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:29:54.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822b7fe700 1 --2- 172.21.15.148:0/3490394427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c15daf0 0x7f822c0c50a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f822c06f140 tx=0x7f82200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc334474e2dd1b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:54.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822affd700 1 -- 172.21.15.148:0/3490394427 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220011630 con 0x7f822c15daf0 2023-10-10T20:29:54.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822affd700 1 -- 172.21.15.148:0/3490394427 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220011790 con 0x7f822c15daf0 2023-10-10T20:29:54.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.908+0000 7f822affd700 1 -- 172.21.15.148:0/3490394427 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220011b30 con 0x7f822c15daf0 2023-10-10T20:29:54.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 -- 172.21.15.148:0/3490394427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c15daf0 msgr2=0x7f822c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 --2- 172.21.15.148:0/3490394427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c15daf0 0x7f822c0c50a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f822c06f140 tx=0x7f82200091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 -- 172.21.15.148:0/3490394427 shutdown_connections 2023-10-10T20:29:54.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 --2- 172.21.15.148:0/3490394427 >> v2:172.21.15.161:3300/0 conn(0x7f822c0c55e0 0x7f822c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 --2- 172.21.15.148:0/3490394427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c15daf0 0x7f822c0c50a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 -- 172.21.15.148:0/3490394427 >> 172.21.15.148:0/3490394427 conn(0x7f822c069870 msgr2=0x7f822c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:54.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 -- 172.21.15.148:0/3490394427 shutdown_connections 2023-10-10T20:29:54.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.909+0000 7f8232a1b700 1 -- 172.21.15.148:0/3490394427 wait complete. 2023-10-10T20:29:54.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.910+0000 7f8232a1b700 1 Processor -- start 2023-10-10T20:29:54.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.910+0000 7f8232a1b700 1 -- start start 2023-10-10T20:29:54.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822c0c55e0 0x7f822c15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f822c15daf0 0x7f822c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0b4280 con 0x7f822c0b0f80 2023-10-10T20:29:54.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0b2bf0 con 0x7f822c0c55e0 2023-10-10T20:29:54.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0b1fe0 con 0x7f822c15daf0 2023-10-10T20:29:54.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35332/0 (socket says 172.21.15.148:35332) 2023-10-10T20:29:54.923 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 -- 172.21.15.148:0/717833820 learned_addr learned my addr 172.21.15.148:0/717833820 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822b7fe700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822c0c55e0 0x7f822c15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8230fb8700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f822c15daf0 0x7f822c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 -- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f822c15daf0 msgr2=0x7f822c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f822c15daf0 0x7f822c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 -- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822c0c55e0 msgr2=0x7f822c15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822c0c55e0 0x7f822c15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220009420 con 0x7f822c0b0f80 2023-10-10T20:29:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f822bfff700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c7c0 tx=0x7f821c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c005b30 con 0x7f822c0b0f80 2023-10-10T20:29:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c005c90 con 0x7f822c0b0f80 2023-10-10T20:29:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c007340 con 0x7f822c0b0f80 2023-10-10T20:29:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.911+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c0b22c0 con 0x7f822c0b0f80 2023-10-10T20:29:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.912+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c0b2880 con 0x7f822c0b0f80 2023-10-10T20:29:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c04c3c0 con 0x7f822c0b0f80 2023-10-10T20:29:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f821c01b400 con 0x7f822c0b0f80 2023-10-10T20:29:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f8213fff700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 0x7f82180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f821c0840b0 con 0x7f822c0b0f80 2023-10-10T20:29:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f822b7fe700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 0x7f82180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.913+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=168}) v3 -- 0x7f821806eb50 con 0x7f822c0b0f80 2023-10-10T20:29:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.914+0000 7f822b7fe700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 0x7f82180693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f82200093f0 tx=0x7f8220017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:54.917+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f821c051330 con 0x7f822c0b0f80 2023-10-10T20:29:55.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.101+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f822c068be0 con 0x7f822c0b0f80 2023-10-10T20:29:55.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.583+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(168..168 src has 1..168) v4 ==== 679+0+0 (secure 0 0 0) 0x7f821c0556f0 con 0x7f822c0b0f80 2023-10-10T20:29:55.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.583+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f821806cf80 con 0x7f822c0b0f80 2023-10-10T20:29:55.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.586+0000 7f8213fff700 1 -- 172.21.15.148:0/717833820 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v168) v1 ==== 181+0+0 (secure 0 0 0) 0x7f821c054950 con 0x7f822c0b0f80 2023-10-10T20:29:55.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 msgr2=0x7f82180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 0x7f82180693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f82200093f0 tx=0x7f8220017000 comp rx=0 tx=0).stop 2023-10-10T20:29:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 msgr2=0x7f822c0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c7c0 tx=0x7f821c007e00 comp rx=0 tx=0).stop 2023-10-10T20:29:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 shutdown_connections 2023-10-10T20:29:55.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8218066f30 0x7f82180693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f822c15daf0 0x7f822c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f822c0b0f80 0x7f822c0b1920 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 --2- 172.21.15.148:0/717833820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822c0c55e0 0x7f822c15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 >> 172.21.15.148:0/717833820 conn(0x7f822c069870 msgr2=0x7f822c154c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:55.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 shutdown_connections 2023-10-10T20:29:55.597 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.592+0000 7f8232a1b700 1 -- 172.21.15.148:0/717833820 wait complete. 2023-10-10T20:29:55.597 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:29:55.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:29:55.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.723+0000 7f0cca04c700 1 Processor -- start 2023-10-10T20:29:55.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.723+0000 7f0cca04c700 1 -- start start 2023-10-10T20:29:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cca04c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0cc4159e50 0x7f0cc40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cca04c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40b0d60 con 0x7f0cc40b3370 2023-10-10T20:29:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cca04c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40d15b0 con 0x7f0cc4068cb0 2023-10-10T20:29:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cca04c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40d1890 con 0x7f0cc4159e50 2023-10-10T20:29:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cc37fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0cc4159e50 0x7f0cc40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cc2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0cc40b3370 0x7f0cc40b11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38364/0 (socket says 172.21.15.148:38364) 2023-10-10T20:29:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.724+0000 7f0cc2ffd700 1 -- 172.21.15.148:0/1472961541 learned_addr learned my addr 172.21.15.148:0/1472961541 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc37fe700 1 -- 172.21.15.148:0/1472961541 >> v1:172.21.15.161:6789/0 conn(0x7f0cc4068cb0 legacy=0x7f0cc40690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc37fe700 1 -- 172.21.15.148:0/1472961541 >> v1:172.21.15.148:6789/0 conn(0x7f0cc40b3370 legacy=0x7f0cc40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc37fe700 1 -- 172.21.15.148:0/1472961541 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40b09f0 con 0x7f0cc4159e50 2023-10-10T20:29:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc27fc700 1 -- 172.21.15.148:0/1472961541 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857510745 0 0) 0x7f0cc40b0d60 con 0x7f0cc40b3370 2023-10-10T20:29:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc37fe700 1 --2- 172.21.15.148:0/1472961541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc4159e50 0x7f0cc40b2e30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009e60 tx=0x7f0cb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79021df80ff4560e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc27fc700 1 -- 172.21.15.148:0/1472961541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400e070 con 0x7f0cc4159e50 2023-10-10T20:29:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc27fc700 1 -- 172.21.15.148:0/1472961541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb400bbf0 con 0x7f0cc4159e50 2023-10-10T20:29:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cc27fc700 1 -- 172.21.15.148:0/1472961541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4012430 con 0x7f0cc4159e50 2023-10-10T20:29:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cca04c700 1 -- 172.21.15.148:0/1472961541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc4159e50 msgr2=0x7f0cc40b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cca04c700 1 --2- 172.21.15.148:0/1472961541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc4159e50 0x7f0cc40b2e30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009e60 tx=0x7f0cb4009780 comp rx=0 tx=0).stop 2023-10-10T20:29:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.725+0000 7f0cca04c700 1 -- 172.21.15.148:0/1472961541 shutdown_connections 2023-10-10T20:29:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.726+0000 7f0cca04c700 1 --2- 172.21.15.148:0/1472961541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc4159e50 0x7f0cc40b2e30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.726+0000 7f0cca04c700 1 -- 172.21.15.148:0/1472961541 >> 172.21.15.148:0/1472961541 conn(0x7f0cc4068130 msgr2=0x7f0cc40d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.726+0000 7f0cca04c700 1 -- 172.21.15.148:0/1472961541 shutdown_connections 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.726+0000 7f0cca04c700 1 -- 172.21.15.148:0/1472961541 wait complete. 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.727+0000 7f0cca04c700 1 Processor -- start 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.727+0000 7f0cca04c700 1 -- start start 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.727+0000 7f0cca04c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cc4068cb0 0x7f0cc41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.727+0000 7f0cca04c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cc4159e50 0x7f0cc415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c8e30 con 0x7f0cc40b3370 2023-10-10T20:29:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c82b0 con 0x7f0cc4159e50 2023-10-10T20:29:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40c8590 con 0x7f0cc4068cb0 2023-10-10T20:29:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:55.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35354/0 (socket says 172.21.15.148:35354) 2023-10-10T20:29:55.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 -- 172.21.15.148:0/980410256 learned_addr learned my addr 172.21.15.148:0/980410256 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:55.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc37fe700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cc4068cb0 0x7f0cc41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:55.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc3fff700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cc4159e50 0x7f0cc415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:55.782 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 -- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cc4068cb0 msgr2=0x7f0cc41581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cc4068cb0 0x7f0cc41581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 -- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cc4159e50 msgr2=0x7f0cc415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:55.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cc4159e50 0x7f0cc415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:55.783 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 -- 172.21.15.148:0/980410256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4009a20 con 0x7f0cc40b3370 2023-10-10T20:29:55.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cc2ffd700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800c810 tx=0x7f0cb8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:55.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8005c00 con 0x7f0cc40b3370 2023-10-10T20:29:55.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb8005d60 con 0x7f0cc40b3370 2023-10-10T20:29:55.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8007320 con 0x7f0cc40b3370 2023-10-10T20:29:55.784 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc40c8870 con 0x7f0cc40b3370 2023-10-10T20:29:55.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.728+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4158730 con 0x7f0cc40b3370 2023-10-10T20:29:55.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.729+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc404c3c0 con 0x7f0cc40b3370 2023-10-10T20:29:55.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.729+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cb801b400 con 0x7f0cc40b3370 2023-10-10T20:29:55.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.730+0000 7f0cab7fe700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:55.785 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.730+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0cb8083eb0 con 0x7f0cc40b3370 2023-10-10T20:29:55.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.730+0000 7f0cc37fe700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:55.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.730+0000 7f0cc37fe700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 0x7f0cb00693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40097c0 tx=0x7f0cb4018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:55.786 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.735+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cb80512e0 con 0x7f0cc40b3370 2023-10-10T20:29:55.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:55.917+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0cc40690c0 con 0x7f0cc40b3370 2023-10-10T20:29:56.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.828+0000 7f0cab7fe700 1 -- 172.21.15.148:0/980410256 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v169) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0cb8054900 con 0x7f0cc40b3370 2023-10-10T20:29:56.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 msgr2=0x7f0cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 0x7f0cb00693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40097c0 tx=0x7f0cb4018000 comp rx=0 tx=0).stop 2023-10-10T20:29:56.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 msgr2=0x7f0cc40c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800c810 tx=0x7f0cb8007db0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 shutdown_connections 2023-10-10T20:29:56.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cc4159e50 0x7f0cc415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cc4068cb0 0x7f0cc41581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 --2- 172.21.15.148:0/980410256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cc40b3370 0x7f0cc40c9e00 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 >> 172.21.15.148:0/980410256 conn(0x7f0cc4068130 msgr2=0x7f0cc40b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:56.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 shutdown_connections 2023-10-10T20:29:56.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.835+0000 7f0cca04c700 1 -- 172.21.15.148:0/980410256 wait complete. 2023-10-10T20:29:56.840 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:29:56.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:29:56.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.965+0000 7f236759e700 1 Processor -- start 2023-10-10T20:29:56.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.965+0000 7f236759e700 1 -- start start 2023-10-10T20:29:56.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f236815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236759e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23680b6580 0x7f236806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236759e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680b4d70 con 0x7f236806a830 2023-10-10T20:29:56.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236759e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236806ac70 con 0x7f23680b6580 2023-10-10T20:29:56.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cd8d0 con 0x7f23680cce30 2023-10-10T20:29:56.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f2366d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f236806a830 0x7f236815d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38368/0 (socket says 172.21.15.148:38368) 2023-10-10T20:29:56.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f2366d9d700 1 -- 172.21.15.148:0/3514125787 learned_addr learned my addr 172.21.15.148:0/3514125787 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:56.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f2365d9b700 1 --2- 172.21.15.148:0/3514125787 >> v2:172.21.15.161:3300/0 conn(0x7f23680b6580 0x7f236806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236659c700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f236815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436743648 0 0) 0x7f23680b4d70 con 0x7f236806a830 2023-10-10T20:29:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.966+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003570 con 0x7f236806a830 2023-10-10T20:29:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140913977 0 0) 0x7f2350003570 con 0x7f236806a830 2023-10-10T20:29:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 msgr2=0x7f236815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:56.974 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f236815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 >> v2:172.21.15.161:3300/0 conn(0x7f23680b6580 msgr2=0x7f236806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 --2- 172.21.15.148:0/3514125787 >> v2:172.21.15.161:3300/0 conn(0x7f23680b6580 0x7f236806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680b4440 con 0x7f236806a830 2023-10-10T20:29:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f2360001470 con 0x7f236806a830 2023-10-10T20:29:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 >> v1:172.21.15.148:6789/0 conn(0x7f236806a830 legacy=0x7f236815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350003a60 0x7f2350003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f2350004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2350004d50 0x7f23500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236806ac70 con 0x7f2350004d50 2023-10-10T20:29:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cd8d0 con 0x7f2350003a60 2023-10-10T20:29:56.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235000ba60 con 0x7f23680cce30 2023-10-10T20:29:56.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2350004d50 0x7f23500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236659c700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350003a60 0x7f2350003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2365d9b700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f2350004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 -- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 msgr2=0x7f2350004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:56.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23680cce30 0x7f2350004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 -- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350003a60 msgr2=0x7f2350003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:56.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350003a60 0x7f2350003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 -- 172.21.15.148:0/3514125787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350003570 con 0x7f2350004d50 2023-10-10T20:29:56.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2350004d50 0x7f23500051c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f236000c3c0 tx=0x7f23680cdca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32c8574d340594 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:56.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000cc30 con 0x7f2350004d50 2023-10-10T20:29:56.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236000cd90 con 0x7f2350004d50 2023-10-10T20:29:56.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.967+0000 7f236559a700 1 -- 172.21.15.148:0/3514125787 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360005b50 con 0x7f2350004d50 2023-10-10T20:29:56.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 -- 172.21.15.148:0/3514125787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2350004d50 msgr2=0x7f23500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 --2- 172.21.15.148:0/3514125787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2350004d50 0x7f23500051c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f236000c3c0 tx=0x7f23680cdca0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f2366d9d700 1 -- 172.21.15.148:0/3514125787 reap_dead start 2023-10-10T20:29:56.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 -- 172.21.15.148:0/3514125787 shutdown_connections 2023-10-10T20:29:56.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 -- 172.21.15.148:0/3514125787 >> 172.21.15.148:0/3514125787 conn(0x7f2368068cb0 msgr2=0x7f23680690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:56.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 -- 172.21.15.148:0/3514125787 shutdown_connections 2023-10-10T20:29:56.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.968+0000 7f236759e700 1 -- 172.21.15.148:0/3514125787 wait complete. 2023-10-10T20:29:56.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.969+0000 7f236759e700 1 Processor -- start 2023-10-10T20:29:56.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.969+0000 7f236759e700 1 -- start start 2023-10-10T20:29:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350004d50 0x7f236814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23681569a0 0x7f2368156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368157350 con 0x7f2368154890 2023-10-10T20:29:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368155270 con 0x7f2350004d50 2023-10-10T20:29:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368155510 con 0x7f23681569a0 2023-10-10T20:29:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35370/0 (socket says 172.21.15.148:35370) 2023-10-10T20:29:56.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 -- 172.21.15.148:0/3071598597 learned_addr learned my addr 172.21.15.148:0/3071598597 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f236659c700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350004d50 0x7f236814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2365d9b700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23681569a0 0x7f2368156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 -- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23681569a0 msgr2=0x7f2368156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23681569a0 0x7f2368156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 -- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350004d50 msgr2=0x7f236814e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350004d50 0x7f236814e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 -- 172.21.15.148:0/3071598597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23681557f0 con 0x7f2368154890 2023-10-10T20:29:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.970+0000 7f2366d9d700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2360009020 tx=0x7f2360015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.971+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000cc30 con 0x7f2368154890 2023-10-10T20:29:56.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.971+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360001430 con 0x7f2368154890 2023-10-10T20:29:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.971+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368155ad0 con 0x7f2368154890 2023-10-10T20:29:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.971+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012630 con 0x7f2368154890 2023-10-10T20:29:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.971+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236814eb70 con 0x7f2368154890 2023-10-10T20:29:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.972+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368155ff0 con 0x7f2368154890 2023-10-10T20:29:56.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.972+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2360019030 con 0x7f2368154890 2023-10-10T20:29:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.972+0000 7f23567fc700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 0x7f23480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.973+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2360084ac0 con 0x7f2368154890 2023-10-10T20:29:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.973+0000 7f236659c700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 0x7f23480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.973+0000 7f236659c700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 0x7f23480693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f235800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:56.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:56.977+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2368155ff0 con 0x7f2368154890 2023-10-10T20:29:57.162 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.160+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2368155ff0 con 0x7f2368154890 2023-10-10T20:29:57.168 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.166+0000 7f23567fc700 1 -- 172.21.15.148:0/3071598597 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2368155ff0 con 0x7f2368154890 2023-10-10T20:29:57.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 msgr2=0x7f23480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.170 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 0x7f23480693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f235800b040 comp rx=0 tx=0).stop 2023-10-10T20:29:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 msgr2=0x7f2368154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2360009020 tx=0x7f2360015000 comp rx=0 tx=0).stop 2023-10-10T20:29:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 shutdown_connections 2023-10-10T20:29:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2348066f30 0x7f23480693e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2368154890 0x7f2368154d30 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2350004d50 0x7f236814e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 --2- 172.21.15.148:0/3071598597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23681569a0 0x7f2368156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 >> 172.21.15.148:0/3071598597 conn(0x7f2368068cb0 msgr2=0x7f2368150670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 shutdown_connections 2023-10-10T20:29:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.168+0000 7f236759e700 1 -- 172.21.15.148:0/3071598597 wait complete. 2023-10-10T20:29:57.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-10T20:29:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.295+0000 7f61da91d700 1 Processor -- start 2023-10-10T20:29:57.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.295+0000 7f61da91d700 1 -- start start 2023-10-10T20:29:57.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61da91d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61d4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61da91d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d415cbb0 con 0x7f61d4157870 2023-10-10T20:29:57.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61da91d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d415bfa0 con 0x7f61d415a620 2023-10-10T20:29:57.302 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61da91d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d415bdb0 con 0x7f61d4157c40 2023-10-10T20:29:57.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61d3fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f61d4157870 0x7f61d415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38376/0 (socket says 172.21.15.148:38376) 2023-10-10T20:29:57.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61d3fff700 1 -- 172.21.15.148:0/105914779 learned_addr learned my addr 172.21.15.148:0/105914779 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:57.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661458986 0 0) 0x7f61d415cbb0 con 0x7f61d4157870 2023-10-10T20:29:57.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61d8eba700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61d4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.296+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61bc003570 con 0x7f61d4157870 2023-10-10T20:29:57.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151065274 0 0) 0x7f61bc003570 con 0x7f61d4157870 2023-10-10T20:29:57.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 msgr2=0x7f61d4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:57.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61d4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 >> v1:172.21.15.161:6789/0 conn(0x7f61d415a620 legacy=0x7f61d415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d415c280 con 0x7f61d4157870 2023-10-10T20:29:57.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f61c4001470 con 0x7f61d4157870 2023-10-10T20:29:57.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 >> v1:172.21.15.148:6789/0 conn(0x7f61d4157870 legacy=0x7f61d415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61bc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61bc003fd0 0x7f61bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00bb10 con 0x7f61bc003fd0 2023-10-10T20:29:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00bc80 con 0x7f61bc004d50 2023-10-10T20:29:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00be40 con 0x7f61d4157c40 2023-10-10T20:29:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61bc003fd0 0x7f61bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d3fff700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61bc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d8eba700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 -- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 msgr2=0x7f61bc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61d4157c40 0x7f61bc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 -- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61bc004d50 msgr2=0x7f61bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:57.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61bc004d50 0x7f61bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 -- 172.21.15.148:0/105914779 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc003570 con 0x7f61bc003fd0 2023-10-10T20:29:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d37fe700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61bc003fd0 0x7f61bc004810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f61c8001970 tx=0x7f61c8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd879b70adc09144 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800c5d0 con 0x7f61bc003fd0 2023-10-10T20:29:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800c750 con 0x7f61bc003fd0 2023-10-10T20:29:57.310 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.297+0000 7f61d2ffd700 1 -- 172.21.15.148:0/105914779 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800cb10 con 0x7f61bc003fd0 2023-10-10T20:29:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 -- 172.21.15.148:0/105914779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61bc003fd0 msgr2=0x7f61bc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 --2- 172.21.15.148:0/105914779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61bc003fd0 0x7f61bc004810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f61c8001970 tx=0x7f61c8007d10 comp rx=0 tx=0).stop 2023-10-10T20:29:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61d8eba700 1 -- 172.21.15.148:0/105914779 reap_dead start 2023-10-10T20:29:57.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 -- 172.21.15.148:0/105914779 shutdown_connections 2023-10-10T20:29:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 -- 172.21.15.148:0/105914779 >> 172.21.15.148:0/105914779 conn(0x7f61d4068ce0 msgr2=0x7f61d40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 -- 172.21.15.148:0/105914779 shutdown_connections 2023-10-10T20:29:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.298+0000 7f61da91d700 1 -- 172.21.15.148:0/105914779 wait complete. 2023-10-10T20:29:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.299+0000 7f61da91d700 1 Processor -- start 2023-10-10T20:29:57.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.299+0000 7f61da91d700 1 -- start start 2023-10-10T20:29:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61bc003fd0 0x7f61d4129d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d4151510 0x7f61d41533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40b15e0 con 0x7f61d414d330 2023-10-10T20:29:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4151980 con 0x7f61d4151510 2023-10-10T20:29:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61da91d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40b1e10 con 0x7f61bc003fd0 2023-10-10T20:29:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35376/0 (socket says 172.21.15.148:35376) 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 -- 172.21.15.148:0/430141183 learned_addr learned my addr 172.21.15.148:0/430141183 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d3fff700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61bc003fd0 0x7f61d4129d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d8eba700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d4151510 0x7f61d41533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 -- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61bc003fd0 msgr2=0x7f61d4129d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61bc003fd0 0x7f61d4129d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 -- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d4151510 msgr2=0x7f61d41533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d4151510 0x7f61d41533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.300+0000 7f61d37fe700 1 -- 172.21.15.148:0/430141183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4154160 con 0x7f61d414d330 2023-10-10T20:29:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61d37fe700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c3c0 tx=0x7f61c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8007af0 con 0x7f61d414d330 2023-10-10T20:29:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800c5d0 con 0x7f61d414d330 2023-10-10T20:29:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d40b17c0 con 0x7f61d414d330 2023-10-10T20:29:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800cb10 con 0x7f61d414d330 2023-10-10T20:29:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.301+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d414d7a0 con 0x7f61d414d330 2023-10-10T20:29:57.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.302+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d40b15e0 con 0x7f61d414d330 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.302+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61c8019030 con 0x7f61d414d330 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.302+0000 7f61bbfff700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.302+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f61c8085690 con 0x7f61d414d330 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.302+0000 7f61d3fff700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.303+0000 7f61d3fff700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001200 tx=0x7f61c4001e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.307+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61c8052c20 con 0x7f61d414d330 2023-10-10T20:29:57.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.563+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f61d404c3c0 con 0x7f61d414d330 2023-10-10T20:29:57.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.840+0000 7f61bbfff700 1 -- 172.21.15.148:0/430141183 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v495) v1 ==== 131+0+0 (secure 0 0 0) 0x7f61c8056240 con 0x7f61d414d330 2023-10-10T20:29:57.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 msgr2=0x7f61c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 0x7f61c00693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001200 tx=0x7f61c4001e70 comp rx=0 tx=0).stop 2023-10-10T20:29:57.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 msgr2=0x7f61d4150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c3c0 tx=0x7f61c8015000 comp rx=0 tx=0).stop 2023-10-10T20:29:57.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 shutdown_connections 2023-10-10T20:29:57.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61c0066f30 0x7f61c00693e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61d4151510 0x7f61d41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61bc003fd0 0x7f61d4129d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 --2- 172.21.15.148:0/430141183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61d414d330 0x7f61d4150fd0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 >> 172.21.15.148:0/430141183 conn(0x7f61d4068ce0 msgr2=0x7f61d40b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:57.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 shutdown_connections 2023-10-10T20:29:57.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.842+0000 7f61da91d700 1 -- 172.21.15.148:0/430141183 wait complete. 2023-10-10T20:29:57.857 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-10T20:29:57.857 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-10T20:29:57.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-10T20:29:57.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.970+0000 7ff885459700 1 Processor -- start 2023-10-10T20:29:57.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.970+0000 7ff885459700 1 -- start start 2023-10-10T20:29:57.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff885459700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 0x7ff8800cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff885459700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff880157ba0 0x7ff88006a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff885459700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88015c360 con 0x7ff88015b0d0 2023-10-10T20:29:57.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff885459700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88015b790 con 0x7ff880157ba0 2023-10-10T20:29:57.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff885459700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88015d800 con 0x7ff8801577d0 2023-10-10T20:29:57.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.971+0000 7ff8767fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff880157ba0 0x7ff88006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff880157ba0 0x7ff88006a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35386/0 (socket says 172.21.15.148:35386) 2023-10-10T20:29:57.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 -- 172.21.15.148:0/1104803006 learned_addr learned my addr 172.21.15.148:0/1104803006 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:57.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff87effd700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 0x7ff8800cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 -- 172.21.15.148:0/1104803006 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 msgr2=0x7ff8800cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:57.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 0x7ff8800cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 -- 172.21.15.148:0/1104803006 >> v1:172.21.15.161:6789/0 conn(0x7ff88015b0d0 legacy=0x7ff88015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:57.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 -- 172.21.15.148:0/1104803006 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88015ba30 con 0x7ff880157ba0 2023-10-10T20:29:57.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff87effd700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 0x7ff8800cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:57.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff8767fc700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880157ba0 0x7ff88006a520 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff8680099b0 tx=0x7ff868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14f53aeb8c43530 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff87e7fc700 1 -- 172.21.15.148:0/1104803006 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800b820 con 0x7ff880157ba0 2023-10-10T20:29:57.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff87e7fc700 1 -- 172.21.15.148:0/1104803006 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800b980 con 0x7ff880157ba0 2023-10-10T20:29:57.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.972+0000 7ff87e7fc700 1 -- 172.21.15.148:0/1104803006 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800bd20 con 0x7ff880157ba0 2023-10-10T20:29:57.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 -- 172.21.15.148:0/1104803006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880157ba0 msgr2=0x7ff88006a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880157ba0 0x7ff88006a520 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff8680099b0 tx=0x7ff868009250 comp rx=0 tx=0).stop 2023-10-10T20:29:57.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 -- 172.21.15.148:0/1104803006 shutdown_connections 2023-10-10T20:29:57.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8801577d0 0x7ff8800cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 --2- 172.21.15.148:0/1104803006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff880157ba0 0x7ff88006a520 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 -- 172.21.15.148:0/1104803006 >> 172.21.15.148:0/1104803006 conn(0x7ff880068ce0 msgr2=0x7ff8800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:57.981 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 -- 172.21.15.148:0/1104803006 shutdown_connections 2023-10-10T20:29:57.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.973+0000 7ff885459700 1 -- 172.21.15.148:0/1104803006 wait complete. 2023-10-10T20:29:57.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.974+0000 7ff885459700 1 Processor -- start 2023-10-10T20:29:57.982 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.974+0000 7ff885459700 1 -- start start 2023-10-10T20:29:57.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8801577d0 0x7ff8800b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 0x7ff8800b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800b3420 con 0x7ff88015b0d0 2023-10-10T20:29:57.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880158ea0 con 0x7ff8801577d0 2023-10-10T20:29:57.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff880159180 con 0x7ff880157ba0 2023-10-10T20:29:57.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35392/0 (socket says 172.21.15.148:35392) 2023-10-10T20:29:57.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 -- 172.21.15.148:0/2145363411 learned_addr learned my addr 172.21.15.148:0/2145363411 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:57.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff8767fc700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 0x7ff8800b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87effd700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8801577d0 0x7ff8800b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 -- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 msgr2=0x7ff8800b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 0x7ff8800b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 -- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8801577d0 msgr2=0x7ff8800b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:57.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8801577d0 0x7ff8800b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:57.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 -- 172.21.15.148:0/2145363411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8680094f0 con 0x7ff88015b0d0 2023-10-10T20:29:57.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff8767fc700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 0x7ff8800b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:57.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff87f7fe700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c930 tx=0x7ff8780026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800ce70 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800a7e0 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff880159460 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.975+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800aba0 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.976+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff880158a20 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.977+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88004c3c0 con 0x7ff88015b0d0 2023-10-10T20:29:57.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.977+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff878012420 con 0x7ff88015b0d0 2023-10-10T20:29:57.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.977+0000 7ff8777fe700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:57.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.977+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff878084600 con 0x7ff88015b0d0 2023-10-10T20:29:57.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.977+0000 7ff87effd700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:57.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.978+0000 7ff87effd700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:57.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:57.982+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8780556b0 con 0x7ff88015b0d0 2023-10-10T20:29:58.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.239+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7ff88015bd10 con 0x7ff88015b0d0 2023-10-10T20:29:58.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.838+0000 7ff8777fe700 1 -- 172.21.15.148:0/2145363411 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v496) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff878055240 con 0x7ff88015b0d0 2023-10-10T20:29:58.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 msgr2=0x7ff8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870008040 comp rx=0 tx=0).stop 2023-10-10T20:29:58.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 msgr2=0x7ff8800b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c930 tx=0x7ff8780026e0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 shutdown_connections 2023-10-10T20:29:58.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff88015b0d0 0x7ff8800b2ee0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8801577d0 0x7ff8800b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 --2- 172.21.15.148:0/2145363411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff880157ba0 0x7ff8800b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 >> 172.21.15.148:0/2145363411 conn(0x7ff880068ce0 msgr2=0x7ff8801564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 shutdown_connections 2023-10-10T20:29:58.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.841+0000 7ff885459700 1 -- 172.21.15.148:0/2145363411 wait complete. 2023-10-10T20:29:58.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-10T20:29:58.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.970+0000 7fb4e0631700 1 Processor -- start 2023-10-10T20:29:58.973 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.970+0000 7fb4e0631700 1 -- start start 2023-10-10T20:29:58.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4e0631700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb4d815a290 0x7fb4d806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4e0631700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4e0631700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d80b0790 con 0x7fb4d8159ec0 2023-10-10T20:29:58.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4e0631700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d806af60 con 0x7fb4d815a290 2023-10-10T20:29:58.975 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4e0631700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d80ce1f0 con 0x7fb4d806aae0 2023-10-10T20:29:58.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4de3cd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb4d8159ec0 0x7fb4d80af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38384/0 (socket says 172.21.15.148:38384) 2023-10-10T20:29:58.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4de3cd700 1 -- 172.21.15.148:0/914195263 learned_addr learned my addr 172.21.15.148:0/914195263 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:29:58.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/914195263 >> v2:172.21.15.148:3300/0 conn(0x7fb4d815a290 0x7fb4d806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4debce700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.976 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/914195263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 msgr2=0x7fb4d80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/914195263 >> v1:172.21.15.148:6789/0 conn(0x7fb4d8159ec0 legacy=0x7fb4d80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.971+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/914195263 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d80b0b20 con 0x7fb4d815a290 2023-10-10T20:29:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d815a290 0x7fb4d806a5a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f934e6ca1df697 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4dd3cb700 1 -- 172.21.15.148:0/914195263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009e80 con 0x7fb4d815a290 2023-10-10T20:29:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4dd3cb700 1 -- 172.21.15.148:0/914195263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800b860 con 0x7fb4d815a290 2023-10-10T20:29:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4dd3cb700 1 -- 172.21.15.148:0/914195263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800bbe0 con 0x7fb4d815a290 2023-10-10T20:29:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 -- 172.21.15.148:0/914195263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d815a290 msgr2=0x7fb4d806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d815a290 0x7fb4d806a5a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098e0 tx=0x7fb4c80091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 -- 172.21.15.148:0/914195263 shutdown_connections 2023-10-10T20:29:58.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.979 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 --2- 172.21.15.148:0/914195263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d815a290 0x7fb4d806a5a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 -- 172.21.15.148:0/914195263 >> 172.21.15.148:0/914195263 conn(0x7fb4d80d1880 msgr2=0x7fb4d80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:58.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 -- 172.21.15.148:0/914195263 shutdown_connections 2023-10-10T20:29:58.980 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.972+0000 7fb4e0631700 1 -- 172.21.15.148:0/914195263 wait complete. 2023-10-10T20:29:58.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.973+0000 7fb4e0631700 1 Processor -- start 2023-10-10T20:29:58.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.973+0000 7fb4e0631700 1 -- start start 2023-10-10T20:29:58.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d81556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.983 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d815a290 0x7fb4d8152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d816e360 con 0x7fb4d8159ec0 2023-10-10T20:29:58.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d816d750 con 0x7fb4d815a290 2023-10-10T20:29:58.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4e0631700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d816da30 con 0x7fb4d806aae0 2023-10-10T20:29:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4ddbcc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4ddbcc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35406/0 (socket says 172.21.15.148:35406) 2023-10-10T20:29:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/3061063892 learned_addr learned my addr 172.21.15.148:0/3061063892 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4de3cd700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d81556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.974+0000 7fb4debce700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d815a290 0x7fb4d8152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 msgr2=0x7fb4d81556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d81556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.986 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d815a290 msgr2=0x7fb4d8152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d815a290 0x7fb4d8152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 -- 172.21.15.148:0/3061063892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009420 con 0x7fb4d8159ec0 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ddbcc700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8071eb0 tx=0x7fb4c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800bbe0 con 0x7fb4d8159ec0 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8011420 con 0x7fb4d8159ec0 2023-10-10T20:29:58.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d816dd10 con 0x7fb4d8159ec0 2023-10-10T20:29:58.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80117c0 con 0x7fb4d8159ec0 2023-10-10T20:29:58.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.975+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d8154a70 con 0x7fb4d8159ec0 2023-10-10T20:29:58.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.976+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d804c3c0 con 0x7fb4d8159ec0 2023-10-10T20:29:58.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.976+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c8011bb0 con 0x7fb4d8159ec0 2023-10-10T20:29:58.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.977+0000 7fb4ce7fc700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:58.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.977+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb4c8085b00 con 0x7fb4d8159ec0 2023-10-10T20:29:58.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.977+0000 7fb4de3cd700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:58.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.977+0000 7fb4de3cd700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4001820 tx=0x7fb4d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:58.981+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c8053070 con 0x7fb4d8159ec0 2023-10-10T20:29:59.163 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.161+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4d80b04b0 con 0x7fb4d8159ec0 2023-10-10T20:29:59.169 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.167+0000 7fb4ce7fc700 1 -- 172.21.15.148:0/3061063892 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4c8011e60 con 0x7fb4d8159ec0 2023-10-10T20:29:59.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 msgr2=0x7fb4bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 0x7fb4bc0693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4001820 tx=0x7fb4d4009040 comp rx=0 tx=0).stop 2023-10-10T20:29:59.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 msgr2=0x7fb4d815f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8071eb0 tx=0x7fb4c8017000 comp rx=0 tx=0).stop 2023-10-10T20:29:59.172 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 shutdown_connections 2023-10-10T20:29:59.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb4bc066f30 0x7fb4bc0693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4d815a290 0x7fb4d8152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4d806aae0 0x7fb4d81556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 --2- 172.21.15.148:0/3061063892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4d8159ec0 0x7fb4d815f4d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.173 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 >> 172.21.15.148:0/3061063892 conn(0x7fb4d80d1880 msgr2=0x7fb4d8158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:59.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 shutdown_connections 2023-10-10T20:29:59.174 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.169+0000 7fb4e0631700 1 -- 172.21.15.148:0/3061063892 wait complete. 2023-10-10T20:29:59.183 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:29:59.184 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:29:59.224 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:29:59.267 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:29:59.267 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-10T20:29:59.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:29:59.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.379+0000 7fc82e624700 1 Processor -- start 2023-10-10T20:29:59.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.379+0000 7fc82e624700 1 -- start start 2023-10-10T20:29:59.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82e624700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82e624700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82e624700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82815cbb0 con 0x7fc82815a620 2023-10-10T20:29:59.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82e624700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82815bfa0 con 0x7fc828157870 2023-10-10T20:29:59.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82e624700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82815c280 con 0x7fc828157c40 2023-10-10T20:29:59.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82cbc1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc827fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82cbc1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:59004/0 (socket says 172.21.15.148:59004) 2023-10-10T20:29:59.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc827fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:57080/0 (socket says 172.21.15.148:57080) 2023-10-10T20:29:59.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc82cbc1700 1 -- 172.21.15.148:0/1006943450 learned_addr learned my addr 172.21.15.148:0/1006943450 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:59.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.380+0000 7fc8277fe700 1 --1- 172.21.15.148:0/1006943450 >> v1:172.21.15.161:6789/0 conn(0x7fc82815a620 0x7fc82815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:45954/0 (socket says 172.21.15.148:45954) 2023-10-10T20:29:59.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82cbc1700 1 -- 172.21.15.148:0/1006943450 >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 msgr2=0x7fc82815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:59.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82cbc1700 1 --2- 172.21.15.148:0/1006943450 >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82cbc1700 1 -- 172.21.15.148:0/1006943450 >> v1:172.21.15.161:6789/0 conn(0x7fc82815a620 legacy=0x7fc82815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82cbc1700 1 -- 172.21.15.148:0/1006943450 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82815c560 con 0x7fc828157c40 2023-10-10T20:29:59.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc827fff700 1 --2- 172.21.15.148:0/1006943450 >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:29:59.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82cbc1700 1 --2- 172.21.15.148:0/1006943450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc820009f20 tx=0x7fc820009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2967a48981289a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc826ffd700 1 -- 172.21.15.148:0/1006943450 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82000e070 con 0x7fc828157c40 2023-10-10T20:29:59.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc826ffd700 1 -- 172.21.15.148:0/1006943450 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82000b9d0 con 0x7fc828157c40 2023-10-10T20:29:59.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc826ffd700 1 -- 172.21.15.148:0/1006943450 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200092e0 con 0x7fc828157c40 2023-10-10T20:29:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 -- 172.21.15.148:0/1006943450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 msgr2=0x7fc828158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 --2- 172.21.15.148:0/1006943450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc820009f20 tx=0x7fc820009840 comp rx=0 tx=0).stop 2023-10-10T20:29:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 -- 172.21.15.148:0/1006943450 shutdown_connections 2023-10-10T20:29:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 --2- 172.21.15.148:0/1006943450 >> v2:172.21.15.161:3300/0 conn(0x7fc828157870 0x7fc82815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 --2- 172.21.15.148:0/1006943450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc828158110 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 -- 172.21.15.148:0/1006943450 >> 172.21.15.148:0/1006943450 conn(0x7fc828068ce0 msgr2=0x7fc8280690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:59.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.381+0000 7fc82e624700 1 -- 172.21.15.148:0/1006943450 shutdown_connections 2023-10-10T20:29:59.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.382+0000 7fc82e624700 1 -- 172.21.15.148:0/1006943450 wait complete. 2023-10-10T20:29:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 Processor -- start 2023-10-10T20:29:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 -- start start 2023-10-10T20:29:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc82814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc82815a620 0x7fc82814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b2fd0 con 0x7fc828157870 2023-10-10T20:29:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b2cf0 con 0x7fc82815a620 2023-10-10T20:29:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.383+0000 7fc82e624700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc828151a80 con 0x7fc828157c40 2023-10-10T20:29:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35418/0 (socket says 172.21.15.148:35418) 2023-10-10T20:29:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 -- 172.21.15.148:0/3511884159 learned_addr learned my addr 172.21.15.148:0/3511884159 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc8277fe700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc82814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc82cbc1700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc82815a620 0x7fc82814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 -- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 msgr2=0x7fc82814d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc82814d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 -- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc82815a620 msgr2=0x7fc82814dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc82815a620 0x7fc82814dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 -- 172.21.15.148:0/3511884159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009420 con 0x7fc828157870 2023-10-10T20:29:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc827fff700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc81800ec80 tx=0x7fc8180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800c850 con 0x7fc828157870 2023-10-10T20:29:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800c9b0 con 0x7fc828157870 2023-10-10T20:29:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc820009ae0 con 0x7fc828157870 2023-10-10T20:29:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800cd30 con 0x7fc828157870 2023-10-10T20:29:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.384+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828152040 con 0x7fc828157870 2023-10-10T20:29:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.385+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82804c3c0 con 0x7fc828157870 2023-10-10T20:29:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.386+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc818013680 con 0x7fc828157870 2023-10-10T20:29:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.386+0000 7fc80ffff700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 0x7fc8140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.386+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc818084670 con 0x7fc828157870 2023-10-10T20:29:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.386+0000 7fc8277fe700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 0x7fc8140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.386+0000 7fc8277fe700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 0x7fc8140693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc8280d1170 tx=0x7fc81c007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.390+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc818055670 con 0x7fc828157870 2023-10-10T20:29:59.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.612+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc82815c840 con 0x7fc828157870 2023-10-10T20:29:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.612+0000 7fc80ffff700 1 -- 172.21.15.148:0/3511884159 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fc818017df0 con 0x7fc828157870 2023-10-10T20:29:59.615 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:29:59.616 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7186,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/887331809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":887331809},{"type":"v1","addr":"172.21.15.161:6834","nonce":887331809}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2733032759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2733032759},{"type":"v1","addr":"172.21.15.161:6835","nonce":2733032759}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1448562888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1448562888},{"type":"v1","addr":"172.21.15.148:6836","nonce":1448562888}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7800,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4175743506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4175743506},{"type":"v1","addr":"172.21.15.148:6837","nonce":4175743506}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2023-10-10T20:29:59.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 msgr2=0x7fc8140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 0x7fc8140693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc8280d1170 tx=0x7fc81c007eb0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 msgr2=0x7fc82814c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc81800ec80 tx=0x7fc8180091a0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 shutdown_connections 2023-10-10T20:29:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc814066f30 0x7fc8140693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc82815a620 0x7fc82814dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc828157870 0x7fc82814c580 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 --2- 172.21.15.148:0/3511884159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc828157c40 0x7fc82814d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 >> 172.21.15.148:0/3511884159 conn(0x7fc828068ce0 msgr2=0x7fc8280690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 shutdown_connections 2023-10-10T20:29:59.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.615+0000 7fc82e624700 1 -- 172.21.15.148:0/3511884159 wait complete. 2023-10-10T20:29:59.620 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 80 2023-10-10T20:29:59.629 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-10T20:29:59.630 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-10T20:29:59.630 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:29:59.630 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:29:59.631 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-10T20:29:59.631 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-10T20:29:59.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:29:59.631 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:29:59.632 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-10T20:29:59.632 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-10T20:29:59.632 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:29:59.632 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:29:59.633 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-10T20:29:59.633 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-10T20:29:59.633 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:29:59.633 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:29:59.638 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:29:59.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-10T20:29:59.640 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:29:59.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-10T20:29:59.646 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:29:59.647 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-10T20:29:59.648 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:29:59.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-10T20:29:59.680 INFO:tasks.ceph.mds.a:Started 2023-10-10T20:29:59.681 DEBUG:teuthology.parallel:result is None 2023-10-10T20:29:59.682 INFO:tasks.ceph.mds.c:Started 2023-10-10T20:29:59.682 DEBUG:teuthology.parallel:result is None 2023-10-10T20:29:59.688 INFO:tasks.ceph.mds.b:Started 2023-10-10T20:29:59.689 DEBUG:teuthology.parallel:result is None 2023-10-10T20:29:59.689 INFO:tasks.ceph.mds.d:Started 2023-10-10T20:29:59.690 DEBUG:teuthology.parallel:result is None 2023-10-10T20:29:59.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-10T20:29:59.753 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2023-10-10T20:29:59.754 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2023-10-10T20:29:59.769 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:29:59.770 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2023-10-10T20:29:59.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.808+0000 7fca381cd700 1 Processor -- start 2023-10-10T20:29:59.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 -- start start 2023-10-10T20:29:59.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 0x7fca3006c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca30158e50 0x7fca300cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3006c010 con 0x7fca30152010 2023-10-10T20:29:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3006bd10 con 0x7fca30158e50 2023-10-10T20:29:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.809+0000 7fca381cd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015c060 con 0x7fca301523e0 2023-10-10T20:29:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca30158e50 0x7fca300cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca30158e50 0x7fca300cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35444/0 (socket says 172.21.15.148:35444) 2023-10-10T20:29:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 -- 172.21.15.148:0/642200495 learned_addr learned my addr 172.21.15.148:0/642200495 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca35768700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 0x7fca3006c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 -- 172.21.15.148:0/642200495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 msgr2=0x7fca3006c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 0x7fca3006c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 -- 172.21.15.148:0/642200495 >> v1:172.21.15.161:6789/0 conn(0x7fca30152010 legacy=0x7fca3015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:29:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 -- 172.21.15.148:0/642200495 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3015c340 con 0x7fca30158e50 2023-10-10T20:29:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca35768700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 0x7fca3006c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:29:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca3676a700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30158e50 0x7fca300cdf30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fca28009f20 tx=0x7fca28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665df15da9ac4ec6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca34f67700 1 -- 172.21.15.148:0/642200495 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2800e070 con 0x7fca30158e50 2023-10-10T20:29:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca34f67700 1 -- 172.21.15.148:0/642200495 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2800ba40 con 0x7fca30158e50 2023-10-10T20:29:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.810+0000 7fca34f67700 1 -- 172.21.15.148:0/642200495 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280092e0 con 0x7fca30158e50 2023-10-10T20:29:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 -- 172.21.15.148:0/642200495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30158e50 msgr2=0x7fca300cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30158e50 0x7fca300cdf30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fca28009f20 tx=0x7fca28009840 comp rx=0 tx=0).stop 2023-10-10T20:29:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 -- 172.21.15.148:0/642200495 shutdown_connections 2023-10-10T20:29:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca30158e50 0x7fca300cdf30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 --2- 172.21.15.148:0/642200495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca301523e0 0x7fca3006c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 -- 172.21.15.148:0/642200495 >> 172.21.15.148:0/642200495 conn(0x7fca30069870 msgr2=0x7fca30069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:29:59.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 -- 172.21.15.148:0/642200495 shutdown_connections 2023-10-10T20:29:59.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.811+0000 7fca381cd700 1 -- 172.21.15.148:0/642200495 wait complete. 2023-10-10T20:29:59.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.812+0000 7fca381cd700 1 Processor -- start 2023-10-10T20:29:59.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.812+0000 7fca381cd700 1 -- start start 2023-10-10T20:29:59.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca30152010 0x7fca3014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca30158e50 0x7fca30156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300c5730 con 0x7fca301523e0 2023-10-10T20:29:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3016ab70 con 0x7fca30158e50 2023-10-10T20:29:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3014c2a0 con 0x7fca30152010 2023-10-10T20:29:59.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35460/0 (socket says 172.21.15.148:35460) 2023-10-10T20:29:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 -- 172.21.15.148:0/3064618469 learned_addr learned my addr 172.21.15.148:0/3064618469 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:29:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35f69700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca30152010 0x7fca3014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca3676a700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca30158e50 0x7fca30156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 -- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca30152010 msgr2=0x7fca3014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca30152010 0x7fca3014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 -- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca30158e50 msgr2=0x7fca30156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:29:59.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca30158e50 0x7fca30156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:29:59.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 -- 172.21.15.148:0/3064618469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28009ae0 con 0x7fca301523e0 2023-10-10T20:29:59.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca35768700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c8b0 tx=0x7fca20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20011600 con 0x7fca301523e0 2023-10-10T20:29:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20011760 con 0x7fca301523e0 2023-10-10T20:29:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca30169f60 con 0x7fca301523e0 2023-10-10T20:29:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.813+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20011b00 con 0x7fca301523e0 2023-10-10T20:29:59.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.814+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3016a520 con 0x7fca301523e0 2023-10-10T20:29:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.815+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3004c3c0 con 0x7fca301523e0 2023-10-10T20:29:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.815+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca200193b0 con 0x7fca301523e0 2023-10-10T20:29:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.815+0000 7fca25ffb700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:29:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.815+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fca200841b0 con 0x7fca301523e0 2023-10-10T20:29:59.827 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.815+0000 7fca35f69700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:29:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.816+0000 7fca35f69700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 0x7fca140693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca30069730 tx=0x7fca2c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:29:59.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:29:59.820+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca20051720 con 0x7fca301523e0 2023-10-10T20:30:00.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.018+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fca3006d1b0 con 0x7fca301523e0 2023-10-10T20:30:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.864+0000 7fca25ffb700 1 -- 172.21.15.148:0/3064618469 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v174) v1 ==== 97+0+0 (secure 0 0 0) 0x7fca20054d40 con 0x7fca301523e0 2023-10-10T20:30:00.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.866+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 msgr2=0x7fca140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:00.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.866+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 0x7fca140693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca30069730 tx=0x7fca2c009060 comp rx=0 tx=0).stop 2023-10-10T20:30:00.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.866+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 msgr2=0x7fca300b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:00.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.866+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c8b0 tx=0x7fca20008040 comp rx=0 tx=0).stop 2023-10-10T20:30:00.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 shutdown_connections 2023-10-10T20:30:00.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fca14066f30 0x7fca140693e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:00.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca30158e50 0x7fca30156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca30152010 0x7fca3014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 --2- 172.21.15.148:0/3064618469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca301523e0 0x7fca300b40d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 >> 172.21.15.148:0/3064618469 conn(0x7fca30069870 msgr2=0x7fca3014cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 shutdown_connections 2023-10-10T20:30:00.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.867+0000 7fca381cd700 1 -- 172.21.15.148:0/3064618469 wait complete. 2023-10-10T20:30:00.873 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2023-10-10T20:30:00.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-10T20:30:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.997+0000 7f18fac83700 1 Processor -- start 2023-10-10T20:30:01.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.997+0000 7f18fac83700 1 -- start start 2023-10-10T20:30:01.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.998+0000 7f18fac83700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18f40c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.004 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.998+0000 7f18fac83700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f18f40cb4f0 0x7f18f40cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.998+0000 7f18fac83700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f415de60 con 0x7f18f40b0e50 2023-10-10T20:30:01.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.998+0000 7f18fac83700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f40cbed0 con 0x7f18f40cb4f0 2023-10-10T20:30:01.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.998+0000 7f18fac83700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415e140 con 0x7f18f40b1220 2023-10-10T20:30:01.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18f8a1f700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f18f40b0e50 0x7f18f406d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38416/0 (socket says 172.21.15.148:38416) 2023-10-10T20:30:01.005 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18f8a1f700 1 -- 172.21.15.148:0/3071409290 learned_addr learned my addr 172.21.15.148:0/3071409290 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:01.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18f9220700 1 --2- 172.21.15.148:0/3071409290 >> v2:172.21.15.161:3300/0 conn(0x7f18f40cb4f0 0x7f18f40cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18ebfff700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18f40c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646846866 0 0) 0x7f18f415de60 con 0x7f18f40b0e50 2023-10-10T20:30:01.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18dc003570 con 0x7f18f40b0e50 2023-10-10T20:30:01.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360435634 0 0) 0x7f18dc003570 con 0x7f18f40b0e50 2023-10-10T20:30:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 msgr2=0x7f18f40c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18f40c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 >> v2:172.21.15.161:3300/0 conn(0x7f18f40cb4f0 msgr2=0x7f18f40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 --2- 172.21.15.148:0/3071409290 >> v2:172.21.15.161:3300/0 conn(0x7f18f40cb4f0 0x7f18f40cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f415e420 con 0x7f18f40b0e50 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f18f0001470 con 0x7f18f40b0e50 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 >> v1:172.21.15.148:6789/0 conn(0x7f18f40b0e50 legacy=0x7f18f406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18dc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18dc003fe0 0x7f18dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc004d50 0x7f18dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40cbed0 con 0x7f18dc004d50 2023-10-10T20:30:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f415e140 con 0x7f18dc003fe0 2023-10-10T20:30:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc00bb40 con 0x7f18f40b1220 2023-10-10T20:30:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:00.999+0000 7f18f9220700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc004d50 0x7f18dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18ebfff700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18dc003fe0 0x7f18dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f8a1f700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18dc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 -- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 msgr2=0x7f18dc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1220 0x7f18dc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 -- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18dc003fe0 msgr2=0x7f18dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18dc003fe0 0x7f18dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 -- 172.21.15.148:0/3071409290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc003570 con 0x7f18dc004d50 2023-10-10T20:30:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc004d50 0x7f18dc0051c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0094d0 tx=0x7f18ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb0f1db56e99b08 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00ea90 con 0x7f18dc004d50 2023-10-10T20:30:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00f040 con 0x7f18dc004d50 2023-10-10T20:30:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18eb7fe700 1 -- 172.21.15.148:0/3071409290 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00ee20 con 0x7f18dc004d50 2023-10-10T20:30:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18fac83700 1 -- 172.21.15.148:0/3071409290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc004d50 msgr2=0x7f18dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18fac83700 1 --2- 172.21.15.148:0/3071409290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc004d50 0x7f18dc0051c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0094d0 tx=0x7f18ec00b040 comp rx=0 tx=0).stop 2023-10-10T20:30:01.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.000+0000 7f18f9220700 1 -- 172.21.15.148:0/3071409290 reap_dead start 2023-10-10T20:30:01.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.001+0000 7f18fac83700 1 -- 172.21.15.148:0/3071409290 shutdown_connections 2023-10-10T20:30:01.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.001+0000 7f18fac83700 1 -- 172.21.15.148:0/3071409290 >> 172.21.15.148:0/3071409290 conn(0x7f18f4069870 msgr2=0x7f18f4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.001+0000 7f18fac83700 1 -- 172.21.15.148:0/3071409290 shutdown_connections 2023-10-10T20:30:01.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.001+0000 7f18fac83700 1 -- 172.21.15.148:0/3071409290 wait complete. 2023-10-10T20:30:01.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.002+0000 7f18fac83700 1 Processor -- start 2023-10-10T20:30:01.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.002+0000 7f18fac83700 1 -- start start 2023-10-10T20:30:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.002+0000 7f18fac83700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18f4153790 0x7f18f40b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.002+0000 7f18fac83700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1e20 0x7f18f4153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.002+0000 7f18fac83700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18fac83700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f416a740 con 0x7f18f41543c0 2023-10-10T20:30:01.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18fac83700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40b2290 con 0x7f18f4153790 2023-10-10T20:30:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18fac83700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4155710 con 0x7f18f40b1e20 2023-10-10T20:30:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35476/0 (socket says 172.21.15.148:35476) 2023-10-10T20:30:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 -- 172.21.15.148:0/4024450913 learned_addr learned my addr 172.21.15.148:0/4024450913 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:01.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f8a1f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18f4153790 0x7f18f40b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18ebfff700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1e20 0x7f18f4153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 -- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1e20 msgr2=0x7f18f4153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1e20 0x7f18f4153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 -- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18f4153790 msgr2=0x7f18f40b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18f4153790 0x7f18f40b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 -- 172.21.15.148:0/4024450913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f4154860 con 0x7f18f41543c0 2023-10-10T20:30:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18f9220700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e7d0 tx=0x7f18ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec014e40 con 0x7f18f41543c0 2023-10-10T20:30:01.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec01d630 con 0x7f18f41543c0 2023-10-10T20:30:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f4154b00 con 0x7f18f41543c0 2023-10-10T20:30:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec01d9b0 con 0x7f18f41543c0 2023-10-10T20:30:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.003+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f41550c0 con 0x7f18f41543c0 2023-10-10T20:30:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.004+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f404c3c0 con 0x7f18f41543c0 2023-10-10T20:30:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.004+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18ec003040 con 0x7f18f41543c0 2023-10-10T20:30:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.005+0000 7f18e8ff9700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 0x7f18e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.005+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f18ec085150 con 0x7f18f41543c0 2023-10-10T20:30:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.005+0000 7f18f8a1f700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 0x7f18e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.005+0000 7f18f8a1f700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 0x7f18e00693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f18f00010e0 tx=0x7f18f0001e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.010+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18ec052790 con 0x7f18f41543c0 2023-10-10T20:30:01.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.263+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f18f4068ba0 con 0x7f18f41543c0 2023-10-10T20:30:01.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.263+0000 7f18e8ff9700 1 -- 172.21.15.148:0/4024450913 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f18ec055db0 con 0x7f18f41543c0 2023-10-10T20:30:01.266 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:01.266 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:01.267 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2023-10-10T20:30:01.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.267 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2023-10-10T20:30:01.267 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlEUOvMBAA6yjKRdjwDsPAtHIhhXH3Ug==", 2023-10-10T20:30:01.268 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.268 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.268 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:30:01.268 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:30:01.268 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.269 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.269 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2023-10-10T20:30:01.269 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlQJNZABAAIe+SZuix64lZ4kQCxPqxWA==", 2023-10-10T20:30:01.270 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.270 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.270 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:30:01.270 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:30:01.270 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.271 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2023-10-10T20:30:01.272 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlpKiANxAAlB5RGULH/EM9h60UT/jGow==", 2023-10-10T20:30:01.272 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.272 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.272 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:30:01.272 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:30:01.273 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.273 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.274 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2023-10-10T20:30:01.274 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlvSctBxAAO1gZDfZdZIcNq0kNrRZHhQ==", 2023-10-10T20:30:01.274 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.274 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.274 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:30:01.275 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:30:01.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.275 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.276 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2023-10-10T20:30:01.276 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlUdU5HxAAy6O74M4yxo6zOyq0b+ac7Q==", 2023-10-10T20:30:01.276 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2023-10-10T20:30:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVluvoOJhAAGpigkXL5Awh3LrPWxuBBAw==", 2023-10-10T20:30:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.279 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.280 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2023-10-10T20:30:01.280 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVlbgcKLxAAjUsyLmlIVocI+3wNcMmWhg==", 2023-10-10T20:30:01.280 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.280 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.281 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.282 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2023-10-10T20:30:01.282 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCysSVl7mooOBAAaKWGmOS+7hI/TnacYc4csA==", 2023-10-10T20:30:01.282 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.283 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlecC9DxAAczFgxueLWTpZEu4Tb2VjyA==", 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.284 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.285 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2023-10-10T20:30:01.286 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlkGTdFhAASr1GO/Shvg6f3c47B3SCCg==", 2023-10-10T20:30:01.286 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.286 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.287 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2023-10-10T20:30:01.287 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlxI3tHxAAL58DDIgcDB52Kzou8UjPvg==", 2023-10-10T20:30:01.288 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.288 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.289 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2023-10-10T20:30:01.289 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQC2sSVlvc8fKRAAwf5p9kLkukpKGBZvRMHtDA==", 2023-10-10T20:30:01.290 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.290 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2023-10-10T20:30:01.290 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2023-10-10T20:30:01.290 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2023-10-10T20:30:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlRRoaCxAANQit6V8DIM88Ewzfomt9eA==", 2023-10-10T20:30:01.291 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2023-10-10T20:30:01.292 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-10T20:30:01.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.293 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2023-10-10T20:30:01.293 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVlLnB2ExAAt0Em0DHQPRuoYss3IeBbkQ==", 2023-10-10T20:30:01.293 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.293 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.294 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:30:01.294 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:30:01.294 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:30:01.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.295 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2023-10-10T20:30:01.295 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl00R1EBAAjbiL4k+m7FmqopXM0R4CLQ==", 2023-10-10T20:30:01.295 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.295 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.296 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:30:01.296 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:30:01.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:30:01.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.297 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2023-10-10T20:30:01.297 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCvsSVl0XXQGBAA4+ifRWooPwXzmj5KF50onA==", 2023-10-10T20:30:01.297 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.297 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2023-10-10T20:30:01.298 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2023-10-10T20:30:01.298 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2023-10-10T20:30:01.298 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2023-10-10T20:30:01.298 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.299 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2023-10-10T20:30:01.299 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVl1cqOFRAArwcO8oge4HrAUY9VBE8PJw==", 2023-10-10T20:30:01.299 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.299 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.301 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2023-10-10T20:30:01.301 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlWOSlLRAAaX/h++hm8Fi65qdffY99Xg==", 2023-10-10T20:30:01.301 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.301 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:30:01.301 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:30:01.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.302 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:01.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:01.302 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2023-10-10T20:30:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQCusSVlmzXkKRAA3N/ucCBVjTAGi9L45vPPtA==", 2023-10-10T20:30:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2023-10-10T20:30:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2023-10-10T20:30:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2023-10-10T20:30:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2023-10-10T20:30:01.304 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.304 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:01.304 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:01.304 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:01.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.265+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 msgr2=0x7f18e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.265+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 0x7f18e00693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f18f00010e0 tx=0x7f18f0001e90 comp rx=0 tx=0).stop 2023-10-10T20:30:01.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.265+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 msgr2=0x7f18f40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.265+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e7d0 tx=0x7f18ec01a000 comp rx=0 tx=0).stop 2023-10-10T20:30:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.265+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 shutdown_connections 2023-10-10T20:30:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f18e0066f30 0x7f18e00693e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.307 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18f41543c0 0x7f18f40b3940 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18f4153790 0x7f18f40b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 --2- 172.21.15.148:0/4024450913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18f40b1e20 0x7f18f4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 >> 172.21.15.148:0/4024450913 conn(0x7f18f4069870 msgr2=0x7f18f40c4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 shutdown_connections 2023-10-10T20:30:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.266+0000 7f18fac83700 1 -- 172.21.15.148:0/4024450913 wait complete. 2023-10-10T20:30:01.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:01.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.423+0000 7fc6d3b63700 1 Processor -- start 2023-10-10T20:30:01.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.423+0000 7fc6d3b63700 1 -- start start 2023-10-10T20:30:01.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.424+0000 7fc6d3b63700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6cc157d70 0x7fc6cc0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.424+0000 7fc6d3b63700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.424+0000 7fc6d3b63700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0b0cb0 con 0x7fc6cc158140 2023-10-10T20:30:01.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.424+0000 7fc6d3b63700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ca070 con 0x7fc6cc157d70 2023-10-10T20:30:01.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.424+0000 7fc6d3b63700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc067800 con 0x7fc6cc0b0880 2023-10-10T20:30:01.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6cc157d70 0x7fc6cc0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6cc157d70 0x7fc6cc0c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.148:57178/0 (socket says 172.21.15.148:57178) 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 -- 172.21.15.148:0/1418056445 learned_addr learned my addr 172.21.15.148:0/1418056445 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d2100700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 -- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 msgr2=0x7fc6cc070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 -- 172.21.15.148:0/1418056445 >> v1:172.21.15.161:6789/0 conn(0x7fc6cc158140 legacy=0x7fc6cc06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 -- 172.21.15.148:0/1418056445 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc067ae0 con 0x7fc6cc157d70 2023-10-10T20:30:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d2100700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d18ff700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc0c9a70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80099b0 tx=0x7fc6c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c329fd2de6b4d054 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d08fd700 1 -- 172.21.15.148:0/1418056445 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8009ea0 con 0x7fc6cc157d70 2023-10-10T20:30:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d08fd700 1 -- 172.21.15.148:0/1418056445 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c800b860 con 0x7fc6cc157d70 2023-10-10T20:30:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.425+0000 7fc6d08fd700 1 -- 172.21.15.148:0/1418056445 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800bc00 con 0x7fc6cc157d70 2023-10-10T20:30:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 msgr2=0x7fc6cc0c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc0c9a70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80099b0 tx=0x7fc6c8009250 comp rx=0 tx=0).stop 2023-10-10T20:30:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1418056445 shutdown_connections 2023-10-10T20:30:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc0c9a70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1418056445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1418056445 >> 172.21.15.148:0/1418056445 conn(0x7fc6cc0ca4f0 msgr2=0x7fc6cc0ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1418056445 shutdown_connections 2023-10-10T20:30:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.426+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1418056445 wait complete. 2023-10-10T20:30:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.427+0000 7fc6d3b63700 1 Processor -- start 2023-10-10T20:30:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.427+0000 7fc6d3b63700 1 -- start start 2023-10-10T20:30:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc157160 con 0x7fc6cc158140 2023-10-10T20:30:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c2cc0 con 0x7fc6cc157d70 2023-10-10T20:30:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d3b63700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc156ba0 con 0x7fc6cc0b0880 2023-10-10T20:30:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35482/0 (socket says 172.21.15.148:35482) 2023-10-10T20:30:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 -- 172.21.15.148:0/1510346457 learned_addr learned my addr 172.21.15.148:0/1510346457 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d18ff700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d10fe700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 -- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 msgr2=0x7fc6cc149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 -- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 msgr2=0x7fc6cc156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 -- 172.21.15.148:0/1510346457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4009a20 con 0x7fc6cc158140 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6d2100700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fc0 tx=0x7fc6c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.428+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400cd30 con 0x7fc6cc158140 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.429+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c400f040 con 0x7fc6cc158140 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.429+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c40146d0 con 0x7fc6cc158140 2023-10-10T20:30:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.429+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c80094f0 con 0x7fc6cc158140 2023-10-10T20:30:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.429+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc0c2950 con 0x7fc6cc158140 2023-10-10T20:30:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.430+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc0c2e30 con 0x7fc6cc158140 2023-10-10T20:30:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.430+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6c4003020 con 0x7fc6cc158140 2023-10-10T20:30:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.430+0000 7fc6c1ffb700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.430+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc6c4084c70 con 0x7fc6cc158140 2023-10-10T20:30:01.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.430+0000 7fc6d18ff700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.431+0000 7fc6d18ff700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 0x7fc6b00693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80099b0 tx=0x7fc6c8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.435+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6c4052230 con 0x7fc6cc158140 2023-10-10T20:30:01.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.621+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6cc04c3c0 con 0x7fc6cc158140 2023-10-10T20:30:01.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.622+0000 7fc6c1ffb700 1 -- 172.21.15.148:0/1510346457 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+12193 (secure 0 0 0) 0x7fc6c4055850 con 0x7fc6cc158140 2023-10-10T20:30:01.624 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:01.624 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":174,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:00.862923+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:01.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 msgr2=0x7fc6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 0x7fc6b00693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80099b0 tx=0x7fc6c8017000 comp rx=0 tx=0).stop 2023-10-10T20:30:01.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 msgr2=0x7fc6cc0c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fc0 tx=0x7fc6c400b040 comp rx=0 tx=0).stop 2023-10-10T20:30:01.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 shutdown_connections 2023-10-10T20:30:01.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6b0066f30 0x7fc6b00693e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6cc158140 0x7fc6cc0c59c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6cc0b0880 0x7fc6cc149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 --2- 172.21.15.148:0/1510346457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6cc157d70 0x7fc6cc156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 >> 172.21.15.148:0/1510346457 conn(0x7fc6cc0ca4f0 msgr2=0x7fc6cc157470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 shutdown_connections 2023-10-10T20:30:01.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.624+0000 7fc6d3b63700 1 -- 172.21.15.148:0/1510346457 wait complete. 2023-10-10T20:30:01.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-10T20:30:01.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:30:01.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.755+0000 7f16c6b6e700 1 Processor -- start 2023-10-10T20:30:01.758 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.755+0000 7f16c6b6e700 1 -- start start 2023-10-10T20:30:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16c6b6e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 0x7f16c0164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16c6b6e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00b0790 con 0x7f16c015a290 2023-10-10T20:30:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16c6b6e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00ce1f0 con 0x7f16c0159ec0 2023-10-10T20:30:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16c6b6e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b0b20 con 0x7f16c00d05a0 2023-10-10T20:30:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16b7fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f16c015a290 0x7f16c006a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38418/0 (socket says 172.21.15.148:38418) 2023-10-10T20:30:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16b7fff700 1 -- 172.21.15.148:0/1737415749 learned_addr learned my addr 172.21.15.148:0/1737415749 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16c510b700 1 --2- 172.21.15.148:0/1737415749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 0x7f16c0164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386960891 0 0) 0x7f16c00b0790 con 0x7f16c015a290 2023-10-10T20:30:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.756+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a8003570 con 0x7f16c015a290 2023-10-10T20:30:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528890835 0 0) 0x7f16a8003570 con 0x7f16c015a290 2023-10-10T20:30:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c510b700 1 -- 172.21.15.148:0/1737415749 >> v1:172.21.15.161:6789/0 conn(0x7f16c0159ec0 legacy=0x7f16c00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c510b700 1 -- 172.21.15.148:0/1737415749 >> v1:172.21.15.148:6789/0 conn(0x7f16c015a290 legacy=0x7f16c006a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c510b700 1 -- 172.21.15.148:0/1737415749 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00b04b0 con 0x7f16c00d05a0 2023-10-10T20:30:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c510b700 1 --2- 172.21.15.148:0/1737415749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 0x7f16c0164ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009910 tx=0x7f16b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30696a396fd4d614 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0001470 con 0x7f16c00d05a0 2023-10-10T20:30:01.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8011420 con 0x7f16c00d05a0 2023-10-10T20:30:01.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16b77fe700 1 -- 172.21.15.148:0/1737415749 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80117c0 con 0x7f16c00d05a0 2023-10-10T20:30:01.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c6b6e700 1 -- 172.21.15.148:0/1737415749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 msgr2=0x7f16c0164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.757+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/1737415749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 0x7f16c0164ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009910 tx=0x7f16b80091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.758+0000 7f16c6b6e700 1 -- 172.21.15.148:0/1737415749 shutdown_connections 2023-10-10T20:30:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.758+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/1737415749 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c00d05a0 0x7f16c0164ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.758+0000 7f16c6b6e700 1 -- 172.21.15.148:0/1737415749 >> 172.21.15.148:0/1737415749 conn(0x7f16c00d1880 msgr2=0x7f16c00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.758+0000 7f16c6b6e700 1 -- 172.21.15.148:0/1737415749 shutdown_connections 2023-10-10T20:30:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.758+0000 7f16c6b6e700 1 -- 172.21.15.148:0/1737415749 wait complete. 2023-10-10T20:30:01.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.759+0000 7f16c6b6e700 1 Processor -- start 2023-10-10T20:30:01.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.759+0000 7f16c6b6e700 1 -- start start 2023-10-10T20:30:01.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.759+0000 7f16c6b6e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.769 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.759+0000 7f16c6b6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16c0159ec0 0x7f16c0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.759+0000 7f16c6b6e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 0x7f16c00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c6b6e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c016da70 con 0x7f16c00d05a0 2023-10-10T20:30:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c6b6e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b5ba0 con 0x7f16c0159ec0 2023-10-10T20:30:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c6b6e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01551f0 con 0x7f16c015a290 2023-10-10T20:30:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35484/0 (socket says 172.21.15.148:35484) 2023-10-10T20:30:01.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 -- 172.21.15.148:0/545067343 learned_addr learned my addr 172.21.15.148:0/545067343 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:01.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16b7fff700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16c0159ec0 0x7f16c0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c510b700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 0x7f16c00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 -- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 msgr2=0x7f16c00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 0x7f16c00c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 -- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16c0159ec0 msgr2=0x7f16c0154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16c0159ec0 0x7f16c0154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 -- 172.21.15.148:0/545067343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009420 con 0x7f16c00d05a0 2023-10-10T20:30:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c510b700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 0x7f16c00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c490a700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00cb90 tx=0x7f16bc001840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00a870 con 0x7f16c00d05a0 2023-10-10T20:30:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00a9d0 con 0x7f16c00d05a0 2023-10-10T20:30:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00ad70 con 0x7f16c00d05a0 2023-10-10T20:30:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00b5e10 con 0x7f16c00d05a0 2023-10-10T20:30:01.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.760+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00b6970 con 0x7f16c00d05a0 2023-10-10T20:30:01.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.761+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c004c3c0 con 0x7f16c00d05a0 2023-10-10T20:30:01.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.761+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16bc012630 con 0x7f16c00d05a0 2023-10-10T20:30:01.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.762+0000 7f16b4ff9700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:01.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.762+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f16bc084530 con 0x7f16c00d05a0 2023-10-10T20:30:01.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.762+0000 7f16b7fff700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:01.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.762+0000 7f16b7fff700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 0x7f16ac0693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f16b00015a0 tx=0x7f16b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:01.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.766+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16bc051af0 con 0x7f16c00d05a0 2023-10-10T20:30:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.948+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16c00ceab0 con 0x7f16c00d05a0 2023-10-10T20:30:01.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.949+0000 7f16b4ff9700 1 -- 172.21.15.148:0/545067343 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16bc055110 con 0x7f16c00d05a0 2023-10-10T20:30:01.952 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:30:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 msgr2=0x7f16ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 0x7f16ac0693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f16b00015a0 tx=0x7f16b0009040 comp rx=0 tx=0).stop 2023-10-10T20:30:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 msgr2=0x7f16c00afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00cb90 tx=0x7f16bc001840 comp rx=0 tx=0).stop 2023-10-10T20:30:01.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 shutdown_connections 2023-10-10T20:30:01.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16c015a290 0x7f16c00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16c00d05a0 0x7f16c00afe80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 --2- 172.21.15.148:0/545067343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16c0159ec0 0x7f16c0154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:01.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 >> 172.21.15.148:0/545067343 conn(0x7f16c00d1880 msgr2=0x7f16c00c7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:01.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 shutdown_connections 2023-10-10T20:30:01.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:01.951+0000 7f16c6b6e700 1 -- 172.21.15.148:0/545067343 wait complete. 2023-10-10T20:30:01.966 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-10T20:30:02.080 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.078+0000 7f6e22f91700 1 Processor -- start 2023-10-10T20:30:02.081 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.078+0000 7f6e22f91700 1 -- start start 2023-10-10T20:30:02.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e1c15a620 0x7f6e1c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6e1c15d580 0x7f6e1c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c15cbb0 con 0x7f6e1c15a620 2023-10-10T20:30:02.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c15bfa0 con 0x7f6e1c15d580 2023-10-10T20:30:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e22f91700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c15c280 con 0x7f6e1c157870 2023-10-10T20:30:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e1c15a620 0x7f6e1c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e1c15a620 0x7f6e1c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35492/0 (socket says 172.21.15.148:35492) 2023-10-10T20:30:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 -- 172.21.15.148:0/3503283140 learned_addr learned my addr 172.21.15.148:0/3503283140 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 -- 172.21.15.148:0/3503283140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 msgr2=0x7f6e1c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:02.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e20d2d700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e2152e700 1 --2- 172.21.15.148:0/3503283140 >> v2:172.21.15.161:3300/0 conn(0x7f6e1c15d580 0x7f6e1c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 -- 172.21.15.148:0/3503283140 >> v2:172.21.15.161:3300/0 conn(0x7f6e1c15d580 msgr2=0x7f6e1c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:02.086 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 --2- 172.21.15.148:0/3503283140 >> v2:172.21.15.161:3300/0 conn(0x7f6e1c15d580 0x7f6e1c158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 -- 172.21.15.148:0/3503283140 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c15c560 con 0x7f6e1c15a620 2023-10-10T20:30:02.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.079+0000 7f6e1bfff700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15a620 0x7f6e1c15d040 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0b03b0 tx=0x7f6e0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657952d66aa1f78 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e1b7fe700 1 -- 172.21.15.148:0/3503283140 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00b860 con 0x7f6e1c15a620 2023-10-10T20:30:02.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e1b7fe700 1 -- 172.21.15.148:0/3503283140 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00b9c0 con 0x7f6e1c15a620 2023-10-10T20:30:02.087 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e1b7fe700 1 -- 172.21.15.148:0/3503283140 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00bd60 con 0x7f6e1c15a620 2023-10-10T20:30:02.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 -- 172.21.15.148:0/3503283140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15a620 msgr2=0x7f6e1c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15a620 0x7f6e1c15d040 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0b03b0 tx=0x7f6e0c009250 comp rx=0 tx=0).stop 2023-10-10T20:30:02.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 -- 172.21.15.148:0/3503283140 shutdown_connections 2023-10-10T20:30:02.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 --2- 172.21.15.148:0/3503283140 >> v2:172.21.15.161:3300/0 conn(0x7f6e1c15d580 0x7f6e1c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.088 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c15b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 --2- 172.21.15.148:0/3503283140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15a620 0x7f6e1c15d040 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 -- 172.21.15.148:0/3503283140 >> 172.21.15.148:0/3503283140 conn(0x7f6e1c068ce0 msgr2=0x7f6e1c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:02.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 -- 172.21.15.148:0/3503283140 shutdown_connections 2023-10-10T20:30:02.089 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.080+0000 7f6e22f91700 1 -- 172.21.15.148:0/3503283140 wait complete. 2023-10-10T20:30:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.081+0000 7f6e22f91700 1 Processor -- start 2023-10-10T20:30:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.081+0000 7f6e22f91700 1 -- start start 2023-10-10T20:30:02.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e1c15a620 0x7f6e1c16b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16b840 con 0x7f6e1c15d580 2023-10-10T20:30:02.093 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16e240 con 0x7f6e1c15a620 2023-10-10T20:30:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e22f91700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16e520 con 0x7f6e1c157870 2023-10-10T20:30:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e2152e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e2152e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35508/0 (socket says 172.21.15.148:35508) 2023-10-10T20:30:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.082+0000 7f6e2152e700 1 -- 172.21.15.148:0/1493113425 learned_addr learned my addr 172.21.15.148:0/1493113425 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:02.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e20d2d700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 -- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 msgr2=0x7f6e1c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e1bfff700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e1c15a620 0x7f6e1c16b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 -- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e1c15a620 msgr2=0x7f6e1c16b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e1c15a620 0x7f6e1c16b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 -- 172.21.15.148:0/1493113425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0094f0 con 0x7f6e1c15d580 2023-10-10T20:30:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e2152e700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c880 tx=0x7f6e04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04011600 con 0x7f6e1c15d580 2023-10-10T20:30:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04011780 con 0x7f6e1c15d580 2023-10-10T20:30:02.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04011b40 con 0x7f6e1c15d580 2023-10-10T20:30:02.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c16e800 con 0x7f6e1c15d580 2023-10-10T20:30:02.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.083+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c14d1c0 con 0x7f6e1c15d580 2023-10-10T20:30:02.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.084+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c0d0680 con 0x7f6e1c15d580 2023-10-10T20:30:02.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.084+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e04019400 con 0x7f6e1c15d580 2023-10-10T20:30:02.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.084+0000 7f6e18ff9700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 0x7f6dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.085+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6e040841e0 con 0x7f6e1c15d580 2023-10-10T20:30:02.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.085+0000 7f6e20d2d700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 0x7f6dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.085+0000 7f6e20d2d700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 0x7f6dfc0693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400d390 tx=0x7f6e14007510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.089+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e1c0d0680 con 0x7f6e1c15d580 2023-10-10T20:30:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.275+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6e1c0d0680 con 0x7f6e1c15d580 2023-10-10T20:30:02.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.275+0000 7f6e18ff9700 1 -- 172.21.15.148:0/1493113425 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v174) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6e1c0d0680 con 0x7f6e1c15d580 2023-10-10T20:30:02.278 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:02.278 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2023-10-10T20:30:02.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:02.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2023-10-10T20:30:02.279 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2023-10-10T20:30:02.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:02.279 INFO:teuthology.orchestra.run.smithi148.stdout:] 2023-10-10T20:30:02.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 msgr2=0x7f6dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 0x7f6dfc0693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400d390 tx=0x7f6e14007510 comp rx=0 tx=0).stop 2023-10-10T20:30:02.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 msgr2=0x7f6e1c14cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c880 tx=0x7f6e04008040 comp rx=0 tx=0).stop 2023-10-10T20:30:02.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 shutdown_connections 2023-10-10T20:30:02.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6dfc066f30 0x7f6dfc0693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e1c15d580 0x7f6e1c14cc80 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e1c157870 0x7f6e1c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 --2- 172.21.15.148:0/1493113425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e1c15a620 0x7f6e1c16b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 >> 172.21.15.148:0/1493113425 conn(0x7f6e1c068ce0 msgr2=0x7f6e1c14fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:02.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 shutdown_connections 2023-10-10T20:30:02.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.277+0000 7f6e22f91700 1 -- 172.21.15.148:0/1493113425 wait complete. 2023-10-10T20:30:02.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-10T20:30:02.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-10T20:30:02.411 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.409+0000 7f4f5a7b5700 1 Processor -- start 2023-10-10T20:30:02.412 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.409+0000 7f4f5a7b5700 1 -- start start 2023-10-10T20:30:02.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f5a7b5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f540cc940 0x7f4f540ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f5a7b5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4f540ce810 0x7f4f540cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f5a7b5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f540cbb80 con 0x7f4f5415d240 2023-10-10T20:30:02.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f5a7b5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540cbe60 con 0x7f4f540cc940 2023-10-10T20:30:02.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f5a7b5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540cc140 con 0x7f4f540ce810 2023-10-10T20:30:02.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f540cc940 0x7f4f540ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f597b3700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4f5415d240 0x7f4f540b3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38430/0 (socket says 172.21.15.148:38430) 2023-10-10T20:30:02.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f597b3700 1 -- 172.21.15.148:0/1088216152 learned_addr learned my addr 172.21.15.148:0/1088216152 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f59fb4700 1 --2- 172.21.15.148:0/1088216152 >> v2:172.21.15.161:3300/0 conn(0x7f4f540ce810 0x7f4f540cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 -- 172.21.15.148:0/1088216152 >> v2:172.21.15.161:3300/0 conn(0x7f4f540ce810 msgr2=0x7f4f540cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 --2- 172.21.15.148:0/1088216152 >> v2:172.21.15.161:3300/0 conn(0x7f4f540ce810 0x7f4f540cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 -- 172.21.15.148:0/1088216152 >> v1:172.21.15.148:6789/0 conn(0x7f4f5415d240 legacy=0x7f4f540b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f53fff700 1 -- 172.21.15.148:0/1088216152 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405292205 0 0) 0x7f4f540cbb80 con 0x7f4f5415d240 2023-10-10T20:30:02.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 -- 172.21.15.148:0/1088216152 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540cc420 con 0x7f4f540cc940 2023-10-10T20:30:02.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.410+0000 7f4f58fb2700 1 --2- 172.21.15.148:0/1088216152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f540cc940 0x7f4f540ccde0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b4110 tx=0x7f4f480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49370b1ad49916db server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f53fff700 1 -- 172.21.15.148:0/1088216152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800b6b0 con 0x7f4f540cc940 2023-10-10T20:30:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f53fff700 1 -- 172.21.15.148:0/1088216152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4800b810 con 0x7f4f540cc940 2023-10-10T20:30:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f53fff700 1 -- 172.21.15.148:0/1088216152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800bbb0 con 0x7f4f540cc940 2023-10-10T20:30:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1088216152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f540cc940 msgr2=0x7f4f540ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1088216152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f540cc940 0x7f4f540ccde0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b4110 tx=0x7f4f480091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1088216152 shutdown_connections 2023-10-10T20:30:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1088216152 >> v2:172.21.15.161:3300/0 conn(0x7f4f540ce810 0x7f4f540cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1088216152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f540cc940 0x7f4f540ccde0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1088216152 >> 172.21.15.148:0/1088216152 conn(0x7f4f54067e40 msgr2=0x7f4f540d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1088216152 shutdown_connections 2023-10-10T20:30:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.411+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1088216152 wait complete. 2023-10-10T20:30:02.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.412+0000 7f4f5a7b5700 1 Processor -- start 2023-10-10T20:30:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.412+0000 7f4f5a7b5700 1 -- start start 2023-10-10T20:30:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f540ce810 0x7f4f5416b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f5415d240 0x7f4f5416cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5416d490 con 0x7f4f5415c0e0 2023-10-10T20:30:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415ca90 con 0x7f4f5415d240 2023-10-10T20:30:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f5a7b5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5409d4a0 con 0x7f4f540ce810 2023-10-10T20:30:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f59fb4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.413+0000 7f4f59fb4700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35536/0 (socket says 172.21.15.148:35536) 2023-10-10T20:30:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 -- 172.21.15.148:0/1124370161 learned_addr learned my addr 172.21.15.148:0/1124370161 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f597b3700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f540ce810 0x7f4f5416b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f58fb2700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f5415d240 0x7f4f5416cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 -- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f540ce810 msgr2=0x7f4f5416b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f540ce810 0x7f4f5416b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 -- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f5415d240 msgr2=0x7f4f5416cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f5415d240 0x7f4f5416cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 -- 172.21.15.148:0/1124370161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48009420 con 0x7f4f5415c0e0 2023-10-10T20:30:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f59fb4700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007f20 tx=0x7f4f44009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400eb90 con 0x7f4f5415c0e0 2023-10-10T20:30:02.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4400ed10 con 0x7f4f5415c0e0 2023-10-10T20:30:02.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5409cb70 con 0x7f4f5415c0e0 2023-10-10T20:30:02.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44014670 con 0x7f4f5415c0e0 2023-10-10T20:30:02.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.414+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5409d130 con 0x7f4f5415c0e0 2023-10-10T20:30:02.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.415+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540cfdd0 con 0x7f4f5415c0e0 2023-10-10T20:30:02.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.415+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f440149e0 con 0x7f4f5415c0e0 2023-10-10T20:30:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.416+0000 7f4f517fa700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 0x7f4f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.416+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4f440846c0 con 0x7f4f5415c0e0 2023-10-10T20:30:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.416+0000 7f4f597b3700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 0x7f4f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.416+0000 7f4f597b3700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 0x7f4f400693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f5415cfc0 tx=0x7f4f4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:02.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.420+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f44051c80 con 0x7f4f5415c0e0 2023-10-10T20:30:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.602+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4f540d1600 con 0x7f4f5415c0e0 2023-10-10T20:30:02.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.873+0000 7f4f517fa700 1 -- 172.21.15.148:0/1124370161 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v175) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f440552a0 con 0x7f4f5415c0e0 2023-10-10T20:30:02.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.876+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 msgr2=0x7f4f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.876+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 0x7f4f400693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4f5415cfc0 tx=0x7f4f4c009040 comp rx=0 tx=0).stop 2023-10-10T20:30:02.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.876+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 msgr2=0x7f4f5415c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:02.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.876+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007f20 tx=0x7f4f44009860 comp rx=0 tx=0).stop 2023-10-10T20:30:02.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 shutdown_connections 2023-10-10T20:30:02.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4f40066f30 0x7f4f400693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5415c0e0 0x7f4f5415c550 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4f540ce810 0x7f4f5416b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 --2- 172.21.15.148:0/1124370161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f5415d240 0x7f4f5416cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 >> 172.21.15.148:0/1124370161 conn(0x7f4f54067e40 msgr2=0x7f4f540cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 shutdown_connections 2023-10-10T20:30:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:02.877+0000 7f4f5a7b5700 1 -- 172.21.15.148:0/1124370161 wait complete. 2023-10-10T20:30:02.882 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2023-10-10T20:30:02.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-10T20:30:03.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.011+0000 7fce2bba7700 1 Processor -- start 2023-10-10T20:30:03.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.011+0000 7fce2bba7700 1 -- start start 2023-10-10T20:30:03.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2bba7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce240d05a0 0x7fce24164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:03.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2bba7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce240b0790 con 0x7fce2415a290 2023-10-10T20:30:03.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2bba7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce240ce1f0 con 0x7fce24159ec0 2023-10-10T20:30:03.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2bba7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240b0b20 con 0x7fce240d05a0 2023-10-10T20:30:03.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce29142700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fce2415a290 0x7fce2406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57324/0 (socket says 172.21.15.148:57324) 2023-10-10T20:30:03.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce29142700 1 -- 172.21.15.148:0/2385932670 learned_addr learned my addr 172.21.15.148:0/2385932670 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:03.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2a144700 1 --2- 172.21.15.148:0/2385932670 >> v2:172.21.15.148:3300/0 conn(0x7fce240d05a0 0x7fce24164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:03.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2a144700 1 -- 172.21.15.148:0/2385932670 >> v1:172.21.15.161:6789/0 conn(0x7fce24159ec0 legacy=0x7fce240af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:03.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2a144700 1 -- 172.21.15.148:0/2385932670 >> v1:172.21.15.148:6789/0 conn(0x7fce2415a290 legacy=0x7fce2406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce2a144700 1 -- 172.21.15.148:0/2385932670 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240b04b0 con 0x7fce240d05a0 2023-10-10T20:30:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.012+0000 7fce28941700 1 -- 172.21.15.148:0/2385932670 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048448615 0 0) 0x7fce240b0790 con 0x7fce2415a290 2023-10-10T20:30:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2a144700 1 --2- 172.21.15.148:0/2385932670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce24164ed0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded5496f8e3eff70 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce28941700 1 -- 172.21.15.148:0/2385932670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c009e80 con 0x7fce240d05a0 2023-10-10T20:30:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce28941700 1 -- 172.21.15.148:0/2385932670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00b820 con 0x7fce240d05a0 2023-10-10T20:30:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce28941700 1 -- 172.21.15.148:0/2385932670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00bba0 con 0x7fce240d05a0 2023-10-10T20:30:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 -- 172.21.15.148:0/2385932670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 msgr2=0x7fce24164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 --2- 172.21.15.148:0/2385932670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce24164ed0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0098e0 tx=0x7fce1c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 -- 172.21.15.148:0/2385932670 shutdown_connections 2023-10-10T20:30:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 --2- 172.21.15.148:0/2385932670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce24164ed0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 -- 172.21.15.148:0/2385932670 >> 172.21.15.148:0/2385932670 conn(0x7fce240d1880 msgr2=0x7fce240cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 -- 172.21.15.148:0/2385932670 shutdown_connections 2023-10-10T20:30:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.013+0000 7fce2bba7700 1 -- 172.21.15.148:0/2385932670 wait complete. 2023-10-10T20:30:03.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.014+0000 7fce2bba7700 1 Processor -- start 2023-10-10T20:30:03.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.014+0000 7fce2bba7700 1 -- start start 2023-10-10T20:30:03.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:03.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fce24159ec0 0x7fce24152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:03.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2415a290 0x7fce2414f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:03.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce24152c80 con 0x7fce240d05a0 2023-10-10T20:30:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b47b0 con 0x7fce2415a290 2023-10-10T20:30:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2bba7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b44a0 con 0x7fce24159ec0 2023-10-10T20:30:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33330/0 (socket says 172.21.15.148:33330) 2023-10-10T20:30:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 -- 172.21.15.148:0/3878706181 learned_addr learned my addr 172.21.15.148:0/3878706181 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:03.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce2a144700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2415a290 0x7fce2414f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:03.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29142700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fce24159ec0 0x7fce24152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:03.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 -- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fce24159ec0 msgr2=0x7fce24152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fce24159ec0 0x7fce24152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 -- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2415a290 msgr2=0x7fce2414f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.015+0000 7fce29943700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2415a290 0x7fce2414f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce29943700 1 -- 172.21.15.148:0/3878706181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c009420 con 0x7fce240d05a0 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce29943700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c980 tx=0x7fce20002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000ce70 con 0x7fce240d05a0 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000a7a0 con 0x7fce240d05a0 2023-10-10T20:30:03.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000ab60 con 0x7fce240d05a0 2023-10-10T20:30:03.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce24160420 con 0x7fce240d05a0 2023-10-10T20:30:03.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.016+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce241609e0 con 0x7fce240d05a0 2023-10-10T20:30:03.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.017+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce24160c00 con 0x7fce240d05a0 2023-10-10T20:30:03.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.017+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce20012420 con 0x7fce240d05a0 2023-10-10T20:30:03.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.018+0000 7fce19ffb700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:03.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.018+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fce20084770 con 0x7fce240d05a0 2023-10-10T20:30:03.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.018+0000 7fce29142700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:03.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.018+0000 7fce29142700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fce2416d4c0 tx=0x7fce14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:03.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.022+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce20051c70 con 0x7fce240d05a0 2023-10-10T20:30:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.206+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce2404c3c0 con 0x7fce240d05a0 2023-10-10T20:30:03.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.883+0000 7fce19ffb700 1 -- 172.21.15.148:0/3878706181 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v176) v1 ==== 170+0+0 (secure 0 0 0) 0x7fce20055290 con 0x7fce240d05a0 2023-10-10T20:30:03.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 msgr2=0x7fce080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fce2416d4c0 tx=0x7fce14009040 comp rx=0 tx=0).stop 2023-10-10T20:30:03.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 msgr2=0x7fce2415fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:03.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c980 tx=0x7fce20002760 comp rx=0 tx=0).stop 2023-10-10T20:30:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 shutdown_connections 2023-10-10T20:30:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.891+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce2415a290 0x7fce2414f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.892+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce240d05a0 0x7fce2415fd60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.892+0000 7fce2bba7700 1 --2- 172.21.15.148:0/3878706181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fce24159ec0 0x7fce24152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.892+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 >> 172.21.15.148:0/3878706181 conn(0x7fce240d1880 msgr2=0x7fce2406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:03.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.892+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 shutdown_connections 2023-10-10T20:30:03.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:03.892+0000 7fce2bba7700 1 -- 172.21.15.148:0/3878706181 wait complete. 2023-10-10T20:30:03.898 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2023-10-10T20:30:03.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-10T20:30:04.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.025+0000 7f74797b2700 1 Processor -- start 2023-10-10T20:30:04.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.025+0000 7f74797b2700 1 -- start start 2023-10-10T20:30:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74797b2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7474159ec0 0x7f74740af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74797b2700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74740b0790 con 0x7f747406acb0 2023-10-10T20:30:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74797b2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74740ce1f0 con 0x7f747415a290 2023-10-10T20:30:04.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74797b2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74740b0b20 con 0x7f7474159ec0 2023-10-10T20:30:04.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74737fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f747406acb0 0x7f74740cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57332/0 (socket says 172.21.15.148:57332) 2023-10-10T20:30:04.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74737fe700 1 -- 172.21.15.148:0/960338916 learned_addr learned my addr 172.21.15.148:0/960338916 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:04.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f7472ffd700 1 --2- 172.21.15.148:0/960338916 >> v2:172.21.15.161:3300/0 conn(0x7f7474159ec0 0x7f74740af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532949129 0 0) 0x7f74740b0790 con 0x7f747406acb0 2023-10-10T20:30:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7460003570 con 0x7f747406acb0 2023-10-10T20:30:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2032922459 0 0) 0x7f7460003570 con 0x7f747406acb0 2023-10-10T20:30:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 >> v2:172.21.15.161:3300/0 conn(0x7f7474159ec0 msgr2=0x7f74740af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 --2- 172.21.15.148:0/960338916 >> v2:172.21.15.161:3300/0 conn(0x7f7474159ec0 0x7f74740af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 >> v1:172.21.15.161:6789/0 conn(0x7f747415a290 legacy=0x7f747406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740b04b0 con 0x7f747406acb0 2023-10-10T20:30:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f746c0017f0 con 0x7f747406acb0 2023-10-10T20:30:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 >> v1:172.21.15.148:6789/0 conn(0x7f747406acb0 legacy=0x7f74740cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7460003a60 0x7f7460003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 0x7f7460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74600051c0 0x7f746000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740b0b20 con 0x7f7460004810 2023-10-10T20:30:04.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746000c160 con 0x7f7460003a60 2023-10-10T20:30:04.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.026+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746000c320 con 0x7f74600051c0 2023-10-10T20:30:04.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 0x7f7460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f7472ffd700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7460003a60 0x7f7460003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f74737fe700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74600051c0 0x7f746000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 -- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74600051c0 msgr2=0x7f746000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74600051c0 0x7f746000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 -- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7460003a60 msgr2=0x7f7460003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7460003a60 0x7f7460003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 -- 172.21.15.148:0/960338916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460003570 con 0x7f7460004810 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f74737fe700 1 -- 172.21.15.148:0/960338916 reap_dead start 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f746a7fc700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 0x7f7460004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f745c0018b0 tx=0x7f745c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245c6c1bce09fb0b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c012420 con 0x7f7460004810 2023-10-10T20:30:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c012580 con 0x7f7460004810 2023-10-10T20:30:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.027+0000 7f74727fc700 1 -- 172.21.15.148:0/960338916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c012920 con 0x7f7460004810 2023-10-10T20:30:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 -- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 msgr2=0x7f7460004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 0x7f7460004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f745c0018b0 tx=0x7f745c007d30 comp rx=0 tx=0).stop 2023-10-10T20:30:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 -- 172.21.15.148:0/960338916 shutdown_connections 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 --2- 172.21.15.148:0/960338916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7460004810 0x7f7460004c80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 -- 172.21.15.148:0/960338916 >> 172.21.15.148:0/960338916 conn(0x7f74740d1880 msgr2=0x7f74740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 -- 172.21.15.148:0/960338916 shutdown_connections 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.028+0000 7f74797b2700 1 -- 172.21.15.148:0/960338916 wait complete. 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.029+0000 7f74797b2700 1 Processor -- start 2023-10-10T20:30:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.029+0000 7f74797b2700 1 -- start start 2023-10-10T20:30:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7460004810 0x7f74740afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74740b59f0 0x7f747414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c9ea0 con 0x7f74740c8680 2023-10-10T20:30:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740c9030 con 0x7f74740b59f0 2023-10-10T20:30:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74741532b0 con 0x7f7460004810 2023-10-10T20:30:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33348/0 (socket says 172.21.15.148:33348) 2023-10-10T20:30:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 -- 172.21.15.148:0/3139909808 learned_addr learned my addr 172.21.15.148:0/3139909808 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f7472ffd700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7460004810 0x7f74740afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f746a7fc700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74740b59f0 0x7f747414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 -- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7460004810 msgr2=0x7f74740afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7460004810 0x7f74740afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 -- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74740b59f0 msgr2=0x7f747414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74740b59f0 0x7f747414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 -- 172.21.15.148:0/3139909808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740b5f90 con 0x7f74740c8680 2023-10-10T20:30:04.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74737fe700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f746c000f80 tx=0x7f746c0018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:04.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c00eb10 con 0x7f74740c8680 2023-10-10T20:30:04.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c00ec70 con 0x7f74740c8680 2023-10-10T20:30:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74740b6270 con 0x7f74740c8680 2023-10-10T20:30:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.030+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c013420 con 0x7f74740c8680 2023-10-10T20:30:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.031+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74740b6830 con 0x7f74740c8680 2023-10-10T20:30:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.031+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7474153440 con 0x7f74740c8680 2023-10-10T20:30:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.032+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f746c0137f0 con 0x7f74740c8680 2023-10-10T20:30:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.032+0000 7f746b7fe700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.032+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f746c084a10 con 0x7f74740c8680 2023-10-10T20:30:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.032+0000 7f7472ffd700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.033+0000 7f7472ffd700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 0x7f74540693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f747404d1b0 tx=0x7f74640091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.037+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f746c051ce0 con 0x7f74740c8680 2023-10-10T20:30:04.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.258+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f747404c3c0 con 0x7f74740c8680 2023-10-10T20:30:04.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.883+0000 7f746b7fe700 1 -- 172.21.15.148:0/3139909808 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f746c055300 con 0x7f74740c8680 2023-10-10T20:30:04.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 msgr2=0x7f74540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 0x7f74540693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f747404d1b0 tx=0x7f74640091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.898 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 msgr2=0x7f74740c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f746c000f80 tx=0x7f746c0018f0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 shutdown_connections 2023-10-10T20:30:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.895+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74740c8680 0x7f74740c8af0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.896+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7460004810 0x7f74740afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.896+0000 7f74797b2700 1 --2- 172.21.15.148:0/3139909808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74740b59f0 0x7f747414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:04.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.896+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 >> 172.21.15.148:0/3139909808 conn(0x7f74740d1880 msgr2=0x7f74740b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:04.900 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.898+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 shutdown_connections 2023-10-10T20:30:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:04.898+0000 7f74797b2700 1 -- 172.21.15.148:0/3139909808 wait complete. 2023-10-10T20:30:04.901 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 22 and data pool 23 2023-10-10T20:30:04.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:05.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.027+0000 7fa60b681700 1 Processor -- start 2023-10-10T20:30:05.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.028+0000 7fa60b681700 1 -- start start 2023-10-10T20:30:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.028+0000 7fa60b681700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa6040adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.028+0000 7fa60b681700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040c87d0 con 0x7fa604156a50 2023-10-10T20:30:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.028+0000 7fa60b681700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa604068750 con 0x7fa604156e20 2023-10-10T20:30:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.028+0000 7fa60b681700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa604068d10 con 0x7fa604068280 2023-10-10T20:30:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa60941d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa604156a50 0x7fa6040c8c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57348/0 (socket says 172.21.15.148:57348) 2023-10-10T20:30:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa60941d700 1 -- 172.21.15.148:0/2086583945 learned_addr learned my addr 172.21.15.148:0/2086583945 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633996119 0 0) 0x7fa6040c87d0 con 0x7fa604156a50 2023-10-10T20:30:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa609c1e700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa6040adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f0005780 con 0x7fa604156a50 2023-10-10T20:30:05.035 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637371822 0 0) 0x7fa5f0005780 con 0x7fa604156a50 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 msgr2=0x7fa6040adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa6040adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 >> v1:172.21.15.161:6789/0 conn(0x7fa604156e20 legacy=0x7fa604067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604068a30 con 0x7fa604156a50 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa600001470 con 0x7fa604156a50 2023-10-10T20:30:05.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 >> v1:172.21.15.148:6789/0 conn(0x7fa604156a50 legacy=0x7fa6040c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0005ca0 0x7fa5f00060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5f0006a20 0x7fa5f0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa5f00073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604068d10 con 0x7fa5f0006a20 2023-10-10T20:30:05.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f000dd70 con 0x7fa5f0005ca0 2023-10-10T20:30:05.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f000df30 con 0x7fa604068280 2023-10-10T20:30:05.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa608c1c700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5f0006a20 0x7fa5f0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa60941d700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0005ca0 0x7fa5f00060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa609c1e700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa5f00073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa608c1c700 1 -- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 msgr2=0x7fa5f00073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.029+0000 7fa608c1c700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa604068280 0x7fa5f00073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa608c1c700 1 -- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0005ca0 msgr2=0x7fa5f00060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa608c1c700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f0005ca0 0x7fa5f00060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa608c1c700 1 -- 172.21.15.148:0/2086583945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0005780 con 0x7fa5f0006a20 2023-10-10T20:30:05.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa608c1c700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5f0006a20 0x7fa5f0006e90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4001880 tx=0x7fa5f4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8956e753ab80f10e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4012420 con 0x7fa5f0006a20 2023-10-10T20:30:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f4012580 con 0x7fa5f0006a20 2023-10-10T20:30:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa5fbfff700 1 -- 172.21.15.148:0/2086583945 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4012920 con 0x7fa5f0006a20 2023-10-10T20:30:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 -- 172.21.15.148:0/2086583945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5f0006a20 msgr2=0x7fa5f0006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 --2- 172.21.15.148:0/2086583945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5f0006a20 0x7fa5f0006e90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4001880 tx=0x7fa5f4007d30 comp rx=0 tx=0).stop 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa609c1e700 1 -- 172.21.15.148:0/2086583945 reap_dead start 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 -- 172.21.15.148:0/2086583945 shutdown_connections 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 -- 172.21.15.148:0/2086583945 >> 172.21.15.148:0/2086583945 conn(0x7fa60406a250 msgr2=0x7fa60406a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 -- 172.21.15.148:0/2086583945 shutdown_connections 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.030+0000 7fa60b681700 1 -- 172.21.15.148:0/2086583945 wait complete. 2023-10-10T20:30:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.031+0000 7fa60b681700 1 Processor -- start 2023-10-10T20:30:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.031+0000 7fa60b681700 1 -- start start 2023-10-10T20:30:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5f0006a20 0x7fa6040c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6040ae1b0 0x7fa60414ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040af330 con 0x7fa6040b3130 2023-10-10T20:30:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60414f4d0 con 0x7fa6040ae1b0 2023-10-10T20:30:05.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa60b681700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040ae720 con 0x7fa5f0006a20 2023-10-10T20:30:05.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa609c1e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa609c1e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33366/0 (socket says 172.21.15.148:33366) 2023-10-10T20:30:05.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.032+0000 7fa609c1e700 1 -- 172.21.15.148:0/1106118495 learned_addr learned my addr 172.21.15.148:0/1106118495 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:05.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa60941d700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5f0006a20 0x7fa6040c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.053 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa608c1c700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6040ae1b0 0x7fa60414ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 -- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5f0006a20 msgr2=0x7fa6040c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5f0006a20 0x7fa6040c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 -- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6040ae1b0 msgr2=0x7fa60414ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6040ae1b0 0x7fa60414ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 -- 172.21.15.148:0/1106118495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040aea00 con 0x7fa6040b3130 2023-10-10T20:30:05.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa609c1e700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa6040c8040 tx=0x7fa5fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00e070 con 0x7fa6040b3130 2023-10-10T20:30:05.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc011c30 con 0x7fa6040b3130 2023-10-10T20:30:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc016470 con 0x7fa6040b3130 2023-10-10T20:30:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040aece0 con 0x7fa6040b3130 2023-10-10T20:30:05.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.033+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040b35d0 con 0x7fa6040b3130 2023-10-10T20:30:05.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.034+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040af180 con 0x7fa6040b3130 2023-10-10T20:30:05.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.034+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5fc016860 con 0x7fa6040b3130 2023-10-10T20:30:05.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.035+0000 7fa5f97fa700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.035+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5fc084f20 con 0x7fa6040b3130 2023-10-10T20:30:05.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.035+0000 7fa60941d700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.035+0000 7fa60941d700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 0x7fa5e80693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa600001200 tx=0x7fa6040c8740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.039+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5fc052250 con 0x7fa6040b3130 2023-10-10T20:30:05.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.224+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa60404c3c0 con 0x7fa6040b3130 2023-10-10T20:30:05.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.224+0000 7fa5f97fa700 1 -- 172.21.15.148:0/1106118495 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+15426 (secure 0 0 0) 0x7fa5fc055870 con 0x7fa6040b3130 2023-10-10T20:30:05.227 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:05.227 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":177,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:04.260187+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 msgr2=0x7fa5e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 0x7fa5e80693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa600001200 tx=0x7fa6040c8740 comp rx=0 tx=0).stop 2023-10-10T20:30:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 msgr2=0x7fa6040b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa6040c8040 tx=0x7fa5fc00b040 comp rx=0 tx=0).stop 2023-10-10T20:30:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 shutdown_connections 2023-10-10T20:30:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa5e8066f30 0x7fa5e80693e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6040b3130 0x7fa6040b0900 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa5f0006a20 0x7fa6040c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 --2- 172.21.15.148:0/1106118495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6040ae1b0 0x7fa60414ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.231 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 >> 172.21.15.148:0/1106118495 conn(0x7fa60406a250 msgr2=0x7fa604154820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 shutdown_connections 2023-10-10T20:30:05.232 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.227+0000 7fa60b681700 1 -- 172.21.15.148:0/1106118495 wait complete. 2023-10-10T20:30:05.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:05.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.353+0000 7f0abf08d700 1 Processor -- start 2023-10-10T20:30:05.356 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.353+0000 7f0abf08d700 1 -- start start 2023-10-10T20:30:05.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abf08d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0ab81577d0 0x7f0ab80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abf08d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab806ac30 0x7f0ab815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.359 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abf08d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab815c360 con 0x7f0ab8157ba0 2023-10-10T20:30:05.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abf08d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab815b790 con 0x7f0ab81577d0 2023-10-10T20:30:05.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abf08d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab815ba30 con 0x7f0ab806ac30 2023-10-10T20:30:05.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0ab7fff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ab8157ba0 0x7f0ab806a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57352/0 (socket says 172.21.15.148:57352) 2023-10-10T20:30:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0ab7fff700 1 -- 172.21.15.148:0/3355354154 learned_addr learned my addr 172.21.15.148:0/3355354154 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.354+0000 7f0abce29700 1 --2- 172.21.15.148:0/3355354154 >> v2:172.21.15.161:3300/0 conn(0x7f0ab81577d0 0x7f0ab80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abd62a700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab806ac30 0x7f0ab815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0ab77fe700 1 -- 172.21.15.148:0/3355354154 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615836904 0 0) 0x7f0ab815c360 con 0x7f0ab8157ba0 2023-10-10T20:30:05.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0ab77fe700 1 -- 172.21.15.148:0/3355354154 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa8003570 con 0x7f0ab8157ba0 2023-10-10T20:30:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abce29700 1 -- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab806ac30 msgr2=0x7f0ab815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abce29700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab806ac30 0x7f0ab815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abce29700 1 -- 172.21.15.148:0/3355354154 >> v1:172.21.15.148:6789/0 conn(0x7f0ab8157ba0 legacy=0x7f0ab806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abce29700 1 -- 172.21.15.148:0/3355354154 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab815bd10 con 0x7f0ab81577d0 2023-10-10T20:30:05.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0abce29700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab81577d0 0x7f0ab80cd770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0098e0 tx=0x7f0aac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b36f1411b563c873 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0ab77fe700 1 -- 172.21.15.148:0/3355354154 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac011630 con 0x7f0ab81577d0 2023-10-10T20:30:05.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0ab77fe700 1 -- 172.21.15.148:0/3355354154 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac011790 con 0x7f0ab81577d0 2023-10-10T20:30:05.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.355+0000 7f0ab77fe700 1 -- 172.21.15.148:0/3355354154 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac011b10 con 0x7f0ab81577d0 2023-10-10T20:30:05.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 -- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab81577d0 msgr2=0x7f0ab80cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab81577d0 0x7f0ab80cd770 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0098e0 tx=0x7f0aac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 -- 172.21.15.148:0/3355354154 shutdown_connections 2023-10-10T20:30:05.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab806ac30 0x7f0ab815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 --2- 172.21.15.148:0/3355354154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab81577d0 0x7f0ab80cd770 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 -- 172.21.15.148:0/3355354154 >> 172.21.15.148:0/3355354154 conn(0x7f0ab8068ce0 msgr2=0x7f0ab80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:05.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 -- 172.21.15.148:0/3355354154 shutdown_connections 2023-10-10T20:30:05.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.356+0000 7f0abf08d700 1 -- 172.21.15.148:0/3355354154 wait complete. 2023-10-10T20:30:05.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.357+0000 7f0abf08d700 1 Processor -- start 2023-10-10T20:30:05.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.357+0000 7f0abf08d700 1 -- start start 2023-10-10T20:30:05.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab81577d0 0x7f0ab80accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8157ba0 0x7f0ab80b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ac870 con 0x7f0ab806ac30 2023-10-10T20:30:05.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ab930 con 0x7f0ab8157ba0 2023-10-10T20:30:05.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abf08d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80aaf60 con 0x7f0ab81577d0 2023-10-10T20:30:05.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33370/0 (socket says 172.21.15.148:33370) 2023-10-10T20:30:05.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 -- 172.21.15.148:0/644147590 learned_addr learned my addr 172.21.15.148:0/644147590 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:05.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 -- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab81577d0 msgr2=0x7f0ab80accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab81577d0 0x7f0ab80accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 -- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8157ba0 msgr2=0x7f0ab80b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8157ba0 0x7f0ab80b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 -- 172.21.15.148:0/644147590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac009420 con 0x7f0ab806ac30 2023-10-10T20:30:05.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0abce29700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009ed0 tx=0x7f0aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.358+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac011b10 con 0x7f0ab806ac30 2023-10-10T20:30:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.359+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac000d10 con 0x7f0ab806ac30 2023-10-10T20:30:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.359+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac01a660 con 0x7f0ab806ac30 2023-10-10T20:30:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.359+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab80ad1f0 con 0x7f0ab806ac30 2023-10-10T20:30:05.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.359+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80aa6b0 con 0x7f0ab806ac30 2023-10-10T20:30:05.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.359+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80ab240 con 0x7f0ab806ac30 2023-10-10T20:30:05.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.360+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0aac01aa90 con 0x7f0ab806ac30 2023-10-10T20:30:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.360+0000 7f0ab4ff9700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 0x7f0a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.360+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0aac0860b0 con 0x7f0ab806ac30 2023-10-10T20:30:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.360+0000 7f0ab7fff700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 0x7f0a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.361+0000 7f0ab7fff700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 0x7f0a9c0693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40058d0 tx=0x7f0aa40079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.365+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aac053320 con 0x7f0ab806ac30 2023-10-10T20:30:05.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.548+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab815c040 con 0x7f0ab806ac30 2023-10-10T20:30:05.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.549+0000 7f0ab4ff9700 1 -- 172.21.15.148:0/644147590 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+15426 (secure 0 0 0) 0x7f0aac056940 con 0x7f0ab806ac30 2023-10-10T20:30:05.551 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:05.552 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":177,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:04.260187+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 msgr2=0x7f0a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 0x7f0a9c0693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40058d0 tx=0x7f0aa40079d0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 msgr2=0x7f0ab80ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009ed0 tx=0x7f0aac017000 comp rx=0 tx=0).stop 2023-10-10T20:30:05.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 shutdown_connections 2023-10-10T20:30:05.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0a9c066f30 0x7f0a9c0693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ab8157ba0 0x7f0ab80b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab806ac30 0x7f0ab80ab3f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.555 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 --2- 172.21.15.148:0/644147590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0ab81577d0 0x7f0ab80accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 >> 172.21.15.148:0/644147590 conn(0x7f0ab8068ce0 msgr2=0x7f0ab80d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:05.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 shutdown_connections 2023-10-10T20:30:05.556 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.551+0000 7f0abf08d700 1 -- 172.21.15.148:0/644147590 wait complete. 2023-10-10T20:30:05.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-10T20:30:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.682+0000 7fcb3278e700 1 Processor -- start 2023-10-10T20:30:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.682+0000 7fcb3278e700 1 -- start start 2023-10-10T20:30:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb3278e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcb2c0b3570 0x7fcb2c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb3278e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c0b14f0 con 0x7fcb2c0b4c30 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb3278e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c0b17d0 con 0x7fcb2c0b4860 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb3278e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c0b1ab0 con 0x7fcb2c0b3570 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb30d2b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb2c0b4c30 0x7fcb2c0cfe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57364/0 (socket says 172.21.15.148:57364) 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb30d2b700 1 -- 172.21.15.148:0/1178894663 learned_addr learned my addr 172.21.15.148:0/1178894663 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.683+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/1178894663 >> v2:172.21.15.161:3300/0 conn(0x7fcb2c0b3570 0x7fcb2c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777759559 0 0) 0x7fcb2c0b14f0 con 0x7fcb2c0b4c30 2023-10-10T20:30:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb14003570 con 0x7fcb2c0b4c30 2023-10-10T20:30:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713524111 0 0) 0x7fcb14003570 con 0x7fcb2c0b4c30 2023-10-10T20:30:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1178894663 >> v1:172.21.15.161:6789/0 conn(0x7fcb2c0b4860 legacy=0x7fcb2c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1178894663 >> v1:172.21.15.148:6789/0 conn(0x7fcb2c0b4c30 legacy=0x7fcb2c0cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1178894663 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c0b1d90 con 0x7fcb2c0b3570 2023-10-10T20:30:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/1178894663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b3570 0x7fcb2c15a1a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb200098e0 tx=0x7fcb200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0808e08addb2060 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240017f0 con 0x7fcb2c0b3570 2023-10-10T20:30:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb20011420 con 0x7fcb2c0b3570 2023-10-10T20:30:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.684+0000 7fcb2affd700 1 -- 172.21.15.148:0/1178894663 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb200117c0 con 0x7fcb2c0b3570 2023-10-10T20:30:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 -- 172.21.15.148:0/1178894663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b3570 msgr2=0x7fcb2c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1178894663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b3570 0x7fcb2c15a1a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb200098e0 tx=0x7fcb200091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 -- 172.21.15.148:0/1178894663 shutdown_connections 2023-10-10T20:30:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1178894663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b3570 0x7fcb2c15a1a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 -- 172.21.15.148:0/1178894663 >> 172.21.15.148:0/1178894663 conn(0x7fcb2c09a0f0 msgr2=0x7fcb2c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:05.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 -- 172.21.15.148:0/1178894663 shutdown_connections 2023-10-10T20:30:05.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.685+0000 7fcb3278e700 1 -- 172.21.15.148:0/1178894663 wait complete. 2023-10-10T20:30:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.686+0000 7fcb3278e700 1 Processor -- start 2023-10-10T20:30:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.686+0000 7fcb3278e700 1 -- start start 2023-10-10T20:30:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 0x7fcb2c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b4c30 0x7fcb2c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c154550 con 0x7fcb2c0b4860 2023-10-10T20:30:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c153940 con 0x7fcb2c0b4c30 2023-10-10T20:30:05.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c153c20 con 0x7fcb2c0b3570 2023-10-10T20:30:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33384/0 (socket says 172.21.15.148:33384) 2023-10-10T20:30:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1819457527 learned_addr learned my addr 172.21.15.148:0/1819457527 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb30d2b700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b4c30 0x7fcb2c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2bfff700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 0x7fcb2c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 msgr2=0x7fcb2c152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 0x7fcb2c152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b4c30 msgr2=0x7fcb2c155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b4c30 0x7fcb2c155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:05.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 -- 172.21.15.148:0/1819457527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20009420 con 0x7fcb2c0b4860 2023-10-10T20:30:05.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2bfff700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 0x7fcb2c152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:05.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb2b7fe700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fcb200030c0 tx=0x7fcb20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20010c20 con 0x7fcb2c0b4860 2023-10-10T20:30:05.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb20010d80 con 0x7fcb2c0b4860 2023-10-10T20:30:05.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c153f00 con 0x7fcb2c0b4860 2023-10-10T20:30:05.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2001d680 con 0x7fcb2c0b4860 2023-10-10T20:30:05.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.687+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1549e0 con 0x7fcb2c0b4860 2023-10-10T20:30:05.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.688+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c1546c0 con 0x7fcb2c0b4860 2023-10-10T20:30:05.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.689+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb2001d9f0 con 0x7fcb2c0b4860 2023-10-10T20:30:05.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.689+0000 7fcb13fff700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:05.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.689+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb20085f80 con 0x7fcb2c0b4860 2023-10-10T20:30:05.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.689+0000 7fcb2bfff700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:05.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.690+0000 7fcb2bfff700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 0x7fcb180693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c007fd0 tx=0x7fcb1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:05.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.694+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb200531f0 con 0x7fcb2c0b4860 2023-10-10T20:30:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:05.919+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcb2c04c3c0 con 0x7fcb2c0b4860 2023-10-10T20:30:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.902+0000 7fcb13fff700 1 -- 172.21.15.148:0/1819457527 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb20056810 con 0x7fcb2c0b4860 2023-10-10T20:30:06.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 msgr2=0x7fcb180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:06.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 0x7fcb180693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c007fd0 tx=0x7fcb1c009040 comp rx=0 tx=0).stop 2023-10-10T20:30:06.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 msgr2=0x7fcb2c16aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:06.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fcb200030c0 tx=0x7fcb20017000 comp rx=0 tx=0).stop 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 shutdown_connections 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c0b4c30 0x7fcb2c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c0b3570 0x7fcb2c152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 --2- 172.21.15.148:0/1819457527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb2c0b4860 0x7fcb2c16aff0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:06.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 >> 172.21.15.148:0/1819457527 conn(0x7fcb2c09a0f0 msgr2=0x7fcb2c15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:06.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 shutdown_connections 2023-10-10T20:30:06.910 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:06.905+0000 7fcb3278e700 1 -- 172.21.15.148:0/1819457527 wait complete. 2023-10-10T20:30:06.922 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-10T20:30:07.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.042+0000 7fcb0f72b700 1 Processor -- start 2023-10-10T20:30:07.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.042+0000 7fcb0f72b700 1 -- start start 2023-10-10T20:30:07.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0f72b700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb0815a290 0x7fcb0806a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:07.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0f72b700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080b0790 con 0x7fcb0806aae0 2023-10-10T20:30:07.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0f72b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0806af20 con 0x7fcb08159ec0 2023-10-10T20:30:07.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0f72b700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb080ce1f0 con 0x7fcb0815a290 2023-10-10T20:30:07.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb0815a290 0x7fcb0806a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:07.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0dcc8700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb0806aae0 0x7fcb080cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57380/0 (socket says 172.21.15.148:57380) 2023-10-10T20:30:07.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb0815a290 0x7fcb0806a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33386/0 (socket says 172.21.15.148:33386) 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0dcc8700 1 -- 172.21.15.148:0/1995429975 learned_addr learned my addr 172.21.15.148:0/1995429975 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb07fff700 1 -- 172.21.15.148:0/1995429975 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995162437 0 0) 0x7fcb080b0790 con 0x7fcb0806aae0 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb07fff700 1 -- 172.21.15.148:0/1995429975 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf0003570 con 0x7fcb0806aae0 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/1995429975 >> v1:172.21.15.161:6789/0 conn(0x7fcb08159ec0 legacy=0x7fcb080af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/1995429975 >> v1:172.21.15.148:6789/0 conn(0x7fcb0806aae0 legacy=0x7fcb080cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/1995429975 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080b0b20 con 0x7fcb0815a290 2023-10-10T20:30:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.043+0000 7fcb0ccc6700 1 --2- 172.21.15.148:0/1995429975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0815a290 0x7fcb0806a5a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f037f75c13c9b93 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb07fff700 1 -- 172.21.15.148:0/1995429975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00b640 con 0x7fcb0815a290 2023-10-10T20:30:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb07fff700 1 -- 172.21.15.148:0/1995429975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00b7a0 con 0x7fcb0815a290 2023-10-10T20:30:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb07fff700 1 -- 172.21.15.148:0/1995429975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00bb40 con 0x7fcb0815a290 2023-10-10T20:30:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 -- 172.21.15.148:0/1995429975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0815a290 msgr2=0x7fcb0806a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/1995429975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0815a290 0x7fcb0806a5a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 -- 172.21.15.148:0/1995429975 shutdown_connections 2023-10-10T20:30:07.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/1995429975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0815a290 0x7fcb0806a5a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 -- 172.21.15.148:0/1995429975 >> 172.21.15.148:0/1995429975 conn(0x7fcb080d1880 msgr2=0x7fcb080cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:07.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 -- 172.21.15.148:0/1995429975 shutdown_connections 2023-10-10T20:30:07.052 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.044+0000 7fcb0f72b700 1 -- 172.21.15.148:0/1995429975 wait complete. 2023-10-10T20:30:07.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.045+0000 7fcb0f72b700 1 Processor -- start 2023-10-10T20:30:07.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.045+0000 7fcb0f72b700 1 -- start start 2023-10-10T20:30:07.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0806aae0 0x7fcb0816b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:07.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:07.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb0815a290 0x7fcb0816a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:07.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0816b490 con 0x7fcb08159ec0 2023-10-10T20:30:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0816d7b0 con 0x7fcb0806aae0 2023-10-10T20:30:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0f72b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0816aed0 con 0x7fcb0815a290 2023-10-10T20:30:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0ccc6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0ccc6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33388/0 (socket says 172.21.15.148:33388) 2023-10-10T20:30:07.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/3804143951 learned_addr learned my addr 172.21.15.148:0/3804143951 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0d4c7700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0806aae0 0x7fcb0816b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.046+0000 7fcb0dcc8700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb0815a290 0x7fcb0816a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb0815a290 msgr2=0x7fcb0816a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb0815a290 0x7fcb0816a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0806aae0 msgr2=0x7fcb0816b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0806aae0 0x7fcb0816b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 -- 172.21.15.148:0/3804143951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009420 con 0x7fcb08159ec0 2023-10-10T20:30:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0ccc6700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0093f0 tx=0x7fcafc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00bb20 con 0x7fcb08159ec0 2023-10-10T20:30:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc011420 con 0x7fcb08159ec0 2023-10-10T20:30:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0117c0 con 0x7fcb08159ec0 2023-10-10T20:30:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0816b1b0 con 0x7fcb08159ec0 2023-10-10T20:30:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.047+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb08156790 con 0x7fcb08159ec0 2023-10-10T20:30:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.048+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0804c3c0 con 0x7fcb08159ec0 2023-10-10T20:30:07.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.048+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcafc011bf0 con 0x7fcb08159ec0 2023-10-10T20:30:07.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.048+0000 7fcb057fa700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:07.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.049+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcafc085b90 con 0x7fcb08159ec0 2023-10-10T20:30:07.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.049+0000 7fcb0d4c7700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:07.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.049+0000 7fcb0d4c7700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b5bd0 tx=0x7fcb00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:07.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.053+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcafc052de0 con 0x7fcb08159ec0 2023-10-10T20:30:07.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.277+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcb080b04b0 con 0x7fcb08159ec0 2023-10-10T20:30:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.911+0000 7fcb057fa700 1 -- 172.21.15.148:0/3804143951 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcafc0150c0 con 0x7fcb08159ec0 2023-10-10T20:30:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.913+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 msgr2=0x7fcaf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.913+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b5bd0 tx=0x7fcb00009040 comp rx=0 tx=0).stop 2023-10-10T20:30:07.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 msgr2=0x7fcb0816d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:07.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0093f0 tx=0x7fcafc017000 comp rx=0 tx=0).stop 2023-10-10T20:30:07.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 shutdown_connections 2023-10-10T20:30:07.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fcaf4066f30 0x7fcaf40693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.917 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb0815a290 0x7fcb0816a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb0806aae0 0x7fcb0816b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 --2- 172.21.15.148:0/3804143951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08159ec0 0x7fcb0816d270 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:07.918 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 >> 172.21.15.148:0/3804143951 conn(0x7fcb080d1880 msgr2=0x7fcb080afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:07.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 shutdown_connections 2023-10-10T20:30:07.919 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:07.914+0000 7fcb0f72b700 1 -- 172.21.15.148:0/3804143951 wait complete. 2023-10-10T20:30:07.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 Processor -- start 2023-10-10T20:30:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 -- start start 2023-10-10T20:30:08.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc9415a620 0x7fbc9415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9415cbb0 con 0x7fbc9415d580 2023-10-10T20:30:08.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9415bfa0 con 0x7fbc94157870 2023-10-10T20:30:08.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.042+0000 7fbc9a94a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9415bdb0 con 0x7fbc9415a620 2023-10-10T20:30:08.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc937fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc9415a620 0x7fbc9415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc98ee7700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbc9415d580 0x7fbc94158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57386/0 (socket says 172.21.15.148:57386) 2023-10-10T20:30:08.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc98ee7700 1 -- 172.21.15.148:0/551851122 learned_addr learned my addr 172.21.15.148:0/551851122 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:08.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc937fe700 1 -- 172.21.15.148:0/551851122 >> v1:172.21.15.161:6789/0 conn(0x7fbc94157870 legacy=0x7fbc9415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc937fe700 1 -- 172.21.15.148:0/551851122 >> v1:172.21.15.148:6789/0 conn(0x7fbc9415d580 legacy=0x7fbc94158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc937fe700 1 -- 172.21.15.148:0/551851122 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9415c280 con 0x7fbc9415a620 2023-10-10T20:30:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc92ffd700 1 -- 172.21.15.148:0/551851122 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417080880 0 0) 0x7fbc9415cbb0 con 0x7fbc9415d580 2023-10-10T20:30:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc937fe700 1 --2- 172.21.15.148:0/551851122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415a620 0x7fbc9415d040 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbc880098e0 tx=0x7fbc880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de2db08d9aaa490 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc92ffd700 1 -- 172.21.15.148:0/551851122 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88009e80 con 0x7fbc9415a620 2023-10-10T20:30:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc92ffd700 1 -- 172.21.15.148:0/551851122 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800b860 con 0x7fbc9415a620 2023-10-10T20:30:08.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.043+0000 7fbc92ffd700 1 -- 172.21.15.148:0/551851122 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800bbe0 con 0x7fbc9415a620 2023-10-10T20:30:08.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 -- 172.21.15.148:0/551851122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415a620 msgr2=0x7fbc9415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/551851122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415a620 0x7fbc9415d040 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbc880098e0 tx=0x7fbc880091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 -- 172.21.15.148:0/551851122 shutdown_connections 2023-10-10T20:30:08.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/551851122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415a620 0x7fbc9415d040 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 -- 172.21.15.148:0/551851122 >> 172.21.15.148:0/551851122 conn(0x7fbc94068ce0 msgr2=0x7fbc940690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 -- 172.21.15.148:0/551851122 shutdown_connections 2023-10-10T20:30:08.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.044+0000 7fbc9a94a700 1 -- 172.21.15.148:0/551851122 wait complete. 2023-10-10T20:30:08.051 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.045+0000 7fbc9a94a700 1 Processor -- start 2023-10-10T20:30:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.045+0000 7fbc9a94a700 1 -- start start 2023-10-10T20:30:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc94157870 0x7fbc9416b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 0x7fbc9413d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9416b6e0 con 0x7fbc9415d580 2023-10-10T20:30:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9413dd10 con 0x7fbc9415a620 2023-10-10T20:30:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc94152f30 con 0x7fbc94157870 2023-10-10T20:30:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33410/0 (socket says 172.21.15.148:33410) 2023-10-10T20:30:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 -- 172.21.15.148:0/2291171295 learned_addr learned my addr 172.21.15.148:0/2291171295 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc937fe700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 0x7fbc9413d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc93fff700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc94157870 0x7fbc9416b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 -- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc94157870 msgr2=0x7fbc9416b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc94157870 0x7fbc9416b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 -- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 msgr2=0x7fbc9413d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 0x7fbc9413d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 -- 172.21.15.148:0/2291171295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88009420 con 0x7fbc9415d580 2023-10-10T20:30:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc937fe700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 0x7fbc9413d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc98ee7700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cbf0 tx=0x7fbc8c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00a980 con 0x7fbc9415d580 2023-10-10T20:30:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c00aae0 con 0x7fbc9415d580 2023-10-10T20:30:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc941531d0 con 0x7fbc9415d580 2023-10-10T20:30:08.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00aea0 con 0x7fbc9415d580 2023-10-10T20:30:08.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.046+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc94153a70 con 0x7fbc9415d580 2023-10-10T20:30:08.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.047+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9404c3c0 con 0x7fbc9415d580 2023-10-10T20:30:08.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.048+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc8c007300 con 0x7fbc9415d580 2023-10-10T20:30:08.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.048+0000 7fbc7bfff700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 0x7fbc800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.048+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc8c0848f0 con 0x7fbc9415d580 2023-10-10T20:30:08.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.048+0000 7fbc93fff700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 0x7fbc800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.048+0000 7fbc93fff700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 0x7fbc800693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbc940d1060 tx=0x7fbc84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.052+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc8c051be0 con 0x7fbc9415d580 2023-10-10T20:30:08.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.273+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc9415c560 con 0x7fbc9415d580 2023-10-10T20:30:08.276 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.274+0000 7fbc7bfff700 1 -- 172.21.15.148:0/2291171295 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fbc8c055200 con 0x7fbc9415d580 2023-10-10T20:30:08.277 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:08.277 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:08.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 msgr2=0x7fbc800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.279 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 0x7fbc800693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbc940d1060 tx=0x7fbc84009040 comp rx=0 tx=0).stop 2023-10-10T20:30:08.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 msgr2=0x7fbc941529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cbf0 tx=0x7fbc8c001980 comp rx=0 tx=0).stop 2023-10-10T20:30:08.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 shutdown_connections 2023-10-10T20:30:08.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc80066f30 0x7fbc800693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc9415d580 0x7fbc941529f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc94157870 0x7fbc9416b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 --2- 172.21.15.148:0/2291171295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc9415a620 0x7fbc9413d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 >> 172.21.15.148:0/2291171295 conn(0x7fbc94068ce0 msgr2=0x7fbc940b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 shutdown_connections 2023-10-10T20:30:08.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.276+0000 7fbc9a94a700 1 -- 172.21.15.148:0/2291171295 wait complete. 2023-10-10T20:30:08.282 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:08.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 Processor -- start 2023-10-10T20:30:08.407 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 -- start start 2023-10-10T20:30:08.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62dc15a620 0x7f62dc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62dc158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc15cbb0 con 0x7f62dc157870 2023-10-10T20:30:08.413 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc15bdb0 con 0x7f62dc15a620 2023-10-10T20:30:08.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62e0c68700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15bfa0 con 0x7f62dc15d580 2023-10-10T20:30:08.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62da59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f62dc157870 0x7f62dc15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57396/0 (socket says 172.21.15.148:57396) 2023-10-10T20:30:08.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62da59c700 1 -- 172.21.15.148:0/837685896 learned_addr learned my addr 172.21.15.148:0/837685896 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:08.414 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62d1d9b700 1 --2- 172.21.15.148:0/837685896 >> v2:172.21.15.161:3300/0 conn(0x7f62dc15a620 0x7f62dc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62dad9d700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62dc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762364465 0 0) 0x7f62dc15cbb0 con 0x7f62dc157870 2023-10-10T20:30:08.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.405+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c8003570 con 0x7f62dc157870 2023-10-10T20:30:08.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2467136706 0 0) 0x7f62c8003570 con 0x7f62dc157870 2023-10-10T20:30:08.415 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 msgr2=0x7f62dc158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62dc158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 >> v2:172.21.15.161:3300/0 conn(0x7f62dc15a620 msgr2=0x7f62dc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 --2- 172.21.15.148:0/837685896 >> v2:172.21.15.161:3300/0 conn(0x7f62dc15a620 0x7f62dc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc15c280 con 0x7f62dc157870 2023-10-10T20:30:08.416 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f62cc001470 con 0x7f62dc157870 2023-10-10T20:30:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d1d9b700 1 --2- 172.21.15.148:0/837685896 >> v2:172.21.15.161:3300/0 conn(0x7f62dc15a620 0x7f62dc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 >> v1:172.21.15.148:6789/0 conn(0x7f62dc157870 legacy=0x7f62dc15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c8003a60 0x7f62c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8004d50 0x7f62c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15bfa0 con 0x7f62c8003a60 2023-10-10T20:30:08.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c800ba60 con 0x7f62c8004d50 2023-10-10T20:30:08.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c800bba0 con 0x7f62dc15d580 2023-10-10T20:30:08.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c8003a60 0x7f62c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.418 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d1d9b700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62dad9d700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8004d50 0x7f62c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 -- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 msgr2=0x7f62c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15d580 0x7f62c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 -- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8004d50 msgr2=0x7f62c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8004d50 0x7f62c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 -- 172.21.15.148:0/837685896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8003570 con 0x7f62c8003a60 2023-10-10T20:30:08.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62da59c700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c8003a60 0x7f62c8003e70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c3c0 tx=0x7f62cc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9216877fc20555d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.406+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00cc30 con 0x7f62c8003a60 2023-10-10T20:30:08.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc00cdb0 con 0x7f62c8003a60 2023-10-10T20:30:08.420 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62d9d9b700 1 -- 172.21.15.148:0/837685896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc005b70 con 0x7f62c8003a60 2023-10-10T20:30:08.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 -- 172.21.15.148:0/837685896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c8003a60 msgr2=0x7f62c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 --2- 172.21.15.148:0/837685896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c8003a60 0x7f62c8003e70 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c3c0 tx=0x7f62cc001e90 comp rx=0 tx=0).stop 2023-10-10T20:30:08.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62dad9d700 1 -- 172.21.15.148:0/837685896 reap_dead start 2023-10-10T20:30:08.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 -- 172.21.15.148:0/837685896 shutdown_connections 2023-10-10T20:30:08.421 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 -- 172.21.15.148:0/837685896 >> 172.21.15.148:0/837685896 conn(0x7f62dc068ce0 msgr2=0x7f62dc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 -- 172.21.15.148:0/837685896 shutdown_connections 2023-10-10T20:30:08.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.407+0000 7f62e0c68700 1 -- 172.21.15.148:0/837685896 wait complete. 2023-10-10T20:30:08.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.408+0000 7f62e0c68700 1 Processor -- start 2023-10-10T20:30:08.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.408+0000 7f62e0c68700 1 -- start start 2023-10-10T20:30:08.422 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8003a60 0x7f62dc14b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 0x7f62dc15f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0b0fc0 con 0x7f62dc0b0610 2023-10-10T20:30:08.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15ea80 con 0x7f62c8003a60 2023-10-10T20:30:08.423 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62e0c68700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15de70 con 0x7f62dc15ef10 2023-10-10T20:30:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33420/0 (socket says 172.21.15.148:33420) 2023-10-10T20:30:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 -- 172.21.15.148:0/1694926972 learned_addr learned my addr 172.21.15.148:0/1694926972 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62da59c700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8003a60 0x7f62dc14b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62d1d9b700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 0x7f62dc15f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 -- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 msgr2=0x7f62dc15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 0x7f62dc15f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 -- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8003a60 msgr2=0x7f62dc14b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8003a60 0x7f62dc14b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.425 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62dad9d700 1 -- 172.21.15.148:0/1694926972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc15e150 con 0x7f62dc0b0610 2023-10-10T20:30:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.409+0000 7f62d1d9b700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 0x7f62dc15f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62dad9d700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f62dc15d9e0 tx=0x7f62d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400e040 con 0x7f62dc0b0610 2023-10-10T20:30:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d400f040 con 0x7f62dc0b0610 2023-10-10T20:30:08.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc15e430 con 0x7f62dc0b0610 2023-10-10T20:30:08.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400fe50 con 0x7f62dc0b0610 2023-10-10T20:30:08.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.410+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc14bd00 con 0x7f62dc0b0610 2023-10-10T20:30:08.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.411+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc15ec10 con 0x7f62dc0b0610 2023-10-10T20:30:08.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.411+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62d4015640 con 0x7f62dc0b0610 2023-10-10T20:30:08.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.411+0000 7f62d2ffd700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 0x7f62bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.411+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62d4084d40 con 0x7f62dc0b0610 2023-10-10T20:30:08.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.411+0000 7f62da59c700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 0x7f62bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.412+0000 7f62da59c700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 0x7f62bc0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001ed0 tx=0x7f62cc015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.429 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.417+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62d4051fb0 con 0x7f62dc0b0610 2023-10-10T20:30:08.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.598+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62dc04c3c0 con 0x7f62dc0b0610 2023-10-10T20:30:08.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.599+0000 7f62d2ffd700 1 -- 172.21.15.148:0/1694926972 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+15413 (secure 0 0 0) 0x7f62d40555d0 con 0x7f62dc0b0610 2023-10-10T20:30:08.601 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:08.601 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":178,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:05.880134+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 msgr2=0x7f62bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 0x7f62bc0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001ed0 tx=0x7f62cc015000 comp rx=0 tx=0).stop 2023-10-10T20:30:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 msgr2=0x7f62dc0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f62dc15d9e0 tx=0x7f62d400b040 comp rx=0 tx=0).stop 2023-10-10T20:30:08.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 shutdown_connections 2023-10-10T20:30:08.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f62bc066f30 0x7f62bc0693e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62dc0b0610 0x7f62dc0b0a80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c8003a60 0x7f62dc14b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 --2- 172.21.15.148:0/1694926972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f62dc15ef10 0x7f62dc15f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 >> 172.21.15.148:0/1694926972 conn(0x7f62dc068ce0 msgr2=0x7f62dc155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 shutdown_connections 2023-10-10T20:30:08.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.601+0000 7f62e0c68700 1 -- 172.21.15.148:0/1694926972 wait complete. 2023-10-10T20:30:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-10T20:30:08.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:08.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.729+0000 7f6893a9a700 1 Processor -- start 2023-10-10T20:30:08.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.729+0000 7f6893a9a700 1 -- start start 2023-10-10T20:30:08.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6893a9a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f688c0cc3c0 0x7f688c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6893a9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c0aff60 0x7f688c163c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6893a9a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0b0580 con 0x7f688c06a9f0 2023-10-10T20:30:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6893a9a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c0b03a0 con 0x7f688c0cc3c0 2023-10-10T20:30:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6893a9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c06ae90 con 0x7f688c0aff60 2023-10-10T20:30:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891035700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f688c06a9f0 0x7f688c0af850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57406/0 (socket says 172.21.15.148:57406) 2023-10-10T20:30:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891035700 1 -- 172.21.15.148:0/3688853937 learned_addr learned my addr 172.21.15.148:0/3688853937 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891836700 1 --2- 172.21.15.148:0/3688853937 >> v2:172.21.15.161:3300/0 conn(0x7f688c0cc3c0 0x7f688c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6892037700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c0aff60 0x7f688c163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269093378 0 0) 0x7f688c0b0580 con 0x7f688c06a9f0 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003570 con 0x7f688c06a9f0 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 667351493 0 0) 0x7f6878003570 con 0x7f688c06a9f0 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891836700 1 -- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c0aff60 msgr2=0x7f688c163c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:08.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891836700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c0aff60 0x7f688c163c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891836700 1 -- 172.21.15.148:0/3688853937 >> v1:172.21.15.148:6789/0 conn(0x7f688c06a9f0 legacy=0x7f688c0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.730+0000 7f6891836700 1 -- 172.21.15.148:0/3688853937 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c15b0c0 con 0x7f688c0cc3c0 2023-10-10T20:30:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6891836700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c06a4b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f68880098e0 tx=0x7f68880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d45d52372428f81 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c001470 con 0x7f688c0cc3c0 2023-10-10T20:30:08.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888011420 con 0x7f688c0cc3c0 2023-10-10T20:30:08.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6890834700 1 -- 172.21.15.148:0/3688853937 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880117c0 con 0x7f688c0cc3c0 2023-10-10T20:30:08.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 -- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 msgr2=0x7f688c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c06a4b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f68880098e0 tx=0x7f68880091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 -- 172.21.15.148:0/3688853937 shutdown_connections 2023-10-10T20:30:08.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c0aff60 0x7f688c163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 --2- 172.21.15.148:0/3688853937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c06a4b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 -- 172.21.15.148:0/3688853937 >> 172.21.15.148:0/3688853937 conn(0x7f688c069800 msgr2=0x7f688c0d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 -- 172.21.15.148:0/3688853937 shutdown_connections 2023-10-10T20:30:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.731+0000 7f6893a9a700 1 -- 172.21.15.148:0/3688853937 wait complete. 2023-10-10T20:30:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.732+0000 7f6893a9a700 1 Processor -- start 2023-10-10T20:30:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.732+0000 7f6893a9a700 1 -- start start 2023-10-10T20:30:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 0x7f688c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0b5980 con 0x7f688c0aff60 2023-10-10T20:30:08.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c14de20 con 0x7f688c0cc3c0 2023-10-10T20:30:08.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6893a9a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0b5c20 con 0x7f688c06a9f0 2023-10-10T20:30:08.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6891035700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6891035700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33436/0 (socket says 172.21.15.148:33436) 2023-10-10T20:30:08.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6891035700 1 -- 172.21.15.148:0/661563577 learned_addr learned my addr 172.21.15.148:0/661563577 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:08.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.733+0000 7f6892037700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891836700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 0x7f688c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.743 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 -- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 msgr2=0x7f688c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 0x7f688c0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 -- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 msgr2=0x7f688c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 -- 172.21.15.148:0/661563577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009420 con 0x7f688c0aff60 2023-10-10T20:30:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891836700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 0x7f688c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6891035700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f687c00cb90 tx=0x7f687c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c007360 con 0x7f688c0aff60 2023-10-10T20:30:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c0074e0 con 0x7f688c0aff60 2023-10-10T20:30:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c0b5f00 con 0x7f688c0aff60 2023-10-10T20:30:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c004830 con 0x7f688c0aff60 2023-10-10T20:30:08.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.734+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c0b64c0 con 0x7f688c0aff60 2023-10-10T20:30:08.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.735+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c089110 con 0x7f688c0aff60 2023-10-10T20:30:08.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.735+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f687c004ba0 con 0x7f688c0aff60 2023-10-10T20:30:08.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.736+0000 7f6881ffb700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 0x7f68700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:08.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.736+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f687c083e90 con 0x7f688c0aff60 2023-10-10T20:30:08.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.736+0000 7f6891836700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 0x7f68700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:08.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.736+0000 7f6891836700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 0x7f68700693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f68880091e0 tx=0x7f6888016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:08.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.740+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f687c051180 con 0x7f688c0aff60 2023-10-10T20:30:08.920 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.919+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f688c14f560 con 0x7f6870066f30 2023-10-10T20:30:08.928 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.927+0000 7f6881ffb700 1 -- 172.21.15.148:0/661563577 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285888 (secure 0 0 0) 0x7f688c14f560 con 0x7f6870066f30 2023-10-10T20:30:08.931 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:08.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 msgr2=0x7f68700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 0x7f68700693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f68880091e0 tx=0x7f6888016000 comp rx=0 tx=0).stop 2023-10-10T20:30:08.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 msgr2=0x7f688c0b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:08.934 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f687c00cb90 tx=0x7f687c001b50 comp rx=0 tx=0).stop 2023-10-10T20:30:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 shutdown_connections 2023-10-10T20:30:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6870066f30 0x7f68700693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f688c0cc3c0 0x7f688c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f688c06a9f0 0x7f688c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 --2- 172.21.15.148:0/661563577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f688c0aff60 0x7f688c0b5440 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 >> 172.21.15.148:0/661563577 conn(0x7f688c069800 msgr2=0x7f688c0c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 shutdown_connections 2023-10-10T20:30:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:08.930+0000 7f6893a9a700 1 -- 172.21.15.148:0/661563577 wait complete. 2023-10-10T20:30:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:08.943 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2023-10-10T20:30:07.468633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553608,"statfs":{"total":1600000000,"available":1590895554,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.595195"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902028+0000","last_change":"2023-10-10T20:30:04.900734+0000","last_active":"2023-10-10T20:30:04.902028+0000","last_peered":"2023-10-10T20:30:04.902028+0000","last_clean":"2023-10-10T20:30:04.902028+0000","last_became_active":"2023-10-10T20:30:04.900371+0000","last_became_peered":"2023-10-10T20:30:04.900371+0000","last_unstale":"2023-10-10T20:30:04.902028+0000","last_undegraded":"2023-10-10T20:30:04.902028+0000","last_fullsized":"2023-10-10T20:30:04.902028+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:38.903594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892166+0000","last_change":"2023-10-10T20:30:04.891339+0000","last_active":"2023-10-10T20:30:04.892166+0000","last_peered":"2023-10-10T20:30:04.892166+0000","last_clean":"2023-10-10T20:30:04.892166+0000","last_became_active":"2023-10-10T20:30:04.890531+0000","last_became_peered":"2023-10-10T20:30:04.890531+0000","last_unstale":"2023-10-10T20:30:04.892166+0000","last_undegraded":"2023-10-10T20:30:04.892166+0000","last_fullsized":"2023-10-10T20:30:04.892166+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:28.533521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902574+0000","last_change":"2023-10-10T20:30:04.902375+0000","last_active":"2023-10-10T20:30:04.902574+0000","last_peered":"2023-10-10T20:30:04.902574+0000","last_clean":"2023-10-10T20:30:04.902574+0000","last_became_active":"2023-10-10T20:30:04.901572+0000","last_became_peered":"2023-10-10T20:30:04.901572+0000","last_unstale":"2023-10-10T20:30:04.902574+0000","last_undegraded":"2023-10-10T20:30:04.902574+0000","last_fullsized":"2023-10-10T20:30:04.902574+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:03.005907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889520+0000","last_change":"2023-10-10T20:30:04.889394+0000","last_active":"2023-10-10T20:30:04.889520+0000","last_peered":"2023-10-10T20:30:04.889520+0000","last_clean":"2023-10-10T20:30:04.889520+0000","last_became_active":"2023-10-10T20:30:04.888798+0000","last_became_peered":"2023-10-10T20:30:04.888798+0000","last_unstale":"2023-10-10T20:30:04.889520+0000","last_undegraded":"2023-10-10T20:30:04.889520+0000","last_fullsized":"2023-10-10T20:30:04.889520+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894846+0000","last_change":"2023-10-10T20:30:04.894434+0000","last_active":"2023-10-10T20:30:04.894846+0000","last_peered":"2023-10-10T20:30:04.894846+0000","last_clean":"2023-10-10T20:30:04.894846+0000","last_became_active":"2023-10-10T20:30:04.894189+0000","last_became_peered":"2023-10-10T20:30:04.894189+0000","last_unstale":"2023-10-10T20:30:04.894846+0000","last_undegraded":"2023-10-10T20:30:04.894846+0000","last_fullsized":"2023-10-10T20:30:04.894846+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894045+0000","last_change":"2023-10-10T20:30:04.894045+0000","last_active":"2023-10-10T20:30:04.894045+0000","last_peered":"2023-10-10T20:30:04.894045+0000","last_clean":"2023-10-10T20:30:04.894045+0000","last_became_active":"2023-10-10T20:30:04.892826+0000","last_became_peered":"2023-10-10T20:30:04.892826+0000","last_unstale":"2023-10-10T20:30:04.894045+0000","last_undegraded":"2023-10-10T20:30:04.894045+0000","last_fullsized":"2023-10-10T20:30:04.894045+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:04.746102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892625+0000","last_change":"2023-10-10T20:30:04.891826+0000","last_active":"2023-10-10T20:30:04.892625+0000","last_peered":"2023-10-10T20:30:04.892625+0000","last_clean":"2023-10-10T20:30:04.892625+0000","last_became_active":"2023-10-10T20:30:04.891123+0000","last_became_peered":"2023-10-10T20:30:04.891123+0000","last_unstale":"2023-10-10T20:30:04.892625+0000","last_undegraded":"2023-10-10T20:30:04.892625+0000","last_fullsized":"2023-10-10T20:30:04.892625+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:08:05.484701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891086+0000","last_change":"2023-10-10T20:30:04.890927+0000","last_active":"2023-10-10T20:30:04.891086+0000","last_peered":"2023-10-10T20:30:04.891086+0000","last_clean":"2023-10-10T20:30:04.891086+0000","last_became_active":"2023-10-10T20:30:04.890655+0000","last_became_peered":"2023-10-10T20:30:04.890655+0000","last_unstale":"2023-10-10T20:30:04.891086+0000","last_undegraded":"2023-10-10T20:30:04.891086+0000","last_fullsized":"2023-10-10T20:30:04.891086+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:43.975389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892183+0000","last_change":"2023-10-10T20:30:04.891897+0000","last_active":"2023-10-10T20:30:04.892183+0000","last_peered":"2023-10-10T20:30:04.892183+0000","last_clean":"2023-10-10T20:30:04.892183+0000","last_became_active":"2023-10-10T20:30:04.891309+0000","last_became_peered":"2023-10-10T20:30:04.891309+0000","last_unstale":"2023-10-10T20:30:04.892183+0000","last_undegraded":"2023-10-10T20:30:04.892183+0000","last_fullsized":"2023-10-10T20:30:04.892183+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:59.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892363+0000","last_change":"2023-10-10T20:30:04.892243+0000","last_active":"2023-10-10T20:30:04.892363+0000","last_peered":"2023-10-10T20:30:04.892363+0000","last_clean":"2023-10-10T20:30:04.892363+0000","last_became_active":"2023-10-10T20:30:04.890671+0000","last_became_peered":"2023-10-10T20:30:04.890671+0000","last_unstale":"2023-10-10T20:30:04.892363+0000","last_undegraded":"2023-10-10T20:30:04.892363+0000","last_fullsized":"2023-10-10T20:30:04.892363+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:21.294621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894864+0000","last_change":"2023-10-10T20:30:04.894754+0000","last_active":"2023-10-10T20:30:04.894864+0000","last_peered":"2023-10-10T20:30:04.894864+0000","last_clean":"2023-10-10T20:30:04.894864+0000","last_became_active":"2023-10-10T20:30:04.894466+0000","last_became_peered":"2023-10-10T20:30:04.894466+0000","last_unstale":"2023-10-10T20:30:04.894864+0000","last_undegraded":"2023-10-10T20:30:04.894864+0000","last_fullsized":"2023-10-10T20:30:04.894864+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:49.395189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894158+0000","last_change":"2023-10-10T20:30:04.893319+0000","last_active":"2023-10-10T20:30:04.894158+0000","last_peered":"2023-10-10T20:30:04.894158+0000","last_clean":"2023-10-10T20:30:04.894158+0000","last_became_active":"2023-10-10T20:30:04.891550+0000","last_became_peered":"2023-10-10T20:30:04.891550+0000","last_unstale":"2023-10-10T20:30:04.894158+0000","last_undegraded":"2023-10-10T20:30:04.894158+0000","last_fullsized":"2023-10-10T20:30:04.894158+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:36.597979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902728+0000","last_change":"2023-10-10T20:30:04.902657+0000","last_active":"2023-10-10T20:30:04.902728+0000","last_peered":"2023-10-10T20:30:04.902728+0000","last_clean":"2023-10-10T20:30:04.902728+0000","last_became_active":"2023-10-10T20:30:04.901862+0000","last_became_peered":"2023-10-10T20:30:04.901862+0000","last_unstale":"2023-10-10T20:30:04.902728+0000","last_undegraded":"2023-10-10T20:30:04.902728+0000","last_fullsized":"2023-10-10T20:30:04.902728+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:08:20.582951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902368+0000","last_change":"2023-10-10T20:30:04.900458+0000","last_active":"2023-10-10T20:30:04.902368+0000","last_peered":"2023-10-10T20:30:04.902368+0000","last_clean":"2023-10-10T20:30:04.902368+0000","last_became_active":"2023-10-10T20:30:04.900051+0000","last_became_peered":"2023-10-10T20:30:04.900051+0000","last_unstale":"2023-10-10T20:30:04.902368+0000","last_undegraded":"2023-10-10T20:30:04.902368+0000","last_fullsized":"2023-10-10T20:30:04.902368+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:08:20.934208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889820+0000","last_change":"2023-10-10T20:30:04.889753+0000","last_active":"2023-10-10T20:30:04.889820+0000","last_peered":"2023-10-10T20:30:04.889820+0000","last_clean":"2023-10-10T20:30:04.889820+0000","last_became_active":"2023-10-10T20:30:04.889523+0000","last_became_peered":"2023-10-10T20:30:04.889523+0000","last_unstale":"2023-10-10T20:30:04.889820+0000","last_undegraded":"2023-10-10T20:30:04.889820+0000","last_fullsized":"2023-10-10T20:30:04.889820+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892642+0000","last_change":"2023-10-10T20:30:04.891883+0000","last_active":"2023-10-10T20:30:04.892642+0000","last_peered":"2023-10-10T20:30:04.892642+0000","last_clean":"2023-10-10T20:30:04.892642+0000","last_became_active":"2023-10-10T20:30:04.891532+0000","last_became_peered":"2023-10-10T20:30:04.891532+0000","last_unstale":"2023-10-10T20:30:04.892642+0000","last_undegraded":"2023-10-10T20:30:04.892642+0000","last_fullsized":"2023-10-10T20:30:04.892642+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:21.574488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903203+0000","last_change":"2023-10-10T20:30:04.903203+0000","last_active":"2023-10-10T20:30:04.903203+0000","last_peered":"2023-10-10T20:30:04.903203+0000","last_clean":"2023-10-10T20:30:04.903203+0000","last_became_active":"2023-10-10T20:30:04.902976+0000","last_became_peered":"2023-10-10T20:30:04.902976+0000","last_unstale":"2023-10-10T20:30:04.903203+0000","last_undegraded":"2023-10-10T20:30:04.903203+0000","last_fullsized":"2023-10-10T20:30:04.903203+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:24.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898025+0000","last_change":"2023-10-10T20:30:04.898025+0000","last_active":"2023-10-10T20:30:04.898025+0000","last_peered":"2023-10-10T20:30:04.898025+0000","last_clean":"2023-10-10T20:30:04.898025+0000","last_became_active":"2023-10-10T20:30:04.897752+0000","last_became_peered":"2023-10-10T20:30:04.897752+0000","last_unstale":"2023-10-10T20:30:04.898025+0000","last_undegraded":"2023-10-10T20:30:04.898025+0000","last_fullsized":"2023-10-10T20:30:04.898025+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:58.402354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.884636+0000","last_change":"2023-10-10T20:30:04.884636+0000","last_active":"2023-10-10T20:30:04.884636+0000","last_peered":"2023-10-10T20:30:04.884636+0000","last_clean":"2023-10-10T20:30:04.884636+0000","last_became_active":"2023-10-10T20:30:04.884502+0000","last_became_peered":"2023-10-10T20:30:04.884502+0000","last_unstale":"2023-10-10T20:30:04.884636+0000","last_undegraded":"2023-10-10T20:30:04.884636+0000","last_fullsized":"2023-10-10T20:30:04.884636+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:48.166772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903632+0000","last_change":"2023-10-10T20:30:04.903548+0000","last_active":"2023-10-10T20:30:04.903632+0000","last_peered":"2023-10-10T20:30:04.903632+0000","last_clean":"2023-10-10T20:30:04.903632+0000","last_became_active":"2023-10-10T20:30:04.903256+0000","last_became_peered":"2023-10-10T20:30:04.903256+0000","last_unstale":"2023-10-10T20:30:04.903632+0000","last_undegraded":"2023-10-10T20:30:04.903632+0000","last_fullsized":"2023-10-10T20:30:04.903632+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:44:21.005942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892616+0000","last_change":"2023-10-10T20:30:04.892616+0000","last_active":"2023-10-10T20:30:04.892616+0000","last_peered":"2023-10-10T20:30:04.892616+0000","last_clean":"2023-10-10T20:30:04.892616+0000","last_became_active":"2023-10-10T20:30:04.892264+0000","last_became_peered":"2023-10-10T20:30:04.892264+0000","last_unstale":"2023-10-10T20:30:04.892616+0000","last_undegraded":"2023-10-10T20:30:04.892616+0000","last_fullsized":"2023-10-10T20:30:04.892616+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:29.170127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894178+0000","last_change":"2023-10-10T20:30:04.894011+0000","last_active":"2023-10-10T20:30:04.894178+0000","last_peered":"2023-10-10T20:30:04.894178+0000","last_clean":"2023-10-10T20:30:04.894178+0000","last_became_active":"2023-10-10T20:30:04.893699+0000","last_became_peered":"2023-10-10T20:30:04.893699+0000","last_unstale":"2023-10-10T20:30:04.894178+0000","last_undegraded":"2023-10-10T20:30:04.894178+0000","last_fullsized":"2023-10-10T20:30:04.894178+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:48.585573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893381+0000","last_change":"2023-10-10T20:30:04.893381+0000","last_active":"2023-10-10T20:30:04.893381+0000","last_peered":"2023-10-10T20:30:04.893381+0000","last_clean":"2023-10-10T20:30:04.893381+0000","last_became_active":"2023-10-10T20:30:04.893182+0000","last_became_peered":"2023-10-10T20:30:04.893182+0000","last_unstale":"2023-10-10T20:30:04.893381+0000","last_undegraded":"2023-10-10T20:30:04.893381+0000","last_fullsized":"2023-10-10T20:30:04.893381+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:09.455246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892197+0000","last_change":"2023-10-10T20:30:04.892060+0000","last_active":"2023-10-10T20:30:04.892197+0000","last_peered":"2023-10-10T20:30:04.892197+0000","last_clean":"2023-10-10T20:30:04.892197+0000","last_became_active":"2023-10-10T20:30:04.891648+0000","last_became_peered":"2023-10-10T20:30:04.891648+0000","last_unstale":"2023-10-10T20:30:04.892197+0000","last_undegraded":"2023-10-10T20:30:04.892197+0000","last_fullsized":"2023-10-10T20:30:04.892197+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:35.730188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902102+0000","last_change":"2023-10-10T20:30:04.898980+0000","last_active":"2023-10-10T20:30:04.902102+0000","last_peered":"2023-10-10T20:30:04.902102+0000","last_clean":"2023-10-10T20:30:04.902102+0000","last_became_active":"2023-10-10T20:30:04.898770+0000","last_became_peered":"2023-10-10T20:30:04.898770+0000","last_unstale":"2023-10-10T20:30:04.902102+0000","last_undegraded":"2023-10-10T20:30:04.902102+0000","last_fullsized":"2023-10-10T20:30:04.902102+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:16:14.375378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892659+0000","last_change":"2023-10-10T20:30:04.891956+0000","last_active":"2023-10-10T20:30:04.892659+0000","last_peered":"2023-10-10T20:30:04.892659+0000","last_clean":"2023-10-10T20:30:04.892659+0000","last_became_active":"2023-10-10T20:30:04.891713+0000","last_became_peered":"2023-10-10T20:30:04.891713+0000","last_unstale":"2023-10-10T20:30:04.892659+0000","last_undegraded":"2023-10-10T20:30:04.892659+0000","last_fullsized":"2023-10-10T20:30:04.892659+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:59.152754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903111+0000","last_change":"2023-10-10T20:30:04.903045+0000","last_active":"2023-10-10T20:30:04.903111+0000","last_peered":"2023-10-10T20:30:04.903111+0000","last_clean":"2023-10-10T20:30:04.903111+0000","last_became_active":"2023-10-10T20:30:04.902617+0000","last_became_peered":"2023-10-10T20:30:04.902617+0000","last_unstale":"2023-10-10T20:30:04.903111+0000","last_undegraded":"2023-10-10T20:30:04.903111+0000","last_fullsized":"2023-10-10T20:30:04.903111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:48.431851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900001+0000","last_change":"2023-10-10T20:30:04.899943+0000","last_active":"2023-10-10T20:30:04.900001+0000","last_peered":"2023-10-10T20:30:04.900001+0000","last_clean":"2023-10-10T20:30:04.900001+0000","last_became_active":"2023-10-10T20:30:04.899717+0000","last_became_peered":"2023-10-10T20:30:04.899717+0000","last_unstale":"2023-10-10T20:30:04.900001+0000","last_undegraded":"2023-10-10T20:30:04.900001+0000","last_fullsized":"2023-10-10T20:30:04.900001+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:48.585573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896995+0000","last_change":"2023-10-10T20:30:04.896375+0000","last_active":"2023-10-10T20:30:04.896995+0000","last_peered":"2023-10-10T20:30:04.896995+0000","last_clean":"2023-10-10T20:30:04.896995+0000","last_became_active":"2023-10-10T20:30:04.896156+0000","last_became_peered":"2023-10-10T20:30:04.896156+0000","last_unstale":"2023-10-10T20:30:04.896995+0000","last_undegraded":"2023-10-10T20:30:04.896995+0000","last_fullsized":"2023-10-10T20:30:04.896995+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:52.866253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886527+0000","last_change":"2023-10-10T20:30:04.886031+0000","last_active":"2023-10-10T20:30:04.886527+0000","last_peered":"2023-10-10T20:30:04.886527+0000","last_clean":"2023-10-10T20:30:04.886527+0000","last_became_active":"2023-10-10T20:30:04.885865+0000","last_became_peered":"2023-10-10T20:30:04.885865+0000","last_unstale":"2023-10-10T20:30:04.886527+0000","last_undegraded":"2023-10-10T20:30:04.886527+0000","last_fullsized":"2023-10-10T20:30:04.886527+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:33:58.541825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898067+0000","last_change":"2023-10-10T20:30:04.895587+0000","last_active":"2023-10-10T20:30:04.898067+0000","last_peered":"2023-10-10T20:30:04.898067+0000","last_clean":"2023-10-10T20:30:04.898067+0000","last_became_active":"2023-10-10T20:30:04.895139+0000","last_became_peered":"2023-10-10T20:30:04.895139+0000","last_unstale":"2023-10-10T20:30:04.898067+0000","last_undegraded":"2023-10-10T20:30:04.898067+0000","last_fullsized":"2023-10-10T20:30:04.898067+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899070+0000","last_change":"2023-10-10T20:30:04.899070+0000","last_active":"2023-10-10T20:30:04.899070+0000","last_peered":"2023-10-10T20:30:04.899070+0000","last_clean":"2023-10-10T20:30:04.899070+0000","last_became_active":"2023-10-10T20:30:04.898511+0000","last_became_peered":"2023-10-10T20:30:04.898511+0000","last_unstale":"2023-10-10T20:30:04.899070+0000","last_undegraded":"2023-10-10T20:30:04.899070+0000","last_fullsized":"2023-10-10T20:30:04.899070+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:08:05.484701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890782+0000","last_change":"2023-10-10T20:30:04.889965+0000","last_active":"2023-10-10T20:30:04.890782+0000","last_peered":"2023-10-10T20:30:04.890782+0000","last_clean":"2023-10-10T20:30:04.890782+0000","last_became_active":"2023-10-10T20:30:04.889714+0000","last_became_peered":"2023-10-10T20:30:04.889714+0000","last_unstale":"2023-10-10T20:30:04.890782+0000","last_undegraded":"2023-10-10T20:30:04.890782+0000","last_fullsized":"2023-10-10T20:30:04.890782+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903738+0000","last_change":"2023-10-10T20:30:03.889349+0000","last_active":"2023-10-10T20:30:04.903738+0000","last_peered":"2023-10-10T20:30:04.903738+0000","last_clean":"2023-10-10T20:30:04.903738+0000","last_became_active":"2023-10-10T20:30:03.889112+0000","last_became_peered":"2023-10-10T20:30:03.889112+0000","last_unstale":"2023-10-10T20:30:04.903738+0000","last_undegraded":"2023-10-10T20:30:04.903738+0000","last_fullsized":"2023-10-10T20:30:04.903738+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:05.003759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891053+0000","last_change":"2023-10-10T20:30:04.890977+0000","last_active":"2023-10-10T20:30:04.891053+0000","last_peered":"2023-10-10T20:30:04.891053+0000","last_clean":"2023-10-10T20:30:04.891053+0000","last_became_active":"2023-10-10T20:30:04.890714+0000","last_became_peered":"2023-10-10T20:30:04.890714+0000","last_unstale":"2023-10-10T20:30:04.891053+0000","last_undegraded":"2023-10-10T20:30:04.891053+0000","last_fullsized":"2023-10-10T20:30:04.891053+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:01:26.914899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897981+0000","last_change":"2023-10-10T20:30:03.892241+0000","last_active":"2023-10-10T20:30:04.897981+0000","last_peered":"2023-10-10T20:30:04.897981+0000","last_clean":"2023-10-10T20:30:04.897981+0000","last_became_active":"2023-10-10T20:30:03.891898+0000","last_became_peered":"2023-10-10T20:30:03.891898+0000","last_unstale":"2023-10-10T20:30:04.897981+0000","last_undegraded":"2023-10-10T20:30:04.897981+0000","last_fullsized":"2023-10-10T20:30:04.897981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:29:12.416336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892483+0000","last_change":"2023-10-10T20:30:04.892483+0000","last_active":"2023-10-10T20:30:04.892483+0000","last_peered":"2023-10-10T20:30:04.892483+0000","last_clean":"2023-10-10T20:30:04.892483+0000","last_became_active":"2023-10-10T20:30:04.891965+0000","last_became_peered":"2023-10-10T20:30:04.891965+0000","last_unstale":"2023-10-10T20:30:04.892483+0000","last_undegraded":"2023-10-10T20:30:04.892483+0000","last_fullsized":"2023-10-10T20:30:04.892483+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:25.470282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896349+0000","last_change":"2023-10-10T20:30:03.881890+0000","last_active":"2023-10-10T20:30:04.896349+0000","last_peered":"2023-10-10T20:30:04.896349+0000","last_clean":"2023-10-10T20:30:04.896349+0000","last_became_active":"2023-10-10T20:30:03.881664+0000","last_became_peered":"2023-10-10T20:30:03.881664+0000","last_unstale":"2023-10-10T20:30:04.896349+0000","last_undegraded":"2023-10-10T20:30:04.896349+0000","last_fullsized":"2023-10-10T20:30:04.896349+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:24.168089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902591+0000","last_change":"2023-10-10T20:30:04.902437+0000","last_active":"2023-10-10T20:30:04.902591+0000","last_peered":"2023-10-10T20:30:04.902591+0000","last_clean":"2023-10-10T20:30:04.902591+0000","last_became_active":"2023-10-10T20:30:04.902155+0000","last_became_peered":"2023-10-10T20:30:04.902155+0000","last_unstale":"2023-10-10T20:30:04.902591+0000","last_undegraded":"2023-10-10T20:30:04.902591+0000","last_fullsized":"2023-10-10T20:30:04.902591+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:29.170127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896712+0000","last_change":"2023-10-10T20:30:03.887746+0000","last_active":"2023-10-10T20:30:04.896712+0000","last_peered":"2023-10-10T20:30:04.896712+0000","last_clean":"2023-10-10T20:30:04.896712+0000","last_became_active":"2023-10-10T20:30:03.887519+0000","last_became_peered":"2023-10-10T20:30:03.887519+0000","last_unstale":"2023-10-10T20:30:04.896712+0000","last_undegraded":"2023-10-10T20:30:04.896712+0000","last_fullsized":"2023-10-10T20:30:04.896712+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:03.618789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891115+0000","last_change":"2023-10-10T20:30:04.891058+0000","last_active":"2023-10-10T20:30:04.891115+0000","last_peered":"2023-10-10T20:30:04.891115+0000","last_clean":"2023-10-10T20:30:04.891115+0000","last_became_active":"2023-10-10T20:30:04.890844+0000","last_became_peered":"2023-10-10T20:30:04.890844+0000","last_unstale":"2023-10-10T20:30:04.891115+0000","last_undegraded":"2023-10-10T20:30:04.891115+0000","last_fullsized":"2023-10-10T20:30:04.891115+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:28.525881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902390+0000","last_change":"2023-10-10T20:30:03.888216+0000","last_active":"2023-10-10T20:30:04.902390+0000","last_peered":"2023-10-10T20:30:04.902390+0000","last_clean":"2023-10-10T20:30:04.902390+0000","last_became_active":"2023-10-10T20:30:03.887646+0000","last_became_peered":"2023-10-10T20:30:03.887646+0000","last_unstale":"2023-10-10T20:30:04.902390+0000","last_undegraded":"2023-10-10T20:30:04.902390+0000","last_fullsized":"2023-10-10T20:30:04.902390+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:52.137385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893091+0000","last_change":"2023-10-10T20:30:04.892902+0000","last_active":"2023-10-10T20:30:04.893091+0000","last_peered":"2023-10-10T20:30:04.893091+0000","last_clean":"2023-10-10T20:30:04.893091+0000","last_became_active":"2023-10-10T20:30:04.892385+0000","last_became_peered":"2023-10-10T20:30:04.892385+0000","last_unstale":"2023-10-10T20:30:04.893091+0000","last_undegraded":"2023-10-10T20:30:04.893091+0000","last_fullsized":"2023-10-10T20:30:04.893091+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:35.730188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896036+0000","last_change":"2023-10-10T20:30:03.886607+0000","last_active":"2023-10-10T20:30:04.896036+0000","last_peered":"2023-10-10T20:30:04.896036+0000","last_clean":"2023-10-10T20:30:04.896036+0000","last_became_active":"2023-10-10T20:30:03.886237+0000","last_became_peered":"2023-10-10T20:30:03.886237+0000","last_unstale":"2023-10-10T20:30:04.896036+0000","last_undegraded":"2023-10-10T20:30:04.896036+0000","last_fullsized":"2023-10-10T20:30:04.896036+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:29:12.416336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892122+0000","last_change":"2023-10-10T20:30:04.892062+0000","last_active":"2023-10-10T20:30:04.892122+0000","last_peered":"2023-10-10T20:30:04.892122+0000","last_clean":"2023-10-10T20:30:04.892122+0000","last_became_active":"2023-10-10T20:30:04.891774+0000","last_became_peered":"2023-10-10T20:30:04.891774+0000","last_unstale":"2023-10-10T20:30:04.892122+0000","last_undegraded":"2023-10-10T20:30:04.892122+0000","last_fullsized":"2023-10-10T20:30:04.892122+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:59.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887097+0000","last_change":"2023-10-10T20:30:03.889405+0000","last_active":"2023-10-10T20:30:04.887097+0000","last_peered":"2023-10-10T20:30:04.887097+0000","last_clean":"2023-10-10T20:30:04.887097+0000","last_became_active":"2023-10-10T20:30:03.889009+0000","last_became_peered":"2023-10-10T20:30:03.889009+0000","last_unstale":"2023-10-10T20:30:04.887097+0000","last_undegraded":"2023-10-10T20:30:04.887097+0000","last_fullsized":"2023-10-10T20:30:04.887097+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891667+0000","last_change":"2023-10-10T20:30:04.891667+0000","last_active":"2023-10-10T20:30:04.891667+0000","last_peered":"2023-10-10T20:30:04.891667+0000","last_clean":"2023-10-10T20:30:04.891667+0000","last_became_active":"2023-10-10T20:30:04.891414+0000","last_became_peered":"2023-10-10T20:30:04.891414+0000","last_unstale":"2023-10-10T20:30:04.891667+0000","last_undegraded":"2023-10-10T20:30:04.891667+0000","last_fullsized":"2023-10-10T20:30:04.891667+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:33.937483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885855+0000","last_change":"2023-10-10T20:30:03.889990+0000","last_active":"2023-10-10T20:30:04.885855+0000","last_peered":"2023-10-10T20:30:04.885855+0000","last_clean":"2023-10-10T20:30:04.885855+0000","last_became_active":"2023-10-10T20:30:03.889779+0000","last_became_peered":"2023-10-10T20:30:03.889779+0000","last_unstale":"2023-10-10T20:30:04.885855+0000","last_undegraded":"2023-10-10T20:30:04.885855+0000","last_fullsized":"2023-10-10T20:30:04.885855+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:54:23.629817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894488+0000","last_change":"2023-10-10T20:30:04.894488+0000","last_active":"2023-10-10T20:30:04.894488+0000","last_peered":"2023-10-10T20:30:04.894488+0000","last_clean":"2023-10-10T20:30:04.894488+0000","last_became_active":"2023-10-10T20:30:04.890305+0000","last_became_peered":"2023-10-10T20:30:04.890305+0000","last_unstale":"2023-10-10T20:30:04.894488+0000","last_undegraded":"2023-10-10T20:30:04.894488+0000","last_fullsized":"2023-10-10T20:30:04.894488+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:59.152754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.888939+0000","last_change":"2023-10-10T20:30:03.888880+0000","last_active":"2023-10-10T20:30:03.888939+0000","last_peered":"2023-10-10T20:30:03.888939+0000","last_clean":"2023-10-10T20:30:03.888939+0000","last_became_active":"2023-10-10T20:30:03.888635+0000","last_became_peered":"2023-10-10T20:30:03.888635+0000","last_unstale":"2023-10-10T20:30:03.888939+0000","last_undegraded":"2023-10-10T20:30:03.888939+0000","last_fullsized":"2023-10-10T20:30:03.888939+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:51.498918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897313+0000","last_change":"2023-10-10T20:30:04.897313+0000","last_active":"2023-10-10T20:30:04.897313+0000","last_peered":"2023-10-10T20:30:04.897313+0000","last_clean":"2023-10-10T20:30:04.897313+0000","last_became_active":"2023-10-10T20:30:04.890219+0000","last_became_peered":"2023-10-10T20:30:04.890219+0000","last_unstale":"2023-10-10T20:30:04.897313+0000","last_undegraded":"2023-10-10T20:30:04.897313+0000","last_fullsized":"2023-10-10T20:30:04.897313+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:46.701566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"177'3","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900328+0000","last_change":"2023-10-10T20:30:03.888305+0000","last_active":"2023-10-10T20:30:04.900328+0000","last_peered":"2023-10-10T20:30:04.900328+0000","last_clean":"2023-10-10T20:30:04.900328+0000","last_became_active":"2023-10-10T20:30:03.888068+0000","last_became_peered":"2023-10-10T20:30:03.888068+0000","last_unstale":"2023-10-10T20:30:04.900328+0000","last_undegraded":"2023-10-10T20:30:04.900328+0000","last_fullsized":"2023-10-10T20:30:04.900328+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:11.071357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.901616+0000","last_change":"2023-10-10T20:30:04.901530+0000","last_active":"2023-10-10T20:30:04.901616+0000","last_peered":"2023-10-10T20:30:04.901616+0000","last_clean":"2023-10-10T20:30:04.901616+0000","last_became_active":"2023-10-10T20:30:04.901306+0000","last_became_peered":"2023-10-10T20:30:04.901306+0000","last_unstale":"2023-10-10T20:30:04.901616+0000","last_undegraded":"2023-10-10T20:30:04.901616+0000","last_fullsized":"2023-10-10T20:30:04.901616+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:10.205666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"177'6","reported_seq":16,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.904953+0000","last_change":"2023-10-10T20:30:03.890316+0000","last_active":"2023-10-10T20:30:04.904953+0000","last_peered":"2023-10-10T20:30:04.904953+0000","last_clean":"2023-10-10T20:30:04.904953+0000","last_became_active":"2023-10-10T20:30:03.890064+0000","last_became_peered":"2023-10-10T20:30:03.890064+0000","last_unstale":"2023-10-10T20:30:04.904953+0000","last_undegraded":"2023-10-10T20:30:04.904953+0000","last_fullsized":"2023-10-10T20:30:04.904953+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:05:38.322381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894631+0000","last_change":"2023-10-10T20:30:04.894631+0000","last_active":"2023-10-10T20:30:04.894631+0000","last_peered":"2023-10-10T20:30:04.894631+0000","last_clean":"2023-10-10T20:30:04.894631+0000","last_became_active":"2023-10-10T20:30:04.890606+0000","last_became_peered":"2023-10-10T20:30:04.890606+0000","last_unstale":"2023-10-10T20:30:04.894631+0000","last_undegraded":"2023-10-10T20:30:04.894631+0000","last_fullsized":"2023-10-10T20:30:04.894631+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:00.971201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898667+0000","last_change":"2023-10-10T20:30:03.883717+0000","last_active":"2023-10-10T20:30:04.898667+0000","last_peered":"2023-10-10T20:30:04.898667+0000","last_clean":"2023-10-10T20:30:04.898667+0000","last_became_active":"2023-10-10T20:30:03.883503+0000","last_became_peered":"2023-10-10T20:30:03.883503+0000","last_unstale":"2023-10-10T20:30:04.898667+0000","last_undegraded":"2023-10-10T20:30:04.898667+0000","last_fullsized":"2023-10-10T20:30:04.898667+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:14:23.295705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903439+0000","last_change":"2023-10-10T20:30:04.903386+0000","last_active":"2023-10-10T20:30:04.903439+0000","last_peered":"2023-10-10T20:30:04.903439+0000","last_clean":"2023-10-10T20:30:04.903439+0000","last_became_active":"2023-10-10T20:30:04.903186+0000","last_became_peered":"2023-10-10T20:30:04.903186+0000","last_unstale":"2023-10-10T20:30:04.903439+0000","last_undegraded":"2023-10-10T20:30:04.903439+0000","last_fullsized":"2023-10-10T20:30:04.903439+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:57.500136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.883931+0000","last_change":"2023-10-10T20:30:03.883809+0000","last_active":"2023-10-10T20:30:03.883931+0000","last_peered":"2023-10-10T20:30:03.883931+0000","last_clean":"2023-10-10T20:30:03.883931+0000","last_became_active":"2023-10-10T20:30:03.883595+0000","last_became_peered":"2023-10-10T20:30:03.883595+0000","last_unstale":"2023-10-10T20:30:03.883931+0000","last_undegraded":"2023-10-10T20:30:03.883931+0000","last_fullsized":"2023-10-10T20:30:03.883931+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:35.592670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903274+0000","last_change":"2023-10-10T20:30:04.903208+0000","last_active":"2023-10-10T20:30:04.903274+0000","last_peered":"2023-10-10T20:30:04.903274+0000","last_clean":"2023-10-10T20:30:04.903274+0000","last_became_active":"2023-10-10T20:30:04.902966+0000","last_became_peered":"2023-10-10T20:30:04.902966+0000","last_unstale":"2023-10-10T20:30:04.903274+0000","last_undegraded":"2023-10-10T20:30:04.903274+0000","last_fullsized":"2023-10-10T20:30:04.903274+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:21.294621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886556+0000","last_change":"2023-10-10T20:30:03.881756+0000","last_active":"2023-10-10T20:30:04.886556+0000","last_peered":"2023-10-10T20:30:04.886556+0000","last_clean":"2023-10-10T20:30:04.886556+0000","last_became_active":"2023-10-10T20:30:03.881565+0000","last_became_peered":"2023-10-10T20:30:03.881565+0000","last_unstale":"2023-10-10T20:30:04.886556+0000","last_undegraded":"2023-10-10T20:30:04.886556+0000","last_fullsized":"2023-10-10T20:30:04.886556+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:55.070436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903092+0000","last_change":"2023-10-10T20:30:04.902893+0000","last_active":"2023-10-10T20:30:04.903092+0000","last_peered":"2023-10-10T20:30:04.903092+0000","last_clean":"2023-10-10T20:30:04.903092+0000","last_became_active":"2023-10-10T20:30:04.902372+0000","last_became_peered":"2023-10-10T20:30:04.902372+0000","last_unstale":"2023-10-10T20:30:04.903092+0000","last_undegraded":"2023-10-10T20:30:04.903092+0000","last_fullsized":"2023-10-10T20:30:04.903092+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:31.428174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"177'8","reported_seq":18,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903004+0000","last_change":"2023-10-10T20:30:03.889705+0000","last_active":"2023-10-10T20:30:04.903004+0000","last_peered":"2023-10-10T20:30:04.903004+0000","last_clean":"2023-10-10T20:30:04.903004+0000","last_became_active":"2023-10-10T20:30:03.889433+0000","last_became_peered":"2023-10-10T20:30:03.889433+0000","last_unstale":"2023-10-10T20:30:04.903004+0000","last_undegraded":"2023-10-10T20:30:04.903004+0000","last_fullsized":"2023-10-10T20:30:04.903004+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898997+0000","last_change":"2023-10-10T20:30:04.898814+0000","last_active":"2023-10-10T20:30:04.898997+0000","last_peered":"2023-10-10T20:30:04.898997+0000","last_clean":"2023-10-10T20:30:04.898997+0000","last_became_active":"2023-10-10T20:30:04.898326+0000","last_became_peered":"2023-10-10T20:30:04.898326+0000","last_unstale":"2023-10-10T20:30:04.898997+0000","last_undegraded":"2023-10-10T20:30:04.898997+0000","last_fullsized":"2023-10-10T20:30:04.898997+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:28.533521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903421+0000","last_change":"2023-10-10T20:30:03.887443+0000","last_active":"2023-10-10T20:30:04.903421+0000","last_peered":"2023-10-10T20:30:04.903421+0000","last_clean":"2023-10-10T20:30:04.903421+0000","last_became_active":"2023-10-10T20:30:03.887135+0000","last_became_peered":"2023-10-10T20:30:03.887135+0000","last_unstale":"2023-10-10T20:30:04.903421+0000","last_undegraded":"2023-10-10T20:30:04.903421+0000","last_fullsized":"2023-10-10T20:30:04.903421+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:23.522665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897448+0000","last_change":"2023-10-10T20:30:04.897448+0000","last_active":"2023-10-10T20:30:04.897448+0000","last_peered":"2023-10-10T20:30:04.897448+0000","last_clean":"2023-10-10T20:30:04.897448+0000","last_became_active":"2023-10-10T20:30:04.890538+0000","last_became_peered":"2023-10-10T20:30:04.890538+0000","last_unstale":"2023-10-10T20:30:04.897448+0000","last_undegraded":"2023-10-10T20:30:04.897448+0000","last_fullsized":"2023-10-10T20:30:04.897448+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:21.574488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902266+0000","last_change":"2023-10-10T20:30:03.885612+0000","last_active":"2023-10-10T20:30:04.902266+0000","last_peered":"2023-10-10T20:30:04.902266+0000","last_clean":"2023-10-10T20:30:04.902266+0000","last_became_active":"2023-10-10T20:30:03.885317+0000","last_became_peered":"2023-10-10T20:30:03.885317+0000","last_unstale":"2023-10-10T20:30:04.902266+0000","last_undegraded":"2023-10-10T20:30:04.902266+0000","last_fullsized":"2023-10-10T20:30:04.902266+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:27.749116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893108+0000","last_change":"2023-10-10T20:30:04.893038+0000","last_active":"2023-10-10T20:30:04.893108+0000","last_peered":"2023-10-10T20:30:04.893108+0000","last_clean":"2023-10-10T20:30:04.893108+0000","last_became_active":"2023-10-10T20:30:04.892690+0000","last_became_peered":"2023-10-10T20:30:04.892690+0000","last_unstale":"2023-10-10T20:30:04.893108+0000","last_undegraded":"2023-10-10T20:30:04.893108+0000","last_fullsized":"2023-10-10T20:30:04.893108+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:36.597979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900591+0000","last_change":"2023-10-10T20:30:03.893140+0000","last_active":"2023-10-10T20:30:04.900591+0000","last_peered":"2023-10-10T20:30:04.900591+0000","last_clean":"2023-10-10T20:30:04.900591+0000","last_became_active":"2023-10-10T20:30:03.892640+0000","last_became_peered":"2023-10-10T20:30:03.892640+0000","last_unstale":"2023-10-10T20:30:04.900591+0000","last_undegraded":"2023-10-10T20:30:04.900591+0000","last_fullsized":"2023-10-10T20:30:04.900591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:30.782134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899227+0000","last_change":"2023-10-10T20:30:04.899159+0000","last_active":"2023-10-10T20:30:04.899227+0000","last_peered":"2023-10-10T20:30:04.899227+0000","last_clean":"2023-10-10T20:30:04.899227+0000","last_became_active":"2023-10-10T20:30:04.898928+0000","last_became_peered":"2023-10-10T20:30:04.898928+0000","last_unstale":"2023-10-10T20:30:04.899227+0000","last_undegraded":"2023-10-10T20:30:04.899227+0000","last_fullsized":"2023-10-10T20:30:04.899227+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:55.837552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885462+0000","last_change":"2023-10-10T20:30:03.886004+0000","last_active":"2023-10-10T20:30:04.885462+0000","last_peered":"2023-10-10T20:30:04.885462+0000","last_clean":"2023-10-10T20:30:04.885462+0000","last_became_active":"2023-10-10T20:30:03.885776+0000","last_became_peered":"2023-10-10T20:30:03.885776+0000","last_unstale":"2023-10-10T20:30:04.885462+0000","last_undegraded":"2023-10-10T20:30:04.885462+0000","last_fullsized":"2023-10-10T20:30:04.885462+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:02.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902340+0000","last_change":"2023-10-10T20:30:04.900286+0000","last_active":"2023-10-10T20:30:04.902340+0000","last_peered":"2023-10-10T20:30:04.902340+0000","last_clean":"2023-10-10T20:30:04.902340+0000","last_became_active":"2023-10-10T20:30:04.899716+0000","last_became_peered":"2023-10-10T20:30:04.899716+0000","last_unstale":"2023-10-10T20:30:04.902340+0000","last_undegraded":"2023-10-10T20:30:04.902340+0000","last_fullsized":"2023-10-10T20:30:04.902340+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:36.275993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896446+0000","last_change":"2023-10-10T20:30:03.892928+0000","last_active":"2023-10-10T20:30:04.896446+0000","last_peered":"2023-10-10T20:30:04.896446+0000","last_clean":"2023-10-10T20:30:04.896446+0000","last_became_active":"2023-10-10T20:30:03.892427+0000","last_became_peered":"2023-10-10T20:30:03.892427+0000","last_unstale":"2023-10-10T20:30:04.896446+0000","last_undegraded":"2023-10-10T20:30:04.896446+0000","last_fullsized":"2023-10-10T20:30:04.896446+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:05:38.322381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890848+0000","last_change":"2023-10-10T20:30:04.890616+0000","last_active":"2023-10-10T20:30:04.890848+0000","last_peered":"2023-10-10T20:30:04.890848+0000","last_clean":"2023-10-10T20:30:04.890848+0000","last_became_active":"2023-10-10T20:30:04.890372+0000","last_became_peered":"2023-10-10T20:30:04.890372+0000","last_unstale":"2023-10-10T20:30:04.890848+0000","last_undegraded":"2023-10-10T20:30:04.890848+0000","last_fullsized":"2023-10-10T20:30:04.890848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:38.903594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885258+0000","last_change":"2023-10-10T20:30:03.888491+0000","last_active":"2023-10-10T20:30:04.885258+0000","last_peered":"2023-10-10T20:30:04.885258+0000","last_clean":"2023-10-10T20:30:04.885258+0000","last_became_active":"2023-10-10T20:30:03.888265+0000","last_became_peered":"2023-10-10T20:30:03.888265+0000","last_unstale":"2023-10-10T20:30:04.885258+0000","last_undegraded":"2023-10-10T20:30:04.885258+0000","last_fullsized":"2023-10-10T20:30:04.885258+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:40:33.366728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889795+0000","last_change":"2023-10-10T20:30:04.889584+0000","last_active":"2023-10-10T20:30:04.889795+0000","last_peered":"2023-10-10T20:30:04.889795+0000","last_clean":"2023-10-10T20:30:04.889795+0000","last_became_active":"2023-10-10T20:30:04.888803+0000","last_became_peered":"2023-10-10T20:30:04.888803+0000","last_unstale":"2023-10-10T20:30:04.889795+0000","last_undegraded":"2023-10-10T20:30:04.889795+0000","last_fullsized":"2023-10-10T20:30:04.889795+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:58.402354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887770+0000","last_change":"2023-10-10T20:30:03.886221+0000","last_active":"2023-10-10T20:30:04.887770+0000","last_peered":"2023-10-10T20:30:04.887770+0000","last_clean":"2023-10-10T20:30:04.887770+0000","last_became_active":"2023-10-10T20:30:03.885993+0000","last_became_peered":"2023-10-10T20:30:03.885993+0000","last_unstale":"2023-10-10T20:30:04.887770+0000","last_undegraded":"2023-10-10T20:30:04.887770+0000","last_fullsized":"2023-10-10T20:30:04.887770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892175+0000","last_change":"2023-10-10T20:30:04.892175+0000","last_active":"2023-10-10T20:30:04.892175+0000","last_peered":"2023-10-10T20:30:04.892175+0000","last_clean":"2023-10-10T20:30:04.892175+0000","last_became_active":"2023-10-10T20:30:04.891786+0000","last_became_peered":"2023-10-10T20:30:04.891786+0000","last_unstale":"2023-10-10T20:30:04.892175+0000","last_undegraded":"2023-10-10T20:30:04.892175+0000","last_fullsized":"2023-10-10T20:30:04.892175+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:02.843513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885892+0000","last_change":"2023-10-10T20:30:03.893285+0000","last_active":"2023-10-10T20:30:04.885892+0000","last_peered":"2023-10-10T20:30:04.885892+0000","last_clean":"2023-10-10T20:30:04.885892+0000","last_became_active":"2023-10-10T20:30:03.892923+0000","last_became_peered":"2023-10-10T20:30:03.892923+0000","last_unstale":"2023-10-10T20:30:04.885892+0000","last_undegraded":"2023-10-10T20:30:04.885892+0000","last_fullsized":"2023-10-10T20:30:04.885892+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:46:58.385171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894999+0000","last_change":"2023-10-10T20:30:04.894999+0000","last_active":"2023-10-10T20:30:04.894999+0000","last_peered":"2023-10-10T20:30:04.894999+0000","last_clean":"2023-10-10T20:30:04.894999+0000","last_became_active":"2023-10-10T20:30:04.894106+0000","last_became_peered":"2023-10-10T20:30:04.894106+0000","last_unstale":"2023-10-10T20:30:04.894999+0000","last_undegraded":"2023-10-10T20:30:04.894999+0000","last_fullsized":"2023-10-10T20:30:04.894999+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"177'2","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903837+0000","last_change":"2023-10-10T20:30:03.893314+0000","last_active":"2023-10-10T20:30:04.903837+0000","last_peered":"2023-10-10T20:30:04.903837+0000","last_clean":"2023-10-10T20:30:04.903837+0000","last_became_active":"2023-10-10T20:30:03.892650+0000","last_became_peered":"2023-10-10T20:30:03.892650+0000","last_unstale":"2023-10-10T20:30:04.903837+0000","last_undegraded":"2023-10-10T20:30:04.903837+0000","last_fullsized":"2023-10-10T20:30:04.903837+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:47.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892675+0000","last_change":"2023-10-10T20:30:04.891965+0000","last_active":"2023-10-10T20:30:04.892675+0000","last_peered":"2023-10-10T20:30:04.892675+0000","last_clean":"2023-10-10T20:30:04.892675+0000","last_became_active":"2023-10-10T20:30:04.891436+0000","last_became_peered":"2023-10-10T20:30:04.891436+0000","last_unstale":"2023-10-10T20:30:04.892675+0000","last_undegraded":"2023-10-10T20:30:04.892675+0000","last_fullsized":"2023-10-10T20:30:04.892675+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:00.971201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897200+0000","last_change":"2023-10-10T20:30:03.887548+0000","last_active":"2023-10-10T20:30:04.897200+0000","last_peered":"2023-10-10T20:30:04.897200+0000","last_clean":"2023-10-10T20:30:04.897200+0000","last_became_active":"2023-10-10T20:30:03.886804+0000","last_became_peered":"2023-10-10T20:30:03.886804+0000","last_unstale":"2023-10-10T20:30:04.897200+0000","last_undegraded":"2023-10-10T20:30:04.897200+0000","last_fullsized":"2023-10-10T20:30:04.897200+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:06:48.718804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892032+0000","last_change":"2023-10-10T20:30:04.892032+0000","last_active":"2023-10-10T20:30:04.892032+0000","last_peered":"2023-10-10T20:30:04.892032+0000","last_clean":"2023-10-10T20:30:04.892032+0000","last_became_active":"2023-10-10T20:30:04.891506+0000","last_became_peered":"2023-10-10T20:30:04.891506+0000","last_unstale":"2023-10-10T20:30:04.892032+0000","last_undegraded":"2023-10-10T20:30:04.892032+0000","last_fullsized":"2023-10-10T20:30:04.892032+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:55:25.955509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887222+0000","last_change":"2023-10-10T20:30:03.889244+0000","last_active":"2023-10-10T20:30:04.887222+0000","last_peered":"2023-10-10T20:30:04.887222+0000","last_clean":"2023-10-10T20:30:04.887222+0000","last_became_active":"2023-10-10T20:30:03.888691+0000","last_became_peered":"2023-10-10T20:30:03.888691+0000","last_unstale":"2023-10-10T20:30:04.887222+0000","last_undegraded":"2023-10-10T20:30:04.887222+0000","last_fullsized":"2023-10-10T20:30:04.887222+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898933+0000","last_change":"2023-10-10T20:30:04.898933+0000","last_active":"2023-10-10T20:30:04.898933+0000","last_peered":"2023-10-10T20:30:04.898933+0000","last_clean":"2023-10-10T20:30:04.898933+0000","last_became_active":"2023-10-10T20:30:04.897632+0000","last_became_peered":"2023-10-10T20:30:04.897632+0000","last_unstale":"2023-10-10T20:30:04.898933+0000","last_undegraded":"2023-10-10T20:30:04.898933+0000","last_fullsized":"2023-10-10T20:30:04.898933+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:18.390010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.890659+0000","last_change":"2023-10-10T20:30:03.890592+0000","last_active":"2023-10-10T20:30:03.890659+0000","last_peered":"2023-10-10T20:30:03.890659+0000","last_clean":"2023-10-10T20:30:03.890659+0000","last_became_active":"2023-10-10T20:30:03.890366+0000","last_became_peered":"2023-10-10T20:30:03.890366+0000","last_unstale":"2023-10-10T20:30:03.890659+0000","last_undegraded":"2023-10-10T20:30:03.890659+0000","last_fullsized":"2023-10-10T20:30:03.890659+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:27.528211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899254+0000","last_change":"2023-10-10T20:30:04.899254+0000","last_active":"2023-10-10T20:30:04.899254+0000","last_peered":"2023-10-10T20:30:04.899254+0000","last_clean":"2023-10-10T20:30:04.899254+0000","last_became_active":"2023-10-10T20:30:04.898807+0000","last_became_peered":"2023-10-10T20:30:04.898807+0000","last_unstale":"2023-10-10T20:30:04.899254+0000","last_undegraded":"2023-10-10T20:30:04.899254+0000","last_fullsized":"2023-10-10T20:30:04.899254+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:22.948362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.883035+0000","last_change":"2023-10-10T20:30:03.894106+0000","last_active":"2023-10-10T20:30:04.883035+0000","last_peered":"2023-10-10T20:30:04.883035+0000","last_clean":"2023-10-10T20:30:04.883035+0000","last_became_active":"2023-10-10T20:30:03.893446+0000","last_became_peered":"2023-10-10T20:30:03.893446+0000","last_unstale":"2023-10-10T20:30:04.883035+0000","last_undegraded":"2023-10-10T20:30:04.883035+0000","last_fullsized":"2023-10-10T20:30:04.883035+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:21.867919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898046+0000","last_change":"2023-10-10T20:30:04.895449+0000","last_active":"2023-10-10T20:30:04.898046+0000","last_peered":"2023-10-10T20:30:04.898046+0000","last_clean":"2023-10-10T20:30:04.898046+0000","last_became_active":"2023-10-10T20:30:04.894865+0000","last_became_peered":"2023-10-10T20:30:04.894865+0000","last_unstale":"2023-10-10T20:30:04.898046+0000","last_undegraded":"2023-10-10T20:30:04.898046+0000","last_fullsized":"2023-10-10T20:30:04.898046+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:42:00.402654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886645+0000","last_change":"2023-10-10T20:30:03.886710+0000","last_active":"2023-10-10T20:30:04.886645+0000","last_peered":"2023-10-10T20:30:04.886645+0000","last_clean":"2023-10-10T20:30:04.886645+0000","last_became_active":"2023-10-10T20:30:03.886484+0000","last_became_peered":"2023-10-10T20:30:03.886484+0000","last_unstale":"2023-10-10T20:30:04.886645+0000","last_undegraded":"2023-10-10T20:30:04.886645+0000","last_fullsized":"2023-10-10T20:30:04.886645+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890830+0000","last_change":"2023-10-10T20:30:04.890424+0000","last_active":"2023-10-10T20:30:04.890830+0000","last_peered":"2023-10-10T20:30:04.890830+0000","last_clean":"2023-10-10T20:30:04.890830+0000","last_became_active":"2023-10-10T20:30:04.890185+0000","last_became_peered":"2023-10-10T20:30:04.890185+0000","last_unstale":"2023-10-10T20:30:04.890830+0000","last_undegraded":"2023-10-10T20:30:04.890830+0000","last_fullsized":"2023-10-10T20:30:04.890830+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:17:01.169446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887185+0000","last_change":"2023-10-10T20:30:03.885345+0000","last_active":"2023-10-10T20:30:04.887185+0000","last_peered":"2023-10-10T20:30:04.887185+0000","last_clean":"2023-10-10T20:30:04.887185+0000","last_became_active":"2023-10-10T20:30:03.885068+0000","last_became_peered":"2023-10-10T20:30:03.885068+0000","last_unstale":"2023-10-10T20:30:04.887185+0000","last_undegraded":"2023-10-10T20:30:04.887185+0000","last_fullsized":"2023-10-10T20:30:04.887185+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:54.029327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902554+0000","last_change":"2023-10-10T20:30:04.902135+0000","last_active":"2023-10-10T20:30:04.902554+0000","last_peered":"2023-10-10T20:30:04.902554+0000","last_clean":"2023-10-10T20:30:04.902554+0000","last_became_active":"2023-10-10T20:30:04.901263+0000","last_became_peered":"2023-10-10T20:30:04.901263+0000","last_unstale":"2023-10-10T20:30:04.902554+0000","last_undegraded":"2023-10-10T20:30:04.902554+0000","last_fullsized":"2023-10-10T20:30:04.902554+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:36:03.064103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.882291+0000","last_change":"2023-10-10T20:30:03.892090+0000","last_active":"2023-10-10T20:30:04.882291+0000","last_peered":"2023-10-10T20:30:04.882291+0000","last_clean":"2023-10-10T20:30:04.882291+0000","last_became_active":"2023-10-10T20:30:03.891639+0000","last_became_peered":"2023-10-10T20:30:03.891639+0000","last_unstale":"2023-10-10T20:30:04.882291+0000","last_undegraded":"2023-10-10T20:30:04.882291+0000","last_fullsized":"2023-10-10T20:30:04.882291+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:54.029327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896666+0000","last_change":"2023-10-10T20:30:04.896666+0000","last_active":"2023-10-10T20:30:04.896666+0000","last_peered":"2023-10-10T20:30:04.896666+0000","last_clean":"2023-10-10T20:30:04.896666+0000","last_became_active":"2023-10-10T20:30:04.896504+0000","last_became_peered":"2023-10-10T20:30:04.896504+0000","last_unstale":"2023-10-10T20:30:04.896666+0000","last_undegraded":"2023-10-10T20:30:04.896666+0000","last_fullsized":"2023-10-10T20:30:04.896666+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:17:01.169446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.882949+0000","last_change":"2023-10-10T20:30:03.893203+0000","last_active":"2023-10-10T20:30:04.882949+0000","last_peered":"2023-10-10T20:30:04.882949+0000","last_clean":"2023-10-10T20:30:04.882949+0000","last_became_active":"2023-10-10T20:30:03.892521+0000","last_became_peered":"2023-10-10T20:30:03.892521+0000","last_unstale":"2023-10-10T20:30:04.882949+0000","last_undegraded":"2023-10-10T20:30:04.882949+0000","last_fullsized":"2023-10-10T20:30:04.882949+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885353+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:04.885353+0000","last_peered":"2023-10-10T20:30:04.885353+0000","last_clean":"2023-10-10T20:30:04.885353+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:04.885353+0000","last_undegraded":"2023-10-10T20:30:04.885353+0000","last_fullsized":"2023-10-10T20:30:04.885353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1542,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193769,"statfs":{"total":200000000,"available":198420386,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.30399999999999999,"15min":0.27500000000000002},"max":{"1min":0.68400000000000005,"5min":0.754,"15min":0.754},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.38900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47999999999999998,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.40899999999999997,"15min":0.33100000000000002},"max":{"1min":0.77800000000000002,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.29599999999999999},"max":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.80800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.50800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194342,"statfs":{"total":200000000,"available":199006694,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.434,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.73299999999999998,"5min":0.877,"15min":1.3140000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.70899999999999996,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78200000000000003,"5min":0.92100000000000004,"15min":1.865},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":0.82299999999999995,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.505}]},{"osd":3,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.748,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.35699999999999998},"max":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":15.862},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45900000000000002,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.26400000000000001,"15min":0.222},"max":{"1min":0.85099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.0409999999999999},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72499999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194342,"statfs":{"total":200000000,"available":199006682,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33700000000000002,"15min":0.36199999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46899999999999997,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":1.647},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54000000000000004,"5min":0.627,"15min":1.524},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38900000000000001,"15min":0.40600000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.59299999999999997,"5min":0.627,"15min":1.673},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":1.633},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.34699999999999998},"max":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":1.506},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.70499999999999996,"5min":0.84899999999999998,"15min":1.696},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.748,"5min":0.877,"15min":1.7490000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.46400000000000002,"5min":0.45200000000000001,"15min":0.39400000000000002},"max":{"1min":0.78300000000000003,"5min":0.98899999999999999,"15min":1.585},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77400000000000002,"5min":1.0009999999999999,"15min":1.478},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":1.722},"last":0.46700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193773,"statfs":{"total":200000000,"available":198423647,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35799999999999998,"15min":0.40899999999999997},"min":{"1min":0.23799999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":1.8740000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.377,"15min":0.436},"min":{"1min":0.25600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.40600000000000003},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51600000000000001,"5min":0.57299999999999995,"15min":5.891},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35599999999999998,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.64100000000000001,"5min":0.68899999999999995,"15min":1.462},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35599999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.79700000000000004},"min":{"1min":0.42599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.873,"5min":0.873,"15min":9.7260000000000009},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":1.3799999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36299999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":1.4970000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":5.6379999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.76100000000000001,"5min":0.84699999999999998,"15min":5.758},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":962,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199013925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.372},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":2.4409999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.42699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23400000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.26100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.38700000000000001},"min":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.22},"max":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.224},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.44600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":11.815},"last":0.307},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.45800000000000002},"min":{"1min":0.317,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.28100000000000003},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.65500000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.873,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.84299999999999997,"5min":0.93600000000000005,"15min":6.399},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.29499999999999998},"max":{"1min":0.88900000000000001,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.92300000000000004,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":963,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194348,"statfs":{"total":200000000,"available":199012970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32100000000000001,"15min":0.311},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.499,"5min":0.622,"15min":0.622},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.32800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36799999999999999,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.51800000000000002,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.372},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.38700000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.751},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.42199999999999999,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.56100000000000005,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.432,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.32100000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.77500000000000002,"5min":0.875,"15min":0.875},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.83899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.56100000000000005},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.877,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":971,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194340,"statfs":{"total":200000000,"available":199005110,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.72599999999999998,"5min":0.76400000000000001,"15min":0.85399999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.753,"15min":0.85899999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.81599999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.51700000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.33700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46200000000000002,"15min":0.46700000000000003},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70699999999999996,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.63}]},{"osd":6,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.34399999999999997,"5min":0.28999999999999998,"15min":0.26100000000000001},"max":{"1min":0.68400000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.88700000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.30399999999999999,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.92600000000000005,"15min":0.98599999999999999},"last":0.60599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194341,"statfs":{"total":200000000,"available":199006140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.66600000000000004,"5min":0.77200000000000002,"15min":0.91200000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.66700000000000004,"5min":2.105,"15min":2.105},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54600000000000004,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.32000000000000001,"15min":0.255},"max":{"1min":0.67600000000000005,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57599999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.65200000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.47199999999999998,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.68500000000000005,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.56599999999999995},"min":{"1min":0.48499999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45700000000000002,"15min":0.48399999999999999},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":0.76900000000000002,"15min":0.79900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.80800000000000005,"15min":0.875},"last":0.47499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.498,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.55800000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.63500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.34799999999999998},"max":{"1min":0.73099999999999998,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.77900000000000003}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:08.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.097+0000 7f0678ed7700 1 Processor -- start 2023-10-10T20:30:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.097+0000 7f0678ed7700 1 -- start start 2023-10-10T20:30:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0678ed7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 0x7f06740cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0678ed7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f067415cbd0 0x7f067415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0678ed7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06740cb7c0 con 0x7f06740cdab0 2023-10-10T20:30:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0678ed7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067415e850 con 0x7f067415cbd0 2023-10-10T20:30:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0678ed7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06740cd030 con 0x7f06740cd6e0 2023-10-10T20:30:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f067415cbd0 0x7f067415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f067415cbd0 0x7f067415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33450/0 (socket says 172.21.15.148:33450) 2023-10-10T20:30:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0672ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06740cdab0 0x7f067415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57420/0 (socket says 172.21.15.148:57420) 2023-10-10T20:30:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 -- 172.21.15.148:0/1694119031 learned_addr learned my addr 172.21.15.148:0/1694119031 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f06737fe700 1 --2- 172.21.15.148:0/1694119031 >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 0x7f06740cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 -- 172.21.15.148:0/1694119031 >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 msgr2=0x7f06740cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 --2- 172.21.15.148:0/1694119031 >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 0x7f06740cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 -- 172.21.15.148:0/1694119031 >> v1:172.21.15.148:6789/0 conn(0x7f06740cdab0 legacy=0x7f067415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f0673fff700 1 -- 172.21.15.148:0/1694119031 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067415e290 con 0x7f067415cbd0 2023-10-10T20:30:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f06727fc700 1 -- 172.21.15.148:0/1694119031 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585324740 0 0) 0x7f06740cb7c0 con 0x7f06740cdab0 2023-10-10T20:30:09.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.098+0000 7f06737fe700 1 --2- 172.21.15.148:0/1694119031 >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 0x7f06740cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:09.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0673fff700 1 --2- 172.21.15.148:0/1694119031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f067415cbd0 0x7f067415d050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f067406b730 tx=0x7f066c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05147b8cf35cca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f06727fc700 1 -- 172.21.15.148:0/1694119031 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00b860 con 0x7f067415cbd0 2023-10-10T20:30:09.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f06727fc700 1 -- 172.21.15.148:0/1694119031 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c00b9c0 con 0x7f067415cbd0 2023-10-10T20:30:09.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f06727fc700 1 -- 172.21.15.148:0/1694119031 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00bd60 con 0x7f067415cbd0 2023-10-10T20:30:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 -- 172.21.15.148:0/1694119031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f067415cbd0 msgr2=0x7f067415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1694119031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f067415cbd0 0x7f067415d050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f067406b730 tx=0x7f066c009250 comp rx=0 tx=0).stop 2023-10-10T20:30:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 -- 172.21.15.148:0/1694119031 shutdown_connections 2023-10-10T20:30:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1694119031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f067415cbd0 0x7f067415d050 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1694119031 >> v2:172.21.15.161:3300/0 conn(0x7f06740cd6e0 0x7f06740cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 -- 172.21.15.148:0/1694119031 >> 172.21.15.148:0/1694119031 conn(0x7f0674067660 msgr2=0x7f0674067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:09.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 -- 172.21.15.148:0/1694119031 shutdown_connections 2023-10-10T20:30:09.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.099+0000 7f0678ed7700 1 -- 172.21.15.148:0/1694119031 wait complete. 2023-10-10T20:30:09.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 Processor -- start 2023-10-10T20:30:09.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 -- start start 2023-10-10T20:30:09.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 0x7f067414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.110 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f067415cbd0 0x7f06740b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740ca180 con 0x7f06740cdab0 2023-10-10T20:30:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067414cd20 con 0x7f067415cbd0 2023-10-10T20:30:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0678ed7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740b62c0 con 0x7f06740cd6e0 2023-10-10T20:30:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0672ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0672ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33454/0 (socket says 172.21.15.148:33454) 2023-10-10T20:30:09.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.101+0000 7f0672ffd700 1 -- 172.21.15.148:0/1978718602 learned_addr learned my addr 172.21.15.148:0/1978718602 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:09.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0673fff700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f067415cbd0 0x7f06740b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f06737fe700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 0x7f067414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 -- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 msgr2=0x7f067414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 0x7f067414c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 -- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f067415cbd0 msgr2=0x7f06740b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f067415cbd0 0x7f06740b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 -- 172.21.15.148:0/1978718602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0094f0 con 0x7f06740cdab0 2023-10-10T20:30:09.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f06737fe700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 0x7f067414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:09.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0672ffd700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f06680011e0 tx=0x7f0668007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668005a70 con 0x7f06740cdab0 2023-10-10T20:30:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0668005bd0 con 0x7f06740cdab0 2023-10-10T20:30:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06740b56f0 con 0x7f06740cdab0 2023-10-10T20:30:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680070d0 con 0x7f06740cdab0 2023-10-10T20:30:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.102+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740b5f50 con 0x7f06740cdab0 2023-10-10T20:30:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.103+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674089280 con 0x7f06740cdab0 2023-10-10T20:30:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.103+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0668007480 con 0x7f06740cdab0 2023-10-10T20:30:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.104+0000 7f065b7fe700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 0x7f06600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.104+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0668083980 con 0x7f06740cdab0 2023-10-10T20:30:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.104+0000 7f06737fe700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 0x7f06600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.104+0000 7f06737fe700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 0x7f06600693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f06640076e0 tx=0x7f06640098f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.108+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f066804b5a0 con 0x7f06740cdab0 2023-10-10T20:30:09.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.288+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06740b6400 con 0x7f0660066f30 2023-10-10T20:30:09.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.296+0000 7f065b7fe700 1 -- 172.21.15.148:0/1978718602 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285888 (secure 0 0 0) 0x7f06740b6400 con 0x7f0660066f30 2023-10-10T20:30:09.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:09.303 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 msgr2=0x7f06600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 0x7f06600693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f06640076e0 tx=0x7f06640098f0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 msgr2=0x7f06740b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f06680011e0 tx=0x7f0668007e10 comp rx=0 tx=0).stop 2023-10-10T20:30:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 shutdown_connections 2023-10-10T20:30:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0660066f30 0x7f06600693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f067415cbd0 0x7f06740b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06740cd6e0 0x7f067414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 --2- 172.21.15.148:0/1978718602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06740cdab0 0x7f06740b4cc0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 >> 172.21.15.148:0/1978718602 conn(0x7f0674067660 msgr2=0x7f067414d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:09.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 shutdown_connections 2023-10-10T20:30:09.306 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.299+0000 7f0678ed7700 1 -- 172.21.15.148:0/1978718602 wait complete. 2023-10-10T20:30:09.306 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:09.313 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2023-10-10T20:30:07.468633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553608,"statfs":{"total":1600000000,"available":1590895554,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.595195"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902028+0000","last_change":"2023-10-10T20:30:04.900734+0000","last_active":"2023-10-10T20:30:04.902028+0000","last_peered":"2023-10-10T20:30:04.902028+0000","last_clean":"2023-10-10T20:30:04.902028+0000","last_became_active":"2023-10-10T20:30:04.900371+0000","last_became_peered":"2023-10-10T20:30:04.900371+0000","last_unstale":"2023-10-10T20:30:04.902028+0000","last_undegraded":"2023-10-10T20:30:04.902028+0000","last_fullsized":"2023-10-10T20:30:04.902028+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:38.903594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892166+0000","last_change":"2023-10-10T20:30:04.891339+0000","last_active":"2023-10-10T20:30:04.892166+0000","last_peered":"2023-10-10T20:30:04.892166+0000","last_clean":"2023-10-10T20:30:04.892166+0000","last_became_active":"2023-10-10T20:30:04.890531+0000","last_became_peered":"2023-10-10T20:30:04.890531+0000","last_unstale":"2023-10-10T20:30:04.892166+0000","last_undegraded":"2023-10-10T20:30:04.892166+0000","last_fullsized":"2023-10-10T20:30:04.892166+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:28.533521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902574+0000","last_change":"2023-10-10T20:30:04.902375+0000","last_active":"2023-10-10T20:30:04.902574+0000","last_peered":"2023-10-10T20:30:04.902574+0000","last_clean":"2023-10-10T20:30:04.902574+0000","last_became_active":"2023-10-10T20:30:04.901572+0000","last_became_peered":"2023-10-10T20:30:04.901572+0000","last_unstale":"2023-10-10T20:30:04.902574+0000","last_undegraded":"2023-10-10T20:30:04.902574+0000","last_fullsized":"2023-10-10T20:30:04.902574+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:03.005907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889520+0000","last_change":"2023-10-10T20:30:04.889394+0000","last_active":"2023-10-10T20:30:04.889520+0000","last_peered":"2023-10-10T20:30:04.889520+0000","last_clean":"2023-10-10T20:30:04.889520+0000","last_became_active":"2023-10-10T20:30:04.888798+0000","last_became_peered":"2023-10-10T20:30:04.888798+0000","last_unstale":"2023-10-10T20:30:04.889520+0000","last_undegraded":"2023-10-10T20:30:04.889520+0000","last_fullsized":"2023-10-10T20:30:04.889520+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894846+0000","last_change":"2023-10-10T20:30:04.894434+0000","last_active":"2023-10-10T20:30:04.894846+0000","last_peered":"2023-10-10T20:30:04.894846+0000","last_clean":"2023-10-10T20:30:04.894846+0000","last_became_active":"2023-10-10T20:30:04.894189+0000","last_became_peered":"2023-10-10T20:30:04.894189+0000","last_unstale":"2023-10-10T20:30:04.894846+0000","last_undegraded":"2023-10-10T20:30:04.894846+0000","last_fullsized":"2023-10-10T20:30:04.894846+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894045+0000","last_change":"2023-10-10T20:30:04.894045+0000","last_active":"2023-10-10T20:30:04.894045+0000","last_peered":"2023-10-10T20:30:04.894045+0000","last_clean":"2023-10-10T20:30:04.894045+0000","last_became_active":"2023-10-10T20:30:04.892826+0000","last_became_peered":"2023-10-10T20:30:04.892826+0000","last_unstale":"2023-10-10T20:30:04.894045+0000","last_undegraded":"2023-10-10T20:30:04.894045+0000","last_fullsized":"2023-10-10T20:30:04.894045+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:22:04.746102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892625+0000","last_change":"2023-10-10T20:30:04.891826+0000","last_active":"2023-10-10T20:30:04.892625+0000","last_peered":"2023-10-10T20:30:04.892625+0000","last_clean":"2023-10-10T20:30:04.892625+0000","last_became_active":"2023-10-10T20:30:04.891123+0000","last_became_peered":"2023-10-10T20:30:04.891123+0000","last_unstale":"2023-10-10T20:30:04.892625+0000","last_undegraded":"2023-10-10T20:30:04.892625+0000","last_fullsized":"2023-10-10T20:30:04.892625+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:08:05.484701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891086+0000","last_change":"2023-10-10T20:30:04.890927+0000","last_active":"2023-10-10T20:30:04.891086+0000","last_peered":"2023-10-10T20:30:04.891086+0000","last_clean":"2023-10-10T20:30:04.891086+0000","last_became_active":"2023-10-10T20:30:04.890655+0000","last_became_peered":"2023-10-10T20:30:04.890655+0000","last_unstale":"2023-10-10T20:30:04.891086+0000","last_undegraded":"2023-10-10T20:30:04.891086+0000","last_fullsized":"2023-10-10T20:30:04.891086+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:39:43.975389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892183+0000","last_change":"2023-10-10T20:30:04.891897+0000","last_active":"2023-10-10T20:30:04.892183+0000","last_peered":"2023-10-10T20:30:04.892183+0000","last_clean":"2023-10-10T20:30:04.892183+0000","last_became_active":"2023-10-10T20:30:04.891309+0000","last_became_peered":"2023-10-10T20:30:04.891309+0000","last_unstale":"2023-10-10T20:30:04.892183+0000","last_undegraded":"2023-10-10T20:30:04.892183+0000","last_fullsized":"2023-10-10T20:30:04.892183+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:59.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892363+0000","last_change":"2023-10-10T20:30:04.892243+0000","last_active":"2023-10-10T20:30:04.892363+0000","last_peered":"2023-10-10T20:30:04.892363+0000","last_clean":"2023-10-10T20:30:04.892363+0000","last_became_active":"2023-10-10T20:30:04.890671+0000","last_became_peered":"2023-10-10T20:30:04.890671+0000","last_unstale":"2023-10-10T20:30:04.892363+0000","last_undegraded":"2023-10-10T20:30:04.892363+0000","last_fullsized":"2023-10-10T20:30:04.892363+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:21.294621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894864+0000","last_change":"2023-10-10T20:30:04.894754+0000","last_active":"2023-10-10T20:30:04.894864+0000","last_peered":"2023-10-10T20:30:04.894864+0000","last_clean":"2023-10-10T20:30:04.894864+0000","last_became_active":"2023-10-10T20:30:04.894466+0000","last_became_peered":"2023-10-10T20:30:04.894466+0000","last_unstale":"2023-10-10T20:30:04.894864+0000","last_undegraded":"2023-10-10T20:30:04.894864+0000","last_fullsized":"2023-10-10T20:30:04.894864+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:30:49.395189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894158+0000","last_change":"2023-10-10T20:30:04.893319+0000","last_active":"2023-10-10T20:30:04.894158+0000","last_peered":"2023-10-10T20:30:04.894158+0000","last_clean":"2023-10-10T20:30:04.894158+0000","last_became_active":"2023-10-10T20:30:04.891550+0000","last_became_peered":"2023-10-10T20:30:04.891550+0000","last_unstale":"2023-10-10T20:30:04.894158+0000","last_undegraded":"2023-10-10T20:30:04.894158+0000","last_fullsized":"2023-10-10T20:30:04.894158+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:36.597979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902728+0000","last_change":"2023-10-10T20:30:04.902657+0000","last_active":"2023-10-10T20:30:04.902728+0000","last_peered":"2023-10-10T20:30:04.902728+0000","last_clean":"2023-10-10T20:30:04.902728+0000","last_became_active":"2023-10-10T20:30:04.901862+0000","last_became_peered":"2023-10-10T20:30:04.901862+0000","last_unstale":"2023-10-10T20:30:04.902728+0000","last_undegraded":"2023-10-10T20:30:04.902728+0000","last_fullsized":"2023-10-10T20:30:04.902728+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:08:20.582951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902368+0000","last_change":"2023-10-10T20:30:04.900458+0000","last_active":"2023-10-10T20:30:04.902368+0000","last_peered":"2023-10-10T20:30:04.902368+0000","last_clean":"2023-10-10T20:30:04.902368+0000","last_became_active":"2023-10-10T20:30:04.900051+0000","last_became_peered":"2023-10-10T20:30:04.900051+0000","last_unstale":"2023-10-10T20:30:04.902368+0000","last_undegraded":"2023-10-10T20:30:04.902368+0000","last_fullsized":"2023-10-10T20:30:04.902368+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:08:20.934208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889820+0000","last_change":"2023-10-10T20:30:04.889753+0000","last_active":"2023-10-10T20:30:04.889820+0000","last_peered":"2023-10-10T20:30:04.889820+0000","last_clean":"2023-10-10T20:30:04.889820+0000","last_became_active":"2023-10-10T20:30:04.889523+0000","last_became_peered":"2023-10-10T20:30:04.889523+0000","last_unstale":"2023-10-10T20:30:04.889820+0000","last_undegraded":"2023-10-10T20:30:04.889820+0000","last_fullsized":"2023-10-10T20:30:04.889820+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892642+0000","last_change":"2023-10-10T20:30:04.891883+0000","last_active":"2023-10-10T20:30:04.892642+0000","last_peered":"2023-10-10T20:30:04.892642+0000","last_clean":"2023-10-10T20:30:04.892642+0000","last_became_active":"2023-10-10T20:30:04.891532+0000","last_became_peered":"2023-10-10T20:30:04.891532+0000","last_unstale":"2023-10-10T20:30:04.892642+0000","last_undegraded":"2023-10-10T20:30:04.892642+0000","last_fullsized":"2023-10-10T20:30:04.892642+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:21.574488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903203+0000","last_change":"2023-10-10T20:30:04.903203+0000","last_active":"2023-10-10T20:30:04.903203+0000","last_peered":"2023-10-10T20:30:04.903203+0000","last_clean":"2023-10-10T20:30:04.903203+0000","last_became_active":"2023-10-10T20:30:04.902976+0000","last_became_peered":"2023-10-10T20:30:04.902976+0000","last_unstale":"2023-10-10T20:30:04.903203+0000","last_undegraded":"2023-10-10T20:30:04.903203+0000","last_fullsized":"2023-10-10T20:30:04.903203+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:24.931195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898025+0000","last_change":"2023-10-10T20:30:04.898025+0000","last_active":"2023-10-10T20:30:04.898025+0000","last_peered":"2023-10-10T20:30:04.898025+0000","last_clean":"2023-10-10T20:30:04.898025+0000","last_became_active":"2023-10-10T20:30:04.897752+0000","last_became_peered":"2023-10-10T20:30:04.897752+0000","last_unstale":"2023-10-10T20:30:04.898025+0000","last_undegraded":"2023-10-10T20:30:04.898025+0000","last_fullsized":"2023-10-10T20:30:04.898025+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:58.402354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.884636+0000","last_change":"2023-10-10T20:30:04.884636+0000","last_active":"2023-10-10T20:30:04.884636+0000","last_peered":"2023-10-10T20:30:04.884636+0000","last_clean":"2023-10-10T20:30:04.884636+0000","last_became_active":"2023-10-10T20:30:04.884502+0000","last_became_peered":"2023-10-10T20:30:04.884502+0000","last_unstale":"2023-10-10T20:30:04.884636+0000","last_undegraded":"2023-10-10T20:30:04.884636+0000","last_fullsized":"2023-10-10T20:30:04.884636+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T05:59:48.166772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903632+0000","last_change":"2023-10-10T20:30:04.903548+0000","last_active":"2023-10-10T20:30:04.903632+0000","last_peered":"2023-10-10T20:30:04.903632+0000","last_clean":"2023-10-10T20:30:04.903632+0000","last_became_active":"2023-10-10T20:30:04.903256+0000","last_became_peered":"2023-10-10T20:30:04.903256+0000","last_unstale":"2023-10-10T20:30:04.903632+0000","last_undegraded":"2023-10-10T20:30:04.903632+0000","last_fullsized":"2023-10-10T20:30:04.903632+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:44:21.005942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892616+0000","last_change":"2023-10-10T20:30:04.892616+0000","last_active":"2023-10-10T20:30:04.892616+0000","last_peered":"2023-10-10T20:30:04.892616+0000","last_clean":"2023-10-10T20:30:04.892616+0000","last_became_active":"2023-10-10T20:30:04.892264+0000","last_became_peered":"2023-10-10T20:30:04.892264+0000","last_unstale":"2023-10-10T20:30:04.892616+0000","last_undegraded":"2023-10-10T20:30:04.892616+0000","last_fullsized":"2023-10-10T20:30:04.892616+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:29.170127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894178+0000","last_change":"2023-10-10T20:30:04.894011+0000","last_active":"2023-10-10T20:30:04.894178+0000","last_peered":"2023-10-10T20:30:04.894178+0000","last_clean":"2023-10-10T20:30:04.894178+0000","last_became_active":"2023-10-10T20:30:04.893699+0000","last_became_peered":"2023-10-10T20:30:04.893699+0000","last_unstale":"2023-10-10T20:30:04.894178+0000","last_undegraded":"2023-10-10T20:30:04.894178+0000","last_fullsized":"2023-10-10T20:30:04.894178+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:48.585573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893381+0000","last_change":"2023-10-10T20:30:04.893381+0000","last_active":"2023-10-10T20:30:04.893381+0000","last_peered":"2023-10-10T20:30:04.893381+0000","last_clean":"2023-10-10T20:30:04.893381+0000","last_became_active":"2023-10-10T20:30:04.893182+0000","last_became_peered":"2023-10-10T20:30:04.893182+0000","last_unstale":"2023-10-10T20:30:04.893381+0000","last_undegraded":"2023-10-10T20:30:04.893381+0000","last_fullsized":"2023-10-10T20:30:04.893381+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:15:09.455246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892197+0000","last_change":"2023-10-10T20:30:04.892060+0000","last_active":"2023-10-10T20:30:04.892197+0000","last_peered":"2023-10-10T20:30:04.892197+0000","last_clean":"2023-10-10T20:30:04.892197+0000","last_became_active":"2023-10-10T20:30:04.891648+0000","last_became_peered":"2023-10-10T20:30:04.891648+0000","last_unstale":"2023-10-10T20:30:04.892197+0000","last_undegraded":"2023-10-10T20:30:04.892197+0000","last_fullsized":"2023-10-10T20:30:04.892197+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:35.730188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902102+0000","last_change":"2023-10-10T20:30:04.898980+0000","last_active":"2023-10-10T20:30:04.902102+0000","last_peered":"2023-10-10T20:30:04.902102+0000","last_clean":"2023-10-10T20:30:04.902102+0000","last_became_active":"2023-10-10T20:30:04.898770+0000","last_became_peered":"2023-10-10T20:30:04.898770+0000","last_unstale":"2023-10-10T20:30:04.902102+0000","last_undegraded":"2023-10-10T20:30:04.902102+0000","last_fullsized":"2023-10-10T20:30:04.902102+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:16:14.375378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892659+0000","last_change":"2023-10-10T20:30:04.891956+0000","last_active":"2023-10-10T20:30:04.892659+0000","last_peered":"2023-10-10T20:30:04.892659+0000","last_clean":"2023-10-10T20:30:04.892659+0000","last_became_active":"2023-10-10T20:30:04.891713+0000","last_became_peered":"2023-10-10T20:30:04.891713+0000","last_unstale":"2023-10-10T20:30:04.892659+0000","last_undegraded":"2023-10-10T20:30:04.892659+0000","last_fullsized":"2023-10-10T20:30:04.892659+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:59.152754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903111+0000","last_change":"2023-10-10T20:30:04.903045+0000","last_active":"2023-10-10T20:30:04.903111+0000","last_peered":"2023-10-10T20:30:04.903111+0000","last_clean":"2023-10-10T20:30:04.903111+0000","last_became_active":"2023-10-10T20:30:04.902617+0000","last_became_peered":"2023-10-10T20:30:04.902617+0000","last_unstale":"2023-10-10T20:30:04.903111+0000","last_undegraded":"2023-10-10T20:30:04.903111+0000","last_fullsized":"2023-10-10T20:30:04.903111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:48.431851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900001+0000","last_change":"2023-10-10T20:30:04.899943+0000","last_active":"2023-10-10T20:30:04.900001+0000","last_peered":"2023-10-10T20:30:04.900001+0000","last_clean":"2023-10-10T20:30:04.900001+0000","last_became_active":"2023-10-10T20:30:04.899717+0000","last_became_peered":"2023-10-10T20:30:04.899717+0000","last_unstale":"2023-10-10T20:30:04.900001+0000","last_undegraded":"2023-10-10T20:30:04.900001+0000","last_fullsized":"2023-10-10T20:30:04.900001+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:48.585573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896995+0000","last_change":"2023-10-10T20:30:04.896375+0000","last_active":"2023-10-10T20:30:04.896995+0000","last_peered":"2023-10-10T20:30:04.896995+0000","last_clean":"2023-10-10T20:30:04.896995+0000","last_became_active":"2023-10-10T20:30:04.896156+0000","last_became_peered":"2023-10-10T20:30:04.896156+0000","last_unstale":"2023-10-10T20:30:04.896995+0000","last_undegraded":"2023-10-10T20:30:04.896995+0000","last_fullsized":"2023-10-10T20:30:04.896995+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:18:52.866253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886527+0000","last_change":"2023-10-10T20:30:04.886031+0000","last_active":"2023-10-10T20:30:04.886527+0000","last_peered":"2023-10-10T20:30:04.886527+0000","last_clean":"2023-10-10T20:30:04.886527+0000","last_became_active":"2023-10-10T20:30:04.885865+0000","last_became_peered":"2023-10-10T20:30:04.885865+0000","last_unstale":"2023-10-10T20:30:04.886527+0000","last_undegraded":"2023-10-10T20:30:04.886527+0000","last_fullsized":"2023-10-10T20:30:04.886527+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:33:58.541825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898067+0000","last_change":"2023-10-10T20:30:04.895587+0000","last_active":"2023-10-10T20:30:04.898067+0000","last_peered":"2023-10-10T20:30:04.898067+0000","last_clean":"2023-10-10T20:30:04.898067+0000","last_became_active":"2023-10-10T20:30:04.895139+0000","last_became_peered":"2023-10-10T20:30:04.895139+0000","last_unstale":"2023-10-10T20:30:04.898067+0000","last_undegraded":"2023-10-10T20:30:04.898067+0000","last_fullsized":"2023-10-10T20:30:04.898067+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:28:26.096300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899070+0000","last_change":"2023-10-10T20:30:04.899070+0000","last_active":"2023-10-10T20:30:04.899070+0000","last_peered":"2023-10-10T20:30:04.899070+0000","last_clean":"2023-10-10T20:30:04.899070+0000","last_became_active":"2023-10-10T20:30:04.898511+0000","last_became_peered":"2023-10-10T20:30:04.898511+0000","last_unstale":"2023-10-10T20:30:04.899070+0000","last_undegraded":"2023-10-10T20:30:04.899070+0000","last_fullsized":"2023-10-10T20:30:04.899070+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:08:05.484701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890782+0000","last_change":"2023-10-10T20:30:04.889965+0000","last_active":"2023-10-10T20:30:04.890782+0000","last_peered":"2023-10-10T20:30:04.890782+0000","last_clean":"2023-10-10T20:30:04.890782+0000","last_became_active":"2023-10-10T20:30:04.889714+0000","last_became_peered":"2023-10-10T20:30:04.889714+0000","last_unstale":"2023-10-10T20:30:04.890782+0000","last_undegraded":"2023-10-10T20:30:04.890782+0000","last_fullsized":"2023-10-10T20:30:04.890782+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"177'4","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903738+0000","last_change":"2023-10-10T20:30:03.889349+0000","last_active":"2023-10-10T20:30:04.903738+0000","last_peered":"2023-10-10T20:30:04.903738+0000","last_clean":"2023-10-10T20:30:04.903738+0000","last_became_active":"2023-10-10T20:30:03.889112+0000","last_became_peered":"2023-10-10T20:30:03.889112+0000","last_unstale":"2023-10-10T20:30:04.903738+0000","last_undegraded":"2023-10-10T20:30:04.903738+0000","last_fullsized":"2023-10-10T20:30:04.903738+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:13:05.003759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891053+0000","last_change":"2023-10-10T20:30:04.890977+0000","last_active":"2023-10-10T20:30:04.891053+0000","last_peered":"2023-10-10T20:30:04.891053+0000","last_clean":"2023-10-10T20:30:04.891053+0000","last_became_active":"2023-10-10T20:30:04.890714+0000","last_became_peered":"2023-10-10T20:30:04.890714+0000","last_unstale":"2023-10-10T20:30:04.891053+0000","last_undegraded":"2023-10-10T20:30:04.891053+0000","last_fullsized":"2023-10-10T20:30:04.891053+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:01:26.914899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897981+0000","last_change":"2023-10-10T20:30:03.892241+0000","last_active":"2023-10-10T20:30:04.897981+0000","last_peered":"2023-10-10T20:30:04.897981+0000","last_clean":"2023-10-10T20:30:04.897981+0000","last_became_active":"2023-10-10T20:30:03.891898+0000","last_became_peered":"2023-10-10T20:30:03.891898+0000","last_unstale":"2023-10-10T20:30:04.897981+0000","last_undegraded":"2023-10-10T20:30:04.897981+0000","last_fullsized":"2023-10-10T20:30:04.897981+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:29:12.416336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892483+0000","last_change":"2023-10-10T20:30:04.892483+0000","last_active":"2023-10-10T20:30:04.892483+0000","last_peered":"2023-10-10T20:30:04.892483+0000","last_clean":"2023-10-10T20:30:04.892483+0000","last_became_active":"2023-10-10T20:30:04.891965+0000","last_became_peered":"2023-10-10T20:30:04.891965+0000","last_unstale":"2023-10-10T20:30:04.892483+0000","last_undegraded":"2023-10-10T20:30:04.892483+0000","last_fullsized":"2023-10-10T20:30:04.892483+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:12:25.470282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896349+0000","last_change":"2023-10-10T20:30:03.881890+0000","last_active":"2023-10-10T20:30:04.896349+0000","last_peered":"2023-10-10T20:30:04.896349+0000","last_clean":"2023-10-10T20:30:04.896349+0000","last_became_active":"2023-10-10T20:30:03.881664+0000","last_became_peered":"2023-10-10T20:30:03.881664+0000","last_unstale":"2023-10-10T20:30:04.896349+0000","last_undegraded":"2023-10-10T20:30:04.896349+0000","last_fullsized":"2023-10-10T20:30:04.896349+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:24.168089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902591+0000","last_change":"2023-10-10T20:30:04.902437+0000","last_active":"2023-10-10T20:30:04.902591+0000","last_peered":"2023-10-10T20:30:04.902591+0000","last_clean":"2023-10-10T20:30:04.902591+0000","last_became_active":"2023-10-10T20:30:04.902155+0000","last_became_peered":"2023-10-10T20:30:04.902155+0000","last_unstale":"2023-10-10T20:30:04.902591+0000","last_undegraded":"2023-10-10T20:30:04.902591+0000","last_fullsized":"2023-10-10T20:30:04.902591+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:02:29.170127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896712+0000","last_change":"2023-10-10T20:30:03.887746+0000","last_active":"2023-10-10T20:30:04.896712+0000","last_peered":"2023-10-10T20:30:04.896712+0000","last_clean":"2023-10-10T20:30:04.896712+0000","last_became_active":"2023-10-10T20:30:03.887519+0000","last_became_peered":"2023-10-10T20:30:03.887519+0000","last_unstale":"2023-10-10T20:30:04.896712+0000","last_undegraded":"2023-10-10T20:30:04.896712+0000","last_fullsized":"2023-10-10T20:30:04.896712+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:38:03.618789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891115+0000","last_change":"2023-10-10T20:30:04.891058+0000","last_active":"2023-10-10T20:30:04.891115+0000","last_peered":"2023-10-10T20:30:04.891115+0000","last_clean":"2023-10-10T20:30:04.891115+0000","last_became_active":"2023-10-10T20:30:04.890844+0000","last_became_peered":"2023-10-10T20:30:04.890844+0000","last_unstale":"2023-10-10T20:30:04.891115+0000","last_undegraded":"2023-10-10T20:30:04.891115+0000","last_fullsized":"2023-10-10T20:30:04.891115+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:39:28.525881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902390+0000","last_change":"2023-10-10T20:30:03.888216+0000","last_active":"2023-10-10T20:30:04.902390+0000","last_peered":"2023-10-10T20:30:04.902390+0000","last_clean":"2023-10-10T20:30:04.902390+0000","last_became_active":"2023-10-10T20:30:03.887646+0000","last_became_peered":"2023-10-10T20:30:03.887646+0000","last_unstale":"2023-10-10T20:30:04.902390+0000","last_undegraded":"2023-10-10T20:30:04.902390+0000","last_fullsized":"2023-10-10T20:30:04.902390+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:14:52.137385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893091+0000","last_change":"2023-10-10T20:30:04.892902+0000","last_active":"2023-10-10T20:30:04.893091+0000","last_peered":"2023-10-10T20:30:04.893091+0000","last_clean":"2023-10-10T20:30:04.893091+0000","last_became_active":"2023-10-10T20:30:04.892385+0000","last_became_peered":"2023-10-10T20:30:04.892385+0000","last_unstale":"2023-10-10T20:30:04.893091+0000","last_undegraded":"2023-10-10T20:30:04.893091+0000","last_fullsized":"2023-10-10T20:30:04.893091+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:38:35.730188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896036+0000","last_change":"2023-10-10T20:30:03.886607+0000","last_active":"2023-10-10T20:30:04.896036+0000","last_peered":"2023-10-10T20:30:04.896036+0000","last_clean":"2023-10-10T20:30:04.896036+0000","last_became_active":"2023-10-10T20:30:03.886237+0000","last_became_peered":"2023-10-10T20:30:03.886237+0000","last_unstale":"2023-10-10T20:30:04.896036+0000","last_undegraded":"2023-10-10T20:30:04.896036+0000","last_fullsized":"2023-10-10T20:30:04.896036+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:29:12.416336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892122+0000","last_change":"2023-10-10T20:30:04.892062+0000","last_active":"2023-10-10T20:30:04.892122+0000","last_peered":"2023-10-10T20:30:04.892122+0000","last_clean":"2023-10-10T20:30:04.892122+0000","last_became_active":"2023-10-10T20:30:04.891774+0000","last_became_peered":"2023-10-10T20:30:04.891774+0000","last_unstale":"2023-10-10T20:30:04.892122+0000","last_undegraded":"2023-10-10T20:30:04.892122+0000","last_fullsized":"2023-10-10T20:30:04.892122+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:20:59.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887097+0000","last_change":"2023-10-10T20:30:03.889405+0000","last_active":"2023-10-10T20:30:04.887097+0000","last_peered":"2023-10-10T20:30:04.887097+0000","last_clean":"2023-10-10T20:30:04.887097+0000","last_became_active":"2023-10-10T20:30:03.889009+0000","last_became_peered":"2023-10-10T20:30:03.889009+0000","last_unstale":"2023-10-10T20:30:04.887097+0000","last_undegraded":"2023-10-10T20:30:04.887097+0000","last_fullsized":"2023-10-10T20:30:04.887097+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.891667+0000","last_change":"2023-10-10T20:30:04.891667+0000","last_active":"2023-10-10T20:30:04.891667+0000","last_peered":"2023-10-10T20:30:04.891667+0000","last_clean":"2023-10-10T20:30:04.891667+0000","last_became_active":"2023-10-10T20:30:04.891414+0000","last_became_peered":"2023-10-10T20:30:04.891414+0000","last_unstale":"2023-10-10T20:30:04.891667+0000","last_undegraded":"2023-10-10T20:30:04.891667+0000","last_fullsized":"2023-10-10T20:30:04.891667+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:03:33.937483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885855+0000","last_change":"2023-10-10T20:30:03.889990+0000","last_active":"2023-10-10T20:30:04.885855+0000","last_peered":"2023-10-10T20:30:04.885855+0000","last_clean":"2023-10-10T20:30:04.885855+0000","last_became_active":"2023-10-10T20:30:03.889779+0000","last_became_peered":"2023-10-10T20:30:03.889779+0000","last_unstale":"2023-10-10T20:30:04.885855+0000","last_undegraded":"2023-10-10T20:30:04.885855+0000","last_fullsized":"2023-10-10T20:30:04.885855+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:54:23.629817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894488+0000","last_change":"2023-10-10T20:30:04.894488+0000","last_active":"2023-10-10T20:30:04.894488+0000","last_peered":"2023-10-10T20:30:04.894488+0000","last_clean":"2023-10-10T20:30:04.894488+0000","last_became_active":"2023-10-10T20:30:04.890305+0000","last_became_peered":"2023-10-10T20:30:04.890305+0000","last_unstale":"2023-10-10T20:30:04.894488+0000","last_undegraded":"2023-10-10T20:30:04.894488+0000","last_fullsized":"2023-10-10T20:30:04.894488+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:59.152754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.888939+0000","last_change":"2023-10-10T20:30:03.888880+0000","last_active":"2023-10-10T20:30:03.888939+0000","last_peered":"2023-10-10T20:30:03.888939+0000","last_clean":"2023-10-10T20:30:03.888939+0000","last_became_active":"2023-10-10T20:30:03.888635+0000","last_became_peered":"2023-10-10T20:30:03.888635+0000","last_unstale":"2023-10-10T20:30:03.888939+0000","last_undegraded":"2023-10-10T20:30:03.888939+0000","last_fullsized":"2023-10-10T20:30:03.888939+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:07:51.498918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897313+0000","last_change":"2023-10-10T20:30:04.897313+0000","last_active":"2023-10-10T20:30:04.897313+0000","last_peered":"2023-10-10T20:30:04.897313+0000","last_clean":"2023-10-10T20:30:04.897313+0000","last_became_active":"2023-10-10T20:30:04.890219+0000","last_became_peered":"2023-10-10T20:30:04.890219+0000","last_unstale":"2023-10-10T20:30:04.897313+0000","last_undegraded":"2023-10-10T20:30:04.897313+0000","last_fullsized":"2023-10-10T20:30:04.897313+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:53:46.701566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"177'3","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900328+0000","last_change":"2023-10-10T20:30:03.888305+0000","last_active":"2023-10-10T20:30:04.900328+0000","last_peered":"2023-10-10T20:30:04.900328+0000","last_clean":"2023-10-10T20:30:04.900328+0000","last_became_active":"2023-10-10T20:30:03.888068+0000","last_became_peered":"2023-10-10T20:30:03.888068+0000","last_unstale":"2023-10-10T20:30:04.900328+0000","last_undegraded":"2023-10-10T20:30:04.900328+0000","last_fullsized":"2023-10-10T20:30:04.900328+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:14:11.071357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.901616+0000","last_change":"2023-10-10T20:30:04.901530+0000","last_active":"2023-10-10T20:30:04.901616+0000","last_peered":"2023-10-10T20:30:04.901616+0000","last_clean":"2023-10-10T20:30:04.901616+0000","last_became_active":"2023-10-10T20:30:04.901306+0000","last_became_peered":"2023-10-10T20:30:04.901306+0000","last_unstale":"2023-10-10T20:30:04.901616+0000","last_undegraded":"2023-10-10T20:30:04.901616+0000","last_fullsized":"2023-10-10T20:30:04.901616+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:10.205666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"177'6","reported_seq":16,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.904953+0000","last_change":"2023-10-10T20:30:03.890316+0000","last_active":"2023-10-10T20:30:04.904953+0000","last_peered":"2023-10-10T20:30:04.904953+0000","last_clean":"2023-10-10T20:30:04.904953+0000","last_became_active":"2023-10-10T20:30:03.890064+0000","last_became_peered":"2023-10-10T20:30:03.890064+0000","last_unstale":"2023-10-10T20:30:04.904953+0000","last_undegraded":"2023-10-10T20:30:04.904953+0000","last_fullsized":"2023-10-10T20:30:04.904953+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:05:38.322381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894631+0000","last_change":"2023-10-10T20:30:04.894631+0000","last_active":"2023-10-10T20:30:04.894631+0000","last_peered":"2023-10-10T20:30:04.894631+0000","last_clean":"2023-10-10T20:30:04.894631+0000","last_became_active":"2023-10-10T20:30:04.890606+0000","last_became_peered":"2023-10-10T20:30:04.890606+0000","last_unstale":"2023-10-10T20:30:04.894631+0000","last_undegraded":"2023-10-10T20:30:04.894631+0000","last_fullsized":"2023-10-10T20:30:04.894631+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:00.971201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898667+0000","last_change":"2023-10-10T20:30:03.883717+0000","last_active":"2023-10-10T20:30:04.898667+0000","last_peered":"2023-10-10T20:30:04.898667+0000","last_clean":"2023-10-10T20:30:04.898667+0000","last_became_active":"2023-10-10T20:30:03.883503+0000","last_became_peered":"2023-10-10T20:30:03.883503+0000","last_unstale":"2023-10-10T20:30:04.898667+0000","last_undegraded":"2023-10-10T20:30:04.898667+0000","last_fullsized":"2023-10-10T20:30:04.898667+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:14:23.295705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903439+0000","last_change":"2023-10-10T20:30:04.903386+0000","last_active":"2023-10-10T20:30:04.903439+0000","last_peered":"2023-10-10T20:30:04.903439+0000","last_clean":"2023-10-10T20:30:04.903439+0000","last_became_active":"2023-10-10T20:30:04.903186+0000","last_became_peered":"2023-10-10T20:30:04.903186+0000","last_unstale":"2023-10-10T20:30:04.903439+0000","last_undegraded":"2023-10-10T20:30:04.903439+0000","last_fullsized":"2023-10-10T20:30:04.903439+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:10:57.500136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.883931+0000","last_change":"2023-10-10T20:30:03.883809+0000","last_active":"2023-10-10T20:30:03.883931+0000","last_peered":"2023-10-10T20:30:03.883931+0000","last_clean":"2023-10-10T20:30:03.883931+0000","last_became_active":"2023-10-10T20:30:03.883595+0000","last_became_peered":"2023-10-10T20:30:03.883595+0000","last_unstale":"2023-10-10T20:30:03.883931+0000","last_undegraded":"2023-10-10T20:30:03.883931+0000","last_fullsized":"2023-10-10T20:30:03.883931+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:35.592670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903274+0000","last_change":"2023-10-10T20:30:04.903208+0000","last_active":"2023-10-10T20:30:04.903274+0000","last_peered":"2023-10-10T20:30:04.903274+0000","last_clean":"2023-10-10T20:30:04.903274+0000","last_became_active":"2023-10-10T20:30:04.902966+0000","last_became_peered":"2023-10-10T20:30:04.902966+0000","last_unstale":"2023-10-10T20:30:04.903274+0000","last_undegraded":"2023-10-10T20:30:04.903274+0000","last_fullsized":"2023-10-10T20:30:04.903274+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:31:21.294621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886556+0000","last_change":"2023-10-10T20:30:03.881756+0000","last_active":"2023-10-10T20:30:04.886556+0000","last_peered":"2023-10-10T20:30:04.886556+0000","last_clean":"2023-10-10T20:30:04.886556+0000","last_became_active":"2023-10-10T20:30:03.881565+0000","last_became_peered":"2023-10-10T20:30:03.881565+0000","last_unstale":"2023-10-10T20:30:04.886556+0000","last_undegraded":"2023-10-10T20:30:04.886556+0000","last_fullsized":"2023-10-10T20:30:04.886556+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:54:55.070436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903092+0000","last_change":"2023-10-10T20:30:04.902893+0000","last_active":"2023-10-10T20:30:04.903092+0000","last_peered":"2023-10-10T20:30:04.903092+0000","last_clean":"2023-10-10T20:30:04.903092+0000","last_became_active":"2023-10-10T20:30:04.902372+0000","last_became_peered":"2023-10-10T20:30:04.902372+0000","last_unstale":"2023-10-10T20:30:04.903092+0000","last_undegraded":"2023-10-10T20:30:04.903092+0000","last_fullsized":"2023-10-10T20:30:04.903092+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:24:31.428174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"177'8","reported_seq":18,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903004+0000","last_change":"2023-10-10T20:30:03.889705+0000","last_active":"2023-10-10T20:30:04.903004+0000","last_peered":"2023-10-10T20:30:04.903004+0000","last_clean":"2023-10-10T20:30:04.903004+0000","last_became_active":"2023-10-10T20:30:03.889433+0000","last_became_peered":"2023-10-10T20:30:03.889433+0000","last_unstale":"2023-10-10T20:30:04.903004+0000","last_undegraded":"2023-10-10T20:30:04.903004+0000","last_fullsized":"2023-10-10T20:30:04.903004+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898997+0000","last_change":"2023-10-10T20:30:04.898814+0000","last_active":"2023-10-10T20:30:04.898997+0000","last_peered":"2023-10-10T20:30:04.898997+0000","last_clean":"2023-10-10T20:30:04.898997+0000","last_became_active":"2023-10-10T20:30:04.898326+0000","last_became_peered":"2023-10-10T20:30:04.898326+0000","last_unstale":"2023-10-10T20:30:04.898997+0000","last_undegraded":"2023-10-10T20:30:04.898997+0000","last_fullsized":"2023-10-10T20:30:04.898997+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:28.533521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903421+0000","last_change":"2023-10-10T20:30:03.887443+0000","last_active":"2023-10-10T20:30:04.903421+0000","last_peered":"2023-10-10T20:30:04.903421+0000","last_clean":"2023-10-10T20:30:04.903421+0000","last_became_active":"2023-10-10T20:30:03.887135+0000","last_became_peered":"2023-10-10T20:30:03.887135+0000","last_unstale":"2023-10-10T20:30:04.903421+0000","last_undegraded":"2023-10-10T20:30:04.903421+0000","last_fullsized":"2023-10-10T20:30:04.903421+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:06:23.522665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897448+0000","last_change":"2023-10-10T20:30:04.897448+0000","last_active":"2023-10-10T20:30:04.897448+0000","last_peered":"2023-10-10T20:30:04.897448+0000","last_clean":"2023-10-10T20:30:04.897448+0000","last_became_active":"2023-10-10T20:30:04.890538+0000","last_became_peered":"2023-10-10T20:30:04.890538+0000","last_unstale":"2023-10-10T20:30:04.897448+0000","last_undegraded":"2023-10-10T20:30:04.897448+0000","last_fullsized":"2023-10-10T20:30:04.897448+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:01:21.574488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902266+0000","last_change":"2023-10-10T20:30:03.885612+0000","last_active":"2023-10-10T20:30:04.902266+0000","last_peered":"2023-10-10T20:30:04.902266+0000","last_clean":"2023-10-10T20:30:04.902266+0000","last_became_active":"2023-10-10T20:30:03.885317+0000","last_became_peered":"2023-10-10T20:30:03.885317+0000","last_unstale":"2023-10-10T20:30:04.902266+0000","last_undegraded":"2023-10-10T20:30:04.902266+0000","last_fullsized":"2023-10-10T20:30:04.902266+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:05:27.749116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.893108+0000","last_change":"2023-10-10T20:30:04.893038+0000","last_active":"2023-10-10T20:30:04.893108+0000","last_peered":"2023-10-10T20:30:04.893108+0000","last_clean":"2023-10-10T20:30:04.893108+0000","last_became_active":"2023-10-10T20:30:04.892690+0000","last_became_peered":"2023-10-10T20:30:04.892690+0000","last_unstale":"2023-10-10T20:30:04.893108+0000","last_undegraded":"2023-10-10T20:30:04.893108+0000","last_fullsized":"2023-10-10T20:30:04.893108+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:14:36.597979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"177'1","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.900591+0000","last_change":"2023-10-10T20:30:03.893140+0000","last_active":"2023-10-10T20:30:04.900591+0000","last_peered":"2023-10-10T20:30:04.900591+0000","last_clean":"2023-10-10T20:30:04.900591+0000","last_became_active":"2023-10-10T20:30:03.892640+0000","last_became_peered":"2023-10-10T20:30:03.892640+0000","last_unstale":"2023-10-10T20:30:04.900591+0000","last_undegraded":"2023-10-10T20:30:04.900591+0000","last_fullsized":"2023-10-10T20:30:04.900591+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:25:30.782134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899227+0000","last_change":"2023-10-10T20:30:04.899159+0000","last_active":"2023-10-10T20:30:04.899227+0000","last_peered":"2023-10-10T20:30:04.899227+0000","last_clean":"2023-10-10T20:30:04.899227+0000","last_became_active":"2023-10-10T20:30:04.898928+0000","last_became_peered":"2023-10-10T20:30:04.898928+0000","last_unstale":"2023-10-10T20:30:04.899227+0000","last_undegraded":"2023-10-10T20:30:04.899227+0000","last_fullsized":"2023-10-10T20:30:04.899227+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:20:55.837552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885462+0000","last_change":"2023-10-10T20:30:03.886004+0000","last_active":"2023-10-10T20:30:04.885462+0000","last_peered":"2023-10-10T20:30:04.885462+0000","last_clean":"2023-10-10T20:30:04.885462+0000","last_became_active":"2023-10-10T20:30:03.885776+0000","last_became_peered":"2023-10-10T20:30:03.885776+0000","last_unstale":"2023-10-10T20:30:04.885462+0000","last_undegraded":"2023-10-10T20:30:04.885462+0000","last_fullsized":"2023-10-10T20:30:04.885462+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:34:02.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902340+0000","last_change":"2023-10-10T20:30:04.900286+0000","last_active":"2023-10-10T20:30:04.902340+0000","last_peered":"2023-10-10T20:30:04.902340+0000","last_clean":"2023-10-10T20:30:04.902340+0000","last_became_active":"2023-10-10T20:30:04.899716+0000","last_became_peered":"2023-10-10T20:30:04.899716+0000","last_unstale":"2023-10-10T20:30:04.902340+0000","last_undegraded":"2023-10-10T20:30:04.902340+0000","last_fullsized":"2023-10-10T20:30:04.902340+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:35:36.275993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"177'2","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896446+0000","last_change":"2023-10-10T20:30:03.892928+0000","last_active":"2023-10-10T20:30:04.896446+0000","last_peered":"2023-10-10T20:30:04.896446+0000","last_clean":"2023-10-10T20:30:04.896446+0000","last_became_active":"2023-10-10T20:30:03.892427+0000","last_became_peered":"2023-10-10T20:30:03.892427+0000","last_unstale":"2023-10-10T20:30:04.896446+0000","last_undegraded":"2023-10-10T20:30:04.896446+0000","last_fullsized":"2023-10-10T20:30:04.896446+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T01:05:38.322381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890848+0000","last_change":"2023-10-10T20:30:04.890616+0000","last_active":"2023-10-10T20:30:04.890848+0000","last_peered":"2023-10-10T20:30:04.890848+0000","last_clean":"2023-10-10T20:30:04.890848+0000","last_became_active":"2023-10-10T20:30:04.890372+0000","last_became_peered":"2023-10-10T20:30:04.890372+0000","last_unstale":"2023-10-10T20:30:04.890848+0000","last_undegraded":"2023-10-10T20:30:04.890848+0000","last_fullsized":"2023-10-10T20:30:04.890848+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:18:38.903594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885258+0000","last_change":"2023-10-10T20:30:03.888491+0000","last_active":"2023-10-10T20:30:04.885258+0000","last_peered":"2023-10-10T20:30:04.885258+0000","last_clean":"2023-10-10T20:30:04.885258+0000","last_became_active":"2023-10-10T20:30:03.888265+0000","last_became_peered":"2023-10-10T20:30:03.888265+0000","last_unstale":"2023-10-10T20:30:04.885258+0000","last_undegraded":"2023-10-10T20:30:04.885258+0000","last_fullsized":"2023-10-10T20:30:04.885258+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:40:33.366728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.889795+0000","last_change":"2023-10-10T20:30:04.889584+0000","last_active":"2023-10-10T20:30:04.889795+0000","last_peered":"2023-10-10T20:30:04.889795+0000","last_clean":"2023-10-10T20:30:04.889795+0000","last_became_active":"2023-10-10T20:30:04.888803+0000","last_became_peered":"2023-10-10T20:30:04.888803+0000","last_unstale":"2023-10-10T20:30:04.889795+0000","last_undegraded":"2023-10-10T20:30:04.889795+0000","last_fullsized":"2023-10-10T20:30:04.889795+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:06:58.402354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887770+0000","last_change":"2023-10-10T20:30:03.886221+0000","last_active":"2023-10-10T20:30:04.887770+0000","last_peered":"2023-10-10T20:30:04.887770+0000","last_clean":"2023-10-10T20:30:04.887770+0000","last_became_active":"2023-10-10T20:30:03.885993+0000","last_became_peered":"2023-10-10T20:30:03.885993+0000","last_unstale":"2023-10-10T20:30:04.887770+0000","last_undegraded":"2023-10-10T20:30:04.887770+0000","last_fullsized":"2023-10-10T20:30:04.887770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892175+0000","last_change":"2023-10-10T20:30:04.892175+0000","last_active":"2023-10-10T20:30:04.892175+0000","last_peered":"2023-10-10T20:30:04.892175+0000","last_clean":"2023-10-10T20:30:04.892175+0000","last_became_active":"2023-10-10T20:30:04.891786+0000","last_became_peered":"2023-10-10T20:30:04.891786+0000","last_unstale":"2023-10-10T20:30:04.892175+0000","last_undegraded":"2023-10-10T20:30:04.892175+0000","last_fullsized":"2023-10-10T20:30:04.892175+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:53:02.843513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885892+0000","last_change":"2023-10-10T20:30:03.893285+0000","last_active":"2023-10-10T20:30:04.885892+0000","last_peered":"2023-10-10T20:30:04.885892+0000","last_clean":"2023-10-10T20:30:04.885892+0000","last_became_active":"2023-10-10T20:30:03.892923+0000","last_became_peered":"2023-10-10T20:30:03.892923+0000","last_unstale":"2023-10-10T20:30:04.885892+0000","last_undegraded":"2023-10-10T20:30:04.885892+0000","last_fullsized":"2023-10-10T20:30:04.885892+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T20:46:58.385171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.894999+0000","last_change":"2023-10-10T20:30:04.894999+0000","last_active":"2023-10-10T20:30:04.894999+0000","last_peered":"2023-10-10T20:30:04.894999+0000","last_clean":"2023-10-10T20:30:04.894999+0000","last_became_active":"2023-10-10T20:30:04.894106+0000","last_became_peered":"2023-10-10T20:30:04.894106+0000","last_unstale":"2023-10-10T20:30:04.894999+0000","last_undegraded":"2023-10-10T20:30:04.894999+0000","last_fullsized":"2023-10-10T20:30:04.894999+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T04:25:00.274809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"177'2","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.903837+0000","last_change":"2023-10-10T20:30:03.893314+0000","last_active":"2023-10-10T20:30:04.903837+0000","last_peered":"2023-10-10T20:30:04.903837+0000","last_clean":"2023-10-10T20:30:04.903837+0000","last_became_active":"2023-10-10T20:30:03.892650+0000","last_became_peered":"2023-10-10T20:30:03.892650+0000","last_unstale":"2023-10-10T20:30:04.903837+0000","last_undegraded":"2023-10-10T20:30:04.903837+0000","last_fullsized":"2023-10-10T20:30:04.903837+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:36:47.580264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892675+0000","last_change":"2023-10-10T20:30:04.891965+0000","last_active":"2023-10-10T20:30:04.892675+0000","last_peered":"2023-10-10T20:30:04.892675+0000","last_clean":"2023-10-10T20:30:04.892675+0000","last_became_active":"2023-10-10T20:30:04.891436+0000","last_became_peered":"2023-10-10T20:30:04.891436+0000","last_unstale":"2023-10-10T20:30:04.892675+0000","last_undegraded":"2023-10-10T20:30:04.892675+0000","last_fullsized":"2023-10-10T20:30:04.892675+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:00.971201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.897200+0000","last_change":"2023-10-10T20:30:03.887548+0000","last_active":"2023-10-10T20:30:04.897200+0000","last_peered":"2023-10-10T20:30:04.897200+0000","last_clean":"2023-10-10T20:30:04.897200+0000","last_became_active":"2023-10-10T20:30:03.886804+0000","last_became_peered":"2023-10-10T20:30:03.886804+0000","last_unstale":"2023-10-10T20:30:04.897200+0000","last_undegraded":"2023-10-10T20:30:04.897200+0000","last_fullsized":"2023-10-10T20:30:04.897200+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T23:06:48.718804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.892032+0000","last_change":"2023-10-10T20:30:04.892032+0000","last_active":"2023-10-10T20:30:04.892032+0000","last_peered":"2023-10-10T20:30:04.892032+0000","last_clean":"2023-10-10T20:30:04.892032+0000","last_became_active":"2023-10-10T20:30:04.891506+0000","last_became_peered":"2023-10-10T20:30:04.891506+0000","last_unstale":"2023-10-10T20:30:04.892032+0000","last_undegraded":"2023-10-10T20:30:04.892032+0000","last_fullsized":"2023-10-10T20:30:04.892032+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:55:25.955509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887222+0000","last_change":"2023-10-10T20:30:03.889244+0000","last_active":"2023-10-10T20:30:04.887222+0000","last_peered":"2023-10-10T20:30:04.887222+0000","last_clean":"2023-10-10T20:30:04.887222+0000","last_became_active":"2023-10-10T20:30:03.888691+0000","last_became_peered":"2023-10-10T20:30:03.888691+0000","last_unstale":"2023-10-10T20:30:04.887222+0000","last_undegraded":"2023-10-10T20:30:04.887222+0000","last_fullsized":"2023-10-10T20:30:04.887222+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898933+0000","last_change":"2023-10-10T20:30:04.898933+0000","last_active":"2023-10-10T20:30:04.898933+0000","last_peered":"2023-10-10T20:30:04.898933+0000","last_clean":"2023-10-10T20:30:04.898933+0000","last_became_active":"2023-10-10T20:30:04.897632+0000","last_became_peered":"2023-10-10T20:30:04.897632+0000","last_unstale":"2023-10-10T20:30:04.898933+0000","last_undegraded":"2023-10-10T20:30:04.898933+0000","last_fullsized":"2023-10-10T20:30:04.898933+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:19:18.390010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-10-10T20:30:03.890659+0000","last_change":"2023-10-10T20:30:03.890592+0000","last_active":"2023-10-10T20:30:03.890659+0000","last_peered":"2023-10-10T20:30:03.890659+0000","last_clean":"2023-10-10T20:30:03.890659+0000","last_became_active":"2023-10-10T20:30:03.890366+0000","last_became_peered":"2023-10-10T20:30:03.890366+0000","last_unstale":"2023-10-10T20:30:03.890659+0000","last_undegraded":"2023-10-10T20:30:03.890659+0000","last_fullsized":"2023-10-10T20:30:03.890659+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:13:27.528211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.899254+0000","last_change":"2023-10-10T20:30:04.899254+0000","last_active":"2023-10-10T20:30:04.899254+0000","last_peered":"2023-10-10T20:30:04.899254+0000","last_clean":"2023-10-10T20:30:04.899254+0000","last_became_active":"2023-10-10T20:30:04.898807+0000","last_became_peered":"2023-10-10T20:30:04.898807+0000","last_unstale":"2023-10-10T20:30:04.899254+0000","last_undegraded":"2023-10-10T20:30:04.899254+0000","last_fullsized":"2023-10-10T20:30:04.899254+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:31:22.948362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.883035+0000","last_change":"2023-10-10T20:30:03.894106+0000","last_active":"2023-10-10T20:30:04.883035+0000","last_peered":"2023-10-10T20:30:04.883035+0000","last_clean":"2023-10-10T20:30:04.883035+0000","last_became_active":"2023-10-10T20:30:03.893446+0000","last_became_peered":"2023-10-10T20:30:03.893446+0000","last_unstale":"2023-10-10T20:30:04.883035+0000","last_undegraded":"2023-10-10T20:30:04.883035+0000","last_fullsized":"2023-10-10T20:30:04.883035+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:02:21.867919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.898046+0000","last_change":"2023-10-10T20:30:04.895449+0000","last_active":"2023-10-10T20:30:04.898046+0000","last_peered":"2023-10-10T20:30:04.898046+0000","last_clean":"2023-10-10T20:30:04.898046+0000","last_became_active":"2023-10-10T20:30:04.894865+0000","last_became_peered":"2023-10-10T20:30:04.894865+0000","last_unstale":"2023-10-10T20:30:04.898046+0000","last_undegraded":"2023-10-10T20:30:04.898046+0000","last_fullsized":"2023-10-10T20:30:04.898046+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:42:00.402654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.886645+0000","last_change":"2023-10-10T20:30:03.886710+0000","last_active":"2023-10-10T20:30:04.886645+0000","last_peered":"2023-10-10T20:30:04.886645+0000","last_clean":"2023-10-10T20:30:04.886645+0000","last_became_active":"2023-10-10T20:30:03.886484+0000","last_became_peered":"2023-10-10T20:30:03.886484+0000","last_unstale":"2023-10-10T20:30:04.886645+0000","last_undegraded":"2023-10-10T20:30:04.886645+0000","last_fullsized":"2023-10-10T20:30:04.886645+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:51:24.325871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.890830+0000","last_change":"2023-10-10T20:30:04.890424+0000","last_active":"2023-10-10T20:30:04.890830+0000","last_peered":"2023-10-10T20:30:04.890830+0000","last_clean":"2023-10-10T20:30:04.890830+0000","last_became_active":"2023-10-10T20:30:04.890185+0000","last_became_peered":"2023-10-10T20:30:04.890185+0000","last_unstale":"2023-10-10T20:30:04.890830+0000","last_undegraded":"2023-10-10T20:30:04.890830+0000","last_fullsized":"2023-10-10T20:30:04.890830+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:17:01.169446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.887185+0000","last_change":"2023-10-10T20:30:03.885345+0000","last_active":"2023-10-10T20:30:04.887185+0000","last_peered":"2023-10-10T20:30:04.887185+0000","last_clean":"2023-10-10T20:30:04.887185+0000","last_became_active":"2023-10-10T20:30:03.885068+0000","last_became_peered":"2023-10-10T20:30:03.885068+0000","last_unstale":"2023-10-10T20:30:04.887185+0000","last_undegraded":"2023-10-10T20:30:04.887185+0000","last_fullsized":"2023-10-10T20:30:04.887185+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:54.029327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.902554+0000","last_change":"2023-10-10T20:30:04.902135+0000","last_active":"2023-10-10T20:30:04.902554+0000","last_peered":"2023-10-10T20:30:04.902554+0000","last_clean":"2023-10-10T20:30:04.902554+0000","last_became_active":"2023-10-10T20:30:04.901263+0000","last_became_peered":"2023-10-10T20:30:04.901263+0000","last_unstale":"2023-10-10T20:30:04.902554+0000","last_undegraded":"2023-10-10T20:30:04.902554+0000","last_fullsized":"2023-10-10T20:30:04.902554+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T22:36:03.064103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.882291+0000","last_change":"2023-10-10T20:30:03.892090+0000","last_active":"2023-10-10T20:30:04.882291+0000","last_peered":"2023-10-10T20:30:04.882291+0000","last_clean":"2023-10-10T20:30:04.882291+0000","last_became_active":"2023-10-10T20:30:03.891639+0000","last_became_peered":"2023-10-10T20:30:03.891639+0000","last_unstale":"2023-10-10T20:30:04.882291+0000","last_undegraded":"2023-10-10T20:30:04.882291+0000","last_fullsized":"2023-10-10T20:30:04.882291+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T03:35:54.029327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":8,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.896666+0000","last_change":"2023-10-10T20:30:04.896666+0000","last_active":"2023-10-10T20:30:04.896666+0000","last_peered":"2023-10-10T20:30:04.896666+0000","last_clean":"2023-10-10T20:30:04.896666+0000","last_became_active":"2023-10-10T20:30:04.896504+0000","last_became_peered":"2023-10-10T20:30:04.896504+0000","last_unstale":"2023-10-10T20:30:04.896666+0000","last_undegraded":"2023-10-10T20:30:04.896666+0000","last_fullsized":"2023-10-10T20:30:04.896666+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:03.873698+0000","last_clean_scrub_stamp":"2023-10-10T20:30:03.873698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:17:01.169446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.882949+0000","last_change":"2023-10-10T20:30:03.893203+0000","last_active":"2023-10-10T20:30:04.882949+0000","last_peered":"2023-10-10T20:30:04.882949+0000","last_clean":"2023-10-10T20:30:04.882949+0000","last_became_active":"2023-10-10T20:30:03.892521+0000","last_became_peered":"2023-10-10T20:30:03.892521+0000","last_unstale":"2023-10-10T20:30:04.882949+0000","last_undegraded":"2023-10-10T20:30:04.882949+0000","last_fullsized":"2023-10-10T20:30:04.882949+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:30:02.868389+0000","last_clean_scrub_stamp":"2023-10-10T20:30:02.868389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T00:11:34.012940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-10-10T20:30:04.885353+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:04.885353+0000","last_peered":"2023-10-10T20:30:04.885353+0000","last_clean":"2023-10-10T20:30:04.885353+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:04.885353+0000","last_undegraded":"2023-10-10T20:30:04.885353+0000","last_fullsized":"2023-10-10T20:30:04.885353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1542,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193769,"statfs":{"total":200000000,"available":198420386,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51700000000000002,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.30399999999999999,"15min":0.27500000000000002},"max":{"1min":0.68400000000000005,"5min":0.754,"15min":0.754},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.314},"max":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.38900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47999999999999998,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.40899999999999997,"15min":0.33100000000000002},"max":{"1min":0.77800000000000002,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Oct 10 20:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.29599999999999999},"max":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.80800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.50800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194342,"statfs":{"total":200000000,"available":199006694,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.434,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.73299999999999998,"5min":0.877,"15min":1.3140000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.70899999999999996,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.41599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78200000000000003,"5min":0.92100000000000004,"15min":1.865},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.39600000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.45700000000000002,"5min":0.32400000000000001,"15min":0.28199999999999997},"max":{"1min":0.82299999999999995,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.505}]},{"osd":3,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.748,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.35699999999999998},"max":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":15.862},"last":0.54800000000000004}]},{"osd":4,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45900000000000002,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.26400000000000001,"15min":0.222},"max":{"1min":0.85099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.46899999999999997},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.34300000000000003,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89500000000000002,"5min":1.02,"15min":1.0409999999999999},"last":0.67700000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:29:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72499999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194342,"statfs":{"total":200000000,"available":199006682,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33700000000000002,"15min":0.36199999999999999},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46899999999999997,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.38200000000000001,"15min":0.38100000000000001},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":1.647},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54000000000000004,"5min":0.627,"15min":1.524},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38900000000000001,"15min":0.40600000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.28100000000000003,"15min":0.253},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40100000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.59299999999999997,"5min":0.627,"15min":1.673},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73399999999999999,"5min":0.84599999999999997,"15min":1.633},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.34699999999999998},"max":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":1.506},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.70499999999999996,"5min":0.84899999999999998,"15min":1.696},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60699999999999998,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.623},"min":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.748,"5min":0.877,"15min":1.7490000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.46400000000000002,"5min":0.45200000000000001,"15min":0.39400000000000002},"max":{"1min":0.78300000000000003,"5min":0.98899999999999999,"15min":1.585},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77400000000000002,"5min":1.0009999999999999,"15min":1.478},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.77800000000000002,"5min":0.93500000000000005,"15min":1.722},"last":0.46700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193773,"statfs":{"total":200000000,"available":198423647,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35799999999999998,"15min":0.40899999999999997},"min":{"1min":0.23799999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":1.8740000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39900000000000002,"15min":0.45500000000000002},"min":{"1min":0.29999999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.39200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.377,"15min":0.436},"min":{"1min":0.25600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.40600000000000003},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51600000000000001,"5min":0.57299999999999995,"15min":5.891},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.35599999999999998,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.64100000000000001,"5min":0.68899999999999995,"15min":1.462},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.32900000000000001},"max":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.61699999999999999},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35599999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.79700000000000004},"min":{"1min":0.42599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.873,"5min":0.873,"15min":9.7260000000000009},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":1.3799999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36299999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":1.4970000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.40799999999999997,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.79200000000000004,"5min":0.79900000000000004,"15min":5.6379999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Oct 10 20:29:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.76100000000000001,"5min":0.84699999999999998,"15min":5.758},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":962,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199013925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.372},"min":{"1min":0.22,"5min":0.189,"15min":0.189},"max":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":2.4409999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.42699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23400000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.26100000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.38700000000000001},"min":{"1min":0.23300000000000001,"5min":0.23000000000000001,"15min":0.22},"max":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.224},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.44600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":11.815},"last":0.307},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.45800000000000002},"min":{"1min":0.317,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.28100000000000003},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.65500000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.873,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.84299999999999997,"5min":0.93600000000000005,"15min":6.399},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.29499999999999998},"max":{"1min":0.88900000000000001,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Oct 10 20:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.441,"5min":0.38800000000000001,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.62,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.92300000000000004,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":963,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194348,"statfs":{"total":200000000,"available":199012970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32100000000000001,"15min":0.311},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.499,"5min":0.622,"15min":0.622},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.32800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36799999999999999,"15min":0.34999999999999998},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.51800000000000002,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.372},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.38700000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.751},"last":0.34699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.42199999999999999,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.25,"15min":0.23300000000000001},"max":{"1min":0.56100000000000005,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.432,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.25900000000000001},"max":{"1min":0.55200000000000005,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.32100000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.77500000000000002,"5min":0.875,"15min":0.875},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.83899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.56100000000000005},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.38700000000000001,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61399999999999999,"15min":0.58399999999999996},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.877,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.59099999999999997},"min":{"1min":0.499,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":971,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194340,"statfs":{"total":200000000,"available":199005110,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.72599999999999998,"5min":0.76400000000000001,"15min":0.85399999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.51500000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.753,"15min":0.85899999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.78200000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.81599999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.51700000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.33700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46200000000000002,"15min":0.46700000000000003},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70699999999999996,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.63}]},{"osd":6,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48699999999999999,"15min":0.47899999999999998},"min":{"1min":0.34399999999999997,"5min":0.28999999999999998,"15min":0.26100000000000001},"max":{"1min":0.68400000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.88700000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:29:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.30399999999999999,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.92600000000000005,"15min":0.98599999999999999},"last":0.60599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":970,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194341,"statfs":{"total":200000000,"available":199006140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29099999999999998,"15min":0.26400000000000001},"max":{"1min":0.66600000000000004,"5min":0.77200000000000002,"15min":0.91200000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.66700000000000004,"5min":2.105,"15min":2.105},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54600000000000004,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.32000000000000001,"15min":0.255},"max":{"1min":0.67600000000000005,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57599999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.65200000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.38},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.47199999999999998,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.68500000000000005,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.56599999999999995},"min":{"1min":0.48499999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45700000000000002,"15min":0.48399999999999999},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":0.76900000000000002,"15min":0.79900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.66400000000000003,"5min":0.80800000000000005,"15min":0.875},"last":0.47499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.498,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Oct 10 20:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.55800000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.63500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.34799999999999998},"max":{"1min":0.73099999999999998,"5min":0.82099999999999995,"15min":0.84699999999999998},"last":0.77900000000000003}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:09.319 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-10T20:30:09.319 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-10T20:30:09.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.472+0000 7f7733dc1700 1 Processor -- start 2023-10-10T20:30:09.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 -- start start 2023-10-10T20:30:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f772c159ec0 0x7f772c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f772c06acb0 0x7f772c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0b0790 con 0x7f772c15a290 2023-10-10T20:30:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0ce1f0 con 0x7f772c159ec0 2023-10-10T20:30:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.473+0000 7f7733dc1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0b0b20 con 0x7f772c06acb0 2023-10-10T20:30:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f772c159ec0 0x7f772c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f773135c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f772c15a290 0x7f772c06a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57424/0 (socket says 172.21.15.148:57424) 2023-10-10T20:30:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f773135c700 1 -- 172.21.15.148:0/2751377784 learned_addr learned my addr 172.21.15.148:0/2751377784 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f773235e700 1 --2- 172.21.15.148:0/2751377784 >> v2:172.21.15.161:3300/0 conn(0x7f772c06acb0 0x7f772c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 -- 172.21.15.148:0/2751377784 >> v2:172.21.15.161:3300/0 conn(0x7f772c06acb0 msgr2=0x7f772c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 --2- 172.21.15.148:0/2751377784 >> v2:172.21.15.161:3300/0 conn(0x7f772c06acb0 0x7f772c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 -- 172.21.15.148:0/2751377784 >> v1:172.21.15.148:6789/0 conn(0x7f772c15a290 legacy=0x7f772c06a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7730b5b700 1 -- 172.21.15.148:0/2751377784 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024009721 0 0) 0x7f772c0b0790 con 0x7f772c15a290 2023-10-10T20:30:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 -- 172.21.15.148:0/2751377784 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0b04b0 con 0x7f772c159ec0 2023-10-10T20:30:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7731b5d700 1 --2- 172.21.15.148:0/2751377784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c159ec0 0x7f772c0af940 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f772c0711f0 tx=0x7f7728009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b73e49ddd7d9f8b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7730b5b700 1 -- 172.21.15.148:0/2751377784 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800e070 con 0x7f772c159ec0 2023-10-10T20:30:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7730b5b700 1 -- 172.21.15.148:0/2751377784 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772800bb60 con 0x7f772c159ec0 2023-10-10T20:30:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.474+0000 7f7730b5b700 1 -- 172.21.15.148:0/2751377784 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728012430 con 0x7f772c159ec0 2023-10-10T20:30:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 -- 172.21.15.148:0/2751377784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c159ec0 msgr2=0x7f772c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 --2- 172.21.15.148:0/2751377784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c159ec0 0x7f772c0af940 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f772c0711f0 tx=0x7f7728009780 comp rx=0 tx=0).stop 2023-10-10T20:30:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 -- 172.21.15.148:0/2751377784 shutdown_connections 2023-10-10T20:30:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 --2- 172.21.15.148:0/2751377784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c159ec0 0x7f772c0af940 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 --2- 172.21.15.148:0/2751377784 >> v2:172.21.15.161:3300/0 conn(0x7f772c06acb0 0x7f772c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 -- 172.21.15.148:0/2751377784 >> 172.21.15.148:0/2751377784 conn(0x7f772c0d1880 msgr2=0x7f772c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 -- 172.21.15.148:0/2751377784 shutdown_connections 2023-10-10T20:30:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.475+0000 7f7733dc1700 1 -- 172.21.15.148:0/2751377784 wait complete. 2023-10-10T20:30:09.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.476+0000 7f7733dc1700 1 Processor -- start 2023-10-10T20:30:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.476+0000 7f7733dc1700 1 -- start start 2023-10-10T20:30:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f772c159ec0 0x7f772c16a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f772c15a290 0x7f772c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c16be30 con 0x7f772c06acb0 2023-10-10T20:30:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c16b760 con 0x7f772c15a290 2023-10-10T20:30:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c16b1a0 con 0x7f772c159ec0 2023-10-10T20:30:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33476/0 (socket says 172.21.15.148:33476) 2023-10-10T20:30:09.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 -- 172.21.15.148:0/1343243943 learned_addr learned my addr 172.21.15.148:0/1343243943 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f773135c700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f772c159ec0 0x7f772c16a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f773235e700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f772c15a290 0x7f772c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 -- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f772c159ec0 msgr2=0x7f772c16a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f772c159ec0 0x7f772c16a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 -- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f772c15a290 msgr2=0x7f772c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f772c15a290 0x7f772c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 -- 172.21.15.148:0/1343243943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7728009a20 con 0x7f772c06acb0 2023-10-10T20:30:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7731b5d700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7728009fd0 tx=0x7f7728019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728012c50 con 0x7f772c06acb0 2023-10-10T20:30:09.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728012db0 con 0x7f772c06acb0 2023-10-10T20:30:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280276c0 con 0x7f772c06acb0 2023-10-10T20:30:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.477+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c16b480 con 0x7f772c06acb0 2023-10-10T20:30:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.478+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0b3350 con 0x7f772c06acb0 2023-10-10T20:30:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.479+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c16c110 con 0x7f772c06acb0 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.479+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7728024070 con 0x7f772c06acb0 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.479+0000 7f7721ffb700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 0x7f77100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.479+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7728086900 con 0x7f772c06acb0 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.479+0000 7f773135c700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 0x7f77100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.480+0000 7f773135c700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 0x7f77100693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f772c0b2420 tx=0x7f771c007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.484+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7728053b70 con 0x7f772c06acb0 2023-10-10T20:30:09.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.740+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f772c0b3540 con 0x7f772c06acb0 2023-10-10T20:30:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.743+0000 7f7721ffb700 1 -- 172.21.15.148:0/1343243943 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7728057190 con 0x7f772c06acb0 2023-10-10T20:30:09.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 msgr2=0x7f77100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 0x7f77100693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f772c0b2420 tx=0x7f771c007f50 comp rx=0 tx=0).stop 2023-10-10T20:30:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 msgr2=0x7f772c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7728009fd0 tx=0x7f7728019000 comp rx=0 tx=0).stop 2023-10-10T20:30:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 shutdown_connections 2023-10-10T20:30:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7710066f30 0x7f77100693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f772c15a290 0x7f772c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772c06acb0 0x7f772c0b2e10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 --2- 172.21.15.148:0/1343243943 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f772c159ec0 0x7f772c16a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 >> 172.21.15.148:0/1343243943 conn(0x7f772c0d1880 msgr2=0x7f772c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:09.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 shutdown_connections 2023-10-10T20:30:09.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.746+0000 7f7733dc1700 1 -- 172.21.15.148:0/1343243943 wait complete. 2023-10-10T20:30:09.750 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2023-10-10T20:30:09.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.872+0000 7fda19cb2700 1 Processor -- start 2023-10-10T20:30:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.872+0000 7fda19cb2700 1 -- start start 2023-10-10T20:30:09.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda19cb2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda14157870 0x7fda1415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda19cb2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda14157c40 0x7fda14158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda19cb2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1415cbb0 con 0x7fda1415a620 2023-10-10T20:30:09.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda19cb2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1415bfa0 con 0x7fda14157c40 2023-10-10T20:30:09.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda19cb2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1415c280 con 0x7fda14157870 2023-10-10T20:30:09.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda13fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda14157c40 0x7fda14158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda13fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda14157c40 0x7fda14158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33492/0 (socket says 172.21.15.148:33492) 2023-10-10T20:30:09.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda13fff700 1 -- 172.21.15.148:0/2032461442 learned_addr learned my addr 172.21.15.148:0/2032461442 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:09.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.873+0000 7fda137fe700 1 --2- 172.21.15.148:0/2032461442 >> v2:172.21.15.161:3300/0 conn(0x7fda14157870 0x7fda1415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda13fff700 1 -- 172.21.15.148:0/2032461442 >> v2:172.21.15.161:3300/0 conn(0x7fda14157870 msgr2=0x7fda1415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda13fff700 1 --2- 172.21.15.148:0/2032461442 >> v2:172.21.15.161:3300/0 conn(0x7fda14157870 0x7fda1415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda13fff700 1 -- 172.21.15.148:0/2032461442 >> v1:172.21.15.161:6789/0 conn(0x7fda1415a620 legacy=0x7fda1415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda13fff700 1 -- 172.21.15.148:0/2032461442 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1415c560 con 0x7fda14157c40 2023-10-10T20:30:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda13fff700 1 --2- 172.21.15.148:0/2032461442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda14158110 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fda14069d80 tx=0x7fda0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f455903e698f7a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda127fc700 1 -- 172.21.15.148:0/2032461442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c009e70 con 0x7fda14157c40 2023-10-10T20:30:09.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda127fc700 1 -- 172.21.15.148:0/2032461442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c00b860 con 0x7fda14157c40 2023-10-10T20:30:09.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda127fc700 1 -- 172.21.15.148:0/2032461442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00bc00 con 0x7fda14157c40 2023-10-10T20:30:09.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 -- 172.21.15.148:0/2032461442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 msgr2=0x7fda14158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 --2- 172.21.15.148:0/2032461442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda14158110 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fda14069d80 tx=0x7fda0c009250 comp rx=0 tx=0).stop 2023-10-10T20:30:09.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 -- 172.21.15.148:0/2032461442 shutdown_connections 2023-10-10T20:30:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 --2- 172.21.15.148:0/2032461442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda14158110 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 --2- 172.21.15.148:0/2032461442 >> v2:172.21.15.161:3300/0 conn(0x7fda14157870 0x7fda1415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 -- 172.21.15.148:0/2032461442 >> 172.21.15.148:0/2032461442 conn(0x7fda14068ce0 msgr2=0x7fda140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 -- 172.21.15.148:0/2032461442 shutdown_connections 2023-10-10T20:30:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.874+0000 7fda19cb2700 1 -- 172.21.15.148:0/2032461442 wait complete. 2023-10-10T20:30:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 Processor -- start 2023-10-10T20:30:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 -- start start 2023-10-10T20:30:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda14157870 0x7fda1413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1415a620 0x7fda140b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda140b1400 con 0x7fda14157c40 2023-10-10T20:30:09.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda140b07f0 con 0x7fda1415a620 2023-10-10T20:30:09.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda19cb2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda140b0ad0 con 0x7fda14157870 2023-10-10T20:30:09.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.876+0000 7fda12ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33500/0 (socket says 172.21.15.148:33500) 2023-10-10T20:30:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 -- 172.21.15.148:0/587168055 learned_addr learned my addr 172.21.15.148:0/587168055 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda13fff700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1415a620 0x7fda140b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda137fe700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda14157870 0x7fda1413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 -- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda14157870 msgr2=0x7fda1413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda14157870 0x7fda1413f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 -- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1415a620 msgr2=0x7fda140b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1415a620 0x7fda140b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 -- 172.21.15.148:0/587168055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0094f0 con 0x7fda14157c40 2023-10-10T20:30:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda12ffd700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c590 tx=0x7fda08007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011590 con 0x7fda14157c40 2023-10-10T20:30:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08011710 con 0x7fda14157c40 2023-10-10T20:30:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda140b0db0 con 0x7fda14157c40 2023-10-10T20:30:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08011ad0 con 0x7fda14157c40 2023-10-10T20:30:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.877+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1414ff20 con 0x7fda14157c40 2023-10-10T20:30:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.878+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda140b1570 con 0x7fda14157c40 2023-10-10T20:30:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.879+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda0801b3b0 con 0x7fda14157c40 2023-10-10T20:30:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.879+0000 7fd9fb7fe700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.879+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda08083810 con 0x7fda14157c40 2023-10-10T20:30:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.879+0000 7fda137fe700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.879+0000 7fda137fe700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 0x7fda000693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda0400f570 tx=0x7fda0400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:09.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:09.883+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda08050a80 con 0x7fda14157c40 2023-10-10T20:30:10.111 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.110+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda1404c3c0 con 0x7fda14157c40 2023-10-10T20:30:10.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.110+0000 7fd9fb7fe700 1 -- 172.21.15.148:0/587168055 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fda08017070 con 0x7fda14157c40 2023-10-10T20:30:10.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:10.113 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 msgr2=0x7fda000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 0x7fda000693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda0400f570 tx=0x7fda0400b040 comp rx=0 tx=0).stop 2023-10-10T20:30:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 msgr2=0x7fda1414f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c590 tx=0x7fda08007f50 comp rx=0 tx=0).stop 2023-10-10T20:30:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 shutdown_connections 2023-10-10T20:30:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1415a620 0x7fda140b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda14157870 0x7fda1413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 --2- 172.21.15.148:0/587168055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda14157c40 0x7fda1414f9e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 >> 172.21.15.148:0/587168055 conn(0x7fda14068ce0 msgr2=0x7fda140b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 shutdown_connections 2023-10-10T20:30:10.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.112+0000 7fda19cb2700 1 -- 172.21.15.148:0/587168055 wait complete. 2023-10-10T20:30:10.117 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:10.177 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-10-10T20:30:10.175+0000 7fb1c683d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:30:10.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-10T20:30:04.260506+0000', 'modified': '2023-10-10T20:30:07.906439+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8139}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8139': {'gid': 8139, 'name': 'b', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/1942118864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6833', 'nonce': 1942118864}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 1942118864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-10T20:30:10.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-10T20:30:10.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-10T20:30:10.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.645+0000 7f9979d84700 1 Processor -- start 2023-10-10T20:30:10.647 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.645+0000 7f9979d84700 1 -- start start 2023-10-10T20:30:10.649 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f9979d84700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9974152010 0x7f997415b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f9979d84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f99740cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f9979d84700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997406c010 con 0x7f99741523e0 2023-10-10T20:30:10.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f9979d84700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997406bd10 con 0x7f9974152010 2023-10-10T20:30:10.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f9979d84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997415c060 con 0x7f9974158e50 2023-10-10T20:30:10.650 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f99737fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9974152010 0x7f997415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f99737fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9974152010 0x7f997415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33502/0 (socket says 172.21.15.148:33502) 2023-10-10T20:30:10.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.646+0000 7f99737fe700 1 -- 172.21.15.148:0/1704765439 learned_addr learned my addr 172.21.15.148:0/1704765439 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f9973fff700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f99740cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99737fe700 1 -- 172.21.15.148:0/1704765439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 msgr2=0x7f99740cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.651 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99737fe700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f99740cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99737fe700 1 -- 172.21.15.148:0/1704765439 >> v1:172.21.15.161:6789/0 conn(0x7f99741523e0 legacy=0x7f997406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99737fe700 1 -- 172.21.15.148:0/1704765439 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997415c340 con 0x7f9974152010 2023-10-10T20:30:10.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f9973fff700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f99740cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:10.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99737fe700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f997415b810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99640099b0 tx=0x7f9964009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cf93fca4bf0f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.652 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99727fc700 1 -- 172.21.15.148:0/1704765439 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400b820 con 0x7f9974152010 2023-10-10T20:30:10.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99727fc700 1 -- 172.21.15.148:0/1704765439 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400b980 con 0x7f9974152010 2023-10-10T20:30:10.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f99727fc700 1 -- 172.21.15.148:0/1704765439 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400bd20 con 0x7f9974152010 2023-10-10T20:30:10.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f9979d84700 1 -- 172.21.15.148:0/1704765439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 msgr2=0x7f997415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.647+0000 7f9979d84700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f997415b810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99640099b0 tx=0x7f9964009250 comp rx=0 tx=0).stop 2023-10-10T20:30:10.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 -- 172.21.15.148:0/1704765439 shutdown_connections 2023-10-10T20:30:10.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f997415b810 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 --2- 172.21.15.148:0/1704765439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f99740cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 -- 172.21.15.148:0/1704765439 >> 172.21.15.148:0/1704765439 conn(0x7f9974069870 msgr2=0x7f9974069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 -- 172.21.15.148:0/1704765439 shutdown_connections 2023-10-10T20:30:10.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.648+0000 7f9979d84700 1 -- 172.21.15.148:0/1704765439 wait complete. 2023-10-10T20:30:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.649+0000 7f9979d84700 1 Processor -- start 2023-10-10T20:30:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.649+0000 7f9979d84700 1 -- start start 2023-10-10T20:30:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.649+0000 7f9979d84700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.649+0000 7f9979d84700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99741523e0 0x7f99741531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f997414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974154210 con 0x7f9974152010 2023-10-10T20:30:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974153720 con 0x7f99741523e0 2023-10-10T20:30:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740b2cc0 con 0x7f9974158e50 2023-10-10T20:30:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33508/0 (socket says 172.21.15.148:33508) 2023-10-10T20:30:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 -- 172.21.15.148:0/625651365 learned_addr learned my addr 172.21.15.148:0/625651365 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9972ffd700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99741523e0 0x7f99741531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9973fff700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f997414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 -- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 msgr2=0x7f997414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f997414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 -- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99741523e0 msgr2=0x7f99741531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99741523e0 0x7f99741531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 -- 172.21.15.148:0/625651365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99640094f0 con 0x7f9974152010 2023-10-10T20:30:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9973fff700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f997414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f99737fe700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9964009290 tx=0x7f9964016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f995b7fe700 1 -- 172.21.15.148:0/625651365 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964011cc0 con 0x7f9974152010 2023-10-10T20:30:10.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f995b7fe700 1 -- 172.21.15.148:0/625651365 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964020070 con 0x7f9974152010 2023-10-10T20:30:10.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740b2fa0 con 0x7f9974152010 2023-10-10T20:30:10.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f995b7fe700 1 -- 172.21.15.148:0/625651365 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996401b6a0 con 0x7f9974152010 2023-10-10T20:30:10.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.650+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974153ea0 con 0x7f9974152010 2023-10-10T20:30:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.652+0000 7f995b7fe700 1 -- 172.21.15.148:0/625651365 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9964022030 con 0x7f9974152010 2023-10-10T20:30:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.652+0000 7f995b7fe700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.652+0000 7f9972ffd700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.652+0000 7f995b7fe700 1 -- 172.21.15.148:0/625651365 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9964086470 con 0x7f9974152010 2023-10-10T20:30:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.653+0000 7f9972ffd700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 0x7f99600693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f99740c8810 tx=0x7f9968007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.654+0000 7f998307db80 1 Processor -- start 2023-10-10T20:30:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.654+0000 7f998307db80 1 -- start start 2023-10-10T20:30:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5bfa50 0x55dbdd5bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd5e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd516a70 con 0x55dbdd5e8f60 2023-10-10T20:30:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd514270 con 0x55dbdd5bfa50 2023-10-10T20:30:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd510060 con 0x55dbdd609d00 2023-10-10T20:30:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33518/0 (socket says 172.21.15.148:33518) 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 -- 172.21.15.148:0/2200379534 learned_addr learned my addr 172.21.15.148:0/2200379534 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f99737fe700 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5bfa50 0x55dbdd5bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9972ffd700 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd5e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 -- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 msgr2=0x55dbdd5e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd5e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 -- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5bfa50 msgr2=0x55dbdd5bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5bfa50 0x55dbdd5bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.655+0000 7f9973fff700 1 -- 172.21.15.148:0/2200379534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbdd3aa270 con 0x55dbdd5e8f60 2023-10-10T20:30:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f9973fff700 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e520 tx=0x7f996c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5a951d8a80faae server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f9958ff9700 1 -- 172.21.15.148:0/2200379534 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c018430 con 0x55dbdd5e8f60 2023-10-10T20:30:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f9958ff9700 1 -- 172.21.15.148:0/2200379534 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c013070 con 0x55dbdd5e8f60 2023-10-10T20:30:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f9958ff9700 1 -- 172.21.15.148:0/2200379534 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0187c0 con 0x55dbdd5e8f60 2023-10-10T20:30:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 -- 172.21.15.148:0/2200379534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 msgr2=0x55dbdd5bbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e520 tx=0x7f996c007a50 comp rx=0 tx=0).stop 2023-10-10T20:30:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 -- 172.21.15.148:0/2200379534 shutdown_connections 2023-10-10T20:30:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5bbcf0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5bfa50 0x55dbdd5bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 --2- 172.21.15.148:0/2200379534 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd5e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 -- 172.21.15.148:0/2200379534 >> 172.21.15.148:0/2200379534 conn(0x55dbdd5c1370 msgr2=0x55dbdd5c1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 -- 172.21.15.148:0/2200379534 shutdown_connections 2023-10-10T20:30:10.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.656+0000 7f998307db80 1 -- 172.21.15.148:0/2200379534 wait complete. 2023-10-10T20:30:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.657+0000 7f998307db80 1 -- start start 2023-10-10T20:30:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.657+0000 7f998307db80 1 Processor -- start 2023-10-10T20:30:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd615fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd5ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd4bb890 con 0x55dbdd5e8f60 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd4d6ea0 con 0x55dbdd609d00 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd392e30 con 0x55dbdd5bfa50 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f998307db80 10 client.0 _ll_register_callbacks cb 0x55dbdd583110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33520/0 (socket says 172.21.15.148:33520) 2023-10-10T20:30:10.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 -- 172.21.15.148:0/345643002 learned_addr learned my addr 172.21.15.148:0/345643002 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9973fff700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd5ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f99737fe700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd615fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 -- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 msgr2=0x55dbdd615fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd615fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 -- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 msgr2=0x55dbdd5ef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd5ef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 -- 172.21.15.148:0/345643002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c007d10 con 0x55dbdd5e8f60 2023-10-10T20:30:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9972ffd700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f997406dde0 tx=0x7f9968015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280ee4e8b371d118 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9951ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:30:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.658+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800f620 con 0x55dbdd5e8f60 2023-10-10T20:30:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f996801c040 con 0x55dbdd5e8f60 2023-10-10T20:30:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbdd5b4b70 con 0x55dbdd5e8f60 2023-10-10T20:30:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9968014e70 con 0x55dbdd5e8f60 2023-10-10T20:30:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 10 client.8277 fetch_fsmap learned FSMap version 85 2023-10-10T20:30:10.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55dbdd34cfe0 con 0x55dbdd5e8f60 2023-10-10T20:30:10.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dbdd5b4b70 con 0x55dbdd5e8f60 2023-10-10T20:30:10.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 85) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9968021640 con 0x55dbdd5e8f60 2023-10-10T20:30:10.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 10 client.8277 fetch_fsmap finished waiting for FSMap version 85 2023-10-10T20:30:10.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 10 client.8277 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 20 client.8277 populate_metadata read hostname 'smithi148' 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd5fe2b0 0x55dbdd5fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 4 client.8277 mds_command: new command op to 8139 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 --> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbdd591aa0 con 0x55dbdd5fe2b0 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.659+0000 7f99737fe700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd5fe2b0 0x55dbdd5fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f9951ffb700 10 client.8277 ms_handle_connect on v2:172.21.15.161:6833/1942118864 2023-10-10T20:30:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f99737fe700 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd5fe2b0 0x55dbdd5fe680 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86966fcc62b40a90 server_cookie=ad0623aaa8094d74 in_seq=0 out_seq=0 2023-10-10T20:30:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f9951ffb700 1 -- 172.21.15.148:0/345643002 <== mds.0 v2:172.21.15.161:6833/1942118864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55dbdd591aa0 con 0x55dbdd5fe2b0 2023-10-10T20:30:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f9951ffb700 10 client.8277 handle_command_reply: tid=0 2023-10-10T20:30:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 client.8277 shutdown 2023-10-10T20:30:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 msgr2=0x55dbdd5eee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f997406dde0 tx=0x7f9968015000 comp rx=0 tx=0).stop 2023-10-10T20:30:10.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 shutdown_connections 2023-10-10T20:30:10.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd5ef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd615fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd5fe2b0 0x55dbdd5fe680 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 --2- 172.21.15.148:0/345643002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd5eee80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f9951ffb700 0 client.8277 ms_handle_reset on v2:172.21.15.161:6833/1942118864 2023-10-10T20:30:10.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 >> 172.21.15.148:0/345643002 conn(0x55dbdd5c1370 msgr2=0x55dbdd5bc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 shutdown_connections 2023-10-10T20:30:10.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.660+0000 7f998307db80 1 -- 172.21.15.148:0/345643002 wait complete. 2023-10-10T20:30:10.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.661+0000 7f998307db80 20 client.8277 trim_cache size 0 max 16384 2023-10-10T20:30:10.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.677+0000 7f998307db80 1 Processor -- start 2023-10-10T20:30:10.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.677+0000 7f998307db80 1 -- start start 2023-10-10T20:30:10.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd5bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd61fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd6051e0 con 0x55dbdd5e8f60 2023-10-10T20:30:10.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd5c2a10 con 0x55dbdd609d00 2023-10-10T20:30:10.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd602ea0 con 0x55dbdd5bfa50 2023-10-10T20:30:10.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33530/0 (socket says 172.21.15.148:33530) 2023-10-10T20:30:10.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 -- 172.21.15.148:0/670431523 learned_addr learned my addr 172.21.15.148:0/670431523 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 -- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 msgr2=0x55dbdd5bbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd5bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 -- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 msgr2=0x55dbdd61fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.695 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.678+0000 7f9972ffd700 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd61fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f9972ffd700 1 -- 172.21.15.148:0/670431523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbdd4de780 con 0x55dbdd5e8f60 2023-10-10T20:30:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f9972ffd700 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f996800ffe0 tx=0x7f9968021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead014a25ab8e178 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f99597fa700 1 -- 172.21.15.148:0/670431523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968021ba0 con 0x55dbdd5e8f60 2023-10-10T20:30:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f99597fa700 1 -- 172.21.15.148:0/670431523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9968021640 con 0x55dbdd5e8f60 2023-10-10T20:30:10.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f99597fa700 1 -- 172.21.15.148:0/670431523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680217b0 con 0x55dbdd5e8f60 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 -- 172.21.15.148:0/670431523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 msgr2=0x55dbdd61f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f996800ffe0 tx=0x7f9968021000 comp rx=0 tx=0).stop 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 -- 172.21.15.148:0/670431523 shutdown_connections 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd609d00 0x55dbdd61fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd5bfa50 0x55dbdd5bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 --2- 172.21.15.148:0/670431523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61f5a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 -- 172.21.15.148:0/670431523 >> 172.21.15.148:0/670431523 conn(0x55dbdd5c1370 msgr2=0x55dbdd5c1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 -- 172.21.15.148:0/670431523 shutdown_connections 2023-10-10T20:30:10.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.679+0000 7f998307db80 1 -- 172.21.15.148:0/670431523 wait complete. 2023-10-10T20:30:10.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.680+0000 7f998307db80 1 -- start start 2023-10-10T20:30:10.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.680+0000 7f998307db80 1 Processor -- start 2023-10-10T20:30:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd668ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd669e60 con 0x55dbdd5bfa50 2023-10-10T20:30:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd5e7a90 con 0x55dbdd5e8f60 2023-10-10T20:30:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbdd5e6e80 con 0x55dbdd609d00 2023-10-10T20:30:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 10 client.0 _ll_register_callbacks cb 0x55dbdd583110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-10T20:30:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33532/0 (socket says 172.21.15.148:33532) 2023-10-10T20:30:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 -- 172.21.15.148:0/3033298966 learned_addr learned my addr 172.21.15.148:0/3033298966 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9972ffd700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9973fff700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd668ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 -- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 msgr2=0x55dbdd668ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd668ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 -- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 msgr2=0x55dbdd61fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 -- 172.21.15.148:0/3033298966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9968014e70 con 0x55dbdd5bfa50 2023-10-10T20:30:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9972ffd700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f99737fe700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f99640030c0 tx=0x7f9964027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef132aa1ea5f0032 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9951ffb700 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2023-10-10T20:30:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964007ea0 con 0x55dbdd5bfa50 2023-10-10T20:30:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9964011e40 con 0x55dbdd5bfa50 2023-10-10T20:30:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f996c013070 con 0x55dbdd5bfa50 2023-10-10T20:30:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9964053a90 con 0x55dbdd5bfa50 2023-10-10T20:30:10.703 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 10 client.8283 fetch_fsmap learned FSMap version 85 2023-10-10T20:30:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55dbdd4c5470 con 0x55dbdd5bfa50 2023-10-10T20:30:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.681+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964062080 con 0x55dbdd5bfa50 2023-10-10T20:30:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 85) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f99640860b0 con 0x55dbdd5bfa50 2023-10-10T20:30:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 10 client.8283 fetch_fsmap finished waiting for FSMap version 85 2023-10-10T20:30:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 10 client.8283 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-10T20:30:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 20 client.8283 populate_metadata read hostname 'smithi148' 2023-10-10T20:30:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd603ae0 0x55dbdd669430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 4 client.8283 mds_command: new command op to 8139 tid=0[{"prefix": "status"}] 2023-10-10T20:30:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 --> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dbdd4b2e00 con 0x55dbdd603ae0 2023-10-10T20:30:10.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9972ffd700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd603ae0 0x55dbdd669430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9951ffb700 10 client.8283 ms_handle_connect on v2:172.21.15.161:6833/1942118864 2023-10-10T20:30:10.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9972ffd700 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd603ae0 0x55dbdd669430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc76478040dc911 server_cookie=7ea7e01784349774 in_seq=0 out_seq=0 2023-10-10T20:30:10.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9951ffb700 1 -- 172.21.15.148:0/3033298966 <== mds.0 v2:172.21.15.161:6833/1942118864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55dbdd4b2e00 con 0x55dbdd603ae0 2023-10-10T20:30:10.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9951ffb700 10 client.8283 handle_command_reply: tid=0 2023-10-10T20:30:10.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 client.8283 shutdown 2023-10-10T20:30:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 msgr2=0x55dbdd61fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f99640030c0 tx=0x7f9964027040 comp rx=0 tx=0).stop 2023-10-10T20:30:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 shutdown_connections 2023-10-10T20:30:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55dbdd609d00 0x55dbdd668ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:6833/1942118864,v1:172.21.15.161:6835/1942118864] conn(0x55dbdd603ae0 0x55dbdd669430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dbdd5bfa50 0x55dbdd61fab0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 --2- 172.21.15.148:0/3033298966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dbdd5e8f60 0x55dbdd61fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f9951ffb700 0 client.8283 ms_handle_reset on v2:172.21.15.161:6833/1942118864 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.682+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 >> 172.21.15.148:0/3033298966 conn(0x55dbdd5c1370 msgr2=0x55dbdd5bbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 shutdown_connections 2023-10-10T20:30:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f998307db80 1 -- 172.21.15.148:0/3033298966 wait complete. 2023-10-10T20:30:10.709 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f998307db80 20 client.8283 trim_cache size 0 max 16384 2023-10-10T20:30:10.709 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:10.709 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2023-10-10T20:30:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 8139, 2023-10-10T20:30:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2023-10-10T20:30:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2023-10-10T20:30:10.710 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.8023941380000004, 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 85, 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 178, 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 178, 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 10.913262007 2023-10-10T20:30:10.711 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:10.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 msgr2=0x7f99600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 0x7f99600693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f99740c8810 tx=0x7f9968007ea0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 msgr2=0x7f99740b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9964009290 tx=0x7f9964016000 comp rx=0 tx=0).stop 2023-10-10T20:30:10.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 shutdown_connections 2023-10-10T20:30:10.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9974158e50 0x7f997414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9974152010 0x7f99740b2630 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 --2- 172.21.15.148:0/625651365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99741523e0 0x7f99741531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 >> 172.21.15.148:0/625651365 conn(0x7f9974069870 msgr2=0x7f9974158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 shutdown_connections 2023-10-10T20:30:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.683+0000 7f9979d84700 1 -- 172.21.15.148:0/625651365 wait complete. 2023-10-10T20:30:10.716 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:30:10.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.829+0000 7efdd74d7700 1 Processor -- start 2023-10-10T20:30:10.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.829+0000 7efdd74d7700 1 -- start start 2023-10-10T20:30:10.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd74d7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdd00b1220 0x7efdd00c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd74d7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd015de60 con 0x7efdd00cb4f0 2023-10-10T20:30:10.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd74d7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd015e140 con 0x7efdd00b0e50 2023-10-10T20:30:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd74d7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd015e420 con 0x7efdd00b1220 2023-10-10T20:30:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd4a72700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdd00b1220 0x7efdd00c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd5a74700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efdd00cb4f0 0x7efdd00679d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57440/0 (socket says 172.21.15.148:57440) 2023-10-10T20:30:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd4a72700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efdd00b1220 0x7efdd00c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33544/0 (socket says 172.21.15.148:33544) 2023-10-10T20:30:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.830+0000 7efdd5a74700 1 -- 172.21.15.148:0/1810048332 learned_addr learned my addr 172.21.15.148:0/1810048332 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd4a72700 1 -- 172.21.15.148:0/1810048332 >> v1:172.21.15.161:6789/0 conn(0x7efdd00b0e50 legacy=0x7efdd006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd4a72700 1 -- 172.21.15.148:0/1810048332 >> v1:172.21.15.148:6789/0 conn(0x7efdd00cb4f0 legacy=0x7efdd00679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd4a72700 1 -- 172.21.15.148:0/1810048332 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd015e700 con 0x7efdd00b1220 2023-10-10T20:30:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdcffff700 1 -- 172.21.15.148:0/1810048332 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674603118 0 0) 0x7efdd015de60 con 0x7efdd00cb4f0 2023-10-10T20:30:10.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd4a72700 1 --2- 172.21.15.148:0/1810048332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c9180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7efdc40098e0 tx=0x7efdc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e5ef39bde395f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdcffff700 1 -- 172.21.15.148:0/1810048332 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4009e80 con 0x7efdd00b1220 2023-10-10T20:30:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdcffff700 1 -- 172.21.15.148:0/1810048332 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc400b820 con 0x7efdd00b1220 2023-10-10T20:30:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdcffff700 1 -- 172.21.15.148:0/1810048332 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400bba0 con 0x7efdd00b1220 2023-10-10T20:30:10.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd74d7700 1 -- 172.21.15.148:0/1810048332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 msgr2=0x7efdd00c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.831+0000 7efdd74d7700 1 --2- 172.21.15.148:0/1810048332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c9180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7efdc40098e0 tx=0x7efdc40091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.832+0000 7efdd74d7700 1 -- 172.21.15.148:0/1810048332 shutdown_connections 2023-10-10T20:30:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.832+0000 7efdd74d7700 1 --2- 172.21.15.148:0/1810048332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c9180 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.832+0000 7efdd74d7700 1 -- 172.21.15.148:0/1810048332 >> 172.21.15.148:0/1810048332 conn(0x7efdd0069870 msgr2=0x7efdd0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.832+0000 7efdd74d7700 1 -- 172.21.15.148:0/1810048332 shutdown_connections 2023-10-10T20:30:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.832+0000 7efdd74d7700 1 -- 172.21.15.148:0/1810048332 wait complete. 2023-10-10T20:30:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.833+0000 7efdd74d7700 1 Processor -- start 2023-10-10T20:30:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.833+0000 7efdd74d7700 1 -- start start 2023-10-10T20:30:10.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 0x7efdd014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efdd00cb4f0 0x7efdd00c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd015b110 con 0x7efdd00b1220 2023-10-10T20:30:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd015a500 con 0x7efdd00b0e50 2023-10-10T20:30:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd015a7e0 con 0x7efdd00cb4f0 2023-10-10T20:30:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33560/0 (socket says 172.21.15.148:33560) 2023-10-10T20:30:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 -- 172.21.15.148:0/2719945416 learned_addr learned my addr 172.21.15.148:0/2719945416 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd5273700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 0x7efdd014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.843 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd5a74700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efdd00cb4f0 0x7efdd00c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 -- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efdd00cb4f0 msgr2=0x7efdd00c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efdd00cb4f0 0x7efdd00c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 -- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 msgr2=0x7efdd014cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 0x7efdd014cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 -- 172.21.15.148:0/2719945416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc4009420 con 0x7efdd00b1220 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd5273700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 0x7efdd014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd4a72700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7efdc40030c0 tx=0x7efdc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400b990 con 0x7efdd00b1220 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc4020070 con 0x7efdd00b1220 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd015aac0 con 0x7efdd00b1220 2023-10-10T20:30:10.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc401b450 con 0x7efdd00b1220 2023-10-10T20:30:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.834+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd014d510 con 0x7efdd00b1220 2023-10-10T20:30:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.835+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd015b280 con 0x7efdd00b1220 2023-10-10T20:30:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.836+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdc4022030 con 0x7efdd00b1220 2023-10-10T20:30:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.836+0000 7efdcd7fa700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 0x7efdbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:10.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.836+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efdc40864d0 con 0x7efdd00b1220 2023-10-10T20:30:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.836+0000 7efdd5273700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 0x7efdbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.836+0000 7efdd5273700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 0x7efdbc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efdc0007fd0 tx=0x7efdc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:10.841+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdc4053740 con 0x7efdd00b1220 2023-10-10T20:30:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.070+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efdd015b250 con 0x7efdd00b1220 2023-10-10T20:30:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.070+0000 7efdcd7fa700 1 -- 172.21.15.148:0/2719945416 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7efdc4056d60 con 0x7efdd00b1220 2023-10-10T20:30:11.072 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:30:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 msgr2=0x7efdbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 0x7efdbc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efdc0007fd0 tx=0x7efdc0009040 comp rx=0 tx=0).stop 2023-10-10T20:30:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 msgr2=0x7efdd00c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7efdc40030c0 tx=0x7efdc4016000 comp rx=0 tx=0).stop 2023-10-10T20:30:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 shutdown_connections 2023-10-10T20:30:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efdbc066f30 0x7efdbc0693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efdd00cb4f0 0x7efdd00c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdd00b0e50 0x7efdd014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 --2- 172.21.15.148:0/2719945416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00b1220 0x7efdd00c5d80 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 >> 172.21.15.148:0/2719945416 conn(0x7efdd0069870 msgr2=0x7efdd014dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 shutdown_connections 2023-10-10T20:30:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.072+0000 7efdd74d7700 1 -- 172.21.15.148:0/2719945416 wait complete. 2023-10-10T20:30:11.085 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:30:11.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:30:11.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2023-10-10T20:30:11.087 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-10T20:30:11.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-10T20:30:11.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:30:11.088 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:30:11.088 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:30:11.088 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:30:11.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.089 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2023-10-10T20:30:11.104 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:30:11.104 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:30:11.105 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:30:11.105 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.105 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2023-10-10T20:30:11.105 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.105 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:11.106 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:11.106 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2023-10-10T20:30:11.106 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:11.106 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:11.106 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2023-10-10T20:30:11.107 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:11.107 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:11.107 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:30:11.107 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:11.107 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 41900sec preferred_lft 41900sec 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:11.108 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:30:11.109 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.109 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8888:aff:fea6:5ce/64 scope link 2023-10-10T20:30:11.109 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.109 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:30:11.109 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 7a:d1:45:b3:07:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:30:11.110 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::78d1:45ff:feb3:7ff/64 scope link 2023-10-10T20:30:11.110 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.110 INFO:teuthology.orchestra.run.smithi148.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:30:11.110 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether fa:e6:b2:08:eb:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:30:11.110 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::f8e6:b2ff:fe08:eb9b/64 scope link 2023-10-10T20:30:11.111 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:11.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.111 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:30:11.167 INFO:teuthology.orchestra.run.smithi148.stdout:1 2023-10-10T20:30:11.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.169 DEBUG:teuthology.orchestra.run.smithi148:> route 2023-10-10T20:30:11.227 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2023-10-10T20:30:11.227 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:30:11.228 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:30:11.228 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:30:11.228 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:30:11.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.230 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2023-10-10T20:30:11.230 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2023-10-10T20:30:11.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:30:11.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:30:11.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:30:11.234 DEBUG:teuthology.orchestra.run.smithi148:> ') 2023-10-10T20:30:11.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.328 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:30:11.342 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:30:11.343 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:30:11.343 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:30:11.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2023-10-10T20:30:11.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:30:11.408 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-10T20:30:11.462 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-10T20:30:11.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:30:11.463 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:30:11.517 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-10T20:30:11.518 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:30:11.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:11.598 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:11.599 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:11.617 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:30:11.618 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:30:11.619 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:30:11.638 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 0@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:30:11.639 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:30:11.640 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:30:11.641 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:30:11.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:30:11.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:11.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-10T20:30:11.707 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-10T20:30:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.860+0000 7f31da70a700 1 Processor -- start 2023-10-10T20:30:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.860+0000 7f31da70a700 1 -- start start 2023-10-10T20:30:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31da70a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f31d4068de0 0x7f31d40cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31da70a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31d4067660 0x7f31d4067a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31da70a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d40cf920 con 0x7f31d4159f80 2023-10-10T20:30:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31da70a700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d415a350 con 0x7f31d4067660 2023-10-10T20:30:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31da70a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d4069730 con 0x7f31d4068de0 2023-10-10T20:30:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31d4067660 0x7f31d4067a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31d4067660 0x7f31d4067a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33572/0 (socket says 172.21.15.148:33572) 2023-10-10T20:30:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 -- 172.21.15.148:0/3045109324 learned_addr learned my addr 172.21.15.148:0/3045109324 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d8f07700 1 --2- 172.21.15.148:0/3045109324 >> v2:172.21.15.161:3300/0 conn(0x7f31d4068de0 0x7f31d40cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.865 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 -- 172.21.15.148:0/3045109324 >> v2:172.21.15.161:3300/0 conn(0x7f31d4068de0 msgr2=0x7f31d40cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 --2- 172.21.15.148:0/3045109324 >> v2:172.21.15.161:3300/0 conn(0x7f31d4068de0 0x7f31d40cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 -- 172.21.15.148:0/3045109324 >> v1:172.21.15.161:6789/0 conn(0x7f31d4159f80 legacy=0x7f31d40cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 -- 172.21.15.148:0/3045109324 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d4067fa0 con 0x7f31d4067660 2023-10-10T20:30:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.861+0000 7f31d9f09700 1 --2- 172.21.15.148:0/3045109324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4067660 0x7f31d4067a60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f31d40649d0 tx=0x7f31cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43b687adfa5988f server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:11.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31cbfff700 1 -- 172.21.15.148:0/3045109324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00e070 con 0x7f31d4067660 2023-10-10T20:30:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31cbfff700 1 -- 172.21.15.148:0/3045109324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc00ba20 con 0x7f31d4067660 2023-10-10T20:30:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31cbfff700 1 -- 172.21.15.148:0/3045109324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0092e0 con 0x7f31d4067660 2023-10-10T20:30:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 -- 172.21.15.148:0/3045109324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4067660 msgr2=0x7f31d4067a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 --2- 172.21.15.148:0/3045109324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4067660 0x7f31d4067a60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f31d40649d0 tx=0x7f31cc009840 comp rx=0 tx=0).stop 2023-10-10T20:30:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 -- 172.21.15.148:0/3045109324 shutdown_connections 2023-10-10T20:30:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 --2- 172.21.15.148:0/3045109324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4067660 0x7f31d4067a60 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 --2- 172.21.15.148:0/3045109324 >> v2:172.21.15.161:3300/0 conn(0x7f31d4068de0 0x7f31d40cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 -- 172.21.15.148:0/3045109324 >> 172.21.15.148:0/3045109324 conn(0x7f31d406c700 msgr2=0x7f31d4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 -- 172.21.15.148:0/3045109324 shutdown_connections 2023-10-10T20:30:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.862+0000 7f31da70a700 1 -- 172.21.15.148:0/3045109324 wait complete. 2023-10-10T20:30:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.863+0000 7f31da70a700 1 Processor -- start 2023-10-10T20:30:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.863+0000 7f31da70a700 1 -- start start 2023-10-10T20:30:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d4067660 0x7f31d413fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4068de0 0x7f31d41586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4140320 con 0x7f31d4159f80 2023-10-10T20:30:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4159120 con 0x7f31d4067660 2023-10-10T20:30:11.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31da70a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4157af0 con 0x7f31d4068de0 2023-10-10T20:30:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31d9f09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31d9f09700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33574/0 (socket says 172.21.15.148:33574) 2023-10-10T20:30:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31d9f09700 1 -- 172.21.15.148:0/2221462693 learned_addr learned my addr 172.21.15.148:0/2221462693 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31d9708700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d4067660 0x7f31d413fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.864+0000 7f31d8f07700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4068de0 0x7f31d41586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 -- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4068de0 msgr2=0x7f31d41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4068de0 0x7f31d41586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 -- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d4067660 msgr2=0x7f31d413fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d4067660 0x7f31d413fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 -- 172.21.15.148:0/2221462693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc009ae0 con 0x7f31d4159f80 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31d9f09700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009880 tx=0x7f31cc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc008a80 con 0x7f31d4159f80 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc008c00 con 0x7f31d4159f80 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d4152a40 con 0x7f31d4159f80 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc026460 con 0x7f31d4159f80 2023-10-10T20:30:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.865+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d40ab560 con 0x7f31d4159f80 2023-10-10T20:30:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.866+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d406e200 con 0x7f31d4159f80 2023-10-10T20:30:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.866+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f31cc003040 con 0x7f31d4159f80 2023-10-10T20:30:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.867+0000 7f31c97fa700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.867+0000 7f31d9708700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.867+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31cc085e10 con 0x7f31d4159f80 2023-10-10T20:30:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.867+0000 7f31d9708700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 0x7f31b80693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f31d416d600 tx=0x7f31d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:11.871+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31cc053080 con 0x7f31d4159f80 2023-10-10T20:30:12.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.103+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31d404bb80 con 0x7f31d4159f80 2023-10-10T20:30:12.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.103+0000 7f31c97fa700 1 -- 172.21.15.148:0/2221462693 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31cc0566a0 con 0x7f31d4159f80 2023-10-10T20:30:12.105 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-10T20:30:12.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 msgr2=0x7f31b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 0x7f31b80693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f31d416d600 tx=0x7f31d0009040 comp rx=0 tx=0).stop 2023-10-10T20:30:12.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 msgr2=0x7f31d4158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009880 tx=0x7f31cc018000 comp rx=0 tx=0).stop 2023-10-10T20:30:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 shutdown_connections 2023-10-10T20:30:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f31b8066f30 0x7f31b80693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31d4159f80 0x7f31d4158be0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d4067660 0x7f31d413fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 --2- 172.21.15.148:0/2221462693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4068de0 0x7f31d41586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 >> 172.21.15.148:0/2221462693 conn(0x7f31d406c700 msgr2=0x7f31d4063230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 shutdown_connections 2023-10-10T20:30:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.105+0000 7f31da70a700 1 -- 172.21.15.148:0/2221462693 wait complete. 2023-10-10T20:30:12.119 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-10T20:30:12.119 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-10T20:30:12.119 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2023-10-10T20:30:12.120 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-10T20:30:12.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-10T20:30:12.120 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-10T20:30:12.121 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-10T20:30:12.121 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-10T20:30:12.121 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:30:12.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.122 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2023-10-10T20:30:12.141 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-10T20:30:12.141 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-10T20:30:12.141 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2023-10-10T20:30:12.142 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.142 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2023-10-10T20:30:12.142 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.142 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:12.142 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:12.143 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2023-10-10T20:30:12.143 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:12.143 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:12.143 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2023-10-10T20:30:12.144 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-10T20:30:12.144 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:12.144 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-10T20:30:12.144 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:12.144 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-10T20:30:12.145 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41895sec preferred_lft 41895sec 2023-10-10T20:30:12.145 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2023-10-10T20:30:12.145 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.145 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-10T20:30:12.145 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6cd1:88ff:feb7:e0ab/64 scope link 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:30:12.146 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether d2:d5:35:54:07:b3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:30:12.147 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::d0d5:35ff:fe54:7b3/64 scope link 2023-10-10T20:30:12.147 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.147 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-10T20:30:12.147 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 3a:3e:e7:97:d2:53 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:30:12.148 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::383e:e7ff:fe97:d253/64 scope link 2023-10-10T20:30:12.148 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2023-10-10T20:30:12.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.148 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-10T20:30:12.210 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-10-10T20:30:12.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.212 DEBUG:teuthology.orchestra.run.smithi161:> route 2023-10-10T20:30:12.270 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2023-10-10T20:30:12.272 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-10T20:30:12.272 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-10T20:30:12.272 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-10T20:30:12.272 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-10T20:30:12.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.273 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2023-10-10T20:30:12.274 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2023-10-10T20:30:12.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-10T20:30:12.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-10T20:30:12.275 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-10T20:30:12.275 DEBUG:teuthology.orchestra.run.smithi161:> ') 2023-10-10T20:30:12.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.378 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:30:12.393 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:30:12.394 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:30:12.395 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:30:12.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2023-10-10T20:30:12.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:30:12.460 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-10T20:30:12.515 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-10T20:30:12.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-10T20:30:12.516 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:30:12.569 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-10T20:30:12.570 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-10T20:30:12.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:12.649 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:12.649 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:30:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:30:12.668 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:30:12.669 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:30:12.669 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:30:12.669 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:30:12.688 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-10T20:30:12.689 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-10T20:30:12.689 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.161:3300,172.21.15.161:3301" 2023-10-10T20:30:12.689 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@374035e8-0861-4d26-af5d-2eadca412b24.cephfs=/ 2023-10-10T20:30:12.689 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.148:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2023-10-10T20:30:12.690 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: v2 mount support check returned 2 2023-10-10T20:30:12.690 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-10T20:30:12.690 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-10T20:30:12.690 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790" 2023-10-10T20:30:12.690 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.161:6789,172.21.15.161:6790:/ 2023-10-10T20:30:12.691 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=374035e8-0861-4d26-af5d-2eadca412b24" will pass to kernel 2023-10-10T20:30:12.692 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-10T20:30:12.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:12.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-10T20:30:12.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.872+0000 7f9dbc704700 1 Processor -- start 2023-10-10T20:30:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.872+0000 7f9dbc704700 1 -- start start 2023-10-10T20:30:12.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbc704700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9db406c630 0x7f9db4067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbc704700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9db4067ba0 0x7f9db4159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.878 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbc704700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40cb370 con 0x7f9db40b0890 2023-10-10T20:30:12.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbc704700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db4067fa0 con 0x7f9db4067ba0 2023-10-10T20:30:12.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbc704700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db40b0c60 con 0x7f9db406c630 2023-10-10T20:30:12.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dba4a0700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9db40b0890 0x7f9db40cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51804/0 (socket says 172.21.15.148:51804) 2023-10-10T20:30:12.879 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dba4a0700 1 -- 172.21.15.148:0/3606166627 learned_addr learned my addr 172.21.15.148:0/3606166627 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:12.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.873+0000 7f9dbaca1700 1 --2- 172.21.15.148:0/3606166627 >> v2:172.21.15.148:3300/0 conn(0x7f9db4067ba0 0x7f9db4159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db9c9f700 1 --2- 172.21.15.148:0/3606166627 >> v2:172.21.15.161:3300/0 conn(0x7f9db406c630 0x7f9db4067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db949e700 1 -- 172.21.15.148:0/3606166627 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394654 0 0) 0x7f9db40cb370 con 0x7f9db40b0890 2023-10-10T20:30:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db949e700 1 -- 172.21.15.148:0/3606166627 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da0003570 con 0x7f9db40b0890 2023-10-10T20:30:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9dbaca1700 1 -- 172.21.15.148:0/3606166627 >> v2:172.21.15.161:3300/0 conn(0x7f9db406c630 msgr2=0x7f9db4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9dbaca1700 1 --2- 172.21.15.148:0/3606166627 >> v2:172.21.15.161:3300/0 conn(0x7f9db406c630 0x7f9db4067660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9dbaca1700 1 -- 172.21.15.148:0/3606166627 >> v1:172.21.15.148:6789/0 conn(0x7f9db40b0890 legacy=0x7f9db40cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9dbaca1700 1 -- 172.21.15.148:0/3606166627 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db406a1e0 con 0x7f9db4067ba0 2023-10-10T20:30:12.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9dbaca1700 1 --2- 172.21.15.148:0/3606166627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db4067ba0 0x7f9db4159790 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9db4064ae0 tx=0x7f9dac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda5ef643b7a7046 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:12.882 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db949e700 1 -- 172.21.15.148:0/3606166627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011630 con 0x7f9db4067ba0 2023-10-10T20:30:12.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db949e700 1 -- 172.21.15.148:0/3606166627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0117b0 con 0x7f9db4067ba0 2023-10-10T20:30:12.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.874+0000 7f9db949e700 1 -- 172.21.15.148:0/3606166627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011b70 con 0x7f9db4067ba0 2023-10-10T20:30:12.883 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 -- 172.21.15.148:0/3606166627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db4067ba0 msgr2=0x7f9db4159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 --2- 172.21.15.148:0/3606166627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db4067ba0 0x7f9db4159790 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9db4064ae0 tx=0x7f9dac0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 -- 172.21.15.148:0/3606166627 shutdown_connections 2023-10-10T20:30:12.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 --2- 172.21.15.148:0/3606166627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db4067ba0 0x7f9db4159790 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 --2- 172.21.15.148:0/3606166627 >> v2:172.21.15.161:3300/0 conn(0x7f9db406c630 0x7f9db4067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 -- 172.21.15.148:0/3606166627 >> 172.21.15.148:0/3606166627 conn(0x7f9db406fe50 msgr2=0x7f9db4069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:12.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 -- 172.21.15.148:0/3606166627 shutdown_connections 2023-10-10T20:30:12.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.875+0000 7f9dbc704700 1 -- 172.21.15.148:0/3606166627 wait complete. 2023-10-10T20:30:12.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 Processor -- start 2023-10-10T20:30:12.885 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 -- start start 2023-10-10T20:30:12.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db4067ba0 0x7f9db414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9db40b0890 0x7f9db40afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db414e8f0 con 0x7f9db406c630 2023-10-10T20:30:12.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40ab4f0 con 0x7f9db4067ba0 2023-10-10T20:30:12.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.876+0000 7f9dbc704700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40aa920 con 0x7f9db40b0890 2023-10-10T20:30:12.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51550/0 (socket says 172.21.15.148:51550) 2023-10-10T20:30:12.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 -- 172.21.15.148:0/82610767 learned_addr learned my addr 172.21.15.148:0/82610767 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:12.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9dba4a0700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db4067ba0 0x7f9db414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9dbaca1700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9db40b0890 0x7f9db40afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 -- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9db40b0890 msgr2=0x7f9db40afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9db40b0890 0x7f9db40afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 -- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db4067ba0 msgr2=0x7f9db414e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:12.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db4067ba0 0x7f9db414e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:12.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 -- 172.21.15.148:0/82610767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac009420 con 0x7f9db406c630 2023-10-10T20:30:12.891 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9db9c9f700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9da400ec50 tx=0x7f9da40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:12.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400c9d0 con 0x7f9db406c630 2023-10-10T20:30:12.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400cb30 con 0x7f9db406c630 2023-10-10T20:30:12.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db40aabc0 con 0x7f9db406c630 2023-10-10T20:30:12.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4013470 con 0x7f9db406c630 2023-10-10T20:30:12.892 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.877+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db40ab180 con 0x7f9db406c630 2023-10-10T20:30:12.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.878+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db40ab660 con 0x7f9db406c630 2023-10-10T20:30:12.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.879+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9da40138a0 con 0x7f9db406c630 2023-10-10T20:30:12.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.879+0000 7f9daa7fc700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 0x7f9d980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:12.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.879+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9da4084800 con 0x7f9db406c630 2023-10-10T20:30:12.893 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.879+0000 7f9dba4a0700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 0x7f9d980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:12.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.879+0000 7f9dba4a0700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 0x7f9d980693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005c50 tx=0x7f9db0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:12.894 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:12.883+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9da4051a70 con 0x7f9db406c630 2023-10-10T20:30:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.111+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9db404c3c0 con 0x7f9db406c630 2023-10-10T20:30:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.112+0000 7f9daa7fc700 1 -- 172.21.15.148:0/82610767 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f9da4055090 con 0x7f9db406c630 2023-10-10T20:30:13.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:13.114 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 msgr2=0x7f9d980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 0x7f9d980693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005c50 tx=0x7f9db0009040 comp rx=0 tx=0).stop 2023-10-10T20:30:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 msgr2=0x7f9db40af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9da400ec50 tx=0x7f9da40091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 shutdown_connections 2023-10-10T20:30:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9d98066f30 0x7f9d980693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9db40b0890 0x7f9db40afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9db4067ba0 0x7f9db414e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 --2- 172.21.15.148:0/82610767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9db406c630 0x7f9db40af4f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 >> 172.21.15.148:0/82610767 conn(0x7f9db406fe50 msgr2=0x7f9db415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 shutdown_connections 2023-10-10T20:30:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.114+0000 7f9dbc704700 1 -- 172.21.15.148:0/82610767 wait complete. 2023-10-10T20:30:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:13.127 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:30:13.261 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":8139,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3698382769999995,"mdsmap_epoch":85,"osdmap_epoch":178,"osdmap_epoch_barrier":178,"uptime":13.480707064000001} 2023-10-10T20:30:13.261 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 8139, "mdsmap_epoch": 85, "osdmap_epoch": 178, "osdmap_epoch_barrier": 178, "rank_uptime": 8.369838277, "state": "up:active", "uptime": 13.480707064, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:30:13.262 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-10T20:30:13.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.379+0000 7f569abab700 1 Processor -- start 2023-10-10T20:30:13.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.379+0000 7f569abab700 1 -- start start 2023-10-10T20:30:13.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f569abab700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f569406ad60 0x7f56940b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:13.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f569abab700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569415eb10 con 0x7f5694159de0 2023-10-10T20:30:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f569abab700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569415e680 con 0x7f569415a1b0 2023-10-10T20:30:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f569abab700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56940b3790 con 0x7f569406ad60 2023-10-10T20:30:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f5698947700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5694159de0 0x7f56940b2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51818/0 (socket says 172.21.15.148:51818) 2023-10-10T20:30:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f5698947700 1 -- 172.21.15.148:0/3113276402 learned_addr learned my addr 172.21.15.148:0/3113276402 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f5699148700 1 --2- 172.21.15.148:0/3113276402 >> v2:172.21.15.161:3300/0 conn(0x7f569406ad60 0x7f56940b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506552759 0 0) 0x7f5690001110 con 0x7f5694159de0 2023-10-10T20:30:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.380+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f567c003570 con 0x7f5694159de0 2023-10-10T20:30:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f5699148700 1 -- 172.21.15.148:0/3113276402 >> v1:172.21.15.161:6789/0 conn(0x7f569415a1b0 legacy=0x7f569406a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246083227 0 0) 0x7f569415e680 con 0x7f569415a1b0 2023-10-10T20:30:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f5699148700 1 -- 172.21.15.148:0/3113276402 >> v1:172.21.15.148:6789/0 conn(0x7f5694159de0 legacy=0x7f56940b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f5699148700 1 -- 172.21.15.148:0/3113276402 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569415dd50 con 0x7f569406ad60 2023-10-10T20:30:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f5699148700 1 --2- 172.21.15.148:0/3113276402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569406ad60 0x7f56940b14e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f568c009e60 tx=0x7f568c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8078a6a76029f098 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c00e070 con 0x7f569406ad60 2023-10-10T20:30:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c00b850 con 0x7f569406ad60 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f568b7fe700 1 -- 172.21.15.148:0/3113276402 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690001580 con 0x7f569406ad60 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 -- 172.21.15.148:0/3113276402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569406ad60 msgr2=0x7f56940b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 --2- 172.21.15.148:0/3113276402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569406ad60 0x7f56940b14e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f568c009e60 tx=0x7f568c009780 comp rx=0 tx=0).stop 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 -- 172.21.15.148:0/3113276402 shutdown_connections 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 --2- 172.21.15.148:0/3113276402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569406ad60 0x7f56940b14e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 -- 172.21.15.148:0/3113276402 >> 172.21.15.148:0/3113276402 conn(0x7f5694068d60 msgr2=0x7f5694069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 -- 172.21.15.148:0/3113276402 shutdown_connections 2023-10-10T20:30:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.381+0000 7f569abab700 1 -- 172.21.15.148:0/3113276402 wait complete. 2023-10-10T20:30:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 Processor -- start 2023-10-10T20:30:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 -- start start 2023-10-10T20:30:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 0x7f56940c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569415a1b0 0x7f56940c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569415d230 con 0x7f569406ad60 2023-10-10T20:30:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569415c860 con 0x7f569415a1b0 2023-10-10T20:30:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.383+0000 7f569abab700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5694158180 con 0x7f5694159de0 2023-10-10T20:30:13.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:13.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51566/0 (socket says 172.21.15.148:51566) 2023-10-10T20:30:13.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 -- 172.21.15.148:0/2704434757 learned_addr learned my addr 172.21.15.148:0/2704434757 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:13.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5699148700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569415a1b0 0x7f56940c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:13.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f568bfff700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 0x7f56940c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 -- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 msgr2=0x7f56940c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 0x7f56940c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 -- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569415a1b0 msgr2=0x7f56940c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569415a1b0 0x7f56940c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 -- 172.21.15.148:0/2704434757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c009a20 con 0x7f569406ad60 2023-10-10T20:30:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f568bfff700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 0x7f56940c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5698947700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f569000cb90 tx=0x7f5690001010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690007360 con 0x7f569406ad60 2023-10-10T20:30:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56900074e0 con 0x7f569406ad60 2023-10-10T20:30:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56940c82f0 con 0x7f569406ad60 2023-10-10T20:30:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690004830 con 0x7f569406ad60 2023-10-10T20:30:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.384+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56940c7560 con 0x7f569406ad60 2023-10-10T20:30:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.385+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569404c3c0 con 0x7f569406ad60 2023-10-10T20:30:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.385+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5690004bc0 con 0x7f569406ad60 2023-10-10T20:30:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.386+0000 7f5688ff9700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:13.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.386+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5690083c30 con 0x7f569406ad60 2023-10-10T20:30:13.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.386+0000 7f568bfff700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:13.400 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.386+0000 7f568bfff700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56940c6b40 tx=0x7f5684009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:13.401 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.390+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5690050f20 con 0x7f569406ad60 2023-10-10T20:30:13.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.573+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f569415eaf0 con 0x7f569406ad60 2023-10-10T20:30:13.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.940+0000 7f5688ff9700 1 -- 172.21.15.148:0/2704434757 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v179) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5690018020 con 0x7f569406ad60 2023-10-10T20:30:13.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 msgr2=0x7f56800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f56940c6b40 tx=0x7f5684009040 comp rx=0 tx=0).stop 2023-10-10T20:30:13.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 msgr2=0x7f569415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:13.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f569000cb90 tx=0x7f5690001010 comp rx=0 tx=0).stop 2023-10-10T20:30:13.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 shutdown_connections 2023-10-10T20:30:13.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f569415a1b0 0x7f56940c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f569406ad60 0x7f569415ccf0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 --2- 172.21.15.148:0/2704434757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5694159de0 0x7f56940c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:13.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 >> 172.21.15.148:0/2704434757 conn(0x7f5694068d60 msgr2=0x7f5694159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:13.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 shutdown_connections 2023-10-10T20:30:13.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:13.943+0000 7f569abab700 1 -- 172.21.15.148:0/2704434757 wait complete. 2023-10-10T20:30:13.947 INFO:teuthology.orchestra.run.smithi148.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-10T20:30:13.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-10T20:30:14.088 INFO:teuthology.orchestra.run.smithi148.stdout:{"osd_mon_report_interval":"5"} 2023-10-10T20:30:14.089 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-10T20:30:14.089 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-10T20:30:14.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:14.091 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-10T20:30:14.201 INFO:teuthology.orchestra.run.smithi148.stderr:16+0 records in 2023-10-10T20:30:14.201 INFO:teuthology.orchestra.run.smithi148.stderr:16+0 records out 2023-10-10T20:30:14.202 INFO:teuthology.orchestra.run.smithi148.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.090915 s, 185 MB/s 2023-10-10T20:30:14.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:14.203 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-10T20:30:14.311 INFO:teuthology.orchestra.run.smithi148.stderr:17+0 records in 2023-10-10T20:30:14.311 INFO:teuthology.orchestra.run.smithi148.stderr:17+0 records out 2023-10-10T20:30:14.311 INFO:teuthology.orchestra.run.smithi148.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0908345 s, 196 MB/s 2023-10-10T20:30:14.312 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-10T20:30:14.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.426+0000 7f61c509a700 1 Processor -- start 2023-10-10T20:30:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.426+0000 7f61c509a700 1 -- start start 2023-10-10T20:30:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61c509a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 0x7f61c00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61c509a700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00cb7c0 con 0x7f61c00cdab0 2023-10-10T20:30:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61c509a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c015e290 con 0x7f61c015cbd0 2023-10-10T20:30:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61c509a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015e850 con 0x7f61c00cd6e0 2023-10-10T20:30:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61bf7fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f61c00cdab0 0x7f61c015d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51822/0 (socket says 172.21.15.148:51822) 2023-10-10T20:30:14.431 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61bf7fe700 1 -- 172.21.15.148:0/1909509912 learned_addr learned my addr 172.21.15.148:0/1909509912 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61bffff700 1 --2- 172.21.15.148:0/1909509912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 0x7f61c00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38686914 0 0) 0x7f61c00cb7c0 con 0x7f61c00cdab0 2023-10-10T20:30:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.427+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a8003570 con 0x7f61c00cdab0 2023-10-10T20:30:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475719074 0 0) 0x7f61c015e290 con 0x7f61c015cbd0 2023-10-10T20:30:14.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c00cb7c0 con 0x7f61c015cbd0 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2384613494 0 0) 0x7f61a8003570 con 0x7f61c00cdab0 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61bffff700 1 -- 172.21.15.148:0/1909509912 >> v1:172.21.15.161:6789/0 conn(0x7f61c015cbd0 legacy=0x7f61c015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61bffff700 1 -- 172.21.15.148:0/1909509912 >> v1:172.21.15.148:6789/0 conn(0x7f61c00cdab0 legacy=0x7f61c015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61bffff700 1 -- 172.21.15.148:0/1909509912 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c015e570 con 0x7f61c00cd6e0 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61bffff700 1 --2- 172.21.15.148:0/1909509912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 0x7f61c00cbc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009910 tx=0x7f61b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88b66e26df51395 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:14.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4001470 con 0x7f61c00cd6e0 2023-10-10T20:30:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b0011420 con 0x7f61c00cd6e0 2023-10-10T20:30:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61beffd700 1 -- 172.21.15.148:0/1909509912 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00117c0 con 0x7f61c00cd6e0 2023-10-10T20:30:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61c509a700 1 -- 172.21.15.148:0/1909509912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 msgr2=0x7f61c00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61c509a700 1 --2- 172.21.15.148:0/1909509912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 0x7f61c00cbc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009910 tx=0x7f61b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61c509a700 1 -- 172.21.15.148:0/1909509912 shutdown_connections 2023-10-10T20:30:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61c509a700 1 --2- 172.21.15.148:0/1909509912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cd6e0 0x7f61c00cbc50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.428+0000 7f61c509a700 1 -- 172.21.15.148:0/1909509912 >> 172.21.15.148:0/1909509912 conn(0x7f61c0067660 msgr2=0x7f61c0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.429+0000 7f61c509a700 1 -- 172.21.15.148:0/1909509912 shutdown_connections 2023-10-10T20:30:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.429+0000 7f61c509a700 1 -- 172.21.15.148:0/1909509912 wait complete. 2023-10-10T20:30:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 Processor -- start 2023-10-10T20:30:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 -- start start 2023-10-10T20:30:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c00cd6e0 0x7f61c014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:14.439 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cdab0 0x7f61c014ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00c82a0 con 0x7f61c015cbd0 2023-10-10T20:30:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00b5a50 con 0x7f61c00cd6e0 2023-10-10T20:30:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.430+0000 7f61c509a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c014f5d0 con 0x7f61c00cdab0 2023-10-10T20:30:14.440 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51582/0 (socket says 172.21.15.148:51582) 2023-10-10T20:30:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 -- 172.21.15.148:0/4203247664 learned_addr learned my addr 172.21.15.148:0/4203247664 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61bf7fe700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cdab0 0x7f61c014ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61bffff700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c00cd6e0 0x7f61c014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:14.441 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 -- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cdab0 msgr2=0x7f61c014ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cdab0 0x7f61c014ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 -- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c00cd6e0 msgr2=0x7f61c014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c00cd6e0 0x7f61c014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 -- 172.21.15.148:0/4203247664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0009420 con 0x7f61c015cbd0 2023-10-10T20:30:14.442 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c4899700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f61b800c860 tx=0x7f61b8002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8002030 con 0x7f61c015cbd0 2023-10-10T20:30:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80021b0 con 0x7f61c015cbd0 2023-10-10T20:30:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c014f8b0 con 0x7f61c015cbd0 2023-10-10T20:30:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8012630 con 0x7f61c015cbd0 2023-10-10T20:30:14.443 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.431+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c014fe70 con 0x7f61c015cbd0 2023-10-10T20:30:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.432+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c00c8070 con 0x7f61c015cbd0 2023-10-10T20:30:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.432+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61b8012a20 con 0x7f61c015cbd0 2023-10-10T20:30:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.433+0000 7f61a7fff700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 0x7f61ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.433+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61b80844d0 con 0x7f61c015cbd0 2023-10-10T20:30:14.444 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.433+0000 7f61bffff700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 0x7f61ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.433+0000 7f61bffff700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 0x7f61ac0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61b00030c0 tx=0x7f61b0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:14.445 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.437+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61c00c8070 con 0x7f61c015cbd0 2023-10-10T20:30:14.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.620+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f61c00c8070 con 0x7f61c015cbd0 2023-10-10T20:30:14.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.621+0000 7f61a7fff700 1 -- 172.21.15.148:0/4203247664 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v179) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f61c00c8070 con 0x7f61c015cbd0 2023-10-10T20:30:14.623 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:14.623 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:14.624 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 179, 2023-10-10T20:30:14.624 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:14.624 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:14.624 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:13.930498+0000", 2023-10-10T20:30:14.624 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:14.625 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:14.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:14.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:14.625 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:14.625 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:14.626 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:14.626 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:14.626 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:14.626 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:14.627 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:14.628 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:14.628 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:14.628 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:14.628 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:14.628 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:14.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.629 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:14.629 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:14.629 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:14.629 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:14.630 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:14.630 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:14.630 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:14.630 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:14.630 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:14.631 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:14.631 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:14.631 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:14.631 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:14.632 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:14.632 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:14.632 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:14.632 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:14.632 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:14.633 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:14.633 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:14.633 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:14.633 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:14.633 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:14.634 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:14.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:14.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:14.635 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:14.635 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:14.636 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:14.636 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:14.636 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:14.636 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:14.636 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:14.637 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:14.638 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:14.638 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:14.638 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:14.638 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:14.638 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:14.639 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:14.639 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:14.639 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:14.639 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:14.639 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:14.640 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:14.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.640 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:14.640 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:14.640 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:14.641 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:14.642 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:14.642 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:14.642 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:14.642 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:14.642 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:14.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.643 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:14.643 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:14.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.644 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:14.644 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:14.644 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:14.644 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:14.645 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:14.646 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:14.646 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:14.646 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:14.646 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:14.646 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:14.647 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:14.647 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:14.647 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:14.647 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:14.647 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:14.648 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:14.649 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:14.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.649 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:14.649 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:14.649 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:14.650 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:14.650 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:14.650 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:14.650 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:14.650 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:14.651 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:14.651 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:14.651 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:14.651 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:14.651 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:14.652 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:14.652 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:14.652 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:14.652 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:14.652 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:14.653 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:14.653 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:14.653 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:14.653 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:14.653 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:14.654 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:14.654 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:14.654 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:14.654 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:14.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.655 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:14.655 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:14.655 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:14.655 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:14.655 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:14.656 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:14.657 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:14.657 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:14.657 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:14.657 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:14.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.658 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:14.658 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:14.658 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:14.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.659 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:14.659 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:14.660 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:14.661 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:14.661 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:14.661 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:14.661 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:14.662 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:14.662 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:14.662 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:14.662 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:14.662 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:14.663 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:14.663 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:14.663 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:14.663 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:14.664 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:14.665 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:14.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.665 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "179", 2023-10-10T20:30:14.665 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:14.665 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:14.666 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:14.666 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:14.666 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:14.667 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:14.668 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:14.668 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:14.668 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:14.668 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:14.668 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:14.669 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:14.669 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:14.669 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:14.669 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:14.669 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:14.670 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:14.671 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:14.672 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:14.672 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:14.672 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:14.672 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:14.672 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:14.673 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:14.673 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:14.673 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:14.673 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:14.673 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:14.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.674 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:14.674 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:14.674 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:14.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.675 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:14.675 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:14.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.676 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:14.676 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.676 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.676 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.677 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.677 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.677 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.677 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.677 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.679 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:14.679 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.680 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.680 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:14.680 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.680 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.681 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.681 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.681 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.682 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.683 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:14.683 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.683 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:14.685 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.685 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.685 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.685 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.685 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.686 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.687 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.687 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.687 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:14.687 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.688 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.688 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:14.688 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:14.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.689 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.689 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:14.689 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:14.690 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:14.690 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:14.690 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.690 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.690 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.691 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.691 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:14.691 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.692 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.692 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.692 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.692 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.692 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.693 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.694 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.694 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:14.694 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.695 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.695 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.695 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:14.695 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.696 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.696 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.696 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.696 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.698 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:14.698 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.699 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.699 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.699 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.699 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.699 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.700 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:14.700 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.700 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.700 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.700 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.701 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:14.701 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.701 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.701 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:14.702 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.703 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.703 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.703 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:14.703 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:14.704 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:14.705 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:14.705 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:14.705 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.705 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.705 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.706 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.706 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:14.706 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.707 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.707 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.707 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.707 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.707 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.708 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.709 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.709 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.709 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:14.709 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.710 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.710 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.710 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:14.710 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.712 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.712 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:14.712 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.712 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.713 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.713 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:14.713 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.713 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.713 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.714 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.714 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.714 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.714 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.714 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.715 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:14.715 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.715 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.716 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.717 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.717 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.717 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:14.717 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.718 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.718 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.718 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.718 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:14.718 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:14.719 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.719 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.719 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.719 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:14.719 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:14.720 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:14.720 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:14.720 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.720 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.720 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.721 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.721 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:14.721 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.722 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.722 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.722 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.722 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.722 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.723 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.723 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.723 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.723 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.723 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:14.724 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.725 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.725 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:14.725 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.726 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.726 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.726 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.726 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.727 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.727 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.727 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:14.727 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.728 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.728 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:14.728 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.728 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.730 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.730 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:14.730 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.731 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.731 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.731 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:14.731 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.731 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.732 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.733 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:14.733 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.733 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.733 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.734 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:14.734 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:14.734 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.734 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.734 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:14.735 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.736 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:14.737 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.737 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.737 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.737 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.737 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.738 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.738 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.738 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.738 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.738 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.739 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:14.740 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.740 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.740 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:14.741 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.741 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.741 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.743 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.743 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:14.743 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.744 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.745 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.746 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.746 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:14.746 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.746 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.746 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.747 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:14.748 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.748 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.748 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.748 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.748 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:14.749 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:14.750 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:14.750 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:14.750 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.750 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.750 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.751 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.751 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:14.751 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.752 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.752 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.752 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.752 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.752 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.753 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.753 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.753 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.753 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.753 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:14.754 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.755 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.755 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:14.755 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.756 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.756 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.757 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:14.757 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.758 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.759 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.759 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.759 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.760 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:14.760 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.760 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.761 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:14.761 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.761 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.761 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:14.762 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.763 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.763 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:14.763 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:14.764 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.764 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.764 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:14.764 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.765 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.766 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.767 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.767 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.767 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.767 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.767 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.768 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.769 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.769 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:14.769 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.770 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.770 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:14.770 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.770 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.771 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.771 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.771 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.772 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.773 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:14.773 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.773 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.773 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.774 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.774 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.774 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.774 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:14.775 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.775 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.775 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:14.776 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.776 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.776 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.778 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.778 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:14.778 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:14.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.779 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.779 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:14.779 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:14.779 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.781 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:14.781 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:14.781 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:14.781 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:14.782 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:14.782 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:14.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:14.782 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:14.782 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:14.783 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:14.783 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:14.783 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:14.783 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:14.783 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.785 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.785 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:14.785 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.785 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.785 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.786 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:14.786 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.786 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.787 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:14.788 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.788 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.788 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.790 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.790 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:14.790 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.791 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.791 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:14.791 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:14.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.792 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:14.793 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:14.793 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:14.793 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.793 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:14.794 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:14.795 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:14.795 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:14.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.795 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:14.795 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.796 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.797 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.797 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:14.797 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.798 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:14.798 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.798 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.798 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.798 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.799 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:14.800 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.800 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.800 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.800 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.800 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:14.801 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.802 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.802 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.802 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.802 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.802 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:14.803 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.803 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:14.803 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.804 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.804 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.804 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.804 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.804 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:14.805 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:14.805 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.806 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.806 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.806 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.806 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.806 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.807 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.808 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.808 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.808 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.808 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:14.808 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:14.809 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:14.809 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:14.809 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:14.810 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.811 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:14.811 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:14.811 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:14.811 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:14.811 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:14.812 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:14.813 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:14.813 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:14.813 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:14.813 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.813 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:14.814 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:14.815 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:14.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:14.816 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:14.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.623+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 msgr2=0x7f61ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.623+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 0x7f61ac0693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61b00030c0 tx=0x7f61b0016000 comp rx=0 tx=0).stop 2023-10-10T20:30:14.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.623+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 msgr2=0x7f61c014ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:14.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.623+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f61b800c860 tx=0x7f61b8002700 comp rx=0 tx=0).stop 2023-10-10T20:30:14.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 shutdown_connections 2023-10-10T20:30:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f61ac066f30 0x7f61ac0693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61c015cbd0 0x7f61c014ef40 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c00cd6e0 0x7f61c014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 --2- 172.21.15.148:0/4203247664 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61c00cdab0 0x7f61c014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 >> 172.21.15.148:0/4203247664 conn(0x7f61c0067660 msgr2=0x7f61c00c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:14.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 shutdown_connections 2023-10-10T20:30:14.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:14.624+0000 7f61c509a700 1 -- 172.21.15.148:0/4203247664 wait complete. 2023-10-10T20:30:14.820 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:30:19.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.949+0000 7fc9f9ab2700 1 Processor -- start 2023-10-10T20:30:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.949+0000 7fc9f9ab2700 1 -- start start 2023-10-10T20:30:19.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc9f4069de0 0x7fc9f4157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc9f40c9930 0x7fc9f415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f4069060 con 0x7fc9f40c9930 2023-10-10T20:30:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f4069300 con 0x7fc9f4069de0 2023-10-10T20:30:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f9ab2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40695e0 con 0x7fc9f40c9560 2023-10-10T20:30:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc9f40c9930 0x7fc9f415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f3fff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc9f40c9930 0x7fc9f415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51598/0 (socket says 172.21.15.148:51598) 2023-10-10T20:30:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f3fff700 1 -- 172.21.15.148:0/3163561158 learned_addr learned my addr 172.21.15.148:0/3163561158 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f37fe700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f4157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.950+0000 7fc9f3fff700 1 -- 172.21.15.148:0/3163561158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 msgr2=0x7fc9f4157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f4157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f3fff700 1 -- 172.21.15.148:0/3163561158 >> v2:172.21.15.161:3300/0 conn(0x7fc9f4069de0 msgr2=0x7fc9f4157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f2ffd700 1 --2- 172.21.15.148:0/3163561158 >> v2:172.21.15.161:3300/0 conn(0x7fc9f4069de0 0x7fc9f4157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/3163561158 >> v2:172.21.15.161:3300/0 conn(0x7fc9f4069de0 0x7fc9f4157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f3fff700 1 -- 172.21.15.148:0/3163561158 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f40698c0 con 0x7fc9f40c9930 2023-10-10T20:30:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f415bc60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc9f406b4f0 tx=0x7fc9ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212990b0db0ec9d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f27fc700 1 -- 172.21.15.148:0/3163561158 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00e070 con 0x7fc9f40c9930 2023-10-10T20:30:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f27fc700 1 -- 172.21.15.148:0/3163561158 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec00ba50 con 0x7fc9f40c9930 2023-10-10T20:30:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f27fc700 1 -- 172.21.15.148:0/3163561158 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0092e0 con 0x7fc9f40c9930 2023-10-10T20:30:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/3163561158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 msgr2=0x7fc9f415bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f415bc60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc9f406b4f0 tx=0x7fc9ec009840 comp rx=0 tx=0).stop 2023-10-10T20:30:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/3163561158 shutdown_connections 2023-10-10T20:30:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f415bc60 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/3163561158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f4157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/3163561158 >> v2:172.21.15.161:3300/0 conn(0x7fc9f4069de0 0x7fc9f4157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/3163561158 >> 172.21.15.148:0/3163561158 conn(0x7fc9f4067660 msgr2=0x7fc9f4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.951+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/3163561158 shutdown_connections 2023-10-10T20:30:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.952+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/3163561158 wait complete. 2023-10-10T20:30:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 Processor -- start 2023-10-10T20:30:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 -- start start 2023-10-10T20:30:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 0x7fc9f41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f40b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f41515e0 con 0x7fc9f40c9930 2023-10-10T20:30:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f416b890 con 0x7fc9f4069de0 2023-10-10T20:30:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.953+0000 7fc9f9ab2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f416bb70 con 0x7fc9f40c9560 2023-10-10T20:30:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51600/0 (socket says 172.21.15.148:51600) 2023-10-10T20:30:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 -- 172.21.15.148:0/1685871652 learned_addr learned my addr 172.21.15.148:0/1685871652 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f37fe700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 0x7fc9f41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f2ffd700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f40b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 -- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 msgr2=0x7fc9f40b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f40b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 -- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 msgr2=0x7fc9f41504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 0x7fc9f41504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 -- 172.21.15.148:0/1685871652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec009ae0 con 0x7fc9f40c9930 2023-10-10T20:30:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f2ffd700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f40b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f37fe700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 0x7fc9f41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f3fff700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40c92a0 tx=0x7fc9ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0032f0 con 0x7fc9f40c9930 2023-10-10T20:30:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec007e70 con 0x7fc9f40c9930 2023-10-10T20:30:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f416be50 con 0x7fc9f40c9930 2023-10-10T20:30:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec01d630 con 0x7fc9f40c9930 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.954+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f40b1fb0 con 0x7fc9f40c9930 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.955+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f404c3c0 con 0x7fc9f40c9930 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.956+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9ec003040 con 0x7fc9f40c9930 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.956+0000 7fc9db7fe700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 0x7fc9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.956+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc9ec086250 con 0x7fc9f40c9930 2023-10-10T20:30:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.956+0000 7fc9f37fe700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 0x7fc9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.956+0000 7fc9f37fe700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 0x7fc9e00693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4008a80 tx=0x7fc9e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:19.960+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9ec0534c0 con 0x7fc9f40c9930 2023-10-10T20:30:20.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.141+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc9f40c9f50 con 0x7fc9f40c9930 2023-10-10T20:30:20.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.142+0000 7fc9db7fe700 1 -- 172.21.15.148:0/1685871652 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v179) v1 ==== 81+0+32330 (secure 0 0 0) 0x7fc9ec056ae0 con 0x7fc9f40c9930 2023-10-10T20:30:20.145 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:20.146 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:20.146 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 179, 2023-10-10T20:30:20.146 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:20.146 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:20.147 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:13.930498+0000", 2023-10-10T20:30:20.147 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:20.147 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:20.149 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:20.149 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:20.149 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:20.149 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:20.150 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:20.151 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:20.151 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:20.151 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:20.151 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:20.151 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:20.152 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.154 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:20.154 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:20.154 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:20.154 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:20.155 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:20.156 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:20.156 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:20.156 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:20.156 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:20.156 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:20.157 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:20.157 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:20.157 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:20.158 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:20.158 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:20.158 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:20.159 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:20.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:20.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:20.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:20.160 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:20.161 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:20.161 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:20.161 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:20.161 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:20.161 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:20.162 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:20.162 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:20.162 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:20.162 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:20.162 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:20.163 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:20.163 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:20.163 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:20.163 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:20.163 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:20.164 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:20.164 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:20.164 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:20.165 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:20.165 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:20.166 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:20.168 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:20.168 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:20.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.169 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:20.169 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:20.169 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:20.169 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:20.170 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:20.170 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:20.170 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:20.171 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:20.171 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:20.171 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:20.171 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:20.171 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:20.172 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:20.172 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:20.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.172 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:20.172 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:20.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.173 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.173 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.173 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:20.173 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:20.174 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:20.174 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:20.174 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:20.174 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:20.174 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:20.175 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:20.175 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:20.175 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:20.175 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:20.175 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:20.176 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:20.176 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:20.176 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:20.176 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:20.176 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:20.177 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:20.177 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:20.177 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:20.177 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:20.178 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:20.178 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:20.178 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:20.178 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:20.178 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:20.179 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:20.179 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.179 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:20.179 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:20.179 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:20.180 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:20.180 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:20.180 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:20.180 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:20.180 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:20.181 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:20.181 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:20.181 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:20.181 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:20.181 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:20.182 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:20.182 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:20.182 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:20.183 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:20.183 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:20.183 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:20.183 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:20.184 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:20.184 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:20.184 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:20.184 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:20.184 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:20.185 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:20.186 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:20.186 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:20.186 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:20.186 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:20.187 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:20.187 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:20.187 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:20.187 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.188 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:20.189 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:20.189 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:20.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.189 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.189 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.190 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:20.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:20.190 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:20.190 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:20.191 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:20.191 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:20.191 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:20.191 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:20.191 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:20.193 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:20.193 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:20.193 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:20.193 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:20.193 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:20.194 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:20.194 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:20.194 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:20.194 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:20.194 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:20.195 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:20.195 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:20.195 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:20.195 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.196 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "179", 2023-10-10T20:30:20.196 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:20.196 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:20.196 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:20.197 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:20.197 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:20.197 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:20.197 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:20.197 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:20.198 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:20.198 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:20.198 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:20.198 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:20.198 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:20.199 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:20.200 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:20.200 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:20.200 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:20.200 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:20.200 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:20.201 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:20.202 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:20.203 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:20.204 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:20.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:20.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:20.206 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.206 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.206 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.207 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.207 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.207 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.207 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.207 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.208 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.208 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.208 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.208 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.209 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.209 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.209 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:20.209 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.210 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.210 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:20.210 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.211 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.211 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.211 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.211 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.212 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.212 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.212 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:20.212 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.212 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.213 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.214 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.214 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.214 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.214 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.215 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:20.215 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.215 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.215 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.215 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.216 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:20.216 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.216 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.216 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.217 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.217 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.217 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.217 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.217 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.218 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:20.218 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.218 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.218 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.218 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.219 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:20.219 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:20.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.219 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.220 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.221 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.221 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.221 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.221 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.221 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:20.222 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.222 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.222 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.222 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.222 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.223 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.224 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.224 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.224 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.224 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.225 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:20.225 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.225 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.226 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.226 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:20.226 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.226 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.226 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.227 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.227 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.227 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.227 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.227 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.228 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:20.228 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.228 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.228 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.229 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.230 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.230 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.230 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.231 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:20.231 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.231 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.231 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.232 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.233 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.233 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.233 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:20.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.235 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.235 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.235 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:20.235 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:20.236 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:20.236 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:20.236 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.237 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:20.238 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.238 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.238 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.238 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.238 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.239 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.239 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.239 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.239 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.239 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.240 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:20.241 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.241 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.241 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:20.242 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.242 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.242 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.242 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.243 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.243 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.243 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.243 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:20.243 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.244 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.244 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:20.245 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.245 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.245 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.245 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.246 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.246 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.246 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.246 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:20.246 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.247 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.248 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.248 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.248 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.248 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.254 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.255 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.255 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.256 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.257 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.258 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.258 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.258 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.258 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.259 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.259 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:20.259 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.260 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.260 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:20.260 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.260 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.261 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.261 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.261 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.261 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.262 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.262 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:20.262 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.263 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:20.263 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.264 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.264 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.264 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.266 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:20.266 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.266 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.267 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.267 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.267 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.268 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:20.268 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.269 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.269 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:20.269 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:20.269 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.269 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.270 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:20.270 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:20.270 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:20.270 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.272 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:20.272 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.272 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.272 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.273 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.274 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.274 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.274 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.274 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.275 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.275 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:20.275 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.276 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.276 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:20.276 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.276 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.276 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.277 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.277 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.277 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.278 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:20.278 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.279 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:20.279 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.279 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.280 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.280 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.280 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.281 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:20.281 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.281 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.282 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:20.282 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.282 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.282 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:20.283 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.284 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:20.284 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:20.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.285 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.285 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:20.285 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:20.286 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:20.286 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:20.286 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.286 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.286 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.287 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.287 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:20.287 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.288 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.289 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.289 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.289 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.289 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.289 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.290 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.290 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.290 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:20.290 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.291 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.291 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:20.292 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.292 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:20.293 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.294 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.294 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:20.294 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.295 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.295 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.295 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.295 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.297 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.297 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:20.297 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.297 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.297 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.299 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:20.300 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:20.300 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.300 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.300 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.300 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:20.301 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:20.301 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:20.301 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:20.302 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.302 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.302 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.302 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.303 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:20.303 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.303 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.303 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.304 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.304 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.304 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.304 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.304 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.305 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.305 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.305 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.305 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.305 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.306 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.306 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:20.306 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.307 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:20.307 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.308 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.308 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.310 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:20.310 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.310 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.310 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.311 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.311 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.311 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.311 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.312 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.312 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:20.312 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.312 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.313 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:20.313 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.313 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.313 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.315 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:20.316 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:20.316 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.316 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.316 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:20.317 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:20.317 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:20.317 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:20.317 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.317 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.318 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.318 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.318 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:20.319 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:20.319 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:20.319 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:20.319 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:20.319 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:20.320 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:20.321 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:20.321 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:20.321 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.321 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.322 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:20.322 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.323 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.323 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:20.323 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:20.325 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.325 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.326 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:20.326 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.326 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.327 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:20.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:20.328 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.328 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:20.329 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.329 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:20.330 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:20.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:20.331 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:20.331 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:20.332 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:20.332 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:20.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.332 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.333 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:20.333 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:20.333 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:20.333 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:20.334 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:20.334 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:20.334 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:20.334 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:20.334 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.335 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:20.335 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:20.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.335 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:20.335 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.336 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.336 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.336 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.336 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.336 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:20.337 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.337 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:20.337 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.338 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.338 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.338 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.338 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.339 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:20.339 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.339 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.339 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:20.340 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.340 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.340 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.340 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.340 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.341 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:20.341 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.341 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:20.342 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.342 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.342 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.342 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.344 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:20.344 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.344 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.344 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.344 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.345 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.345 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:20.345 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.346 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:20.346 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.347 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.347 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.347 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.347 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.347 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:20.348 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.348 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.348 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:20.349 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.349 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.349 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.349 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.349 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.350 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:20.350 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.350 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.350 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:20.350 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:20.351 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:20.351 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:20.351 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:20.351 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:20.352 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:20.353 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:20.353 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:20.353 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:20.353 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:20.353 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:20.354 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:20.354 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:20.354 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:20.354 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:20.354 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:20.355 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:20.355 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:20.355 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.355 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:20.356 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:20.356 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:20.356 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:20.357 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:20.357 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:20.357 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:20.357 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:20.358 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:20.358 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:20.358 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:20.358 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:20.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:20.359 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:20.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 msgr2=0x7fc9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:20.360 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 0x7fc9e00693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4008a80 tx=0x7fc9e4008040 comp rx=0 tx=0).stop 2023-10-10T20:30:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 msgr2=0x7fc9f40b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40c92a0 tx=0x7fc9ec018000 comp rx=0 tx=0).stop 2023-10-10T20:30:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 shutdown_connections 2023-10-10T20:30:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc9e0066f30 0x7fc9e00693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9f40c9930 0x7fc9f40b1980 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f4069de0 0x7fc9f41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 --2- 172.21.15.148:0/1685871652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f40c9560 0x7fc9f40b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 >> 172.21.15.148:0/1685871652 conn(0x7fc9f4067660 msgr2=0x7fc9f40ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 shutdown_connections 2023-10-10T20:30:20.363 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:20.144+0000 7fc9f9ab2700 1 -- 172.21.15.148:0/1685871652 wait complete. 2023-10-10T20:30:20.363 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:30:23.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:30:23.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:30:25.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:25.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.490+0000 7f146b32c700 1 Processor -- start 2023-10-10T20:30:25.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.490+0000 7f146b32c700 1 -- start start 2023-10-10T20:30:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640cd6e0 0x7f14640cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14640cdab0 0x7f146415d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.503 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f146415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146415e850 con 0x7f14640cd6e0 2023-10-10T20:30:25.504 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14640cd030 con 0x7f14640cdab0 2023-10-10T20:30:25.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146b32c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640cb7c0 con 0x7f146415dda0 2023-10-10T20:30:25.505 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146a32a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640cd6e0 0x7f14640cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146a32a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f14640cd6e0 0x7f14640cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43024/0 (socket says 172.21.15.148:43024) 2023-10-10T20:30:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146a32a700 1 -- 172.21.15.148:0/3056680553 learned_addr learned my addr 172.21.15.148:0/3056680553 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146a32a700 1 -- 172.21.15.148:0/3056680553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 msgr2=0x7f146415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.491+0000 7f146ab2b700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f146415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f1469b29700 1 --2- 172.21.15.148:0/3056680553 >> v2:172.21.15.161:3300/0 conn(0x7f14640cdab0 0x7f146415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146a32a700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f146415cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146a32a700 1 -- 172.21.15.148:0/3056680553 >> v2:172.21.15.161:3300/0 conn(0x7f14640cdab0 msgr2=0x7f146415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146a32a700 1 --2- 172.21.15.148:0/3056680553 >> v2:172.21.15.161:3300/0 conn(0x7f14640cdab0 0x7f146415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146a32a700 1 -- 172.21.15.148:0/3056680553 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146415e290 con 0x7f14640cd6e0 2023-10-10T20:30:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146a32a700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cd6e0 0x7f14640cbc50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f14640b6c70 tx=0x7f1460009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248ba8740792b651 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f1469328700 1 -- 172.21.15.148:0/3056680553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000b860 con 0x7f14640cd6e0 2023-10-10T20:30:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f1469328700 1 -- 172.21.15.148:0/3056680553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146000b9c0 con 0x7f14640cd6e0 2023-10-10T20:30:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f1469328700 1 -- 172.21.15.148:0/3056680553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000bd60 con 0x7f14640cd6e0 2023-10-10T20:30:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146b32c700 1 -- 172.21.15.148:0/3056680553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cd6e0 msgr2=0x7f14640cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:25.508 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.492+0000 7f146b32c700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cd6e0 0x7f14640cbc50 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f14640b6c70 tx=0x7f1460009250 comp rx=0 tx=0).stop 2023-10-10T20:30:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 -- 172.21.15.148:0/3056680553 shutdown_connections 2023-10-10T20:30:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f146415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 --2- 172.21.15.148:0/3056680553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cd6e0 0x7f14640cbc50 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 --2- 172.21.15.148:0/3056680553 >> v2:172.21.15.161:3300/0 conn(0x7f14640cdab0 0x7f146415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 -- 172.21.15.148:0/3056680553 >> 172.21.15.148:0/3056680553 conn(0x7f1464067660 msgr2=0x7f1464067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 -- 172.21.15.148:0/3056680553 shutdown_connections 2023-10-10T20:30:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.493+0000 7f146b32c700 1 -- 172.21.15.148:0/3056680553 wait complete. 2023-10-10T20:30:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 Processor -- start 2023-10-10T20:30:25.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 -- start start 2023-10-10T20:30:25.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14640cd6e0 0x7f14640c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f14640c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146415b7a0 con 0x7f14640cdab0 2023-10-10T20:30:25.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640c8690 con 0x7f14640cd6e0 2023-10-10T20:30:25.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.494+0000 7f146b32c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640c8970 con 0x7f146415dda0 2023-10-10T20:30:25.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43026/0 (socket says 172.21.15.148:43026) 2023-10-10T20:30:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 -- 172.21.15.148:0/1113218655 learned_addr learned my addr 172.21.15.148:0/1113218655 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f146a32a700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14640cd6e0 0x7f14640c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f146ab2b700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f14640c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 -- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 msgr2=0x7f14640c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f14640c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 -- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14640cd6e0 msgr2=0x7f14640c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14640cd6e0 0x7f14640c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600094f0 con 0x7f14640cdab0 2023-10-10T20:30:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f1469b29700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f145400c7d0 tx=0x7f1454008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454011600 con 0x7f14640cdab0 2023-10-10T20:30:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454011760 con 0x7f14640cdab0 2023-10-10T20:30:25.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14640c8c50 con 0x7f14640cdab0 2023-10-10T20:30:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454011b20 con 0x7f14640cdab0 2023-10-10T20:30:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.495+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14640c8260 con 0x7f14640cdab0 2023-10-10T20:30:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.496+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146415b290 con 0x7f14640cdab0 2023-10-10T20:30:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.496+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14540193f0 con 0x7f14640cdab0 2023-10-10T20:30:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.497+0000 7f145a7fc700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.497+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f14540847f0 con 0x7f14640cdab0 2023-10-10T20:30:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.497+0000 7f146a32a700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.497+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7f144806ea50 con 0x7f14640cdab0 2023-10-10T20:30:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.497+0000 7f146a32a700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 0x7f14480693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f14640b6c70 tx=0x7f1460017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:25.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.501+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f146415b290 con 0x7f14640cdab0 2023-10-10T20:30:25.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.687+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f146415b290 con 0x7f14640cdab0 2023-10-10T20:30:25.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.688+0000 7f145a7fc700 1 -- 172.21.15.148:0/1113218655 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v180) v1 ==== 81+0+32349 (secure 0 0 0) 0x7f146415b290 con 0x7f14640cdab0 2023-10-10T20:30:25.691 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:25.692 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:25.692 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 180, 2023-10-10T20:30:25.692 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:25.693 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:25.693 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:20.587875+0000", 2023-10-10T20:30:25.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:25.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:25.694 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:25.695 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:25.696 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:25.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:25.696 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:25.696 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:25.697 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:25.697 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:25.697 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:25.697 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:25.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.698 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:25.698 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:25.698 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:25.698 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:25.700 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:25.700 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:25.700 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:25.700 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:25.701 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:25.701 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:25.701 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:25.702 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:25.702 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:25.702 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:25.703 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:25.703 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:25.703 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:25.704 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:25.704 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:25.704 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:25.705 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:25.705 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:25.705 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:25.706 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:25.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:25.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:25.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:25.707 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:25.707 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:25.707 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:25.707 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:25.707 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:25.708 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:25.708 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:25.708 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:25.708 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:25.708 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:25.709 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:25.709 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:25.709 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:25.709 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:25.710 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:25.710 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:25.710 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:25.710 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:25.711 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:25.712 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:25.713 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:25.713 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:25.713 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:25.713 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:25.714 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:25.714 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:25.714 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:25.714 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:25.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:25.715 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:25.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.715 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:25.715 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:25.715 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.716 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.716 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:25.716 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:25.716 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:25.717 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:25.717 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:25.717 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:25.717 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:25.717 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:25.718 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:25.718 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:25.718 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:25.718 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:25.718 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:25.719 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:25.719 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:25.719 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:25.719 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:25.719 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:25.720 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:25.721 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:25.721 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:25.721 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:25.721 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:25.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.722 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:25.722 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:25.722 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:25.722 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:25.722 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:25.723 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:25.724 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:25.725 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:25.726 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.727 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:25.728 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:25.729 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:25.730 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.731 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.732 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:25.733 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:30:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:30:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:25.734 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:25.735 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:25.736 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:25.737 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:25.738 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:25.739 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:25.740 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "180", 2023-10-10T20:30:25.741 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:25.742 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:25.743 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:25.744 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:25.745 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:25.746 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:25.747 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:25.748 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:25.749 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:25.750 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:25.751 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:25.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.753 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.754 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.756 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.758 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.759 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:25.761 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:25.762 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:25.769 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.770 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:25.772 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.773 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.774 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.775 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.776 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.777 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:25.779 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.782 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.783 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:25.785 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.787 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.788 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:25.790 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:25.792 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:25.793 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.795 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.796 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.797 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.799 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.801 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.802 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.804 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.805 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.806 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:25.807 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:25.808 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:25.809 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.810 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:25.811 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.812 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.813 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.814 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.815 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.818 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:25.820 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:25.821 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.822 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:25.823 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:25.824 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:25.825 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.826 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:25.827 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.828 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.829 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.830 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:25.831 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.834 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.835 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:25.837 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.838 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.838 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.838 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.839 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.839 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.839 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:25.840 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.840 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.841 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.842 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.842 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.842 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:25.843 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.844 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:25.844 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:25.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.845 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.846 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:25.846 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:25.847 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:25.847 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:25.847 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.848 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.848 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.848 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.850 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:25.850 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.850 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.851 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.851 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.851 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.852 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.852 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.852 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.853 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.853 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.853 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.854 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.854 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.855 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:25.856 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.857 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.857 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:25.857 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.858 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.859 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.859 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.860 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.860 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.860 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:25.861 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.862 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.862 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:25.862 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.864 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.864 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.865 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.865 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:25.865 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.866 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.867 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:25.867 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.868 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.869 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.869 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.870 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:25.870 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.871 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.872 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.872 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:25.872 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:25.873 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.873 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.873 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.874 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:25.874 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:25.874 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:25.875 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:25.875 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.875 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.876 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.876 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.877 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.877 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.877 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:25.878 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.878 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.878 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.879 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.879 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.879 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.880 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.880 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.880 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.881 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.881 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.881 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.882 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.882 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.882 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:25.883 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.883 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.883 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.884 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:25.884 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.885 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.885 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.886 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.886 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.886 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:25.886 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.887 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.888 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.888 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.888 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.888 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.888 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.889 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.889 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:25.889 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.889 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.890 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.890 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.890 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:25.890 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.891 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.891 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.892 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.892 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.892 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.892 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.893 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:25.893 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.893 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.894 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.894 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.895 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:25.895 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:25.895 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.896 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.896 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:25.897 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:25.897 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:25.897 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:25.897 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.898 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.899 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:25.900 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:25.901 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:25.902 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:25.903 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:25.904 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.905 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.906 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.907 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.908 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.909 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.910 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:25.911 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.912 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:25.913 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:25.914 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.915 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:25.916 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.917 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:25.918 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:25.919 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:25.920 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.921 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.922 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.923 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.924 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.925 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.926 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.927 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.928 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.929 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:25.930 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.931 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.932 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:25.933 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.934 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:25.935 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:25.936 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:25.937 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:25.938 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:25.939 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:25.940 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:25.941 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:25.942 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.690+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 msgr2=0x7f14480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.690+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 0x7f14480693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f14640b6c70 tx=0x7f1460017000 comp rx=0 tx=0).stop 2023-10-10T20:30:25.944 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.690+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 msgr2=0x7f14640b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.690+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f145400c7d0 tx=0x7f1454008040 comp rx=0 tx=0).stop 2023-10-10T20:30:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 shutdown_connections 2023-10-10T20:30:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f146415dda0 0x7f14640c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14640cd6e0 0x7f14640c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 --2- 172.21.15.148:0/1113218655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14640cdab0 0x7f14640b5070 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:25.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 >> 172.21.15.148:0/1113218655 conn(0x7f1464067660 msgr2=0x7f1464156950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:25.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 shutdown_connections 2023-10-10T20:30:25.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:25.691+0000 7f146b32c700 1 -- 172.21.15.148:0/1113218655 wait complete. 2023-10-10T20:30:25.947 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:30:25.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:25.948 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-10T20:30:26.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:30:26.197 INFO:teuthology.orchestra.run.smithi148.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-10T20:30:26.197 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:30:26.198 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:30:26.198 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.229111 s, 229 MB/s 2023-10-10T20:30:26.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 Processor -- start 2023-10-10T20:30:26.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 -- start start 2023-10-10T20:30:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 0x7f00a015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00a0157c40 0x7f00a0158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a015cbb0 con 0x7f00a015a620 2023-10-10T20:30:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a015bfa0 con 0x7f00a0157c40 2023-10-10T20:30:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.311+0000 7f00a6163700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a015c280 con 0x7f00a0157870 2023-10-10T20:30:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00a0157c40 0x7f00a0158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00a0157c40 0x7f00a0158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43036/0 (socket says 172.21.15.148:43036) 2023-10-10T20:30:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009effd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f00a015a620 0x7f00a015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50644/0 (socket says 172.21.15.148:50644) 2023-10-10T20:30:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 -- 172.21.15.148:0/348220311 learned_addr learned my addr 172.21.15.148:0/348220311 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:26.317 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009f7fe700 1 --2- 172.21.15.148:0/348220311 >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 0x7f00a015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 -- 172.21.15.148:0/348220311 >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 msgr2=0x7f00a015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 --2- 172.21.15.148:0/348220311 >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 0x7f00a015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 -- 172.21.15.148:0/348220311 >> v1:172.21.15.148:6789/0 conn(0x7f00a015a620 legacy=0x7f00a015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009e7fc700 1 -- 172.21.15.148:0/348220311 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611779871 0 0) 0x7f00a015cbb0 con 0x7f00a015a620 2023-10-10T20:30:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 -- 172.21.15.148:0/348220311 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a015c560 con 0x7f00a0157c40 2023-10-10T20:30:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009f7fe700 1 --2- 172.21.15.148:0/348220311 >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 0x7f00a015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009ffff700 1 --2- 172.21.15.148:0/348220311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a0157c40 0x7f00a0158110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f00a0069d80 tx=0x7f0098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b256a59863cbabb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009e7fc700 1 -- 172.21.15.148:0/348220311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800b860 con 0x7f00a0157c40 2023-10-10T20:30:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009e7fc700 1 -- 172.21.15.148:0/348220311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800b9c0 con 0x7f00a0157c40 2023-10-10T20:30:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.312+0000 7f009e7fc700 1 -- 172.21.15.148:0/348220311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800bd60 con 0x7f00a0157c40 2023-10-10T20:30:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 -- 172.21.15.148:0/348220311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a0157c40 msgr2=0x7f00a0158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 --2- 172.21.15.148:0/348220311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a0157c40 0x7f00a0158110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f00a0069d80 tx=0x7f0098009250 comp rx=0 tx=0).stop 2023-10-10T20:30:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 -- 172.21.15.148:0/348220311 shutdown_connections 2023-10-10T20:30:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 --2- 172.21.15.148:0/348220311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a0157c40 0x7f00a0158110 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 --2- 172.21.15.148:0/348220311 >> v2:172.21.15.161:3300/0 conn(0x7f00a0157870 0x7f00a015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 -- 172.21.15.148:0/348220311 >> 172.21.15.148:0/348220311 conn(0x7f00a0068ce0 msgr2=0x7f00a00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 -- 172.21.15.148:0/348220311 shutdown_connections 2023-10-10T20:30:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.313+0000 7f00a6163700 1 -- 172.21.15.148:0/348220311 wait complete. 2023-10-10T20:30:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.314+0000 7f00a6163700 1 Processor -- start 2023-10-10T20:30:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.314+0000 7f00a6163700 1 -- start start 2023-10-10T20:30:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00a0157870 0x7f00a014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00a0157c40 0x7f00a0150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a014e600 con 0x7f00a015a620 2023-10-10T20:30:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a014dff0 con 0x7f00a0157c40 2023-10-10T20:30:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0153a10 con 0x7f00a0157870 2023-10-10T20:30:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43048/0 (socket says 172.21.15.148:43048) 2023-10-10T20:30:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 -- 172.21.15.148:0/1571864523 learned_addr learned my addr 172.21.15.148:0/1571864523 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 -- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00a0157870 msgr2=0x7f00a014c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00a0157870 0x7f00a014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009effd700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00a0157c40 0x7f00a0150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 -- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00a0157c40 msgr2=0x7f00a0150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00a0157c40 0x7f00a0150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980094f0 con 0x7f00a015a620 2023-10-10T20:30:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f009ffff700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f00980090b0 tx=0x7f0098017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009801a640 con 0x7f00a015a620 2023-10-10T20:30:26.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009801f070 con 0x7f00a015a620 2023-10-10T20:30:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a014e290 con 0x7f00a015a620 2023-10-10T20:30:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.315+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009801a9f0 con 0x7f00a015a620 2023-10-10T20:30:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.316+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a014cec0 con 0x7f00a015a620 2023-10-10T20:30:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.316+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a004c3c0 con 0x7f00a015a620 2023-10-10T20:30:26.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.317+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0098026480 con 0x7f00a015a620 2023-10-10T20:30:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.317+0000 7f00877fe700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 0x7f008c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.317+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0098021070 con 0x7f00a015a620 2023-10-10T20:30:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.317+0000 7f009f7fe700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 0x7f008c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.317+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7f008c06ea80 con 0x7f00a015a620 2023-10-10T20:30:26.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.318+0000 7f009f7fe700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 0x7f008c069410 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f00900094d0 tx=0x7f00900096b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.322+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f009802b060 con 0x7f00a015a620 2023-10-10T20:30:26.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.504+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f00a014e740 con 0x7f00a015a620 2023-10-10T20:30:26.507 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.505+0000 7f00877fe700 1 -- 172.21.15.148:0/1571864523 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v180) v1 ==== 81+0+32349 (secure 0 0 0) 0x7f0098015020 con 0x7f00a015a620 2023-10-10T20:30:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:26.508 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 180, 2023-10-10T20:30:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:26.508 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:20.587875+0000", 2023-10-10T20:30:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:26.509 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:26.510 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:26.511 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:26.512 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:26.513 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:26.514 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:26.515 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:26.516 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:26.517 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.518 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:26.519 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:26.520 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:26.521 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:26.522 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:26.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:26.524 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:26.525 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:26.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.527 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:26.528 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:26.529 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:26.530 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:26.531 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:26.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:26.533 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:26.534 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:26.535 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:26.536 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:26.537 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:26.538 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:26.539 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:26.540 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:26.541 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:26.542 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:26.543 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:26.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:26.546 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:30:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:30:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:26.547 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:26.548 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:26.549 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:26.550 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.551 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "180", 2023-10-10T20:30:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:26.552 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:26.553 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:26.554 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:26.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:26.556 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:26.557 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:26.559 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:26.560 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:26.561 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.562 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.563 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.564 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:26.565 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.566 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.568 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.569 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.571 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.572 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.574 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.575 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.576 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.577 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:26.578 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.581 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:26.582 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.584 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.585 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.586 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.587 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.589 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.590 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.591 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:26.592 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:26.593 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.594 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.595 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.596 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.599 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.600 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.601 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.602 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.603 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.604 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.605 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.606 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:26.607 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.608 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:26.609 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.610 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.611 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.612 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.613 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:26.615 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.616 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.617 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.618 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.619 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.620 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.621 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.622 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.623 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.625 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.626 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.627 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.628 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.629 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.630 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.631 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.632 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.634 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.635 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.636 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.637 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:26.638 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.639 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.640 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:26.641 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.642 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.642 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.642 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.642 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:26.643 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.643 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.643 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.643 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.643 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.644 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.644 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.644 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.644 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:26.645 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.645 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.645 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.645 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.645 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:26.646 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:26.646 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.646 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.647 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.647 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:26.647 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:26.647 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:26.647 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:26.648 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.648 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.648 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.648 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.649 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:26.649 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.650 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.650 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.650 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.651 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.651 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.651 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.651 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.652 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.652 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.652 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.653 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.653 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.653 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.654 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:26.654 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.654 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.656 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:26.657 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.657 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.657 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.658 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.660 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.660 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:26.660 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.662 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.662 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:26.662 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.662 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.662 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.663 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.663 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:26.663 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:26.663 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.664 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.664 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.664 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:26.664 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:26.664 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.665 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.666 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.666 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:26.666 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.666 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.666 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.667 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.667 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.667 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.667 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.667 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.668 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.668 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.668 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.668 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.668 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.669 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.670 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.670 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:26.670 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.670 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.670 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.671 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.671 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.671 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.671 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.671 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.672 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:26.672 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.672 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.672 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.673 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:26.673 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.673 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.674 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.674 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.674 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.675 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.676 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:26.676 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.677 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.677 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.677 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.678 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.678 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:26.678 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.679 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.679 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:26.679 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:26.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.680 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.680 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:26.681 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:26.681 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:26.681 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:26.682 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.682 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.682 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.682 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.683 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:26.684 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:26.684 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:26.684 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:26.685 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:26.685 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:26.685 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:26.686 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:26.686 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:26.686 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:26.687 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:26.687 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:26.687 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:26.687 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.688 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.689 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:26.689 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.689 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.689 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.689 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.690 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:26.690 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.690 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.690 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.690 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:26.691 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.691 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.691 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.691 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.691 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:26.692 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.692 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.692 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.692 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.693 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:26.694 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.694 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.694 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.694 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.694 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:26.695 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.695 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.695 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.695 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.695 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:26.696 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:26.696 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.696 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:26.696 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:26.697 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.697 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.697 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.697 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:26.697 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:26.698 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:26.698 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.698 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.699 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:26.699 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:26.699 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:26.699 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:26.699 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:26.700 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.701 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.703 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.703 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:26.703 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.703 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.703 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.704 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.704 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.704 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:26.704 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.704 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.705 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.706 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:26.706 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.706 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.706 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.707 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:26.707 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.707 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.707 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.707 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.708 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:26.708 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.709 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.709 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:26.709 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.709 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.709 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.710 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.710 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.710 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:26.710 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.710 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.711 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.712 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.712 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.712 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:26.712 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.712 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.713 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.713 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:26.713 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.713 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.714 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.714 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.714 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.714 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:26.715 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.715 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:26.715 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:26.715 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:26.716 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:26.716 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:26.716 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:26.716 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:26.717 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:26.717 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:26.717 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.717 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:26.717 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:26.718 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:26.718 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:26.718 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:26.718 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:26.718 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:26.719 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:26.719 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:26.719 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:26.719 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:26.720 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:26.720 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:26.720 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:26.720 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:26.721 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:26.721 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:26.721 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:26.722 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:26.722 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:26.722 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:26.722 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:26.723 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:26.723 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:26.723 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:26.723 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:26.724 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:26.724 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:26.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 msgr2=0x7f008c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 0x7f008c069410 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f00900094d0 tx=0x7f00900096b0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 msgr2=0x7f00a00b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f00980090b0 tx=0x7f0098017000 comp rx=0 tx=0).stop 2023-10-10T20:30:26.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 shutdown_connections 2023-10-10T20:30:26.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f008c066f60 0x7f008c069410 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00a015a620 0x7f00a00b1690 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00a0157870 0x7f00a014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 --2- 172.21.15.148:0/1571864523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00a0157c40 0x7f00a0150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 >> 172.21.15.148:0/1571864523 conn(0x7f00a0068ce0 msgr2=0x7f00a00d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 shutdown_connections 2023-10-10T20:30:26.728 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.507+0000 7f00a6163700 1 -- 172.21.15.148:0/1571864523 wait complete. 2023-10-10T20:30:26.729 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:26.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 Processor -- start 2023-10-10T20:30:26.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 -- start start 2023-10-10T20:30:26.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d4157c40 0x7f47d4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d415cbb0 con 0x7f47d415a620 2023-10-10T20:30:26.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d415bfa0 con 0x7f47d4157870 2023-10-10T20:30:26.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.842+0000 7f47da286700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d415bdb0 con 0x7f47d4157c40 2023-10-10T20:30:26.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d4157c40 0x7f47d4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47d4157c40 0x7f47d4158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43064/0 (socket says 172.21.15.148:43064) 2023-10-10T20:30:26.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d37fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f47d415a620 0x7f47d415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50652/0 (socket says 172.21.15.148:50652) 2023-10-10T20:30:26.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 -- 172.21.15.148:0/3521899087 learned_addr learned my addr 172.21.15.148:0/3521899087 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:26.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 -- 172.21.15.148:0/3521899087 >> v1:172.21.15.161:6789/0 conn(0x7f47d4157870 legacy=0x7f47d415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:26.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 -- 172.21.15.148:0/3521899087 >> v1:172.21.15.148:6789/0 conn(0x7f47d415a620 legacy=0x7f47d415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 -- 172.21.15.148:0/3521899087 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d415c280 con 0x7f47d4157c40 2023-10-10T20:30:26.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d2ffd700 1 -- 172.21.15.148:0/3521899087 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830931095 0 0) 0x7f47d415cbb0 con 0x7f47d415a620 2023-10-10T20:30:26.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d8823700 1 --2- 172.21.15.148:0/3521899087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d4158110 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009f20 tx=0x7f47cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c69e4f866c9c551 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d2ffd700 1 -- 172.21.15.148:0/3521899087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00e070 con 0x7f47d4157c40 2023-10-10T20:30:26.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d2ffd700 1 -- 172.21.15.148:0/3521899087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00ba40 con 0x7f47d4157c40 2023-10-10T20:30:26.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.843+0000 7f47d2ffd700 1 -- 172.21.15.148:0/3521899087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0092e0 con 0x7f47d4157c40 2023-10-10T20:30:26.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 -- 172.21.15.148:0/3521899087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 msgr2=0x7f47d4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 --2- 172.21.15.148:0/3521899087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d4158110 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009f20 tx=0x7f47cc009840 comp rx=0 tx=0).stop 2023-10-10T20:30:26.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 -- 172.21.15.148:0/3521899087 shutdown_connections 2023-10-10T20:30:26.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 --2- 172.21.15.148:0/3521899087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d4158110 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 -- 172.21.15.148:0/3521899087 >> 172.21.15.148:0/3521899087 conn(0x7f47d4068ce0 msgr2=0x7f47d40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:26.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 -- 172.21.15.148:0/3521899087 shutdown_connections 2023-10-10T20:30:26.851 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.844+0000 7f47da286700 1 -- 172.21.15.148:0/3521899087 wait complete. 2023-10-10T20:30:26.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.845+0000 7f47da286700 1 Processor -- start 2023-10-10T20:30:26.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.845+0000 7f47da286700 1 -- start start 2023-10-10T20:30:26.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d4157870 0x7f47d40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d415a620 0x7f47d4152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d414e830 con 0x7f47d4157c40 2023-10-10T20:30:26.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b16b0 con 0x7f47d415a620 2023-10-10T20:30:26.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b1990 con 0x7f47d4157870 2023-10-10T20:30:26.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.855 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43076/0 (socket says 172.21.15.148:43076) 2023-10-10T20:30:26.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 -- 172.21.15.148:0/2323128711 learned_addr learned my addr 172.21.15.148:0/2323128711 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:26.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d3fff700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d4157870 0x7f47d40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d8823700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d415a620 0x7f47d4152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 -- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d4157870 msgr2=0x7f47d40b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d4157870 0x7f47d40b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 -- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d415a620 msgr2=0x7f47d4152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:26.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d415a620 0x7f47d4152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:26.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009ae0 con 0x7f47d4157c40 2023-10-10T20:30:26.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47d37fe700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c510 tx=0x7f47c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8005a70 con 0x7f47d4157c40 2023-10-10T20:30:26.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8005bf0 con 0x7f47d4157c40 2023-10-10T20:30:26.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40b1c70 con 0x7f47d4157c40 2023-10-10T20:30:26.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80070d0 con 0x7f47d4157c40 2023-10-10T20:30:26.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.846+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d413ec20 con 0x7f47d4157c40 2023-10-10T20:30:26.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.847+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d414e600 con 0x7f47d4157c40 2023-10-10T20:30:26.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.848+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47c8007440 con 0x7f47d4157c40 2023-10-10T20:30:26.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.848+0000 7f47bbfff700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:26.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.848+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47c8083a60 con 0x7f47d4157c40 2023-10-10T20:30:26.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.848+0000 7f47d3fff700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:26.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.848+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7f47c006ea50 con 0x7f47d4157c40 2023-10-10T20:30:26.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.849+0000 7f47d3fff700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 0x7f47c00693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f47c40027e0 tx=0x7f47c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:26.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:26.853+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c8017020 con 0x7f47d4157c40 2023-10-10T20:30:27.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.076+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d404c3c0 con 0x7f47d4157c40 2023-10-10T20:30:27.079 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.077+0000 7f47bbfff700 1 -- 172.21.15.148:0/2323128711 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f47d414e600 con 0x7f47d4157c40 2023-10-10T20:30:27.080 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:27.080 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:27.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 msgr2=0x7f47c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.082 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 0x7f47c00693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f47c40027e0 tx=0x7f47c4009040 comp rx=0 tx=0).stop 2023-10-10T20:30:27.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 msgr2=0x7f47d413e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c510 tx=0x7f47c800c8d0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 shutdown_connections 2023-10-10T20:30:27.083 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47d415a620 0x7f47d4152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f47d4157870 0x7f47d40b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.084 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 --2- 172.21.15.148:0/2323128711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d4157c40 0x7f47d413e6e0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 >> 172.21.15.148:0/2323128711 conn(0x7f47d4068ce0 msgr2=0x7f47d40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:27.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 shutdown_connections 2023-10-10T20:30:27.085 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.079+0000 7f47da286700 1 -- 172.21.15.148:0/2323128711 wait complete. 2023-10-10T20:30:27.085 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:27.093 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:30:27.234 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":8139,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.342295158999999,"mdsmap_epoch":85,"osdmap_epoch":180,"osdmap_epoch_barrier":180,"uptime":27.453163863} 2023-10-10T20:30:27.235 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 8139, "mdsmap_epoch": 85, "osdmap_epoch": 180, "osdmap_epoch_barrier": 180, "rank_uptime": 22.342295159, "state": "up:active", "uptime": 27.453163863, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:30:27.235 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:30:27.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:27.236 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-10T20:30:27.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:27.253 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-10T20:30:27.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:27.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.467+0000 7f684eba9700 1 Processor -- start 2023-10-10T20:30:27.470 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.467+0000 7f684eba9700 1 -- start start 2023-10-10T20:30:27.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684eba9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157870 0x7f684815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684eba9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6848157c40 0x7f6848158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684eba9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684815cbb0 con 0x7f684815a620 2023-10-10T20:30:27.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684eba9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684815bfa0 con 0x7f6848157c40 2023-10-10T20:30:27.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684eba9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684815c280 con 0x7f6848157870 2023-10-10T20:30:27.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684d146700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6848157c40 0x7f6848158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f683ffff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f684815a620 0x7f684815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50664/0 (socket says 172.21.15.148:50664) 2023-10-10T20:30:27.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f683ffff700 1 -- 172.21.15.148:0/1715366337 learned_addr learned my addr 172.21.15.148:0/1715366337 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:27.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.468+0000 7f684c945700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157870 0x7f684815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684d146700 1 -- 172.21.15.148:0/1715366337 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157870 msgr2=0x7f684815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:27.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684d146700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157870 0x7f684815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684d146700 1 -- 172.21.15.148:0/1715366337 >> v1:172.21.15.148:6789/0 conn(0x7f684815a620 legacy=0x7f684815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684d146700 1 -- 172.21.15.148:0/1715366337 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684815c560 con 0x7f6848157c40 2023-10-10T20:30:27.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f683f7fe700 1 -- 172.21.15.148:0/1715366337 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777580426 0 0) 0x7f684815cbb0 con 0x7f684815a620 2023-10-10T20:30:27.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684d146700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157c40 0x7f6848158110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f68400098e0 tx=0x7f68400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958f106070be8259 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:27.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f683f7fe700 1 -- 172.21.15.148:0/1715366337 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000b640 con 0x7f6848157c40 2023-10-10T20:30:27.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f683f7fe700 1 -- 172.21.15.148:0/1715366337 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684000b7a0 con 0x7f6848157c40 2023-10-10T20:30:27.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f683f7fe700 1 -- 172.21.15.148:0/1715366337 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000bb40 con 0x7f6848157c40 2023-10-10T20:30:27.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684eba9700 1 -- 172.21.15.148:0/1715366337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157c40 msgr2=0x7f6848158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.469+0000 7f684eba9700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157c40 0x7f6848158110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f68400098e0 tx=0x7f68400091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 -- 172.21.15.148:0/1715366337 shutdown_connections 2023-10-10T20:30:27.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157c40 0x7f6848158110 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 --2- 172.21.15.148:0/1715366337 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157870 0x7f684815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 -- 172.21.15.148:0/1715366337 >> 172.21.15.148:0/1715366337 conn(0x7f6848068ce0 msgr2=0x7f68480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:27.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 -- 172.21.15.148:0/1715366337 shutdown_connections 2023-10-10T20:30:27.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.470+0000 7f684eba9700 1 -- 172.21.15.148:0/1715366337 wait complete. 2023-10-10T20:30:27.479 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.471+0000 7f684eba9700 1 Processor -- start 2023-10-10T20:30:27.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.471+0000 7f684eba9700 1 -- start start 2023-10-10T20:30:27.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157c40 0x7f68481446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f684815a620 0x7f6848158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.483 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6848152ca0 con 0x7f6848157870 2023-10-10T20:30:27.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6848152f80 con 0x7f684815a620 2023-10-10T20:30:27.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6848153260 con 0x7f6848157c40 2023-10-10T20:30:27.484 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43096/0 (socket says 172.21.15.148:43096) 2023-10-10T20:30:27.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 -- 172.21.15.148:0/419501759 learned_addr learned my addr 172.21.15.148:0/419501759 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:27.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f683ffff700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157c40 0x7f68481446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.485 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684d146700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f684815a620 0x7f6848158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 -- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157c40 msgr2=0x7f68481446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157c40 0x7f68481446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.486 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 -- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f684815a620 msgr2=0x7f6848158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f684815a620 0x7f6848158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009420 con 0x7f6848157870 2023-10-10T20:30:27.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684c945700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6844009f10 tx=0x7f684400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:27.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400cb70 con 0x7f6848157870 2023-10-10T20:30:27.487 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400f040 con 0x7f6848157870 2023-10-10T20:30:27.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844014480 con 0x7f6848157870 2023-10-10T20:30:27.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6848153540 con 0x7f6848157870 2023-10-10T20:30:27.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.472+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6848158d70 con 0x7f6848157870 2023-10-10T20:30:27.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.473+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684804c3c0 con 0x7f6848157870 2023-10-10T20:30:27.488 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6844014830 con 0x7f6848157870 2023-10-10T20:30:27.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683cff9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 0x7f68340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:27.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6844084eb0 con 0x7f6848157870 2023-10-10T20:30:27.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683ffff700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 0x7f68340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:27.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7f683406ea50 con 0x7f6848157870 2023-10-10T20:30:27.489 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.474+0000 7f683ffff700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 0x7f68340693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f68481570b0 tx=0x7f6838007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:27.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.478+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6844052120 con 0x7f6848157870 2023-10-10T20:30:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.663+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f684815c840 con 0x7f6848157870 2023-10-10T20:30:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.664+0000 7f683cff9700 1 -- 172.21.15.148:0/419501759 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v180) v1 ==== 81+0+32349 (secure 0 0 0) 0x7f6844018030 con 0x7f6848157870 2023-10-10T20:30:27.667 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:27.667 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:27.668 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 180, 2023-10-10T20:30:27.668 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:27.668 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:27.668 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:20.587875+0000", 2023-10-10T20:30:27.668 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:27.669 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:27.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:27.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:27.669 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:27.669 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:27.670 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:27.671 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:27.671 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:27.671 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:27.671 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:27.671 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:27.685 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:27.686 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:27.686 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:27.686 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:27.686 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.686 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:27.687 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:27.687 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.687 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.687 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.687 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:27.688 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:27.688 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:27.688 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:27.688 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:27.688 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:27.689 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:27.690 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:27.691 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:27.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:27.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:27.693 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:27.693 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:27.694 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:27.694 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:27.694 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:27.694 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:27.694 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:27.695 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:27.695 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:27.695 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:27.695 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:27.695 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:27.696 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:27.697 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:27.697 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:27.697 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:27.697 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:27.697 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:27.698 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:27.698 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.698 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:27.698 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:27.698 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:27.699 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:27.700 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:27.700 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:27.700 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:27.700 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:27.700 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:27.701 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:27.701 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.701 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:27.701 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:27.701 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:27.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.702 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.702 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.702 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.702 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:27.703 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:27.703 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:27.703 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:30:27.703 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:30:27.703 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:27.704 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:27.704 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:27.704 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:27.704 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:27.704 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:27.705 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:27.706 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:27.707 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:27.707 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:27.707 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:27.707 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:27.707 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "180", 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:27.708 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:27.709 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:27.709 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:27.709 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:27.709 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:27.709 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:27.710 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:27.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:27.711 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:27.711 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:27.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:27.711 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:27.712 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:27.712 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:27.712 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:27.712 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:27.712 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:27.713 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:27.714 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:27.714 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:27.714 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:27.714 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:27.714 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:27.715 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.716 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:27.716 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:27.716 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:27.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.716 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.717 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.718 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.719 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:27.720 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.721 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.721 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.721 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.721 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:27.721 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.722 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.722 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.722 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.722 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.723 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.724 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.724 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:27.724 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.724 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.724 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.725 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.725 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.725 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.725 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.726 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:27.727 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.727 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.727 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.727 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.727 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.728 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.728 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.728 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.728 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:27.728 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:27.729 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:27.730 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.732 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.732 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:27.732 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.732 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.732 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.733 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.734 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.734 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.734 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.734 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.734 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.735 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.736 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:27.736 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.736 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.736 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.737 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.737 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.737 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.737 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:27.738 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.739 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.739 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.739 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.739 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.740 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.741 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.741 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:27.741 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.741 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.742 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.742 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.742 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.742 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.743 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:27.744 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:27.744 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.744 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.744 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.744 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.745 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.746 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.746 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.746 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:27.746 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.747 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.747 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.747 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.747 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.747 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.748 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.749 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.749 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.749 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.749 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:27.749 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.750 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.751 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.751 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.751 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.751 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.752 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.752 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:27.752 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.752 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.754 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.754 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.754 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.754 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.754 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:27.755 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.755 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.755 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.755 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:27.756 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.756 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.756 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.756 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.757 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.757 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.757 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.757 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:27.757 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:27.758 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.759 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.759 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:27.759 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:27.759 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:27.760 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:27.760 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.760 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.760 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.761 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.762 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.763 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.764 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.764 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:27.764 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.766 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.766 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.766 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.766 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:27.767 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.768 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.768 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.769 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.770 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.771 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:27.772 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.772 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.772 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.772 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:27.773 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:27.774 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:27.774 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:27.774 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.774 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.774 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.775 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.776 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.776 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.776 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.776 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.776 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.777 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.778 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.778 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:27.778 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.779 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:27.779 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.779 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.780 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.781 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:27.781 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.781 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.782 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.783 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.783 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.783 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.783 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:27.783 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.785 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.785 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.785 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.786 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.786 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:27.786 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.788 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:27.788 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:27.788 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:27.788 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:27.788 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.789 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.789 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.789 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.791 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.791 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.791 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.791 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.791 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.792 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.793 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:27.793 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.793 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.794 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:27.794 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.794 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.794 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:27.796 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.796 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.796 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.797 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.798 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.798 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:27.798 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.799 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.799 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:27.799 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.800 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.801 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:27.801 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.801 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.801 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.801 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.802 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:27.803 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:27.803 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:27.803 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:27.804 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.804 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.804 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.804 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.805 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.806 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.807 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.807 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.807 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.807 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.807 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.808 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.808 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:27.808 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.808 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.809 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.809 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:27.809 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.809 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.810 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.811 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:27.812 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.812 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.812 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.813 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.813 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.813 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.813 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:27.813 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.814 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.814 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:27.814 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.815 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.816 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.816 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:27.816 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.816 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.816 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:27.817 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:27.817 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.817 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:27.818 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.819 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.820 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:27.820 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:27.820 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:27.820 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:27.821 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:27.822 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:27.822 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:27.822 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:27.822 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.824 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.824 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:27.824 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.824 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.825 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:27.826 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.826 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:27.827 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.827 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:27.828 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.828 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.828 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.828 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:27.828 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.829 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.829 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:27.829 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.830 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.830 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.830 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.830 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:27.831 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:27.831 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.831 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:27.831 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:27.831 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:27.832 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.833 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.833 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:27.833 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:27.833 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:27.834 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.835 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:27.836 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.836 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.836 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.836 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.836 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.837 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:27.837 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.837 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.837 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:27.838 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.838 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.838 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.838 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.838 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.839 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:27.840 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.840 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.840 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.840 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.840 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.841 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:27.841 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.841 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.841 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:27.842 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.842 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.842 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.842 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.842 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.843 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:27.843 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.843 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:27.844 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.844 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.844 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.844 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.844 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:27.845 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.846 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.846 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.846 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.846 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.846 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:27.847 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.847 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:27.847 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.848 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.848 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.848 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.848 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.848 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:27.849 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:27.850 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:27.850 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:27.850 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:27.850 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:27.850 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:27.851 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.851 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:27.851 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:27.851 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:27.851 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:27.852 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:27.852 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:27.852 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:27.852 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:27.852 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:27.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:27.854 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:27.854 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:27.854 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:27.854 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:27.855 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:27.856 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:27.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:27.856 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:27.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 msgr2=0x7f68340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 0x7f68340693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f68481570b0 tx=0x7f6838007f50 comp rx=0 tx=0).stop 2023-10-10T20:30:27.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 msgr2=0x7f684816a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:27.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6844009f10 tx=0x7f684400b040 comp rx=0 tx=0).stop 2023-10-10T20:30:27.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 shutdown_connections 2023-10-10T20:30:27.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6834066f30 0x7f68340693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f684815a620 0x7f6848158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6848157870 0x7f684816a010 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 --2- 172.21.15.148:0/419501759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6848157c40 0x7f68481446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:27.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.666+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 >> 172.21.15.148:0/419501759 conn(0x7f6848068ce0 msgr2=0x7f68480d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:27.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.667+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 shutdown_connections 2023-10-10T20:30:27.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:27.667+0000 7f684eba9700 1 -- 172.21.15.148:0/419501759 wait complete. 2023-10-10T20:30:27.861 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:30:32.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:32.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.981+0000 7fbf59153700 1 Processor -- start 2023-10-10T20:30:32.984 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.981+0000 7fbf59153700 1 -- start start 2023-10-10T20:30:32.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf59153700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf541523e0 0x7fbf5406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:32.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf59153700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5406c010 con 0x7fbf54158e50 2023-10-10T20:30:32.987 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf59153700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5406bd10 con 0x7fbf54152010 2023-10-10T20:30:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf59153700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5415c060 con 0x7fbf541523e0 2023-10-10T20:30:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf4a59c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf541523e0 0x7fbf5406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf5359e700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbf54158e50 0x7fbf540cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51660/0 (socket says 172.21.15.148:51660) 2023-10-10T20:30:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.982+0000 7fbf5359e700 1 -- 172.21.15.148:0/1797165611 learned_addr learned my addr 172.21.15.148:0/1797165611 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:32.988 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf4a59c700 1 -- 172.21.15.148:0/1797165611 >> v1:172.21.15.161:6789/0 conn(0x7fbf54152010 legacy=0x7fbf5415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf4a59c700 1 -- 172.21.15.148:0/1797165611 >> v1:172.21.15.148:6789/0 conn(0x7fbf54158e50 legacy=0x7fbf540cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf4a59c700 1 -- 172.21.15.148:0/1797165611 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5415c340 con 0x7fbf541523e0 2023-10-10T20:30:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf5259c700 1 -- 172.21.15.148:0/1797165611 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669930226 0 0) 0x7fbf5406c010 con 0x7fbf54158e50 2023-10-10T20:30:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf4a59c700 1 --2- 172.21.15.148:0/1797165611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf541523e0 0x7fbf5406c4a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0098e0 tx=0x7fbf3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b791202fd36ea75c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf5259c700 1 -- 172.21.15.148:0/1797165611 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c009e80 con 0x7fbf541523e0 2023-10-10T20:30:32.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf5259c700 1 -- 172.21.15.148:0/1797165611 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c00b860 con 0x7fbf541523e0 2023-10-10T20:30:32.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.983+0000 7fbf5259c700 1 -- 172.21.15.148:0/1797165611 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00bbe0 con 0x7fbf541523e0 2023-10-10T20:30:32.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 -- 172.21.15.148:0/1797165611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf541523e0 msgr2=0x7fbf5406c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:32.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 --2- 172.21.15.148:0/1797165611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf541523e0 0x7fbf5406c4a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0098e0 tx=0x7fbf3c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:32.990 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 -- 172.21.15.148:0/1797165611 shutdown_connections 2023-10-10T20:30:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 --2- 172.21.15.148:0/1797165611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf541523e0 0x7fbf5406c4a0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 -- 172.21.15.148:0/1797165611 >> 172.21.15.148:0/1797165611 conn(0x7fbf54069870 msgr2=0x7fbf54069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 -- 172.21.15.148:0/1797165611 shutdown_connections 2023-10-10T20:30:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.984+0000 7fbf59153700 1 -- 172.21.15.148:0/1797165611 wait complete. 2023-10-10T20:30:32.991 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.985+0000 7fbf59153700 1 Processor -- start 2023-10-10T20:30:32.992 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.985+0000 7fbf59153700 1 -- start start 2023-10-10T20:30:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf54152010 0x7fbf5414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:32.994 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf541523e0 0x7fbf5414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540c6990 con 0x7fbf54158e50 2023-10-10T20:30:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5414ea70 con 0x7fbf541523e0 2023-10-10T20:30:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5414f5a0 con 0x7fbf54152010 2023-10-10T20:30:32.995 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46720/0 (socket says 172.21.15.148:46720) 2023-10-10T20:30:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 -- 172.21.15.148:0/2796525400 learned_addr learned my addr 172.21.15.148:0/2796525400 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf4a59c700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf541523e0 0x7fbf5414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf52d9d700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf54152010 0x7fbf5414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:32.996 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 -- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf54152010 msgr2=0x7fbf5414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf54152010 0x7fbf5414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 -- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf541523e0 msgr2=0x7fbf5414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf541523e0 0x7fbf5414eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:32.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c009420 con 0x7fbf54158e50 2023-10-10T20:30:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf5359e700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00cc10 tx=0x7fbf4c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00a850 con 0x7fbf54158e50 2023-10-10T20:30:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00a9d0 con 0x7fbf54158e50 2023-10-10T20:30:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5414f880 con 0x7fbf54158e50 2023-10-10T20:30:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00ad90 con 0x7fbf54158e50 2023-10-10T20:30:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.986+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf540c6540 con 0x7fbf54158e50 2023-10-10T20:30:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.987+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf540c6b00 con 0x7fbf54158e50 2023-10-10T20:30:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.988+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf4c007300 con 0x7fbf54158e50 2023-10-10T20:30:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.988+0000 7fbf4b7fe700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 0x7fbf340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.988+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf4c0847d0 con 0x7fbf54158e50 2023-10-10T20:30:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.988+0000 7fbf52d9d700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 0x7fbf340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.988+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7fbf3406ea50 con 0x7fbf54158e50 2023-10-10T20:30:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.989+0000 7fbf52d9d700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 0x7fbf340693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbf44003a80 tx=0x7fbf44007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:32.993+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf4c051ac0 con 0x7fbf54158e50 2023-10-10T20:30:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.176+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbf5404c3c0 con 0x7fbf54158e50 2023-10-10T20:30:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.177+0000 7fbf4b7fe700 1 -- 172.21.15.148:0/2796525400 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v180) v1 ==== 81+0+32349 (secure 0 0 0) 0x7fbf4c0550e0 con 0x7fbf54158e50 2023-10-10T20:30:33.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:33.181 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:33.181 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 180, 2023-10-10T20:30:33.182 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:33.182 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:33.182 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:20.587875+0000", 2023-10-10T20:30:33.182 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:33.183 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:33.183 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:33.183 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:33.183 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:33.184 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:33.184 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:33.184 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:33.184 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:33.184 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:33.185 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:33.186 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:33.186 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:33.186 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:33.186 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:33.186 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:33.187 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:33.187 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.187 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:33.187 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:33.187 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:33.188 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:33.188 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:33.188 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:33.188 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:33.189 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:33.189 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:33.189 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:33.189 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:33.190 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:33.190 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:33.190 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:33.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:33.190 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:33.191 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:33.191 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:33.191 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:33.191 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:33.191 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:33.192 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:33.192 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:33.192 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:33.192 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:33.192 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:33.193 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:33.193 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.193 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:33.193 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:33.194 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:33.194 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:33.194 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:33.194 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:33.194 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:33.195 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:33.195 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:33.195 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:33.196 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:33.196 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:33.196 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:33.196 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:33.196 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:33.197 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:33.197 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:33.197 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:33.197 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:33.198 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:33.199 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:33.199 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:33.199 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:33.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.199 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:33.200 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:33.200 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:33.200 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:33.200 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:33.200 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:33.201 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:33.201 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:33.201 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:33.201 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:33.201 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.202 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:33.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:33.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.204 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:33.204 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:33.204 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:33.204 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:33.204 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:33.205 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:33.205 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:33.205 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:33.205 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:33.205 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:33.206 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:33.206 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:33.206 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:33.206 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:33.207 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:33.207 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:33.207 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:33.207 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:33.208 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:33.208 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:33.208 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:33.208 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:33.208 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:33.209 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:33.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:33.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:33.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:33.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:33.211 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:33.212 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:33.212 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:33.212 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:33.212 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:33.212 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:33.213 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:33.213 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:33.213 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:33.213 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:33.213 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:33.214 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:33.214 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:33.214 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:33.215 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:33.215 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:33.215 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:33.215 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:33.215 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:33.216 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:33.217 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:33.218 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.219 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:33.220 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:33.221 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1027, 2023-10-10T20:30:33.221 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-10T20:30:33.221 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:33.221 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:33.222 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:33.222 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:33.222 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:33.222 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:33.223 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:33.224 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:33.224 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:33.224 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:33.224 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:33.224 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:33.225 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:33.225 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:33.225 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:33.225 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:33.226 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:33.226 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.226 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "180", 2023-10-10T20:30:33.226 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:33.226 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:33.227 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:33.228 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:33.228 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:33.228 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:33.228 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:33.228 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:33.229 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:33.230 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:33.230 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:33.230 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:33.230 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:33.230 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:33.231 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:33.231 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:33.231 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:33.231 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.231 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:33.232 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:33.233 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:33.233 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:33.233 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:33.233 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:33.233 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:33.234 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:33.234 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:33.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.234 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:33.234 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:33.235 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:33.236 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.236 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:33.236 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.236 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.236 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.237 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.237 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.237 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.237 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.237 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.238 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.238 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.238 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.238 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.238 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.239 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.239 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.239 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:33.239 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.241 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.242 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.242 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:33.242 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.242 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.243 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:33.245 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.245 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.245 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.245 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.245 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:33.246 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.246 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.246 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.246 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.246 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.247 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.248 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.248 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.248 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:33.248 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:33.249 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.249 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.249 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:33.250 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:33.250 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:33.250 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:33.250 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.250 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.251 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.251 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.251 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.251 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.251 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:33.252 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.252 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.252 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.252 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.252 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.253 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.254 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.254 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.254 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.254 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.254 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:33.255 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.256 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.256 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.256 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.256 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.256 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.257 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.257 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.257 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:33.257 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:33.258 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.259 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.259 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.259 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.259 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.259 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.260 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.260 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.260 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:33.260 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.260 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.261 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.262 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.262 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.262 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.263 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:33.263 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.264 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:33.265 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:33.265 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:33.265 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:33.265 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.265 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.266 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.266 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.267 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.268 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.269 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.269 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.269 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.269 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.269 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.270 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:33.270 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.270 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.271 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:33.271 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.271 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.271 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:33.272 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.273 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.274 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.275 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:33.275 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.276 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.276 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:33.276 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.276 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.278 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:33.278 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.279 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.280 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:33.280 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:33.280 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:33.280 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.282 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:33.282 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.282 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.282 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.282 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.283 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.283 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.283 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.283 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.283 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.284 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.285 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:33.285 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.285 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.286 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:33.286 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.286 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.287 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.287 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.288 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.289 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:33.289 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.289 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:33.291 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.291 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.291 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.292 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.294 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.294 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:33.294 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:33.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.295 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.295 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:33.295 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:33.296 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:33.296 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:33.296 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.296 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.296 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.297 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.298 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.298 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.298 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.298 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.298 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.299 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.300 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.300 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.300 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:33.300 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.301 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.302 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.302 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.302 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.302 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.303 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.303 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:33.303 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.303 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.304 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.304 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:33.304 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.305 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.306 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.306 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:33.306 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.306 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.307 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:33.307 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.307 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:33.309 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.309 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.309 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:33.310 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:33.310 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.310 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.311 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:33.311 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:33.311 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:33.311 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:33.311 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.312 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.312 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.312 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.313 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:33.313 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.313 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.314 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.314 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.314 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.314 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.314 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.315 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.315 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.315 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.315 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.315 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.317 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.317 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.317 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:33.317 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.318 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.318 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.318 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.318 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.319 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.319 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.319 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:33.320 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.320 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.320 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:33.321 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.321 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.322 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.322 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.323 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.323 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:33.323 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.324 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:33.324 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.325 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.325 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.326 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.326 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:33.326 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.328 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.329 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.329 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.329 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:33.330 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.330 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.330 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.330 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.330 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.331 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.331 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.331 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.331 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.331 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.332 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:33.333 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.333 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.333 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.333 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.333 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:33.334 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.334 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.334 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.334 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.335 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.335 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.335 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.335 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:33.335 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.336 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.337 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.337 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.337 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.337 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.338 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.338 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:33.338 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.340 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:33.341 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.341 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.342 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:33.342 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:33.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:33.343 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.345 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:33.345 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:33.345 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:33.345 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:33.346 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:33.346 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:33.346 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:33.346 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:33.346 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:33.347 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:33.347 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:33.347 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:33.347 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:33.347 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.348 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.348 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.348 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:33.348 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.349 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.350 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:33.350 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.350 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.350 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.351 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:33.352 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.352 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.352 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.352 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:33.353 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.354 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.354 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.354 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:33.354 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.355 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:33.355 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.355 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:33.357 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:33.357 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.357 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.357 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:33.358 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:33.358 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:33.358 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.358 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.359 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:33.359 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:33.359 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:33.359 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.360 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:33.361 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:33.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.361 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:33.361 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.361 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.362 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.362 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.362 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.362 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:33.362 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.363 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.363 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.363 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:33.363 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.363 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.364 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.364 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.364 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.364 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:33.364 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.365 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.366 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.367 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.367 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:33.367 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.369 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.369 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.369 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.370 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.370 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:33.370 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.371 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:33.371 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.371 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.371 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.372 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.372 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.372 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:33.372 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.373 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.375 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.376 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:33.376 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.376 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:33.376 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:33.377 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:33.377 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:33.377 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:33.377 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:33.377 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:33.378 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:33.378 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:33.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.378 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:33.378 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:33.379 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:33.379 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:33.379 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:33.379 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:33.380 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:33.380 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:33.380 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:33.380 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:33.381 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:33.381 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:33.381 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:33.381 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:33.381 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:33.383 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:33.383 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:33.383 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:33.383 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:33.383 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:33.384 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:33.384 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:33.384 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:33.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 msgr2=0x7fbf340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:33.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 0x7fbf340693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbf44003a80 tx=0x7fbf44007ec0 comp rx=0 tx=0).stop 2023-10-10T20:30:33.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 msgr2=0x7fbf5415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:33.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00cc10 tx=0x7fbf4c001970 comp rx=0 tx=0).stop 2023-10-10T20:30:33.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 shutdown_connections 2023-10-10T20:30:33.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbf34066f30 0x7fbf340693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:33.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf54158e50 0x7fbf5415a210 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:33.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbf54152010 0x7fbf5414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:33.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 --2- 172.21.15.148:0/2796525400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf541523e0 0x7fbf5414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:33.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 >> 172.21.15.148:0/2796525400 conn(0x7fbf54069870 msgr2=0x7fbf5415b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:33.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 shutdown_connections 2023-10-10T20:30:33.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:33.179+0000 7fbf59153700 1 -- 172.21.15.148:0/2796525400 wait complete. 2023-10-10T20:30:33.389 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-10T20:30:38.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.507+0000 7f5f4f5d5700 1 Processor -- start 2023-10-10T20:30:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.507+0000 7f5f4f5d5700 1 -- start start 2023-10-10T20:30:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4f5d5700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f48157870 0x7f5f4815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4f5d5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 0x7f5f4815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4f5d5700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4815cbb0 con 0x7f5f4815d580 2023-10-10T20:30:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4f5d5700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4815bfa0 con 0x7f5f48157870 2023-10-10T20:30:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4f5d5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4815c280 con 0x7f5f4815a620 2023-10-10T20:30:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f48157870 0x7f5f4815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f48157870 0x7f5f4815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46724/0 (socket says 172.21.15.148:46724) 2023-10-10T20:30:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 -- 172.21.15.148:0/2122154010 learned_addr learned my addr 172.21.15.148:0/2122154010 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4db72700 1 --1- 172.21.15.148:0/2122154010 >> v1:172.21.15.148:6789/0 conn(0x7f5f4815d580 0x7f5f48158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51676/0 (socket says 172.21.15.148:51676) 2023-10-10T20:30:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4cb70700 1 --2- 172.21.15.148:0/2122154010 >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 0x7f5f4815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 -- 172.21.15.148:0/2122154010 >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 msgr2=0x7f5f4815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 --2- 172.21.15.148:0/2122154010 >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 0x7f5f4815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 -- 172.21.15.148:0/2122154010 >> v1:172.21.15.148:6789/0 conn(0x7f5f4815d580 legacy=0x7f5f48158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 -- 172.21.15.148:0/2122154010 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4815c560 con 0x7f5f48157870 2023-10-10T20:30:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f47fff700 1 -- 172.21.15.148:0/2122154010 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201076167 0 0) 0x7f5f4815cbb0 con 0x7f5f4815d580 2023-10-10T20:30:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4cb70700 1 --2- 172.21.15.148:0/2122154010 >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 0x7f5f4815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f4d371700 1 --2- 172.21.15.148:0/2122154010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48157870 0x7f5f4815b6a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5f4815cb20 tx=0x7f5f38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adb080d7870f241 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f47fff700 1 -- 172.21.15.148:0/2122154010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800b860 con 0x7f5f48157870 2023-10-10T20:30:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f47fff700 1 -- 172.21.15.148:0/2122154010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800b9c0 con 0x7f5f48157870 2023-10-10T20:30:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.508+0000 7f5f47fff700 1 -- 172.21.15.148:0/2122154010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800bd60 con 0x7f5f48157870 2023-10-10T20:30:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2122154010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48157870 msgr2=0x7f5f4815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2122154010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48157870 0x7f5f4815b6a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5f4815cb20 tx=0x7f5f38009250 comp rx=0 tx=0).stop 2023-10-10T20:30:38.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2122154010 shutdown_connections 2023-10-10T20:30:38.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2122154010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48157870 0x7f5f4815b6a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2122154010 >> v2:172.21.15.161:3300/0 conn(0x7f5f4815a620 0x7f5f4815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2122154010 >> 172.21.15.148:0/2122154010 conn(0x7f5f48068ce0 msgr2=0x7f5f480690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:38.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2122154010 shutdown_connections 2023-10-10T20:30:38.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.509+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2122154010 wait complete. 2023-10-10T20:30:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.510+0000 7f5f4f5d5700 1 Processor -- start 2023-10-10T20:30:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.510+0000 7f5f4f5d5700 1 -- start start 2023-10-10T20:30:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48157870 0x7f5f4814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f4815a620 0x7f5f481518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48152f10 con 0x7f5f4815d580 2023-10-10T20:30:38.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4814b010 con 0x7f5f48157870 2023-10-10T20:30:38.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4f5d5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48152300 con 0x7f5f4815a620 2023-10-10T20:30:38.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46734/0 (socket says 172.21.15.148:46734) 2023-10-10T20:30:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 -- 172.21.15.148:0/2591402447 learned_addr learned my addr 172.21.15.148:0/2591402447 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 -- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f4815a620 msgr2=0x7f5f481518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f4815a620 0x7f5f481518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 -- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48157870 msgr2=0x7f5f4814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48157870 0x7f5f4814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.511+0000 7f5f4db72700 1 -- 172.21.15.148:0/2591402447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380094f0 con 0x7f5f4815d580 2023-10-10T20:30:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f4db72700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000ccc0 tx=0x7f5f40001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40005df0 con 0x7f5f4815d580 2023-10-10T20:30:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4000a920 con 0x7f5f4815d580 2023-10-10T20:30:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f481525e0 con 0x7f5f4815d580 2023-10-10T20:30:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000ace0 con 0x7f5f4815d580 2023-10-10T20:30:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.512+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f48152ba0 con 0x7f5f4815d580 2023-10-10T20:30:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.513+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4804c3c0 con 0x7f5f4815d580 2023-10-10T20:30:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.513+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f40007300 con 0x7f5f4815d580 2023-10-10T20:30:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.513+0000 7f5f457fa700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 0x7f5f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.514+0000 7f5f4d371700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 0x7f5f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.514+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f400846b0 con 0x7f5f4815d580 2023-10-10T20:30:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.514+0000 7f5f4d371700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 0x7f5f340693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5f4815cb20 tx=0x7f5f38017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.518+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f40051900 con 0x7f5f4815d580 2023-10-10T20:30:38.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.701+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5f48153050 con 0x7f5f4815d580 2023-10-10T20:30:38.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.702+0000 7f5f457fa700 1 -- 172.21.15.148:0/2591402447 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v181) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f5f40054f20 con 0x7f5f4815d580 2023-10-10T20:30:38.705 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:38.705 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:38.706 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 181, 2023-10-10T20:30:38.706 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:38.706 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:38.706 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:35.590886+0000", 2023-10-10T20:30:38.706 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:38.707 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:38.707 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:38.707 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:38.707 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:38.707 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:38.708 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:38.708 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:38.708 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:38.708 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:38.708 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:38.709 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:38.709 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:38.709 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:38.709 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:38.710 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:38.710 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:38.710 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:38.710 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:38.711 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:38.712 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:38.712 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:38.712 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:38.712 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:38.712 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:38.713 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:38.713 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:38.713 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:38.713 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:38.713 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:38.714 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:38.715 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:38.715 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:38.715 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:38.715 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:38.715 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:38.716 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:38.716 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:38.716 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:38.716 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.716 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:38.717 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:38.717 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:38.717 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:38.717 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:38.717 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:38.719 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:38.719 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:38.719 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:38.719 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:38.720 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:38.721 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:38.721 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:38.721 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:38.721 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:38.721 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:38.722 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:38.723 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:38.723 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:38.723 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:38.723 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:38.723 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:38.724 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:38.724 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:38.724 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:38.724 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:38.724 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.726 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.726 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.726 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:38.726 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:38.726 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:38.727 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:38.728 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:38.728 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:38.728 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:38.728 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:38.728 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:38.729 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:38.729 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:38.729 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:38.729 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:38.729 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.731 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:38.732 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:38.733 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:38.734 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:38.734 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:38.734 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:38.734 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:38.734 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:38.735 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.736 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:38.737 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:38.738 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:38.738 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:38.738 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:38.738 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:38.738 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.739 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:38.740 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:38.740 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:38.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.740 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:38.741 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:38.742 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:38.742 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:38.742 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:38.742 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:38.742 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:38.743 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:38.743 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:38.743 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:38.743 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:38.743 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:38.744 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:38.745 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:38.745 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:38.745 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:38.745 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:38.745 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "181", 2023-10-10T20:30:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:38.746 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:38.747 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:38.748 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:38.748 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:38.748 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:38.748 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:38.748 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:38.749 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:38.750 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:38.750 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:38.750 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:38.750 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:38.750 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:38.753 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:38.753 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:38.753 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:38.753 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:38.753 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:38.754 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:38.754 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:38.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.754 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:38.754 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:38.756 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.756 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:38.756 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.756 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.756 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.757 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.758 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.758 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.758 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.758 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.758 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.759 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.759 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:38.759 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.759 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.760 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.761 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:38.762 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.762 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.762 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.762 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:38.763 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.763 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.763 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.763 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.765 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.765 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.765 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:38.765 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.765 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.766 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.766 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.766 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.766 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.767 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.767 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:38.767 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.767 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.769 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:38.769 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:38.769 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:38.769 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:38.769 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.771 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.817 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:38.818 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.818 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.818 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.818 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.818 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:38.819 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.819 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.819 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.819 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.819 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.820 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.821 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.821 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.821 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:38.821 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.821 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.822 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.823 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:38.823 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.823 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.823 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.823 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.824 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.825 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.825 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.825 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.825 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:38.825 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.826 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.826 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.826 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.826 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:38.826 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:38.827 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.827 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.827 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.827 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:38.827 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.828 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.829 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.829 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.829 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:38.829 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.829 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.830 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.830 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.830 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.830 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.830 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.831 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.832 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.832 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.832 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:38.832 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.832 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.833 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.834 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.834 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.834 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.834 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.834 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.835 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:38.836 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.836 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.836 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.836 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.836 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.838 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.839 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.839 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.839 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.839 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.839 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.841 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:38.841 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:38.841 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.841 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.841 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.842 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:38.842 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:38.842 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:38.842 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:38.842 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.843 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:38.844 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.844 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.844 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.844 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.844 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.846 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.846 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.846 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.846 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:38.847 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.847 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.847 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:38.848 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.848 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.848 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.850 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.850 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:38.850 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.851 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.851 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.851 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.852 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.853 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:38.853 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.853 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:38.854 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.855 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.855 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:38.855 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.857 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.858 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.858 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.858 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:38.858 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:38.858 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:38.859 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:38.859 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:38.859 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:38.859 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:38.859 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:38.860 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.861 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.861 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.861 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:38.861 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.861 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.862 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.862 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.862 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:38.862 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.862 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.863 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.864 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:38.864 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.864 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.864 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.864 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.865 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:38.865 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.865 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.865 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:38.866 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.867 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.867 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.867 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.867 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:38.867 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:38.868 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.869 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.870 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:38.870 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:38.870 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:38.870 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.870 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:38.871 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:38.872 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.874 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.874 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.874 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:38.874 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.874 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.875 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.876 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.876 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:38.876 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.876 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.876 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.877 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.878 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:38.878 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.878 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.878 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.879 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:38.880 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.881 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.881 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.881 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.881 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.881 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:38.882 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.882 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.882 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.883 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:38.884 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.884 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.884 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:38.884 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:38.885 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.886 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.886 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:38.886 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:38.886 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:38.886 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:38.887 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:38.887 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:38.887 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:38.887 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:38.887 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:38.888 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:38.889 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:38.889 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:38.889 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:38.889 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:38.889 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:38.890 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:38.890 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:38.890 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:38.890 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:38.890 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:38.891 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:38.892 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:38.893 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:38.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:38.893 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:38.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.704+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 msgr2=0x7f5f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.704+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 0x7f5f340693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5f4815cb20 tx=0x7f5f38017000 comp rx=0 tx=0).stop 2023-10-10T20:30:38.895 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.704+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 msgr2=0x7f5f480b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:38.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.704+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000ccc0 tx=0x7f5f40001a40 comp rx=0 tx=0).stop 2023-10-10T20:30:38.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 shutdown_connections 2023-10-10T20:30:38.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5f34066f30 0x7f5f340693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f4815d580 0x7f5f480b59a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.896 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48157870 0x7f5f4814aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 --2- 172.21.15.148:0/2591402447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f4815a620 0x7f5f481518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:38.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 >> 172.21.15.148:0/2591402447 conn(0x7f5f48068ce0 msgr2=0x7f5f4814c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:38.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 shutdown_connections 2023-10-10T20:30:38.897 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:38.705+0000 7f5f4f5d5700 1 -- 172.21.15.148:0/2591402447 wait complete. 2023-10-10T20:30:38.898 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-10T20:30:38.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-10T20:30:39.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.012+0000 7f5bec57f700 1 Processor -- start 2023-10-10T20:30:39.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.012+0000 7f5bec57f700 1 -- start start 2023-10-10T20:30:39.016 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bec57f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5be415a620 0x7f5be415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bec57f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 0x7f5be4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bec57f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be415cbb0 con 0x7f5be4157870 2023-10-10T20:30:39.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bec57f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be415bdb0 con 0x7f5be415a620 2023-10-10T20:30:39.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bec57f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be415bfa0 con 0x7f5be415d580 2023-10-10T20:30:39.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5be415a620 0x7f5be415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bea31b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5be4157870 0x7f5be415b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51684/0 (socket says 172.21.15.148:51684) 2023-10-10T20:30:39.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5bea31b700 1 -- 172.21.15.148:0/3232536522 learned_addr learned my addr 172.21.15.148:0/3232536522 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:39.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5beab1c700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 0x7f5be4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 -- 172.21.15.148:0/3232536522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 msgr2=0x7f5be4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 0x7f5be4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 -- 172.21.15.148:0/3232536522 >> v1:172.21.15.148:6789/0 conn(0x7f5be4157870 legacy=0x7f5be415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 -- 172.21.15.148:0/3232536522 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be415c280 con 0x7f5be415a620 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9319700 1 -- 172.21.15.148:0/3232536522 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366618380 0 0) 0x7f5be415cbb0 con 0x7f5be4157870 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5beab1c700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 0x7f5be4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:39.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9b1a700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415a620 0x7f5be415d040 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40098e0 tx=0x7f5bd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea94b735e94b10 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9319700 1 -- 172.21.15.148:0/3232536522 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4009e80 con 0x7f5be415a620 2023-10-10T20:30:39.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9319700 1 -- 172.21.15.148:0/3232536522 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd400b860 con 0x7f5be415a620 2023-10-10T20:30:39.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.013+0000 7f5be9319700 1 -- 172.21.15.148:0/3232536522 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd400bbe0 con 0x7f5be415a620 2023-10-10T20:30:39.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 -- 172.21.15.148:0/3232536522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415a620 msgr2=0x7f5be415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415a620 0x7f5be415d040 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40098e0 tx=0x7f5bd40091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 -- 172.21.15.148:0/3232536522 shutdown_connections 2023-10-10T20:30:39.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be415d580 0x7f5be4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 --2- 172.21.15.148:0/3232536522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415a620 0x7f5be415d040 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 -- 172.21.15.148:0/3232536522 >> 172.21.15.148:0/3232536522 conn(0x7f5be4068ce0 msgr2=0x7f5be40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:39.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 -- 172.21.15.148:0/3232536522 shutdown_connections 2023-10-10T20:30:39.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.014+0000 7f5bec57f700 1 -- 172.21.15.148:0/3232536522 wait complete. 2023-10-10T20:30:39.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.015+0000 7f5bec57f700 1 Processor -- start 2023-10-10T20:30:39.024 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.015+0000 7f5bec57f700 1 -- start start 2023-10-10T20:30:39.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be4157870 0x7f5be416b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be415a620 0x7f5be40b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be416af80 con 0x7f5be415d580 2023-10-10T20:30:39.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be40b3210 con 0x7f5be415a620 2023-10-10T20:30:39.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bec57f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be40b34f0 con 0x7f5be4157870 2023-10-10T20:30:39.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46756/0 (socket says 172.21.15.148:46756) 2023-10-10T20:30:39.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 -- 172.21.15.148:0/2720163699 learned_addr learned my addr 172.21.15.148:0/2720163699 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:39.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5bea31b700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be4157870 0x7f5be416b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5be9b1a700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be415a620 0x7f5be40b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 -- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be4157870 msgr2=0x7f5be416b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be4157870 0x7f5be416b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 -- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be415a620 msgr2=0x7f5be40b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be415a620 0x7f5be40b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 -- 172.21.15.148:0/2720163699 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4009420 con 0x7f5be415d580 2023-10-10T20:30:39.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.016+0000 7f5beab1c700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00cef0 tx=0x7f5bdc007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00ae30 con 0x7f5be415d580 2023-10-10T20:30:39.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc016420 con 0x7f5be415d580 2023-10-10T20:30:39.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be416ac10 con 0x7f5be415d580 2023-10-10T20:30:39.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0167e0 con 0x7f5be415d580 2023-10-10T20:30:39.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be40b2f00 con 0x7f5be415d580 2023-10-10T20:30:39.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.017+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be416b0f0 con 0x7f5be415d580 2023-10-10T20:30:39.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.018+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bdc016bd0 con 0x7f5be415d580 2023-10-10T20:30:39.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.018+0000 7f5bda7fc700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.018+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bdc012070 con 0x7f5be415d580 2023-10-10T20:30:39.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.018+0000 7f5bea31b700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.019+0000 7f5bea31b700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 0x7f5bc80693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5be0007fd0 tx=0x7f5be0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.023+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bdc04e5a0 con 0x7f5be415d580 2023-10-10T20:30:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.204+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5be404c3c0 con 0x7f5be415d580 2023-10-10T20:30:39.207 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.205+0000 7f5bda7fc700 1 -- 172.21.15.148:0/2720163699 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v181) v1 ==== 81+0+32330 (secure 0 0 0) 0x7f5bdc0521b0 con 0x7f5be415d580 2023-10-10T20:30:39.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:39.209 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "epoch": 181, 2023-10-10T20:30:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "fsid": "374035e8-0861-4d26-af5d-2eadca412b24", 2023-10-10T20:30:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "created": "2023-10-10T20:19:05.720110+0000", 2023-10-10T20:30:39.209 INFO:teuthology.orchestra.run.smithi148.stdout: "modified": "2023-10-10T20:30:35.590886+0000", 2023-10-10T20:30:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_up_change": "2023-10-10T20:19:11.195666+0000", 2023-10-10T20:30:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "last_in_change": "2023-10-10T20:19:08.182843+0000", 2023-10-10T20:30:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-10T20:30:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_num": 5799936, 2023-10-10T20:30:39.210 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_set": [ 2023-10-10T20:30:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "pglog_hardlimit", 2023-10-10T20:30:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "purged_snapdirs", 2023-10-10T20:30:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_deletes", 2023-10-10T20:30:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: "sortbitwise" 2023-10-10T20:30:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_version": 4, 2023-10-10T20:30:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "full_ratio": 0.69999998807907104, 2023-10-10T20:30:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-10T20:30:39.212 INFO:teuthology.orchestra.run.smithi148.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-10T20:30:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_snapshot": "", 2023-10-10T20:30:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_max": 23, 2023-10-10T20:30:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "max_osd": 8, 2023-10-10T20:30:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "require_min_compat_client": "luminous", 2023-10-10T20:30:39.213 INFO:teuthology.orchestra.run.smithi148.stdout: "min_compat_client": "jewel", 2023-10-10T20:30:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "require_osd_release": "quincy", 2023-10-10T20:30:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "pools": [ 2023-10-10T20:30:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 1, 2023-10-10T20:30:39.214 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": ".mgr", 2023-10-10T20:30:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:19:11.361845+0000", 2023-10-10T20:30:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:39.215 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:39.216 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 1, 2023-10-10T20:30:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 1, 2023-10-10T20:30:39.217 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 1, 2023-10-10T20:30:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 1, 2023-10-10T20:30:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 1, 2023-10-10T20:30:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:39.218 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:39.219 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "104", 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:39.220 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:39.221 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:39.223 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:39.224 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:39.226 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_max": 32, 2023-10-10T20:30:39.227 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 1 2023-10-10T20:30:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": {} 2023-10-10T20:30:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 22, 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_metadata", 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:02.621495+0000", 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:39.230 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:39.231 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 32, 2023-10-10T20:30:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 32, 2023-10-10T20:30:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 32, 2023-10-10T20:30:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 32, 2023-10-10T20:30:39.232 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 32, 2023-10-10T20:30:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:39.233 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "177", 2023-10-10T20:30:39.234 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:39.235 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 0, 2023-10-10T20:30:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:39.236 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:39.237 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:39.238 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:39.239 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:39.240 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:39.241 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_bias": 4, 2023-10-10T20:30:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 16, 2023-10-10T20:30:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_priority": 5 2023-10-10T20:30:39.242 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": "cephfs" 2023-10-10T20:30:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.243 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 23, 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_name": "cephfs_data", 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "create_time": "2023-10-10T20:30:03.225751+0000", 2023-10-10T20:30:39.244 INFO:teuthology.orchestra.run.smithi148.stdout: "flags": 1, 2023-10-10T20:30:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "flags_names": "hashpspool", 2023-10-10T20:30:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "type": 1, 2023-10-10T20:30:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "size": 2, 2023-10-10T20:30:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "min_size": 1, 2023-10-10T20:30:39.245 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_rule": 0, 2023-10-10T20:30:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_count": 0, 2023-10-10T20:30:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_target": 0, 2023-10-10T20:30:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_barrier": 0, 2023-10-10T20:30:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-10T20:30:39.246 INFO:teuthology.orchestra.run.smithi148.stdout: "object_hash": 2, 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_autoscale_mode": "off", 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num": 64, 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num": 64, 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_placement_num_target": 64, 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_target": 64, 2023-10-10T20:30:39.247 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_pending": 64, 2023-10-10T20:30:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_pg_merge_meta": { 2023-10-10T20:30:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "source_pgid": "0.0", 2023-10-10T20:30:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "ready_epoch": 0, 2023-10-10T20:30:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_started": 0, 2023-10-10T20:30:39.248 INFO:teuthology.orchestra.run.smithi148.stdout: "last_epoch_clean": 0, 2023-10-10T20:30:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "source_version": "0'0", 2023-10-10T20:30:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "target_version": "0'0" 2023-10-10T20:30:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "last_change": "181", 2023-10-10T20:30:39.249 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend": "0", 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "last_force_op_resend_preluminous": "0", 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "auid": 0, 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_mode": "selfmanaged", 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_seq": 0, 2023-10-10T20:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout: "snap_epoch": 0, 2023-10-10T20:30:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "pool_snaps": [], 2023-10-10T20:30:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps": "[]", 2023-10-10T20:30:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_bytes": 33554432, 2023-10-10T20:30:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "quota_max_objects": 0, 2023-10-10T20:30:39.251 INFO:teuthology.orchestra.run.smithi148.stdout: "tiers": [], 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tier_of": -1, 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "read_tier": -1, 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "write_tier": -1, 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_mode": "none", 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_bytes": 0, 2023-10-10T20:30:39.252 INFO:teuthology.orchestra.run.smithi148.stdout: "target_max_objects": 0, 2023-10-10T20:30:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-10T20:30:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-10T20:30:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-10T20:30:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_flush_age": 0, 2023-10-10T20:30:39.253 INFO:teuthology.orchestra.run.smithi148.stdout: "cache_min_evict_age": 0, 2023-10-10T20:30:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profile": "", 2023-10-10T20:30:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_params": { 2023-10-10T20:30:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "none" 2023-10-10T20:30:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.254 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_period": 0, 2023-10-10T20:30:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_count": 0, 2023-10-10T20:30:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "use_gmt_hitset": true, 2023-10-10T20:30:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "min_read_recency_for_promote": 0, 2023-10-10T20:30:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "min_write_recency_for_promote": 0, 2023-10-10T20:30:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_grade_decay_rate": 0, 2023-10-10T20:30:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "hit_set_search_last_n": 0, 2023-10-10T20:30:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "grade_table": [], 2023-10-10T20:30:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "stripe_width": 0, 2023-10-10T20:30:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "expected_num_objects": 0, 2023-10-10T20:30:39.256 INFO:teuthology.orchestra.run.smithi148.stdout: "fast_read": false, 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "options": { 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_num_min": 64, 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "target_size_ratio": 0.89999997615814209 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "application_metadata": { 2023-10-10T20:30:39.257 INFO:teuthology.orchestra.run.smithi148.stdout: "cephfs": { 2023-10-10T20:30:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: "data": "cephfs" 2023-10-10T20:30:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.258 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "osds": [ 2023-10-10T20:30:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.259 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.260 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.261 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6800", 2023-10-10T20:30:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6801", 2023-10-10T20:30:39.263 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.264 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6802", 2023-10-10T20:30:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6803", 2023-10-10T20:30:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.266 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.267 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6806", 2023-10-10T20:30:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.268 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6807", 2023-10-10T20:30:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.269 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.270 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6804", 2023-10-10T20:30:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.271 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6805", 2023-10-10T20:30:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86309 2023-10-10T20:30:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.272 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6801/86309", 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6803/86309", 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6807/86309", 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6805/86309", 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.273 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:39.274 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.275 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.276 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6816", 2023-10-10T20:30:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6817", 2023-10-10T20:30:39.278 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.279 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6818", 2023-10-10T20:30:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6819", 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.281 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.282 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6822", 2023-10-10T20:30:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.283 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6823", 2023-10-10T20:30:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.285 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6820", 2023-10-10T20:30:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.286 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6821", 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86315 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6817/86315", 2023-10-10T20:30:39.287 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6819/86315", 2023-10-10T20:30:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6823/86315", 2023-10-10T20:30:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6821/86315", 2023-10-10T20:30:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.288 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:39.289 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.290 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.291 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6808", 2023-10-10T20:30:39.292 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6809", 2023-10-10T20:30:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6810", 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.295 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6811", 2023-10-10T20:30:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6814", 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.298 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6815", 2023-10-10T20:30:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.299 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6812", 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6813", 2023-10-10T20:30:39.301 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86312 2023-10-10T20:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6809/86312", 2023-10-10T20:30:39.302 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6811/86312", 2023-10-10T20:30:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6815/86312", 2023-10-10T20:30:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6813/86312", 2023-10-10T20:30:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.303 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:39.304 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.305 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.306 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6824", 2023-10-10T20:30:39.307 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6825", 2023-10-10T20:30:39.308 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.309 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6826", 2023-10-10T20:30:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.310 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6827", 2023-10-10T20:30:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.311 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.312 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6830", 2023-10-10T20:30:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.313 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6831", 2023-10-10T20:30:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.314 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.315 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6828", 2023-10-10T20:30:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.316 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.148:6829", 2023-10-10T20:30:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86339 2023-10-10T20:30:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.317 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.148:6825/86339", 2023-10-10T20:30:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.148:6827/86339", 2023-10-10T20:30:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.148:6831/86339", 2023-10-10T20:30:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.148:6829/86339", 2023-10-10T20:30:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.320 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.321 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.322 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6802", 2023-10-10T20:30:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.323 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6806", 2023-10-10T20:30:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6810", 2023-10-10T20:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6814", 2023-10-10T20:30:39.326 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.327 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6829", 2023-10-10T20:30:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6831", 2023-10-10T20:30:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.330 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6818", 2023-10-10T20:30:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.331 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6822", 2023-10-10T20:30:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86554 2023-10-10T20:30:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.332 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6806/86554", 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6814/86554", 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6831/86554", 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6822/86554", 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.333 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.334 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.335 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.336 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.337 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6801", 2023-10-10T20:30:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.338 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6804", 2023-10-10T20:30:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.339 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6808", 2023-10-10T20:30:39.340 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6812", 2023-10-10T20:30:39.341 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.342 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6824", 2023-10-10T20:30:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.343 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6826", 2023-10-10T20:30:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.344 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6816", 2023-10-10T20:30:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.346 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6820", 2023-10-10T20:30:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86555 2023-10-10T20:30:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.347 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6804/86555", 2023-10-10T20:30:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6812/86555", 2023-10-10T20:30:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6826/86555", 2023-10-10T20:30:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6820/86555", 2023-10-10T20:30:39.348 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.349 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.350 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.351 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.352 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6805", 2023-10-10T20:30:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.353 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6809", 2023-10-10T20:30:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.354 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6813", 2023-10-10T20:30:39.355 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6817", 2023-10-10T20:30:39.356 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.357 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6828", 2023-10-10T20:30:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.358 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6830", 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.359 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.360 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6821", 2023-10-10T20:30:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.361 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6825", 2023-10-10T20:30:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86553 2023-10-10T20:30:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.362 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6809/86553", 2023-10-10T20:30:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6817/86553", 2023-10-10T20:30:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6830/86553", 2023-10-10T20:30:39.363 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6825/86553", 2023-10-10T20:30:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.364 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-10T20:30:39.365 INFO:teuthology.orchestra.run.smithi148.stdout: "up": 1, 2023-10-10T20:30:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "in": 1, 2023-10-10T20:30:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "weight": 1, 2023-10-10T20:30:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_affinity": 1, 2023-10-10T20:30:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_begin": 0, 2023-10-10T20:30:39.366 INFO:teuthology.orchestra.run.smithi148.stdout: "last_clean_end": 0, 2023-10-10T20:30:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "up_from": 13, 2023-10-10T20:30:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "up_thru": 176, 2023-10-10T20:30:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "down_at": 0, 2023-10-10T20:30:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "lost_at": 0, 2023-10-10T20:30:39.367 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addrs": { 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6800", 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.368 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6803", 2023-10-10T20:30:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.369 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addrs": { 2023-10-10T20:30:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.370 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6807", 2023-10-10T20:30:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.371 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6811", 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.372 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addrs": { 2023-10-10T20:30:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.373 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6823", 2023-10-10T20:30:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.374 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6827", 2023-10-10T20:30:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.375 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addrs": { 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "addrvec": [ 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v2", 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6815", 2023-10-10T20:30:39.376 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2023-10-10T20:30:39.377 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "172.21.15.161:6819", 2023-10-10T20:30:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 86574 2023-10-10T20:30:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.378 INFO:teuthology.orchestra.run.smithi148.stdout: "public_addr": "172.21.15.161:6803/86574", 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_addr": "172.21.15.161:6811/86574", 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_back_addr": "172.21.15.161:6827/86574", 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "heartbeat_front_addr": "172.21.15.161:6819/86574", 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "state": [ 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "exists", 2023-10-10T20:30:39.379 INFO:teuthology.orchestra.run.smithi148.stdout: "up" 2023-10-10T20:30:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2023-10-10T20:30:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_xinfo": [ 2023-10-10T20:30:39.380 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 0, 2023-10-10T20:30:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.381 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.316456+0000", 2023-10-10T20:30:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 1, 2023-10-10T20:30:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.287039+0000", 2023-10-10T20:30:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.384 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 2, 2023-10-10T20:30:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.385 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.282821+0000", 2023-10-10T20:30:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.386 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 3, 2023-10-10T20:30:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.387 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.385603+0000", 2023-10-10T20:30:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.388 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 4, 2023-10-10T20:30:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.389 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.354337+0000", 2023-10-10T20:30:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.390 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 5, 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.363589+0000", 2023-10-10T20:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 6, 2023-10-10T20:30:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.393 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.376276+0000", 2023-10-10T20:30:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: { 2023-10-10T20:30:39.394 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": 7, 2023-10-10T20:30:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "down_stamp": "0.000000", 2023-10-10T20:30:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_probability": 0, 2023-10-10T20:30:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "laggy_interval": 0, 2023-10-10T20:30:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "features": 4540138320759226367, 2023-10-10T20:30:39.395 INFO:teuthology.orchestra.run.smithi148.stdout: "old_weight": 0, 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "last_purged_snaps_scrub": "2023-10-10T20:19:09.321072+0000", 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "dead_epoch": 0 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap": [], 2023-10-10T20:30:39.396 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_upmap_items": [], 2023-10-10T20:30:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "pg_temp": [], 2023-10-10T20:30:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "primary_temp": [], 2023-10-10T20:30:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklist": {}, 2023-10-10T20:30:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "range_blocklist": {}, 2023-10-10T20:30:39.397 INFO:teuthology.orchestra.run.smithi148.stdout: "erasure_code_profiles": { 2023-10-10T20:30:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "default": { 2023-10-10T20:30:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "crush-failure-domain": "osd", 2023-10-10T20:30:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "k": "2", 2023-10-10T20:30:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "m": "1", 2023-10-10T20:30:39.398 INFO:teuthology.orchestra.run.smithi148.stdout: "plugin": "jerasure", 2023-10-10T20:30:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "technique": "reed_sol_van" 2023-10-10T20:30:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2023-10-10T20:30:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "removed_snaps_queue": [], 2023-10-10T20:30:39.399 INFO:teuthology.orchestra.run.smithi148.stdout: "new_removed_snaps": [], 2023-10-10T20:30:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "new_purged_snaps": [], 2023-10-10T20:30:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "crush_node_flags": {}, 2023-10-10T20:30:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "device_class_flags": {}, 2023-10-10T20:30:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode": { 2023-10-10T20:30:39.400 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_enabled": false, 2023-10-10T20:30:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_bucket_count": 0, 2023-10-10T20:30:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "degraded_stretch_mode": 0, 2023-10-10T20:30:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "recovering_stretch_mode": 0, 2023-10-10T20:30:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: "stretch_mode_bucket": 0 2023-10-10T20:30:39.401 INFO:teuthology.orchestra.run.smithi148.stdout: } 2023-10-10T20:30:39.402 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:39.403 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 msgr2=0x7f5bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 0x7f5bc80693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5be0007fd0 tx=0x7f5be0007d30 comp rx=0 tx=0).stop 2023-10-10T20:30:39.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 msgr2=0x7f5be40b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00cef0 tx=0x7f5bdc007830 comp rx=0 tx=0).stop 2023-10-10T20:30:39.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 shutdown_connections 2023-10-10T20:30:39.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5bc8066f30 0x7f5bc80693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be415d580 0x7f5be40b29c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5be4157870 0x7f5be416b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 --2- 172.21.15.148:0/2720163699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5be415a620 0x7f5be40b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 >> 172.21.15.148:0/2720163699 conn(0x7f5be4068ce0 msgr2=0x7f5be41605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:39.405 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 shutdown_connections 2023-10-10T20:30:39.406 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.208+0000 7f5bec57f700 1 -- 172.21.15.148:0/2720163699 wait complete. 2023-10-10T20:30:39.406 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:39.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.520+0000 7f55af72f700 1 Processor -- start 2023-10-10T20:30:39.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.521+0000 7f55af72f700 1 -- start start 2023-10-10T20:30:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55af72f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55af72f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55a815d580 0x7f55a8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55af72f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a815cbb0 con 0x7f55a8157870 2023-10-10T20:30:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55af72f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a815bdb0 con 0x7f55a815d580 2023-10-10T20:30:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55af72f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a815bfa0 con 0x7f55a815a620 2023-10-10T20:30:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55adccc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55a815d580 0x7f55a8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:47832/0 (socket says 172.21.15.148:47832) 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 -- 172.21.15.148:0/1741795506 learned_addr learned my addr 172.21.15.148:0/1741795506 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55ad4cb700 1 --1- 172.21.15.148:0/1741795506 >> v1:172.21.15.161:6789/0 conn(0x7f55a8157870 0x7f55a815b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:57340/0 (socket says 172.21.15.148:57340) 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 -- 172.21.15.148:0/1741795506 >> v2:172.21.15.161:3300/0 conn(0x7f55a815d580 msgr2=0x7f55a8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 --2- 172.21.15.148:0/1741795506 >> v2:172.21.15.161:3300/0 conn(0x7f55a815d580 0x7f55a8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 -- 172.21.15.148:0/1741795506 >> v1:172.21.15.161:6789/0 conn(0x7f55a8157870 legacy=0x7f55a815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 -- 172.21.15.148:0/1741795506 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a815c280 con 0x7f55a815a620 2023-10-10T20:30:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.522+0000 7f55accca700 1 --2- 172.21.15.148:0/1741795506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55980099b0 tx=0x7f5598009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c90fd1fd3ad1e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f559ffff700 1 -- 172.21.15.148:0/1741795506 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598009ea0 con 0x7f55a815a620 2023-10-10T20:30:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f559ffff700 1 -- 172.21.15.148:0/1741795506 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800b860 con 0x7f55a815a620 2023-10-10T20:30:39.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f559ffff700 1 -- 172.21.15.148:0/1741795506 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800bc00 con 0x7f55a815a620 2023-10-10T20:30:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 -- 172.21.15.148:0/1741795506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 msgr2=0x7f55a815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 --2- 172.21.15.148:0/1741795506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55980099b0 tx=0x7f5598009250 comp rx=0 tx=0).stop 2023-10-10T20:30:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 -- 172.21.15.148:0/1741795506 shutdown_connections 2023-10-10T20:30:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 --2- 172.21.15.148:0/1741795506 >> v2:172.21.15.161:3300/0 conn(0x7f55a815d580 0x7f55a8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 --2- 172.21.15.148:0/1741795506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a815a620 0x7f55a815d040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 -- 172.21.15.148:0/1741795506 >> 172.21.15.148:0/1741795506 conn(0x7f55a8068ce0 msgr2=0x7f55a80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 -- 172.21.15.148:0/1741795506 shutdown_connections 2023-10-10T20:30:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.523+0000 7f55af72f700 1 -- 172.21.15.148:0/1741795506 wait complete. 2023-10-10T20:30:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.524+0000 7f55af72f700 1 Processor -- start 2023-10-10T20:30:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.524+0000 7f55af72f700 1 -- start start 2023-10-10T20:30:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a8157870 0x7f55a816b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 0x7f55a816a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8153ac0 con 0x7f55a815d580 2023-10-10T20:30:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8153630 con 0x7f55a815a620 2023-10-10T20:30:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55af72f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a816a860 con 0x7f55a8157870 2023-10-10T20:30:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55adccc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55adccc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46762/0 (socket says 172.21.15.148:46762) 2023-10-10T20:30:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.525+0000 7f55adccc700 1 -- 172.21.15.148:0/1998268810 learned_addr learned my addr 172.21.15.148:0/1998268810 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55ad4cb700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a8157870 0x7f55a816b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55accca700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 0x7f55a816a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 -- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a8157870 msgr2=0x7f55a816b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a8157870 0x7f55a816b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 -- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 msgr2=0x7f55a816a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 0x7f55a816a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 -- 172.21.15.148:0/1998268810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980094f0 con 0x7f55a815d580 2023-10-10T20:30:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55accca700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 0x7f55a816a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55adccc700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009ef0 tx=0x7f55a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000cbf0 con 0x7f55a815d580 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000f040 con 0x7f55a815d580 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a8152a20 con 0x7f55a815d580 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0014480 con 0x7f55a815d580 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.526+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8152fe0 con 0x7f55a815d580 2023-10-10T20:30:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.527+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a80b1ea0 con 0x7f55a815d580 2023-10-10T20:30:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.527+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55a00147f0 con 0x7f55a815d580 2023-10-10T20:30:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.528+0000 7f559d7fa700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.528+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55a0085300 con 0x7f55a815d580 2023-10-10T20:30:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.528+0000 7f55ad4cb700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.528+0000 7f55ad4cb700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 0x7f558c0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005f80 tx=0x7f55a4007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:39.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.532+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55a0052570 con 0x7f55a815d580 2023-10-10T20:30:39.760 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.758+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55a804c3c0 con 0x7f55a815d580 2023-10-10T20:30:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.758+0000 7f559d7fa700 1 -- 172.21.15.148:0/1998268810 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f55a0055b90 con 0x7f55a815d580 2023-10-10T20:30:39.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:39.761 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:39.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 msgr2=0x7f558c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 0x7f558c0693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005f80 tx=0x7f55a4007ce0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 msgr2=0x7f55a81521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:39.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009ef0 tx=0x7f55a000b040 comp rx=0 tx=0).stop 2023-10-10T20:30:39.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 shutdown_connections 2023-10-10T20:30:39.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f558c066f30 0x7f558c0693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a815d580 0x7f55a81521d0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55a8157870 0x7f55a816b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 --2- 172.21.15.148:0/1998268810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a815a620 0x7f55a816a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.760+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 >> 172.21.15.148:0/1998268810 conn(0x7f55a8068ce0 msgr2=0x7f55a80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.761+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 shutdown_connections 2023-10-10T20:30:39.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:39.761+0000 7f55af72f700 1 -- 172.21.15.148:0/1998268810 wait complete. 2023-10-10T20:30:39.766 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:39.775 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-10T20:30:39.917 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"374035e8-0861-4d26-af5d-2eadca412b24","whoami":0,"id":8139,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.026036726999997,"mdsmap_epoch":85,"osdmap_epoch":181,"osdmap_epoch_barrier":180,"uptime":40.136903461000003} 2023-10-10T20:30:39.918 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "374035e8-0861-4d26-af5d-2eadca412b24", "fs_name": "cephfs", "id": 8139, "mdsmap_epoch": 85, "osdmap_epoch": 181, "osdmap_epoch_barrier": 180, "rank_uptime": 35.026036727, "state": "up:active", "uptime": 40.136903461, "want_state": "up:active", "whoami": 0 } 2023-10-10T20:30:39.918 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-10T20:30:39.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:39.919 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-10T20:30:40.207 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records in 2023-10-10T20:30:40.208 INFO:teuthology.orchestra.run.smithi148.stderr:50+0 records out 2023-10-10T20:30:40.208 INFO:teuthology.orchestra.run.smithi148.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.270673 s, 194 MB/s 2023-10-10T20:30:40.208 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-10T20:30:40.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:30:40.254 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-10T20:30:40.293 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-10T20:30:40.294 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-10T20:30:40.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:40.295 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-10T20:30:40.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:30:40.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:40.368 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-10T20:30:40.383 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-10T20:30:40.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-10T20:30:40.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-10T20:30:40.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:30:40.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:40.420 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-10T20:30:40.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:40.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.586+0000 7f05a81d7700 1 Processor -- start 2023-10-10T20:30:40.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.586+0000 7f05a81d7700 1 -- start start 2023-10-10T20:30:40.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a81d7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 0x7f05a015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a81d7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05a015a620 0x7f05a015d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a81d7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a015cbb0 con 0x7f05a015d580 2023-10-10T20:30:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a81d7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a015bfa0 con 0x7f05a015a620 2023-10-10T20:30:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a81d7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015c280 con 0x7f05a0157870 2023-10-10T20:30:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a5772700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05a015a620 0x7f05a015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a6774700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f05a015d580 0x7f05a0158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51690/0 (socket says 172.21.15.148:51690) 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.587+0000 7f05a6774700 1 -- 172.21.15.148:0/2520019834 learned_addr learned my addr 172.21.15.148:0/2520019834 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5f73700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 0x7f05a015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5772700 1 -- 172.21.15.148:0/2520019834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 msgr2=0x7f05a015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5772700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 0x7f05a015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5772700 1 -- 172.21.15.148:0/2520019834 >> v1:172.21.15.148:6789/0 conn(0x7f05a015d580 legacy=0x7f05a0158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a4f71700 1 -- 172.21.15.148:0/2520019834 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399204578 0 0) 0x7f05a015cbb0 con 0x7f05a015d580 2023-10-10T20:30:40.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5772700 1 -- 172.21.15.148:0/2520019834 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a015c560 con 0x7f05a015a620 2023-10-10T20:30:40.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5f73700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 0x7f05a015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:40.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a5772700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a015a620 0x7f05a015d040 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f05900098e0 tx=0x7f05900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d96036291196617 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a4f71700 1 -- 172.21.15.148:0/2520019834 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590009e80 con 0x7f05a015a620 2023-10-10T20:30:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a4f71700 1 -- 172.21.15.148:0/2520019834 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059000b860 con 0x7f05a015a620 2023-10-10T20:30:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.588+0000 7f05a4f71700 1 -- 172.21.15.148:0/2520019834 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000bbe0 con 0x7f05a015a620 2023-10-10T20:30:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 -- 172.21.15.148:0/2520019834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a015a620 msgr2=0x7f05a015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a015a620 0x7f05a015d040 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f05900098e0 tx=0x7f05900091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 -- 172.21.15.148:0/2520019834 shutdown_connections 2023-10-10T20:30:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a0157870 0x7f05a015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 --2- 172.21.15.148:0/2520019834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a015a620 0x7f05a015d040 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 -- 172.21.15.148:0/2520019834 >> 172.21.15.148:0/2520019834 conn(0x7f05a0068ce0 msgr2=0x7f05a00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 -- 172.21.15.148:0/2520019834 shutdown_connections 2023-10-10T20:30:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.589+0000 7f05a81d7700 1 -- 172.21.15.148:0/2520019834 wait complete. 2023-10-10T20:30:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.590+0000 7f05a81d7700 1 Processor -- start 2023-10-10T20:30:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.590+0000 7f05a81d7700 1 -- start start 2023-10-10T20:30:40.599 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.590+0000 7f05a81d7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.590+0000 7f05a81d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05a015a620 0x7f05a0139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.590+0000 7f05a81d7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 0x7f05a013a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a81d7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a014ff10 con 0x7f05a0157870 2023-10-10T20:30:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a81d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00b4a10 con 0x7f05a015a620 2023-10-10T20:30:40.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a81d7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00b4cf0 con 0x7f05a015d580 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46792/0 (socket says 172.21.15.148:46792) 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 -- 172.21.15.148:0/1256327040 learned_addr learned my addr 172.21.15.148:0/1256327040 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a6774700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 0x7f05a013a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5772700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05a015a620 0x7f05a0139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.601 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 -- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 msgr2=0x7f05a013a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 0x7f05a013a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 -- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05a015a620 msgr2=0x7f05a0139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05a015a620 0x7f05a0139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.602 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 -- 172.21.15.148:0/1256327040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009420 con 0x7f05a0157870 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a6774700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 0x7f05a013a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a5f73700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f059c00d1d0 tx=0x7f059c009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00dd10 con 0x7f05a0157870 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c00f040 con 0x7f05a0157870 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a00b4fd0 con 0x7f05a0157870 2023-10-10T20:30:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.591+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c015670 con 0x7f05a0157870 2023-10-10T20:30:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.592+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a014d6a0 con 0x7f05a0157870 2023-10-10T20:30:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.592+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a004c3c0 con 0x7f05a0157870 2023-10-10T20:30:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.593+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f059c015a60 con 0x7f05a0157870 2023-10-10T20:30:40.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.593+0000 7f0595ffb700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 0x7f05840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.593+0000 7f05a5772700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 0x7f05840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.594+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f059c055930 con 0x7f05a0157870 2023-10-10T20:30:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.594+0000 7f05a5772700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 0x7f05840693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f05900091e0 tx=0x7f0590017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.598+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f059c051c70 con 0x7f05a0157870 2023-10-10T20:30:40.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.823+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05a015c840 con 0x7f05a0157870 2023-10-10T20:30:40.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.823+0000 7f0595ffb700 1 -- 172.21.15.148:0/1256327040 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f059c01a070 con 0x7f05a0157870 2023-10-10T20:30:40.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:40.826 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 msgr2=0x7f05840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 0x7f05840693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f05900091e0 tx=0x7f0590017000 comp rx=0 tx=0).stop 2023-10-10T20:30:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 msgr2=0x7f05a014d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f059c00d1d0 tx=0x7f059c009a90 comp rx=0 tx=0).stop 2023-10-10T20:30:40.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 shutdown_connections 2023-10-10T20:30:40.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f0584066f30 0x7f05840693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05a015d580 0x7f05a013a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.829 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a0157870 0x7f05a014d160 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 --2- 172.21.15.148:0/1256327040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05a015a620 0x7f05a0139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.830 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 >> 172.21.15.148:0/1256327040 conn(0x7f05a0068ce0 msgr2=0x7f05a00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:40.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 shutdown_connections 2023-10-10T20:30:40.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.825+0000 7f05a81d7700 1 -- 172.21.15.148:0/1256327040 wait complete. 2023-10-10T20:30:40.831 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:40.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:40.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.953+0000 7f9272b0d700 1 Processor -- start 2023-10-10T20:30:40.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.953+0000 7f9272b0d700 1 -- start start 2023-10-10T20:30:40.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f9272b0d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f926c157870 0x7f926c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f9272b0d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c15cbb0 con 0x7f926c15a620 2023-10-10T20:30:40.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f9272b0d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c15bfa0 con 0x7f926c157c40 2023-10-10T20:30:40.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f9272b0d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c15c280 con 0x7f926c157870 2023-10-10T20:30:40.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926bfff700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f926c15a620 0x7f926c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51694/0 (socket says 172.21.15.148:51694) 2023-10-10T20:30:40.959 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926bfff700 1 -- 172.21.15.148:0/3341330412 learned_addr learned my addr 172.21.15.148:0/3341330412 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f92708a9700 1 --2- 172.21.15.148:0/3341330412 >> v2:172.21.15.161:3300/0 conn(0x7f926c157870 0x7f926c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371744473 0 0) 0x7f926c15cbb0 con 0x7f926c15a620 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c003570 con 0x7f926c15a620 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183194336 0 0) 0x7f926c15bfa0 con 0x7f926c157c40 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f926c15cbb0 con 0x7f926c157c40 2023-10-10T20:30:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f92708a9700 1 -- 172.21.15.148:0/3341330412 >> v1:172.21.15.161:6789/0 conn(0x7f926c157c40 legacy=0x7f926c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885787806 0 0) 0x7f925c003570 con 0x7f926c15a620 2023-10-10T20:30:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f92708a9700 1 -- 172.21.15.148:0/3341330412 >> v1:172.21.15.148:6789/0 conn(0x7f926c15a620 legacy=0x7f926c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.954+0000 7f92708a9700 1 -- 172.21.15.148:0/3341330412 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c15c560 con 0x7f926c157870 2023-10-10T20:30:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f92708a9700 1 --2- 172.21.15.148:0/3341330412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157870 0x7f926c15b6a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f92600098e0 tx=0x7f92600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a15cfc813dcf0a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258001470 con 0x7f926c157870 2023-10-10T20:30:40.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260011420 con 0x7f926c157870 2023-10-10T20:30:40.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f926b7fe700 1 -- 172.21.15.148:0/3341330412 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600117c0 con 0x7f926c157870 2023-10-10T20:30:40.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 -- 172.21.15.148:0/3341330412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157870 msgr2=0x7f926c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 --2- 172.21.15.148:0/3341330412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157870 0x7f926c15b6a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f92600098e0 tx=0x7f92600091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 -- 172.21.15.148:0/3341330412 shutdown_connections 2023-10-10T20:30:40.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 --2- 172.21.15.148:0/3341330412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157870 0x7f926c15b6a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 -- 172.21.15.148:0/3341330412 >> 172.21.15.148:0/3341330412 conn(0x7f926c068ce0 msgr2=0x7f926c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:40.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 -- 172.21.15.148:0/3341330412 shutdown_connections 2023-10-10T20:30:40.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.955+0000 7f9272b0d700 1 -- 172.21.15.148:0/3341330412 wait complete. 2023-10-10T20:30:40.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.956+0000 7f9272b0d700 1 Processor -- start 2023-10-10T20:30:40.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.956+0000 7f9272b0d700 1 -- start start 2023-10-10T20:30:40.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f926c157870 0x7f926c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 0x7f926c151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0b2010 con 0x7f926c15a620 2023-10-10T20:30:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0b1400 con 0x7f926c157c40 2023-10-10T20:30:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f9272b0d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0b16e0 con 0x7f926c157870 2023-10-10T20:30:40.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f92710aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f92710aa700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46794/0 (socket says 172.21.15.148:46794) 2023-10-10T20:30:40.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f92710aa700 1 -- 172.21.15.148:0/610322347 learned_addr learned my addr 172.21.15.148:0/610322347 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:40.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f926bfff700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 0x7f926c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.968 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.957+0000 7f92708a9700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f926c157870 0x7f926c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 -- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f926c157870 msgr2=0x7f926c1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f926c157870 0x7f926c1513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 -- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 msgr2=0x7f926c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 0x7f926c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 -- 172.21.15.148:0/610322347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9260009420 con 0x7f926c15a620 2023-10-10T20:30:40.969 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f926bfff700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 0x7f926c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:40.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f92710aa700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f926400c910 tx=0x7f9264007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264012630 con 0x7f926c15a620 2023-10-10T20:30:40.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640127b0 con 0x7f926c15a620 2023-10-10T20:30:40.970 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264012b70 con 0x7f926c15a620 2023-10-10T20:30:40.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0b19c0 con 0x7f926c15a620 2023-10-10T20:30:40.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.958+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c148a20 con 0x7f926c15a620 2023-10-10T20:30:40.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.959+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c04c3c0 con 0x7f926c15a620 2023-10-10T20:30:40.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.959+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f926401b400 con 0x7f926c15a620 2023-10-10T20:30:40.971 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.960+0000 7f9268ff9700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 0x7f92500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:40.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.960+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9264084780 con 0x7f926c15a620 2023-10-10T20:30:40.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.960+0000 7f92708a9700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 0x7f92500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:40.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.960+0000 7f92708a9700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 0x7f92500693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f92600093f0 tx=0x7f9260016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:40.972 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:40.964+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92640519f0 con 0x7f926c15a620 2023-10-10T20:30:41.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.147+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f926c0b3c30 con 0x7f926c15a620 2023-10-10T20:30:41.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.147+0000 7f9268ff9700 1 -- 172.21.15.148:0/610322347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f9264016090 con 0x7f926c15a620 2023-10-10T20:30:41.150 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:41.151 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":181,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:35.590886+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:41.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 msgr2=0x7f92500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 0x7f92500693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f92600093f0 tx=0x7f9260016000 comp rx=0 tx=0).stop 2023-10-10T20:30:41.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 msgr2=0x7f926c151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f926400c910 tx=0x7f9264007f90 comp rx=0 tx=0).stop 2023-10-10T20:30:41.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 shutdown_connections 2023-10-10T20:30:41.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f9250066f30 0x7f92500693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f926c15a620 0x7f926c151e60 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f926c157870 0x7f926c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 --2- 172.21.15.148:0/610322347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f926c157c40 0x7f926c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 >> 172.21.15.148:0/610322347 conn(0x7f926c068ce0 msgr2=0x7f926c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 shutdown_connections 2023-10-10T20:30:41.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.150+0000 7f9272b0d700 1 -- 172.21.15.148:0/610322347 wait complete. 2023-10-10T20:30:41.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-10T20:19:11.361845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-10T20:30:02.621495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-10T20:30:03.225751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-10T20:30:41.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-10T20:30:41.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.280+0000 7f5a788a0700 1 Processor -- start 2023-10-10T20:30:41.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.280+0000 7f5a788a0700 1 -- start start 2023-10-10T20:30:41.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a788a0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a74159ec0 0x7f5a740af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a788a0700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a7406acb0 0x7f5a740afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a788a0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740b0790 con 0x7f5a7415a290 2023-10-10T20:30:41.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a788a0700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740ce1f0 con 0x7f5a7406acb0 2023-10-10T20:30:41.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a788a0700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740b0b20 con 0x7f5a74159ec0 2023-10-10T20:30:41.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a72d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a7406acb0 0x7f5a740afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a71d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5a7415a290 0x7f5a7406a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51704/0 (socket says 172.21.15.148:51704) 2023-10-10T20:30:41.286 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a71d9b700 1 -- 172.21.15.148:0/4264859 learned_addr learned my addr 172.21.15.148:0/4264859 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a7259c700 1 --2- 172.21.15.148:0/4264859 >> v2:172.21.15.161:3300/0 conn(0x7f5a74159ec0 0x7f5a740af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a72d9d700 1 -- 172.21.15.148:0/4264859 >> v2:172.21.15.161:3300/0 conn(0x7f5a74159ec0 msgr2=0x7f5a740af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a72d9d700 1 --2- 172.21.15.148:0/4264859 >> v2:172.21.15.161:3300/0 conn(0x7f5a74159ec0 0x7f5a740af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a72d9d700 1 -- 172.21.15.148:0/4264859 >> v1:172.21.15.148:6789/0 conn(0x7f5a7415a290 legacy=0x7f5a7406a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a72d9d700 1 -- 172.21.15.148:0/4264859 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740b04b0 con 0x7f5a7406acb0 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.281+0000 7f5a7159a700 1 -- 172.21.15.148:0/4264859 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273980474 0 0) 0x7f5a740b0790 con 0x7f5a7415a290 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a72d9d700 1 --2- 172.21.15.148:0/4264859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a740afe80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5a740898e0 tx=0x7f5a6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7decac2c851b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a7159a700 1 -- 172.21.15.148:0/4264859 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00b6b0 con 0x7f5a7406acb0 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a7159a700 1 -- 172.21.15.148:0/4264859 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c00b810 con 0x7f5a7406acb0 2023-10-10T20:30:41.288 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a7159a700 1 -- 172.21.15.148:0/4264859 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00bbb0 con 0x7f5a7406acb0 2023-10-10T20:30:41.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 -- 172.21.15.148:0/4264859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 msgr2=0x7f5a740afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 --2- 172.21.15.148:0/4264859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a740afe80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5a740898e0 tx=0x7f5a6c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 -- 172.21.15.148:0/4264859 shutdown_connections 2023-10-10T20:30:41.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 --2- 172.21.15.148:0/4264859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a740afe80 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 --2- 172.21.15.148:0/4264859 >> v2:172.21.15.161:3300/0 conn(0x7f5a74159ec0 0x7f5a740af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 -- 172.21.15.148:0/4264859 >> 172.21.15.148:0/4264859 conn(0x7f5a740d1880 msgr2=0x7f5a740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 -- 172.21.15.148:0/4264859 shutdown_connections 2023-10-10T20:30:41.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.282+0000 7f5a788a0700 1 -- 172.21.15.148:0/4264859 wait complete. 2023-10-10T20:30:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.283+0000 7f5a788a0700 1 Processor -- start 2023-10-10T20:30:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.283+0000 7f5a788a0700 1 -- start start 2023-10-10T20:30:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 0x7f5a740b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a7415a290 0x7f5a740b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740c8070 con 0x7f5a7406acb0 2023-10-10T20:30:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a74155ec0 con 0x7f5a74159ec0 2023-10-10T20:30:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a788a0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cb1f0 con 0x7f5a7415a290 2023-10-10T20:30:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46818/0 (socket says 172.21.15.148:46818) 2023-10-10T20:30:41.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a7259c700 1 -- 172.21.15.148:0/2868914040 learned_addr learned my addr 172.21.15.148:0/2868914040 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:41.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a72d9d700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a7415a290 0x7f5a740b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.284+0000 7f5a71d9b700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 0x7f5a740b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 -- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a7415a290 msgr2=0x7f5a740b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a7415a290 0x7f5a740b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 -- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 msgr2=0x7f5a740b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 0x7f5a740b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 -- 172.21.15.148:0/2868914040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c009420 con 0x7f5a7406acb0 2023-10-10T20:30:41.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a71d9b700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 0x7f5a740b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:41.296 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a7259c700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e190 tx=0x7f5a64009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400ed80 con 0x7f5a7406acb0 2023-10-10T20:30:41.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64014420 con 0x7f5a7406acb0 2023-10-10T20:30:41.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a740cb4d0 con 0x7f5a7406acb0 2023-10-10T20:30:41.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640147e0 con 0x7f5a7406acb0 2023-10-10T20:30:41.297 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.285+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a740b6650 con 0x7f5a7406acb0 2023-10-10T20:30:41.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.286+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a740c8350 con 0x7f5a7406acb0 2023-10-10T20:30:41.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.286+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a64014b90 con 0x7f5a7406acb0 2023-10-10T20:30:41.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.287+0000 7f5a6a7fc700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 0x7f5a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.287+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a64084c80 con 0x7f5a7406acb0 2023-10-10T20:30:41.298 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.287+0000 7f5a71d9b700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 0x7f5a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.287+0000 7f5a71d9b700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 0x7f5a540693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5a7416b120 tx=0x7f5a5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.299 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.291+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a64051ef0 con 0x7f5a7406acb0 2023-10-10T20:30:41.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.475+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a7404c3c0 con 0x7f5a7406acb0 2023-10-10T20:30:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.475+0000 7f5a6a7fc700 1 -- 172.21.15.148:0/2868914040 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a64009a60 con 0x7f5a7406acb0 2023-10-10T20:30:41.478 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2023-10-10T20:30:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 msgr2=0x7f5a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 0x7f5a540693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5a7416b120 tx=0x7f5a5c009040 comp rx=0 tx=0).stop 2023-10-10T20:30:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 msgr2=0x7f5a74155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e190 tx=0x7f5a64009920 comp rx=0 tx=0).stop 2023-10-10T20:30:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 shutdown_connections 2023-10-10T20:30:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5a54066f30 0x7f5a540693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5a7415a290 0x7f5a740b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a7406acb0 0x7f5a74155980 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 --2- 172.21.15.148:0/2868914040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a74159ec0 0x7f5a740b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 >> 172.21.15.148:0/2868914040 conn(0x7f5a740d1880 msgr2=0x7f5a74156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 shutdown_connections 2023-10-10T20:30:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.477+0000 7f5a788a0700 1 -- 172.21.15.148:0/2868914040 wait complete. 2023-10-10T20:30:41.491 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-10T20:30:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.606+0000 7efe8f4ea700 1 Processor -- start 2023-10-10T20:30:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.606+0000 7efe8f4ea700 1 -- start start 2023-10-10T20:30:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8f4ea700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8f4ea700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe880c9390 0x7efe8815a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8f4ea700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe880cae70 con 0x7efe880678b0 2023-10-10T20:30:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8f4ea700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880cb150 con 0x7efe880c9390 2023-10-10T20:30:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8f4ea700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe880cb430 con 0x7efe880cbf10 2023-10-10T20:30:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8d286700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe880678b0 0x7efe8815db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51718/0 (socket says 172.21.15.148:51718) 2023-10-10T20:30:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8d286700 1 -- 172.21.15.148:0/2732603189 learned_addr learned my addr 172.21.15.148:0/2732603189 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8da87700 1 --2- 172.21.15.148:0/2732603189 >> v2:172.21.15.148:3300/0 conn(0x7efe880c9390 0x7efe8815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8ca85700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe7ffff700 1 -- 172.21.15.148:0/2732603189 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036878089 0 0) 0x7efe880cae70 con 0x7efe880678b0 2023-10-10T20:30:41.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe7ffff700 1 -- 172.21.15.148:0/2732603189 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe74003570 con 0x7efe880678b0 2023-10-10T20:30:41.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8da87700 1 -- 172.21.15.148:0/2732603189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 msgr2=0x7efe8815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:41.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8da87700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8da87700 1 -- 172.21.15.148:0/2732603189 >> v1:172.21.15.148:6789/0 conn(0x7efe880678b0 legacy=0x7efe8815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.607+0000 7efe8da87700 1 -- 172.21.15.148:0/2732603189 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880cb710 con 0x7efe880c9390 2023-10-10T20:30:41.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8da87700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe8815a370 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efe800098e0 tx=0x7efe800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fbf26669e1ee09 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe7ffff700 1 -- 172.21.15.148:0/2732603189 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8000b640 con 0x7efe880c9390 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe7ffff700 1 -- 172.21.15.148:0/2732603189 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8000b7a0 con 0x7efe880c9390 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe7ffff700 1 -- 172.21.15.148:0/2732603189 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8000bb40 con 0x7efe880c9390 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 -- 172.21.15.148:0/2732603189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 msgr2=0x7efe8815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe8815a370 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efe800098e0 tx=0x7efe800091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 -- 172.21.15.148:0/2732603189 shutdown_connections 2023-10-10T20:30:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe8815a370 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/2732603189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 -- 172.21.15.148:0/2732603189 >> 172.21.15.148:0/2732603189 conn(0x7efe88069870 msgr2=0x7efe88069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 -- 172.21.15.148:0/2732603189 shutdown_connections 2023-10-10T20:30:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.608+0000 7efe8f4ea700 1 -- 172.21.15.148:0/2732603189 wait complete. 2023-10-10T20:30:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.609+0000 7efe8f4ea700 1 Processor -- start 2023-10-10T20:30:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 -- start start 2023-10-10T20:30:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe880678b0 0x7efe8814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8816b080 con 0x7efe880c9390 2023-10-10T20:30:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe88151570 con 0x7efe880678b0 2023-10-10T20:30:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.610+0000 7efe8f4ea700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe88151240 con 0x7efe880cbf10 2023-10-10T20:30:41.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46834/0 (socket says 172.21.15.148:46834) 2023-10-10T20:30:41.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 -- 172.21.15.148:0/1024447996 learned_addr learned my addr 172.21.15.148:0/1024447996 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:41.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8d286700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe880678b0 0x7efe8814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8da87700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 -- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 msgr2=0x7efe8816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 -- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe880678b0 msgr2=0x7efe8814e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe880678b0 0x7efe8814e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 -- 172.21.15.148:0/1024447996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80009420 con 0x7efe880c9390 2023-10-10T20:30:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8ca85700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d860 tx=0x7efe78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe780125f0 con 0x7efe880c9390 2023-10-10T20:30:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78012750 con 0x7efe880c9390 2023-10-10T20:30:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8816aac0 con 0x7efe880c9390 2023-10-10T20:30:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78012b10 con 0x7efe880c9390 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.611+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8814ef20 con 0x7efe880c9390 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.612+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe88151080 con 0x7efe880c9390 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.613+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe7801e460 con 0x7efe880c9390 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.613+0000 7efe7d7fa700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 0x7efe6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.613+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe780849c0 con 0x7efe880c9390 2023-10-10T20:30:41.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.613+0000 7efe8d286700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 0x7efe6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.613+0000 7efe8d286700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 0x7efe6c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efe84005c50 tx=0x7efe84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.617+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe780556c0 con 0x7efe880c9390 2023-10-10T20:30:41.801 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.799+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe8804c3c0 con 0x7efe880c9390 2023-10-10T20:30:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.799+0000 7efe7d7fa700 1 -- 172.21.15.148:0/1024447996 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v181) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe78055250 con 0x7efe880c9390 2023-10-10T20:30:41.802 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2023-10-10T20:30:41.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.801+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 msgr2=0x7efe6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.804 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.801+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 0x7efe6c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7efe84005c50 tx=0x7efe84009040 comp rx=0 tx=0).stop 2023-10-10T20:30:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.801+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 msgr2=0x7efe88151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.801+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d860 tx=0x7efe78008040 comp rx=0 tx=0).stop 2023-10-10T20:30:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 shutdown_connections 2023-10-10T20:30:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7efe6c066f30 0x7efe6c0693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.805 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efe880cbf10 0x7efe8816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe880678b0 0x7efe8814e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 --2- 172.21.15.148:0/1024447996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe880c9390 0x7efe88151a00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 >> 172.21.15.148:0/1024447996 conn(0x7efe88069870 msgr2=0x7efe88150820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.806 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 shutdown_connections 2023-10-10T20:30:41.807 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.802+0000 7efe8f4ea700 1 -- 172.21.15.148:0/1024447996 wait complete. 2023-10-10T20:30:41.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-10T20:30:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.931+0000 7f8c338fd700 1 Processor -- start 2023-10-10T20:30:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.931+0000 7f8c338fd700 1 -- start start 2023-10-10T20:30:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c338fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c338fd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 0x7f8c2c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c338fd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c0b0790 con 0x7f8c2c159ec0 2023-10-10T20:30:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c338fd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c06af60 con 0x7f8c2c06aae0 2023-10-10T20:30:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c338fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0ce1f0 con 0x7f8c2c15a290 2023-10-10T20:30:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c30e98700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c30e98700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.148:47912/0 (socket says 172.21.15.148:47912) 2023-10-10T20:30:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c30e98700 1 -- 172.21.15.148:0/740484792 learned_addr learned my addr 172.21.15.148:0/740484792 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c31e9a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 0x7f8c2c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.932+0000 7f8c31699700 1 --1- 172.21.15.148:0/740484792 >> v1:172.21.15.161:6789/0 conn(0x7f8c2c159ec0 0x7f8c2c0af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:57366/0 (socket says 172.21.15.148:57366) 2023-10-10T20:30:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c30e98700 1 -- 172.21.15.148:0/740484792 >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 msgr2=0x7f8c2c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c30e98700 1 --2- 172.21.15.148:0/740484792 >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 0x7f8c2c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c30e98700 1 -- 172.21.15.148:0/740484792 >> v1:172.21.15.161:6789/0 conn(0x7f8c2c159ec0 legacy=0x7f8c2c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c30e98700 1 -- 172.21.15.148:0/740484792 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0b0b20 con 0x7f8c2c15a290 2023-10-10T20:30:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c31e9a700 1 --2- 172.21.15.148:0/740484792 >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 0x7f8c2c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c30e98700 1 --2- 172.21.15.148:0/740484792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099b0 tx=0x7f8c1c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ab23cb44ca829c server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c23fff700 1 -- 172.21.15.148:0/740484792 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c009ea0 con 0x7f8c2c15a290 2023-10-10T20:30:41.940 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c23fff700 1 -- 172.21.15.148:0/740484792 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c00b860 con 0x7f8c2c15a290 2023-10-10T20:30:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c23fff700 1 -- 172.21.15.148:0/740484792 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00bc00 con 0x7f8c2c15a290 2023-10-10T20:30:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.933+0000 7f8c338fd700 1 -- 172.21.15.148:0/740484792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 msgr2=0x7f8c2c06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 --2- 172.21.15.148:0/740484792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099b0 tx=0x7f8c1c009250 comp rx=0 tx=0).stop 2023-10-10T20:30:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 -- 172.21.15.148:0/740484792 shutdown_connections 2023-10-10T20:30:41.941 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 --2- 172.21.15.148:0/740484792 >> v2:172.21.15.161:3300/0 conn(0x7f8c2c06aae0 0x7f8c2c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 --2- 172.21.15.148:0/740484792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c15a290 0x7f8c2c06a5a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 -- 172.21.15.148:0/740484792 >> 172.21.15.148:0/740484792 conn(0x7f8c2c0d1880 msgr2=0x7f8c2c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 -- 172.21.15.148:0/740484792 shutdown_connections 2023-10-10T20:30:41.942 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.934+0000 7f8c338fd700 1 -- 172.21.15.148:0/740484792 wait complete. 2023-10-10T20:30:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.935+0000 7f8c338fd700 1 Processor -- start 2023-10-10T20:30:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.935+0000 7f8c338fd700 1 -- start start 2023-10-10T20:30:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c159ec0 0x7f8c2c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c15a290 0x7f8c2c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c1592a0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c158430 con 0x7f8c2c15a290 2023-10-10T20:30:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c157820 con 0x7f8c2c159ec0 2023-10-10T20:30:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46838/0 (socket says 172.21.15.148:46838) 2023-10-10T20:30:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 -- 172.21.15.148:0/609088521 learned_addr learned my addr 172.21.15.148:0/609088521 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31e9a700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c15a290 0x7f8c2c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.947 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c30e98700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c159ec0 0x7f8c2c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 -- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c159ec0 msgr2=0x7f8c2c154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c159ec0 0x7f8c2c154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 -- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c15a290 msgr2=0x7f8c2c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c15a290 0x7f8c2c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 -- 172.21.15.148:0/609088521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24009420 con 0x7f8c2c06aae0 2023-10-10T20:30:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c31699700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cd80 tx=0x7f8c28001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005df0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800a9f0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0094f0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.936+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800adb0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.937+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c157de0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.937+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c04c3c0 con 0x7f8c2c06aae0 2023-10-10T20:30:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.938+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c28007300 con 0x7f8c2c06aae0 2023-10-10T20:30:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.938+0000 7f8c217fa700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 0x7f8c10069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.938+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c28084680 con 0x7f8c2c06aae0 2023-10-10T20:30:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.938+0000 7f8c30e98700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 0x7f8c10069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.939+0000 7f8c30e98700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 0x7f8c10069410 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099b0 tx=0x7f8c1c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:41.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:41.942+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c280519b0 con 0x7f8c2c06aae0 2023-10-10T20:30:42.124 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.122+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c2c0b04b0 con 0x7f8c2c06aae0 2023-10-10T20:30:42.125 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.122+0000 7f8c217fa700 1 -- 172.21.15.148:0/609088521 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v181) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c28054fd0 con 0x7f8c2c06aae0 2023-10-10T20:30:42.125 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2023-10-10T20:30:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 msgr2=0x7f8c10069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 0x7f8c10069410 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0099b0 tx=0x7f8c1c017000 comp rx=0 tx=0).stop 2023-10-10T20:30:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 msgr2=0x7f8c2c155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cd80 tx=0x7f8c28001b90 comp rx=0 tx=0).stop 2023-10-10T20:30:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 shutdown_connections 2023-10-10T20:30:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8c10066f60 0x7f8c10069410 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c15a290 0x7f8c2c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c2c06aae0 0x7f8c2c155280 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 --2- 172.21.15.148:0/609088521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c2c159ec0 0x7f8c2c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 >> 172.21.15.148:0/609088521 conn(0x7f8c2c0d1880 msgr2=0x7f8c2c0b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 shutdown_connections 2023-10-10T20:30:42.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.124+0000 7f8c338fd700 1 -- 172.21.15.148:0/609088521 wait complete. 2023-10-10T20:30:42.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:42.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.250+0000 7f7bd8a39700 1 Processor -- start 2023-10-10T20:30:42.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.250+0000 7f7bd8a39700 1 -- start start 2023-10-10T20:30:42.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd8a39700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 0x7f7bd4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd8a39700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd415cbb0 con 0x7f7bd415a620 2023-10-10T20:30:42.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd8a39700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd415bfa0 con 0x7f7bd4157870 2023-10-10T20:30:42.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd8a39700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd415bdb0 con 0x7f7bd4157c40 2023-10-10T20:30:42.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd1d9b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7bd415a620 0x7f7bd415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51724/0 (socket says 172.21.15.148:51724) 2023-10-10T20:30:42.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd1d9b700 1 -- 172.21.15.148:0/1103828184 learned_addr learned my addr 172.21.15.148:0/1103828184 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:42.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.251+0000 7f7bd2d9d700 1 --2- 172.21.15.148:0/1103828184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 0x7f7bd4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176276046 0 0) 0x7f7bd415cbb0 con 0x7f7bd415a620 2023-10-10T20:30:42.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc0003570 con 0x7f7bd415a620 2023-10-10T20:30:42.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78394351 0 0) 0x7f7bd415bfa0 con 0x7f7bd4157870 2023-10-10T20:30:42.257 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd415cbb0 con 0x7f7bd4157870 2023-10-10T20:30:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd2d9d700 1 -- 172.21.15.148:0/1103828184 >> v1:172.21.15.161:6789/0 conn(0x7f7bd4157870 legacy=0x7f7bd415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd2d9d700 1 -- 172.21.15.148:0/1103828184 >> v1:172.21.15.148:6789/0 conn(0x7f7bd415a620 legacy=0x7f7bd415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd2d9d700 1 -- 172.21.15.148:0/1103828184 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd415c280 con 0x7f7bd4157c40 2023-10-10T20:30:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd2d9d700 1 --2- 172.21.15.148:0/1103828184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 0x7f7bd4158110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009910 tx=0x7f7bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f1e46397cb9561 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.258 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc011630 con 0x7f7bd4157c40 2023-10-10T20:30:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc011790 con 0x7f7bd4157c40 2023-10-10T20:30:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.252+0000 7f7bd159a700 1 -- 172.21.15.148:0/1103828184 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc011b30 con 0x7f7bd4157c40 2023-10-10T20:30:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 -- 172.21.15.148:0/1103828184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 msgr2=0x7f7bd4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/1103828184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 0x7f7bd4158110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009910 tx=0x7f7bcc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.259 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 -- 172.21.15.148:0/1103828184 shutdown_connections 2023-10-10T20:30:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/1103828184 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd4157c40 0x7f7bd4158110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 -- 172.21.15.148:0/1103828184 >> 172.21.15.148:0/1103828184 conn(0x7f7bd4068ce0 msgr2=0x7f7bd40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 -- 172.21.15.148:0/1103828184 shutdown_connections 2023-10-10T20:30:42.260 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.253+0000 7f7bd8a39700 1 -- 172.21.15.148:0/1103828184 wait complete. 2023-10-10T20:30:42.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 Processor -- start 2023-10-10T20:30:42.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 -- start start 2023-10-10T20:30:42.263 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bd4157c40 0x7f7bd4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd415a620 0x7f7bd414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd41501f0 con 0x7f7bd4157870 2023-10-10T20:30:42.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd414f5e0 con 0x7f7bd4157c40 2023-10-10T20:30:42.264 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.254+0000 7f7bd8a39700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd414f8c0 con 0x7f7bd415a620 2023-10-10T20:30:42.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46854/0 (socket says 172.21.15.148:46854) 2023-10-10T20:30:42.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bd4157c40 0x7f7bd4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 -- 172.21.15.148:0/4232192466 learned_addr learned my addr 172.21.15.148:0/4232192466 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:42.265 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd2d9d700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd415a620 0x7f7bd414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 -- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd415a620 msgr2=0x7f7bd414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd415a620 0x7f7bd414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 -- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bd4157c40 msgr2=0x7f7bd4150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bd4157c40 0x7f7bd4150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.266 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 -- 172.21.15.148:0/4232192466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc009420 con 0x7f7bd4157870 2023-10-10T20:30:42.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd259c700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400cb10 tx=0x7f7bc4002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400ce20 con 0x7f7bd4157870 2023-10-10T20:30:42.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc400a7e0 con 0x7f7bd4157870 2023-10-10T20:30:42.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd414fba0 con 0x7f7bd4157870 2023-10-10T20:30:42.267 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400ab80 con 0x7f7bd4157870 2023-10-10T20:30:42.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.255+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd414d4c0 con 0x7f7bd4157870 2023-10-10T20:30:42.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.256+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd41504d0 con 0x7f7bd4157870 2023-10-10T20:30:42.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.257+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7bc4012420 con 0x7f7bd4157870 2023-10-10T20:30:42.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.257+0000 7f7bca7fc700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 0x7f7bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.268 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.257+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bc40847c0 con 0x7f7bd4157870 2023-10-10T20:30:42.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.257+0000 7f7bd1d9b700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 0x7f7bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.258+0000 7f7bd1d9b700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 0x7f7bb40693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40b36c0 tx=0x7f7bbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.269 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.262+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc4051a30 con 0x7f7bd4157870 2023-10-10T20:30:42.490 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.487+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bd4150040 con 0x7f7bd4157870 2023-10-10T20:30:42.491 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.488+0000 7f7bca7fc700 1 -- 172.21.15.148:0/4232192466 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f7bc4055050 con 0x7f7bd4157870 2023-10-10T20:30:42.491 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:42.491 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:42.492 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 msgr2=0x7f7bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 0x7f7bb40693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40b36c0 tx=0x7f7bbc009040 comp rx=0 tx=0).stop 2023-10-10T20:30:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 msgr2=0x7f7bd414cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400cb10 tx=0x7f7bc4002820 comp rx=0 tx=0).stop 2023-10-10T20:30:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 shutdown_connections 2023-10-10T20:30:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7bb4066f30 0x7f7bb40693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bd415a620 0x7f7bd414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd4157870 0x7f7bd414cf80 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 --2- 172.21.15.148:0/4232192466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bd4157c40 0x7f7bd4150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 >> 172.21.15.148:0/4232192466 conn(0x7f7bd4068ce0 msgr2=0x7f7bd40b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 shutdown_connections 2023-10-10T20:30:42.495 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.490+0000 7f7bd8a39700 1 -- 172.21.15.148:0/4232192466 wait complete. 2023-10-10T20:30:42.495 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:42.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:42.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.616+0000 7fd08b9ae700 1 Processor -- start 2023-10-10T20:30:42.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.616+0000 7fd08b9ae700 1 -- start start 2023-10-10T20:30:42.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd084157870 0x7fd08415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 0x7fd08415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 0x7fd084158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415cbb0 con 0x7fd084157870 2023-10-10T20:30:42.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415bfa0 con 0x7fd08415a620 2023-10-10T20:30:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08b9ae700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415c280 con 0x7fd08415d580 2023-10-10T20:30:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08974a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd084157870 0x7fd08415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08974a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd084157870 0x7fd08415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37118/0 (socket says 172.21.15.148:37118) 2023-10-10T20:30:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd08974a700 1 -- 172.21.15.148:0/1658974792 learned_addr learned my addr 172.21.15.148:0/1658974792 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd088f49700 1 --2- 172.21.15.148:0/1658974792 >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 0x7fd08415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.617+0000 7fd089f4b700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 0x7fd084158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 -- 172.21.15.148:0/1658974792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 msgr2=0x7fd084158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:42.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 0x7fd084158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 -- 172.21.15.148:0/1658974792 >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 msgr2=0x7fd08415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:42.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 --2- 172.21.15.148:0/1658974792 >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 0x7fd08415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 -- 172.21.15.148:0/1658974792 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08415c560 con 0x7fd084157870 2023-10-10T20:30:42.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd088f49700 1 --2- 172.21.15.148:0/1658974792 >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 0x7fd08415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:42.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd089f4b700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 0x7fd084158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:42.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08974a700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd08415b6a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd08415cb20 tx=0x7fd080009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6442c1c125a877 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd07bfff700 1 -- 172.21.15.148:0/1658974792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000e070 con 0x7fd084157870 2023-10-10T20:30:42.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd07bfff700 1 -- 172.21.15.148:0/1658974792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000ba50 con 0x7fd084157870 2023-10-10T20:30:42.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd07bfff700 1 -- 172.21.15.148:0/1658974792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800092e0 con 0x7fd084157870 2023-10-10T20:30:42.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 -- 172.21.15.148:0/1658974792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 msgr2=0x7fd08415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd08415b6a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd08415cb20 tx=0x7fd080009840 comp rx=0 tx=0).stop 2023-10-10T20:30:42.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 -- 172.21.15.148:0/1658974792 shutdown_connections 2023-10-10T20:30:42.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415d580 0x7fd084158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/1658974792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd08415b6a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/1658974792 >> v2:172.21.15.161:3300/0 conn(0x7fd08415a620 0x7fd08415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.618+0000 7fd08b9ae700 1 -- 172.21.15.148:0/1658974792 >> 172.21.15.148:0/1658974792 conn(0x7fd084068ce0 msgr2=0x7fd0840690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.619+0000 7fd08b9ae700 1 -- 172.21.15.148:0/1658974792 shutdown_connections 2023-10-10T20:30:42.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.619+0000 7fd08b9ae700 1 -- 172.21.15.148:0/1658974792 wait complete. 2023-10-10T20:30:42.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 Processor -- start 2023-10-10T20:30:42.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 -- start start 2023-10-10T20:30:42.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415a620 0x7fd084151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 0x7fd084152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084150220 con 0x7fd084157870 2023-10-10T20:30:42.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840d0f90 con 0x7fd08415d580 2023-10-10T20:30:42.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.620+0000 7fd08b9ae700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840d00e0 con 0x7fd08415a620 2023-10-10T20:30:42.632 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37128/0 (socket says 172.21.15.148:37128) 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 -- 172.21.15.148:0/3675176871 learned_addr learned my addr 172.21.15.148:0/3675176871 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd089f4b700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 0x7fd084152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd088f49700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415a620 0x7fd084151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 -- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415a620 msgr2=0x7fd084151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415a620 0x7fd084151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 -- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 msgr2=0x7fd084152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 0x7fd084152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 -- 172.21.15.148:0/3675176871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080009ae0 con 0x7fd084157870 2023-10-10T20:30:42.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd089f4b700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 0x7fd084152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:42.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08974a700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd08406b530 tx=0x7fd080018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080008b70 con 0x7fd084157870 2023-10-10T20:30:42.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080008d10 con 0x7fd084157870 2023-10-10T20:30:42.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800266b0 con 0x7fd084157870 2023-10-10T20:30:42.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0840d0380 con 0x7fd084157870 2023-10-10T20:30:42.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.621+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0840d0940 con 0x7fd084157870 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.622+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08404c3c0 con 0x7fd084157870 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.622+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd080003040 con 0x7fd084157870 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.623+0000 7fd0797fa700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 0x7fd0680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.623+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd080085d50 con 0x7fd084157870 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.623+0000 7fd088f49700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 0x7fd0680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.636 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.623+0000 7fd088f49700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 0x7fd0680693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd074006f00 tx=0x7fd074008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.637 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.627+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd080089030 con 0x7fd084157870 2023-10-10T20:30:42.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.811+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd08415c840 con 0x7fd084157870 2023-10-10T20:30:42.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.812+0000 7fd0797fa700 1 -- 172.21.15.148:0/3675176871 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+15420 (secure 0 0 0) 0x7fd080052fc0 con 0x7fd084157870 2023-10-10T20:30:42.815 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:42.815 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":181,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:35.590886+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:42.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 msgr2=0x7fd0680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 0x7fd0680693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd074006f00 tx=0x7fd074008040 comp rx=0 tx=0).stop 2023-10-10T20:30:42.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 msgr2=0x7fd0840b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd08406b530 tx=0x7fd080018000 comp rx=0 tx=0).stop 2023-10-10T20:30:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 shutdown_connections 2023-10-10T20:30:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd068066f30 0x7fd0680693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd08415d580 0x7fd084152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd084157870 0x7fd0840b2f60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 --2- 172.21.15.148:0/3675176871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd08415a620 0x7fd084151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 >> 172.21.15.148:0/3675176871 conn(0x7fd084068ce0 msgr2=0x7fd08415e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 shutdown_connections 2023-10-10T20:30:42.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.814+0000 7fd08b9ae700 1 -- 172.21.15.148:0/3675176871 wait complete. 2023-10-10T20:30:42.829 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:30:42.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:42.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.942+0000 7f7681725700 1 Processor -- start 2023-10-10T20:30:42.945 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.942+0000 7f7681725700 1 -- start start 2023-10-10T20:30:42.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f7681725700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f767c157c40 0x7f767c158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f7681725700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c15cbb0 con 0x7f767c157870 2023-10-10T20:30:42.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f7681725700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c15bfa0 con 0x7f767c15a620 2023-10-10T20:30:42.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f7681725700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c15bdb0 con 0x7f767c157c40 2023-10-10T20:30:42.949 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f767affd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f767c157870 0x7f767c15b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33328/0 (socket says 172.21.15.148:33328) 2023-10-10T20:30:42.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.943+0000 7f767affd700 1 -- 172.21.15.148:0/3183581683 learned_addr learned my addr 172.21.15.148:0/3183581683 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:42.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250308767 0 0) 0x7f767c15cbb0 con 0x7f767c157870 2023-10-10T20:30:42.950 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7668003570 con 0x7f767c157870 2023-10-10T20:30:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3183581683 >> v2:172.21.15.161:3300/0 conn(0x7f767c157c40 0x7f767c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462284952 0 0) 0x7f7668003570 con 0x7f767c157870 2023-10-10T20:30:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 >> v2:172.21.15.161:3300/0 conn(0x7f767c157c40 msgr2=0x7f767c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 --2- 172.21.15.148:0/3183581683 >> v2:172.21.15.161:3300/0 conn(0x7f767c157c40 0x7f767c158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 >> v1:172.21.15.161:6789/0 conn(0x7f767c15a620 legacy=0x7f767c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c15c280 con 0x7f767c157870 2023-10-10T20:30:42.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f766c001470 con 0x7f767c157870 2023-10-10T20:30:42.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 >> v1:172.21.15.148:6789/0 conn(0x7f767c157870 legacy=0x7f767c15b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.952 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7668003a60 0x7f7668003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7668004810 0x7f7668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766800c130 con 0x7f76680051c0 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766800c2a0 con 0x7f7668003a60 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766800c460 con 0x7f7668004810 2023-10-10T20:30:42.953 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767affd700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7668003a60 0x7f7668003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f76727fc700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7668004810 0x7f7668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 -- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7668004810 msgr2=0x7f7668004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7668004810 0x7f7668004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.954 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 -- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7668003a60 msgr2=0x7f7668003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:42.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7668003a60 0x7f7668003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 -- 172.21.15.148:0/3183581683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668003570 con 0x7f76680051c0 2023-10-10T20:30:42.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.944+0000 7f767b7fe700 1 -- 172.21.15.148:0/3183581683 reap_dead start 2023-10-10T20:30:42.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f766800ba60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f767c1580c0 tx=0x7f767400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5101eaae65a67ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.955 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400ea90 con 0x7f76680051c0 2023-10-10T20:30:42.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767400f040 con 0x7f76680051c0 2023-10-10T20:30:42.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f767a7fc700 1 -- 172.21.15.148:0/3183581683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400ee60 con 0x7f76680051c0 2023-10-10T20:30:42.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 -- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 msgr2=0x7f766800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f766800ba60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f767c1580c0 tx=0x7f767400b040 comp rx=0 tx=0).stop 2023-10-10T20:30:42.956 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 -- 172.21.15.148:0/3183581683 shutdown_connections 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 --2- 172.21.15.148:0/3183581683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f766800ba60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 -- 172.21.15.148:0/3183581683 >> 172.21.15.148:0/3183581683 conn(0x7f767c068ce0 msgr2=0x7f767c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 -- 172.21.15.148:0/3183581683 shutdown_connections 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.945+0000 7f7681725700 1 -- 172.21.15.148:0/3183581683 wait complete. 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.946+0000 7f7681725700 1 Processor -- start 2023-10-10T20:30:42.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.946+0000 7f7681725700 1 -- start start 2023-10-10T20:30:42.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f767c1558a0 0x7f767c14d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 0x7f767c140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c151a20 con 0x7f76680051c0 2023-10-10T20:30:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c16dd50 con 0x7f767c1558a0 2023-10-10T20:30:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f7681725700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c1514a0 con 0x7f767c155d10 2023-10-10T20:30:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f767affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f767affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37138/0 (socket says 172.21.15.148:37138) 2023-10-10T20:30:42.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f767affd700 1 -- 172.21.15.148:0/3714271707 learned_addr learned my addr 172.21.15.148:0/3714271707 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:42.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f76727fc700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f767c1558a0 0x7f767c14d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.947+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 0x7f767c140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 -- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 msgr2=0x7f767c140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 0x7f767c140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 -- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f767c1558a0 msgr2=0x7f767c14d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:42.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f767c1558a0 0x7f767c14d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:42.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 -- 172.21.15.148:0/3714271707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c151740 con 0x7f76680051c0 2023-10-10T20:30:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767b7fe700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 0x7f767c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f767affd700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f766c007390 tx=0x7f766c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c018030 con 0x7f76680051c0 2023-10-10T20:30:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00f8d0 con 0x7f76680051c0 2023-10-10T20:30:42.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00fcb0 con 0x7f76680051c0 2023-10-10T20:30:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c16d9e0 con 0x7f76680051c0 2023-10-10T20:30:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.948+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c14fd70 con 0x7f76680051c0 2023-10-10T20:30:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.949+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c16d840 con 0x7f76680051c0 2023-10-10T20:30:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.949+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f766c01b070 con 0x7f76680051c0 2023-10-10T20:30:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.950+0000 7f76737fe700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 0x7f765c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.950+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f766c019070 con 0x7f76680051c0 2023-10-10T20:30:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.950+0000 7f76727fc700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 0x7f765c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.950+0000 7f76727fc700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 0x7f765c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f767c15b530 tx=0x7f7664007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:42.954+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f767c16d840 con 0x7f76680051c0 2023-10-10T20:30:43.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.177+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f767c16d840 con 0x7f76680051c0 2023-10-10T20:30:43.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.177+0000 7f76737fe700 1 -- 172.21.15.148:0/3714271707 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7f767c16d840 con 0x7f76680051c0 2023-10-10T20:30:43.180 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:43.180 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:43.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 msgr2=0x7f765c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 0x7f765c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f767c15b530 tx=0x7f7664007d30 comp rx=0 tx=0).stop 2023-10-10T20:30:43.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 msgr2=0x7f767c16e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f766c007390 tx=0x7f766c001e50 comp rx=0 tx=0).stop 2023-10-10T20:30:43.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 shutdown_connections 2023-10-10T20:30:43.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f765c066f30 0x7f765c0693e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f767c155d10 0x7f767c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76680051c0 0x7f767c16e1e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 --2- 172.21.15.148:0/3714271707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f767c1558a0 0x7f767c14d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.179+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 >> 172.21.15.148:0/3714271707 conn(0x7f767c068ce0 msgr2=0x7f767c159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:43.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.180+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 shutdown_connections 2023-10-10T20:30:43.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.180+0000 7f7681725700 1 -- 172.21.15.148:0/3714271707 wait complete. 2023-10-10T20:30:43.185 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:43.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:30:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.306+0000 7fa336629700 1 Processor -- start 2023-10-10T20:30:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.306+0000 7fa336629700 1 -- start start 2023-10-10T20:30:43.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa336629700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa330157e60 0x7fa3300b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.311 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa336629700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300cb0b0 con 0x7fa33006fef0 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa336629700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300b0830 con 0x7fa330157a90 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa336629700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3300b0ad0 con 0x7fa330157e60 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa335e28700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa33006fef0 0x7fa3300ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33344/0 (socket says 172.21.15.148:33344) 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa335e28700 1 -- 172.21.15.148:0/3439363107 learned_addr learned my addr 172.21.15.148:0/3439363107 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.307+0000 7fa334e26700 1 --2- 172.21.15.148:0/3439363107 >> v2:172.21.15.161:3300/0 conn(0x7fa330157e60 0x7fa3300b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.312 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020179989 0 0) 0x7fa3300cb0b0 con 0x7fa33006fef0 2023-10-10T20:30:43.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31c003570 con 0x7fa33006fef0 2023-10-10T20:30:43.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086056352 0 0) 0x7fa3300b0830 con 0x7fa330157a90 2023-10-10T20:30:43.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3300cb0b0 con 0x7fa330157a90 2023-10-10T20:30:43.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa334e26700 1 -- 172.21.15.148:0/3439363107 >> v1:172.21.15.161:6789/0 conn(0x7fa330157a90 legacy=0x7fa3300b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.313 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa334e26700 1 -- 172.21.15.148:0/3439363107 >> v1:172.21.15.148:6789/0 conn(0x7fa33006fef0 legacy=0x7fa3300ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa334e26700 1 -- 172.21.15.148:0/3439363107 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300ca780 con 0x7fa330157e60 2023-10-10T20:30:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa334e26700 1 --2- 172.21.15.148:0/3439363107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330157e60 0x7fa3300b1790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098e0 tx=0x7fa3200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9637e61c06eb3db9 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011630 con 0x7fa330157e60 2023-10-10T20:30:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320011790 con 0x7fa330157e60 2023-10-10T20:30:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa327fff700 1 -- 172.21.15.148:0/3439363107 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320011b30 con 0x7fa330157e60 2023-10-10T20:30:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa336629700 1 -- 172.21.15.148:0/3439363107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330157e60 msgr2=0x7fa3300b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.315 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.308+0000 7fa336629700 1 --2- 172.21.15.148:0/3439363107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330157e60 0x7fa3300b1790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098e0 tx=0x7fa3200091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.309+0000 7fa336629700 1 -- 172.21.15.148:0/3439363107 shutdown_connections 2023-10-10T20:30:43.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.309+0000 7fa336629700 1 --2- 172.21.15.148:0/3439363107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa330157e60 0x7fa3300b1790 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.309+0000 7fa336629700 1 -- 172.21.15.148:0/3439363107 >> 172.21.15.148:0/3439363107 conn(0x7fa33006bc70 msgr2=0x7fa33006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:43.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.309+0000 7fa336629700 1 -- 172.21.15.148:0/3439363107 shutdown_connections 2023-10-10T20:30:43.316 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.309+0000 7fa336629700 1 -- 172.21.15.148:0/3439363107 wait complete. 2023-10-10T20:30:43.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.310+0000 7fa336629700 1 Processor -- start 2023-10-10T20:30:43.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.310+0000 7fa336629700 1 -- start start 2023-10-10T20:30:43.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa33006fef0 0x7fa3300c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.319 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 0x7fa3300b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300b2810 con 0x7fa330157e60 2023-10-10T20:30:43.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300b3020 con 0x7fa33006fef0 2023-10-10T20:30:43.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300b2a10 con 0x7fa330157a90 2023-10-10T20:30:43.320 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37154/0 (socket says 172.21.15.148:37154) 2023-10-10T20:30:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 -- 172.21.15.148:0/2789123850 learned_addr learned my addr 172.21.15.148:0/2789123850 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa334e26700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 0x7fa3300b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335627700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa33006fef0 0x7fa3300c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 -- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 msgr2=0x7fa3300b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 0x7fa3300b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 -- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa33006fef0 msgr2=0x7fa3300c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.322 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa33006fef0 0x7fa3300c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 -- 172.21.15.148:0/2789123850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009420 con 0x7fa330157e60 2023-10-10T20:30:43.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa334e26700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 0x7fa3300b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:43.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa335e28700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa32800cc20 tx=0x7fa328001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32800a960 con 0x7fa330157e60 2023-10-10T20:30:43.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32800aac0 con 0x7fa330157e60 2023-10-10T20:30:43.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3300c6aa0 con 0x7fa330157e60 2023-10-10T20:30:43.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32800ae80 con 0x7fa330157e60 2023-10-10T20:30:43.324 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.311+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3300b2cb0 con 0x7fa330157e60 2023-10-10T20:30:43.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.312+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3300b3190 con 0x7fa330157e60 2023-10-10T20:30:43.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.313+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa328007300 con 0x7fa330157e60 2023-10-10T20:30:43.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.313+0000 7fa3257fa700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 0x7fa3140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.313+0000 7fa335627700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 0x7fa3140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.325 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.313+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa328084900 con 0x7fa330157e60 2023-10-10T20:30:43.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.314+0000 7fa335627700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 0x7fa3140693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa32c003870 tx=0x7fa32c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.326 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.318+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa328055680 con 0x7fa330157e60 2023-10-10T20:30:43.546 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.544+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa33004bb80 con 0x7fa330157e60 2023-10-10T20:30:43.547 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.544+0000 7fa3257fa700 1 -- 172.21.15.148:0/2789123850 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fa328055210 con 0x7fa330157e60 2023-10-10T20:30:43.547 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:43.548 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-10T20:30:04.260506+0000","modified":"2023-10-10T20:30:07.906439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8139":{"gid":8139,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/1942118864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":1942118864},{"type":"v1","addr":"172.21.15.161:6835","nonce":1942118864}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-10T20:30:43.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.546+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 msgr2=0x7fa3140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.549 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.546+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 0x7fa3140693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa32c003870 tx=0x7fa32c009040 comp rx=0 tx=0).stop 2023-10-10T20:30:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.546+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 msgr2=0x7fa3301539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.546+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fa32800cc20 tx=0x7fa328001a40 comp rx=0 tx=0).stop 2023-10-10T20:30:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 shutdown_connections 2023-10-10T20:30:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa314066f30 0x7fa3140693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa330157e60 0x7fa3301539d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa33006fef0 0x7fa3300c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 --2- 172.21.15.148:0/2789123850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa330157a90 0x7fa3300b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 >> 172.21.15.148:0/2789123850 conn(0x7fa33006bc70 msgr2=0x7fa3300cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 shutdown_connections 2023-10-10T20:30:43.551 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.547+0000 7fa336629700 1 -- 172.21.15.148:0/2789123850 wait complete. 2023-10-10T20:30:43.552 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 85 2023-10-10T20:30:43.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:43.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.674+0000 7f5de5fcf700 1 Processor -- start 2023-10-10T20:30:43.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.674+0000 7f5de5fcf700 1 -- start start 2023-10-10T20:30:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5de5fcf700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5de0159cf0 0x7f5de015e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5de5fcf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de0068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5de5fcf700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de00cca20 con 0x7f5de015a0c0 2023-10-10T20:30:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5de5fcf700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de015ea40 con 0x7f5de0159cf0 2023-10-10T20:30:43.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5de5fcf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0067660 con 0x7f5de00cf1c0 2023-10-10T20:30:43.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5ddf7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5de0159cf0 0x7f5de015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5ddf7fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5de0159cf0 0x7f5de015e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37166/0 (socket says 172.21.15.148:37166) 2023-10-10T20:30:43.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5ddeffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5de015a0c0 0x7f5de00ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33354/0 (socket says 172.21.15.148:33354) 2023-10-10T20:30:43.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.675+0000 7f5ddf7fe700 1 -- 172.21.15.148:0/3413711436 learned_addr learned my addr 172.21.15.148:0/3413711436 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:43.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddffff700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddf7fe700 1 -- 172.21.15.148:0/3413711436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 msgr2=0x7f5de0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:43.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddf7fe700 1 -- 172.21.15.148:0/3413711436 >> v1:172.21.15.148:6789/0 conn(0x7f5de015a0c0 legacy=0x7f5de00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5dde7fc700 1 -- 172.21.15.148:0/3413711436 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603982069 0 0) 0x7f5de00cca20 con 0x7f5de015a0c0 2023-10-10T20:30:43.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddf7fe700 1 -- 172.21.15.148:0/3413711436 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0067f00 con 0x7f5de0159cf0 2023-10-10T20:30:43.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de0159cf0 0x7f5de015e500 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009f20 tx=0x7f5dd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aa149ec1b912e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5dde7fc700 1 -- 172.21.15.148:0/3413711436 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000e070 con 0x7f5de0159cf0 2023-10-10T20:30:43.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5dde7fc700 1 -- 172.21.15.148:0/3413711436 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000b9d0 con 0x7f5de0159cf0 2023-10-10T20:30:43.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.676+0000 7f5dde7fc700 1 -- 172.21.15.148:0/3413711436 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00092e0 con 0x7f5de0159cf0 2023-10-10T20:30:43.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 -- 172.21.15.148:0/3413711436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de0159cf0 msgr2=0x7f5de015e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de0159cf0 0x7f5de015e500 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009f20 tx=0x7f5dd0009840 comp rx=0 tx=0).stop 2023-10-10T20:30:43.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 -- 172.21.15.148:0/3413711436 shutdown_connections 2023-10-10T20:30:43.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/3413711436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de0159cf0 0x7f5de015e500 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 -- 172.21.15.148:0/3413711436 >> 172.21.15.148:0/3413711436 conn(0x7f5de006c6e0 msgr2=0x7f5de0069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:43.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 -- 172.21.15.148:0/3413711436 shutdown_connections 2023-10-10T20:30:43.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.677+0000 7f5de5fcf700 1 -- 172.21.15.148:0/3413711436 wait complete. 2023-10-10T20:30:43.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.678+0000 7f5de5fcf700 1 Processor -- start 2023-10-10T20:30:43.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.678+0000 7f5de5fcf700 1 -- start start 2023-10-10T20:30:43.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de00b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de0159cf0 0x7f5de0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00c6d10 con 0x7f5de015a0c0 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00b3d50 con 0x7f5de0159cf0 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00c6f60 con 0x7f5de00cf1c0 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37176/0 (socket says 172.21.15.148:37176) 2023-10-10T20:30:43.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 -- 172.21.15.148:0/312608588 learned_addr learned my addr 172.21.15.148:0/312608588 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:43.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de00b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddeffd700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de0159cf0 0x7f5de0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 -- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 msgr2=0x7f5de00b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de00b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 -- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de0159cf0 msgr2=0x7f5de0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de0159cf0 0x7f5de0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 -- 172.21.15.148:0/312608588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009ae0 con 0x7f5de015a0c0 2023-10-10T20:30:43.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:43.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5ddffff700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800ea80 tx=0x7f5dd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800c8c0 con 0x7f5de015a0c0 2023-10-10T20:30:43.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800ca20 con 0x7f5de015a0c0 2023-10-10T20:30:43.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de00c7240 con 0x7f5de015a0c0 2023-10-10T20:30:43.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.679+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800cde0 con 0x7f5de015a0c0 2023-10-10T20:30:43.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.680+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de00b4720 con 0x7f5de015a0c0 2023-10-10T20:30:43.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.680+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de00b3ee0 con 0x7f5de015a0c0 2023-10-10T20:30:43.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.681+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dd8013680 con 0x7f5de015a0c0 2023-10-10T20:30:43.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.681+0000 7f5dc77fe700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 0x7f5dcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:43.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.681+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dd8084810 con 0x7f5de015a0c0 2023-10-10T20:30:43.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.681+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 0x7f5dcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:43.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.682+0000 7f5ddf7fe700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 0x7f5dcc0693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009880 tx=0x7f5dd0018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:43.694 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.686+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dd8051a80 con 0x7f5de015a0c0 2023-10-10T20:30:43.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.869+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5de004c3c0 con 0x7f5de015a0c0 2023-10-10T20:30:43.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.870+0000 7f5dc77fe700 1 -- 172.21.15.148:0/312608588 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f5dd80550a0 con 0x7f5de015a0c0 2023-10-10T20:30:43.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:43.873 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":181,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:35.590886+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-10T20:30:02.621495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-10T20:30:03.225751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:43.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 msgr2=0x7f5dcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 0x7f5dcc0693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009880 tx=0x7f5dd0018000 comp rx=0 tx=0).stop 2023-10-10T20:30:43.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 msgr2=0x7f5de0158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:43.875 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800ea80 tx=0x7f5dd80091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 shutdown_connections 2023-10-10T20:30:43.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f5dcc066f30 0x7f5dcc0693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5de015a0c0 0x7f5de0158b20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5de00cf1c0 0x7f5de00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.876 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 --2- 172.21.15.148:0/312608588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de0159cf0 0x7f5de0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:43.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 >> 172.21.15.148:0/312608588 conn(0x7f5de006c6e0 msgr2=0x7f5de00c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:43.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 shutdown_connections 2023-10-10T20:30:43.877 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:43.872+0000 7f5de5fcf700 1 -- 172.21.15.148:0/312608588 wait complete. 2023-10-10T20:30:43.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-10T20:30:44.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.000+0000 7f98d74b6700 1 Processor -- start 2023-10-10T20:30:44.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.000+0000 7f98d74b6700 1 -- start start 2023-10-10T20:30:44.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d74b6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98d015a170 0x7f98d0068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d74b6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00694a0 con 0x7f98d0159da0 2023-10-10T20:30:44.006 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d74b6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d015ea70 con 0x7f98d0069040 2023-10-10T20:30:44.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d74b6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d00b34b0 con 0x7f98d015a170 2023-10-10T20:30:44.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d64b4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f98d0159da0 0x7f98d00b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33358/0 (socket says 172.21.15.148:33358) 2023-10-10T20:30:44.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d64b4700 1 -- 172.21.15.148:0/35037087 learned_addr learned my addr 172.21.15.148:0/35037087 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:44.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300347184 0 0) 0x7f98cc001490 con 0x7f98d0159da0 2023-10-10T20:30:44.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98bc003570 con 0x7f98d0159da0 2023-10-10T20:30:44.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.001+0000 7f98d5cb3700 1 --2- 172.21.15.148:0/35037087 >> v2:172.21.15.161:3300/0 conn(0x7f98d015a170 0x7f98d0068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2343877404 0 0) 0x7f98bc003570 con 0x7f98d0159da0 2023-10-10T20:30:44.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 >> v2:172.21.15.161:3300/0 conn(0x7f98d015a170 msgr2=0x7f98d0068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:44.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 --2- 172.21.15.148:0/35037087 >> v2:172.21.15.161:3300/0 conn(0x7f98d015a170 0x7f98d0068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 >> v1:172.21.15.161:6789/0 conn(0x7f98d0069040 legacy=0x7f98d00af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00b3990 con 0x7f98d0159da0 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f98cc001980 con 0x7f98d0159da0 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 >> v1:172.21.15.148:6789/0 conn(0x7f98d0159da0 legacy=0x7f98d00b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 0x7f98bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc004810 0x7f98bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc0053c0 0x7f98bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00b34b0 con 0x7f98bc003a60 2023-10-10T20:30:44.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc00c2d0 con 0x7f98bc004810 2023-10-10T20:30:44.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc00c490 con 0x7f98bc0053c0 2023-10-10T20:30:44.010 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 0x7f98bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d5cb3700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc004810 0x7f98bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d6cb5700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc0053c0 0x7f98bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 -- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc0053c0 msgr2=0x7f98bc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc0053c0 0x7f98bc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 -- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc004810 msgr2=0x7f98bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:44.011 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc004810 0x7f98bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 -- 172.21.15.148:0/35037087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc003570 con 0x7f98bc003a60 2023-10-10T20:30:44.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d6cb5700 1 -- 172.21.15.148:0/35037087 reap_dead start 2023-10-10T20:30:44.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d64b4700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 0x7f98bc003e70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00aaa0 tx=0x7f98cc00a170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cade1f0a516835 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00c5d0 con 0x7f98bc003a60 2023-10-10T20:30:44.012 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00c750 con 0x7f98bc003a60 2023-10-10T20:30:44.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.002+0000 7f98d54b2700 1 -- 172.21.15.148:0/35037087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00cb10 con 0x7f98bc003a60 2023-10-10T20:30:44.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 -- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 msgr2=0x7f98bc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 0x7f98bc003e70 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00aaa0 tx=0x7f98cc00a170 comp rx=0 tx=0).stop 2023-10-10T20:30:44.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 -- 172.21.15.148:0/35037087 shutdown_connections 2023-10-10T20:30:44.013 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 --2- 172.21.15.148:0/35037087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98bc003a60 0x7f98bc003e70 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 -- 172.21.15.148:0/35037087 >> 172.21.15.148:0/35037087 conn(0x7f98d00d0bc0 msgr2=0x7f98d00d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:44.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 -- 172.21.15.148:0/35037087 shutdown_connections 2023-10-10T20:30:44.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.003+0000 7f98d74b6700 1 -- 172.21.15.148:0/35037087 wait complete. 2023-10-10T20:30:44.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.004+0000 7f98d74b6700 1 Processor -- start 2023-10-10T20:30:44.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.004+0000 7f98d74b6700 1 -- start start 2023-10-10T20:30:44.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc003a60 0x7f98d00afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d00c7490 0x7f98d00ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d015b2f0 con 0x7f98d00c7020 2023-10-10T20:30:44.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00b0100 con 0x7f98d00c7490 2023-10-10T20:30:44.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d015a6e0 con 0x7f98bc003a60 2023-10-10T20:30:44.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37202/0 (socket says 172.21.15.148:37202) 2023-10-10T20:30:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 -- 172.21.15.148:0/3936199881 learned_addr learned my addr 172.21.15.148:0/3936199881 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d64b4700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc003a60 0x7f98d00afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d6cb5700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d00c7490 0x7f98d00ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 -- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc003a60 msgr2=0x7f98d00afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc003a60 0x7f98d00afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 -- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d00c7490 msgr2=0x7f98d00ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d00c7490 0x7f98d00ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 -- 172.21.15.148:0/3936199881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d015a9c0 con 0x7f98d00c7020 2023-10-10T20:30:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d5cb3700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f98c0010d30 tx=0x7f98c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000be00 con 0x7f98d00c7020 2023-10-10T20:30:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000ec60 con 0x7f98d00c7020 2023-10-10T20:30:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d015aca0 con 0x7f98d00c7020 2023-10-10T20:30:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0014430 con 0x7f98d00c7020 2023-10-10T20:30:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.005+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d0158260 con 0x7f98d00c7020 2023-10-10T20:30:44.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.006+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d015b460 con 0x7f98d00c7020 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.007+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98c0014820 con 0x7f98d00c7020 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.007+0000 7f98c67fc700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.007+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f98c0084cf0 con 0x7f98d00c7020 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.007+0000 7f98d64b4700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.008+0000 7f98d64b4700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00a0b0 tx=0x7f98cc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.012+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98c0089070 con 0x7f98d00c7020 2023-10-10T20:30:44.238 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.235+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f98d00b0b70 con 0x7f98d00c7020 2023-10-10T20:30:44.238 INFO:tasks.ceph.mon.a.smithi148.stderr:2023-10-10T20:30:44.236+0000 7f2ae13e0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-10T20:30:44.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.240+0000 7f98c67fc700 1 -- 172.21.15.148:0/3936199881 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f98c00559f0 con 0x7f98d00c7020 2023-10-10T20:30:44.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 msgr2=0x7f98b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 0x7f98b40693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00a0b0 tx=0x7f98cc016000 comp rx=0 tx=0).stop 2023-10-10T20:30:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 msgr2=0x7f98d00c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f98c0010d30 tx=0x7f98c0008040 comp rx=0 tx=0).stop 2023-10-10T20:30:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 shutdown_connections 2023-10-10T20:30:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f98b4066f30 0x7f98b40693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d00c7490 0x7f98d00ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98bc003a60 0x7f98d00afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 --2- 172.21.15.148:0/3936199881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98d00c7020 0x7f98d00c9d20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 >> 172.21.15.148:0/3936199881 conn(0x7f98d00d0bc0 msgr2=0x7f98d00cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 shutdown_connections 2023-10-10T20:30:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.243+0000 7f98d74b6700 1 -- 172.21.15.148:0/3936199881 wait complete. 2023-10-10T20:30:44.248 INFO:tasks.ceph.mds.b.smithi161.stderr: -7727> 2023-10-10T20:30:10.175+0000 7fb1c683d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-10T20:30:44.248 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-10T20:30:44.257 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-10T20:30:44.277 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-10-10T20:30:44.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.371+0000 7f8f3abbc700 1 Processor -- start 2023-10-10T20:30:44.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.371+0000 7f8f3abbc700 1 -- start start 2023-10-10T20:30:44.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3abbc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f340b6580 0x7f8f3406a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3abbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 0x7f8f340b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3abbc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3406ac90 con 0x7f8f340cce30 2023-10-10T20:30:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3abbc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f340b4d70 con 0x7f8f340b6580 2023-10-10T20:30:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3abbc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3415b3b0 con 0x7f8f3406a830 2023-10-10T20:30:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f393b9700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f340b6580 0x7f8f3406a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f3a3bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 0x7f8f340b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f39bba700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8f340cce30 0x7f8f3415aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:35262/0 (socket says 172.21.15.148:35262) 2023-10-10T20:30:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.372+0000 7f8f39bba700 1 -- 172.21.15.148:0/1602208765 learned_addr learned my addr 172.21.15.148:0/1602208765 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:44.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f38bb8700 1 -- 172.21.15.148:0/1602208765 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50762044 0 0) 0x7f8f30001110 con 0x7f8f340cce30 2023-10-10T20:30:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f38bb8700 1 -- 172.21.15.148:0/1602208765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f20003570 con 0x7f8f340cce30 2023-10-10T20:30:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f393b9700 1 -- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 msgr2=0x7f8f340b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f393b9700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 0x7f8f340b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f393b9700 1 -- 172.21.15.148:0/1602208765 >> v1:172.21.15.161:6789/0 conn(0x7f8f340cce30 legacy=0x7f8f3415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f393b9700 1 -- 172.21.15.148:0/1602208765 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340cd8d0 con 0x7f8f340b6580 2023-10-10T20:30:44.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f3a3bb700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 0x7f8f340b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:44.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f393b9700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f340b6580 0x7f8f3406a2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5acbe6b796023cec server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f38bb8700 1 -- 172.21.15.148:0/1602208765 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24011630 con 0x7f8f340b6580 2023-10-10T20:30:44.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f38bb8700 1 -- 172.21.15.148:0/1602208765 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24011790 con 0x7f8f340b6580 2023-10-10T20:30:44.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.373+0000 7f8f38bb8700 1 -- 172.21.15.148:0/1602208765 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24011b30 con 0x7f8f340b6580 2023-10-10T20:30:44.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 -- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f340b6580 msgr2=0x7f8f3406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f340b6580 0x7f8f3406a2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f240091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 -- 172.21.15.148:0/1602208765 shutdown_connections 2023-10-10T20:30:44.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f3406a830 0x7f8f340b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/1602208765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f340b6580 0x7f8f3406a2f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 -- 172.21.15.148:0/1602208765 >> 172.21.15.148:0/1602208765 conn(0x7f8f34068cb0 msgr2=0x7f8f340690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:44.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 -- 172.21.15.148:0/1602208765 shutdown_connections 2023-10-10T20:30:44.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.374+0000 7f8f3abbc700 1 -- 172.21.15.148:0/1602208765 wait complete. 2023-10-10T20:30:44.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.375+0000 7f8f3abbc700 1 Processor -- start 2023-10-10T20:30:44.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.375+0000 7f8f3abbc700 1 -- start start 2023-10-10T20:30:44.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f3406a830 0x7f8f34142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f340b6580 0x7f8f3415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34155f50 con 0x7f8f340cce30 2023-10-10T20:30:44.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34142c90 con 0x7f8f3406a830 2023-10-10T20:30:44.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34155340 con 0x7f8f340b6580 2023-10-10T20:30:44.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f39bba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f3406a830 0x7f8f34142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f393b9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f340b6580 0x7f8f3415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37208/0 (socket says 172.21.15.148:37208) 2023-10-10T20:30:44.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 -- 172.21.15.148:0/3891871781 learned_addr learned my addr 172.21.15.148:0/3891871781 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:44.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 -- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f340b6580 msgr2=0x7f8f3415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f340b6580 0x7f8f3415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 -- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f3406a830 msgr2=0x7f8f34142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:44.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f3406a830 0x7f8f34142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:44.388 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 -- 172.21.15.148:0/3891871781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c009a20 con 0x7f8f340cce30 2023-10-10T20:30:44.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3a3bb700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fc0 tx=0x7f8f2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00cbd0 con 0x7f8f340cce30 2023-10-10T20:30:44.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c00f040 con 0x7f8f340cce30 2023-10-10T20:30:44.389 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f24009420 con 0x7f8f340cce30 2023-10-10T20:30:44.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.376+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c014480 con 0x7f8f340cce30 2023-10-10T20:30:44.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.377+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f34155900 con 0x7f8f340cce30 2023-10-10T20:30:44.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.377+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f341560c0 con 0x7f8f340cce30 2023-10-10T20:30:44.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.378+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f2c003020 con 0x7f8f340cce30 2023-10-10T20:30:44.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.378+0000 7f8f29ffb700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 0x7f8f180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:44.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.378+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8f2c084fb0 con 0x7f8f340cce30 2023-10-10T20:30:44.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.378+0000 7f8f39bba700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 0x7f8f180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:44.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.379+0000 7f8f39bba700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 0x7f8f180693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007fd0 tx=0x7f8f30007bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:44.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.383+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f2c052130 con 0x7f8f340cce30 2023-10-10T20:30:44.609 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:44.607+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f340b4440 con 0x7f8f340cce30 2023-10-10T20:30:45.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.241+0000 7f8f29ffb700 1 -- 172.21.15.148:0/3891871781 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f2c055750 con 0x7f8f340cce30 2023-10-10T20:30:45.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 msgr2=0x7f8f180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 0x7f8f180693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007fd0 tx=0x7f8f30007bf0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 msgr2=0x7f8f3415ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fc0 tx=0x7f8f2c00b040 comp rx=0 tx=0).stop 2023-10-10T20:30:45.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 shutdown_connections 2023-10-10T20:30:45.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8f18066f30 0x7f8f180693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f340cce30 0x7f8f3415ed20 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f3406a830 0x7f8f34142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 --2- 172.21.15.148:0/3891871781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f340b6580 0x7f8f3415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 >> 172.21.15.148:0/3891871781 conn(0x7f8f34068cb0 msgr2=0x7f8f3415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 shutdown_connections 2023-10-10T20:30:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.243+0000 7f8f3abbc700 1 -- 172.21.15.148:0/3891871781 wait complete. 2023-10-10T20:30:45.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-10T20:30:45.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.375+0000 7fe4c7c28700 1 Processor -- start 2023-10-10T20:30:45.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.375+0000 7fe4c7c28700 1 -- start start 2023-10-10T20:30:45.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c7c28700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c00b28a0 0x7fe4c0158470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c7c28700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b40c0 0x7fe4c01589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c7c28700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00b2410 con 0x7fe4c00b0840 2023-10-10T20:30:45.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c7c28700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c015a1d0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c7c28700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00b1b00 con 0x7fe4c00b40c0 2023-10-10T20:30:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c51c3700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c00b28a0 0x7fe4c0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c59c4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe4c00b0840 0x7fe4c00b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33370/0 (socket says 172.21.15.148:33370) 2023-10-10T20:30:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.376+0000 7fe4c59c4700 1 -- 172.21.15.148:0/2337259309 learned_addr learned my addr 172.21.15.148:0/2337259309 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:45.382 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c61c5700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b40c0 0x7fe4c01589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c49c2700 1 -- 172.21.15.148:0/2337259309 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058963945 0 0) 0x7fe4c00b2410 con 0x7fe4c00b0840 2023-10-10T20:30:45.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c49c2700 1 -- 172.21.15.148:0/2337259309 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ac003570 con 0x7fe4c00b0840 2023-10-10T20:30:45.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c51c3700 1 -- 172.21.15.148:0/2337259309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b40c0 msgr2=0x7fe4c01589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:45.383 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c51c3700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b40c0 0x7fe4c01589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c51c3700 1 -- 172.21.15.148:0/2337259309 >> v1:172.21.15.148:6789/0 conn(0x7fe4c00b0840 legacy=0x7fe4c00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c51c3700 1 -- 172.21.15.148:0/2337259309 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0158ef0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c51c3700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0158470 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00098e0 tx=0x7fe4b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c069a8cdead09 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c49c2700 1 -- 172.21.15.148:0/2337259309 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0011630 con 0x7fe4c00b28a0 2023-10-10T20:30:45.384 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c49c2700 1 -- 172.21.15.148:0/2337259309 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0011790 con 0x7fe4c00b28a0 2023-10-10T20:30:45.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.377+0000 7fe4c49c2700 1 -- 172.21.15.148:0/2337259309 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0011b30 con 0x7fe4c00b28a0 2023-10-10T20:30:45.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 -- 172.21.15.148:0/2337259309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 msgr2=0x7fe4c0158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0158470 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00098e0 tx=0x7fe4b00091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.385 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 -- 172.21.15.148:0/2337259309 shutdown_connections 2023-10-10T20:30:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b40c0 0x7fe4c01589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/2337259309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0158470 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.386 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 -- 172.21.15.148:0/2337259309 >> 172.21.15.148:0/2337259309 conn(0x7fe4c006cf10 msgr2=0x7fe4c006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:45.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 -- 172.21.15.148:0/2337259309 shutdown_connections 2023-10-10T20:30:45.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.378+0000 7fe4c7c28700 1 -- 172.21.15.148:0/2337259309 wait complete. 2023-10-10T20:30:45.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 Processor -- start 2023-10-10T20:30:45.387 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 -- start start 2023-10-10T20:30:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b0840 0x7fe4c0157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.390 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 0x7fe4c0168070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c0169080 con 0x7fe4c00b28a0 2023-10-10T20:30:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c014a220 con 0x7fe4c00b40c0 2023-10-10T20:30:45.391 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.379+0000 7fe4c7c28700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c0157c40 con 0x7fe4c00b0840 2023-10-10T20:30:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37222/0 (socket says 172.21.15.148:37222) 2023-10-10T20:30:45.392 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 -- 172.21.15.148:0/4072071157 learned_addr learned my addr 172.21.15.148:0/4072071157 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c61c5700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 0x7fe4c0168070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c59c4700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b0840 0x7fe4c0157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 -- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b0840 msgr2=0x7fe4c0157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.393 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b0840 0x7fe4c0157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 -- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 msgr2=0x7fe4c0168070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 0x7fe4c0168070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 -- 172.21.15.148:0/4072071157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b0009420 con 0x7fe4c00b28a0 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c61c5700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 0x7fe4c0168070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c51c3700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0013000 tx=0x7fe4b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.394 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0011b10 con 0x7fe4c00b28a0 2023-10-10T20:30:45.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b00253f0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00257b0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c0168ac0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.380+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c00cc920 con 0x7fe4c00b28a0 2023-10-10T20:30:45.395 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.381+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c00cc780 con 0x7fe4c00b28a0 2023-10-10T20:30:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.382+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4b0025ba0 con 0x7fe4c00b28a0 2023-10-10T20:30:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.382+0000 7fe4b5ffb700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.382+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fe4b0085e70 con 0x7fe4c00b28a0 2023-10-10T20:30:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.382+0000 7fe4c59c4700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.382+0000 7fe4c59c4700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 0x7fe4a40693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe4c006d9b0 tx=0x7fe4bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.397 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.386+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4c00cc780 con 0x7fe4c00b28a0 2023-10-10T20:30:45.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.571+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe4c00cc780 con 0x7fe4c00b28a0 2023-10-10T20:30:45.600 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.598+0000 7fe4b5ffb700 1 -- 172.21.15.148:0/4072071157 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v183) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe4c00cc780 con 0x7fe4c00b28a0 2023-10-10T20:30:45.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 msgr2=0x7fe4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 0x7fe4a40693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe4c006d9b0 tx=0x7fe4bc009040 comp rx=0 tx=0).stop 2023-10-10T20:30:45.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 msgr2=0x7fe4c0169240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.603 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0013000 tx=0x7fe4b0017000 comp rx=0 tx=0).stop 2023-10-10T20:30:45.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 shutdown_connections 2023-10-10T20:30:45.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fe4a4066f30 0x7fe4a40693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c00b40c0 0x7fe4c0168070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c00b0840 0x7fe4c0157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.604 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 --2- 172.21.15.148:0/4072071157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c00b28a0 0x7fe4c0169240 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 >> 172.21.15.148:0/4072071157 conn(0x7fe4c006cf10 msgr2=0x7fe4c015c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:45.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 shutdown_connections 2023-10-10T20:30:45.605 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.600+0000 7fe4c7c28700 1 -- 172.21.15.148:0/4072071157 wait complete. 2023-10-10T20:30:45.606 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2023-10-10T20:30:45.614 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-10T20:30:45.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.731+0000 7f7d2cc75700 1 Processor -- start 2023-10-10T20:30:45.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.731+0000 7f7d2cc75700 1 -- start start 2023-10-10T20:30:45.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d28159c10 0x7f7d280cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.736 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d28159fe0 0x7f7d280cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7d280ce800 0x7f7d280cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280d0140 con 0x7f7d28159fe0 2023-10-10T20:30:45.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280cf530 con 0x7f7d280ce800 2023-10-10T20:30:45.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d2cc75700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280cf810 con 0x7f7d28159c10 2023-10-10T20:30:45.737 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d28159fe0 0x7f7d280cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d28159fe0 0x7f7d280cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37230/0 (socket says 172.21.15.148:37230) 2023-10-10T20:30:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 -- 172.21.15.148:0/2060563738 learned_addr learned my addr 172.21.15.148:0/2060563738 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d277fe700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d28159c10 0x7f7d280cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d27fff700 1 --2- 172.21.15.148:0/2060563738 >> v2:172.21.15.161:3300/0 conn(0x7f7d280ce800 0x7f7d280cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 -- 172.21.15.148:0/2060563738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d28159c10 msgr2=0x7f7d280cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d28159c10 0x7f7d280cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 -- 172.21.15.148:0/2060563738 >> v2:172.21.15.161:3300/0 conn(0x7f7d280ce800 msgr2=0x7f7d280cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/2060563738 >> v2:172.21.15.161:3300/0 conn(0x7f7d280ce800 0x7f7d280cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 -- 172.21.15.148:0/2060563738 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280cfaf0 con 0x7f7d28159fe0 2023-10-10T20:30:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159fe0 0x7f7d280cdaf0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7d28063eb0 tx=0x7f7d1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a18e3a8a7b25acf server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.732+0000 7f7d267fc700 1 -- 172.21.15.148:0/2060563738 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00b860 con 0x7f7d28159fe0 2023-10-10T20:30:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d267fc700 1 -- 172.21.15.148:0/2060563738 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00b9c0 con 0x7f7d28159fe0 2023-10-10T20:30:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d267fc700 1 -- 172.21.15.148:0/2060563738 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00bd60 con 0x7f7d28159fe0 2023-10-10T20:30:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 -- 172.21.15.148:0/2060563738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159fe0 msgr2=0x7f7d280cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159fe0 0x7f7d280cdaf0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7d28063eb0 tx=0x7f7d1c009250 comp rx=0 tx=0).stop 2023-10-10T20:30:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 -- 172.21.15.148:0/2060563738 shutdown_connections 2023-10-10T20:30:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/2060563738 >> v2:172.21.15.161:3300/0 conn(0x7f7d280ce800 0x7f7d280cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d28159c10 0x7f7d280cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/2060563738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159fe0 0x7f7d280cdaf0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 -- 172.21.15.148:0/2060563738 >> 172.21.15.148:0/2060563738 conn(0x7f7d280696b0 msgr2=0x7f7d28067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:45.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 -- 172.21.15.148:0/2060563738 shutdown_connections 2023-10-10T20:30:45.742 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.733+0000 7f7d2cc75700 1 -- 172.21.15.148:0/2060563738 wait complete. 2023-10-10T20:30:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.734+0000 7f7d2cc75700 1 Processor -- start 2023-10-10T20:30:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.734+0000 7f7d2cc75700 1 -- start start 2023-10-10T20:30:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 0x7f7d2816af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d28159fe0 0x7f7d2815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2816b4a0 con 0x7f7d28159c10 2023-10-10T20:30:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d28155b40 con 0x7f7d28159fe0 2023-10-10T20:30:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d2cc75700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280cc420 con 0x7f7d280ce800 2023-10-10T20:30:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37232/0 (socket says 172.21.15.148:37232) 2023-10-10T20:30:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 -- 172.21.15.148:0/1762680777 learned_addr learned my addr 172.21.15.148:0/1762680777 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d27fff700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d28159fe0 0x7f7d2815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d277fe700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 0x7f7d2816af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 -- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 msgr2=0x7f7d2816af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 0x7f7d2816af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 -- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d28159fe0 msgr2=0x7f7d2815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d28159fe0 0x7f7d2815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.735+0000 7f7d26ffd700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0094f0 con 0x7f7d28159c10 2023-10-10T20:30:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d277fe700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 0x7f7d2816af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d26ffd700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00b3c0 tx=0x7f7d1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c011ea0 con 0x7f7d28159c10 2023-10-10T20:30:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c01a450 con 0x7f7d28159c10 2023-10-10T20:30:45.749 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d28152af0 con 0x7f7d28159c10 2023-10-10T20:30:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01a810 con 0x7f7d28159c10 2023-10-10T20:30:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.736+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281554f0 con 0x7f7d28159c10 2023-10-10T20:30:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.737+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28155c80 con 0x7f7d28159c10 2023-10-10T20:30:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.737+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d1c01abc0 con 0x7f7d28159c10 2023-10-10T20:30:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.737+0000 7f7d0f7fe700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 0x7f7d140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.738+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f7d1c085ec0 con 0x7f7d28159c10 2023-10-10T20:30:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.738+0000 7f7d277fe700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 0x7f7d140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.738+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f7d1406dcf0 con 0x7f7d28159c10 2023-10-10T20:30:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.738+0000 7f7d277fe700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 0x7f7d140693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7d18006fd0 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:45.752 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.742+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d1c053290 con 0x7f7d28159c10 2023-10-10T20:30:45.926 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:45.924+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7d2804bb80 con 0x7f7d28159c10 2023-10-10T20:30:46.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.667+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 7 ==== osd_map(184..184 src has 1..184) v4 ==== 288+0+0 (secure 0 0 0) 0x7f7d1c0243a0 con 0x7f7d28159c10 2023-10-10T20:30:46.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.667+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7f7d1406ca80 con 0x7f7d28159c10 2023-10-10T20:30:46.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.671+0000 7f7d0f7fe700 1 -- 172.21.15.148:0/1762680777 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v184) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7d1c015030 con 0x7f7d28159c10 2023-10-10T20:30:46.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.674+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 msgr2=0x7f7d140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:46.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.674+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 0x7f7d140693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7d18006fd0 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2023-10-10T20:30:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.674+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 msgr2=0x7f7d281525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.674+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00b3c0 tx=0x7f7d1c017000 comp rx=0 tx=0).stop 2023-10-10T20:30:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 shutdown_connections 2023-10-10T20:30:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7d14066f30 0x7f7d140693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.678 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d28159fe0 0x7f7d2815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d280ce800 0x7f7d2816af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 --2- 172.21.15.148:0/1762680777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d28159c10 0x7f7d281525b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 >> 172.21.15.148:0/1762680777 conn(0x7f7d280696b0 msgr2=0x7f7d28153400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 shutdown_connections 2023-10-10T20:30:46.679 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.675+0000 7f7d2cc75700 1 -- 172.21.15.148:0/1762680777 wait complete. 2023-10-10T20:30:46.680 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2023-10-10T20:30:46.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-10T20:30:46.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.806+0000 7fc72d662700 1 Processor -- start 2023-10-10T20:30:46.809 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.806+0000 7fc72d662700 1 -- start start 2023-10-10T20:30:46.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc728159ec0 0x7fc7280af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72815a290 0x7fc7280afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc72806a5a0 0x7fc72806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.813 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72806af60 con 0x7fc72806a5a0 2023-10-10T20:30:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280b0790 con 0x7fc728159ec0 2023-10-10T20:30:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc72d662700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280ce1f0 con 0x7fc72815a290 2023-10-10T20:30:46.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc72806a5a0 0x7fc72806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc72806a5a0 0x7fc72806aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37240/0 (socket says 172.21.15.148:37240) 2023-10-10T20:30:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 -- 172.21.15.148:0/3501017901 learned_addr learned my addr 172.21.15.148:0/3501017901 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:46.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc726ffd700 1 --2- 172.21.15.148:0/3501017901 >> v2:172.21.15.161:3300/0 conn(0x7fc728159ec0 0x7fc7280af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 -- 172.21.15.148:0/3501017901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72815a290 msgr2=0x7fc7280afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 --2- 172.21.15.148:0/3501017901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72815a290 0x7fc7280afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 -- 172.21.15.148:0/3501017901 >> v2:172.21.15.161:3300/0 conn(0x7fc728159ec0 msgr2=0x7fc7280af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:46.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 --2- 172.21.15.148:0/3501017901 >> v2:172.21.15.161:3300/0 conn(0x7fc728159ec0 0x7fc7280af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.807+0000 7fc7277fe700 1 -- 172.21.15.148:0/3501017901 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280b0b20 con 0x7fc72806a5a0 2023-10-10T20:30:46.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc7277fe700 1 --2- 172.21.15.148:0/3501017901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72806a5a0 0x7fc72806aa20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc7280898e0 tx=0x7fc720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dc28b83c7b9ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:46.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc7267fc700 1 -- 172.21.15.148:0/3501017901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000e070 con 0x7fc72806a5a0 2023-10-10T20:30:46.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc7267fc700 1 -- 172.21.15.148:0/3501017901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000ba50 con 0x7fc72806a5a0 2023-10-10T20:30:46.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc7267fc700 1 -- 172.21.15.148:0/3501017901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200092e0 con 0x7fc72806a5a0 2023-10-10T20:30:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 -- 172.21.15.148:0/3501017901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72806a5a0 msgr2=0x7fc72806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 --2- 172.21.15.148:0/3501017901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72806a5a0 0x7fc72806aa20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc7280898e0 tx=0x7fc720009840 comp rx=0 tx=0).stop 2023-10-10T20:30:46.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 -- 172.21.15.148:0/3501017901 shutdown_connections 2023-10-10T20:30:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 --2- 172.21.15.148:0/3501017901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72806a5a0 0x7fc72806aa20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 --2- 172.21.15.148:0/3501017901 >> v2:172.21.15.161:3300/0 conn(0x7fc728159ec0 0x7fc7280af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 --2- 172.21.15.148:0/3501017901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72815a290 0x7fc7280afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 -- 172.21.15.148:0/3501017901 >> 172.21.15.148:0/3501017901 conn(0x7fc7280d1880 msgr2=0x7fc7280cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:46.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 -- 172.21.15.148:0/3501017901 shutdown_connections 2023-10-10T20:30:46.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.808+0000 7fc72d662700 1 -- 172.21.15.148:0/3501017901 wait complete. 2023-10-10T20:30:46.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 Processor -- start 2023-10-10T20:30:46.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 -- start start 2023-10-10T20:30:46.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72806a5a0 0x7fc728152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.831 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc728159ec0 0x7fc7280b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728152ec0 con 0x7fc72815a290 2023-10-10T20:30:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281547d0 con 0x7fc728159ec0 2023-10-10T20:30:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc72d662700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc728154ab0 con 0x7fc72806a5a0 2023-10-10T20:30:46.832 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc7277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc7277fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37252/0 (socket says 172.21.15.148:37252) 2023-10-10T20:30:46.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.810+0000 7fc7277fe700 1 -- 172.21.15.148:0/1158848161 learned_addr learned my addr 172.21.15.148:0/1158848161 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:46.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc726ffd700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72806a5a0 0x7fc728152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 -- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72806a5a0 msgr2=0x7fc728152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:46.833 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72806a5a0 0x7fc728152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 -- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc728159ec0 msgr2=0x7fc7280b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc71e7fc700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc728159ec0 0x7fc7280b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc728159ec0 0x7fc7280b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 -- 172.21.15.148:0/1158848161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720009ae0 con 0x7fc72815a290 2023-10-10T20:30:46.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc7277fe700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc728071eb0 tx=0x7fc720018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200032f0 con 0x7fc72815a290 2023-10-10T20:30:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720008e80 con 0x7fc72815a290 2023-10-10T20:30:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728154d90 con 0x7fc72815a290 2023-10-10T20:30:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200266b0 con 0x7fc72815a290 2023-10-10T20:30:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.811+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc728155350 con 0x7fc72815a290 2023-10-10T20:30:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.812+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72804c3c0 con 0x7fc72815a290 2023-10-10T20:30:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.812+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc720003040 con 0x7fc72815a290 2023-10-10T20:30:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.813+0000 7fc71f7fe700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.813+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc720086380 con 0x7fc72815a290 2023-10-10T20:30:46.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.813+0000 7fc726ffd700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.813+0000 7fc726ffd700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 0x7fc7080693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7180089b0 tx=0x7fc718008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:46.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.817+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7200538f0 con 0x7fc72815a290 2023-10-10T20:30:47.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:46.998+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc7280b04b0 con 0x7fc72815a290 2023-10-10T20:30:47.003 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.001+0000 7fc71f7fe700 1 -- 172.21.15.148:0/1158848161 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc720056f10 con 0x7fc72815a290 2023-10-10T20:30:47.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.004+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 msgr2=0x7fc7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.007 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.004+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 0x7fc7080693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7180089b0 tx=0x7fc718008040 comp rx=0 tx=0).stop 2023-10-10T20:30:47.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.004+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 msgr2=0x7fc7280b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.004+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc728071eb0 tx=0x7fc720018000 comp rx=0 tx=0).stop 2023-10-10T20:30:47.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 shutdown_connections 2023-10-10T20:30:47.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc708066f30 0x7fc7080693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.008 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc72815a290 0x7fc7280b6670 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc72806a5a0 0x7fc728152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 --2- 172.21.15.148:0/1158848161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc728159ec0 0x7fc7280b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 >> 172.21.15.148:0/1158848161 conn(0x7fc7280d1880 msgr2=0x7fc7280c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:47.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 shutdown_connections 2023-10-10T20:30:47.009 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.005+0000 7fc72d662700 1 -- 172.21.15.148:0/1158848161 wait complete. 2023-10-10T20:30:47.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-10T20:30:47.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.133+0000 7f201f59e700 1 Processor -- start 2023-10-10T20:30:47.138 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.133+0000 7f201f59e700 1 -- start start 2023-10-10T20:30:47.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201f59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20200cd6e0 0x7f20200cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f202015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201f59e700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20200cb7c0 con 0x7f202015dda0 2023-10-10T20:30:47.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201f59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202015e850 con 0x7f20200cd6e0 2023-10-10T20:30:47.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201f59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200cd030 con 0x7f20200cdab0 2023-10-10T20:30:47.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201ed9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f202015dda0 0x7f202015cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33382/0 (socket says 172.21.15.148:33382) 2023-10-10T20:30:47.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201ed9d700 1 -- 172.21.15.148:0/1634969445 learned_addr learned my addr 172.21.15.148:0/1634969445 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:47.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f202015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878150243 0 0) 0x7f20200cb7c0 con 0x7f202015dda0 2023-10-10T20:30:47.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201e59c700 1 --2- 172.21.15.148:0/1634969445 >> v2:172.21.15.161:3300/0 conn(0x7f20200cd6e0 0x7f20200cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.134+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f200c003570 con 0x7f202015dda0 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203800146 0 0) 0x7f200c003570 con 0x7f202015dda0 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 msgr2=0x7f202015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f202015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 >> v2:172.21.15.161:3300/0 conn(0x7f20200cd6e0 msgr2=0x7f20200cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 --2- 172.21.15.148:0/1634969445 >> v2:172.21.15.161:3300/0 conn(0x7f20200cd6e0 0x7f20200cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202015e290 con 0x7f202015dda0 2023-10-10T20:30:47.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f20180017f0 con 0x7f202015dda0 2023-10-10T20:30:47.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 >> v1:172.21.15.148:6789/0 conn(0x7f202015dda0 legacy=0x7f202015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c003a60 0x7f200c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200c004810 0x7f200c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f200c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202015e850 con 0x7f200c004810 2023-10-10T20:30:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200cd030 con 0x7f200c003a60 2023-10-10T20:30:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c00ba60 con 0x7f20200cdab0 2023-10-10T20:30:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200c004810 0x7f200c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201e59c700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c003a60 0x7f200c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201ed9d700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f200c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 -- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 msgr2=0x7f200c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f200c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 -- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c003a60 msgr2=0x7f200c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:47.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c003a60 0x7f200c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 -- 172.21.15.148:0/1634969445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c003570 con 0x7f200c004810 2023-10-10T20:30:47.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201ed9d700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200cdab0 0x7f200c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:47.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200c004810 0x7f200c004c80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f2008009c30 tx=0x7f20080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e509537dfd3361 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:47.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200800e710 con 0x7f200c004810 2023-10-10T20:30:47.151 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200800e870 con 0x7f200c004810 2023-10-10T20:30:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.135+0000 7f201dd9b700 1 -- 172.21.15.148:0/1634969445 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200800ec30 con 0x7f200c004810 2023-10-10T20:30:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 -- 172.21.15.148:0/1634969445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200c004810 msgr2=0x7f200c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 --2- 172.21.15.148:0/1634969445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200c004810 0x7f200c004c80 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f2008009c30 tx=0x7f20080091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201ed9d700 1 -- 172.21.15.148:0/1634969445 reap_dead start 2023-10-10T20:30:47.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 -- 172.21.15.148:0/1634969445 shutdown_connections 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 -- 172.21.15.148:0/1634969445 >> 172.21.15.148:0/1634969445 conn(0x7f2020067660 msgr2=0x7f2020067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 -- 172.21.15.148:0/1634969445 shutdown_connections 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.136+0000 7f201f59e700 1 -- 172.21.15.148:0/1634969445 wait complete. 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.137+0000 7f201f59e700 1 Processor -- start 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.137+0000 7f201f59e700 1 -- start start 2023-10-10T20:30:47.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 0x7f20200c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200c9400 0x7f20200b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202016a460 con 0x7f20200c8f90 2023-10-10T20:30:47.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202016a720 con 0x7f200c004810 2023-10-10T20:30:47.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201f59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202016aa00 con 0x7f20200c9400 2023-10-10T20:30:47.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37270/0 (socket says 172.21.15.148:37270) 2023-10-10T20:30:47.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 -- 172.21.15.148:0/1300339879 learned_addr learned my addr 172.21.15.148:0/1300339879 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:47.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201ed9d700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200c9400 0x7f20200b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.155 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201e59c700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 0x7f20200c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 -- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200c9400 msgr2=0x7f20200b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200c9400 0x7f20200b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 -- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 msgr2=0x7f20200c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 0x7f20200c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 -- 172.21.15.148:0/1300339879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202016ace0 con 0x7f20200c8f90 2023-10-10T20:30:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f201e59c700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 0x7f20200c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.138+0000 7f2015d9b700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f200c004700 tx=0x7f200801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.139+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008022070 con 0x7f20200c8f90 2023-10-10T20:30:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.139+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200800e9e0 con 0x7f20200c8f90 2023-10-10T20:30:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.139+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202016afc0 con 0x7f20200c8f90 2023-10-10T20:30:47.157 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.139+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200801c450 con 0x7f20200c8f90 2023-10-10T20:30:47.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.139+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020158810 con 0x7f20200c8f90 2023-10-10T20:30:47.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.140+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020089280 con 0x7f20200c8f90 2023-10-10T20:30:47.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.140+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f200801c840 con 0x7f20200c8f90 2023-10-10T20:30:47.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.140+0000 7f2016ffd700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 0x7f20000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:47.158 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.140+0000 7f201e59c700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 0x7f20000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:47.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.140+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2008085ac0 con 0x7f20200c8f90 2023-10-10T20:30:47.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.141+0000 7f201e59c700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 0x7f20000693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f20100093f0 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:47.159 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.145+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2008053030 con 0x7f20200c8f90 2023-10-10T20:30:47.404 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.401+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f202015e570 con 0x7f20200c8f90 2023-10-10T20:30:47.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.680+0000 7f2016ffd700 1 -- 172.21.15.148:0/1300339879 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v536) v1 ==== 135+0+0 (secure 0 0 0) 0x7f2008056650 con 0x7f20200c8f90 2023-10-10T20:30:47.684 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 msgr2=0x7f20000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 0x7f20000693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f20100093f0 tx=0x7f2010008040 comp rx=0 tx=0).stop 2023-10-10T20:30:47.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 msgr2=0x7f20200ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:47.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f200c004700 tx=0x7f200801a000 comp rx=0 tx=0).stop 2023-10-10T20:30:47.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 shutdown_connections 2023-10-10T20:30:47.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2000066f30 0x7f20000693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20200c9400 0x7f20200b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f200c004810 0x7f20200c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 --2- 172.21.15.148:0/1300339879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20200c8f90 0x7f20200ca530 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:47.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 >> 172.21.15.148:0/1300339879 conn(0x7f2020067660 msgr2=0x7f20200b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:47.686 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 shutdown_connections 2023-10-10T20:30:47.687 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:47.683+0000 7f201f59e700 1 -- 172.21.15.148:0/1300339879 wait complete. 2023-10-10T20:30:47.696 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-10T20:30:47.697 INFO:tasks.cephfs_test_runner: 2023-10-10T20:30:47.697 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-10T20:30:47.697 INFO:tasks.cephfs_test_runner:Ran 10 tests in 653.595s 2023-10-10T20:30:47.698 INFO:tasks.cephfs_test_runner: 2023-10-10T20:30:47.698 INFO:tasks.cephfs_test_runner:OK 2023-10-10T20:30:47.698 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-10T20:30:47.709 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-10T20:30:47.722 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-10T20:30:47.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:47.723 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:30:47.738 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-10T20:30:47.738 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-10T20:30:47.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:47.739 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-10T20:30:47.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:47.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-10T20:30:47.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:47.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-10-10T20:30:47.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:47.993 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2023-10-10T20:30:48.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.009 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2023-10-10T20:30:48.078 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2023-10-10T20:30:48.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:30:48.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.080 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:30:48.096 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-10T20:30:48.097 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-10T20:30:48.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-10T20:30:48.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.183 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-10T20:30:48.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-10-10T20:30:48.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.310 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2023-10-10T20:30:48.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-10T20:30:48.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2023-10-10T20:30:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2023-10-10T20:30:48.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:30:48.397 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-10T20:30:48.407 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-10T20:30:48.408 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:48.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.521+0000 7fc6f8d3c700 1 Processor -- start 2023-10-10T20:30:48.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.521+0000 7fc6f8d3c700 1 -- start start 2023-10-10T20:30:48.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f8d3c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6f40abae0 0x7fc6f40c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f8d3c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 0x7fc6f40c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f8d3c700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f40c9cb0 con 0x7fc6f40c8400 2023-10-10T20:30:48.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f8d3c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40c90a0 con 0x7fc6f40abae0 2023-10-10T20:30:48.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f8d3c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c9380 con 0x7fc6f40abeb0 2023-10-10T20:30:48.528 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f2d9d700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc6f40c8400 0x7fc6f40c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33388/0 (socket says 172.21.15.148:33388) 2023-10-10T20:30:48.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f2d9d700 1 -- 172.21.15.148:0/555654196 learned_addr learned my addr 172.21.15.148:0/555654196 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:48.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 --2- 172.21.15.148:0/555654196 >> v2:172.21.15.161:3300/0 conn(0x7fc6f40abae0 0x7fc6f40c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6e9d9b700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 0x7fc6f40c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.530 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059695947 0 0) 0x7fc6f40c9cb0 con 0x7fc6f40c8400 2023-10-10T20:30:48.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e0003570 con 0x7fc6f40c8400 2023-10-10T20:30:48.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2725473819 0 0) 0x7fc6e0003570 con 0x7fc6f40c8400 2023-10-10T20:30:48.531 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 -- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 msgr2=0x7fc6f40c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:48.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 0x7fc6f40c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 -- 172.21.15.148:0/555654196 >> v1:172.21.15.148:6789/0 conn(0x7fc6f40c8400 legacy=0x7fc6f40c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 -- 172.21.15.148:0/555654196 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40c9660 con 0x7fc6f40abae0 2023-10-10T20:30:48.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6e9d9b700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 0x7fc6f40c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:48.532 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f259c700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f40c6eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40098e0 tx=0x7fc6e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896aa25cc9ea9cbc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0017f0 con 0x7fc6f40abae0 2023-10-10T20:30:48.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4011420 con 0x7fc6f40abae0 2023-10-10T20:30:48.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.523+0000 7fc6f1d9b700 1 -- 172.21.15.148:0/555654196 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40117c0 con 0x7fc6f40abae0 2023-10-10T20:30:48.533 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 msgr2=0x7fc6f40c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f40c6eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40098e0 tx=0x7fc6e40091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/555654196 shutdown_connections 2023-10-10T20:30:48.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f40c6eb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/555654196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40abeb0 0x7fc6f40c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.534 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/555654196 >> 172.21.15.148:0/555654196 conn(0x7fc6f4067660 msgr2=0x7fc6f4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:48.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/555654196 shutdown_connections 2023-10-10T20:30:48.535 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.525+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/555654196 wait complete. 2023-10-10T20:30:48.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.526+0000 7fc6f8d3c700 1 Processor -- start 2023-10-10T20:30:48.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.526+0000 7fc6f8d3c700 1 -- start start 2023-10-10T20:30:48.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f4168440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.538 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40c8400 0x7fc6f4149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f416bb90 con 0x7fc6f40abeb0 2023-10-10T20:30:48.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4168a40 con 0x7fc6f40abae0 2023-10-10T20:30:48.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4168d20 con 0x7fc6f40c8400 2023-10-10T20:30:48.539 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37274/0 (socket says 172.21.15.148:37274) 2023-10-10T20:30:48.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 -- 172.21.15.148:0/566871723 learned_addr learned my addr 172.21.15.148:0/566871723 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:48.540 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f259c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f4168440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f2d9d700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40c8400 0x7fc6f4149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 -- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40c8400 msgr2=0x7fc6f4149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40c8400 0x7fc6f4149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.541 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 -- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 msgr2=0x7fc6f4168440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f4168440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 -- 172.21.15.148:0/566871723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc009420 con 0x7fc6f40abeb0 2023-10-10T20:30:48.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6e9d9b700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00ec80 tx=0x7fc6dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.542 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00c920 con 0x7fc6f40abeb0 2023-10-10T20:30:48.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc00ca80 con 0x7fc6f40abeb0 2023-10-10T20:30:48.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e4009420 con 0x7fc6f40abeb0 2023-10-10T20:30:48.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00ce00 con 0x7fc6f40abeb0 2023-10-10T20:30:48.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f41692e0 con 0x7fc6f40abeb0 2023-10-10T20:30:48.543 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.527+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f404c3c0 con 0x7fc6f40abeb0 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.528+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6dc013680 con 0x7fc6f40abeb0 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.528+0000 7fc6eaffd700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.528+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc6dc0846b0 con 0x7fc6f40abeb0 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.528+0000 7fc6f259c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.529+0000 7fc6f259c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 0x7fc6d40693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40093f0 tx=0x7fc6e4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.544 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.533+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6dc051b80 con 0x7fc6f40abeb0 2023-10-10T20:30:48.720 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.719+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6f406aeb0 con 0x7fc6f40abeb0 2023-10-10T20:30:48.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.720+0000 7fc6eaffd700 1 -- 172.21.15.148:0/566871723 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+12324 (secure 0 0 0) 0x7fc6dc06be60 con 0x7fc6f40abeb0 2023-10-10T20:30:48.722 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:48.722 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":184,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:46.666878+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6835/1942118864":"2023-10-11T20:30:44.237534+0000","172.21.15.161:6833/1942118864":"2023-10-11T20:30:44.237534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:48.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 msgr2=0x7fc6d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 0x7fc6d40693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40093f0 tx=0x7fc6e4016000 comp rx=0 tx=0).stop 2023-10-10T20:30:48.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 msgr2=0x7fc6f40ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00ec80 tx=0x7fc6dc0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 shutdown_connections 2023-10-10T20:30:48.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f40c8400 0x7fc6f4149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f40abae0 0x7fc6f4168440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 --2- 172.21.15.148:0/566871723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6f40abeb0 0x7fc6f40ca440 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 >> 172.21.15.148:0/566871723 conn(0x7fc6f4067660 msgr2=0x7fc6f40c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:48.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 shutdown_connections 2023-10-10T20:30:48.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.722+0000 7fc6f8d3c700 1 -- 172.21.15.148:0/566871723 wait complete. 2023-10-10T20:30:48.738 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-10T20:30:48.739 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:48.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.852+0000 7fbc1de47700 1 Processor -- start 2023-10-10T20:30:48.854 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.852+0000 7fbc1de47700 1 -- start start 2023-10-10T20:30:48.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc1de47700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc18159cf0 0x7fbc1815e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.856 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc1de47700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc180cf1c0 0x7fbc18068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc1de47700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc180cca20 con 0x7fbc1815a0c0 2023-10-10T20:30:48.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc1de47700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1815ea40 con 0x7fbc18159cf0 2023-10-10T20:30:48.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc1de47700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc18067660 con 0x7fbc180cf1c0 2023-10-10T20:30:48.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc177fe700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc18159cf0 0x7fbc1815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.857 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc16ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbc1815a0c0 0x7fbc180ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33402/0 (socket says 172.21.15.148:33402) 2023-10-10T20:30:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.853+0000 7fbc16ffd700 1 -- 172.21.15.148:0/2587989252 learned_addr learned my addr 172.21.15.148:0/2587989252 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc17fff700 1 --2- 172.21.15.148:0/2587989252 >> v2:172.21.15.161:3300/0 conn(0x7fbc180cf1c0 0x7fbc18068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc167fc700 1 -- 172.21.15.148:0/2587989252 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428800746 0 0) 0x7fbc180cca20 con 0x7fbc1815a0c0 2023-10-10T20:30:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc167fc700 1 -- 172.21.15.148:0/2587989252 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc00003570 con 0x7fbc1815a0c0 2023-10-10T20:30:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc177fe700 1 -- 172.21.15.148:0/2587989252 >> v2:172.21.15.161:3300/0 conn(0x7fbc180cf1c0 msgr2=0x7fbc18068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:48.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc177fe700 1 --2- 172.21.15.148:0/2587989252 >> v2:172.21.15.161:3300/0 conn(0x7fbc180cf1c0 0x7fbc18068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc177fe700 1 -- 172.21.15.148:0/2587989252 >> v1:172.21.15.148:6789/0 conn(0x7fbc1815a0c0 legacy=0x7fbc180ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc177fe700 1 -- 172.21.15.148:0/2587989252 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18067f00 con 0x7fbc18159cf0 2023-10-10T20:30:48.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc177fe700 1 --2- 172.21.15.148:0/2587989252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc18159cf0 0x7fbc1815e500 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbc180cc5d0 tx=0x7fbc08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fe36aff37af5bb server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.859 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc167fc700 1 -- 172.21.15.148:0/2587989252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800e070 con 0x7fbc18159cf0 2023-10-10T20:30:48.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc167fc700 1 -- 172.21.15.148:0/2587989252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0800bb60 con 0x7fbc18159cf0 2023-10-10T20:30:48.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.854+0000 7fbc167fc700 1 -- 172.21.15.148:0/2587989252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012430 con 0x7fbc18159cf0 2023-10-10T20:30:48.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 -- 172.21.15.148:0/2587989252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc18159cf0 msgr2=0x7fbc1815e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2587989252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc18159cf0 0x7fbc1815e500 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbc180cc5d0 tx=0x7fbc08009780 comp rx=0 tx=0).stop 2023-10-10T20:30:48.860 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 -- 172.21.15.148:0/2587989252 shutdown_connections 2023-10-10T20:30:48.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2587989252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc18159cf0 0x7fbc1815e500 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2587989252 >> v2:172.21.15.161:3300/0 conn(0x7fbc180cf1c0 0x7fbc18068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 -- 172.21.15.148:0/2587989252 >> 172.21.15.148:0/2587989252 conn(0x7fbc1806c6e0 msgr2=0x7fbc18069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:48.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 -- 172.21.15.148:0/2587989252 shutdown_connections 2023-10-10T20:30:48.861 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.855+0000 7fbc1de47700 1 -- 172.21.15.148:0/2587989252 wait complete. 2023-10-10T20:30:48.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.856+0000 7fbc1de47700 1 Processor -- start 2023-10-10T20:30:48.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.856+0000 7fbc1de47700 1 -- start start 2023-10-10T20:30:48.866 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 0x7fbc180b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 0x7fbc18158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181440c0 con 0x7fbc180cf1c0 2023-10-10T20:30:48.867 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180b3670 con 0x7fbc18159cf0 2023-10-10T20:30:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180b2a60 con 0x7fbc1815a0c0 2023-10-10T20:30:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37284/0 (socket says 172.21.15.148:37284) 2023-10-10T20:30:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 -- 172.21.15.148:0/2278718756 learned_addr learned my addr 172.21.15.148:0/2278718756 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc16ffd700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 0x7fbc180b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc17fff700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 0x7fbc18158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 -- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 msgr2=0x7fbc18158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 0x7fbc18158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 -- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 msgr2=0x7fbc180b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 0x7fbc180b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 -- 172.21.15.148:0/2278718756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009a20 con 0x7fbc180cf1c0 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc16ffd700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 0x7fbc180b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc17fff700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 0x7fbc18158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc177fe700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012db0 con 0x7fbc180cf1c0 2023-10-10T20:30:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08027470 con 0x7fbc180cf1c0 2023-10-10T20:30:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc180b2d40 con 0x7fbc180cf1c0 2023-10-10T20:30:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08027830 con 0x7fbc180cf1c0 2023-10-10T20:30:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.857+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc180b3300 con 0x7fbc180cf1c0 2023-10-10T20:30:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.858+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1804c3c0 con 0x7fbc180cf1c0 2023-10-10T20:30:48.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.859+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc08024070 con 0x7fbc180cf1c0 2023-10-10T20:30:48.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.859+0000 7fbbff7fe700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 0x7fbc040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:48.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.859+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbc08086ca0 con 0x7fbc180cf1c0 2023-10-10T20:30:48.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.859+0000 7fbc16ffd700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 0x7fbc040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:48.872 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.859+0000 7fbc16ffd700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 0x7fbc040693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbc1816d5e0 tx=0x7fbc0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:48.873 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:48.863+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc08054210 con 0x7fbc180cf1c0 2023-10-10T20:30:49.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.042+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc180ce730 con 0x7fbc04066f30 2023-10-10T20:30:49.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.045+0000 7fbbff7fe700 1 -- 172.21.15.148:0/2278718756 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50513 (secure 0 0 0) 0x7fbc180ce730 con 0x7fbc04066f30 2023-10-10T20:30:49.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:49.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 msgr2=0x7fbc040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 0x7fbc040693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbc1816d5e0 tx=0x7fbc0c009040 comp rx=0 tx=0).stop 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 msgr2=0x7fbc181553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08019000 comp rx=0 tx=0).stop 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 shutdown_connections 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fbc04066f30 0x7fbc040693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc1815a0c0 0x7fbc18158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc180cf1c0 0x7fbc181553b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 --2- 172.21.15.148:0/2278718756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc18159cf0 0x7fbc180b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.046+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 >> 172.21.15.148:0/2278718756 conn(0x7fbc1806c6e0 msgr2=0x7fbc180c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.047+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 shutdown_connections 2023-10-10T20:30:49.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.047+0000 7fbc1de47700 1 -- 172.21.15.148:0/2278718756 wait complete. 2023-10-10T20:30:49.051 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:49.060 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2023-10-10T20:30:47.477574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":58317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1504182,"statfs":{"total":1600000000,"available":1540283364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-44025,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-97,"num_write_kb":-33860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001168"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":307,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-10T20:30:46.335457+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:46.335457+0000","last_peered":"2023-10-10T20:30:46.335457+0000","last_clean":"2023-10-10T20:30:46.335457+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:46.335457+0000","last_undegraded":"2023-10-10T20:30:46.335457+0000","last_fullsized":"2023-10-10T20:30:46.335457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9769,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185543,"statfs":{"total":200000000,"available":189996425,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.27500000000000002},"max":{"1min":0.623,"5min":0.754,"15min":0.754},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.501},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.32200000000000001},"max":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70199999999999996,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"last":0.437},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.54000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.877,"15min":1.3140000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.66700000000000004,"5min":0.92100000000000004,"15min":1.865},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":0.749,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.439}]},{"osd":3,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54800000000000004,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":15.862},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46000000000000002,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.222},"max":{"1min":0.64800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":1.02,"15min":1.0409999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.498},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13282,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182030,"statfs":{"total":200000000,"available":186398837,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":1.647},"last":0.22600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.624,"15min":1.524},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.5,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.254}]},{"osd":3,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":1.673},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.84599999999999997,"15min":1.633},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.506},"last":0.44700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":1.696},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58999999999999997,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.432}]},{"osd":6,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36699999999999999},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":1.7490000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.81299999999999994,"5min":0.98899999999999999,"15min":1.585},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.478},"last":0.373},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.624,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.93500000000000005,"15min":1.722},"last":0.41399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185550,"statfs":{"total":200000000,"available":190003818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.40100000000000002},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.8740000000000001},"last":0.251},{"interface":"front","average":{"1min":0.38,"5min":0.39700000000000002,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48199999999999998,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.375,"15min":0.40300000000000002},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":5.891},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.41999999999999998,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":1.462},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.40999999999999998,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77000000000000002,"5min":0.873,"15min":9.7260000000000009},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39100000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3799999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.4970000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":5.6379999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.69299999999999995,"5min":0.84699999999999998,"15min":5.758},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21479,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173832,"statfs":{"total":200000000,"available":178004892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.36799999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.53800000000000003,"15min":2.4409999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":0.504,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35899999999999999,"15min":0.38500000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42699999999999999,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56299999999999994,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":11.815},"last":0.371},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.45500000000000002},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.52600000000000002,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.28100000000000003},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.93600000000000005,"15min":6.399},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.32400000000000001},"max":{"1min":0.76300000000000001,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.313},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.622,"15min":0.622},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34899999999999998,"15min":0.33100000000000002},"min":{"1min":0.22700000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.44}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.21099999999999999},"max":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.751},"last":0.377}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42799999999999999,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.23300000000000001},"max":{"1min":0.58899999999999997,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.35299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.90100000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.85899999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81599999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":0.623,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.77600000000000002,"15min":0.88700000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.98599999999999999},"last":0.496}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.91200000000000003},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.255},"max":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":2.9159999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.31},"max":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.627},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.69599999999999995,"15min":0.79900000000000004},"last":0.438},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.60299999999999998,"5min":0.80800000000000005,"15min":0.875},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.63600000000000001,"5min":0.72099999999999997,"15min":0.78400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.79400000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.84699999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:49.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.177+0000 7ffbaf9a9700 1 Processor -- start 2023-10-10T20:30:49.179 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 -- start start 2023-10-10T20:30:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffba8067d70 0x7ffba8159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80cb370 con 0x7ffba80cbd40 2023-10-10T20:30:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80b0c60 con 0x7ffba8067d70 2023-10-10T20:30:49.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbaf9a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba806a1e0 con 0x7ffba80b0890 2023-10-10T20:30:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.178+0000 7ffbadf46700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffba8067d70 0x7ffba8159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffba8067d70 0x7ffba8159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37288/0 (socket says 172.21.15.148:37288) 2023-10-10T20:30:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 -- 172.21.15.148:0/1317590893 learned_addr learned my addr 172.21.15.148:0/1317590893 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbad745700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 -- 172.21.15.148:0/1317590893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 msgr2=0x7ffba80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 -- 172.21.15.148:0/1317590893 >> v1:172.21.15.161:6789/0 conn(0x7ffba80cbd40 legacy=0x7ffba8067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 -- 172.21.15.148:0/1317590893 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba806cb80 con 0x7ffba8067d70 2023-10-10T20:30:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffbadf46700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8159790 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ffba00099b0 tx=0x7ffba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d45482aca2a005 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffb9ffff700 1 -- 172.21.15.148:0/1317590893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000b860 con 0x7ffba8067d70 2023-10-10T20:30:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffb9ffff700 1 -- 172.21.15.148:0/1317590893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000b9c0 con 0x7ffba8067d70 2023-10-10T20:30:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.179+0000 7ffb9ffff700 1 -- 172.21.15.148:0/1317590893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000bd60 con 0x7ffba8067d70 2023-10-10T20:30:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/1317590893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 msgr2=0x7ffba8159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8159790 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ffba00099b0 tx=0x7ffba0009250 comp rx=0 tx=0).stop 2023-10-10T20:30:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/1317590893 shutdown_connections 2023-10-10T20:30:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8159790 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/1317590893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/1317590893 >> 172.21.15.148:0/1317590893 conn(0x7ffba806fe50 msgr2=0x7ffba8069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/1317590893 shutdown_connections 2023-10-10T20:30:49.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.180+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/1317590893 wait complete. 2023-10-10T20:30:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.181+0000 7ffbaf9a9700 1 Processor -- start 2023-10-10T20:30:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.181+0000 7ffbaf9a9700 1 -- start start 2023-10-10T20:30:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80ac210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba80cbd40 0x7ffba815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80ad230 con 0x7ffba8067d70 2023-10-10T20:30:49.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b2b90 con 0x7ffba80cbd40 2023-10-10T20:30:49.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80acc70 con 0x7ffba80b0890 2023-10-10T20:30:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37304/0 (socket says 172.21.15.148:37304) 2023-10-10T20:30:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 -- 172.21.15.148:0/3539121861 learned_addr learned my addr 172.21.15.148:0/3539121861 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbadf46700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba80cbd40 0x7ffba815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbacf44700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 -- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 msgr2=0x7ffba80ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 -- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba80cbd40 msgr2=0x7ffba815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba80cbd40 0x7ffba815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 -- 172.21.15.148:0/3539121861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00094f0 con 0x7ffba8067d70 2023-10-10T20:30:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbad745700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ffba400e190 tx=0x7ffba4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400ee30 con 0x7ffba8067d70 2023-10-10T20:30:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4014420 con 0x7ffba8067d70 2023-10-10T20:30:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba80acf50 con 0x7ffba8067d70 2023-10-10T20:30:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.182+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40147e0 con 0x7ffba8067d70 2023-10-10T20:30:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.183+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba8159440 con 0x7ffba8067d70 2023-10-10T20:30:49.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.183+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba80b2680 con 0x7ffba8067d70 2023-10-10T20:30:49.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.184+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffba4014b50 con 0x7ffba8067d70 2023-10-10T20:30:49.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.184+0000 7ffb9d7fa700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.184+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffba4084a00 con 0x7ffba8067d70 2023-10-10T20:30:49.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.184+0000 7ffbacf44700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.185+0000 7ffbacf44700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffba80cc170 tx=0x7ffb98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.188+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffba80b2680 con 0x7ffba8067d70 2023-10-10T20:30:49.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.369+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffba80b2680 con 0x7ffb8c066f30 2023-10-10T20:30:49.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.371+0000 7ffb9d7fa700 1 -- 172.21.15.148:0/3539121861 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50513 (secure 0 0 0) 0x7ffba80b2680 con 0x7ffb8c066f30 2023-10-10T20:30:49.373 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:49.374 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 msgr2=0x7ffb8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffba80cc170 tx=0x7ffb98009040 comp rx=0 tx=0).stop 2023-10-10T20:30:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 msgr2=0x7ffba8158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ffba400e190 tx=0x7ffba4009920 comp rx=0 tx=0).stop 2023-10-10T20:30:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 shutdown_connections 2023-10-10T20:30:49.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7ffb8c066f30 0x7ffb8c0693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba80cbd40 0x7ffba815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffba8067d70 0x7ffba8158f00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 --2- 172.21.15.148:0/3539121861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffba80b0890 0x7ffba80ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 >> 172.21.15.148:0/3539121861 conn(0x7ffba806fe50 msgr2=0x7ffba815bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 shutdown_connections 2023-10-10T20:30:49.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.373+0000 7ffbaf9a9700 1 -- 172.21.15.148:0/3539121861 wait complete. 2023-10-10T20:30:49.378 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:49.386 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2023-10-10T20:30:47.477574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":58317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1504182,"statfs":{"total":1600000000,"available":1540283364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-44025,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-97,"num_write_kb":-33860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001168"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":307,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-10T20:30:46.335457+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:46.335457+0000","last_peered":"2023-10-10T20:30:46.335457+0000","last_clean":"2023-10-10T20:30:46.335457+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:46.335457+0000","last_undegraded":"2023-10-10T20:30:46.335457+0000","last_fullsized":"2023-10-10T20:30:46.335457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9769,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185543,"statfs":{"total":200000000,"available":189996425,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.27500000000000002},"max":{"1min":0.623,"5min":0.754,"15min":0.754},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.501},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.32200000000000001},"max":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70199999999999996,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"last":0.437},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.54000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.877,"15min":1.3140000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.66700000000000004,"5min":0.92100000000000004,"15min":1.865},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":0.749,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.439}]},{"osd":3,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54800000000000004,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":15.862},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46000000000000002,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.222},"max":{"1min":0.64800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":1.02,"15min":1.0409999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.498},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13282,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182030,"statfs":{"total":200000000,"available":186398837,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":1.647},"last":0.22600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.624,"15min":1.524},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.5,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.254}]},{"osd":3,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":1.673},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.84599999999999997,"15min":1.633},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.506},"last":0.44700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":1.696},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58999999999999997,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.432}]},{"osd":6,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36699999999999999},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":1.7490000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.81299999999999994,"5min":0.98899999999999999,"15min":1.585},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.478},"last":0.373},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.624,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.93500000000000005,"15min":1.722},"last":0.41399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185550,"statfs":{"total":200000000,"available":190003818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.40100000000000002},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.8740000000000001},"last":0.251},{"interface":"front","average":{"1min":0.38,"5min":0.39700000000000002,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48199999999999998,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.375,"15min":0.40300000000000002},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":5.891},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.41999999999999998,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":1.462},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.40999999999999998,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77000000000000002,"5min":0.873,"15min":9.7260000000000009},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39100000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3799999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.4970000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":5.6379999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.69299999999999995,"5min":0.84699999999999998,"15min":5.758},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21479,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173832,"statfs":{"total":200000000,"available":178004892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.36799999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.53800000000000003,"15min":2.4409999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":0.504,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35899999999999999,"15min":0.38500000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42699999999999999,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56299999999999994,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":11.815},"last":0.371},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.45500000000000002},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.52600000000000002,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.28100000000000003},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.93600000000000005,"15min":6.399},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.32400000000000001},"max":{"1min":0.76300000000000001,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.313},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.622,"15min":0.622},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34899999999999998,"15min":0.33100000000000002},"min":{"1min":0.22700000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.44}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.21099999999999999},"max":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.751},"last":0.377}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42799999999999999,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.23300000000000001},"max":{"1min":0.58899999999999997,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.35299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.90100000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.85899999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81599999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":0.623,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.77600000000000002,"15min":0.88700000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.98599999999999999},"last":0.496}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.91200000000000003},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.255},"max":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":2.9159999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.31},"max":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.627},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.69599999999999995,"15min":0.79900000000000004},"last":0.438},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.60299999999999998,"5min":0.80800000000000005,"15min":0.875},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.63600000000000001,"5min":0.72099999999999997,"15min":0.78400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.79400000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.84699999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:49.388 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-10T20:30:49.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:49.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.504+0000 7f223feef700 1 Processor -- start 2023-10-10T20:30:49.506 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.504+0000 7f223feef700 1 -- start start 2023-10-10T20:30:49.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223feef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238156ea0 0x7f22380c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223feef700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f223806e2b0 0x7f223806e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.509 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223feef700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22380c42b0 con 0x7f2238157270 2023-10-10T20:30:49.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223feef700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223806fac0 con 0x7f223806e2b0 2023-10-10T20:30:49.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223feef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223806ec70 con 0x7f2238156ea0 2023-10-10T20:30:49.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223e48c700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f223806e2b0 0x7f223806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.510 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223d48a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2238157270 0x7f223806f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33416/0 (socket says 172.21.15.148:33416) 2023-10-10T20:30:49.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.505+0000 7f223d48a700 1 -- 172.21.15.148:0/1308113304 learned_addr learned my addr 172.21.15.148:0/1308113304 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:49.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223dc8b700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238156ea0 0x7f22380c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223e48c700 1 -- 172.21.15.148:0/1308113304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238156ea0 msgr2=0x7f22380c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:49.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223e48c700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238156ea0 0x7f22380c30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.511 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223e48c700 1 -- 172.21.15.148:0/1308113304 >> v1:172.21.15.148:6789/0 conn(0x7f2238157270 legacy=0x7f223806f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223e48c700 1 -- 172.21.15.148:0/1308113304 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380c36a0 con 0x7f223806e2b0 2023-10-10T20:30:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223cc89700 1 -- 172.21.15.148:0/1308113304 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318619943 0 0) 0x7f22380c42b0 con 0x7f2238157270 2023-10-10T20:30:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223e48c700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f223806e2b0 0x7f223806e730 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f22300098e0 tx=0x7f22300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec71105eb412db7b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223cc89700 1 -- 172.21.15.148:0/1308113304 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230009e80 con 0x7f223806e2b0 2023-10-10T20:30:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223cc89700 1 -- 172.21.15.148:0/1308113304 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000b860 con 0x7f223806e2b0 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223cc89700 1 -- 172.21.15.148:0/1308113304 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000bbe0 con 0x7f223806e2b0 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223feef700 1 -- 172.21.15.148:0/1308113304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f223806e2b0 msgr2=0x7f223806e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.506+0000 7f223feef700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f223806e2b0 0x7f223806e730 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f22300098e0 tx=0x7f22300091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 -- 172.21.15.148:0/1308113304 shutdown_connections 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f223806e2b0 0x7f223806e730 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.513 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 --2- 172.21.15.148:0/1308113304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238156ea0 0x7f22380c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 -- 172.21.15.148:0/1308113304 >> 172.21.15.148:0/1308113304 conn(0x7f22380ca4f0 msgr2=0x7f22380ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 -- 172.21.15.148:0/1308113304 shutdown_connections 2023-10-10T20:30:49.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.507+0000 7f223feef700 1 -- 172.21.15.148:0/1308113304 wait complete. 2023-10-10T20:30:49.514 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 Processor -- start 2023-10-10T20:30:49.515 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 -- start start 2023-10-10T20:30:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223806e2b0 0x7f22380c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.517 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238157270 0x7f22380cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2238145ad0 con 0x7f2238156ea0 2023-10-10T20:30:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223816c330 con 0x7f223806e2b0 2023-10-10T20:30:49.518 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.508+0000 7f223feef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2238158560 con 0x7f2238157270 2023-10-10T20:30:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37316/0 (socket says 172.21.15.148:37316) 2023-10-10T20:30:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 -- 172.21.15.148:0/1628872448 learned_addr learned my addr 172.21.15.148:0/1628872448 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223e48c700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238157270 0x7f22380cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223dc8b700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223806e2b0 0x7f22380c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 -- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238157270 msgr2=0x7f22380cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238157270 0x7f22380cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 -- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223806e2b0 msgr2=0x7f22380c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223806e2b0 0x7f22380c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 -- 172.21.15.148:0/1628872448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2230009420 con 0x7f2238156ea0 2023-10-10T20:30:49.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223d48a700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f222800c610 tx=0x7f2228000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228005b30 con 0x7f2238156ea0 2023-10-10T20:30:49.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228005c90 con 0x7f2238156ea0 2023-10-10T20:30:49.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228007300 con 0x7f2238156ea0 2023-10-10T20:30:49.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2238151dc0 con 0x7f2238156ea0 2023-10-10T20:30:49.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.509+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223816c050 con 0x7f2238156ea0 2023-10-10T20:30:49.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.510+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223804c3c0 con 0x7f2238156ea0 2023-10-10T20:30:49.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.511+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22280076b0 con 0x7f2238156ea0 2023-10-10T20:30:49.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.511+0000 7f222dffb700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 0x7f221c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.522 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.511+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2228083760 con 0x7f2238156ea0 2023-10-10T20:30:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.511+0000 7f223dc8b700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 0x7f221c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.511+0000 7f223dc8b700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 0x7f221c0693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f223400d390 tx=0x7f2234007510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.515+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2228050cd0 con 0x7f2238156ea0 2023-10-10T20:30:49.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.698+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22381586a0 con 0x7f221c066f30 2023-10-10T20:30:49.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.700+0000 7f222dffb700 1 -- 172.21.15.148:0/1628872448 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50517 (secure 0 0 0) 0x7f22381586a0 con 0x7f221c066f30 2023-10-10T20:30:49.703 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:49.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.702+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 msgr2=0x7f221c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.704 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.702+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 0x7f221c0693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f223400d390 tx=0x7f2234007510 comp rx=0 tx=0).stop 2023-10-10T20:30:49.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 msgr2=0x7f2238145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f222800c610 tx=0x7f2228000fe0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 shutdown_connections 2023-10-10T20:30:49.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f221c066f30 0x7f221c0693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.705 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2238157270 0x7f22380cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f223806e2b0 0x7f22380c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 --2- 172.21.15.148:0/1628872448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2238156ea0 0x7f2238145590 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 >> 172.21.15.148:0/1628872448 conn(0x7f22380ca4f0 msgr2=0x7f2238152380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 shutdown_connections 2023-10-10T20:30:49.706 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.703+0000 7f223feef700 1 -- 172.21.15.148:0/1628872448 wait complete. 2023-10-10T20:30:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:49.716 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2023-10-10T20:30:49.477883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":58317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1504182,"statfs":{"total":1600000000,"available":1540283364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489239,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-120,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001133"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":307,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-10T20:30:46.335457+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:46.335457+0000","last_peered":"2023-10-10T20:30:46.335457+0000","last_clean":"2023-10-10T20:30:46.335457+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:46.335457+0000","last_undegraded":"2023-10-10T20:30:46.335457+0000","last_fullsized":"2023-10-10T20:30:46.335457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9769,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185543,"statfs":{"total":200000000,"available":189996425,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.27500000000000002},"max":{"1min":0.623,"5min":0.754,"15min":0.754},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.501},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.32200000000000001},"max":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70199999999999996,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"last":0.437},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.54000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.877,"15min":1.3140000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.66700000000000004,"5min":0.92100000000000004,"15min":1.865},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":0.749,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.439}]},{"osd":3,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54800000000000004,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":15.862},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46000000000000002,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.222},"max":{"1min":0.64800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":1.02,"15min":1.0409999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.498},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13282,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182030,"statfs":{"total":200000000,"available":186398837,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":1.647},"last":0.22600000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.624,"15min":1.524},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.5,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.254}]},{"osd":3,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":1.673},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.84599999999999997,"15min":1.633},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.506},"last":0.44700000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":1.696},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58999999999999997,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.432}]},{"osd":6,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36699999999999999},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":1.7490000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.81299999999999994,"5min":0.98899999999999999,"15min":1.585},"last":0.48399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.478},"last":0.373},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.624,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.93500000000000005,"15min":1.722},"last":0.41399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185550,"statfs":{"total":200000000,"available":190003818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.40100000000000002},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.8740000000000001},"last":0.251},{"interface":"front","average":{"1min":0.38,"5min":0.39700000000000002,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48199999999999998,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.375,"15min":0.40300000000000002},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":5.891},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.41999999999999998,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":1.462},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.40999999999999998,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77000000000000002,"5min":0.873,"15min":9.7260000000000009},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39100000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3799999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.4970000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":5.6379999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.69299999999999995,"5min":0.84699999999999998,"15min":5.758},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21479,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173832,"statfs":{"total":200000000,"available":178004892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.36799999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.53800000000000003,"15min":2.4409999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":0.504,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35899999999999999,"15min":0.38500000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42699999999999999,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56299999999999994,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":11.815},"last":0.371},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.45500000000000002},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.52600000000000002,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.28100000000000003},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.93600000000000005,"15min":6.399},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.32400000000000001},"max":{"1min":0.76300000000000001,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.313},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.622,"15min":0.622},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34899999999999998,"15min":0.33100000000000002},"min":{"1min":0.22700000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.44}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.21099999999999999},"max":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.751},"last":0.377}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42799999999999999,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.23300000000000001},"max":{"1min":0.58899999999999997,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.35299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.90100000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.439},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.85899999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81599999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":0.623,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.77600000000000002,"15min":0.88700000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.98599999999999999},"last":0.496}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.91200000000000003},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.255},"max":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.39300000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":2.9159999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.31},"max":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.627},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.69599999999999995,"15min":0.79900000000000004},"last":0.438},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.60299999999999998,"5min":0.80800000000000005,"15min":0.875},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.63600000000000001,"5min":0.72099999999999997,"15min":0.78400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.79400000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.84699999999999998},"last":0.60499999999999998}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:49.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-10T20:30:49.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.832+0000 7f2604330700 1 Processor -- start 2023-10-10T20:30:49.834 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.832+0000 7f2604330700 1 -- start start 2023-10-10T20:30:49.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f2604330700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25fc154470 0x7f25fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.836 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f2604330700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc154840 0x7f25fc0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f2604330700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25fc0ccfa0 con 0x7f25fc0cc230 2023-10-10T20:30:49.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f2604330700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25fc0cd840 con 0x7f25fc154470 2023-10-10T20:30:49.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f2604330700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0ccda0 con 0x7f25fc154840 2023-10-10T20:30:49.837 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f26020cc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25fc154470 0x7f25fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f26020cc700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25fc154470 0x7f25fc0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37318/0 (socket says 172.21.15.148:37318) 2023-10-10T20:30:49.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.833+0000 7f26020cc700 1 -- 172.21.15.148:0/737545531 learned_addr learned my addr 172.21.15.148:0/737545531 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:49.838 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26018cb700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc154840 0x7f25fc0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26020cc700 1 -- 172.21.15.148:0/737545531 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc154840 msgr2=0x7f25fc0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26020cc700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc154840 0x7f25fc0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26020cc700 1 -- 172.21.15.148:0/737545531 >> v1:172.21.15.161:6789/0 conn(0x7f25fc0cc230 legacy=0x7f25fc0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26020cc700 1 -- 172.21.15.148:0/737545531 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0cd280 con 0x7f25fc154470 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26020cc700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154470 0x7f25fc0c8a20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f25f800bc50 tx=0x7f25f800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0090f0d4555a849 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26010ca700 1 -- 172.21.15.148:0/737545531 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800be60 con 0x7f25fc154470 2023-10-10T20:30:49.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26010ca700 1 -- 172.21.15.148:0/737545531 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f800de00 con 0x7f25fc154470 2023-10-10T20:30:49.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f26010ca700 1 -- 172.21.15.148:0/737545531 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f80196b0 con 0x7f25fc154470 2023-10-10T20:30:49.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f2604330700 1 -- 172.21.15.148:0/737545531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154470 msgr2=0x7f25fc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.840 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.834+0000 7f2604330700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154470 0x7f25fc0c8a20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f25f800bc50 tx=0x7f25f800b4f0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 -- 172.21.15.148:0/737545531 shutdown_connections 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154470 0x7f25fc0c8a20 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 --2- 172.21.15.148:0/737545531 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc154840 0x7f25fc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 -- 172.21.15.148:0/737545531 >> 172.21.15.148:0/737545531 conn(0x7f25fc06ad20 msgr2=0x7f25fc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 -- 172.21.15.148:0/737545531 shutdown_connections 2023-10-10T20:30:49.841 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.835+0000 7f2604330700 1 -- 172.21.15.148:0/737545531 wait complete. 2023-10-10T20:30:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 Processor -- start 2023-10-10T20:30:49.844 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 -- start start 2023-10-10T20:30:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc0cc230 0x7f25fc0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25fc154470 0x7f25fc14f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.845 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b2310 con 0x7f25fc154840 2023-10-10T20:30:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b1700 con 0x7f25fc154470 2023-10-10T20:30:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f2604330700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc0b19e0 con 0x7f25fc0cc230 2023-10-10T20:30:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.836+0000 7f26028cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37322/0 (socket says 172.21.15.148:37322) 2023-10-10T20:30:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 -- 172.21.15.148:0/3783635723 learned_addr learned my addr 172.21.15.148:0/3783635723 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26020cc700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc0cc230 0x7f25fc0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26018cb700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25fc154470 0x7f25fc14f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 -- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc0cc230 msgr2=0x7f25fc0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc0cc230 0x7f25fc0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 -- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25fc154470 msgr2=0x7f25fc14f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25fc154470 0x7f25fc14f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 -- 172.21.15.148:0/3783635723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f800b790 con 0x7f25fc154840 2023-10-10T20:30:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f26028cd700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c8e0 tx=0x7f25f4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4012630 con 0x7f25fc154840 2023-10-10T20:30:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4012790 con 0x7f25fc154840 2023-10-10T20:30:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc0b1cc0 con 0x7f25fc154840 2023-10-10T20:30:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4012b50 con 0x7f25fc154840 2023-10-10T20:30:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.837+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc0b13f0 con 0x7f25fc154840 2023-10-10T20:30:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.838+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc04c3c0 con 0x7f25fc154840 2023-10-10T20:30:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.839+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25f401a3a0 con 0x7f25fc154840 2023-10-10T20:30:49.849 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.839+0000 7f25f27fc700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:49.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.839+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f25f4084400 con 0x7f25fc154840 2023-10-10T20:30:49.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.839+0000 7f26020cc700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:49.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.839+0000 7f26020cc700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 0x7f25e00693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f800bc50 tx=0x7f25f801b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:49.850 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:49.843+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25f4051970 con 0x7f25fc154840 2023-10-10T20:30:50.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.024+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25fc0cd560 con 0x7f25fc154840 2023-10-10T20:30:50.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.025+0000 7f25f27fc700 1 -- 172.21.15.148:0/3783635723 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+12324 (secure 0 0 0) 0x7f25f4054f90 con 0x7f25fc154840 2023-10-10T20:30:50.027 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:50.027 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":184,"fsid":"374035e8-0861-4d26-af5d-2eadca412b24","created":"2023-10-10T20:19:05.720110+0000","modified":"2023-10-10T20:30:46.666878+0000","last_up_change":"2023-10-10T20:19:11.195666+0000","last_in_change":"2023-10-10T20:19:08.182843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-10T20:19:11.361845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":86309},{"type":"v1","addr":"172.21.15.148:6801","nonce":86309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":86309},{"type":"v1","addr":"172.21.15.148:6803","nonce":86309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":86309},{"type":"v1","addr":"172.21.15.148:6807","nonce":86309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":86309},{"type":"v1","addr":"172.21.15.148:6805","nonce":86309}]},"public_addr":"172.21.15.148:6801/86309","cluster_addr":"172.21.15.148:6803/86309","heartbeat_back_addr":"172.21.15.148:6807/86309","heartbeat_front_addr":"172.21.15.148:6805/86309","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":86315},{"type":"v1","addr":"172.21.15.148:6817","nonce":86315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":86315},{"type":"v1","addr":"172.21.15.148:6819","nonce":86315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":86315},{"type":"v1","addr":"172.21.15.148:6823","nonce":86315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":86315},{"type":"v1","addr":"172.21.15.148:6821","nonce":86315}]},"public_addr":"172.21.15.148:6817/86315","cluster_addr":"172.21.15.148:6819/86315","heartbeat_back_addr":"172.21.15.148:6823/86315","heartbeat_front_addr":"172.21.15.148:6821/86315","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":86312},{"type":"v1","addr":"172.21.15.148:6809","nonce":86312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":86312},{"type":"v1","addr":"172.21.15.148:6811","nonce":86312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":86312},{"type":"v1","addr":"172.21.15.148:6815","nonce":86312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":86312},{"type":"v1","addr":"172.21.15.148:6813","nonce":86312}]},"public_addr":"172.21.15.148:6809/86312","cluster_addr":"172.21.15.148:6811/86312","heartbeat_back_addr":"172.21.15.148:6815/86312","heartbeat_front_addr":"172.21.15.148:6813/86312","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":86339},{"type":"v1","addr":"172.21.15.148:6825","nonce":86339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":86339},{"type":"v1","addr":"172.21.15.148:6827","nonce":86339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":86339},{"type":"v1","addr":"172.21.15.148:6831","nonce":86339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":86339},{"type":"v1","addr":"172.21.15.148:6829","nonce":86339}]},"public_addr":"172.21.15.148:6825/86339","cluster_addr":"172.21.15.148:6827/86339","heartbeat_back_addr":"172.21.15.148:6831/86339","heartbeat_front_addr":"172.21.15.148:6829/86339","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":86554},{"type":"v1","addr":"172.21.15.161:6806","nonce":86554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":86554},{"type":"v1","addr":"172.21.15.161:6814","nonce":86554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6829","nonce":86554},{"type":"v1","addr":"172.21.15.161:6831","nonce":86554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":86554},{"type":"v1","addr":"172.21.15.161:6822","nonce":86554}]},"public_addr":"172.21.15.161:6806/86554","cluster_addr":"172.21.15.161:6814/86554","heartbeat_back_addr":"172.21.15.161:6831/86554","heartbeat_front_addr":"172.21.15.161:6822/86554","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":86555},{"type":"v1","addr":"172.21.15.161:6804","nonce":86555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":86555},{"type":"v1","addr":"172.21.15.161:6812","nonce":86555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":86555},{"type":"v1","addr":"172.21.15.161:6826","nonce":86555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":86555},{"type":"v1","addr":"172.21.15.161:6820","nonce":86555}]},"public_addr":"172.21.15.161:6804/86555","cluster_addr":"172.21.15.161:6812/86555","heartbeat_back_addr":"172.21.15.161:6826/86555","heartbeat_front_addr":"172.21.15.161:6820/86555","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":86553},{"type":"v1","addr":"172.21.15.161:6809","nonce":86553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":86553},{"type":"v1","addr":"172.21.15.161:6817","nonce":86553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":86553},{"type":"v1","addr":"172.21.15.161:6830","nonce":86553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6821","nonce":86553},{"type":"v1","addr":"172.21.15.161:6825","nonce":86553}]},"public_addr":"172.21.15.161:6809/86553","cluster_addr":"172.21.15.161:6817/86553","heartbeat_back_addr":"172.21.15.161:6830/86553","heartbeat_front_addr":"172.21.15.161:6825/86553","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":86574},{"type":"v1","addr":"172.21.15.161:6803","nonce":86574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":86574},{"type":"v1","addr":"172.21.15.161:6811","nonce":86574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6823","nonce":86574},{"type":"v1","addr":"172.21.15.161:6827","nonce":86574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":86574},{"type":"v1","addr":"172.21.15.161:6819","nonce":86574}]},"public_addr":"172.21.15.161:6803/86574","cluster_addr":"172.21.15.161:6811/86574","heartbeat_back_addr":"172.21.15.161:6827/86574","heartbeat_front_addr":"172.21.15.161:6819/86574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.316456+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.287039+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.282821+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.385603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.354337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.363589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.376276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-10T20:19:09.321072+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6835/1942118864":"2023-10-11T20:30:44.237534+0000","172.21.15.161:6833/1942118864":"2023-10-11T20:30:44.237534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-10T20:30:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 msgr2=0x7f25e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 0x7f25e00693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f800bc50 tx=0x7f25f801b000 comp rx=0 tx=0).stop 2023-10-10T20:30:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 msgr2=0x7f25fc150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c8e0 tx=0x7f25f4001710 comp rx=0 tx=0).stop 2023-10-10T20:30:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 shutdown_connections 2023-10-10T20:30:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f25e0066f30 0x7f25e00693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc154840 0x7f25fc150e90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:50.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25fc0cc230 0x7f25fc0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:50.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 --2- 172.21.15.148:0/3783635723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25fc154470 0x7f25fc14f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:50.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 >> 172.21.15.148:0/3783635723 conn(0x7f25fc06ad20 msgr2=0x7f25fc1505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:50.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 shutdown_connections 2023-10-10T20:30:50.032 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:50.027+0000 7f2604330700 1 -- 172.21.15.148:0/3783635723 wait complete. 2023-10-10T20:30:51.042 INFO:tasks.ceph:Scrubbing osd.0 2023-10-10T20:30:51.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:51.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.169+0000 7fab0817f700 1 Processor -- start 2023-10-10T20:30:51.171 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.170+0000 7fab0817f700 1 -- start start 2023-10-10T20:30:51.180 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.170+0000 7fab0817f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 0x7fab000adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.170+0000 7fab0817f700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab000ae810 con 0x7fab000ca880 2023-10-10T20:30:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.170+0000 7fab0817f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab000aeaf0 con 0x7fab000abf00 2023-10-10T20:30:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.170+0000 7fab0817f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab000aedd0 con 0x7fab00157b20 2023-10-10T20:30:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab0571a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fab000ca880 0x7fab000ab7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33424/0 (socket says 172.21.15.148:33424) 2023-10-10T20:30:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab0571a700 1 -- 172.21.15.148:0/2150223301 learned_addr learned my addr 172.21.15.148:0/2150223301 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:51.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab05f1b700 1 --2- 172.21.15.148:0/2150223301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 0x7fab000adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528796585 0 0) 0x7fab000ae810 con 0x7fab000ca880 2023-10-10T20:30:51.183 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae8003570 con 0x7fab000ca880 2023-10-10T20:30:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690754398 0 0) 0x7fab000aeaf0 con 0x7fab000abf00 2023-10-10T20:30:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab000ae810 con 0x7fab000abf00 2023-10-10T20:30:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab05f1b700 1 -- 172.21.15.148:0/2150223301 >> v1:172.21.15.161:6789/0 conn(0x7fab000abf00 legacy=0x7fab0006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 110309054 0 0) 0x7faae8003570 con 0x7fab000ca880 2023-10-10T20:30:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab05f1b700 1 -- 172.21.15.148:0/2150223301 >> v1:172.21.15.148:6789/0 conn(0x7fab000ca880 legacy=0x7fab000ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab05f1b700 1 -- 172.21.15.148:0/2150223301 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000af0b0 con 0x7fab00157b20 2023-10-10T20:30:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab05f1b700 1 --2- 172.21.15.148:0/2150223301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 0x7fab000adcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ec0 tx=0x7faaf0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30cf1a5e169976e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000e070 con 0x7fab00157b20 2023-10-10T20:30:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000b870 con 0x7fab00157b20 2023-10-10T20:30:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.171+0000 7fab04f19700 1 -- 172.21.15.148:0/2150223301 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4001470 con 0x7fab00157b20 2023-10-10T20:30:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 -- 172.21.15.148:0/2150223301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 msgr2=0x7fab000adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 --2- 172.21.15.148:0/2150223301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 0x7fab000adcc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ec0 tx=0x7faaf0009780 comp rx=0 tx=0).stop 2023-10-10T20:30:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 -- 172.21.15.148:0/2150223301 shutdown_connections 2023-10-10T20:30:51.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 --2- 172.21.15.148:0/2150223301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab00157b20 0x7fab000adcc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 -- 172.21.15.148:0/2150223301 >> 172.21.15.148:0/2150223301 conn(0x7fab000cd450 msgr2=0x7fab000c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 -- 172.21.15.148:0/2150223301 shutdown_connections 2023-10-10T20:30:51.188 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.172+0000 7fab0817f700 1 -- 172.21.15.148:0/2150223301 wait complete. 2023-10-10T20:30:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.173+0000 7fab0817f700 1 Processor -- start 2023-10-10T20:30:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.173+0000 7fab0817f700 1 -- start start 2023-10-10T20:30:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab000abf00 0x7fab0016c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab000ca880 0x7fab00149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.189 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 0x7fab00151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001585e0 con 0x7fab00157b20 2023-10-10T20:30:51.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0016c940 con 0x7fab000abf00 2023-10-10T20:30:51.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0817f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0014a330 con 0x7fab000ca880 2023-10-10T20:30:51.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 0x7fab00151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.190 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 0x7fab00151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37334/0 (socket says 172.21.15.148:37334) 2023-10-10T20:30:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 -- 172.21.15.148:0/1357558134 learned_addr learned my addr 172.21.15.148:0/1357558134 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0571a700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab000ca880 0x7fab00149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab05f1b700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab000abf00 0x7fab0016c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 -- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab000ca880 msgr2=0x7fab00149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.191 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab000ca880 0x7fab00149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 -- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab000abf00 msgr2=0x7fab0016c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab000abf00 0x7fab0016c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0671c700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf0009a20 con 0x7fab00157b20 2023-10-10T20:30:51.192 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.174+0000 7fab0571a700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab000ca880 0x7fab00149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7fab0671c700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 0x7fab00151170 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c4b0 tx=0x7faaf8007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011590 con 0x7fab00157b20 2023-10-10T20:30:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf80116f0 con 0x7fab00157b20 2023-10-10T20:30:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab000b16a0 con 0x7fab00157b20 2023-10-10T20:30:51.193 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011a90 con 0x7fab00157b20 2023-10-10T20:30:51.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.175+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab00158270 con 0x7fab00157b20 2023-10-10T20:30:51.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.176+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab0006f4f0 con 0x7fab00157b20 2023-10-10T20:30:51.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.176+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faaf801c460 con 0x7fab00157b20 2023-10-10T20:30:51.194 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.176+0000 7faafdffb700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaec066f30 0x7faaec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faaf8083ec0 con 0x7fab00157b20 2023-10-10T20:30:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7fab05f1b700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaec066f30 0x7faaec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7faafdffb700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7faaec06cd80 0x7faaec06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faaec06f860 con 0x7faaec06cd80 2023-10-10T20:30:51.195 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaf8086500 con 0x7fab00157b20 2023-10-10T20:30:51.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7fab0571a700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7faaec06cd80 0x7faaec06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7fab05f1b700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaec066f30 0x7faaec0693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009e90 tx=0x7faaf0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.196 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.177+0000 7fab0571a700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7faaec06cd80 0x7faaec06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.197 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.178+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== osd.0 v2:172.21.15.148:6800/86309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7faaec06f860 con 0x7faaec06cd80 2023-10-10T20:30:51.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.190+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 --> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab0004c3c0 con 0x7faaec06cd80 2023-10-10T20:30:51.198 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7faafdffb700 1 -- 172.21.15.148:0/1357558134 <== osd.0 v2:172.21.15.148:6800/86309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fab0004c3c0 con 0x7faaec06cd80 2023-10-10T20:30:51.199 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:51.199 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:51.200 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:51.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7faaec06cd80 msgr2=0x7faaec06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6800/86309,v1:172.21.15.148:6801/86309] conn(0x7faaec06cd80 0x7faaec06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.202 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaec066f30 msgr2=0x7faaec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7faaec066f30 0x7faaec0693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009e90 tx=0x7faaf0017000 comp rx=0 tx=0).stop 2023-10-10T20:30:51.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 msgr2=0x7fab00151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.203 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 --2- 172.21.15.148:0/1357558134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab00157b20 0x7fab00151170 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c4b0 tx=0x7faaf8007d50 comp rx=0 tx=0).stop 2023-10-10T20:30:51.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0671c700 1 -- 172.21.15.148:0/1357558134 reap_dead start 2023-10-10T20:30:51.204 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 shutdown_connections 2023-10-10T20:30:51.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 >> 172.21.15.148:0/1357558134 conn(0x7fab000cd450 msgr2=0x7fab0006d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 shutdown_connections 2023-10-10T20:30:51.205 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.191+0000 7fab0817f700 1 -- 172.21.15.148:0/1357558134 wait complete. 2023-10-10T20:30:51.207 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-10T20:30:51.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.321+0000 7fb0e4669700 1 Processor -- start 2023-10-10T20:30:51.323 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.321+0000 7fb0e4669700 1 -- start start 2023-10-10T20:30:51.327 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.322+0000 7fb0e4669700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0dc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.322+0000 7fb0e4669700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc15de60 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.322+0000 7fb0e4669700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc15e140 con 0x7fb0dc0b1220 2023-10-10T20:30:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.322+0000 7fb0e4669700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc15e420 con 0x7fb0dc0b0e50 2023-10-10T20:30:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e2c06700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb0dc0cb4f0 0x7fb0dc0679d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33426/0 (socket says 172.21.15.148:33426) 2023-10-10T20:30:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e2c06700 1 -- 172.21.15.148:0/4246369885 learned_addr learned my addr 172.21.15.148:0/4246369885 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e2405700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0dc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197329985 0 0) 0x7fb0dc15de60 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0c4003570 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110532730 0 0) 0x7fb0c4003570 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 msgr2=0x7fb0dc0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0dc0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 >> v1:172.21.15.161:6789/0 conn(0x7fb0dc0b1220 legacy=0x7fb0dc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc15e700 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fb0d4001470 con 0x7fb0dc0cb4f0 2023-10-10T20:30:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 >> v1:172.21.15.148:6789/0 conn(0x7fb0dc0cb4f0 legacy=0x7fb0dc0679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c4003a90 0x7fb0c4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4004810 0x7fb0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc15e420 con 0x7fb0c4003a90 2023-10-10T20:30:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4005520 con 0x7fb0c4004810 2023-10-10T20:30:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.323+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4005690 con 0x7fb0dc0b0e50 2023-10-10T20:30:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c4003a90 0x7fb0c4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e1c04700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4004810 0x7fb0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.332 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2c06700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 -- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 msgr2=0x7fb0c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:51.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0dc0b0e50 0x7fb0c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 -- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4004810 msgr2=0x7fb0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:51.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4004810 0x7fb0c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.333 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 -- 172.21.15.148:0/4246369885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4003570 con 0x7fb0c4003a90 2023-10-10T20:30:51.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e2405700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c4003a90 0x7fb0c4003ea0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00e210 tx=0x7fb0cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8926eea492f294 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00eac0 con 0x7fb0c4003a90 2023-10-10T20:30:51.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc00f040 con 0x7fb0c4003a90 2023-10-10T20:30:51.334 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.324+0000 7fb0e1403700 1 -- 172.21.15.148:0/4246369885 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc00ee70 con 0x7fb0c4003a90 2023-10-10T20:30:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 -- 172.21.15.148:0/4246369885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c4003a90 msgr2=0x7fb0c4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 --2- 172.21.15.148:0/4246369885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c4003a90 0x7fb0c4003ea0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00e210 tx=0x7fb0cc00b040 comp rx=0 tx=0).stop 2023-10-10T20:30:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e2c06700 1 -- 172.21.15.148:0/4246369885 reap_dead start 2023-10-10T20:30:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 -- 172.21.15.148:0/4246369885 shutdown_connections 2023-10-10T20:30:51.335 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 -- 172.21.15.148:0/4246369885 >> 172.21.15.148:0/4246369885 conn(0x7fb0dc069870 msgr2=0x7fb0dc069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 -- 172.21.15.148:0/4246369885 shutdown_connections 2023-10-10T20:30:51.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.325+0000 7fb0e4669700 1 -- 172.21.15.148:0/4246369885 wait complete. 2023-10-10T20:30:51.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.326+0000 7fb0e4669700 1 Processor -- start 2023-10-10T20:30:51.336 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.326+0000 7fb0e4669700 1 -- start start 2023-10-10T20:30:51.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c4003a90 0x7fb0dc14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.339 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 0x7fb0dc0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b4760 con 0x7fb0dc14fc00 2023-10-10T20:30:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b18e0 con 0x7fb0dc0b3db0 2023-10-10T20:30:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0b1bc0 con 0x7fb0c4003a90 2023-10-10T20:30:51.340 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37342/0 (socket says 172.21.15.148:37342) 2023-10-10T20:30:51.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 -- 172.21.15.148:0/3968324787 learned_addr learned my addr 172.21.15.148:0/3968324787 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:51.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e1c04700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 0x7fb0dc0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2405700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c4003a90 0x7fb0dc14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.341 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 -- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c4003a90 msgr2=0x7fb0dc14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c4003a90 0x7fb0dc14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 -- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 msgr2=0x7fb0dc0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 0x7fb0dc0b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 -- 172.21.15.148:0/3968324787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc1526a0 con 0x7fb0dc14fc00 2023-10-10T20:30:51.342 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e1c04700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 0x7fb0dc0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e2c06700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc15e9e0 tx=0x7fb0d4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4011590 con 0x7fb0dc14fc00 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d40116f0 con 0x7fb0dc14fc00 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc150970 con 0x7fb0dc14fc00 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4011ab0 con 0x7fb0dc14fc00 2023-10-10T20:30:51.343 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.327+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc150f30 con 0x7fb0dc14fc00 2023-10-10T20:30:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.328+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc0b1d50 con 0x7fb0dc14fc00 2023-10-10T20:30:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.329+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0d401d3b0 con 0x7fb0dc14fc00 2023-10-10T20:30:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.329+0000 7fb0da7fc700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 0x7fb0c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.329+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0d4083d90 con 0x7fb0dc14fc00 2023-10-10T20:30:51.344 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.329+0000 7fb0e2405700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 0x7fb0c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.329+0000 7fb0e2405700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 0x7fb0c80693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00afd0 tx=0x7fb0cc01a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.345 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.333+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0d4051300 con 0x7fb0dc14fc00 2023-10-10T20:30:51.520 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.518+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0dc04c3c0 con 0x7fb0c8066f30 2023-10-10T20:30:51.521 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.519+0000 7fb0da7fc700 1 -- 172.21.15.148:0/3968324787 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb0dc04c3c0 con 0x7fb0c8066f30 2023-10-10T20:30:51.523 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 msgr2=0x7fb0c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 0x7fb0c80693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00afd0 tx=0x7fb0cc01a000 comp rx=0 tx=0).stop 2023-10-10T20:30:51.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 msgr2=0x7fb0dc1500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.524 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc15e9e0 tx=0x7fb0d4001d80 comp rx=0 tx=0).stop 2023-10-10T20:30:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 shutdown_connections 2023-10-10T20:30:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fb0c8066f30 0x7fb0c80693e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.525 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0dc14fc00 0x7fb0dc1500a0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c4003a90 0x7fb0dc14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 --2- 172.21.15.148:0/3968324787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0dc0b3db0 0x7fb0dc0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.526 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 >> 172.21.15.148:0/3968324787 conn(0x7fb0dc069870 msgr2=0x7fb0dc0c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 shutdown_connections 2023-10-10T20:30:51.527 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.521+0000 7fb0e4669700 1 -- 172.21.15.148:0/3968324787 wait complete. 2023-10-10T20:30:51.527 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 0 to deep-scrub 2023-10-10T20:30:51.534 INFO:tasks.ceph:Scrubbing osd.1 2023-10-10T20:30:51.534 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.652+0000 7f17ba428700 1 Processor -- start 2023-10-10T20:30:51.654 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.652+0000 7f17ba428700 1 -- start start 2023-10-10T20:30:51.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17ba428700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17b40cd7f0 0x7f17b415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17ba428700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 0x7f17b415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.657 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17ba428700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b415a180 con 0x7f17b41595f0 2023-10-10T20:30:51.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17ba428700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b415e890 con 0x7f17b40cd7f0 2023-10-10T20:30:51.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17ba428700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b415e340 con 0x7f17b40cdbc0 2023-10-10T20:30:51.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17b40cd7f0 0x7f17b415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.658 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f17b40cd7f0 0x7f17b415d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37358/0 (socket says 172.21.15.148:37358) 2023-10-10T20:30:51.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 -- 172.21.15.148:0/3473340901 learned_addr learned my addr 172.21.15.148:0/3473340901 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:51.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b8c25700 1 --2- 172.21.15.148:0/3473340901 >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 0x7f17b415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 -- 172.21.15.148:0/3473340901 >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 msgr2=0x7f17b415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:51.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 --2- 172.21.15.148:0/3473340901 >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 0x7f17b415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.659 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 -- 172.21.15.148:0/3473340901 >> v1:172.21.15.161:6789/0 conn(0x7f17b41595f0 legacy=0x7f17b4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 -- 172.21.15.148:0/3473340901 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b415e5b0 con 0x7f17b40cd7f0 2023-10-10T20:30:51.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b8c25700 1 --2- 172.21.15.148:0/3473340901 >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 0x7f17b415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:51.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b9426700 1 --2- 172.21.15.148:0/3473340901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cd7f0 0x7f17b415d8c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f17b40cb8c0 tx=0x7f17ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0694ab7bfae3146 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.660 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.653+0000 7f17b3fff700 1 -- 172.21.15.148:0/3473340901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00e070 con 0x7f17b40cd7f0 2023-10-10T20:30:51.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17b3fff700 1 -- 172.21.15.148:0/3473340901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac00ba50 con 0x7f17b40cd7f0 2023-10-10T20:30:51.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17b3fff700 1 -- 172.21.15.148:0/3473340901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0092e0 con 0x7f17b40cd7f0 2023-10-10T20:30:51.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 -- 172.21.15.148:0/3473340901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cd7f0 msgr2=0x7f17b415d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 --2- 172.21.15.148:0/3473340901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cd7f0 0x7f17b415d8c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f17b40cb8c0 tx=0x7f17ac009840 comp rx=0 tx=0).stop 2023-10-10T20:30:51.661 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 -- 172.21.15.148:0/3473340901 shutdown_connections 2023-10-10T20:30:51.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 --2- 172.21.15.148:0/3473340901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cd7f0 0x7f17b415d8c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 --2- 172.21.15.148:0/3473340901 >> v2:172.21.15.161:3300/0 conn(0x7f17b40cdbc0 0x7f17b415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 -- 172.21.15.148:0/3473340901 >> 172.21.15.148:0/3473340901 conn(0x7f17b4068bf0 msgr2=0x7f17b4068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 -- 172.21.15.148:0/3473340901 shutdown_connections 2023-10-10T20:30:51.662 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.654+0000 7f17ba428700 1 -- 172.21.15.148:0/3473340901 wait complete. 2023-10-10T20:30:51.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.655+0000 7f17ba428700 1 Processor -- start 2023-10-10T20:30:51.663 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.655+0000 7f17ba428700 1 -- start start 2023-10-10T20:30:51.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17b40cd7f0 0x7f17b416d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 0x7f17b416b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.666 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17b41595f0 0x7f17b416b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40b4470 con 0x7f17b40cdbc0 2023-10-10T20:30:51.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40b3b40 con 0x7f17b41595f0 2023-10-10T20:30:51.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17ba428700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b40b3e20 con 0x7f17b40cd7f0 2023-10-10T20:30:51.667 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17b8c25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 0x7f17b416b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17b8c25700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 0x7f17b416b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37374/0 (socket says 172.21.15.148:37374) 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17b8c25700 1 -- 172.21.15.148:0/701101965 learned_addr learned my addr 172.21.15.148:0/701101965 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.656+0000 7f17b9426700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17b40cd7f0 0x7f17b416d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b9c27700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17b41595f0 0x7f17b416b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 -- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17b40cd7f0 msgr2=0x7f17b416d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.668 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17b40cd7f0 0x7f17b416d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 -- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17b41595f0 msgr2=0x7f17b416b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17b41595f0 0x7f17b416b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac009ae0 con 0x7f17b40cdbc0 2023-10-10T20:30:51.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b9426700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17b40cd7f0 0x7f17b416d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:51.669 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b8c25700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 0x7f17b416b8f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e8d0 tx=0x7f17a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800ee70 con 0x7f17b40cdbc0 2023-10-10T20:30:51.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a800c670 con 0x7f17b40cdbc0 2023-10-10T20:30:51.670 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b40b4100 con 0x7f17b40cdbc0 2023-10-10T20:30:51.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800ca10 con 0x7f17b40cdbc0 2023-10-10T20:30:51.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.657+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b40ca610 con 0x7f17b40cdbc0 2023-10-10T20:30:51.671 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.658+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17b40ca8f0 con 0x7f17b40cdbc0 2023-10-10T20:30:51.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.658+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17a8013410 con 0x7f17b40cdbc0 2023-10-10T20:30:51.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b17fa700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f17a0066f30 0x7f17a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.672 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f17a8084100 con 0x7f17b40cdbc0 2023-10-10T20:30:51.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b9426700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f17a0066f30 0x7f17a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b17fa700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7f17a006cd80 0x7f17a006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.673 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17a006f860 con 0x7f17a006cd80 2023-10-10T20:30:51.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a8086740 con 0x7f17b40cdbc0 2023-10-10T20:30:51.674 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b9c27700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7f17a006cd80 0x7f17a006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b9426700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f17a0066f30 0x7f17a00693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009880 tx=0x7f17ac018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.659+0000 7f17b9c27700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7f17a006cd80 0x7f17a006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.675 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.660+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== osd.1 v2:172.21.15.148:6816/86315 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f17a006f860 con 0x7f17a006cd80 2023-10-10T20:30:51.676 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.672+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 --> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f17b40b4290 con 0x7f17a006cd80 2023-10-10T20:30:51.677 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.672+0000 7f17b17fa700 1 -- 172.21.15.148:0/701101965 <== osd.1 v2:172.21.15.148:6816/86315 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f17b40b4290 con 0x7f17a006cd80 2023-10-10T20:30:51.677 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:51.678 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:51.680 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7f17a006cd80 msgr2=0x7f17a006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6816/86315,v1:172.21.15.148:6817/86315] conn(0x7f17a006cd80 0x7f17a006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.681 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f17a0066f30 msgr2=0x7f17a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f17a0066f30 0x7f17a00693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009880 tx=0x7f17ac018000 comp rx=0 tx=0).stop 2023-10-10T20:30:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 msgr2=0x7f17b416b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 --2- 172.21.15.148:0/701101965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17b40cdbc0 0x7f17b416b8f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f17a800e8d0 tx=0x7f17a8008040 comp rx=0 tx=0).stop 2023-10-10T20:30:51.682 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17b9c27700 1 -- 172.21.15.148:0/701101965 reap_dead start 2023-10-10T20:30:51.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 shutdown_connections 2023-10-10T20:30:51.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 >> 172.21.15.148:0/701101965 conn(0x7f17b4068bf0 msgr2=0x7f17b40c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 shutdown_connections 2023-10-10T20:30:51.683 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.673+0000 7f17ba428700 1 -- 172.21.15.148:0/701101965 wait complete. 2023-10-10T20:30:51.686 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-10T20:30:51.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.800+0000 7f50cc356700 1 Processor -- start 2023-10-10T20:30:51.802 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.800+0000 7f50cc356700 1 -- start start 2023-10-10T20:30:51.810 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50cc356700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50c41523e0 0x7f50c406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.811 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50cc356700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c4158e50 0x7f50c40cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.812 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50cc356700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c406c010 con 0x7f50c4152010 2023-10-10T20:30:51.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50cc356700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c406bd10 con 0x7f50c41523e0 2023-10-10T20:30:51.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50cc356700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c415c060 con 0x7f50c4158e50 2023-10-10T20:30:51.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50c98f1700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50c41523e0 0x7f50c406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50ca0f2700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50c4152010 0x7f50c415b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33432/0 (socket says 172.21.15.148:33432) 2023-10-10T20:30:51.814 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.801+0000 7f50ca0f2700 1 -- 172.21.15.148:0/3846137385 learned_addr learned my addr 172.21.15.148:0/3846137385 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c98f1700 1 -- 172.21.15.148:0/3846137385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c4158e50 msgr2=0x7f50c40cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50ca8f3700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c4158e50 0x7f50c40cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c98f1700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c4158e50 0x7f50c40cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c98f1700 1 -- 172.21.15.148:0/3846137385 >> v1:172.21.15.148:6789/0 conn(0x7f50c4152010 legacy=0x7f50c415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.815 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c90f0700 1 -- 172.21.15.148:0/3846137385 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064978925 0 0) 0x7f50c406c010 con 0x7f50c4152010 2023-10-10T20:30:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c98f1700 1 -- 172.21.15.148:0/3846137385 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c415c340 con 0x7f50c41523e0 2023-10-10T20:30:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c98f1700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c41523e0 0x7f50c406c4a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f50b40098e0 tx=0x7f50b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe9daa4847f2947 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c90f0700 1 -- 172.21.15.148:0/3846137385 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4009e80 con 0x7f50c41523e0 2023-10-10T20:30:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c90f0700 1 -- 172.21.15.148:0/3846137385 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400b860 con 0x7f50c41523e0 2023-10-10T20:30:51.816 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.802+0000 7f50c90f0700 1 -- 172.21.15.148:0/3846137385 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400bbe0 con 0x7f50c41523e0 2023-10-10T20:30:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 -- 172.21.15.148:0/3846137385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c41523e0 msgr2=0x7f50c406c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c41523e0 0x7f50c406c4a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f50b40098e0 tx=0x7f50b40091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 -- 172.21.15.148:0/3846137385 shutdown_connections 2023-10-10T20:30:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c4158e50 0x7f50c40cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.817 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 --2- 172.21.15.148:0/3846137385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c41523e0 0x7f50c406c4a0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 -- 172.21.15.148:0/3846137385 >> 172.21.15.148:0/3846137385 conn(0x7f50c4069870 msgr2=0x7f50c4069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:51.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 -- 172.21.15.148:0/3846137385 shutdown_connections 2023-10-10T20:30:51.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.803+0000 7f50cc356700 1 -- 172.21.15.148:0/3846137385 wait complete. 2023-10-10T20:30:51.818 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 Processor -- start 2023-10-10T20:30:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 -- start start 2023-10-10T20:30:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c41523e0 0x7f50c414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50c4158e50 0x7f50c415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.819 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4168e60 con 0x7f50c4152010 2023-10-10T20:30:51.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4168250 con 0x7f50c4158e50 2023-10-10T20:30:51.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.804+0000 7f50cc356700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4168530 con 0x7f50c41523e0 2023-10-10T20:30:51.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.820 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37392/0 (socket says 172.21.15.148:37392) 2023-10-10T20:30:51.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 -- 172.21.15.148:0/3672305060 learned_addr learned my addr 172.21.15.148:0/3672305060 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:51.821 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca8f3700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50c4158e50 0x7f50c415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50c98f1700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c41523e0 0x7f50c414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 -- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c41523e0 msgr2=0x7f50c414f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.822 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c41523e0 0x7f50c414f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 -- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50c4158e50 msgr2=0x7f50c415a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:51.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50c4158e50 0x7f50c415a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:51.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 -- 172.21.15.148:0/3672305060 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b4009420 con 0x7f50c4152010 2023-10-10T20:30:51.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ca0f2700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c5b0 tx=0x7f50c0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.823 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0005b30 con 0x7f50c4152010 2023-10-10T20:30:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0005c90 con 0x7f50c4152010 2023-10-10T20:30:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c4168810 con 0x7f50c4152010 2023-10-10T20:30:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007300 con 0x7f50c4152010 2023-10-10T20:30:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.805+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c415a9a0 con 0x7f50c4152010 2023-10-10T20:30:51.824 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.806+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c404c3c0 con 0x7f50c4152010 2023-10-10T20:30:51.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.807+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50c001b3b0 con 0x7f50c4152010 2023-10-10T20:30:51.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.807+0000 7f50ba7fc700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 0x7f50a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:51.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.807+0000 7f50c98f1700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 0x7f50a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:51.825 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.807+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f50c0083660 con 0x7f50c4152010 2023-10-10T20:30:51.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.808+0000 7f50c98f1700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 0x7f50a80693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f50b40091e0 tx=0x7f50b4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:51.826 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.812+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50c0050bd0 con 0x7f50c4152010 2023-10-10T20:30:51.997 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.996+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c4168fa0 con 0x7f50a8066f30 2023-10-10T20:30:51.998 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.996+0000 7f50ba7fc700 1 -- 172.21.15.148:0/3672305060 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f50b400bb10 con 0x7f50a8066f30 2023-10-10T20:30:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 msgr2=0x7f50a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 0x7f50a80693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f50b40091e0 tx=0x7f50b4017000 comp rx=0 tx=0).stop 2023-10-10T20:30:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 msgr2=0x7f50c414cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c5b0 tx=0x7f50c0001070 comp rx=0 tx=0).stop 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 shutdown_connections 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f50a8066f30 0x7f50a80693e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50c4158e50 0x7f50c415a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c4152010 0x7f50c414cfc0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 --2- 172.21.15.148:0/3672305060 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50c41523e0 0x7f50c414f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 >> 172.21.15.148:0/3672305060 conn(0x7f50c4069870 msgr2=0x7f50c40ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 shutdown_connections 2023-10-10T20:30:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:51.998+0000 7f50cc356700 1 -- 172.21.15.148:0/3672305060 wait complete. 2023-10-10T20:30:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 1 to deep-scrub 2023-10-10T20:30:52.011 INFO:tasks.ceph:Scrubbing osd.2 2023-10-10T20:30:52.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:52.129 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.128+0000 7f263b058700 1 Processor -- start 2023-10-10T20:30:52.130 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.128+0000 7f263b058700 1 -- start start 2023-10-10T20:30:52.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f263b058700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f26340b4e00 0x7f263415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.132 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f263b058700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26340b17b0 con 0x7f2634159ef0 2023-10-10T20:30:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f263b058700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26340b1a60 con 0x7f26340b6a10 2023-10-10T20:30:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f263b058700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26340b1d40 con 0x7f26340b4e00 2023-10-10T20:30:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f2638df4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2634159ef0 0x7f26340b10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33438/0 (socket says 172.21.15.148:33438) 2023-10-10T20:30:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f2638df4700 1 -- 172.21.15.148:0/2076335456 learned_addr learned my addr 172.21.15.148:0/2076335456 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:52.133 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f26395f5700 1 --2- 172.21.15.148:0/2076335456 >> v2:172.21.15.161:3300/0 conn(0x7f26340b4e00 0x7f263415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989943142 0 0) 0x7f26340b17b0 con 0x7f2634159ef0 2023-10-10T20:30:52.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c003570 con 0x7f2634159ef0 2023-10-10T20:30:52.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283361188 0 0) 0x7f26340b1a60 con 0x7f26340b6a10 2023-10-10T20:30:52.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26340b17b0 con 0x7f26340b6a10 2023-10-10T20:30:52.134 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f26395f5700 1 -- 172.21.15.148:0/2076335456 >> v1:172.21.15.161:6789/0 conn(0x7f26340b6a10 legacy=0x7f26340b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f26395f5700 1 -- 172.21.15.148:0/2076335456 >> v1:172.21.15.148:6789/0 conn(0x7f2634159ef0 legacy=0x7f26340b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539162587 0 0) 0x7f261c003570 con 0x7f2634159ef0 2023-10-10T20:30:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.129+0000 7f26395f5700 1 -- 172.21.15.148:0/2076335456 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26340b2020 con 0x7f26340b4e00 2023-10-10T20:30:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f26395f5700 1 --2- 172.21.15.148:0/2076335456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f263415a2c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098e0 tx=0x7f262c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bed702f730c10b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c011630 con 0x7f26340b4e00 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c011790 con 0x7f26340b4e00 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f262b7fe700 1 -- 172.21.15.148:0/2076335456 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c011b30 con 0x7f26340b4e00 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 -- 172.21.15.148:0/2076335456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 msgr2=0x7f263415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 --2- 172.21.15.148:0/2076335456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f263415a2c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098e0 tx=0x7f262c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 -- 172.21.15.148:0/2076335456 shutdown_connections 2023-10-10T20:30:52.136 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 --2- 172.21.15.148:0/2076335456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f263415a2c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 -- 172.21.15.148:0/2076335456 >> 172.21.15.148:0/2076335456 conn(0x7f263409a0a0 msgr2=0x7f2634067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 -- 172.21.15.148:0/2076335456 shutdown_connections 2023-10-10T20:30:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.130+0000 7f263b058700 1 -- 172.21.15.148:0/2076335456 wait complete. 2023-10-10T20:30:52.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.131+0000 7f263b058700 1 Processor -- start 2023-10-10T20:30:52.140 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 -- start start 2023-10-10T20:30:52.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f2634154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 0x7f2634153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2634159ef0 0x7f263414e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341555f0 con 0x7f26340b6a10 2023-10-10T20:30:52.141 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2634155a80 con 0x7f26340b4e00 2023-10-10T20:30:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f263b058700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26341549e0 con 0x7f2634159ef0 2023-10-10T20:30:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.132+0000 7f262bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 0x7f2634153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 0x7f2634153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37402/0 (socket says 172.21.15.148:37402) 2023-10-10T20:30:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 -- 172.21.15.148:0/1368756109 learned_addr learned my addr 172.21.15.148:0/1368756109 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:52.142 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f26395f5700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2634159ef0 0x7f263414e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f2638df4700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f2634154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 -- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2634159ef0 msgr2=0x7f263414e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2634159ef0 0x7f263414e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 -- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 msgr2=0x7f2634154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.143 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f2634154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c009420 con 0x7f26340b6a10 2023-10-10T20:30:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f2638df4700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f26340b4e00 0x7f2634154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f26395f5700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2634159ef0 0x7f263414e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f262bfff700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 0x7f2634153a70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f262400c7c0 tx=0x7f2624007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.144 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624010510 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2624010690 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2634154cc0 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624010a50 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.133+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2634155280 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.134+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f263404c3c0 con 0x7f26340b6a10 2023-10-10T20:30:52.145 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.134+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f262401b400 con 0x7f26340b6a10 2023-10-10T20:30:52.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2628ff9700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2620066f30 0x7f26200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f26240840e0 con 0x7f26340b6a10 2023-10-10T20:30:52.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2638df4700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2620066f30 0x7f26200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.146 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2628ff9700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f262006cd80 0x7f262006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f262006f860 con 0x7f262006cd80 2023-10-10T20:30:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2624086720 con 0x7f26340b6a10 2023-10-10T20:30:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f26395f5700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f262006cd80 0x7f262006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f2638df4700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2620066f30 0x7f26200693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f26300016e0 tx=0x7f2630009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.147 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.135+0000 7f26395f5700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f262006cd80 0x7f262006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.148 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.136+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== osd.2 v2:172.21.15.148:6808/86312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f262006f860 con 0x7f262006cd80 2023-10-10T20:30:52.149 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.148+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 --> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f263406a9f0 con 0x7f262006cd80 2023-10-10T20:30:52.150 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.148+0000 7f2628ff9700 1 -- 172.21.15.148:0/1368756109 <== osd.2 v2:172.21.15.148:6808/86312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f263406a9f0 con 0x7f262006cd80 2023-10-10T20:30:52.150 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:52.150 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:52.151 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:52.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f262006cd80 msgr2=0x7f262006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6808/86312,v1:172.21.15.148:6809/86312] conn(0x7f262006cd80 0x7f262006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2620066f30 msgr2=0x7f26200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.152 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2620066f30 0x7f26200693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f26300016e0 tx=0x7f2630009040 comp rx=0 tx=0).stop 2023-10-10T20:30:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 msgr2=0x7f2634153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 --2- 172.21.15.148:0/1368756109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340b6a10 0x7f2634153a70 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f262400c7c0 tx=0x7f2624007de0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f26395f5700 1 -- 172.21.15.148:0/1368756109 reap_dead start 2023-10-10T20:30:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 shutdown_connections 2023-10-10T20:30:52.153 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 >> 172.21.15.148:0/1368756109 conn(0x7f263409a0a0 msgr2=0x7f263414efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 shutdown_connections 2023-10-10T20:30:52.154 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.149+0000 7f263b058700 1 -- 172.21.15.148:0/1368756109 wait complete. 2023-10-10T20:30:52.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-10T20:30:52.277 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.276+0000 7f253b28d700 1 Processor -- start 2023-10-10T20:30:52.278 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.276+0000 7f253b28d700 1 -- start start 2023-10-10T20:30:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253b28d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534157870 0x7f253415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253b28d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f253415a620 0x7f253415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.280 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253b28d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253415cbb0 con 0x7f253415d580 2023-10-10T20:30:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253b28d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253415bfa0 con 0x7f253415a620 2023-10-10T20:30:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253b28d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253415c280 con 0x7f2534157870 2023-10-10T20:30:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2538828700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f253415a620 0x7f253415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.281 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253982a700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f253415d580 0x7f2534158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33440/0 (socket says 172.21.15.148:33440) 2023-10-10T20:30:52.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2538828700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f253415a620 0x7f253415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37408/0 (socket says 172.21.15.148:37408) 2023-10-10T20:30:52.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f253982a700 1 -- 172.21.15.148:0/1952124925 learned_addr learned my addr 172.21.15.148:0/1952124925 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:52.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2539029700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534157870 0x7f253415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2538828700 1 -- 172.21.15.148:0/1952124925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534157870 msgr2=0x7f253415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.282 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2538828700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534157870 0x7f253415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f2538828700 1 -- 172.21.15.148:0/1952124925 >> v1:172.21.15.148:6789/0 conn(0x7f253415d580 legacy=0x7f2534158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f2538828700 1 -- 172.21.15.148:0/1952124925 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253415c560 con 0x7f253415a620 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.277+0000 7f252bfff700 1 -- 172.21.15.148:0/1952124925 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636171714 0 0) 0x7f253415cbb0 con 0x7f253415d580 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f2538828700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415a620 0x7f253415d040 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f25240098e0 tx=0x7f25240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eadb17be0087506 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f252bfff700 1 -- 172.21.15.148:0/1952124925 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400b640 con 0x7f253415a620 2023-10-10T20:30:52.283 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f252bfff700 1 -- 172.21.15.148:0/1952124925 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400b7a0 con 0x7f253415a620 2023-10-10T20:30:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f252bfff700 1 -- 172.21.15.148:0/1952124925 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400bb40 con 0x7f253415a620 2023-10-10T20:30:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 -- 172.21.15.148:0/1952124925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415a620 msgr2=0x7f253415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415a620 0x7f253415d040 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f25240098e0 tx=0x7f25240091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 -- 172.21.15.148:0/1952124925 shutdown_connections 2023-10-10T20:30:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534157870 0x7f253415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 --2- 172.21.15.148:0/1952124925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415a620 0x7f253415d040 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.278+0000 7f253b28d700 1 -- 172.21.15.148:0/1952124925 >> 172.21.15.148:0/1952124925 conn(0x7f2534068ce0 msgr2=0x7f25340690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.279+0000 7f253b28d700 1 -- 172.21.15.148:0/1952124925 shutdown_connections 2023-10-10T20:30:52.285 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.279+0000 7f253b28d700 1 -- 172.21.15.148:0/1952124925 wait complete. 2023-10-10T20:30:52.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 Processor -- start 2023-10-10T20:30:52.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 -- start start 2023-10-10T20:30:52.289 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534157870 0x7f2534152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 0x7f25340b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.280+0000 7f253b28d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25340b27f0 con 0x7f253415d580 2023-10-10T20:30:52.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253b28d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253416a720 con 0x7f2534157870 2023-10-10T20:30:52.290 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253b28d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340b1f50 con 0x7f253415a620 2023-10-10T20:30:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37424/0 (socket says 172.21.15.148:37424) 2023-10-10T20:30:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 -- 172.21.15.148:0/103112857 learned_addr learned my addr 172.21.15.148:0/103112857 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f2539029700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534157870 0x7f2534152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.291 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f2538828700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 0x7f25340b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 -- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 msgr2=0x7f25340b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 0x7f25340b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 -- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534157870 msgr2=0x7f2534152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534157870 0x7f2534152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.292 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 -- 172.21.15.148:0/103112857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009420 con 0x7f253415d580 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f2538828700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 0x7f25340b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253982a700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c4f0 tx=0x7f252c007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c005b30 con 0x7f253415d580 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c005c90 con 0x7f253415d580 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340b2230 con 0x7f253415d580 2023-10-10T20:30:52.293 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0072e0 con 0x7f253415d580 2023-10-10T20:30:52.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.281+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2534152bc0 con 0x7f253415d580 2023-10-10T20:30:52.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.282+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253416a4f0 con 0x7f253415d580 2023-10-10T20:30:52.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.283+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f252c007690 con 0x7f253415d580 2023-10-10T20:30:52.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.283+0000 7f25297fa700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.294 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.283+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f252c0834c0 con 0x7f253415d580 2023-10-10T20:30:52.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.283+0000 7f2539029700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.283+0000 7f2539029700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 0x7f25180693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f253000f570 tx=0x7f253000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.295 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.287+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f252c04b5a0 con 0x7f253415d580 2023-10-10T20:30:52.472 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.471+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f253404c3c0 con 0x7f2518066f30 2023-10-10T20:30:52.473 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.471+0000 7f25297fa700 1 -- 172.21.15.148:0/103112857 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f253404c3c0 con 0x7f2518066f30 2023-10-10T20:30:52.474 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 msgr2=0x7f25180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 0x7f25180693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f253000f570 tx=0x7f253000b040 comp rx=0 tx=0).stop 2023-10-10T20:30:52.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 msgr2=0x7f2534153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c4f0 tx=0x7f252c007f20 comp rx=0 tx=0).stop 2023-10-10T20:30:52.475 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 shutdown_connections 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2518066f30 0x7f25180693e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f253415d580 0x7f2534153e00 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534157870 0x7f2534152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 --2- 172.21.15.148:0/103112857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253415a620 0x7f25340b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 >> 172.21.15.148:0/103112857 conn(0x7f2534068ce0 msgr2=0x7f2534159480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.476 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 shutdown_connections 2023-10-10T20:30:52.477 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.473+0000 7f253b28d700 1 -- 172.21.15.148:0/103112857 wait complete. 2023-10-10T20:30:52.477 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 2 to deep-scrub 2023-10-10T20:30:52.486 INFO:tasks.ceph:Scrubbing osd.3 2023-10-10T20:30:52.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:52.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.604+0000 7f25520d0700 1 Processor -- start 2023-10-10T20:30:52.606 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.604+0000 7f25520d0700 1 -- start start 2023-10-10T20:30:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f25520d0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f254c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f25520d0700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c0aff80 con 0x7f254c068ec0 2023-10-10T20:30:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f25520d0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c0b0350 con 0x7f254c0b1950 2023-10-10T20:30:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f25520d0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b15c0 con 0x7f254c068af0 2023-10-10T20:30:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f254affd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f254c068ec0 0x7f254c15aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56450/0 (socket says 172.21.15.148:56450) 2023-10-10T20:30:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.605+0000 7f254affd700 1 -- 172.21.15.148:0/677232586 learned_addr learned my addr 172.21.15.148:0/677232586 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254b7fe700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f254c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157432652 0 0) 0x7f254c0aff80 con 0x7f254c068ec0 2023-10-10T20:30:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2534003570 con 0x7f254c068ec0 2023-10-10T20:30:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531669204 0 0) 0x7f254c0b0350 con 0x7f254c0b1950 2023-10-10T20:30:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f254c0aff80 con 0x7f254c0b1950 2023-10-10T20:30:52.617 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2805363313 0 0) 0x7f2534003570 con 0x7f254c068ec0 2023-10-10T20:30:52.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 msgr2=0x7f254c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f254c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.618 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 >> v1:172.21.15.161:6789/0 conn(0x7f254c0b1950 legacy=0x7f254c0af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0b0bf0 con 0x7f254c068ec0 2023-10-10T20:30:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f2540001470 con 0x7f254c068ec0 2023-10-10T20:30:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254b7fe700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f254c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 >> v1:172.21.15.148:6789/0 conn(0x7f254c068ec0 legacy=0x7f254c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f2534003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534004160 0x7f2534004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534004d50 0x7f25340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b15c0 con 0x7f2534004160 2023-10-10T20:30:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253400bba0 con 0x7f2534004d50 2023-10-10T20:30:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253400bd60 con 0x7f254c068af0 2023-10-10T20:30:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254affd700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534004160 0x7f2534004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254b7fe700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f2534003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254bfff700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534004d50 0x7f25340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254affd700 1 -- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 msgr2=0x7f2534003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254affd700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f254c068af0 0x7f2534003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254affd700 1 -- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534004d50 msgr2=0x7f25340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.606+0000 7f254affd700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534004d50 0x7f25340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254affd700 1 -- 172.21.15.148:0/677232586 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c009420 con 0x7f2534004160 2023-10-10T20:30:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254bfff700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2534004d50 0x7f25340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254affd700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534004160 0x7f2534004810 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f254000c2d0 tx=0x7f2540001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3da327344fc522 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000cba0 con 0x7f2534004160 2023-10-10T20:30:52.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000cd20 con 0x7f2534004160 2023-10-10T20:30:52.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254a7fc700 1 -- 172.21.15.148:0/677232586 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540005b70 con 0x7f2534004160 2023-10-10T20:30:52.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 -- 172.21.15.148:0/677232586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534004160 msgr2=0x7f2534004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.623 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 --2- 172.21.15.148:0/677232586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2534004160 0x7f2534004810 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f254000c2d0 tx=0x7f2540001e10 comp rx=0 tx=0).stop 2023-10-10T20:30:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f254bfff700 1 -- 172.21.15.148:0/677232586 reap_dead start 2023-10-10T20:30:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 -- 172.21.15.148:0/677232586 shutdown_connections 2023-10-10T20:30:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 -- 172.21.15.148:0/677232586 >> 172.21.15.148:0/677232586 conn(0x7f254c069870 msgr2=0x7f254c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 -- 172.21.15.148:0/677232586 shutdown_connections 2023-10-10T20:30:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.607+0000 7f25520d0700 1 -- 172.21.15.148:0/677232586 wait complete. 2023-10-10T20:30:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 Processor -- start 2023-10-10T20:30:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 -- start start 2023-10-10T20:30:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534004160 0x7f254c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 0x7f254c14a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.625 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f254c06bf10 0x7f254c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06c8d0 con 0x7f254c0caf50 2023-10-10T20:30:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0cdab0 con 0x7f254c06bf10 2023-10-10T20:30:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f25520d0700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0cdd90 con 0x7f2534004160 2023-10-10T20:30:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f254affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 0x7f254c14a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.609+0000 7f254affd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 0x7f254c14a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36398/0 (socket says 172.21.15.148:36398) 2023-10-10T20:30:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 -- 172.21.15.148:0/3939118326 learned_addr learned my addr 172.21.15.148:0/3939118326 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254b7fe700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534004160 0x7f254c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254bfff700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f254c06bf10 0x7f254c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 -- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534004160 msgr2=0x7f254c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2534004160 0x7f254c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 -- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f254c06bf10 msgr2=0x7f254c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f254c06bf10 0x7f254c06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0c7d40 con 0x7f254c0caf50 2023-10-10T20:30:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f254affd700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 0x7f254c14a600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2540005ca0 tx=0x7f2540015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000ce90 con 0x7f254c0caf50 2023-10-10T20:30:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400072e0 con 0x7f254c0caf50 2023-10-10T20:30:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c0c8020 con 0x7f254c0caf50 2023-10-10T20:30:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400076a0 con 0x7f254c0caf50 2023-10-10T20:30:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.610+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0cd6f0 con 0x7f254c0caf50 2023-10-10T20:30:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.611+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f254c06b900 con 0x7f254c0caf50 2023-10-10T20:30:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2540019030 con 0x7f254c0caf50 2023-10-10T20:30:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2538066f30 0x7f25380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2540085500 con 0x7f254c0caf50 2023-10-10T20:30:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f254b7fe700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2538066f30 0x7f25380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f253806cd80 0x7f253806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f253806f860 con 0x7f253806cd80 2023-10-10T20:30:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2540087b40 con 0x7f254c0caf50 2023-10-10T20:30:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f254bfff700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f253806cd80 0x7f253806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.612+0000 7f254b7fe700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2538066f30 0x7f25380693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f254c06db10 tx=0x7f253c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.613+0000 7f254bfff700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f253806cd80 0x7f253806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.613+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== osd.3 v2:172.21.15.148:6824/86339 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f253806f860 con 0x7f253806cd80 2023-10-10T20:30:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.625+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 --> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f254c04c3c0 con 0x7f253806cd80 2023-10-10T20:30:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25337fe700 1 -- 172.21.15.148:0/3939118326 <== osd.3 v2:172.21.15.148:6824/86339 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f254c04c3c0 con 0x7f253806cd80 2023-10-10T20:30:52.636 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:52.636 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:52.636 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f253806cd80 msgr2=0x7f253806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6824/86339,v1:172.21.15.148:6825/86339] conn(0x7f253806cd80 0x7f253806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2538066f30 msgr2=0x7f25380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f2538066f30 0x7f25380693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f254c06db10 tx=0x7f253c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 msgr2=0x7f254c14a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 --2- 172.21.15.148:0/3939118326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f254c0caf50 0x7f254c14a600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f2540005ca0 tx=0x7f2540015000 comp rx=0 tx=0).stop 2023-10-10T20:30:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f254bfff700 1 -- 172.21.15.148:0/3939118326 reap_dead start 2023-10-10T20:30:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 shutdown_connections 2023-10-10T20:30:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 >> 172.21.15.148:0/3939118326 conn(0x7f254c069870 msgr2=0x7f254c1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 shutdown_connections 2023-10-10T20:30:52.640 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.626+0000 7f25520d0700 1 -- 172.21.15.148:0/3939118326 wait complete. 2023-10-10T20:30:52.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-10T20:30:52.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.757+0000 7fee7b942700 1 Processor -- start 2023-10-10T20:30:52.759 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.757+0000 7fee7b942700 1 -- start start 2023-10-10T20:30:52.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee7b942700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee74159cf0 0x7fee7415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee7b942700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee740cf1c0 0x7fee74068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.762 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee7b942700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee740cca20 con 0x7fee7415a0c0 2023-10-10T20:30:52.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee7b942700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7415ea40 con 0x7fee74159cf0 2023-10-10T20:30:52.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee7b942700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee74067660 con 0x7fee740cf1c0 2023-10-10T20:30:52.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee796de700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee74159cf0 0x7fee7415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.763 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee796de700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee74159cf0 0x7fee7415e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36408/0 (socket says 172.21.15.148:36408) 2023-10-10T20:30:52.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.758+0000 7fee796de700 1 -- 172.21.15.148:0/1829429636 learned_addr learned my addr 172.21.15.148:0/1829429636 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:52.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee79edf700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee740cf1c0 0x7fee74068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee796de700 1 -- 172.21.15.148:0/1829429636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee740cf1c0 msgr2=0x7fee74068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee796de700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee740cf1c0 0x7fee74068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.764 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee796de700 1 -- 172.21.15.148:0/1829429636 >> v1:172.21.15.161:6789/0 conn(0x7fee7415a0c0 legacy=0x7fee740ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee796de700 1 -- 172.21.15.148:0/1829429636 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74067f00 con 0x7fee74159cf0 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee796de700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74159cf0 0x7fee7415e500 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fee700099b0 tx=0x7fee70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d933870c2740a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee6bfff700 1 -- 172.21.15.148:0/1829429636 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000b860 con 0x7fee74159cf0 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee6bfff700 1 -- 172.21.15.148:0/1829429636 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000b9c0 con 0x7fee74159cf0 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee6bfff700 1 -- 172.21.15.148:0/1829429636 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000bd60 con 0x7fee74159cf0 2023-10-10T20:30:52.765 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee7b942700 1 -- 172.21.15.148:0/1829429636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74159cf0 msgr2=0x7fee7415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.759+0000 7fee7b942700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74159cf0 0x7fee7415e500 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fee700099b0 tx=0x7fee70009250 comp rx=0 tx=0).stop 2023-10-10T20:30:52.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 -- 172.21.15.148:0/1829429636 shutdown_connections 2023-10-10T20:30:52.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74159cf0 0x7fee7415e500 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.766 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 --2- 172.21.15.148:0/1829429636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee740cf1c0 0x7fee74068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 -- 172.21.15.148:0/1829429636 >> 172.21.15.148:0/1829429636 conn(0x7fee7406c6e0 msgr2=0x7fee74069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 -- 172.21.15.148:0/1829429636 shutdown_connections 2023-10-10T20:30:52.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.760+0000 7fee7b942700 1 -- 172.21.15.148:0/1829429636 wait complete. 2023-10-10T20:30:52.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.761+0000 7fee7b942700 1 Processor -- start 2023-10-10T20:30:52.767 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.761+0000 7fee7b942700 1 -- start start 2023-10-10T20:30:52.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee740cf1c0 0x7fee74154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.770 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee74159cf0 0x7fee740ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee74158530 con 0x7fee7415a0c0 2023-10-10T20:30:52.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee74157920 con 0x7fee740cf1c0 2023-10-10T20:30:52.771 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee74157c00 con 0x7fee74159cf0 2023-10-10T20:30:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36412/0 (socket says 172.21.15.148:36412) 2023-10-10T20:30:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 -- 172.21.15.148:0/4218619642 learned_addr learned my addr 172.21.15.148:0/4218619642 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:52.772 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee796de700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee740cf1c0 0x7fee74154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee78edd700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee74159cf0 0x7fee740ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 -- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee74159cf0 msgr2=0x7fee740ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee74159cf0 0x7fee740ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 -- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee740cf1c0 msgr2=0x7fee74154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.773 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee740cf1c0 0x7fee74154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 -- 172.21.15.148:0/4218619642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee700094f0 con 0x7fee7415a0c0 2023-10-10T20:30:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee79edf700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007f30 tx=0x7fee6c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00eb90 con 0x7fee7415a0c0 2023-10-10T20:30:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c00ecf0 con 0x7fee7415a0c0 2023-10-10T20:30:52.774 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c014670 con 0x7fee7415a0c0 2023-10-10T20:30:52.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee74157ee0 con 0x7fee7415a0c0 2023-10-10T20:30:52.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.762+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee74154ed0 con 0x7fee7415a0c0 2023-10-10T20:30:52.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.763+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7404c3c0 con 0x7fee7415a0c0 2023-10-10T20:30:52.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.764+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee6c014a20 con 0x7fee7415a0c0 2023-10-10T20:30:52.775 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.764+0000 7fee697fa700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 0x7fee580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:52.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.764+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fee6c0848c0 con 0x7fee7415a0c0 2023-10-10T20:30:52.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.764+0000 7fee796de700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 0x7fee580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:52.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.765+0000 7fee796de700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 0x7fee580693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fee700099b0 tx=0x7fee70017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:52.776 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.768+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee6c051e30 con 0x7fee7415a0c0 2023-10-10T20:30:52.957 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.956+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fee74158670 con 0x7fee58066f30 2023-10-10T20:30:52.958 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.956+0000 7fee697fa700 1 -- 172.21.15.148:0/4218619642 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fee74158670 con 0x7fee58066f30 2023-10-10T20:30:52.960 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 msgr2=0x7fee580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 0x7fee580693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fee700099b0 tx=0x7fee70017000 comp rx=0 tx=0).stop 2023-10-10T20:30:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 msgr2=0x7fee740ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:52.961 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007f30 tx=0x7fee6c0098e0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 shutdown_connections 2023-10-10T20:30:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fee58066f30 0x7fee580693e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.962 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee7415a0c0 0x7fee740ca220 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee740cf1c0 0x7fee74154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 --2- 172.21.15.148:0/4218619642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee74159cf0 0x7fee740ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 >> 172.21.15.148:0/4218619642 conn(0x7fee7406c6e0 msgr2=0x7fee74068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:52.963 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 shutdown_connections 2023-10-10T20:30:52.964 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:52.958+0000 7fee7b942700 1 -- 172.21.15.148:0/4218619642 wait complete. 2023-10-10T20:30:52.964 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 3 to deep-scrub 2023-10-10T20:30:52.972 INFO:tasks.ceph:Scrubbing osd.4 2023-10-10T20:30:52.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:53.091 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.090+0000 7f6d66fe7700 1 Processor -- start 2023-10-10T20:30:53.092 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.090+0000 7f6d66fe7700 1 -- start start 2023-10-10T20:30:53.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d66fe7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d600b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d66fe7700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d600ce810 0x7f6d600cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d66fe7700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d600cd4c0 con 0x7f6d600cc940 2023-10-10T20:30:53.094 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d66fe7700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d600b3f40 con 0x7f6d600ce810 2023-10-10T20:30:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d66fe7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600cbb80 con 0x7f6d6015d240 2023-10-10T20:30:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d600ce810 0x7f6d600cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d657e4700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d600cc940 0x7f6d600ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56454/0 (socket says 172.21.15.148:56454) 2023-10-10T20:30:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d600ce810 0x7f6d600cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36416/0 (socket says 172.21.15.148:36416) 2023-10-10T20:30:53.095 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d657e4700 1 -- 172.21.15.148:0/1906714870 learned_addr learned my addr 172.21.15.148:0/1906714870 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:53.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d65fe5700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d600b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 -- 172.21.15.148:0/1906714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 msgr2=0x7f6d600b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d600b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 -- 172.21.15.148:0/1906714870 >> v1:172.21.15.148:6789/0 conn(0x7f6d600cc940 legacy=0x7f6d600ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.096 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d667e6700 1 -- 172.21.15.148:0/1906714870 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600cbe60 con 0x7f6d600ce810 2023-10-10T20:30:53.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.091+0000 7f6d64fe3700 1 -- 172.21.15.148:0/1906714870 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651194274 0 0) 0x7f6d600cd4c0 con 0x7f6d600cc940 2023-10-10T20:30:53.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d667e6700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600ce810 0x7f6d600cec90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009e60 tx=0x7f6d58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da05d5bf0213f656 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d64fe3700 1 -- 172.21.15.148:0/1906714870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800e070 con 0x7f6d600ce810 2023-10-10T20:30:53.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d64fe3700 1 -- 172.21.15.148:0/1906714870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5800bb60 con 0x7f6d600ce810 2023-10-10T20:30:53.097 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d64fe3700 1 -- 172.21.15.148:0/1906714870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58012430 con 0x7f6d600ce810 2023-10-10T20:30:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 -- 172.21.15.148:0/1906714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600ce810 msgr2=0x7f6d600cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600ce810 0x7f6d600cec90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009e60 tx=0x7f6d58009780 comp rx=0 tx=0).stop 2023-10-10T20:30:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 -- 172.21.15.148:0/1906714870 shutdown_connections 2023-10-10T20:30:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600ce810 0x7f6d600cec90 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.098 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/1906714870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d600b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 -- 172.21.15.148:0/1906714870 >> 172.21.15.148:0/1906714870 conn(0x7f6d60067e40 msgr2=0x7f6d600d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 -- 172.21.15.148:0/1906714870 shutdown_connections 2023-10-10T20:30:53.099 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.092+0000 7f6d66fe7700 1 -- 172.21.15.148:0/1906714870 wait complete. 2023-10-10T20:30:53.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.093+0000 7f6d66fe7700 1 Processor -- start 2023-10-10T20:30:53.102 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.093+0000 7f6d66fe7700 1 -- start start 2023-10-10T20:30:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 0x7f6d6015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d600ce810 0x7f6d6014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d6016a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6014aca0 con 0x7f6d600cc940 2023-10-10T20:30:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c84e0 con 0x7f6d600ce810 2023-10-10T20:30:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d66fe7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6016ae90 con 0x7f6d6015d240 2023-10-10T20:30:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d65fe5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 0x7f6d6015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d65fe5700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 0x7f6d6015c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36418/0 (socket says 172.21.15.148:36418) 2023-10-10T20:30:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d65fe5700 1 -- 172.21.15.148:0/2414539722 learned_addr learned my addr 172.21.15.148:0/2414539722 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d667e6700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d6016a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.094+0000 7f6d657e4700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d600ce810 0x7f6d6014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 -- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 msgr2=0x7f6d6016a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d6015d240 0x7f6d6016a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 -- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d600ce810 msgr2=0x7f6d6014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.105 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d600ce810 0x7f6d6014a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d58009a20 con 0x7f6d600cc940 2023-10-10T20:30:53.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d65fe5700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 0x7f6d6015c170 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ec80 tx=0x7f6d5c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00c9d0 con 0x7f6d600cc940 2023-10-10T20:30:53.106 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00cb30 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c013470 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6015c850 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.095+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d600c8170 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.096+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d600c7cf0 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.096+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d5c013820 con 0x7f6d600cc940 2023-10-10T20:30:53.107 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d55ffb700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d44066f30 0x7f6d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d5c084980 con 0x7f6d600cc940 2023-10-10T20:30:53.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d657e4700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d44066f30 0x7f6d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d55ffb700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f6d4406cd80 0x7f6d4406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d4406f860 con 0x7f6d4406cd80 2023-10-10T20:30:53.108 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d5c089240 con 0x7f6d600cc940 2023-10-10T20:30:53.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d667e6700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f6d4406cd80 0x7f6d4406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.097+0000 7f6d657e4700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d44066f30 0x7f6d440693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6d6015bf60 tx=0x7f6d50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.098+0000 7f6d667e6700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f6d4406cd80 0x7f6d4406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.109 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.098+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== osd.4 v2:172.21.15.161:6802/86554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f6d4406f860 con 0x7f6d4406cd80 2023-10-10T20:30:53.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.110+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 --> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6d6006ac60 con 0x7f6d4406cd80 2023-10-10T20:30:53.112 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.111+0000 7f6d55ffb700 1 -- 172.21.15.148:0/2414539722 <== osd.4 v2:172.21.15.161:6802/86554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f6d6006ac60 con 0x7f6d4406cd80 2023-10-10T20:30:53.113 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:53.113 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:53.113 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:53.114 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.111+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f6d4406cd80 msgr2=0x7f6d4406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.111+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.161:6802/86554,v1:172.21.15.161:6806/86554] conn(0x7f6d4406cd80 0x7f6d4406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d44066f30 msgr2=0x7f6d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f6d44066f30 0x7f6d440693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6d6015bf60 tx=0x7f6d50009040 comp rx=0 tx=0).stop 2023-10-10T20:30:53.115 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 msgr2=0x7f6d6015c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 --2- 172.21.15.148:0/2414539722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d600cc940 0x7f6d6015c170 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ec80 tx=0x7f6d5c009120 comp rx=0 tx=0).stop 2023-10-10T20:30:53.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d667e6700 1 -- 172.21.15.148:0/2414539722 reap_dead start 2023-10-10T20:30:53.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 shutdown_connections 2023-10-10T20:30:53.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 >> 172.21.15.148:0/2414539722 conn(0x7f6d60067e40 msgr2=0x7f6d60158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.116 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 shutdown_connections 2023-10-10T20:30:53.117 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.112+0000 7f6d66fe7700 1 -- 172.21.15.148:0/2414539722 wait complete. 2023-10-10T20:30:53.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-10T20:30:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.238+0000 7fd8250db700 1 Processor -- start 2023-10-10T20:30:53.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.238+0000 7fd8250db700 1 -- start start 2023-10-10T20:30:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8250db700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd820159da0 0x7fd8200b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8250db700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 0x7fd820068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8250db700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82015ea70 con 0x7fd820069040 2023-10-10T20:30:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8250db700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200694e0 con 0x7fd820159da0 2023-10-10T20:30:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8250db700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200b3990 con 0x7fd82015a170 2023-10-10T20:30:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd81ffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd820159da0 0x7fd8200b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 0x7fd820068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8248da700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd820069040 0x7fd8200af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.148:43094/0 (socket says 172.21.15.148:43094) 2023-10-10T20:30:53.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.239+0000 7fd8248da700 1 -- 172.21.15.148:0/460490568 learned_addr learned my addr 172.21.15.148:0/460490568 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:53.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81ffff700 1 -- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 msgr2=0x7fd820068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81ffff700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 0x7fd820068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81ffff700 1 -- 172.21.15.148:0/460490568 >> v1:172.21.15.161:6789/0 conn(0x7fd820069040 legacy=0x7fd8200af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81ffff700 1 -- 172.21.15.148:0/460490568 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200b0b70 con 0x7fd820159da0 2023-10-10T20:30:53.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81f7fe700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 0x7fd820068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:53.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81ffff700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200b2da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098e0 tx=0x7fd8100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde4b1782aef6a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81effd700 1 -- 172.21.15.148:0/460490568 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000b640 con 0x7fd820159da0 2023-10-10T20:30:53.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81effd700 1 -- 172.21.15.148:0/460490568 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b7a0 con 0x7fd820159da0 2023-10-10T20:30:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd81effd700 1 -- 172.21.15.148:0/460490568 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000bb40 con 0x7fd820159da0 2023-10-10T20:30:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 -- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 msgr2=0x7fd8200b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200b2da0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098e0 tx=0x7fd8100091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 -- 172.21.15.148:0/460490568 shutdown_connections 2023-10-10T20:30:53.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200b2da0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 --2- 172.21.15.148:0/460490568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd82015a170 0x7fd820068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 -- 172.21.15.148:0/460490568 >> 172.21.15.148:0/460490568 conn(0x7fd8200d0bc0 msgr2=0x7fd8200d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.240+0000 7fd8250db700 1 -- 172.21.15.148:0/460490568 shutdown_connections 2023-10-10T20:30:53.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.241+0000 7fd8250db700 1 -- 172.21.15.148:0/460490568 wait complete. 2023-10-10T20:30:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 Processor -- start 2023-10-10T20:30:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 -- start start 2023-10-10T20:30:53.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd820069040 0x7fd8200c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.242+0000 7fd8250db700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200c69b0 con 0x7fd82015a170 2023-10-10T20:30:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8250db700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200c5da0 con 0x7fd820159da0 2023-10-10T20:30:53.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8250db700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200c6080 con 0x7fd820069040 2023-10-10T20:30:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36432/0 (socket says 172.21.15.148:36432) 2023-10-10T20:30:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 -- 172.21.15.148:0/3103888345 learned_addr learned my addr 172.21.15.148:0/3103888345 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd81ffff700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd820069040 0x7fd8200c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.252 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd81f7fe700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 -- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd820069040 msgr2=0x7fd8200c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd820069040 0x7fd8200c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 -- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 msgr2=0x7fd8200ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 -- 172.21.15.148:0/3103888345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009420 con 0x7fd82015a170 2023-10-10T20:30:53.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8248da700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cc40 tx=0x7fd818001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800a8b0 con 0x7fd82015a170 2023-10-10T20:30:53.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81800aa10 con 0x7fd82015a170 2023-10-10T20:30:53.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810009420 con 0x7fd82015a170 2023-10-10T20:30:53.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800add0 con 0x7fd82015a170 2023-10-10T20:30:53.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.243+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8200c6640 con 0x7fd82015a170 2023-10-10T20:30:53.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.244+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8200c6b20 con 0x7fd82015a170 2023-10-10T20:30:53.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.245+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd818007300 con 0x7fd82015a170 2023-10-10T20:30:53.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.245+0000 7fd807fff700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 0x7fd80c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.245+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd8180844f0 con 0x7fd82015a170 2023-10-10T20:30:53.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.245+0000 7fd81ffff700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 0x7fd80c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.245+0000 7fd81ffff700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 0x7fd80c0693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd8100093f0 tx=0x7fd810017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.249+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd818051ae0 con 0x7fd82015a170 2023-10-10T20:30:53.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.431+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd82015def0 con 0x7fd80c066f30 2023-10-10T20:30:53.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.431+0000 7fd807fff700 1 -- 172.21.15.148:0/3103888345 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd82015def0 con 0x7fd80c066f30 2023-10-10T20:30:53.435 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 msgr2=0x7fd80c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 0x7fd80c0693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd8100093f0 tx=0x7fd810017000 comp rx=0 tx=0).stop 2023-10-10T20:30:53.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 msgr2=0x7fd8200b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cc40 tx=0x7fd818001a50 comp rx=0 tx=0).stop 2023-10-10T20:30:53.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 shutdown_connections 2023-10-10T20:30:53.436 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd80c066f30 0x7fd80c0693e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82015a170 0x7fd8200b5ae0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd820069040 0x7fd8200c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 --2- 172.21.15.148:0/3103888345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd820159da0 0x7fd8200ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 >> 172.21.15.148:0/3103888345 conn(0x7fd8200d0bc0 msgr2=0x7fd8200d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.437 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 shutdown_connections 2023-10-10T20:30:53.438 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.434+0000 7fd8250db700 1 -- 172.21.15.148:0/3103888345 wait complete. 2023-10-10T20:30:53.438 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 4 to deep-scrub 2023-10-10T20:30:53.447 INFO:tasks.ceph:Scrubbing osd.5 2023-10-10T20:30:53.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:53.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.565+0000 7f35b4e4d700 1 Processor -- start 2023-10-10T20:30:53.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.565+0000 7f35b4e4d700 1 -- start start 2023-10-10T20:30:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35b4e4d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f35b015d240 0x7f35b00b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35b4e4d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f35b00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35b4e4d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00cd4f0 con 0x7f35b00cc940 2023-10-10T20:30:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35b4e4d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00b3f40 con 0x7f35b015d240 2023-10-10T20:30:53.571 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35b4e4d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00cbb80 con 0x7f35b00ce810 2023-10-10T20:30:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35aeffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f35b00cc940 0x7f35b00ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56462/0 (socket says 172.21.15.148:56462) 2023-10-10T20:30:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35aeffd700 1 -- 172.21.15.148:0/1485215451 learned_addr learned my addr 172.21.15.148:0/1485215451 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35af7fe700 1 --2- 172.21.15.148:0/1485215451 >> v2:172.21.15.161:3300/0 conn(0x7f35b015d240 0x7f35b00b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.572 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35affff700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f35b00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350307594 0 0) 0x7f35b00cd4f0 con 0x7f35b00cc940 2023-10-10T20:30:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3598003570 con 0x7f35b00cc940 2023-10-10T20:30:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420794092 0 0) 0x7f3598003570 con 0x7f35b00cc940 2023-10-10T20:30:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 msgr2=0x7f35b00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f35b00cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 >> v2:172.21.15.161:3300/0 conn(0x7f35b015d240 msgr2=0x7f35b00b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 --2- 172.21.15.148:0/1485215451 >> v2:172.21.15.161:3300/0 conn(0x7f35b015d240 0x7f35b00b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00cbe60 con 0x7f35b00cc940 2023-10-10T20:30:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7f35a4001470 con 0x7f35b00cc940 2023-10-10T20:30:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.566+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 >> v1:172.21.15.148:6789/0 conn(0x7f35b00cc940 legacy=0x7f35b00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f3598003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3598003fe0 0x7f3598004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598004d50 0x7f35980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00b3f40 con 0x7f3598003fe0 2023-10-10T20:30:53.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00cbb80 con 0x7f3598004d50 2023-10-10T20:30:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359800ba90 con 0x7f35b00ce810 2023-10-10T20:30:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35af7fe700 1 --2- 172.21.15.148:0/1485215451 >> v2:172.21.15.161:3300/0 conn(0x7f35b015d240 0x7f35b00b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3598003fe0 0x7f3598004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35affff700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598004d50 0x7f35980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35af7fe700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f3598003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 -- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 msgr2=0x7f3598003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b00ce810 0x7f3598003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 -- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598004d50 msgr2=0x7f35980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598004d50 0x7f35980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 -- 172.21.15.148:0/1485215451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598003570 con 0x7f3598003fe0 2023-10-10T20:30:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35aeffd700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3598003fe0 0x7f3598004810 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c3c0 tx=0x7f35a4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d296fb7a2522ff server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400cc30 con 0x7f3598003fe0 2023-10-10T20:30:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a400cdd0 con 0x7f3598003fe0 2023-10-10T20:30:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.567+0000 7f35ae7fc700 1 -- 172.21.15.148:0/1485215451 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4005b70 con 0x7f3598003fe0 2023-10-10T20:30:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 -- 172.21.15.148:0/1485215451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3598003fe0 msgr2=0x7f3598004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 --2- 172.21.15.148:0/1485215451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3598003fe0 0x7f3598004810 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c3c0 tx=0x7f35a4001e90 comp rx=0 tx=0).stop 2023-10-10T20:30:53.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35affff700 1 -- 172.21.15.148:0/1485215451 reap_dead start 2023-10-10T20:30:53.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 -- 172.21.15.148:0/1485215451 shutdown_connections 2023-10-10T20:30:53.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 -- 172.21.15.148:0/1485215451 >> 172.21.15.148:0/1485215451 conn(0x7f35b0067e40 msgr2=0x7f35b00d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 -- 172.21.15.148:0/1485215451 shutdown_connections 2023-10-10T20:30:53.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.568+0000 7f35b4e4d700 1 -- 172.21.15.148:0/1485215451 wait complete. 2023-10-10T20:30:53.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.569+0000 7f35b4e4d700 1 Processor -- start 2023-10-10T20:30:53.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.569+0000 7f35b4e4d700 1 -- start start 2023-10-10T20:30:53.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598003fe0 0x7f35b009e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b009a810 0x7f35b014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 0x7f35b009c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b009cde0 con 0x7f35b009c400 2023-10-10T20:30:53.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0159050 con 0x7f3598003fe0 2023-10-10T20:30:53.584 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35b4e4d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b009ac80 con 0x7f35b009a810 2023-10-10T20:30:53.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 0x7f35b009c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 0x7f35b009c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36450/0 (socket says 172.21.15.148:36450) 2023-10-10T20:30:53.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 -- 172.21.15.148:0/3872819426 learned_addr learned my addr 172.21.15.148:0/3872819426 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:53.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 -- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b009a810 msgr2=0x7f35b014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.585 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35af7fe700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598003fe0 0x7f35b009e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35aeffd700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b009a810 0x7f35b014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35b009a810 0x7f35b014b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 -- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598003fe0 msgr2=0x7f35b009e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3598003fe0 0x7f35b009e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b014bc40 con 0x7f35b009c400 2023-10-10T20:30:53.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.570+0000 7f35affff700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 0x7f35b009c8a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f35b0089160 tx=0x7f35a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800e040 con 0x7f35b009c400 2023-10-10T20:30:53.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a800f040 con 0x7f35b009c400 2023-10-10T20:30:53.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b009af10 con 0x7f35b009c400 2023-10-10T20:30:53.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800fdc0 con 0x7f35b009c400 2023-10-10T20:30:53.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b009b4d0 con 0x7f35b009c400 2023-10-10T20:30:53.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.571+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35b006ac60 con 0x7f35b009c400 2023-10-10T20:30:53.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.572+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a8015640 con 0x7f35b009c400 2023-10-10T20:30:53.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.572+0000 7f35977fe700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f359c066f30 0x7f359c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.572+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f35a80849d0 con 0x7f35b009c400 2023-10-10T20:30:53.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.572+0000 7f35af7fe700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f359c066f30 0x7f359c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35977fe700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7f359c06cd80 0x7f359c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f359c06f860 con 0x7f359c06cd80 2023-10-10T20:30:53.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a8089240 con 0x7f35b009c400 2023-10-10T20:30:53.589 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35aeffd700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7f359c06cd80 0x7f359c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35af7fe700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f359c066f30 0x7f359c0693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006aa0 tx=0x7f35a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.573+0000 7f35aeffd700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7f359c06cd80 0x7f359c06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.590 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.574+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== osd.5 v2:172.21.15.161:6801/86555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7f359c06f860 con 0x7f359c06cd80 2023-10-10T20:30:53.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.586+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 --> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f35b00cc140 con 0x7f359c06cd80 2023-10-10T20:30:53.591 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35977fe700 1 -- 172.21.15.148:0/3872819426 <== osd.5 v2:172.21.15.161:6801/86555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f35b00cc140 con 0x7f359c06cd80 2023-10-10T20:30:53.591 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:53.592 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:53.592 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:53.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7f359c06cd80 msgr2=0x7f359c06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.593 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.161:6801/86555,v1:172.21.15.161:6804/86555] conn(0x7f359c06cd80 0x7f359c06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f359c066f30 msgr2=0x7f359c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f359c066f30 0x7f359c0693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006aa0 tx=0x7f35a0008040 comp rx=0 tx=0).stop 2023-10-10T20:30:53.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 msgr2=0x7f35b009c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 --2- 172.21.15.148:0/3872819426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35b009c400 0x7f35b009c8a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f35b0089160 tx=0x7f35a800b040 comp rx=0 tx=0).stop 2023-10-10T20:30:53.594 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35affff700 1 -- 172.21.15.148:0/3872819426 reap_dead start 2023-10-10T20:30:53.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 shutdown_connections 2023-10-10T20:30:53.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 >> 172.21.15.148:0/3872819426 conn(0x7f35b0067e40 msgr2=0x7f35b0069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 shutdown_connections 2023-10-10T20:30:53.595 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.587+0000 7f35b4e4d700 1 -- 172.21.15.148:0/3872819426 wait complete. 2023-10-10T20:30:53.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-10T20:30:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.716+0000 7f865d195700 1 Processor -- start 2023-10-10T20:30:53.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.716+0000 7f865d195700 1 -- start start 2023-10-10T20:30:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86581568d0 0x7f8658069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f865806a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658156ca0 0x7f86580c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865815aa90 con 0x7f8658156ca0 2023-10-10T20:30:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8658067ba0 con 0x7f86581568d0 2023-10-10T20:30:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865d195700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658067870 con 0x7f86580ad9b0 2023-10-10T20:30:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658156ca0 0x7f86580c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865759e700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8658156ca0 0x7f86580c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36466/0 (socket says 172.21.15.148:36466) 2023-10-10T20:30:53.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.717+0000 7f865759e700 1 -- 172.21.15.148:0/1088891044 learned_addr learned my addr 172.21.15.148:0/1088891044 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f8656d9d700 1 --2- 172.21.15.148:0/1088891044 >> v2:172.21.15.161:3300/0 conn(0x7f86581568d0 0x7f8658069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 -- 172.21.15.148:0/1088891044 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 msgr2=0x7f865806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865659c700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f865806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f865806a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 -- 172.21.15.148:0/1088891044 >> v2:172.21.15.161:3300/0 conn(0x7f86581568d0 msgr2=0x7f8658069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 --2- 172.21.15.148:0/1088891044 >> v2:172.21.15.161:3300/0 conn(0x7f86581568d0 0x7f8658069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 -- 172.21.15.148:0/1088891044 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580c8ca0 con 0x7f8658156ca0 2023-10-10T20:30:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865759e700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580c8090 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f86580b3bd0 tx=0x7f8650009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8faa7ebbca19e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f8655d9b700 1 -- 172.21.15.148:0/1088891044 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000b860 con 0x7f8658156ca0 2023-10-10T20:30:53.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f8655d9b700 1 -- 172.21.15.148:0/1088891044 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000b9c0 con 0x7f8658156ca0 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f8655d9b700 1 -- 172.21.15.148:0/1088891044 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000bd60 con 0x7f8658156ca0 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865d195700 1 -- 172.21.15.148:0/1088891044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 msgr2=0x7f86580c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.718+0000 7f865d195700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580c8090 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f86580b3bd0 tx=0x7f8650009250 comp rx=0 tx=0).stop 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 -- 172.21.15.148:0/1088891044 shutdown_connections 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580c8090 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 --2- 172.21.15.148:0/1088891044 >> v2:172.21.15.161:3300/0 conn(0x7f86581568d0 0x7f8658069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 --2- 172.21.15.148:0/1088891044 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f865806a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 -- 172.21.15.148:0/1088891044 >> 172.21.15.148:0/1088891044 conn(0x7f86580cda50 msgr2=0x7f86580cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 -- 172.21.15.148:0/1088891044 shutdown_connections 2023-10-10T20:30:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.719+0000 7f865d195700 1 -- 172.21.15.148:0/1088891044 wait complete. 2023-10-10T20:30:53.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 Processor -- start 2023-10-10T20:30:53.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 -- start start 2023-10-10T20:30:53.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f8658070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.729 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 0x7f86580aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.720+0000 7f865d195700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865816e780 con 0x7f8658156ca0 2023-10-10T20:30:53.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865d195700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865816dc00 con 0x7f86581568d0 2023-10-10T20:30:53.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865d195700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865816dee0 con 0x7f86580ad9b0 2023-10-10T20:30:53.730 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36470/0 (socket says 172.21.15.148:36470) 2023-10-10T20:30:53.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 -- 172.21.15.148:0/1724056300 learned_addr learned my addr 172.21.15.148:0/1724056300 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:53.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865659c700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 0x7f86580aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f8656d9d700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f8658070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.731 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 -- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 msgr2=0x7f8658070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f8658070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 -- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 msgr2=0x7f86580aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 0x7f86580aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 -- 172.21.15.148:0/1724056300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500094f0 con 0x7f8658156ca0 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f8656d9d700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f8658070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:53.732 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865659c700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 0x7f86580aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:53.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865759e700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f86580ce880 tx=0x7f8650017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001a630 con 0x7f8658156ca0 2023-10-10T20:30:53.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865001a7b0 con 0x7f8658156ca0 2023-10-10T20:30:53.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001ab70 con 0x7f8658156ca0 2023-10-10T20:30:53.733 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865816e1c0 con 0x7f8658156ca0 2023-10-10T20:30:53.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.721+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865816d890 con 0x7f8658156ca0 2023-10-10T20:30:53.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.722+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865804c3c0 con 0x7f8658156ca0 2023-10-10T20:30:53.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.723+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f865002d440 con 0x7f8658156ca0 2023-10-10T20:30:53.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.723+0000 7f863effd700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 0x7f86440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:53.734 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.723+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8650085ff0 con 0x7f8658156ca0 2023-10-10T20:30:53.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.723+0000 7f8656d9d700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 0x7f86440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:53.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.723+0000 7f8656d9d700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 0x7f86440693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f86580af400 tx=0x7f8648007710 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:53.735 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.727+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8650053520 con 0x7f8658156ca0 2023-10-10T20:30:53.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.910+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f86580c8f80 con 0x7f8644066f30 2023-10-10T20:30:53.912 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.910+0000 7f863effd700 1 -- 172.21.15.148:0/1724056300 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f86580c8f80 con 0x7f8644066f30 2023-10-10T20:30:53.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 msgr2=0x7f86440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 0x7f86440693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f86580af400 tx=0x7f8648007710 comp rx=0 tx=0).stop 2023-10-10T20:30:53.914 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 msgr2=0x7f86580aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:53.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f86580ce880 tx=0x7f8650017000 comp rx=0 tx=0).stop 2023-10-10T20:30:53.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 shutdown_connections 2023-10-10T20:30:53.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f8644066f30 0x7f86440693e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8658156ca0 0x7f86580aadc0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.915 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86580ad9b0 0x7f8658070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 --2- 172.21.15.148:0/1724056300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86581568d0 0x7f86580aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:53.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 >> 172.21.15.148:0/1724056300 conn(0x7f86580cda50 msgr2=0x7f86580c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:53.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 shutdown_connections 2023-10-10T20:30:53.916 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:53.912+0000 7f865d195700 1 -- 172.21.15.148:0/1724056300 wait complete. 2023-10-10T20:30:53.917 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 5 to deep-scrub 2023-10-10T20:30:53.925 INFO:tasks.ceph:Scrubbing osd.6 2023-10-10T20:30:53.926 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:53.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:30:53.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:30:54.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.046+0000 7fa175068700 1 Processor -- start 2023-10-10T20:30:54.054 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.046+0000 7fa175068700 1 -- start start 2023-10-10T20:30:54.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa175068700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa170161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa175068700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17006f4f0 con 0x7fa170156de0 2023-10-10T20:30:54.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa175068700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700c9880 con 0x7fa170156a10 2023-10-10T20:30:54.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa175068700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006e8e0 con 0x7fa1700cc820 2023-10-10T20:30:54.055 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa16e59c700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa170156de0 0x7fa1700cc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56472/0 (socket says 172.21.15.148:56472) 2023-10-10T20:30:54.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.047+0000 7fa16e59c700 1 -- 172.21.15.148:0/782398680 learned_addr learned my addr 172.21.15.148:0/782398680 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:54.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816296952 0 0) 0x7fa17006f4f0 con 0x7fa170156de0 2023-10-10T20:30:54.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16f59e700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa170161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa15c003570 con 0x7fa170156de0 2023-10-10T20:30:54.056 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560547546 0 0) 0x7fa15c003570 con 0x7fa170156de0 2023-10-10T20:30:54.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 msgr2=0x7fa170161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa170161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 >> v1:172.21.15.161:6789/0 conn(0x7fa170156a10 legacy=0x7fa17015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17006ebc0 con 0x7fa170156de0 2023-10-10T20:30:54.057 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 619219862 0 0) 0x7fa158001470 con 0x7fa170156de0 2023-10-10T20:30:54.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16f59e700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa170161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:54.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 >> v1:172.21.15.148:6789/0 conn(0x7fa170156de0 legacy=0x7fa1700cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c003a90 0x7fa15c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa15c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.058 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa15c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17006e8e0 con 0x7fa15c004810 2023-10-10T20:30:54.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00bbf0 con 0x7fa15c003a90 2023-10-10T20:30:54.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00bdb0 con 0x7fa1700cc820 2023-10-10T20:30:54.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa15c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.059 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16ed9d700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c003a90 0x7fa15c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16f59e700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa15c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 -- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 msgr2=0x7fa15c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1700cc820 0x7fa15c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 -- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c003a90 msgr2=0x7fa15c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.060 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c003a90 0x7fa15c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.048+0000 7fa16e59c700 1 -- 172.21.15.148:0/782398680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c003570 con 0x7fa15c004810 2023-10-10T20:30:54.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa16e59c700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa15c004c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c3c0 tx=0x7fa158001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194683b063c5be19 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800cc90 con 0x7fa15c004810 2023-10-10T20:30:54.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15800cdf0 con 0x7fa15c004810 2023-10-10T20:30:54.061 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa16dd9b700 1 -- 172.21.15.148:0/782398680 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158005b70 con 0x7fa15c004810 2023-10-10T20:30:54.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 -- 172.21.15.148:0/782398680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 msgr2=0x7fa15c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 --2- 172.21.15.148:0/782398680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa15c004c80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c3c0 tx=0x7fa158001e50 comp rx=0 tx=0).stop 2023-10-10T20:30:54.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa16f59e700 1 -- 172.21.15.148:0/782398680 reap_dead start 2023-10-10T20:30:54.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 -- 172.21.15.148:0/782398680 shutdown_connections 2023-10-10T20:30:54.062 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 -- 172.21.15.148:0/782398680 >> 172.21.15.148:0/782398680 conn(0x7fa1700ce5c0 msgr2=0x7fa1700ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 -- 172.21.15.148:0/782398680 shutdown_connections 2023-10-10T20:30:54.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.049+0000 7fa175068700 1 -- 172.21.15.148:0/782398680 wait complete. 2023-10-10T20:30:54.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.050+0000 7fa175068700 1 Processor -- start 2023-10-10T20:30:54.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 -- start start 2023-10-10T20:30:54.063 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa1700b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1701517b0 0x7fa17016a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa17016ac30 0x7fa1700c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170153210 con 0x7fa15c004810 2023-10-10T20:30:54.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170169fe0 con 0x7fa1701517b0 2023-10-10T20:30:54.064 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa175068700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700b34e0 con 0x7fa17016ac30 2023-10-10T20:30:54.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa16ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa1700b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa16ed9d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa1700b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36500/0 (socket says 172.21.15.148:36500) 2023-10-10T20:30:54.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa16ed9d700 1 -- 172.21.15.148:0/279850366 learned_addr learned my addr 172.21.15.148:0/279850366 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:54.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa16e59c700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1701517b0 0x7fa17016a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.065 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.051+0000 7fa16f59e700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa17016ac30 0x7fa1700c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 -- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa17016ac30 msgr2=0x7fa1700c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa17016ac30 0x7fa1700c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 -- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1701517b0 msgr2=0x7fa17016a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1701517b0 0x7fa17016a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.066 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17016a260 con 0x7fa15c004810 2023-10-10T20:30:54.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa16ed9d700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa1700b2fa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa16000ec40 tx=0x7fa160001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160001c80 con 0x7fa15c004810 2023-10-10T20:30:54.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160001de0 con 0x7fa15c004810 2023-10-10T20:30:54.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1700c5830 con 0x7fa15c004810 2023-10-10T20:30:54.067 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000c920 con 0x7fa15c004810 2023-10-10T20:30:54.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.052+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1700c5ad0 con 0x7fa15c004810 2023-10-10T20:30:54.068 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.053+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1700c5cc0 con 0x7fa15c004810 2023-10-10T20:30:54.069 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.053+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa16000cc90 con 0x7fa15c004810 2023-10-10T20:30:54.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa166ffd700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa150066f30 0x7fa1500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa1600843c0 con 0x7fa15c004810 2023-10-10T20:30:54.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa16e59c700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa150066f30 0x7fa1500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.070 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa166ffd700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7fa15006cd80 0x7fa15006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa15006f860 con 0x7fa15006cd80 2023-10-10T20:30:54.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa160086a00 con 0x7fa15c004810 2023-10-10T20:30:54.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa16f59e700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7fa15006cd80 0x7fa15006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.071 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa16e59c700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa150066f30 0x7fa1500693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa158001e90 tx=0x7fa158015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.054+0000 7fa16f59e700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7fa15006cd80 0x7fa15006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.055+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== osd.6 v2:172.21.15.161:6805/86553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fa15006f860 con 0x7fa15006cd80 2023-10-10T20:30:54.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.067+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 --> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa1700b0390 con 0x7fa15006cd80 2023-10-10T20:30:54.072 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa166ffd700 1 -- 172.21.15.148:0/279850366 <== osd.6 v2:172.21.15.161:6805/86553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fa1700b0390 con 0x7fa15006cd80 2023-10-10T20:30:54.072 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:54.073 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:54.073 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:54.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7fa15006cd80 msgr2=0x7fa15006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.074 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.161:6805/86553,v1:172.21.15.161:6809/86553] conn(0x7fa15006cd80 0x7fa15006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa150066f30 msgr2=0x7fa1500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fa150066f30 0x7fa1500693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa158001e90 tx=0x7fa158015000 comp rx=0 tx=0).stop 2023-10-10T20:30:54.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 msgr2=0x7fa1700b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 --2- 172.21.15.148:0/279850366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa15c004810 0x7fa1700b2fa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa16000ec40 tx=0x7fa160001670 comp rx=0 tx=0).stop 2023-10-10T20:30:54.075 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa16f59e700 1 -- 172.21.15.148:0/279850366 reap_dead start 2023-10-10T20:30:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 shutdown_connections 2023-10-10T20:30:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 >> 172.21.15.148:0/279850366 conn(0x7fa1700ce5c0 msgr2=0x7fa1700c38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 shutdown_connections 2023-10-10T20:30:54.076 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.068+0000 7fa175068700 1 -- 172.21.15.148:0/279850366 wait complete. 2023-10-10T20:30:54.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-10T20:30:54.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.235+0000 7f09f07a6700 1 Processor -- start 2023-10-10T20:30:54.237 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.235+0000 7f09f07a6700 1 -- start start 2023-10-10T20:30:54.239 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09f07a6700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f09e80c9af0 0x7f09e806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09f07a6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 0x7f09e8070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09f07a6700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e8071040 con 0x7f09e81510f0 2023-10-10T20:30:54.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09f07a6700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e80ca0b0 con 0x7f09e80c9af0 2023-10-10T20:30:54.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09f07a6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e8068a10 con 0x7f09e8070700 2023-10-10T20:30:54.240 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09edd41700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f09e80c9af0 0x7f09e806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09ee542700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f09e81510f0 0x7f09e815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56476/0 (socket says 172.21.15.148:56476) 2023-10-10T20:30:54.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.236+0000 7f09ee542700 1 -- 172.21.15.148:0/4245648153 learned_addr learned my addr 172.21.15.148:0/4245648153 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:54.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09eed43700 1 --2- 172.21.15.148:0/4245648153 >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 0x7f09e8070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09edd41700 1 -- 172.21.15.148:0/4245648153 >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 msgr2=0x7f09e8070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.241 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09edd41700 1 --2- 172.21.15.148:0/4245648153 >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 0x7f09e8070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09edd41700 1 -- 172.21.15.148:0/4245648153 >> v1:172.21.15.148:6789/0 conn(0x7f09e81510f0 legacy=0x7f09e815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09ed540700 1 -- 172.21.15.148:0/4245648153 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226675264 0 0) 0x7f09e8071040 con 0x7f09e81510f0 2023-10-10T20:30:54.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09edd41700 1 -- 172.21.15.148:0/4245648153 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8068cf0 con 0x7f09e80c9af0 2023-10-10T20:30:54.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09eed43700 1 --2- 172.21.15.148:0/4245648153 >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 0x7f09e8070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:30:54.242 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09edd41700 1 --2- 172.21.15.148:0/4245648153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e80c9af0 0x7f09e806ecc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f09e806df40 tx=0x7f09d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27896df040fc433a server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09ed540700 1 -- 172.21.15.148:0/4245648153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800b6b0 con 0x7f09e80c9af0 2023-10-10T20:30:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09ed540700 1 -- 172.21.15.148:0/4245648153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800b810 con 0x7f09e80c9af0 2023-10-10T20:30:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09ed540700 1 -- 172.21.15.148:0/4245648153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800bbb0 con 0x7f09e80c9af0 2023-10-10T20:30:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09f07a6700 1 -- 172.21.15.148:0/4245648153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e80c9af0 msgr2=0x7f09e806ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.243 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.237+0000 7f09f07a6700 1 --2- 172.21.15.148:0/4245648153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e80c9af0 0x7f09e806ecc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f09e806df40 tx=0x7f09d80091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 -- 172.21.15.148:0/4245648153 shutdown_connections 2023-10-10T20:30:54.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 --2- 172.21.15.148:0/4245648153 >> v2:172.21.15.161:3300/0 conn(0x7f09e8070700 0x7f09e8070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 --2- 172.21.15.148:0/4245648153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e80c9af0 0x7f09e806ecc0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 -- 172.21.15.148:0/4245648153 >> 172.21.15.148:0/4245648153 conn(0x7f09e80ca4f0 msgr2=0x7f09e80ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.244 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 -- 172.21.15.148:0/4245648153 shutdown_connections 2023-10-10T20:30:54.245 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.238+0000 7f09f07a6700 1 -- 172.21.15.148:0/4245648153 wait complete. 2023-10-10T20:30:54.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 Processor -- start 2023-10-10T20:30:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 -- start start 2023-10-10T20:30:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09e8070700 0x7f09e80c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09e80c9af0 0x7f09e8159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80c62a0 con 0x7f09e81510f0 2023-10-10T20:30:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8159c90 con 0x7f09e80c9af0 2023-10-10T20:30:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09f07a6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80c5690 con 0x7f09e8070700 2023-10-10T20:30:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09eed43700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09eed43700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36526/0 (socket says 172.21.15.148:36526) 2023-10-10T20:30:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.239+0000 7f09eed43700 1 -- 172.21.15.148:0/2663379861 learned_addr learned my addr 172.21.15.148:0/2663379861 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:54.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09ee542700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09e8070700 0x7f09e80c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09edd41700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09e80c9af0 0x7f09e8159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 -- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09e8070700 msgr2=0x7f09e80c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09e8070700 0x7f09e80c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 -- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09e80c9af0 msgr2=0x7f09e8159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09e80c9af0 0x7f09e8159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 -- 172.21.15.148:0/2663379861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d8009420 con 0x7f09e81510f0 2023-10-10T20:30:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09eed43700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c880 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0011600 con 0x7f09e81510f0 2023-10-10T20:30:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e0011760 con 0x7f09e81510f0 2023-10-10T20:30:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80c5970 con 0x7f09e81510f0 2023-10-10T20:30:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0011b20 con 0x7f09e81510f0 2023-10-10T20:30:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.240+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e80c5f30 con 0x7f09e81510f0 2023-10-10T20:30:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.241+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e804c3c0 con 0x7f09e81510f0 2023-10-10T20:30:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.241+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09e00193b0 con 0x7f09e81510f0 2023-10-10T20:30:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.242+0000 7f09de7fc700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.242+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f09e0083e60 con 0x7f09e81510f0 2023-10-10T20:30:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.242+0000 7f09ee542700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.242+0000 7f09ee542700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007240 tx=0x7f09e4001970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.246+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09e0054e60 con 0x7f09e81510f0 2023-10-10T20:30:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.428+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f09e80c63e0 con 0x7f09cc066f30 2023-10-10T20:30:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.428+0000 7f09de7fc700 1 -- 172.21.15.148:0/2663379861 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09e80c63e0 con 0x7f09cc066f30 2023-10-10T20:30:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 msgr2=0x7f09cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007240 tx=0x7f09e4001970 comp rx=0 tx=0).stop 2023-10-10T20:30:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 msgr2=0x7f09e80c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f09e000c880 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2023-10-10T20:30:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 shutdown_connections 2023-10-10T20:30:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e81510f0 0x7f09e80c71d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09e8070700 0x7f09e80c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 --2- 172.21.15.148:0/2663379861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09e80c9af0 0x7f09e8159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.430+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 >> 172.21.15.148:0/2663379861 conn(0x7f09e80ca4f0 msgr2=0x7f09e80ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.431+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 shutdown_connections 2023-10-10T20:30:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.431+0000 7f09f07a6700 1 -- 172.21.15.148:0/2663379861 wait complete. 2023-10-10T20:30:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 6 to deep-scrub 2023-10-10T20:30:54.444 INFO:tasks.ceph:Scrubbing osd.7 2023-10-10T20:30:54.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-10T20:30:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.560+0000 7fd02c2ee700 1 Processor -- start 2023-10-10T20:30:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.560+0000 7fd02c2ee700 1 -- start start 2023-10-10T20:30:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.561+0000 7fd02c2ee700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd024159ef0 0x7fd02415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.561+0000 7fd02c2ee700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240b17b0 con 0x7fd0240b46f0 2023-10-10T20:30:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.561+0000 7fd02c2ee700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240b1a60 con 0x7fd0240b6a10 2023-10-10T20:30:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.561+0000 7fd02c2ee700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0240b1d40 con 0x7fd024159ef0 2023-10-10T20:30:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a08a700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd024159ef0 0x7fd02415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a88b700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd0240b46f0 0x7fd0240b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56478/0 (socket says 172.21.15.148:56478) 2023-10-10T20:30:54.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a88b700 1 -- 172.21.15.148:0/1791405413 learned_addr learned my addr 172.21.15.148:0/1791405413 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:54.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a08a700 1 -- 172.21.15.148:0/1791405413 >> v1:172.21.15.161:6789/0 conn(0x7fd0240b6a10 legacy=0x7fd0240b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a08a700 1 -- 172.21.15.148:0/1791405413 >> v1:172.21.15.148:6789/0 conn(0x7fd0240b46f0 legacy=0x7fd0240b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.566 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a08a700 1 -- 172.21.15.148:0/1791405413 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240b2020 con 0x7fd024159ef0 2023-10-10T20:30:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd029088700 1 -- 172.21.15.148:0/1791405413 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377502082 0 0) 0x7fd0240b17b0 con 0x7fd0240b46f0 2023-10-10T20:30:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd02a08a700 1 --2- 172.21.15.148:0/1791405413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd024159ef0 0x7fd02415a2c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd0200098d0 tx=0x7fd020009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c21f5202240ebd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd029088700 1 -- 172.21.15.148:0/1791405413 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020009e70 con 0x7fd024159ef0 2023-10-10T20:30:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd029088700 1 -- 172.21.15.148:0/1791405413 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020005c10 con 0x7fd024159ef0 2023-10-10T20:30:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.562+0000 7fd029088700 1 -- 172.21.15.148:0/1791405413 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02000b6a0 con 0x7fd024159ef0 2023-10-10T20:30:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 -- 172.21.15.148:0/1791405413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd024159ef0 msgr2=0x7fd02415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 --2- 172.21.15.148:0/1791405413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd024159ef0 0x7fd02415a2c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fd0200098d0 tx=0x7fd020009120 comp rx=0 tx=0).stop 2023-10-10T20:30:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 -- 172.21.15.148:0/1791405413 shutdown_connections 2023-10-10T20:30:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 --2- 172.21.15.148:0/1791405413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd024159ef0 0x7fd02415a2c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 -- 172.21.15.148:0/1791405413 >> 172.21.15.148:0/1791405413 conn(0x7fd02409a0a0 msgr2=0x7fd024067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 -- 172.21.15.148:0/1791405413 shutdown_connections 2023-10-10T20:30:54.570 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.563+0000 7fd02c2ee700 1 -- 172.21.15.148:0/1791405413 wait complete. 2023-10-10T20:30:54.573 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.564+0000 7fd02c2ee700 1 Processor -- start 2023-10-10T20:30:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.564+0000 7fd02c2ee700 1 -- start start 2023-10-10T20:30:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 0x7fd02416ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0240b6a10 0x7fd02416cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd024159ef0 0x7fd0240aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c7010 con 0x7fd0240b46f0 2023-10-10T20:30:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02409bae0 con 0x7fd0240b6a10 2023-10-10T20:30:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02416b3a0 con 0x7fd024159ef0 2023-10-10T20:30:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 0x7fd02416ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 0x7fd02416ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36548/0 (socket says 172.21.15.148:36548) 2023-10-10T20:30:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 -- 172.21.15.148:0/2061869963 learned_addr learned my addr 172.21.15.148:0/2061869963 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 -- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd024159ef0 msgr2=0x7fd0240aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd024159ef0 0x7fd0240aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 -- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0240b6a10 msgr2=0x7fd02416cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0240b6a10 0x7fd02416cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200093c0 con 0x7fd0240b46f0 2023-10-10T20:30:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02a08a700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 0x7fd02416ae60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fd024071f80 tx=0x7fd020015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020022070 con 0x7fd0240b46f0 2023-10-10T20:30:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd020006630 con 0x7fd0240b46f0 2023-10-10T20:30:54.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02409aed0 con 0x7fd0240b46f0 2023-10-10T20:30:54.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02001c450 con 0x7fd0240b46f0 2023-10-10T20:30:54.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.565+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02409b490 con 0x7fd0240b46f0 2023-10-10T20:30:54.578 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.566+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd02404c3c0 con 0x7fd0240b46f0 2023-10-10T20:30:54.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd02001c800 con 0x7fd0240b46f0 2023-10-10T20:30:54.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd008066f30 0x7fd0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd0200867c0 con 0x7fd0240b46f0 2023-10-10T20:30:54.579 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd029889700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd008066f30 0x7fd0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7fd00806cd80 0x7fd00806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd00806f860 con 0x7fd00806cd80 2023-10-10T20:30:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=184) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd020086b90 con 0x7fd0240b46f0 2023-10-10T20:30:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd02a88b700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7fd00806cd80 0x7fd00806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.567+0000 7fd029889700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd008066f30 0x7fd0080693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd014007fd0 tx=0x7fd014007b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.568+0000 7fd02a88b700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7fd00806cd80 0x7fd00806f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.568+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== osd.7 v2:172.21.15.161:6800/86574 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22340 (crc 0 0 0) 0x7fd00806f860 con 0x7fd00806cd80 2023-10-10T20:30:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.581+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 --> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd024159970 con 0x7fd00806cd80 2023-10-10T20:30:54.583 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.581+0000 7fd01a7fc700 1 -- 172.21.15.148:0/2061869963 <== osd.7 v2:172.21.15.161:6800/86574 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fd024159970 con 0x7fd00806cd80 2023-10-10T20:30:54.584 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2023-10-10T20:30:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-10-10T20:30:54.585 INFO:teuthology.orchestra.run.smithi148.stdout:} 2023-10-10T20:30:54.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7fd00806cd80 msgr2=0x7fd00806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.161:6800/86574,v1:172.21.15.161:6803/86574] conn(0x7fd00806cd80 0x7fd00806f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd008066f30 msgr2=0x7fd0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7fd008066f30 0x7fd0080693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd014007fd0 tx=0x7fd014007b20 comp rx=0 tx=0).stop 2023-10-10T20:30:54.586 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 msgr2=0x7fd02416ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 --2- 172.21.15.148:0/2061869963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0240b46f0 0x7fd02416ae60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fd024071f80 tx=0x7fd020015000 comp rx=0 tx=0).stop 2023-10-10T20:30:54.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02a88b700 1 -- 172.21.15.148:0/2061869963 reap_dead start 2023-10-10T20:30:54.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 shutdown_connections 2023-10-10T20:30:54.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 >> 172.21.15.148:0/2061869963 conn(0x7fd02409a0a0 msgr2=0x7fd0240b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.587 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 shutdown_connections 2023-10-10T20:30:54.588 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.582+0000 7fd02c2ee700 1 -- 172.21.15.148:0/2061869963 wait complete. 2023-10-10T20:30:54.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-10T20:30:54.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.709+0000 7f3a78fda700 1 Processor -- start 2023-10-10T20:30:54.711 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.709+0000 7f3a78fda700 1 -- start start 2023-10-10T20:30:54.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a78fda700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.713 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a78fda700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a7415d580 0x7f3a74158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a78fda700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7415cbb0 con 0x7f3a74157870 2023-10-10T20:30:54.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a78fda700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7415bdb0 con 0x7f3a7415d580 2023-10-10T20:30:54.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a78fda700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7415bfa0 con 0x7f3a7415a620 2023-10-10T20:30:54.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a7415d580 0x7f3a74158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.714 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a7415d580 0x7f3a74158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36562/0 (socket says 172.21.15.148:36562) 2023-10-10T20:30:54.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 -- 172.21.15.148:0/4123863878 learned_addr learned my addr 172.21.15.148:0/4123863878 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:54.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a69d9b700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 -- 172.21.15.148:0/4123863878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 msgr2=0x7f3a7415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:54.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.715 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 -- 172.21.15.148:0/4123863878 >> v1:172.21.15.161:6789/0 conn(0x7f3a74157870 legacy=0x7f3a7415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.710+0000 7f3a72d9d700 1 -- 172.21.15.148:0/4123863878 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7415c280 con 0x7f3a7415d580 2023-10-10T20:30:54.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a72d9d700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a7415d580 0x7f3a74158110 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009f20 tx=0x7f3a6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac46cfb208c2dd5b server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a71d9b700 1 -- 172.21.15.148:0/4123863878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00e070 con 0x7f3a7415d580 2023-10-10T20:30:54.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a71d9b700 1 -- 172.21.15.148:0/4123863878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c00ba80 con 0x7f3a7415d580 2023-10-10T20:30:54.716 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a71d9b700 1 -- 172.21.15.148:0/4123863878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0092e0 con 0x7f3a7415d580 2023-10-10T20:30:54.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 -- 172.21.15.148:0/4123863878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a7415d580 msgr2=0x7f3a74158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a7415d580 0x7f3a74158110 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009f20 tx=0x7f3a6c009840 comp rx=0 tx=0).stop 2023-10-10T20:30:54.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 -- 172.21.15.148:0/4123863878 shutdown_connections 2023-10-10T20:30:54.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a7415d580 0x7f3a74158110 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.717 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4123863878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 -- 172.21.15.148:0/4123863878 >> 172.21.15.148:0/4123863878 conn(0x7f3a74068ce0 msgr2=0x7f3a740690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 -- 172.21.15.148:0/4123863878 shutdown_connections 2023-10-10T20:30:54.718 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.711+0000 7f3a78fda700 1 -- 172.21.15.148:0/4123863878 wait complete. 2023-10-10T20:30:54.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.712+0000 7f3a78fda700 1 Processor -- start 2023-10-10T20:30:54.721 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 -- start start 2023-10-10T20:30:54.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7416a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7415d580 0x7f3a7416aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.722 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740b1650 con 0x7f3a74157870 2023-10-10T20:30:54.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74159810 con 0x7f3a7415d580 2023-10-10T20:30:54.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a78fda700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74159af0 con 0x7f3a7415a620 2023-10-10T20:30:54.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a7259c700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36576/0 (socket says 172.21.15.148:36576) 2023-10-10T20:30:54.723 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a7259c700 1 -- 172.21.15.148:0/4059966800 learned_addr learned my addr 172.21.15.148:0/4059966800 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:54.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a72d9d700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7415d580 0x7f3a7416aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.713+0000 7f3a69d9b700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7416a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 -- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 msgr2=0x7f3a7416a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7416a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.724 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 -- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7415d580 msgr2=0x7f3a7416aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7415d580 0x7f3a7416aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 -- 172.21.15.148:0/4059966800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009ae0 con 0x7f3a74157870 2023-10-10T20:30:54.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a7259c700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c590 tx=0x7f3a6400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64011590 con 0x7f3a74157870 2023-10-10T20:30:54.725 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64011710 con 0x7f3a74157870 2023-10-10T20:30:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64011ad0 con 0x7f3a74157870 2023-10-10T20:30:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a740b18c0 con 0x7f3a74157870 2023-10-10T20:30:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.714+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a74153930 con 0x7f3a74157870 2023-10-10T20:30:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.715+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a740b1ab0 con 0x7f3a74157870 2023-10-10T20:30:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.715+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a6401b3b0 con 0x7f3a74157870 2023-10-10T20:30:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.716+0000 7f3a6affd700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 0x7f3a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.716+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3a640836a0 con 0x7f3a74157870 2023-10-10T20:30:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.716+0000 7f3a69d9b700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 0x7f3a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.716+0000 7f3a69d9b700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 0x7f3a540693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.720+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a64050c10 con 0x7f3a74157870 2023-10-10T20:30:54.904 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.903+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7404c3c0 con 0x7f3a54066f30 2023-10-10T20:30:54.905 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.903+0000 7f3a6affd700 1 -- 172.21.15.148:0/4059966800 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3a7404c3c0 con 0x7f3a54066f30 2023-10-10T20:30:54.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 msgr2=0x7f3a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 0x7f3a540693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c008040 comp rx=0 tx=0).stop 2023-10-10T20:30:54.907 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 msgr2=0x7f3a74158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:54.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c590 tx=0x7f3a6400c8a0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 shutdown_connections 2023-10-10T20:30:54.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f3a54066f30 0x7f3a540693e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7415d580 0x7f3a7416aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.908 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a74157870 0x7f3a74158e20 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 --2- 172.21.15.148:0/4059966800 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a7415a620 0x7f3a7416a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 >> 172.21.15.148:0/4059966800 conn(0x7f3a74068ce0 msgr2=0x7f3a74155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 shutdown_connections 2023-10-10T20:30:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:54.905+0000 7f3a78fda700 1 -- 172.21.15.148:0/4059966800 wait complete. 2023-10-10T20:30:54.909 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 7 to deep-scrub 2023-10-10T20:30:54.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:30:55.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.031+0000 7f121e3fb700 1 Processor -- start 2023-10-10T20:30:55.033 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.031+0000 7f121e3fb700 1 -- start start 2023-10-10T20:30:55.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.032+0000 7f121e3fb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f12180af000 0x7f12180b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.032+0000 7f121e3fb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.032+0000 7f121e3fb700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12180ccac0 con 0x7f12180af3d0 2023-10-10T20:30:55.036 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.032+0000 7f121e3fb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12180a8050 con 0x7f12180af000 2023-10-10T20:30:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.032+0000 7f121e3fb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12180b1930 con 0x7f12180affa0 2023-10-10T20:30:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f121c998700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f12180af3d0 0x7f12180a7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56488/0 (socket says 172.21.15.148:56488) 2023-10-10T20:30:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f121c998700 1 -- 172.21.15.148:0/1431294500 learned_addr learned my addr 172.21.15.148:0/1431294500 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:30:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f12177fe700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 --2- 172.21.15.148:0/1431294500 >> v2:172.21.15.161:3300/0 conn(0x7f12180af000 0x7f12180b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716093119 0 0) 0x7f12180ccac0 con 0x7f12180af3d0 2023-10-10T20:30:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1200003570 con 0x7f12180af3d0 2023-10-10T20:30:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1475684775 0 0) 0x7f1200003570 con 0x7f12180af3d0 2023-10-10T20:30:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 -- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 msgr2=0x7f12180a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:30:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 -- 172.21.15.148:0/1431294500 >> v1:172.21.15.148:6789/0 conn(0x7f12180af3d0 legacy=0x7f12180a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 -- 172.21.15.148:0/1431294500 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180b1bd0 con 0x7f12180af000 2023-10-10T20:30:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f12177fe700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-10T20:30:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1217fff700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f12180b13f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f12080098e0 tx=0x7f12080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6164e64249000139 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.033+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100017f0 con 0x7f12180af000 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1208011420 con 0x7f12180af000 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f1216ffd700 1 -- 172.21.15.148:0/1431294500 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12080117a0 con 0x7f12180af000 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 -- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 msgr2=0x7f12180b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f12180b13f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f12080098e0 tx=0x7f12080091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.040 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 -- 172.21.15.148:0/1431294500 shutdown_connections 2023-10-10T20:30:55.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f12180b13f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 --2- 172.21.15.148:0/1431294500 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 -- 172.21.15.148:0/1431294500 >> 172.21.15.148:0/1431294500 conn(0x7f12180cd1e0 msgr2=0x7f12180cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:55.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 -- 172.21.15.148:0/1431294500 shutdown_connections 2023-10-10T20:30:55.041 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.034+0000 7f121e3fb700 1 -- 172.21.15.148:0/1431294500 wait complete. 2023-10-10T20:30:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.035+0000 7f121e3fb700 1 Processor -- start 2023-10-10T20:30:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.035+0000 7f121e3fb700 1 -- start start 2023-10-10T20:30:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f1218157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218157c40 con 0x7f12180af3d0 2023-10-10T20:30:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12180990c0 con 0x7f12180af000 2023-10-10T20:30:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121e3fb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12180993a0 con 0x7f12180affa0 2023-10-10T20:30:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36584/0 (socket says 172.21.15.148:36584) 2023-10-10T20:30:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 -- 172.21.15.148:0/585029882 learned_addr learned my addr 172.21.15.148:0/585029882 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:30:55.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f1217fff700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f1218157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f121c998700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 -- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 msgr2=0x7f12180cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.047 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 -- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 msgr2=0x7f1218157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f1218157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.036+0000 7f12177fe700 1 -- 172.21.15.148:0/585029882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009420 con 0x7f12180af3d0 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f1217fff700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f1218157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f12177fe700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f120c00ec80 tx=0x7f120c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00c9d0 con 0x7f12180af3d0 2023-10-10T20:30:55.048 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00cb30 con 0x7f12180af3d0 2023-10-10T20:30:55.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00ceb0 con 0x7f12180af3d0 2023-10-10T20:30:55.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1208009420 con 0x7f12180af3d0 2023-10-10T20:30:55.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.037+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218099960 con 0x7f12180af3d0 2023-10-10T20:30:55.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.038+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121804c3c0 con 0x7f12180af3d0 2023-10-10T20:30:55.049 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.038+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f120c013680 con 0x7f12180af3d0 2023-10-10T20:30:55.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.038+0000 7f11fffff700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 0x7f12040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:30:55.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.039+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f120c0845d0 con 0x7f12180af3d0 2023-10-10T20:30:55.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.039+0000 7f1217fff700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 0x7f12040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:30:55.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.039+0000 7f1217fff700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 0x7f12040693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f12080091e0 tx=0x7f1208016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:30:55.050 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.043+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f120c051b40 con 0x7f12180af3d0 2023-10-10T20:30:55.221 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.219+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12180b1eb0 con 0x7f1204066f30 2023-10-10T20:30:55.223 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.222+0000 7f11fffff700 1 -- 172.21.15.148:0/585029882 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50606 (secure 0 0 0) 0x7f12180b1eb0 con 0x7f1204066f30 2023-10-10T20:30:55.224 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:30:55.225 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 msgr2=0x7f12040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 0x7f12040693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f12080091e0 tx=0x7f1208016000 comp rx=0 tx=0).stop 2023-10-10T20:30:55.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 msgr2=0x7f121809a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:30:55.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f120c00ec80 tx=0x7f120c0091a0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.226 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 shutdown_connections 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f1204066f30 0x7f12040693e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12180affa0 0x7f12180cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12180af000 0x7f1218157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 --2- 172.21.15.148:0/585029882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12180af3d0 0x7f121809a160 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 >> 172.21.15.148:0/585029882 conn(0x7f12180cd1e0 msgr2=0x7f12180cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:30:55.227 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 shutdown_connections 2023-10-10T20:30:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:30:55.224+0000 7f121e3fb700 1 -- 172.21.15.148:0/585029882 wait complete. 2023-10-10T20:30:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:30:55.237 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2023-10-10T20:30:53.478455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553300,"statfs":{"total":1600000000,"available":1590580070,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489239,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-120,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001233"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":307,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-10T20:30:46.335457+0000","last_change":"2023-10-10T20:19:13.265302+0000","last_active":"2023-10-10T20:30:46.335457+0000","last_peered":"2023-10-10T20:30:46.335457+0000","last_clean":"2023-10-10T20:30:46.335457+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:46.335457+0000","last_undegraded":"2023-10-10T20:30:46.335457+0000","last_fullsized":"2023-10-10T20:30:46.335457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-10T20:19:12.213049+0000","last_clean_scrub_stamp":"2023-10-10T20:19:12.213049+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-11T21:31:28.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1582,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193729,"statfs":{"total":200000000,"available":198379480,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.27500000000000002},"max":{"1min":0.623,"5min":0.754,"15min":0.754},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.32200000000000001},"max":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.53300000000000003}]},{"osd":4,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.621},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70199999999999996,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.64200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.877,"15min":1.3140000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.66700000000000004,"5min":0.92100000000000004,"15min":1.865},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":0.749,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.497}]},{"osd":3,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54800000000000004,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":15.862},"last":0.434}]},{"osd":4,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46000000000000002,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.222},"max":{"1min":0.64800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.29599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":1.02,"15min":1.0409999999999999},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":1.647},"last":0.35299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.624,"15min":1.524},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.5,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.377},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":1.673},"last":0.40300000000000002}]},{"osd":4,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.84599999999999997,"15min":1.633},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.506},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":1.696},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58999999999999997,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36699999999999999},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":1.7490000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.81299999999999994,"5min":0.98899999999999999,"15min":1.585},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.478},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.624,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.93500000000000005,"15min":1.722},"last":0.501}]}]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.40100000000000002},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.8740000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.39700000000000002,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48199999999999998,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.375,"15min":0.40300000000000002},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":5.891},"last":0.312}]},{"osd":3,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.41999999999999998,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":1.462},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.40999999999999998,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77000000000000002,"5min":0.873,"15min":9.7260000000000009},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39100000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3799999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.4970000000000001},"last":0.498},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":5.6379999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.69299999999999995,"5min":0.84699999999999998,"15min":5.758},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.55400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970859,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.36799999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.53800000000000003,"15min":2.4409999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":0.504,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35899999999999999,"15min":0.38500000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42699999999999999,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56299999999999994,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":11.815},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.45500000000000002},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.52600000000000002,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.28100000000000003},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.93600000000000005,"15min":6.399},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.32400000000000001},"max":{"1min":0.76300000000000001,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.59499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.313},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.622,"15min":0.622},"last":0.254},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34899999999999998,"15min":0.33100000000000002},"min":{"1min":0.22700000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35499999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.21099999999999999},"max":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.751},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42799999999999999,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.23300000000000001},"max":{"1min":0.58899999999999997,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.311},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.35299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.90100000000000002},"last":0.54500000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.85899999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63300000000000001}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81599999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.61099999999999999}]},{"osd":3,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.40200000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":0.623,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.77600000000000002,"15min":0.88700000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.98599999999999999},"last":0.48399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.91200000000000003},"last":0.747}]},{"osd":1,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.255},"max":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":2.9159999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.31},"max":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.73899999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.69599999999999995,"15min":0.79900000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.60299999999999998,"5min":0.80800000000000005,"15min":0.875},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.63600000000000001,"5min":0.72099999999999997,"15min":0.78400000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.79400000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.84699999999999998},"last":0.63300000000000001}]}]}],"pool_statfs":[]}} 2023-10-10T20:30:55.242 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-10T20:19:12.213049+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=10, tm_hour=20, tm_min=19, tm_sec=12, tm_wday=1, tm_yday=283, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=10, tm_hour=20, tm_min=30, tm_sec=50, tm_wday=1, tm_yday=283, tm_isdst=0) 2023-10-10T20:30:55.242 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-10T20:31:15.243 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-10T20:31:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.359+0000 7f447394f700 1 Processor -- start 2023-10-10T20:31:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.359+0000 7f447394f700 1 -- start start 2023-10-10T20:31:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f446c069de0 0x7f446c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f446c0c9930 0x7f446c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c069060 con 0x7f446c069de0 2023-10-10T20:31:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c069300 con 0x7f446c0c9930 2023-10-10T20:31:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f447394f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0695e0 con 0x7f446c0c9560 2023-10-10T20:31:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f446c069de0 0x7f446c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f446c069de0 0x7f446c157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52992/0 (socket says 172.21.15.148:52992) 2023-10-10T20:31:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 -- 172.21.15.148:0/2602843256 learned_addr learned my addr 172.21.15.148:0/2602843256 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:31:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4471eec700 1 --2- 172.21.15.148:0/2602843256 >> v2:172.21.15.161:3300/0 conn(0x7f446c0c9930 0x7f446c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f44716eb700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 -- 172.21.15.148:0/2602843256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 msgr2=0x7f446c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:31:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 -- 172.21.15.148:0/2602843256 >> v2:172.21.15.161:3300/0 conn(0x7f446c0c9930 msgr2=0x7f446c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:31:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 --2- 172.21.15.148:0/2602843256 >> v2:172.21.15.161:3300/0 conn(0x7f446c0c9930 0x7f446c15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.360+0000 7f4470eea700 1 -- 172.21.15.148:0/2602843256 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0698c0 con 0x7f446c069de0 2023-10-10T20:31:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f4470eea700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c069de0 0x7f446c157f90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c6e40 tx=0x7f445c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaf9039d59125bc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f4463fff700 1 -- 172.21.15.148:0/2602843256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00b860 con 0x7f446c069de0 2023-10-10T20:31:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f4463fff700 1 -- 172.21.15.148:0/2602843256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00b9c0 con 0x7f446c069de0 2023-10-10T20:31:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f4463fff700 1 -- 172.21.15.148:0/2602843256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00bd60 con 0x7f446c069de0 2023-10-10T20:31:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 -- 172.21.15.148:0/2602843256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c069de0 msgr2=0x7f446c157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c069de0 0x7f446c157f90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c6e40 tx=0x7f445c009250 comp rx=0 tx=0).stop 2023-10-10T20:31:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 -- 172.21.15.148:0/2602843256 shutdown_connections 2023-10-10T20:31:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 --2- 172.21.15.148:0/2602843256 >> v2:172.21.15.161:3300/0 conn(0x7f446c0c9930 0x7f446c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 --2- 172.21.15.148:0/2602843256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c069de0 0x7f446c157f90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 -- 172.21.15.148:0/2602843256 >> 172.21.15.148:0/2602843256 conn(0x7f446c067660 msgr2=0x7f446c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 -- 172.21.15.148:0/2602843256 shutdown_connections 2023-10-10T20:31:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.361+0000 7f447394f700 1 -- 172.21.15.148:0/2602843256 wait complete. 2023-10-10T20:31:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.362+0000 7f447394f700 1 Processor -- start 2023-10-10T20:31:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.362+0000 7f447394f700 1 -- start start 2023-10-10T20:31:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f446c069de0 0x7f446c0c1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b12a0 con 0x7f446c0c9930 2023-10-10T20:31:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b0690 con 0x7f446c069de0 2023-10-10T20:31:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f447394f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b0970 con 0x7f446c0c9560 2023-10-10T20:31:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f4471eec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f4471eec700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53002/0 (socket says 172.21.15.148:53002) 2023-10-10T20:31:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f4471eec700 1 -- 172.21.15.148:0/3354023091 learned_addr learned my addr 172.21.15.148:0/3354023091 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:31:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.363+0000 7f44716eb700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f446c069de0 0x7f446c0c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4470eea700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 -- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 msgr2=0x7f446c14ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c14ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 -- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f446c069de0 msgr2=0x7f446c0c1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f446c069de0 0x7f446c0c1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 -- 172.21.15.148:0/3354023091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0094f0 con 0x7f446c0c9930 2023-10-10T20:31:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f4471eec700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4464007f20 tx=0x7f4464009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400eb90 con 0x7f446c0c9930 2023-10-10T20:31:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400ed10 con 0x7f446c0c9930 2023-10-10T20:31:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c0b0c50 con 0x7f446c0c9930 2023-10-10T20:31:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464014670 con 0x7f446c0c9930 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.364+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c0c1c60 con 0x7f446c0c9930 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.365+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c0b1410 con 0x7f446c0c9930 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.365+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44640149e0 con 0x7f446c0c9930 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.366+0000 7f44617fa700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.366+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4464084830 con 0x7f446c0c9930 2023-10-10T20:31:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.366+0000 7f44716eb700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.366+0000 7f44716eb700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 0x7f44500693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4468006fd0 tx=0x7f4468008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.370+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4464051da0 con 0x7f446c0c9930 2023-10-10T20:31:15.552 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.549+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 --> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f446c04c3c0 con 0x7f4450066f30 2023-10-10T20:31:15.554 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.551+0000 7f44617fa700 1 -- 172.21.15.148:0/3354023091 <== mgr.4100 v2:172.21.15.148:6832/86081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50509 (secure 0 0 0) 0x7f446c04c3c0 con 0x7f4450066f30 2023-10-10T20:31:15.555 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:31:15.557 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 msgr2=0x7f44500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 0x7f44500693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4468006fd0 tx=0x7f4468008040 comp rx=0 tx=0).stop 2023-10-10T20:31:15.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 msgr2=0x7f446c0c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4464007f20 tx=0x7f4464009860 comp rx=0 tx=0).stop 2023-10-10T20:31:15.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 shutdown_connections 2023-10-10T20:31:15.558 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f4450066f30 0x7f44500693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f446c0c9930 0x7f446c0c3920 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f446c069de0 0x7f446c0c1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 --2- 172.21.15.148:0/3354023091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f446c0c9560 0x7f446c14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.559 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 >> 172.21.15.148:0/3354023091 conn(0x7f446c067660 msgr2=0x7f446c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 shutdown_connections 2023-10-10T20:31:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.554+0000 7f447394f700 1 -- 172.21.15.148:0/3354023091 wait complete. 2023-10-10T20:31:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2023-10-10T20:31:15.568 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-10-10T20:31:15.482710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553300,"statfs":{"total":1600000000,"available":1590580070,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002272"},"pg_stats":[{"pgid":"1.0","version":"184'119","reported_seq":320,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-10T20:30:57.628795+0000","last_change":"2023-10-10T20:30:57.628764+0000","last_active":"2023-10-10T20:30:57.628795+0000","last_peered":"2023-10-10T20:30:57.628795+0000","last_clean":"2023-10-10T20:30:57.628795+0000","last_became_active":"2023-10-10T20:19:13.264710+0000","last_became_peered":"2023-10-10T20:19:13.264710+0000","last_unstale":"2023-10-10T20:30:57.628795+0000","last_undegraded":"2023-10-10T20:30:57.628795+0000","last_fullsized":"2023-10-10T20:30:57.628795+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'119","last_scrub_stamp":"2023-10-10T20:30:57.628707+0000","last_deep_scrub":"184'119","last_deep_scrub_stamp":"2023-10-10T20:30:57.628707+0000","last_clean_scrub_stamp":"2023-10-10T20:30:57.628707+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T02:13:30.805688+0000","scrub_duration":0.0028264240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1582,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193729,"statfs":{"total":200000000,"available":198379480,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54100000000000004,"15min":0.503},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.27500000000000002},"max":{"1min":0.623,"5min":0.754,"15min":0.754},"last":0.54900000000000004}]},{"osd":1,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53600000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64400000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.314},"max":{"1min":0.71699999999999997,"5min":0.78400000000000003,"15min":1.1339999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.32200000000000001},"max":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.51800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.56499999999999995},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"last":0.56000000000000005}]},{"osd":4,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.70699999999999996,"15min":0.74199999999999999},"last":0.375},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.29899999999999999,"15min":0.28799999999999998},"max":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.72499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.441,"5min":0.41299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70199999999999996,"5min":0.79800000000000004,"15min":0.81599999999999995},"last":0.5}]},{"osd":6,"last update":"Tue Oct 10 20:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29599999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.40899999999999997,"5min":0.34799999999999998,"15min":0.33700000000000002},"max":{"1min":0.70399999999999996,"5min":0.79000000000000004,"15min":0.79200000000000004},"last":0.437}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54200000000000004,"15min":0.52300000000000002},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.877,"15min":1.3140000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":4.4219999999999997},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.27700000000000002},"max":{"1min":0.66700000000000004,"5min":0.92100000000000004,"15min":1.865},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.79000000000000004,"15min":4.7709999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.78300000000000003,"15min":1.2549999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.28199999999999997},"max":{"1min":0.749,"5min":0.95499999999999996,"15min":1.1890000000000001},"last":0.496}]},{"osd":3,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":0.69799999999999995,"5min":0.83799999999999997,"15min":4.0430000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54800000000000004,"15min":0.58499999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":15.862},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46000000000000002,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.222},"max":{"1min":0.64800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.435},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47999999999999998,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.22700000000000001},"max":{"1min":0.72899999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55000000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48399999999999999,"15min":0.47699999999999998},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.93899999999999995,"15min":1.0840000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":1.02,"15min":1.0409999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Oct 10 20:30:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66200000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.623},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.40899999999999997,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.35599999999999998},"min":{"1min":0.21099999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.50900000000000001,"5min":0.52100000000000002,"15min":1.6100000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":1.647},"last":0.40100000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.624,"15min":1.524},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.5,"5min":0.61299999999999999,"15min":1.3959999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.253},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":1.4470000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":1.673},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.84599999999999997,"15min":1.633},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.506},"last":0.56699999999999995}]},{"osd":5,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":1.696},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58999999999999997,"15min":0.627},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73699999999999999,"5min":0.94699999999999995,"15min":1.9359999999999999},"last":0.443}]},{"osd":6,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36699999999999999},"max":{"1min":0.80200000000000005,"5min":0.877,"15min":1.7490000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.81299999999999994,"5min":0.98899999999999999,"15min":1.585},"last":0.46999999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.478},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.624,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.434,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.93500000000000005,"15min":1.722},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1581,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193730,"statfs":{"total":200000000,"available":198380491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.40100000000000002},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.8740000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.39700000000000002,"15min":0.44600000000000001},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.48199999999999998,"5min":0.60999999999999999,"15min":5.5449999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.48599999999999999,"5min":0.57199999999999995,"15min":2.4830000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.375,"15min":0.40300000000000002},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":5.891},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.41999999999999998,"5min":0.58099999999999996,"15min":5.6689999999999996},"last":0.31},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.68899999999999995,"15min":1.462},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.626},"min":{"1min":0.48599999999999999,"5min":0.40999999999999998,"15min":0.32900000000000001},"max":{"1min":0.82699999999999996,"5min":0.85399999999999998,"15min":5.8529999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.90400000000000003,"15min":4.0289999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60099999999999998,"15min":0.76400000000000001},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77000000000000002,"5min":0.873,"15min":9.7260000000000009},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64200000000000002,"15min":0.61499999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.39100000000000001},"max":{"1min":1.135,"5min":1.135,"15min":1.3799999999999999},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.4970000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":5.6379999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Tue Oct 10 20:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.38800000000000001,"15min":0.36899999999999999},"max":{"1min":0.69299999999999995,"5min":0.84699999999999998,"15min":5.758},"last":0.753},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.47499999999999998,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":0.90600000000000003,"15min":4.0720000000000001},"last":0.68500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194307,"statfs":{"total":200000000,"available":198970859,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34699999999999998,"15min":0.36799999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.53800000000000003,"15min":2.4409999999999998},"last":0.308},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.39800000000000002,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23400000000000001},"max":{"1min":0.504,"5min":0.55500000000000005,"15min":8.2629999999999999},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35899999999999999,"15min":0.38500000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":6.5229999999999997},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42699999999999999,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56299999999999994,"5min":0.61399999999999999,"15min":9.2949999999999999},"last":0.38400000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":11.815},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.45500000000000002},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.20799999999999999},"max":{"1min":0.52600000000000002,"5min":0.59799999999999998,"15min":18.399999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.28100000000000003},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":2.1389999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.41199999999999998,"5min":0.36299999999999999,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.89800000000000002,"15min":9.3740000000000006},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.93600000000000005,"15min":6.399},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":2.1040000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60199999999999998,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.32400000000000001},"max":{"1min":0.76300000000000001,"5min":0.81899999999999995,"15min":9.4019999999999992},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.90800000000000003,"15min":6.4969999999999999},"last":0.44900000000000001}]},{"osd":7,"last update":"Tue Oct 10 20:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33200000000000002},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":9.3450000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.38400000000000001},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":6.4630000000000001},"last":0.373}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.313},"min":{"1min":0.217,"5min":0.19900000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.622,"15min":0.622},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34899999999999998,"15min":0.33100000000000002},"min":{"1min":0.22700000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53100000000000003,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.21099999999999999},"max":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.67900000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41499999999999998,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61899999999999999,"5min":0.622,"15min":0.751},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42799999999999999,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.27000000000000002,"15min":0.23300000000000001},"max":{"1min":0.58899999999999997,"5min":0.71499999999999997,"15min":0.74099999999999999},"last":0.443},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.32100000000000001,"5min":0.30599999999999999,"15min":0.25900000000000001},"max":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.80700000000000005},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59799999999999998,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.39000000000000001,"15min":0.35299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.64100000000000001}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83899999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.626}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.35299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.374},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.90100000000000002},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61799999999999999,"15min":0.58699999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.432,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.26400000000000001,"15min":0.22600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.85399999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.26100000000000001,"15min":0.24099999999999999},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.28499999999999998},"max":{"1min":0.80300000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81599999999999995},"last":0.745},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.94599999999999995},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.30499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66900000000000004}]},{"osd":5,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67400000000000004,"5min":0.78200000000000003,"15min":0.82099999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":0.92800000000000005},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":0.623,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.77600000000000002,"15min":0.88700000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":0.90100000000000002,"15min":0.97199999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.98599999999999999},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1006,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198969848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.78500000000000003},"last":0.502},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.33800000000000002,"15min":0.26400000000000001},"max":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.91200000000000003},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":0.71899999999999997,"5min":2.105,"15min":2.105},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55600000000000005,"15min":0.52400000000000002},"min":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.255},"max":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.79900000000000004},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.77400000000000002,"15min":2.9159999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.31},"max":{"1min":0.73399999999999999,"5min":0.76900000000000002,"15min":0.89000000000000001},"last":0.622}]},{"osd":3,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.32500000000000001},"max":{"1min":0.746,"5min":0.80400000000000005,"15min":0.96299999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69499999999999995}]},{"osd":4,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.69599999999999995,"15min":0.79900000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.499,"15min":0.51600000000000001},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.28299999999999997},"max":{"1min":0.60299999999999998,"5min":0.80800000000000005,"15min":0.875},"last":0.74299999999999999}]},{"osd":6,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48599999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.63600000000000001,"5min":0.72099999999999997,"15min":0.78400000000000003},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.53800000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.29699999999999999},"max":{"1min":0.65500000000000003,"5min":0.752,"15min":0.78200000000000003},"last":0.71899999999999997}]},{"osd":7,"last update":"Tue Oct 10 20:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.29199999999999998},"max":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.79400000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.84699999999999998},"last":0.76000000000000001}]}]}],"pool_statfs":[]}} 2023-10-10T20:31:15.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-10T20:31:15.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.682+0000 7febba1a9700 1 Processor -- start 2023-10-10T20:31:15.685 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.682+0000 7febba1a9700 1 -- start start 2023-10-10T20:31:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febba1a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 0x7febb4156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febba1a9700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb40cc440 con 0x7febb40cb280 2023-10-10T20:31:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febba1a9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb40cb830 con 0x7febb40c71d0 2023-10-10T20:31:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febba1a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb40c31a0 con 0x7febb4156160 2023-10-10T20:31:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febb2ffd700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7febb40cb280 0x7febb40c2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59110/0 (socket says 172.21.15.148:59110) 2023-10-10T20:31:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febb2ffd700 1 -- 172.21.15.148:0/1772773744 learned_addr learned my addr 172.21.15.148:0/1772773744 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:31:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.683+0000 7febb37fe700 1 --2- 172.21.15.148:0/1772773744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 0x7febb4156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486873972 0 0) 0x7febb40cc440 con 0x7febb40cb280 2023-10-10T20:31:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb9c003570 con 0x7febb40cb280 2023-10-10T20:31:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874864615 0 0) 0x7febb40cb830 con 0x7febb40c71d0 2023-10-10T20:31:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb40cc440 con 0x7febb40c71d0 2023-10-10T20:31:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2917073699 0 0) 0x7feb9c003570 con 0x7febb40cb280 2023-10-10T20:31:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb37fe700 1 -- 172.21.15.148:0/1772773744 >> v1:172.21.15.161:6789/0 conn(0x7febb40c71d0 legacy=0x7febb406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb37fe700 1 -- 172.21.15.148:0/1772773744 >> v1:172.21.15.148:6789/0 conn(0x7febb40cb280 legacy=0x7febb40c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb37fe700 1 -- 172.21.15.148:0/1772773744 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40cbb10 con 0x7febb4156160 2023-10-10T20:31:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb37fe700 1 --2- 172.21.15.148:0/1772773744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 0x7febb4156530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feba4009910 tx=0x7feba40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4542c6a733f1dc server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8001470 con 0x7febb4156160 2023-10-10T20:31:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba4011420 con 0x7febb4156160 2023-10-10T20:31:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.684+0000 7febb27fc700 1 -- 172.21.15.148:0/1772773744 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40117c0 con 0x7febb4156160 2023-10-10T20:31:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 -- 172.21.15.148:0/1772773744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 msgr2=0x7febb4156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 --2- 172.21.15.148:0/1772773744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 0x7febb4156530 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feba4009910 tx=0x7feba40091a0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 -- 172.21.15.148:0/1772773744 shutdown_connections 2023-10-10T20:31:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 --2- 172.21.15.148:0/1772773744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb4156160 0x7febb4156530 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 -- 172.21.15.148:0/1772773744 >> 172.21.15.148:0/1772773744 conn(0x7febb4067660 msgr2=0x7febb4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 -- 172.21.15.148:0/1772773744 shutdown_connections 2023-10-10T20:31:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.685+0000 7febba1a9700 1 -- 172.21.15.148:0/1772773744 wait complete. 2023-10-10T20:31:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.686+0000 7febba1a9700 1 Processor -- start 2023-10-10T20:31:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.686+0000 7febba1a9700 1 -- start start 2023-10-10T20:31:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 0x7febb40ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.696 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 0x7febb41544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb40a9ae0 con 0x7febb40cb280 2023-10-10T20:31:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb40a9dc0 con 0x7febb4156160 2023-10-10T20:31:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb40aa0a0 con 0x7febb40c71d0 2023-10-10T20:31:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53012/0 (socket says 172.21.15.148:53012) 2023-10-10T20:31:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 -- 172.21.15.148:0/935316842 learned_addr learned my addr 172.21.15.148:0/935316842 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:31:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb37fe700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 0x7febb40ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb3fff700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 0x7febb41544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.698 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 -- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 msgr2=0x7febb40ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 0x7febb40ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 -- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 msgr2=0x7febb41544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 0x7febb41544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 -- 172.21.15.148:0/935316842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4009420 con 0x7febb40cb280 2023-10-10T20:31:15.699 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb37fe700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 0x7febb40ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:31:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb3fff700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 0x7febb41544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-10T20:31:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febb2ffd700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7feba800cb70 tx=0x7feba8001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8007360 con 0x7febb40cb280 2023-10-10T20:31:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba80074e0 con 0x7febb40cb280 2023-10-10T20:31:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.687+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb40aa380 con 0x7febb40cb280 2023-10-10T20:31:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.688+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8004830 con 0x7febb40cb280 2023-10-10T20:31:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.688+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb40abb10 con 0x7febb40cb280 2023-10-10T20:31:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.688+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb404c3c0 con 0x7febb40cb280 2023-10-10T20:31:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.689+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feba8004ba0 con 0x7febb40cb280 2023-10-10T20:31:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.689+0000 7feb9b7fe700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 0x7feba0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.689+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7feba80839c0 con 0x7febb40cb280 2023-10-10T20:31:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.689+0000 7febb37fe700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 0x7feba0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.690+0000 7febb37fe700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 0x7feba0069410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feba40030c0 tx=0x7feba4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.694+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feba8050fb0 con 0x7febb40cb280 2023-10-10T20:31:15.880 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.878+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7febb40cbdf0 con 0x7febb40cb280 2023-10-10T20:31:15.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.882+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feba8050fb0 con 0x7febb40cb280 2023-10-10T20:31:15.884 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.882+0000 7feb9b7fe700 1 -- 172.21.15.148:0/935316842 <== mon.0 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7feba80180c0 con 0x7febb40cb280 2023-10-10T20:31:15.887 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 msgr2=0x7feba0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 0x7feba0069410 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feba40030c0 tx=0x7feba4016000 comp rx=0 tx=0).stop 2023-10-10T20:31:15.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 msgr2=0x7febb4153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:15.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7feba800cb70 tx=0x7feba8001b50 comp rx=0 tx=0).stop 2023-10-10T20:31:15.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 shutdown_connections 2023-10-10T20:31:15.888 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7feba0066f60 0x7feba0069410 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb4156160 0x7febb41544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb40c71d0 0x7febb40ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 --2- 172.21.15.148:0/935316842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb40cb280 0x7febb4153fa0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:15.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 >> 172.21.15.148:0/935316842 conn(0x7febb4067660 msgr2=0x7febb406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:15.889 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 shutdown_connections 2023-10-10T20:31:15.890 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:15.885+0000 7febba1a9700 1 -- 172.21.15.148:0/935316842 wait complete. 2023-10-10T20:31:15.899 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-10T20:31:15.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-10T20:31:16.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.011+0000 7f7cfe41d700 1 Processor -- start 2023-10-10T20:31:16.014 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 -- start start 2023-10-10T20:31:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 0x7f7cf80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7cf806ac30 0x7f7cf815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf815c360 con 0x7f7cf8157ba0 2023-10-10T20:31:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf815b790 con 0x7f7cf806ac30 2023-10-10T20:31:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.012+0000 7f7cfe41d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf815ba30 con 0x7f7cf81577d0 2023-10-10T20:31:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7cf806ac30 0x7f7cf815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf77fe700 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7cf8157ba0 0x7f7cf806a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59116/0 (socket says 172.21.15.148:59116) 2023-10-10T20:31:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf77fe700 1 -- 172.21.15.148:0/687670755 learned_addr learned my addr 172.21.15.148:0/687670755 (peer_addr_for_me v1:172.21.15.148:0/0) 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf7fff700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 0x7f7cf80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 -- 172.21.15.148:0/687670755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 msgr2=0x7f7cf80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 0x7f7cf80cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 -- 172.21.15.148:0/687670755 >> v1:172.21.15.148:6789/0 conn(0x7f7cf8157ba0 legacy=0x7f7cf806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 -- 172.21.15.148:0/687670755 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf815bd10 con 0x7f7cf806ac30 2023-10-10T20:31:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf6ffd700 1 -- 172.21.15.148:0/687670755 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616481532 0 0) 0x7f7cf815c360 con 0x7f7cf8157ba0 2023-10-10T20:31:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf7fff700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 0x7f7cf80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-10T20:31:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cfc9ba700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf806ac30 0x7f7cf815c7f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009e60 tx=0x7f7cf0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47bd5962a1ad964 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf6ffd700 1 -- 172.21.15.148:0/687670755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000e070 con 0x7f7cf806ac30 2023-10-10T20:31:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf6ffd700 1 -- 172.21.15.148:0/687670755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cf000bc30 con 0x7f7cf806ac30 2023-10-10T20:31:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.013+0000 7f7cf6ffd700 1 -- 172.21.15.148:0/687670755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0012430 con 0x7f7cf806ac30 2023-10-10T20:31:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 -- 172.21.15.148:0/687670755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf806ac30 msgr2=0x7f7cf815c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf806ac30 0x7f7cf815c7f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009e60 tx=0x7f7cf0009780 comp rx=0 tx=0).stop 2023-10-10T20:31:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 -- 172.21.15.148:0/687670755 shutdown_connections 2023-10-10T20:31:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf806ac30 0x7f7cf815c7f0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/687670755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf81577d0 0x7f7cf80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 -- 172.21.15.148:0/687670755 >> 172.21.15.148:0/687670755 conn(0x7f7cf8068ce0 msgr2=0x7f7cf80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 -- 172.21.15.148:0/687670755 shutdown_connections 2023-10-10T20:31:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.014+0000 7f7cfe41d700 1 -- 172.21.15.148:0/687670755 wait complete. 2023-10-10T20:31:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.015+0000 7f7cfe41d700 1 Processor -- start 2023-10-10T20:31:16.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.015+0000 7f7cfe41d700 1 -- start start 2023-10-10T20:31:16.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf806ac30 0x7f7cf80d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.025 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf8157ba0 0x7f7cf816a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf816d570 con 0x7f7cf81577d0 2023-10-10T20:31:16.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80a7a60 con 0x7f7cf8157ba0 2023-10-10T20:31:16.026 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80a7d40 con 0x7f7cf806ac30 2023-10-10T20:31:16.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53032/0 (socket says 172.21.15.148:53032) 2023-10-10T20:31:16.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 -- 172.21.15.148:0/1755357966 learned_addr learned my addr 172.21.15.148:0/1755357966 (peer_addr_for_me v2:172.21.15.148:0/0) 2023-10-10T20:31:16.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf7fff700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf806ac30 0x7f7cf80d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.027 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 -- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf806ac30 msgr2=0x7f7cf80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf806ac30 0x7f7cf80d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 -- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf8157ba0 msgr2=0x7f7cf816a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfc9ba700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf8157ba0 0x7f7cf816a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf8157ba0 0x7f7cf816a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 -- 172.21.15.148:0/1755357966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0009a20 con 0x7f7cf81577d0 2023-10-10T20:31:16.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cf77fe700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c7e0 tx=0x7f7cec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:16.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec005b30 con 0x7f7cf81577d0 2023-10-10T20:31:16.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cec005c90 con 0x7f7cf81577d0 2023-10-10T20:31:16.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec007300 con 0x7f7cf81577d0 2023-10-10T20:31:16.029 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.016+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf80a8080 con 0x7f7cf81577d0 2023-10-10T20:31:16.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.017+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf80a7610 con 0x7f7cf81577d0 2023-10-10T20:31:16.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.018+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80a8820 con 0x7f7cf81577d0 2023-10-10T20:31:16.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.018+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cec01b400 con 0x7f7cf81577d0 2023-10-10T20:31:16.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.018+0000 7f7cdffff700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 0x7f7ce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-10T20:31:16.030 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.018+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7cec083db0 con 0x7f7cf81577d0 2023-10-10T20:31:16.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.018+0000 7f7cf7fff700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 0x7f7ce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-10T20:31:16.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.019+0000 7f7cf7fff700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 0x7f7ce40693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006fd0 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-10T20:31:16.031 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.023+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cf80a8820 con 0x7f7cf81577d0 2023-10-10T20:31:16.246 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.244+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cf80a8820 con 0x7f7cf81577d0 2023-10-10T20:31:16.247 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.244+0000 7f7cdffff700 1 -- 172.21.15.148:0/1755357966 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f7cf80a8820 con 0x7f7cf81577d0 2023-10-10T20:31:16.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-10-10T20:31:16.247 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6834/3749207882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3749207882},{"type":"v1","addr":"172.21.15.161:6834","nonce":3749207882}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7663,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2812949456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6833","nonce":2812949456},{"type":"v1","addr":"172.21.15.161:6835","nonce":2812949456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":8127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2965003827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2965003827},{"type":"v1","addr":"172.21.15.148:6836","nonce":2965003827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8130,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2107444757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2107444757},{"type":"v1","addr":"172.21.15.148:6837","nonce":2107444757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[]} 2023-10-10T20:31:16.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 msgr2=0x7f7ce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 0x7f7ce40693e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006fd0 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2023-10-10T20:31:16.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 msgr2=0x7f7cf815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-10T20:31:16.249 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c7e0 tx=0x7f7cec007df0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 shutdown_connections 2023-10-10T20:31:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:6832/86081,v1:172.21.15.148:6833/86081] conn(0x7f7ce4066f30 0x7f7ce40693e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf8157ba0 0x7f7cf816a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cf806ac30 0x7f7cf80d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 --2- 172.21.15.148:0/1755357966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cf81577d0 0x7f7cf815d240 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-10T20:31:16.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 >> 172.21.15.148:0/1755357966 conn(0x7f7cf8068ce0 msgr2=0x7f7cf80b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-10T20:31:16.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 shutdown_connections 2023-10-10T20:31:16.251 INFO:teuthology.orchestra.run.smithi148.stderr:2023-10-10T20:31:16.246+0000 7f7cfe41d700 1 -- 172.21.15.148:0/1755357966 wait complete. 2023-10-10T20:31:16.251 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2023-10-10T20:31:16.260 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-10T20:31:16.261 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-10T20:31:16.261 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-10T20:31:16.261 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.269 INFO:tasks.ceph.mds.a:Stopped 2023-10-10T20:31:16.270 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-10T20:31:16.270 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.318 INFO:tasks.ceph.mds.c:Stopped 2023-10-10T20:31:16.318 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-10T20:31:16.318 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.329 INFO:tasks.ceph.mds.b:Stopped 2023-10-10T20:31:16.329 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-10T20:31:16.330 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.380 INFO:tasks.ceph.mds.d:Stopped 2023-10-10T20:31:16.380 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-10T20:31:16.381 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-10T20:31:16.381 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.401 INFO:tasks.ceph.osd.0:Stopped 2023-10-10T20:31:16.402 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-10T20:31:16.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.464 INFO:tasks.ceph.osd.1:Stopped 2023-10-10T20:31:16.465 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-10T20:31:16.465 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.530 INFO:tasks.ceph.osd.2:Stopped 2023-10-10T20:31:16.530 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-10T20:31:16.531 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.590 INFO:tasks.ceph.osd.3:Stopped 2023-10-10T20:31:16.590 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-10T20:31:16.590 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.613 INFO:tasks.ceph.osd.4:Stopped 2023-10-10T20:31:16.613 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-10T20:31:16.614 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.672 INFO:tasks.ceph.osd.5:Stopped 2023-10-10T20:31:16.673 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-10T20:31:16.673 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.739 INFO:tasks.ceph.osd.6:Stopped 2023-10-10T20:31:16.739 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-10T20:31:16.739 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.800 INFO:tasks.ceph.osd.7:Stopped 2023-10-10T20:31:16.801 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-10T20:31:16.801 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-10T20:31:16.801 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.826 INFO:tasks.ceph.mgr.y:Stopped 2023-10-10T20:31:16.826 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-10T20:31:16.826 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.864 INFO:tasks.ceph.mgr.x:Stopped 2023-10-10T20:31:16.865 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-10T20:31:16.865 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-10T20:31:16.866 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.877 INFO:tasks.ceph.mon.a:Stopped 2023-10-10T20:31:16.878 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-10T20:31:16.878 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.916 INFO:tasks.ceph.mon.b:Stopped 2023-10-10T20:31:16.917 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-10T20:31:16.917 INFO:teuthology.orchestra.run:waiting for 300 2023-10-10T20:31:16.968 INFO:tasks.ceph.mon.c:Stopped 2023-10-10T20:31:16.968 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-10T20:31:16.969 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-10T20:31:16.999 INFO:teuthology.orchestra.run.smithi148.stdout:1696969150.205412 osd.0 (osd.0) 3 : cluster [WRN] OSD bench result of 232801.263459 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.0. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-10T20:31:17.000 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-10T20:31:17.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-10T20:31:17.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-10T20:31:17.140 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-10T20:31:17.211 INFO:teuthology.orchestra.run.smithi148.stdout:1696969150.205412 osd.0 (osd.0) 3 : cluster [WRN] OSD bench result of 232801.263459 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.0. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-10T20:31:17.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:31:17.211 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-10T20:31:17.422 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:31:17.423 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-10T20:31:17.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:31:17.529 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-10T20:31:17.616 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:31:17.616 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-10T20:31:17.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:31:17.683 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-10T20:31:17.861 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:31:17.862 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-10T20:31:17.967 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:31:17.968 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-10T20:31:18.041 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:31:18.042 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-10T20:31:18.135 INFO:tasks.ceph:Archiving mon data... 2023-10-10T20:31:18.137 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/data/mon.a.tgz 2023-10-10T20:31:18.137 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2023-10-10T20:31:18.152 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.IB8k6MULvR 2023-10-10T20:31:18.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IB8k6MULvR 2023-10-10T20:31:18.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.IB8k6MULvR 2023-10-10T20:31:18.382 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.IB8k6MULvR is 375KB 2023-10-10T20:31:18.478 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.IB8k6MULvR 2023-10-10T20:31:18.493 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/data/mon.b.tgz 2023-10-10T20:31:18.494 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-10-10T20:31:18.510 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.m0ioG6V6SI 2023-10-10T20:31:18.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.m0ioG6V6SI 2023-10-10T20:31:18.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.m0ioG6V6SI 2023-10-10T20:31:18.742 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.m0ioG6V6SI is 380KB 2023-10-10T20:31:18.838 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.m0ioG6V6SI 2023-10-10T20:31:18.852 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/data/mon.c.tgz 2023-10-10T20:31:18.852 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-10-10T20:31:18.905 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.8u552EHnLW 2023-10-10T20:31:18.906 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8u552EHnLW 2023-10-10T20:31:19.038 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.8u552EHnLW 2023-10-10T20:31:19.120 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.8u552EHnLW is 380KB 2023-10-10T20:31:19.216 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.8u552EHnLW 2023-10-10T20:31:19.230 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-10T20:31:19.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-10T20:31:19.234 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-10T20:31:19.295 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-10T20:31:19.295 DEBUG:teuthology.orchestra.run.smithi148:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-10T20:31:19.299 DEBUG:teuthology.orchestra.run.smithi161:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-10T20:31:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-10T20:31:19.359 INFO:teuthology.orchestra.run.smithi161.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-10T20:31:19.365 INFO:tasks.ceph:Archiving crash dumps... 2023-10-10T20:31:19.367 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi148/crash 2023-10-10T20:31:19.368 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-10T20:31:19.392 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi161/crash 2023-10-10T20:31:19.394 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-10T20:31:19.428 INFO:tasks.ceph:Shutting down logrotate 2023-10-10T20:31:19.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:31:19.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-10T20:31:19.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-10T20:31:19.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-10T20:31:19.565 INFO:tasks.ceph:Compressing logs... 2023-10-10T20:31:19.566 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-10T20:31:19.585 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-10T20:31:19.648 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.87755.log 2023-10-10T20:31:19.648 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86410.log 2023-10-10T20:31:19.648 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.3.87755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88164.log 2023-10-10T20:31:19.649 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.87755.log.gz 2023-10-10T20:31:19.649 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86282.log 2023-10-10T20:31:19.649 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.88164.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86410.log.gz 2023-10-10T20:31:19.649 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87389.log 2023-10-10T20:31:19.649 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88164.log.gz 2023-10-10T20:31:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89481.log 2023-10-10T20:31:19.650 INFO:teuthology.orchestra.run.smithi161.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86282.log.gz 2023-10-10T20:31:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.87389.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-10T20:31:19.650 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87389.log.gz 2023-10-10T20:31:19.650 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90731.log 2023-10-10T20:31:19.651 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89481.log.gz 2023-10-10T20:31:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88608.log 2023-10-10T20:31:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.90731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86156.log 2023-10-10T20:31:19.651 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90731.log.gz 2023-10-10T20:31:19.651 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.88608.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88608.log.gz 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86241.log 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86367.log 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86156.log.gz 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86241.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-10T20:31:19.652 INFO:teuthology.orchestra.run.smithi161.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86241.log.gz 2023-10-10T20:31:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88168.log 2023-10-10T20:31:19.653 INFO:teuthology.orchestra.run.smithi161.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86367.log.gz 2023-10-10T20:31:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-10T20:31:19.653 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.1.88168.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-10T20:31:19.653 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88168.log.gz 2023-10-10T20:31:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-10T20:31:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86199.log 2023-10-10T20:31:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.86199.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-10T20:31:19.654 INFO:teuthology.orchestra.run.smithi161.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86199.log.gz 2023-10-10T20:31:19.654 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90351.log 2023-10-10T20:31:19.655 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-10T20:31:19.655 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.90351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90351.log.gz 2023-10-10T20:31:19.655 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mon.c.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-10T20:31:19.655 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89485.log 2023-10-10T20:31:19.655 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86454.log 2023-10-10T20:31:19.656 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86324.log 2023-10-10T20:31:19.656 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89485.log.gz 2023-10-10T20:31:19.656 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89076.log 2023-10-10T20:31:19.656 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.86324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86454.log.gz 2023-10-10T20:31:19.656 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89076.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86324.log.gz 2023-10-10T20:31:19.657 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89076.log.gz 2023-10-10T20:31:19.657 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90735.log 2023-10-10T20:31:19.657 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-10T20:31:19.657 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.90735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.87759.log 2023-10-10T20:31:19.657 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90735.log.gz 2023-10-10T20:31:19.658 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91174.log 2023-10-10T20:31:19.658 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.3.87759.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-10T20:31:19.658 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.87759.log.gz 2023-10-10T20:31:19.658 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.91174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91174.log.gz 2023-10-10T20:31:19.658 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92103.log 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89884.log 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.92103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92103.log.gz 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89884.log.gz 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89072.log 2023-10-10T20:31:19.659 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91644.log 2023-10-10T20:31:19.660 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89072.log.gz 2023-10-10T20:31:19.660 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.91644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91644.log.gz 2023-10-10T20:31:19.660 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91178.log 2023-10-10T20:31:19.660 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-10T20:31:19.660 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.91178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91178.log.gz 2023-10-10T20:31:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89880.log 2023-10-10T20:31:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.89880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89880.log.gz 2023-10-10T20:31:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91640.log 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.91640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.91640.log.gz 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87397.log 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.87397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87397.log.gz 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92099.log 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.92099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92099.log.gz 2023-10-10T20:31:19.665 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-10T20:31:19.666 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90347.log 2023-10-10T20:31:19.676 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.90347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90347.log.gz 2023-10-10T20:31:19.689 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88604.log 2023-10-10T20:31:19.696 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.88604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88604.log.gz 2023-10-10T20:31:19.746 INFO:teuthology.orchestra.run.smithi161.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-10T20:31:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92842.log 2023-10-10T20:31:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107914.log 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111664.log 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107914.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92842.log.gzgzip 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:-5 --verbose -- /var/log/ceph/ceph-client.0.96129.log 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111664.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107914.log.gz 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92758.log 2023-10-10T20:31:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.96129.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111664.log.gz 2023-10-10T20:31:19.796 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96129.log.gzgzip 2023-10-10T20:31:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.104404.log 2023-10-10T20:31:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118559.log 2023-10-10T20:31:19.796 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104740.log 2023-10-10T20:31:19.796 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92758.log.gz 2023-10-10T20:31:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118559.log: gzip -5 --verbose 88.2%-- /var/log/ceph/ceph-client.admin.106530.log -- replaced with /var/log/ceph/ceph-client.admin.104404.log.gz 2023-10-10T20:31:19.797 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:19.797 INFO:teuthology.orchestra.run.smithi148.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118559.log.gz 2023-10-10T20:31:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113359.log 2023-10-10T20:31:19.797 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108608.log 2023-10-10T20:31:19.798 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104740.log.gz 2023-10-10T20:31:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113359.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106530.log.gz 2023-10-10T20:31:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110794.log 2023-10-10T20:31:19.798 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.113359.log.gz 2023-10-10T20:31:19.798 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87902.log 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103130.log 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87902.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110794.log.gz 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105983.log 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108608.log.gz 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87902.log.gz 2023-10-10T20:31:19.799 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89330.log 2023-10-10T20:31:19.800 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103130.log.gz 2023-10-10T20:31:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90826.log 2023-10-10T20:31:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89330.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105983.log.gz 2023-10-10T20:31:19.800 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112713.log 2023-10-10T20:31:19.800 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89330.log.gz 2023-10-10T20:31:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116846.log 2023-10-10T20:31:19.801 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6%/var/log/ceph/ceph-client.admin.112713.log: -- replaced with /var/log/ceph/ceph-client.admin.90826.log.gz 2023-10-10T20:31:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100270.log 2023-10-10T20:31:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116846.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112713.log.gz 2023-10-10T20:31:19.801 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94658.log 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100270.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116846.log.gz 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91794.log 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7%/var/log/ceph/ceph-client.admin.94658.log: -- replaced with /var/log/ceph/ceph-client.admin.100270.log.gz 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94490.log 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91794.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94658.log.gz 2023-10-10T20:31:19.802 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107515.log 2023-10-10T20:31:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94490.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91794.log.gz 2023-10-10T20:31:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114801.log 2023-10-10T20:31:19.803 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94490.log.gz 2023-10-10T20:31:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94742.log 2023-10-10T20:31:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111726.log 2023-10-10T20:31:19.804 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107515.log.gz 2023-10-10T20:31:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94742.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114801.log.gz 2023-10-10T20:31:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105707.log 2023-10-10T20:31:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111726.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94742.log.gz 2023-10-10T20:31:19.804 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87041.log 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111726.log.gz 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97259.log 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88994.log 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105707.log.gz 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87041.log.gz 2023-10-10T20:31:19.805 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105214.log 2023-10-10T20:31:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102537.log 2023-10-10T20:31:19.806 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97259.log.gz 2023-10-10T20:31:19.806 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88994.log.gz/var/log/ceph/ceph-client.admin.105214.log: 2023-10-10T20:31:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99714.log 2023-10-10T20:31:19.806 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102537.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.105214.log.gz 2023-10-10T20:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104446.log 2023-10-10T20:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0%/var/log/ceph/ceph-client.admin.99714.log: -- replaced with /var/log/ceph/ceph-client.admin.102537.log.gz 2023-10-10T20:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102086.log 2023-10-10T20:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105495.log 2023-10-10T20:31:19.807 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99714.log.gz 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91668.log 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104446.log.gz 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90868.log 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102086.log.gz 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91668.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105495.log.gz 2023-10-10T20:31:19.808 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115363.log 2023-10-10T20:31:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90868.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91668.log.gz 2023-10-10T20:31:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98380.log 2023-10-10T20:31:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115363.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.90868.log.gz 2023-10-10T20:31:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109646.log 2023-10-10T20:31:19.809 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98380.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115363.log.gz 2023-10-10T20:31:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98168.log 2023-10-10T20:31:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109646.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98380.log.gz 2023-10-10T20:31:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112460.log 2023-10-10T20:31:19.810 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109646.log.gz 2023-10-10T20:31:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96727.log 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112460.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98168.log.gz 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106263.log 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96727.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112460.log.gz 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94700.log 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106263.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96727.log.gzgzip 2023-10-10T20:31:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.95236.log 2023-10-10T20:31:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94700.log: 88.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.106263.log.gz 2023-10-10T20:31:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.90910.log 2023-10-10T20:31:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95492.log 2023-10-10T20:31:19.812 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94700.log.gz 2023-10-10T20:31:19.812 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92968.log 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95236.log.gz 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95492.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90910.log.gz 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115745.log 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95492.log.gz/var/log/ceph/ceph-client.admin.92968.log: 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116678.log 2023-10-10T20:31:19.813 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8%/var/log/ceph/ceph-client.admin.115745.log: -- replaced with /var/log/ceph/ceph-client.admin.92968.log.gz 2023-10-10T20:31:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-10T20:31:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110234.log 2023-10-10T20:31:19.814 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115745.log.gz 2023-10-10T20:31:19.814 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116678.log.gz 2023-10-10T20:31:19.814 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109175.log 2023-10-10T20:31:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87608.log 2023-10-10T20:31:19.815 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110234.log.gz 2023-10-10T20:31:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97581.log 2023-10-10T20:31:19.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98254.log 88.3% 2023-10-10T20:31:19.815 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109175.log.gz 2023-10-10T20:31:19.816 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87608.log.gz 2023-10-10T20:31:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87083.log 2023-10-10T20:31:19.816 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97581.log.gz 2023-10-10T20:31:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115577.log 2023-10-10T20:31:19.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91878.log 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98254.log.gz 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87083.log.gz 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108398.log 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115577.log.gz 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97469.log 2023-10-10T20:31:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108398.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91878.log.gz 2023-10-10T20:31:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109534.log 2023-10-10T20:31:19.818 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8%/var/log/ceph/ceph-client.admin.97469.log: -- replaced with /var/log/ceph/ceph-client.admin.108398.log.gz 2023-10-10T20:31:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113401.log 2023-10-10T20:31:19.818 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104038.log 2023-10-10T20:31:19.818 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97469.log.gz 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113401.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109534.log.gz 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103863.log 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113401.log.gz 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110582.log 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103863.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104038.log.gz 2023-10-10T20:31:19.819 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102836.log 2023-10-10T20:31:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110582.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103863.log.gz 2023-10-10T20:31:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96427.log 2023-10-10T20:31:19.820 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110582.log.gz 2023-10-10T20:31:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115493.log 2023-10-10T20:31:19.820 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118206.log 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102836.log.gz 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115493.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96427.log.gz 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116804.log 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115493.log.gz 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103256.log 2023-10-10T20:31:19.821 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118080.log 2023-10-10T20:31:19.822 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118206.log.gz 2023-10-10T20:31:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103256.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116804.log.gz 2023-10-10T20:31:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102579.log 2023-10-10T20:31:19.822 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117996.log 2023-10-10T20:31:19.822 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.103256.log.gz 2023-10-10T20:31:19.823 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118080.log.gz 2023-10-10T20:31:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100354.log 2023-10-10T20:31:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86042.log 2023-10-10T20:31:19.823 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102579.log.gz 2023-10-10T20:31:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113317.log 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100354.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117996.log.gz 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100354.log.gz 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94992.log 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113317.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86042.log.gz 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93566.log 2023-10-10T20:31:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94992.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113317.log.gz 2023-10-10T20:31:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116176.log 2023-10-10T20:31:19.825 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94992.log.gz 2023-10-10T20:31:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105749.log 2023-10-10T20:31:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.116176.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.93566.log.gz-5 2023-10-10T20:31:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.110422.log 2023-10-10T20:31:19.826 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116176.log.gz 2023-10-10T20:31:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102664.log 2023-10-10T20:31:19.826 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105749.log.gz 2023-10-10T20:31:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111016.log 2023-10-10T20:31:19.826 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102664.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110422.log.gz 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109275.log 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111016.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.102664.log.gz-5 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.104530.log 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111016.log.gz 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93286.log 2023-10-10T20:31:19.827 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109275.log.gz 2023-10-10T20:31:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105623.log 2023-10-10T20:31:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93286.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.104530.log.gz--verbose 2023-10-10T20:31:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:-- /var/log/ceph/ceph-client.admin.107998.log 2023-10-10T20:31:19.828 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93286.log.gz 2023-10-10T20:31:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101647.log 2023-10-10T20:31:19.829 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105623.log.gz 2023-10-10T20:31:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91137.log 2023-10-10T20:31:19.829 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101647.log: gzip -5 89.5%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.107998.log.gz/var/log/ceph/ceph-client.admin.95070.log 2023-10-10T20:31:19.829 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:19.829 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%/var/log/ceph/ceph-client.admin.91137.log: -- replaced with /var/log/ceph/ceph-client.admin.101647.log.gz 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93178.log 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91137.log.gz 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100188.log 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93178.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95070.log.gz 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102378.log 2023-10-10T20:31:19.830 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93178.log.gz 2023-10-10T20:31:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115979.log 2023-10-10T20:31:19.831 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100188.log.gz 2023-10-10T20:31:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108524.log 2023-10-10T20:31:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.90533.log 2023-10-10T20:31:19.831 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102378.log.gz 2023-10-10T20:31:19.832 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115979.log.gz 2023-10-10T20:31:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101941.log 2023-10-10T20:31:19.832 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108524.log.gz 2023-10-10T20:31:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.2.90533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102878.log 2023-10-10T20:31:19.832 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.90533.log.gz 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116562.log 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101941.log.gz 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106908.log 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116562.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.102878.log.gz 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111384.log 2023-10-10T20:31:19.833 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95194.log 2023-10-10T20:31:19.834 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116562.log.gz 2023-10-10T20:31:19.834 INFO:teuthology.orchestra.run.smithi148.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106908.log.gz 2023-10-10T20:31:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97007.log 2023-10-10T20:31:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95194.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111384.log.gz 2023-10-10T20:31:19.834 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112775.log 2023-10-10T20:31:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97007.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95194.log.gz 2023-10-10T20:31:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104614.log 2023-10-10T20:31:19.835 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97007.log.gz 2023-10-10T20:31:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86825.log 2023-10-10T20:31:19.835 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112775.log.gz 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91266.log 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86825.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104614.log.gz 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107202.log 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86825.log.gz 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116762.log 2023-10-10T20:31:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107202.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91266.log.gz 2023-10-10T20:31:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88730.log 2023-10-10T20:31:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116762.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107202.log.gz 2023-10-10T20:31:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89582.log 2023-10-10T20:31:19.837 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116762.log.gz 2023-10-10T20:31:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117463.log 2023-10-10T20:31:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89582.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88730.log.gz 2023-10-10T20:31:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108272.log 2023-10-10T20:31:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86314.log 2023-10-10T20:31:19.838 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3%/var/log/ceph/ceph-client.admin.108272.log: -- replaced with /var/log/ceph/ceph-client.admin.117463.log.gz 2023-10-10T20:31:19.838 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 91.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.89582.log.gz--verbose 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:-- /var/log/ceph/ceph-client.admin.102621.log 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86314.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108272.log.gz 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115535.log 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86314.log.gz 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114658.log 2023-10-10T20:31:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115031.log 2023-10-10T20:31:19.840 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102621.log.gz 2023-10-10T20:31:19.840 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114658.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115535.log.gz 2023-10-10T20:31:19.840 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97133.log 2023-10-10T20:31:19.840 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114658.log.gz 2023-10-10T20:31:19.840 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98032.log 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115031.log: /var/log/ceph/ceph-client.admin.97133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116720.log 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115031.log.gz 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98032.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97133.log.gz 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91962.log 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98032.log.gz 2023-10-10T20:31:19.841 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88724.log 2023-10-10T20:31:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91962.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116720.log.gz 2023-10-10T20:31:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92291.log 2023-10-10T20:31:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88724.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91962.log.gz 2023-10-10T20:31:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104908.log 2023-10-10T20:31:19.842 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88724.log.gz 2023-10-10T20:31:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114927.log 2023-10-10T20:31:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104908.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92291.log.gz 2023-10-10T20:31:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94406.log 2023-10-10T20:31:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114927.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104908.log.gz 2023-10-10T20:31:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112274.log 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114927.log.gz 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103602.log 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112274.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94406.log.gz 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103821.log 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103602.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112274.log.gz 2023-10-10T20:31:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104656.log 2023-10-10T20:31:19.845 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103602.log.gz 2023-10-10T20:31:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102254.log 2023-10-10T20:31:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104656.log: 88.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.103821.log.gz--verbose 2023-10-10T20:31:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:-- /var/log/ceph/ceph-client.admin.98464.log 2023-10-10T20:31:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97904.log 2023-10-10T20:31:19.846 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104656.log.gz 2023-10-10T20:31:19.846 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102254.log.gz 2023-10-10T20:31:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98126.log 2023-10-10T20:31:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97904.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98464.log.gz 2023-10-10T20:31:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95828.log 2023-10-10T20:31:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98126.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97904.log.gz 2023-10-10T20:31:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96469.log 2023-10-10T20:31:19.847 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98126.log.gz 2023-10-10T20:31:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101068.log 2023-10-10T20:31:19.847 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96469.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95828.log.gz 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99630.log 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101068.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96469.log.gz 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115913.log 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101068.log.gz 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107956.log 2023-10-10T20:31:19.848 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115913.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99630.log.gz 2023-10-10T20:31:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103704.log 2023-10-10T20:31:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107118.log 2023-10-10T20:31:19.849 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107956.log.gz 2023-10-10T20:31:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103704.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.115913.log.gz 2023-10-10T20:31:19.849 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108146.log 2023-10-10T20:31:19.850 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103704.log.gz 2023-10-10T20:31:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99546.log 2023-10-10T20:31:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108146.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107118.log.gz 2023-10-10T20:31:19.850 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113103.log 2023-10-10T20:31:19.850 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108146.log.gz 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110624.log 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113103.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99546.log.gz 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111058.log 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110624.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113103.log.gz 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92168.log 2023-10-10T20:31:19.851 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110624.log.gz 2023-10-10T20:31:19.852 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105357.log 2023-10-10T20:31:19.852 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111058.log.gz 2023-10-10T20:31:19.852 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92502.log 2023-10-10T20:31:19.852 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105357.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92168.log.gz 2023-10-10T20:31:19.852 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102316.log 2023-10-10T20:31:19.853 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105357.log.gz 2023-10-10T20:31:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91078.log 2023-10-10T20:31:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102316.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92502.log.gz 2023-10-10T20:31:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91224.log 2023-10-10T20:31:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91078.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102316.log.gz 2023-10-10T20:31:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98979.log 2023-10-10T20:31:19.854 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91078.log.gz 2023-10-10T20:31:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104698.log 2023-10-10T20:31:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.98979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87524.log 2023-10-10T20:31:19.854 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91224.log.gz 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98979.log.gz 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95278.log 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87524.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104698.log.gz 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87524.log.gz 2023-10-10T20:31:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103004.log 2023-10-10T20:31:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95278.log.gz 2023-10-10T20:31:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91350.log 2023-10-10T20:31:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94784.log 2023-10-10T20:31:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91350.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103004.log.gz 2023-10-10T20:31:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91584.log 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91350.log.gz 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95870.log 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99672.log 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91584.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94784.log.gz 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109233.log 2023-10-10T20:31:19.857 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91584.log.gz 2023-10-10T20:31:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109391.log 2023-10-10T20:31:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99672.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.95870.log.gz 2023-10-10T20:31:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95534.log 2023-10-10T20:31:19.858 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.99672.log.gz 2023-10-10T20:31:19.858 INFO:teuthology.orchestra.run.smithi148.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-10T20:31:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88721.log 2023-10-10T20:31:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109391.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109233.log.gz 2023-10-10T20:31:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92628.log 2023-10-10T20:31:19.859 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109391.log.gz 2023-10-10T20:31:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88721.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95534.log.gz 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117253.log 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88721.log.gz 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114000.log 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92628.log.gz 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115237.log 2023-10-10T20:31:19.860 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115279.log 2023-10-10T20:31:19.861 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117253.log.gz 2023-10-10T20:31:19.861 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114000.log.gz 2023-10-10T20:31:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97988.log 2023-10-10T20:31:19.861 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115237.log.gz 2023-10-10T20:31:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113912.log 2023-10-10T20:31:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97988.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115279.log.gzgzip 2023-10-10T20:31:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:-5 --verbose -- /var/log/ceph/ceph-client.0.101273.log 2023-10-10T20:31:19.862 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97988.log.gz 2023-10-10T20:31:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.113912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112400.log 2023-10-10T20:31:19.862 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113912.log.gz 2023-10-10T20:31:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.101273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97217.log 2023-10-10T20:31:19.863 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101273.log.gz 2023-10-10T20:31:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106221.log 2023-10-10T20:31:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97217.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112400.log.gz 2023-10-10T20:31:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87860.log 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108867.log 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106221.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97217.log.gz 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90616.log 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.106221.log.gz 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87860.log.gz 2023-10-10T20:31:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.108867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87650.log 2023-10-10T20:31:19.865 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108867.log.gz 2023-10-10T20:31:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103515.log 2023-10-10T20:31:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96133.log 2023-10-10T20:31:19.865 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90616.log.gz 2023-10-10T20:31:19.865 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87650.log.gz 2023-10-10T20:31:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.103515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114490.log 2023-10-10T20:31:19.866 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103515.log.gz 2023-10-10T20:31:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.96133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102170.log 2023-10-10T20:31:19.866 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96133.log.gz 2023-10-10T20:31:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111318.log 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102170.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114490.log.gz 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94532.log 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102170.log.gz 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107244.log 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112853.log 2023-10-10T20:31:19.867 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94532.log.gz 2023-10-10T20:31:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92376.log 2023-10-10T20:31:19.868 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.111318.log.gz 2023-10-10T20:31:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99378.log 2023-10-10T20:31:19.868 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107244.log.gz 2023-10-10T20:31:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112853.log: /var/log/ceph/ceph-client.admin.92376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87211.log 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112853.log.gz 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1%/var/log/ceph/ceph-client.admin.99378.log: -- replaced with /var/log/ceph/ceph-client.admin.92376.log.gz 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107704.log 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87211.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99378.log.gz 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.90529.log 2023-10-10T20:31:19.869 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107704.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87211.log.gz 2023-10-10T20:31:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108040.log 2023-10-10T20:31:19.870 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107704.log.gz 2023-10-10T20:31:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.2.90529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108482.log 2023-10-10T20:31:19.870 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.90529.log.gz 2023-10-10T20:31:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98076.log 2023-10-10T20:31:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108482.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108040.log.gz 2023-10-10T20:31:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87356.log 2023-10-10T20:31:19.871 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108482.log.gz 2023-10-10T20:31:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104950.log 2023-10-10T20:31:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87356.log: gzip -5 --verbose -- 89.3%/var/log/ceph/ceph-client.admin.93094.log 2023-10-10T20:31:19.872 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98076.log.gz 2023-10-10T20:31:19.872 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104950.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87356.log.gz 2023-10-10T20:31:19.872 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99588.log 2023-10-10T20:31:19.872 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104950.log.gz 2023-10-10T20:31:19.872 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114238.log 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99588.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.93094.log.gz-5 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.96601.log 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114238.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99588.log.gz 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100664.log 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114238.log.gz 2023-10-10T20:31:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115703.log 2023-10-10T20:31:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96601.log.gz 2023-10-10T20:31:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107388.log 2023-10-10T20:31:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115703.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100664.log.gz 2023-10-10T20:31:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106992.log 2023-10-10T20:31:19.874 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115703.log.gz 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117043.log 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97862.log 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107388.log.gz 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117043.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106992.log.gz 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105833.log 2023-10-10T20:31:19.875 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117043.log.gz 2023-10-10T20:31:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100397.log 2023-10-10T20:31:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95702.log 2023-10-10T20:31:19.876 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97862.log.gz 2023-10-10T20:31:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100397.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.105833.log.gz-5 2023-10-10T20:31:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.96369.log 2023-10-10T20:31:19.877 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100397.log.gz 2023-10-10T20:31:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100622.log 2023-10-10T20:31:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96369.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95702.log.gz 2023-10-10T20:31:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115661.log 2023-10-10T20:31:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100622.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96369.log.gz 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90658.log 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100622.log.gz 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115661.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90658.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115661.log.gz 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116959.log 2023-10-10T20:31:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90658.log.gz 2023-10-10T20:31:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118416.log 2023-10-10T20:31:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109898.log 2023-10-10T20:31:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118416.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116959.log.gz 2023-10-10T20:31:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103996.log 2023-10-10T20:31:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86948.log 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109898.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118416.log.gz 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88409.log 88.4% 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109898.log.gz 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103996.log.gz 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108440.log 2023-10-10T20:31:19.880 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86948.log: /var/log/ceph/ceph-client.admin.88409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88070.log 2023-10-10T20:31:19.881 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86948.log.gz 2023-10-10T20:31:19.881 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114280.log 2023-10-10T20:31:19.881 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88409.log.gz 2023-10-10T20:31:19.881 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108440.log.gz 2023-10-10T20:31:19.881 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88073.log 2023-10-10T20:31:19.882 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-10T20:31:19.882 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88070.log.gz 2023-10-10T20:31:19.882 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95576.log 2023-10-10T20:31:19.882 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114280.log: gzip -5 --verbose 88.9%-- -- replaced with /var/log/ceph/ceph-client.admin.114280.log.gz/var/log/ceph/ceph-client.admin.108997.log 2023-10-10T20:31:19.882 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:19.883 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88073.log: /var/log/ceph/ceph-client.admin.95576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99504.log 2023-10-10T20:31:19.883 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88073.log.gz 2023-10-10T20:31:19.883 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108997.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95576.log.gz 2023-10-10T20:31:19.883 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112148.log 2023-10-10T20:31:19.883 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108997.log.gz 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115871.log 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112148.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99504.log.gz 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100110.log 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115871.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112148.log.gz 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96965.log 2023-10-10T20:31:19.884 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115871.log.gz 2023-10-10T20:31:19.885 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98716.log 2023-10-10T20:31:19.885 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96965.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100110.log.gz 2023-10-10T20:31:19.885 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109133.log 2023-10-10T20:31:19.885 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1%/var/log/ceph/ceph-client.admin.98716.log: -- replaced with /var/log/ceph/ceph-client.admin.96965.log.gz 2023-10-10T20:31:19.885 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117870.log 2023-10-10T20:31:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100312.log 2023-10-10T20:31:19.886 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.98716.log.gz 2023-10-10T20:31:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117870.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109133.log.gz 2023-10-10T20:31:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114532.log 2023-10-10T20:31:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108356.log 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100312.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117870.log.gz 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112064.log 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100312.log.gz 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108356.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114532.log.gz 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118770.log 2023-10-10T20:31:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112064.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108356.log.gz 2023-10-10T20:31:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87776.log 2023-10-10T20:31:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118770.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112064.log.gz 2023-10-10T20:31:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112358.log 2023-10-10T20:31:19.888 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118770.log.gz 2023-10-10T20:31:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87566.log 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99020.log 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87776.log.gz 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87566.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112358.log.gz 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103322.log 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87566.log.gz 2023-10-10T20:31:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100876.log 2023-10-10T20:31:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103322.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99020.log.gz 2023-10-10T20:31:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97301.log 2023-10-10T20:31:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100876.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103322.log.gz 2023-10-10T20:31:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101314.log 2023-10-10T20:31:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97301.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100876.log.gz 2023-10-10T20:31:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89204.log 2023-10-10T20:31:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101314.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.97301.log.gz-5 2023-10-10T20:31:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.93608.log 2023-10-10T20:31:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89204.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101314.log.gz 2023-10-10T20:31:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98590.log 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89204.log.gz 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115451.log 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98590.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.93608.log.gz--verbose 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:-- /var/log/ceph/ceph-client.admin.88751.log 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98590.log.gz 2023-10-10T20:31:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91036.log 2023-10-10T20:31:19.893 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115451.log.gz 2023-10-10T20:31:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117379.log 2023-10-10T20:31:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91036.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88751.log.gz 2023-10-10T20:31:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112316.log 2023-10-10T20:31:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117379.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91036.log.gz 2023-10-10T20:31:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95408.log 2023-10-10T20:31:19.894 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117379.log.gz 2023-10-10T20:31:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110750.log /var/log/ceph/ceph-client.admin.112316.log: 2023-10-10T20:31:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95408.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-10T20:31:19.894 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112316.log.gz 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110750.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95408.log.gz 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105256.log 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110750.log.gz 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110192.log 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91542.log 2023-10-10T20:31:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105256.log.gz 2023-10-10T20:31:19.896 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108674.log 2023-10-10T20:31:19.896 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110192.log.gz 2023-10-10T20:31:19.896 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118332.log 2023-10-10T20:31:19.896 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108674.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91542.log.gz 2023-10-10T20:31:19.896 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117127.log 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108674.log.gz 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90952.log 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117127.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118332.log.gz 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112503.log 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117127.log.gz 2023-10-10T20:31:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113916.log 2023-10-10T20:31:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100708.log 2023-10-10T20:31:19.898 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90952.log.gz 2023-10-10T20:31:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112503.log: /var/log/ceph/ceph-client.0.113916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115109.log 2023-10-10T20:31:19.898 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112503.log.gz 2023-10-10T20:31:19.898 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113916.log.gz 2023-10-10T20:31:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112935.log 2023-10-10T20:31:19.899 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100708.log.gz 2023-10-10T20:31:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96174.log 2023-10-10T20:31:19.899 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115109.log.gz 2023-10-10T20:31:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97946.log 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112935.log: /var/log/ceph/ceph-client.admin.96174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99817.log 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112935.log.gz 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102794.log 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97946.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96174.log.gz 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99817.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97946.log.gz 2023-10-10T20:31:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101983.log 2023-10-10T20:31:19.901 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113527.log 2023-10-10T20:31:19.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102794.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99817.log.gz 2023-10-10T20:31:19.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114843.log 2023-10-10T20:31:19.901 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102794.log.gz 2023-10-10T20:31:19.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94322.log 2023-10-10T20:31:19.902 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101983.log.gz 2023-10-10T20:31:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114843.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113527.log.gz 2023-10-10T20:31:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101605.log 2023-10-10T20:31:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90129.log 2023-10-10T20:31:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94322.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114843.log.gz 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89498.log 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94322.log.gz 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110066.log 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.90129.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101605.log.gz 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90129.log.gz 2023-10-10T20:31:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91920.log 2023-10-10T20:31:19.904 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107076.log 2023-10-10T20:31:19.904 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89498.log.gz 2023-10-10T20:31:19.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110066.log: /var/log/ceph/ceph-client.admin.91920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99462.log 2023-10-10T20:31:19.904 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110066.log.gz 2023-10-10T20:31:19.904 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88074.log 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107076.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.91920.log.gz 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99462.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107076.log.gz 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107328.log 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100580.log 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99462.log.gz 2023-10-10T20:31:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88074.log: /var/log/ceph/ceph-client.admin.107328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91836.log 2023-10-10T20:31:19.906 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88074.log.gz 2023-10-10T20:31:19.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95450.log 2023-10-10T20:31:19.906 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%/var/log/ceph/ceph-client.admin.100580.log: -- replaced with /var/log/ceph/ceph-client.admin.107328.log.gz 2023-10-10T20:31:19.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91836.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100580.log.gz 2023-10-10T20:31:19.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104122.log 2023-10-10T20:31:19.907 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93479.log 2023-10-10T20:31:19.907 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91836.log.gz 2023-10-10T20:31:19.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95450.log: /var/log/ceph/ceph-client.admin.104122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110276.log 2023-10-10T20:31:19.907 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95450.log.gz 2023-10-10T20:31:19.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.93479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113275.log 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104122.log.gz 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.93479.log.gz 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87734.log 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113275.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110276.log.gz 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108230.log 2023-10-10T20:31:19.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113275.log.gz 2023-10-10T20:31:19.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108188.log 2023-10-10T20:31:19.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118601.log 2023-10-10T20:31:19.909 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87734.log.gz 2023-10-10T20:31:19.909 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108230.log.gz 2023-10-10T20:31:19.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107286.log 2023-10-10T20:31:19.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118601.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108188.log.gz 2023-10-10T20:31:19.910 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104782.log 2023-10-10T20:31:19.910 INFO:teuthology.orchestra.run.smithi148.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-10T20:31:19.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118601.log.gz 2023-10-10T20:31:19.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100918.log 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104782.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107286.log.gz 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99901.log 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104782.log.gz 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92714.log 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99901.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100918.log.gz 2023-10-10T20:31:19.911 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115619.log 2023-10-10T20:31:19.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92714.log: gzip -5 --verbose 87.9%-- /var/log/ceph/ceph-client.admin.99774.log 2023-10-10T20:31:19.912 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99901.log.gz 2023-10-10T20:31:19.912 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92714.log.gz 2023-10-10T20:31:19.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105315.log 2023-10-10T20:31:19.912 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115619.log.gz 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95618.log 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105315.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99774.log.gz 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87314.log 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95618.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105315.log.gz 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102750.log 2023-10-10T20:31:19.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87314.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95618.log.gz 2023-10-10T20:31:19.914 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86574.log 2023-10-10T20:31:19.914 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87314.log.gz 2023-10-10T20:31:19.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102128.log 2023-10-10T20:31:19.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108955.log 2023-10-10T20:31:19.914 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102750.log.gz 2023-10-10T20:31:19.915 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86574.log.gz 2023-10-10T20:31:19.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114969.log 2023-10-10T20:31:19.915 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102128.log.gz 2023-10-10T20:31:19.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89162.log 2023-10-10T20:31:19.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114969.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108955.log.gz 2023-10-10T20:31:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112190.log 2023-10-10T20:31:19.916 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114969.log.gz 2023-10-10T20:31:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103088.log 2023-10-10T20:31:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89162.log: /var/log/ceph/ceph-client.admin.112190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110838.log 2023-10-10T20:31:19.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103088.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89162.log.gz 2023-10-10T20:31:19.917 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112190.log.gz 2023-10-10T20:31:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93779.log 2023-10-10T20:31:19.917 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103088.log.gz 2023-10-10T20:31:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.110838.log: -- /var/log/ceph/ceph-client.admin.97175.log 2023-10-10T20:31:19.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93779.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110838.log.gz 2023-10-10T20:31:19.918 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93779.log.gz 2023-10-10T20:31:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96923.log 2023-10-10T20:31:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94196.log 2023-10-10T20:31:19.918 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97175.log.gz 2023-10-10T20:31:19.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111100.log 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88411.log 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94196.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96923.log.gz 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111100.log: gzip -5 88.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.94196.log.gz/var/log/ceph/ceph-client.admin.116259.log 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111100.log.gz 2023-10-10T20:31:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87125.log 2023-10-10T20:31:19.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88411.log: /var/log/ceph/ceph-client.admin.116259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101857.log 2023-10-10T20:31:19.920 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88411.log.gz 2023-10-10T20:31:19.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87125.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116259.log.gz 2023-10-10T20:31:19.920 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100960.log 2023-10-10T20:31:19.920 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87125.log.gz 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94238.log 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105579.log 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100960.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101857.log.gz 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104278.log 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100960.log.gz 2023-10-10T20:31:19.921 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94238.log.gz 2023-10-10T20:31:19.922 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117828.log 2023-10-10T20:31:19.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105579.log: /var/log/ceph/ceph-client.admin.104278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98338.log 2023-10-10T20:31:19.922 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105579.log.gz 2023-10-10T20:31:19.922 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104278.log.gz 2023-10-10T20:31:19.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106488.log 2023-10-10T20:31:19.923 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117828.log.gz 2023-10-10T20:31:19.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90170.log 2023-10-10T20:31:19.923 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104320.log 2023-10-10T20:31:19.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106488.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98338.log.gz 2023-10-10T20:31:19.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90170.log: gzip -5 --verbose -- 88.3%/var/log/ceph/ceph-client.admin.88726.log -- replaced with /var/log/ceph/ceph-client.admin.106488.log.gz 2023-10-10T20:31:19.924 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:19.924 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90170.log.gz 2023-10-10T20:31:19.924 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88407.log 2023-10-10T20:31:19.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104320.log: /var/log/ceph/ceph-client.admin.88726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93052.log 2023-10-10T20:31:19.924 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104320.log.gz 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88726.log.gz 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98632.log 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88407.log.gz 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110024.log 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93821.log 2023-10-10T20:31:19.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98632.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93052.log.gz 2023-10-10T20:31:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98632.log.gz 2023-10-10T20:31:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111226.log 2023-10-10T20:31:19.926 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110024.log.gz 2023-10-10T20:31:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97519.log 2023-10-10T20:31:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89372.log 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111226.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93821.log.gz 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97643.log 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111226.log.gz 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97519.log.gz 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94616.log 2023-10-10T20:31:19.927 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89372.log.gz 2023-10-10T20:31:19.928 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88728.log 2023-10-10T20:31:19.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97643.log: /var/log/ceph/ceph-client.admin.94616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88077.log 2023-10-10T20:31:19.928 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97643.log.gz 2023-10-10T20:31:19.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88728.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94616.log.gz 2023-10-10T20:31:19.928 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92800.log 2023-10-10T20:31:19.929 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3%/var/log/ceph/ceph-client.admin.88077.log: -- replaced with /var/log/ceph/ceph-client.admin.88728.log.gz 2023-10-10T20:31:19.929 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111577.log 2023-10-10T20:31:19.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92800.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88077.log.gz 2023-10-10T20:31:19.929 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87440.log 2023-10-10T20:31:19.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.111577.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92800.log.gz 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94930.log 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111577.log.gz 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip /var/log/ceph/ceph-client.admin.87440.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.117786.log 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94930.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87440.log.gz 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90700.log 2023-10-10T20:31:19.930 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94930.log.gz 2023-10-10T20:31:19.931 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97820.log 2023-10-10T20:31:19.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117786.log: /var/log/ceph/ceph-client.admin.90700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112106.log 2023-10-10T20:31:19.931 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117786.log.gz 2023-10-10T20:31:19.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97820.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.90700.log.gz-5 2023-10-10T20:31:19.931 INFO:teuthology.orchestra.run.smithi148.stderr:--verbose -- /var/log/ceph/ceph-client.admin.86906.log 2023-10-10T20:31:19.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112106.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97820.log.gz 2023-10-10T20:31:19.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96685.log 2023-10-10T20:31:19.932 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112106.log.gz 2023-10-10T20:31:19.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip /var/log/ceph/ceph-client.admin.86906.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.88065.log 2023-10-10T20:31:19.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96685.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86906.log.gz 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105445.log 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96685.log.gz 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90784.log 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88065.log: /var/log/ceph/ceph-client.admin.105445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109940.log 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88065.log.gz 2023-10-10T20:31:19.933 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105445.log.gz 2023-10-10T20:31:19.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104236.log 2023-10-10T20:31:19.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109940.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90784.log.gz 2023-10-10T20:31:19.934 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113653.log 2023-10-10T20:31:19.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104236.log: gzip 89.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.109940.log.gz 2023-10-10T20:31:19.934 INFO:teuthology.orchestra.run.smithi148.stderr:-- /var/log/ceph/ceph-client.admin.97049.log 2023-10-10T20:31:19.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113653.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104236.log.gz 2023-10-10T20:31:19.935 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116301.log 2023-10-10T20:31:19.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92418.log 2023-10-10T20:31:19.935 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97049.log.gz 2023-10-10T20:31:19.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96839.log 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113653.log.gz 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116301.log.gz 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92418.log: /var/log/ceph/ceph-client.admin.96839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88408.log 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92418.log.gz 2023-10-10T20:31:19.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.3.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96839.log.gz 2023-10-10T20:31:19.937 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95152.log 2023-10-10T20:31:19.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87167.log 2023-10-10T20:31:19.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95152.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88408.log.gz 2023-10-10T20:31:19.937 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99294.log 2023-10-10T20:31:19.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87167.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95152.log.gz 2023-10-10T20:31:19.938 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117211.log 2023-10-10T20:31:19.938 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87167.log.gz 2023-10-10T20:31:19.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103560.log 2023-10-10T20:31:19.938 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99294.log.gz 2023-10-10T20:31:19.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93220.log 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118728.log 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103560.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.117211.log.gz 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93220.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103560.log.gz 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93136.log 2023-10-10T20:31:19.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118728.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109091.log 2023-10-10T20:31:19.940 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118728.log.gz 2023-10-10T20:31:19.940 INFO:teuthology.orchestra.run.smithi148.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.93220.log.gz 2023-10-10T20:31:19.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92090.log 2023-10-10T20:31:19.940 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105537.log /var/log/ceph/ceph-client.admin.109091.log: 2023-10-10T20:31:19.940 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93136.log.gz 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101356.log 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109091.log.gz 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92090.log.gz 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89456.log 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111184.log 2023-10-10T20:31:19.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105537.log.gz/var/log/ceph/ceph-client.admin.101356.log: 2023-10-10T20:31:19.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89456.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84592.log 2023-10-10T20:31:19.942 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101356.log.gz 2023-10-10T20:31:19.942 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89456.log.gz 2023-10-10T20:31:19.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92249.log 2023-10-10T20:31:19.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106950.log 2023-10-10T20:31:19.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.tmp-client.admin.84592.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111184.log.gz 2023-10-10T20:31:19.943 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84592.log.gz 2023-10-10T20:31:19.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105401.log 2023-10-10T20:31:19.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106950.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92249.log.gz 2023-10-10T20:31:19.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88396.log 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106950.log.gz 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103172.log 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110500.log 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88396.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105401.log.gz 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110108.log 2023-10-10T20:31:19.944 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88396.log.gz 2023-10-10T20:31:19.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103172.log.gz 2023-10-10T20:31:19.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99943.log 2023-10-10T20:31:19.945 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110500.log.gz 2023-10-10T20:31:19.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98782.log 2023-10-10T20:31:19.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109450.log 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99943.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110108.log.gz 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96310.log 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99943.log.gz 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98782.log.gz 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103519.log 2023-10-10T20:31:19.946 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89540.log 2023-10-10T20:31:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96310.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109450.log.gz 2023-10-10T20:31:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.103519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112545.log 2023-10-10T20:31:19.947 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103519.log.gz 2023-10-10T20:31:19.947 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96310.log.gz 2023-10-10T20:31:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86732.log 2023-10-10T20:31:19.948 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89540.log.gz 2023-10-10T20:31:19.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94364.log 2023-10-10T20:31:19.948 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110360.log 2023-10-10T20:31:19.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86732.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112545.log.gz 2023-10-10T20:31:19.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86999.log 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86732.log.gz 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.94364.log.gz 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94574.log 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109814.log 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86999.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110360.log.gz 2023-10-10T20:31:19.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117505.log 2023-10-10T20:31:19.950 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86999.log.gz 2023-10-10T20:31:19.950 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94574.log.gz 2023-10-10T20:31:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96216.log 2023-10-10T20:31:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96559.log 2023-10-10T20:31:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109814.log.gz 2023-10-10T20:31:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92004.log 2023-10-10T20:31:19.951 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117505.log.gz 2023-10-10T20:31:19.951 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96216.log.gz 2023-10-10T20:31:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112022.log 2023-10-10T20:31:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114406.log 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96559.log.gz 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98548.log 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92004.log.gz 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112022.log.gz 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98296.log 2023-10-10T20:31:19.952 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92048.log 2023-10-10T20:31:19.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98548.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114406.log.gz 2023-10-10T20:31:19.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118458.log 2023-10-10T20:31:19.953 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98548.log.gz 2023-10-10T20:31:19.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118248.log 2023-10-10T20:31:19.953 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98296.log.gz 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92048.log.gz 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92544.log 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97385.log 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118458.log.gz 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88729.log 2023-10-10T20:31:19.954 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118248.log.gz 2023-10-10T20:31:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97385.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92544.log.gz 2023-10-10T20:31:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100047.log 2023-10-10T20:31:19.955 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97385.log.gz 2023-10-10T20:31:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97091.log 2023-10-10T20:31:19.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109856.log 2023-10-10T20:31:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100047.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88729.log.gz 2023-10-10T20:31:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100530.log 2023-10-10T20:31:19.956 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100047.log.gz 2023-10-10T20:31:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109856.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97091.log.gz 2023-10-10T20:31:19.956 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114700.log 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4%/var/log/ceph/ceph-client.admin.100530.log: -- replaced with /var/log/ceph/ceph-client.admin.109856.log.gz 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114700.log: gzip -5 --verbose -- 88.1%/var/log/ceph/ceph-client.admin.112629.log 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100530.log.gz 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114700.log.gz 2023-10-10T20:31:19.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105875.log 2023-10-10T20:31:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104362.log 2023-10-10T20:31:19.958 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112629.log.gz 2023-10-10T20:31:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105875.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105875.log.gz 2023-10-10T20:31:19.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98674.log 2023-10-10T20:31:19.958 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104362.log.gz 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116172.log 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93483.log 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98674.log.gz 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.116172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116172.log.gz 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.93483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.93483.log.gz 2023-10-10T20:31:19.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110708.log 2023-10-10T20:31:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111142.log 2023-10-10T20:31:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98975.log 2023-10-10T20:31:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111142.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110708.log.gz 2023-10-10T20:31:19.960 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111142.log.gz 2023-10-10T20:31:19.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.98975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98975.log.gz 2023-10-10T20:31:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89120.log 2023-10-10T20:31:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107473.log 2023-10-10T20:31:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89120.log: /var/log/ceph/ceph-client.admin.88616.log 2023-10-10T20:31:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107473.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89120.log.gz 2023-10-10T20:31:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88616.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107473.log.gz 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88616.log.gz 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95936.log 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114364.log 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109982.log 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114364.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95936.log.gz 2023-10-10T20:31:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109982.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114364.log.gz 2023-10-10T20:31:19.963 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109982.log.gz 2023-10-10T20:31:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105012.log 2023-10-10T20:31:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105665.log 2023-10-10T20:31:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114448.log 2023-10-10T20:31:19.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105665.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105012.log.gz 2023-10-10T20:31:19.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105665.log.gz 2023-10-10T20:31:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114448.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114448.log.gz 2023-10-10T20:31:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114885.log 2023-10-10T20:31:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118038.log 2023-10-10T20:31:19.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113485.log 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118038.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114885.log.gz 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118038.log.gz 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113485.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113485.log.gz 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89078.log 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113443.log 2023-10-10T20:31:19.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89288.log 2023-10-10T20:31:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88071.log 2023-10-10T20:31:19.966 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89078.log.gz 2023-10-10T20:31:19.966 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113443.log.gz 2023-10-10T20:31:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114759.log 2023-10-10T20:31:19.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88410.log 2023-10-10T20:31:19.967 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89288.log.gz 2023-10-10T20:31:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108085.log 2023-10-10T20:31:19.967 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88071.log.gz 2023-10-10T20:31:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114759.log.gz 2023-10-10T20:31:19.967 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88410.log.gz 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108085.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108085.log.gz 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115194.log 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101773.log 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102920.log 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101773.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115194.log.gz 2023-10-10T20:31:19.968 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101773.log.gz 2023-10-10T20:31:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102920.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102920.log.gz 2023-10-10T20:31:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103046.log 2023-10-10T20:31:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108913.log 2023-10-10T20:31:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87272.log 2023-10-10T20:31:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104488.log 2023-10-10T20:31:19.970 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103046.log.gz 2023-10-10T20:31:19.970 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108913.log.gz 2023-10-10T20:31:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100486.log 2023-10-10T20:31:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91626.log 2023-10-10T20:31:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87272.log.gz 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100486.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104488.log.gz 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91626.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100486.log.gz 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95364.log 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91626.log.gz 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115829.log 2023-10-10T20:31:19.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97721.log 2023-10-10T20:31:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115829.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95364.log.gz 2023-10-10T20:31:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97721.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115829.log.gz 2023-10-10T20:31:19.972 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97721.log.gz 2023-10-10T20:31:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111276.log 2023-10-10T20:31:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113191.log 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91179.log 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111276.log.gz 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113191.log: /var/log/ceph/ceph-client.admin.91179.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113191.log.gz 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91179.log.gz 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109730.log 2023-10-10T20:31:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102706.log 2023-10-10T20:31:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102043.log 2023-10-10T20:31:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109730.log.gz 2023-10-10T20:31:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101563.log 2023-10-10T20:31:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102706.log.gz 2023-10-10T20:31:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101899.log 2023-10-10T20:31:19.975 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102043.log.gz 2023-10-10T20:31:19.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110666.log 2023-10-10T20:31:19.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101899.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101563.log.gz 2023-10-10T20:31:19.975 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101899.log.gz 2023-10-10T20:31:19.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110666.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110666.log.gz 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110972.log 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107872.log 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112977.log 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110972.log.gz 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112977.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107872.log.gz 2023-10-10T20:31:19.976 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112977.log.gz 2023-10-10T20:31:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88397.log 2023-10-10T20:31:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116395.log 2023-10-10T20:31:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96643.log 2023-10-10T20:31:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116395.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88397.log.gz 2023-10-10T20:31:19.977 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116395.log.gz 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96643.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96643.log.gz 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110888.log 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88072.log 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117421.log 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88072.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110888.log.gz 2023-10-10T20:31:19.978 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88072.log.gz 2023-10-10T20:31:19.979 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117421.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117421.log.gz 2023-10-10T20:31:19.979 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118643.log 2023-10-10T20:31:19.979 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99420.log 2023-10-10T20:31:19.979 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117589.log 2023-10-10T20:31:19.979 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99420.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118643.log.gz 2023-10-10T20:31:19.980 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99420.log.gz 2023-10-10T20:31:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117589.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117589.log.gz 2023-10-10T20:31:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91412.log 2023-10-10T20:31:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114574.log 2023-10-10T20:31:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105172.log 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114574.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91412.log.gz 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114574.log.gz 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105172.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105172.log.gz 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117001.log 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91710.log 2023-10-10T20:31:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96881.log 2023-10-10T20:31:19.982 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91710.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117001.log.gz 2023-10-10T20:31:19.982 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91710.log.gz 2023-10-10T20:31:19.982 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96881.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96881.log.gz 2023-10-10T20:31:19.982 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101689.log 2023-10-10T20:31:19.982 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89414.log 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93524.log 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89414.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101689.log.gz 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89414.log.gz 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93524.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93524.log.gz 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93737.log 2023-10-10T20:31:19.983 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97343.log 2023-10-10T20:31:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110318.log 2023-10-10T20:31:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97343.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93737.log.gz 2023-10-10T20:31:19.984 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97343.log.gz 2023-10-10T20:31:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110318.log.gz 2023-10-10T20:31:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100442.log 2023-10-10T20:31:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113611.log 2023-10-10T20:31:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114042.log 2023-10-10T20:31:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113611.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100442.log.gz 2023-10-10T20:31:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104824.log 2023-10-10T20:31:19.985 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113611.log.gz 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116445.log 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114042.log.gz 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91308.log 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5%/var/log/ceph/ceph-client.admin.116445.log: -- replaced with /var/log/ceph/ceph-client.admin.104824.log.gz 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.116445.log.gz 2023-10-10T20:31:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91308.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91308.log.gz 2023-10-10T20:31:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100792.log 2023-10-10T20:31:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-10T20:31:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118688.log 2023-10-10T20:31:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.0.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100792.log.gz 2023-10-10T20:31:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118688.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118688.log.gz 2023-10-10T20:31:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107431.log 2023-10-10T20:31:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113147.log 2023-10-10T20:31:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98210.log 2023-10-10T20:31:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113147.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107431.log.gz 2023-10-10T20:31:19.988 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113147.log.gz 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98210.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98210.log.gz 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88413.log 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92926.log 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90742.log 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88413.log.gz 2023-10-10T20:31:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92926.log: /var/log/ceph/ceph-client.admin.90742.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92926.log.gz 2023-10-10T20:31:19.990 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90742.log.gz 2023-10-10T20:31:19.990 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107034.log 2023-10-10T20:31:19.990 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113019.log 2023-10-10T20:31:19.990 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107034.log: /var/log/ceph/ceph-client.admin.113019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107034.log.gz 2023-10-10T20:31:19.990 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.113019.log.gz 2023-10-10T20:31:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112587.log 2023-10-10T20:31:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104572.log 2023-10-10T20:31:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112587.log: /var/log/ceph/ceph-client.admin.104572.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112587.log.gz 2023-10-10T20:31:19.991 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104572.log.gz 2023-10-10T20:31:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98422.log 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94826.log 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.98422.log: /var/log/ceph/ceph-client.admin.94826.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98422.log.gz 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94826.log.gz 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101815.log 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99859.log 2023-10-10T20:31:19.992 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103953.log 2023-10-10T20:31:19.993 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99859.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101815.log.gz 2023-10-10T20:31:19.993 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92586.log 2023-10-10T20:31:19.993 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99859.log.gz 2023-10-10T20:31:19.993 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90121.log 2023-10-10T20:31:19.993 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103953.log.gz 2023-10-10T20:31:19.994 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.90121.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92586.log.gz 2023-10-10T20:31:19.994 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90121.log.gz 2023-10-10T20:31:19.994 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87692.log 2023-10-10T20:31:19.994 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116217.log 2023-10-10T20:31:19.994 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105791.log 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116217.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87692.log.gz 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116217.log.gz 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105791.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105791.log.gz 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107557.log 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110150.log 2023-10-10T20:31:19.995 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101269.log 2023-10-10T20:31:19.996 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110150.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107557.log.gz 2023-10-10T20:31:19.996 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110150.log.gz 2023-10-10T20:31:19.996 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.101269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101269.log.gz 2023-10-10T20:31:19.996 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115321.log 2023-10-10T20:31:19.996 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94280.log 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115321.log: /var/log/ceph/ceph-client.admin.94280.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.115321.log.gz 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94280.log.gz 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101731.log 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94448.log 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101731.log: /var/log/ceph/ceph-client.admin.94448.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101731.log.gz 2023-10-10T20:31:19.997 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94448.log.gz 2023-10-10T20:31:19.998 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95320.log 2023-10-10T20:31:19.998 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87482.log 2023-10-10T20:31:19.998 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95320.log: /var/log/ceph/ceph-client.admin.87482.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95320.log.gz 2023-10-10T20:31:19.998 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87482.log.gz 2023-10-10T20:31:19.998 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102962.log 2023-10-10T20:31:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103214.log 2023-10-10T20:31:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102962.log: /var/log/ceph/ceph-client.admin.103214.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102962.log.gz 2023-10-10T20:31:19.999 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103214.log.gz 2023-10-10T20:31:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108566.log 2023-10-10T20:31:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118164.log 2023-10-10T20:31:20.000 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108566.log: /var/log/ceph/ceph-client.admin.118164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108566.log.gz 2023-10-10T20:31:20.000 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118164.log.gz 2023-10-10T20:31:20.000 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102212.log 2023-10-10T20:31:20.000 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98506.log 2023-10-10T20:31:20.000 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102212.log: /var/log/ceph/ceph-client.admin.98506.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102212.log.gz 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98506.log.gz 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103763.log 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115787.log 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103763.log: /var/log/ceph/ceph-client.admin.115787.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103763.log.gz 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115787.log.gz 2023-10-10T20:31:20.001 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-10T20:31:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113719.log 2023-10-10T20:31:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99336.log 2023-10-10T20:31:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113719.log: /var/log/ceph/ceph-client.admin.99336.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113719.log.gz 2023-10-10T20:31:20.002 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99336.log.gz 2023-10-10T20:31:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88952.log 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101002.log 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88952.log: /var/log/ceph/ceph-client.admin.101002.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88952.log.gz 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.101002.log.gz 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112671.log 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106651.log 2023-10-10T20:31:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112671.log: /var/log/ceph/ceph-client.admin.106651.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112671.log.gz 2023-10-10T20:31:20.004 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106651.log.gz 2023-10-10T20:31:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89246.log 2023-10-10T20:31:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95660.log 2023-10-10T20:31:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89246.log: /var/log/ceph/ceph-client.admin.95660.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89246.log.gz 2023-10-10T20:31:20.004 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95660.log.gz 2023-10-10T20:31:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107641.log 2023-10-10T20:31:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113569.log 2023-10-10T20:31:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107641.log: /var/log/ceph/ceph-client.admin.113569.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107641.log.gz 2023-10-10T20:31:20.005 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113569.log.gz 2023-10-10T20:31:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92333.log 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107782.log 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92333.log: 87.9%/var/log/ceph/ceph-client.admin.107782.log: -- replaced with /var/log/ceph/ceph-client.admin.92333.log.gz 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107782.log.gz 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99062.log 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90994.log 2023-10-10T20:31:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99062.log: 88.6%/var/log/ceph/ceph-client.admin.90994.log: -- replaced with /var/log/ceph/ceph-client.admin.99062.log.gz 2023-10-10T20:31:20.007 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90994.log.gz 2023-10-10T20:31:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104866.log 2023-10-10T20:31:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116504.log 2023-10-10T20:31:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104866.log: /var/log/ceph/ceph-client.admin.116504.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104866.log.gz 2023-10-10T20:31:20.007 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116504.log.gz 2023-10-10T20:31:20.008 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117295.log 2023-10-10T20:31:20.008 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114322.log 2023-10-10T20:31:20.008 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117295.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117295.log.gz 2023-10-10T20:31:20.009 INFO:teuthology.orchestra.run.smithi161.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-10T20:31:20.009 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-10T20:31:20.010 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-10T20:31:20.021 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114322.log.gz 2023-10-10T20:31:20.021 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92884.log 2023-10-10T20:31:20.021 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105917.log 2023-10-10T20:31:20.022 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92884.log: /var/log/ceph/ceph-client.admin.105917.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92884.log.gz 2023-10-10T20:31:20.022 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105917.log.gz 2023-10-10T20:31:20.022 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107160.log 2023-10-10T20:31:20.022 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90574.log 2023-10-10T20:31:20.022 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107160.log: /var/log/ceph/ceph-client.admin.90574.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107160.log.gz 2023-10-10T20:31:20.023 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90574.log.gz 2023-10-10T20:31:20.023 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87398.log 2023-10-10T20:31:20.023 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100750.log 2023-10-10T20:31:20.023 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87398.log: /var/log/ceph/ceph-client.admin.100750.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87398.log.gz 2023-10-10T20:31:20.023 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100750.log.gz 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100834.log 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111980.log 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100834.log: /var/log/ceph/ceph-client.admin.111980.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100834.log.gz 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111980.log.gz 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114616.log 2023-10-10T20:31:20.024 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108314.log 2023-10-10T20:31:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114616.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114616.log.gz 2023-10-10T20:31:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108314.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108314.log.gz 2023-10-10T20:31:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116604.log 2023-10-10T20:31:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95786.log 2023-10-10T20:31:20.025 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.116604.log: /var/log/ceph/ceph-client.admin.95786.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116604.log.gz 2023-10-10T20:31:20.026 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95786.log.gz 2023-10-10T20:31:20.026 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117547.log 2023-10-10T20:31:20.026 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106325.log 2023-10-10T20:31:20.026 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117547.log: /var/log/ceph/ceph-client.admin.106325.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117547.log.gz 2023-10-10T20:31:20.026 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106325.log.gz 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87818.log 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89036.log 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87818.log: /var/log/ceph/ceph-client.admin.89036.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87818.log.gz 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89036.log.gz 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113958.log 2023-10-10T20:31:20.027 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93959.log 2023-10-10T20:31:20.028 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113958.log: /var/log/ceph/ceph-client.admin.93959.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113958.log.gz 2023-10-10T20:31:20.028 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93959.log.gz 2023-10-10T20:31:20.028 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89716.log 2023-10-10T20:31:20.028 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102456.log 2023-10-10T20:31:20.028 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89716.log: /var/log/ceph/ceph-client.admin.102456.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89716.log.gz 2023-10-10T20:31:20.029 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102456.log.gz 2023-10-10T20:31:20.029 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88069.log 2023-10-10T20:31:20.029 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92670.log 2023-10-10T20:31:20.029 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.88069.log: /var/log/ceph/ceph-client.admin.92670.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88069.log.gz 2023-10-10T20:31:20.029 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92670.log.gz 2023-10-10T20:31:20.030 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93010.log 2023-10-10T20:31:20.030 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118290.log 2023-10-10T20:31:20.030 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.93010.log: /var/log/ceph/ceph-client.admin.118290.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93010.log.gz 2023-10-10T20:31:20.030 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118290.log.gz 2023-10-10T20:31:20.030 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106176.log 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108871.log 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.106176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106176.log.gz 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.108871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108871.log.gz 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118517.log 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91752.log 2023-10-10T20:31:20.031 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118517.log: /var/log/ceph/ceph-client.admin.91752.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118517.log.gz 2023-10-10T20:31:20.032 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91752.log.gz 2023-10-10T20:31:20.032 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109688.log 2023-10-10T20:31:20.032 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95744.log 2023-10-10T20:31:20.032 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109688.log: /var/log/ceph/ceph-client.admin.95744.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109688.log.gz 2023-10-10T20:31:20.032 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95744.log.gz 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109492.log 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107599.log 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109492.log: /var/log/ceph/ceph-client.admin.107599.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109492.log.gz 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107599.log.gz 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113061.log 2023-10-10T20:31:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94868.log 2023-10-10T20:31:20.034 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113061.log: /var/log/ceph/ceph-client.admin.94868.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113061.log.gz 2023-10-10T20:31:20.034 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94868.log.gz 2023-10-10T20:31:20.034 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104080.log 2023-10-10T20:31:20.034 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112232.log 2023-10-10T20:31:20.034 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104080.log: /var/log/ceph/ceph-client.admin.112232.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104080.log.gz 2023-10-10T20:31:20.035 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112232.log.gz 2023-10-10T20:31:20.035 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117337.log 2023-10-10T20:31:20.035 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117744.log 2023-10-10T20:31:20.035 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117337.log: /var/log/ceph/ceph-client.admin.117744.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117337.log.gz 2023-10-10T20:31:20.035 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117744.log.gz 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113233.log 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109772.log 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113233.log: /var/log/ceph/ceph-client.admin.109772.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.113233.log.gz 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109772.log.gz 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97427.log 2023-10-10T20:31:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92460.log 2023-10-10T20:31:20.037 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97427.log: /var/log/ceph/ceph-client.admin.92460.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97427.log.gz 2023-10-10T20:31:20.037 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92460.log.gz 2023-10-10T20:31:20.037 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115407.log 2023-10-10T20:31:20.037 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105090.log 2023-10-10T20:31:20.037 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.115407.log: /var/log/ceph/ceph-client.admin.105090.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115407.log.gz 2023-10-10T20:31:20.038 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105090.log.gz 2023-10-10T20:31:20.038 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117169.log 2023-10-10T20:31:20.038 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106446.log 2023-10-10T20:31:20.038 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117169.log: /var/log/ceph/ceph-client.admin.106446.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117169.log.gz 2023-10-10T20:31:20.038 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106446.log.gz 2023-10-10T20:31:20.039 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110930.log 2023-10-10T20:31:20.039 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106180.log 2023-10-10T20:31:20.039 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110930.log: /var/log/ceph/ceph-client.0.106180.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110930.log.gz 2023-10-10T20:31:20.039 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106180.log.gz 2023-10-10T20:31:20.039 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118374.log 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99985.log 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.118374.log: /var/log/ceph/ceph-client.admin.99985.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118374.log.gz 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99985.log.gz 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117085.log 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117954.log 2023-10-10T20:31:20.040 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117085.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117085.log.gz 2023-10-10T20:31:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111622.log 2023-10-10T20:31:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.117954.log: /var/log/ceph/ceph-client.admin.111622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117954.log.gz 2023-10-10T20:31:20.041 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111622.log.gz 2023-10-10T20:31:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109349.log 2023-10-10T20:31:20.041 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117912.log 2023-10-10T20:31:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109349.log: /var/log/ceph/ceph-client.admin.117912.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109349.log.gz 2023-10-10T20:31:20.042 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117912.log.gz 2023-10-10T20:31:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-10T20:31:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118122.log 2023-10-10T20:31:20.042 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.118122.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118122.log.gz 2023-10-10T20:31:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111581.log 2023-10-10T20:31:20.043 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.111581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111581.log.gz 2023-10-10T20:31:20.043 INFO:teuthology.orchestra.run.smithi148.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-10T20:31:20.043 INFO:teuthology.orchestra.run.smithi148.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-10T20:31:20.365 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-10T20:31:20.401 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-10T20:31:20.439 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-10T20:31:20.450 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-10T20:31:20.464 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-10T20:31:20.466 INFO:teuthology.orchestra.run.smithi161.stderr: 2023-10-10T20:31:20.466 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m0.847s 2023-10-10T20:31:20.466 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m3.986s 2023-10-10T20:31:20.467 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.169s 2023-10-10T20:31:20.519 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-10T20:31:20.525 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-10T20:31:20.625 INFO:teuthology.orchestra.run.smithi148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-10T20:31:20.664 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-10T20:31:20.666 INFO:teuthology.orchestra.run.smithi148.stderr: 2023-10-10T20:31:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m1.070s 2023-10-10T20:31:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m4.599s 2023-10-10T20:31:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.480s 2023-10-10T20:31:20.667 INFO:tasks.ceph:Archiving logs... 2023-10-10T20:31:20.667 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi148/log 2023-10-10T20:31:20.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-10T20:31:21.814 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi161/log 2023-10-10T20:31:21.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-10T20:31:22.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-10T20:31:22.363 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-10T20:31:22.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-10T20:31:22.367 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-10T20:31:22.447 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-10T20:31:22.456 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-10T20:31:22.457 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:31:22.460 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-10T20:31:22.473 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2023-10-10T20:31:22.475 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-10-10T20:31:22.477 INFO:teuthology.orchestra.run.smithi148.stdout:210 Number of sources = 4 2023-10-10T20:31:22.477 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:31:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2023-10-10T20:31:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -1254us[-1315us] +/- 78ms 2023-10-10T20:31:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -664us[ -725us] +/- 69ms 2023-10-10T20:31:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +2902us[+2902us] +/- 61ms 2023-10-10T20:31:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:22.480 INFO:teuthology.orchestra.run.smithi161.stdout:210 Number of sources = 4 2023-10-10T20:31:22.480 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-10T20:31:22.480 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-10-10T20:31:22.480 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 -1692us[-1243us] +/- 78ms 2023-10-10T20:31:22.481 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1109us[ -660us] +/- 69ms 2023-10-10T20:31:22.481 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +2472us[+2921us] +/- 61ms 2023-10-10T20:31:22.481 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-10T20:31:22.481 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-10T20:31:22.491 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-10T20:31:22.492 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-10T20:31:22.502 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-10T20:31:22.632 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-10T20:31:22.732 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:31:22.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:31:22.765 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-10T20:31:22.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:31:22.797 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-10T20:31:22.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-10T20:31:22.816 INFO:teuthology.task.internal:Duration was 1366.120462 seconds 2023-10-10T20:31:22.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-10T20:31:22.825 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-10T20:31:22.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-10T20:31:22.828 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-10T20:31:22.871 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:31:22.882 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-10T20:31:23.066 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-10T20:31:23.067 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2023-10-10T20:31:23.067 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-10T20:31:23.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2023-10-10T20:31:23.127 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-10T20:31:23.146 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-10T20:31:23.146 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-10T20:31:23.169 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-10T20:31:23.225 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-10T20:31:23.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-10T20:31:23.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-10T20:31:23.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-10T20:31:23.352 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-10T20:31:23.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-10T20:31:23.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-10T20:31:23.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-10T20:31:23.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:31:23.421 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:31:23.442 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2023-10-10T20:31:23.466 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2023-10-10T20:31:23.480 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:31:23.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:31:23.510 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-10T20:31:23.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-10T20:31:23.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-10T20:31:23.547 INFO:teuthology.task.internal:Transferring archived files... 2023-10-10T20:31:23.548 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi148 2023-10-10T20:31:23.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-10T20:31:23.588 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/remote/smithi161 2023-10-10T20:31:23.588 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-10T20:31:23.624 INFO:teuthology.task.internal:Removing archive directory... 2023-10-10T20:31:23.625 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-10T20:31:23.627 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-10T20:31:23.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-10T20:31:23.686 INFO:teuthology.task.internal:Not uploading archives. 2023-10-10T20:31:23.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-10T20:31:23.698 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-10T20:31:23.698 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-10T20:31:23.701 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-10T20:31:23.713 INFO:teuthology.orchestra.run.smithi148.stdout: 262691 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 10 20:31 /home/ubuntu/cephtest 2023-10-10T20:31:23.728 INFO:teuthology.orchestra.run.smithi161.stdout: 262495 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 10 20:31 /home/ubuntu/cephtest 2023-10-10T20:31:23.729 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-10T20:31:23.740 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-10T20:31:23.762 INFO:teuthology.nuke:Checking targets against current locks 2023-10-10T20:31:23.788 DEBUG:teuthology.nuke:shortname: smithi148 2023-10-10T20:31:23.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:31:23.811 DEBUG:teuthology.nuke:shortname: smithi161 2023-10-10T20:31:23.812 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T20:31:23.816 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:00:59.485629', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGi6gJsTVaMUlBg6Me2PzbT0mPxCeJVMjlPY47SvIQiGWjBu3snTBdzepBBQn2t82Ii4rS/ZdXuBT+0mjcs1yd4='} 2023-10-10T20:31:23.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 20:00:59.483661', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYkYz3NSyUPaG3igeoA55L9b/m1sneuW3V9rXMKdl1KO1GrLsuBTmFe2n4Be3W4R8N7xmc2kremECT0tzLawKs='} 2023-10-10T20:31:23.868 INFO:teuthology.orchestra.console.smithi148:Power off 2023-10-10T20:31:23.869 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T20:31:23.892 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-10-10T20:31:23.892 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:23.910 INFO:teuthology.orchestra.console.smithi161:Power off 2023-10-10T20:31:23.911 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T20:31:23.923 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-10-10T20:31:23.936 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2023-10-10T20:31:23.937 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:23.961 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2023-10-10T20:31:27.925 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:27.963 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:28.035 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-10-10T20:31:28.075 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2023-10-10T20:31:32.038 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:32.076 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:32.151 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-10-10T20:31:32.187 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2023-10-10T20:31:32.252 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-10-10T20:31:36.190 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T20:31:36.301 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2023-10-10T20:31:36.402 INFO:teuthology.orchestra.console.smithi161:Power off completed 2023-10-10T20:31:36.507 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1366.1204617023468 failure_reason: '"1696969150.205412 osd.0 (osd.0) 3 : cluster [WRN] OSD bench result of 232801.263459 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.0. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-10T20:31:36.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T20:31:36.583 INFO:teuthology.run:FAIL